Field Engine modules

The Field Engine (FE) has limited number of instructions/operations to support most use cases. There is a plan to add LuaJIT to be more flexible at the cost of performance. The FE can allocate stream variables in a stream context, write a stream variable to a packet offset, change packet size, and so on.

Examples of Field Engine uses:

  • Change ipv4.tos 1-10

  • Change packet size to a random value in the range 64 to 9K

  • Create a range of flows (change src_ip, dest_ip, src_port, dest_port)

  • Update IPv4 checksum

The following snippet creates a range of 64 bytes packets

# split the range of IP to cores
#
class STLS1(object):

  def __init__ (self):
      self.fsize  =64;

  def create_stream (self):
      # create a base packet and pad it to size
      size = self.fsize - 4; # no FCS

      base_pkt = Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)

      pad = max(0, size - len(base_pkt)) * 'x'

      vm = STLScVmRaw( [   STLVmFlowVar ( "ip_src",
                                          min_value="10.0.0.1",
                                          max_value="10.0.0.255",
                                          size=4,
                                          step=1,
                                          op="inc"),
                           STLVmWrFlowVar (fv_name="ip_src",
                                           pkt_offset= "IP.src" ), # write ip to packet IP.src
                           STLVmFixIpv4(offset = "IP")                                # fix checksum
                         ],
                         split_by_field = "ip_src",
                         cache_size =255 # cache the packets, much better performance
                      );

      pkt = STLPktBuilder(pkt = base_pkt/pad,
                          vm = vm)
      stream = STLStream(packet = pkt,
                       mode = STLTXCont())
      #print(stream.to_code())
      return stream


  def get_streams (self, direction = 0, **kwargs):
      # create 1 stream
      return [ self.create_stream() ]

The following snippet creates a SYN attack:

# create attack from random src_ip from 16.0.0.0-18.0.0.254 and random src_port 1025-65000
# attack 48.0.0.1 server

def create_stream (self):


    # TCP SYN
    base_pkt  = Ether()/IP(dst="48.0.0.1")/TCP(dport=80,flags="S")


    # vm
    vm = STLScVmRaw( [ STLVmFlowVar(name="ip_src",
                                          min_value="16.0.0.0",
                                          max_value="18.0.0.254",
                                          size=4, op="random"),

                        STLVmFlowVar(name="src_port",
                                          min_value=1025,
                                          max_value=65000,
                                          size=2, op="random"),

                       STLVmWrFlowVar(fv_name="ip_src", pkt_offset= "IP.src" ),

                       STLVmFixIpv4(offset = "IP"), # fix checksum

                       STLVmWrFlowVar(fv_name="src_port",
                                            pkt_offset= "TCP.sport") # fix udp len

                      ]
                   )

    pkt = STLPktBuilder(pkt = base_pkt,
                        vm = vm)

    return STLStream(packet = pkt,
                     random_seed = 0x1234,# can be remove. will give the same random value any run
                     mode = STLTXCont())

STLScVmRaw class

Aggregate raw instructions objects

class trex.stl.trex_stl_packet_builder_scapy.STLScVmRaw(list_of_commands=None, cache_size=None)[source]

Raw instructions

Include a list of a basic instructions objects.

Parameters
list_of_commandslist

list of instructions

cache_sizeuint16_t

In case it is bigger than zero, FE results will be cached - this will speedup of the program at the cost of limiting the number of possible packets to the number of cache. The cache size is limited to the pool size

The following example splits the generated traffic by “ip_src” variable.

# Split by 

# TCP SYN
base_pkt  = Ether()/IP(dst="48.0.0.1")/TCP(dport=80,flags="S")     


# vm
vm = STLScVmRaw( [ STLVmFlowVar(name="ip_src", 
                                      min_value="16.0.0.0", 
                                      max_value="16.0.0.254", 
                                      size=4, op="inc"),                     


                   STLVmWrFlowVar(fv_name="ip_src", pkt_offset= "IP.src" ),  

                   STLVmFixIpv4(offset = "IP"), # fix checksum              
                  ]
                 cache_size = 1000
               )

STLVmFlowVar

class trex.stl.trex_stl_packet_builder_scapy.STLVmFlowVar(name, init_value=None, min_value=0, max_value=255, size=4, step=1, op='inc', value_list=None, split_to_cores=True, next_var=None)[source]

Flow variable instruction. Allocates a variable on a stream context. The size argument determines the variable size. The operation can be inc, dec, and random. For increment and decrement operations, can set the “step” size. For all operations, can set initialization value, minimum and maximum value.

Parameters
namestring

Name of the stream variable

init_valueint

Init value of the variable. If not specified, it will be min_value

min_valueint

Min value

max_valueint

Max value

sizeint

Number of bytes of the variable. Possible values: 1,2,4,8 for uint8_t, uint16_t, uint32_t, uint64_t

stepint

Step in case of “inc” or “dec” operations

opstring

Possible values: “inc”, “dec”, “random”

value_listint

List of values instead of init_value, min_value and max_value

split_to_coresbool

Default value is True. If the value is True then each core updates the variable by step * num of cores. If split_to_cores = False then each core updates the variable by step.

next_varstring or None

In case it isn’t None it is the name of the next variable. The next variable will perform its operation only when this variable completes a full wrap around.

# Example1

# input 
STLVmFlowVar(min_value=0, max_value=3, size=1,op="inc")

# output 0,1,2,3,0,1,2,3 ..

# input 
STLVmFlowVar(min_value=0, max_value=3, size=1,op="dec")

# output 0,3,2,1,0,3,2,1 ..


# input 
STLVmFlowVar(min_value=0, max_value=3, size=1,op="random")

# output 1,1,2,3,1,2,1,0 ..

# input 
STLVmFlowVar(min_value=0, max_value=10, size=1,op="inc",step=3)

# output 0,3,6,9,0,3,6,9,0..

# Example2

# input value_list
STLVmFlowVar(value_list=["16.0.0.1","16.0.0.2","16.0.0.3","16.0.0.4"], op="inc"))

# output 268435457,268435458,268435459,268435460,268435457,268435458,268435459,268435460,268435457..

# input value_list
STLVmFlowVar(value_list=[9,0,4,7,5], op="dec"))

# output 5,7,4,0,9,5,7,4,0,9,5,7..
get_var_name()[source]

Virtual function returns the varible name if it exists.

get_next_var_name()[source]

Virtual function that returns the name of the next variable name if it exists.

STLVmWrFlowVar

class trex.stl.trex_stl_packet_builder_scapy.STLVmWrFlowVar(fv_name, pkt_offset, offset_fixup=0, add_val=0, is_big=True)[source]

Write a stream variable into a packet field. The write position is determined by the packet offset + offset fixup. The size of the write is determined by the stream variable. Example: Offset 10, fixup 0, variable size 4. This function writes at 10, 11, 12, and 13.

For inromation about chaning the write size, offset, or fixup, see the STLVmWrMaskFlowVar command. The Field name/offset can be given by name in the following format: header[:id].field.

Parameters
fv_namestring

Stream variable to write to a packet offset.

pkt_offsetstring or int

Name of the field or offset in bytes from packet start.

offset_fixupint

Number of bytes to move forward. If negative, move backward.

add_valueint

Value to add to the stream variable before writing it to the packet field. Can be used as a constant offset.

is_bigbool

How to write the variable to the the packet. True=big-endian, False=little-endian

# Example3

pkt = Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)


# write to ip.src offset 
STLVmWrFlowVar (fv_name="tuple", pkt_offset= "IP.src" )

# packet offset is varible 
STLVmWrFlowVar (fv_name="tuple", pkt_offset= 26 )

# add l3_len_fix before writing fv_rand into IP.len field 
STLVmWrFlowVar(fv_name="fv_rand", pkt_offset= "IP.len", add_val=l3_len_fix) 
get_var_ref()[source]

Virtual function returns a ref var name.

compile(parent)[source]

Virtual function to take parent that has function name_to_offset.

STLVmWrMaskFlowVar

class trex.stl.trex_stl_packet_builder_scapy.STLVmWrMaskFlowVar(fv_name, pkt_offset, pkt_cast_size=1, mask=255, shift=0, add_value=0, offset_fixup=0, is_big=True)[source]

Write a stream variable into a packet field with some operations. Using this instruction, the variable size and the field can have different sizes.

Pseudocode of this code:

uint32_t val=(cast_to_size)rd_from_variable("name") # read flow-var
val+=m_add_value                                    # add value

if (m_shift>0) {                                    # shift 
    val=val<<m_shift
}else{
    if (m_shift<0) {
        val=val>>(-m_shift)
    }
}

pkt_val=rd_from_pkt(pkt_offset)                     # RMW to the packet
pkt_val = (pkt_val & ~m_mask) | (val & m_mask)
wr_to_pkt(pkt_offset,pkt_val)
Parameters
fv_namestring

The stream variable name to write to a packet field

pkt_cast_sizeuint8_t

The size in bytes of the packet field

maskuint32_t

The mask of the field. 1 means to write. 0 don’t care

shiftuint8_t

How many bits to shift

pkt_offsetstring or int

the name of the field or offset in byte from packet start.

offset_fixupint

how many bytes to go forward. In case of a negative value go backward

add_valint

value to add to stream variable before writing it to packet field. can be used as a constant offset

is_bigbool

how to write the variable to the the packet. is it big-edian or little edian

Example 1 - Cast from uint16_t (var) to uint8_t (pkt):

base_pkt =  Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)

vm = STLScVmRaw( [ STLVmFlowVar(name="mac_src", 
                                min_value=1, 
                                max_value=30, 
                                size=2, 
                                op="dec",step=1), 
                   STLVmWrMaskFlowVar(fv_name="mac_src", 
                                      pkt_offset= 11,
                                      pkt_cast_size=1, 
                                      mask=0xff) # mask command ->write it as one byte
              ]
           )

pkt = Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)

Example 2 - Change MSB of uint16_t variable:

vm = STLScVmRaw( [ STLVmFlowVar(name="mac_src", 
                                min_value=1, 
                                max_value=30, 
                                size=2, op="dec",step=1), 
                   STLVmWrMaskFlowVar(fv_name="mac_src", 
                                      pkt_offset= 10,
                                      pkt_cast_size=2, 
                                      mask=0xff00,
                                      shift=8) # take the var shift it 8 (x256) write only to LSB
                  ]
                )

Example 3 - Every 2 packets, change the MAC (shift right):

vm = STLScVmRaw( [ STLVmFlowVar(name="mac_src", 
                                min_value=1, 
                                max_value=30, 
                                size=2, op="dec",step=1), 
                   STLVmWrMaskFlowVar(fv_name="mac_src", 
                                      pkt_offset= 10,
                                      pkt_cast_size=1, 
                                      mask=0x1,
                                      shift=-1) # take var mac_src>>1 and write the LSB every two packet there should be a change
                 ]
                )
get_var_ref()[source]

Virtual function returns a ref var name.

compile(parent)[source]

Virtual function to take parent that has function name_to_offset.

STLVmFixChecksumHw

class trex.stl.trex_stl_packet_builder_scapy.STLVmFixChecksumHw(l3_offset, l4_offset, l4_type)[source]

Fix IPv4 header checksum and/or TCP/UDP checksum using hardware assist. Use this if the packet header has changed or data payload has changed as it is necessary to fix the checksums. This instruction works on NICS that support this hardware offload.

For fixing only IPv4 header checksum use STLVmFixIpv4. This instruction should be used if both L4 and L3 need to be fixed.

example for supported packets

Ether()/IPv4 SomeTunnel()/IPv4 Ether()/(IPv4|IPv6)/(UDP|TCP) Ether()/(IPv4|IPv6)/(UDP|TCP) SomeTunnel()/(IPv4|IPv6)/(UDP|TCP) SomeTunnel()/(IPv4|IPv6)/(UDP|TCP)

Parameters
l3_offsetoffset in bytes

IPv4/IPv6 header offset from packet start. It is not the offset of the checksum field itself. in could be string in case of scapy packet. format IP[:[id]]

l4_offset : offset in bytes to UDP/TCP header or IPv4 payload. in case of IPv4 checksum CTRexVmInsFixHwCs.L4_TYPE_IP could be set to zero to be auto calculated by the server

l4_type : [CTRexVmInsFixHwCs.L4_TYPE_UDP or CTRexVmInsFixHwCs.L4_TYPE_TCP or CTRexVmInsFixHwCs.L4_TYPE_IP]

see full example stl/syn_attack_fix_cs_hw.py

# Example2

pkt = Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)

# by offset 
STLVmFixChecksumHw(l3_offset=14,l4_offset=14+20,l4_type=CTRexVmInsFixHwCs.L4_TYPE_UDP)

# in case of scapy packet can be defined by header name 
STLVmFixChecksumHw(l3_offset="IP",l4_offset="UDP",l4_type=CTRexVmInsFixHwCs.L4_TYPE_UDP)

# string for second "IP" header in the packet is IP:1
STLVmFixChecksumHw(offset="IP:1")
compile(parent)[source]

Virtual function to take parent that has function name_to_offset.

STLVmFixIpv4

class trex.stl.trex_stl_packet_builder_scapy.STLVmFixIpv4(offset)[source]

Fix IPv4 header checksum. Use this if the packet header has changed and it is necessary to change the checksum.

Parameters
offsetuint16_t or string

IPv4 header offset from packet start. It is not the offset of the checksum field itself. in could be string in case of scapy packet. format IP[:[id]]

# Example2

pkt = Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)

# by offset 
STLVmFixIpv4(offset=14)

# in case of scapy packet can be defined by header name 
STLVmFixIpv4(offset="IP")

# string for second "IP" header in the packet is IP:1
STLVmFixIpv4(offset="IP:1")
compile(parent)[source]

Virtual function to take parent that has function name_to_offset.

STLVmFixIcmpv6

class trex.stl.trex_stl_packet_builder_scapy.STLVmFixIcmpv6(l3_offset, l4_offset)[source]

Fix ICMPv6 header checksum Use this if the packet header has changed or data payload has changed as it is necessary to fix the checksums.

example for supported packets

Ether()/IPv6()/ICMPv6ND_NS() Ether()/IPv6()/ICMPv6ND_NA() Ether()/IPv6()/ICMPv6ND_NS()/ICMPv6NDOptDstLLAddr() Ether()Dot1Q()/IPv6()/ICMPv6ND_NS()/ICMPv6NDOptDstLLAddr() Ether()Dot1Q()/Dot1Q()/IPv6()/ICMPv6ND_NS() SomeTunnel()/IPv6()/ICMPv6ND_NS()

Ether()/(IPv4|IPv6)/(UDP|TCP) Ether()/(IPv4|IPv6)/(UDP|TCP) SomeTunnel()/(IPv4|IPv6)/(UDP|TCP) SomeTunnel()/(IPv4|IPv6)/(UDP|TCP)

Parameters
l3_offsetoffset in bytes

IPv6 header offset from packet start. It is not the offset of the checksum field itself. it could be string in case of scapy packet. format IPv6[:[id]]

l4_offset : offset in bytes to ICMPv6 header.

# Example

pkt = Ether()/Dot1Q()/Dot1Q()/IPv6(src='::1', dst='::2')/ICMPv6ND_NS()

# by offset name 
STLVmFixIcmpv6(l3_offset='IPv6', l4_offset=ICMPv6ND_NS().name)

# manually
STLVmFixIcmpv6(l3_offset=22, l4_offset=62)
compile(parent)[source]

Virtual function to take parent that has function name_to_offset.

STLVmTrimPktSize

class trex.stl.trex_stl_packet_builder_scapy.STLVmTrimPktSize(fv_name)[source]

Trim the packet size by the stream variable size. This instruction only changes the total packet size, and does not repair the fields to match the new size.

Parameters
fv_namestring

Stream variable name. The value of this variable is the new total packet size.

For Example:

def create_stream (self):
    # pkt
    p_l2  = Ether();
    p_l3  = IP(src="16.0.0.1",dst="48.0.0.1")
    p_l4  = UDP(dport=12,sport=1025)
    pyld_size = max(0, self.max_pkt_size_l3 - len(p_l3/p_l4));
    base_pkt = p_l2/p_l3/p_l4/('U'*(pyld_size))

    l3_len_fix =-(len(p_l2));
    l4_len_fix =-(len(p_l2/p_l3));


    # vm
    vm = STLScVmRaw( [ STLVmFlowVar(name="fv_rand", min_value=64,
                                    max_value=len(base_pkt),
                                    size=2, op="inc"),

                       STLVmTrimPktSize("fv_rand"),                         # change total packet size <<<

                       STLVmWrFlowVar(fv_name="fv_rand",
                                      pkt_offset= "IP.len",
                                      add_val=l3_len_fix), # fix ip len

                       STLVmFixIpv4(offset = "IP"),                       # fix checksum

                       STLVmWrFlowVar(fv_name="fv_rand",
                                      pkt_offset= "UDP.len",
                                      add_val=l4_len_fix) # fix udp len
                      ]
                   )

    pkt = STLPktBuilder(pkt = base_pkt,
                        vm = vm)

    return STLStream(packet = pkt,
                     mode = STLTXCont())
get_var_ref()[source]

Virtual function returns a ref var name.

STLVmTupleGen

class trex.stl.trex_stl_packet_builder_scapy.STLVmTupleGen(name, ip_min='0.0.0.1', ip_max='0.0.0.10', port_min=1025, port_max=65535, limit_flows=100000, flags=0)[source]

Generate a struct with two variables: var_name.ip as uint32_t and var_name.port as uint16_t The variables are dependent. When the ip variable value reaches its maximum, the port is incremented.

For:

  • ip_min = 10.0.0.1

  • ip_max = 10.0.0.5

  • port_min = 1025

  • port_max = 1028

  • limit_flows = 10

The result:

ip

port

flow_id

10.0.0.1

1025

1

10.0.0.2

1025

2

10.0.0.3

1025

3

10.0.0.4

1025

4

10.0.0.5

1025

5

10.0.0.1

1026

6

10.0.0.2

1026

7

10.0.0.3

1026

8

10.0.0.4

1026

9

10.0.0.5

1026

10

10.0.0.1

1025

1

Parameters
namestring

Name of the stream struct.

ip_minstring or int

Min value of the ip value. Number or IPv4 format.

ip_maxstring or int

Max value of the ip value. Number or IPv4 format.

port_minint

Min value of port variable.

port_maxint

Max value of port variable.

limit_flowsint

Limit of number of flows.

flagsint

0 - noop 1 - ignore port min and max.

# Example5

def create_stream (self):
    # pkt 
    p_l2  = Ether();
    p_l3  = IP(src="16.0.0.1",dst="48.0.0.1")
    p_l4  = UDP(dport=12,sport=1025)
    pyld_size = max(0, self.max_pkt_size_l3 - len(p_l3/p_l4));
    base_pkt = p_l2/p_l3/p_l4/('U'*(pyld_size))

    l3_len_fix =-(len(p_l2));
    l4_len_fix =-(len(p_l2/p_l3));


    # vm
    vm = STLScVmRaw( [ STLVmFlowVar(name="fv_rand", min_value=64, 
                                    max_value=len(base_pkt), 
                                    size=2, op="inc"),

                       STLVmTrimPktSize("fv_rand"),                         # change total packet size <<<

                       STLVmWrFlowVar(fv_name="fv_rand", 
                                      pkt_offset= "IP.len", 
                                      add_val=l3_len_fix), # fix ip len 

                       STLVmFixIpv4(offset = "IP"),                       # fix checksum

                       STLVmWrFlowVar(fv_name="fv_rand", 
                                      pkt_offset= "UDP.len", 
                                      add_val=l4_len_fix) # fix udp len  
                      ]
                   )

    pkt = STLPktBuilder(pkt = base_pkt,
                        vm = vm)

    return STLStream(packet = pkt,
                     mode = STLTXCont())
get_var_name()[source]

Virtual function returns the varible name if it exists.

STLVmFlowVarRepeatableRandom

class trex.stl.trex_stl_packet_builder_scapy.STLVmFlowVarRepeatableRandom(name, size=4, limit=100, seed=None, min_value=0, max_value=None, split_to_cores=True, next_var=None)[source]

Flow variable instruction for repeatable random with limit number of generating numbers. Allocates memory on a stream context. The size argument determines the variable size. Could be 1,2,4 or 8

  1. The maximum number of distinct values will ‘limit’. There could be a case of repetition

  2. The values will be repeated after ‘limit’ number of values.

Parameters
namestring

Name of the stream variable

sizeint

Number of bytes of the variable. Possible values: 1,2,4,8 for uint8_t, uint16_t, uint32_t, uint64_t

limitint

The number of distinct repetable random number

seedint

For deterministic result, you can set this to a uint16_t number

min_valueint

Min value

max_valueint

Max value

split_to_coresbool

Default value is True. If the value is True then each core updates the variable by step * num of cores. If split_to_cores = False then each core updates the variable by step.

next_varstring or None

In case it isn’t None it is the name of the next variable. The next variable will perform its operation only when this variable completes a full wrap around.

# Example1

# input , 1 byte or random with limit of 5 
STLVmFlowVarRepeatableRandom("var1",size=1,limit=5)

# output 255,1,7,129,8, ==> repeat 255,1,7,129,8

STLVmFlowVarRepeatableRandom("var1",size=4,limit=100,min_value=0x12345678, max_value=0x32345678)
get_var_name()[source]

Virtual function returns the varible name if it exists.

get_next_var_name()[source]

Virtual function that returns the name of the next variable name if it exists.

Field Engine snippet

# FE Example1


    base_pkt = Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)

    pad = max(0, size - len(base_pkt)) * 'x'

    vm = STLScVmRaw( [   STLVmTupleGen ( ip_min="16.0.0.1", ip_max="16.0.0.2",
                                         port_min=1025, port_max=65535,
                                         name="tuple"), # define tuple gen

                         # write ip to packet IP.src
                         STLVmWrFlowVar (fv_name="tuple.ip", pkt_offset= "IP.src" ),

                         STLVmFixIpv4(offset = "IP"),  # fix checksum
                         STLVmWrFlowVar (fv_name="tuple.port", pkt_offset= "UDP.sport" )  #write udp.port
                              ]
                          );

    pkt = STLPktBuilder(pkt = base_pkt/pad,
                        vm = vm)
# FE Example2


    #range of source mac-addr

    base_pkt =  Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)
    pad = max(0, size - len(base_pkt)) * 'x'

    vm = STLScVmRaw( [ STLVmFlowVar(name="mac_src",
                                    min_value=1,
                                    max_value=30,
                                    size=2, op="dec",step=1),
                       STLVmWrMaskFlowVar(fv_name="mac_src",
                                          pkt_offset= 11,
                                          pkt_cast_size=1,
                                          mask=0xff)
                     ]
                    )
# FE Example3


    #IP dest would have 10 random values betwean  48.0.0.1   48.0.0.255

    base_pkt = Ether()/IP(src=src_ip,dst=dst_ip)/UDP(dport=12,sport=1025)

    pad = max(0, size - len(base_pkt)) * 'x'

    vm = STLScVmRaw( [   STLVmFlowVar ( "ip_src",  min_value="10.0.0.1",
                                        max_value="10.0.0.255", size=4, step=1,op="inc"),
                         STLVmFlowVarRepeatableRandom ( "ip_dst",
                                                       min_value="48.0.0.1",
                                                       max_value="48.0.0.255",
                                                       size=4,
                                                       limit=10, seed=0x1235),

                         STLVmWrFlowVar (fv_name="ip_src", pkt_offset= "IP.src" ), # write ip to packet IP.src
                         STLVmWrFlowVar (fv_name="ip_dst", pkt_offset= "IP.dst" ), # write ip to packet IP.dst

                         STLVmFixIpv4(offset = "IP")                                # fix checksum
                              ]
                          ,split_by_field = "ip_src"  # split to cores base on the tuple generator
                          ,cache_size = cache_size # the cache size
                          );