Field Engine modules

The Field Engine (FE) has limited number of instructions/operations to support most use cases. There is a plan to add LuaJIT to be more flexible at the cost of performance. The FE can allocate stream variables in a stream context, write a stream variable to a packet offset, change packet size, and so on.

Examples of Field Engine uses:

  • Change ipv4.tos 1-10
  • Change packet size to a random value in the range 64 to 9K
  • Create a range of flows (change src_ip, dest_ip, src_port, dest_port)
  • Update IPv4 checksum

The following snippet creates a range of 64 bytes packets

# split the range of IP to cores
#
class STLS1(object):

  def __init__ (self):
      self.fsize  =64;

  def create_stream (self):
      # create a base packet and pad it to size
      size = self.fsize - 4; # no FCS

      base_pkt = Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)

      pad = max(0, size - len(base_pkt)) * 'x'

      vm = STLScVmRaw( [   STLVmFlowVar ( "ip_src",
                                          min_value="10.0.0.1",
                                          max_value="10.0.0.255",
                                          size=4,
                                          step=1,
                                          op="inc"),
                           STLVmWrFlowVar (fv_name="ip_src",
                                           pkt_offset= "IP.src" ), # write ip to packet IP.src
                           STLVmFixIpv4(offset = "IP")                                # fix checksum
                         ],
                         split_by_field = "ip_src",
                         cache_size =255 # cache the packets, much better performance
                      );

      pkt = STLPktBuilder(pkt = base_pkt/pad,
                          vm = vm)
      stream = STLStream(packet = pkt,
                       mode = STLTXCont())
      #print(stream.to_code())
      return stream


  def get_streams (self, direction = 0, **kwargs):
      # create 1 stream
      return [ self.create_stream() ]

The following snippet creates a SYN attack:

# create attack from random src_ip from 16.0.0.0-18.0.0.254 and random src_port 1025-65000
# attack 48.0.0.1 server

def create_stream (self):


    # TCP SYN
    base_pkt  = Ether()/IP(dst="48.0.0.1")/TCP(dport=80,flags="S")


    # vm
    vm = STLScVmRaw( [ STLVmFlowVar(name="ip_src",
                                          min_value="16.0.0.0",
                                          max_value="18.0.0.254",
                                          size=4, op="random"),

                        STLVmFlowVar(name="src_port",
                                          min_value=1025,
                                          max_value=65000,
                                          size=2, op="random"),

                       STLVmWrFlowVar(fv_name="ip_src", pkt_offset= "IP.src" ),

                       STLVmFixIpv4(offset = "IP"), # fix checksum

                       STLVmWrFlowVar(fv_name="src_port",
                                            pkt_offset= "TCP.sport") # fix udp len

                      ]
                   )

    pkt = STLPktBuilder(pkt = base_pkt,
                        vm = vm)

    return STLStream(packet = pkt,
                     random_seed = 0x1234,# can be remove. will give the same random value any run
                     mode = STLTXCont())

STLScVmRaw class

Aggregate raw instructions objects

class trex.stl.trex_stl_packet_builder_scapy.STLScVmRaw(list_of_commands=None, cache_size=None)[source]

Raw instructions

Include a list of a basic instructions objects.

Parameters:
list_of_commands : list

list of instructions

cache_size : uint16_t

In case it is bigger than zero, FE results will be cached - this will speedup of the program at the cost of limiting the number of possible packets to the number of cache. The cache size is limited to the pool size

The following example splits the generated traffic by “ip_src” variable.

# Split by 

# TCP SYN
base_pkt  = Ether()/IP(dst="48.0.0.1")/TCP(dport=80,flags="S")     


# vm
vm = STLScVmRaw( [ STLVmFlowVar(name="ip_src", 
                                      min_value="16.0.0.0", 
                                      max_value="16.0.0.254", 
                                      size=4, op="inc"),                     


                   STLVmWrFlowVar(fv_name="ip_src", pkt_offset= "IP.src" ),  

                   STLVmFixIpv4(offset = "IP"), # fix checksum              
                  ]
                 cache_size = 1000
               )

STLVmFlowVar

class trex.stl.trex_stl_packet_builder_scapy.STLVmFlowVar(name, init_value=None, min_value=0, max_value=255, size=4, step=1, op='inc', value_list=None, split_to_cores=True, next_var=None)[source]

Flow variable instruction. Allocates a variable on a stream context. The size argument determines the variable size. The operation can be inc, dec, and random. For increment and decrement operations, can set the “step” size. For all operations, can set initialization value, minimum and maximum value.

Parameters:
name : string

Name of the stream variable

init_value : int

Init value of the variable. If not specified, it will be min_value

min_value : int

Min value

max_value : int

Max value

size : int

Number of bytes of the variable. Possible values: 1,2,4,8 for uint8_t, uint16_t, uint32_t, uint64_t

step : int

Step in case of “inc” or “dec” operations

op : string

Possible values: “inc”, “dec”, “random”

value_list : int

List of values instead of init_value, min_value and max_value

split_to_cores : bool

Default value is True. If the value is True then each core updates the variable by step * num of cores. If split_to_cores = False then each core updates the variable by step.

next_var : string or None

In case it isn’t None it is the name of the next variable. The next variable will perform its operation only when this variable completes a full wrap around.

# Example1

# input 
STLVmFlowVar(min_value=0, max_value=3, size=1,op="inc")

# output 0,1,2,3,0,1,2,3 ..

# input 
STLVmFlowVar(min_value=0, max_value=3, size=1,op="dec")

# output 0,3,2,1,0,3,2,1 ..


# input 
STLVmFlowVar(min_value=0, max_value=3, size=1,op="random")

# output 1,1,2,3,1,2,1,0 ..

# input 
STLVmFlowVar(min_value=0, max_value=10, size=1,op="inc",step=3)

# output 0,3,6,9,0,3,6,9,0..

# Example2

# input value_list
STLVmFlowVar(value_list=["16.0.0.1","16.0.0.2","16.0.0.3","16.0.0.4"], op="inc"))

# output 268435457,268435458,268435459,268435460,268435457,268435458,268435459,268435460,268435457..

# input value_list
STLVmFlowVar(value_list=[9,0,4,7,5], op="dec"))

# output 5,7,4,0,9,5,7,4,0,9,5,7..

STLVmWrFlowVar

class trex.stl.trex_stl_packet_builder_scapy.STLVmWrFlowVar(fv_name, pkt_offset, offset_fixup=0, add_val=0, is_big=True)[source]

Write a stream variable into a packet field. The write position is determined by the packet offset + offset fixup. The size of the write is determined by the stream variable. Example: Offset 10, fixup 0, variable size 4. This function writes at 10, 11, 12, and 13.

For inromation about chaning the write size, offset, or fixup, see the STLVmWrMaskFlowVar command. The Field name/offset can be given by name in the following format: header[:id].field.

Parameters:
fv_name : string

Stream variable to write to a packet offset.

pkt_offset : string or int

Name of the field or offset in bytes from packet start.

offset_fixup : int

Number of bytes to move forward. If negative, move backward.

add_value : int

Value to add to the stream variable before writing it to the packet field. Can be used as a constant offset.

is_big : bool

How to write the variable to the the packet. True=big-endian, False=little-endian

# Example3

pkt = Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)


# write to ip.src offset 
STLVmWrFlowVar (fv_name="tuple", pkt_offset= "IP.src" )

# packet offset is varible 
STLVmWrFlowVar (fv_name="tuple", pkt_offset= 26 )

# add l3_len_fix before writing fv_rand into IP.len field 
STLVmWrFlowVar(fv_name="fv_rand", pkt_offset= "IP.len", add_val=l3_len_fix) 

STLVmWrMaskFlowVar

class trex.stl.trex_stl_packet_builder_scapy.STLVmWrMaskFlowVar(fv_name, pkt_offset, pkt_cast_size=1, mask=255, shift=0, add_value=0, offset_fixup=0, is_big=True)[source]

Write a stream variable into a packet field with some operations. Using this instruction, the variable size and the field can have different sizes.

Pseudocode of this code:

uint32_t val=(cast_to_size)rd_from_variable("name") # read flow-var
val+=m_add_value                                    # add value

if (m_shift>0) {                                    # shift 
    val=val<<m_shift
}else{
    if (m_shift<0) {
        val=val>>(-m_shift)
    }
}

pkt_val=rd_from_pkt(pkt_offset)                     # RMW to the packet
pkt_val = (pkt_val & ~m_mask) | (val & m_mask)
wr_to_pkt(pkt_offset,pkt_val)
Parameters:
fv_name : string

The stream variable name to write to a packet field

pkt_cast_size : uint8_t

The size in bytes of the packet field

mask : uint32_t

The mask of the field. 1 means to write. 0 don’t care

shift : uint8_t

How many bits to shift

pkt_offset : string or int

the name of the field or offset in byte from packet start.

offset_fixup : int

how many bytes to go forward. In case of a negative value go backward

add_val : int

value to add to stream variable before writing it to packet field. can be used as a constant offset

is_big : bool

how to write the variable to the the packet. is it big-edian or little edian

Example 1 - Cast from uint16_t (var) to uint8_t (pkt):

base_pkt =  Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)

vm = STLScVmRaw( [ STLVmFlowVar(name="mac_src", 
                                min_value=1, 
                                max_value=30, 
                                size=2, 
                                op="dec",step=1), 
                   STLVmWrMaskFlowVar(fv_name="mac_src", 
                                      pkt_offset= 11,
                                      pkt_cast_size=1, 
                                      mask=0xff) # mask command ->write it as one byte
              ]
           )

pkt = Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)

Example 2 - Change MSB of uint16_t variable:

vm = STLScVmRaw( [ STLVmFlowVar(name="mac_src", 
                                min_value=1, 
                                max_value=30, 
                                size=2, op="dec",step=1), 
                   STLVmWrMaskFlowVar(fv_name="mac_src", 
                                      pkt_offset= 10,
                                      pkt_cast_size=2, 
                                      mask=0xff00,
                                      shift=8) # take the var shift it 8 (x256) write only to LSB
                  ]
                )

Example 3 - Every 2 packets, change the MAC (shift right):

vm = STLScVmRaw( [ STLVmFlowVar(name="mac_src", 
                                min_value=1, 
                                max_value=30, 
                                size=2, op="dec",step=1), 
                   STLVmWrMaskFlowVar(fv_name="mac_src", 
                                      pkt_offset= 10,
                                      pkt_cast_size=1, 
                                      mask=0x1,
                                      shift=-1) # take var mac_src>>1 and write the LSB every two packet there should be a change
                 ]
                )

STLVmFixChecksumHw

class trex.stl.trex_stl_packet_builder_scapy.STLVmFixChecksumHw(l3_offset, l4_offset, l4_type)[source]

Fix IPv4 header checksum and/or TCP/UDP checksum using hardware assist. Use this if the packet header has changed or data payload has changed as it is necessary to fix the checksums. This instruction works on NICS that support this hardware offload.

For fixing only IPv4 header checksum use STLVmFixIpv4. This instruction should be used if both L4 and L3 need to be fixed.

example for supported packets

Ether()/IPv4 SomeTunnel()/IPv4 Ether()/(IPv4|IPv6)/(UDP|TCP) Ether()/(IPv4|IPv6)/(UDP|TCP) SomeTunnel()/(IPv4|IPv6)/(UDP|TCP) SomeTunnel()/(IPv4|IPv6)/(UDP|TCP)

Parameters:
l3_offset : offset in bytes

IPv4/IPv6 header offset from packet start. It is not the offset of the checksum field itself. in could be string in case of scapy packet. format IP[:[id]]

l4_offset : offset in bytes to UDP/TCP header or IPv4 payload. in case of IPv4 checksum CTRexVmInsFixHwCs.L4_TYPE_IP could be set to zero to be auto calculated by the server

l4_type : [CTRexVmInsFixHwCs.L4_TYPE_UDP or CTRexVmInsFixHwCs.L4_TYPE_TCP or CTRexVmInsFixHwCs.L4_TYPE_IP]

see full example stl/syn_attack_fix_cs_hw.py

# Example2

pkt = Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)

# by offset 
STLVmFixChecksumHw(l3_offset=14,l4_offset=14+20,l4_type=CTRexVmInsFixHwCs.L4_TYPE_UDP)

# in case of scapy packet can be defined by header name 
STLVmFixChecksumHw(l3_offset="IP",l4_offset="UDP",l4_type=CTRexVmInsFixHwCs.L4_TYPE_UDP)

# string for second "IP" header in the packet is IP:1
STLVmFixChecksumHw(offset="IP:1")

STLVmFixIpv4

class trex.stl.trex_stl_packet_builder_scapy.STLVmFixIpv4(offset)[source]

Fix IPv4 header checksum. Use this if the packet header has changed and it is necessary to change the checksum.

Parameters:
offset : uint16_t or string

IPv4 header offset from packet start. It is not the offset of the checksum field itself. in could be string in case of scapy packet. format IP[:[id]]

# Example2

pkt = Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)

# by offset 
STLVmFixIpv4(offset=14)

# in case of scapy packet can be defined by header name 
STLVmFixIpv4(offset="IP")

# string for second "IP" header in the packet is IP:1
STLVmFixIpv4(offset="IP:1")

STLVmTrimPktSize

class trex.stl.trex_stl_packet_builder_scapy.STLVmTrimPktSize(fv_name)[source]

Trim the packet size by the stream variable size. This instruction only changes the total packet size, and does not repair the fields to match the new size.

Parameters:
fv_name : string

Stream variable name. The value of this variable is the new total packet size.

For Example:

def create_stream (self):
    # pkt
    p_l2  = Ether();
    p_l3  = IP(src="16.0.0.1",dst="48.0.0.1")
    p_l4  = UDP(dport=12,sport=1025)
    pyld_size = max(0, self.max_pkt_size_l3 - len(p_l3/p_l4));
    base_pkt = p_l2/p_l3/p_l4/('U'*(pyld_size))

    l3_len_fix =-(len(p_l2));
    l4_len_fix =-(len(p_l2/p_l3));


    # vm
    vm = STLScVmRaw( [ STLVmFlowVar(name="fv_rand", min_value=64,
                                    max_value=len(base_pkt),
                                    size=2, op="inc"),

                       STLVmTrimPktSize("fv_rand"),                         # change total packet size <<<

                       STLVmWrFlowVar(fv_name="fv_rand",
                                      pkt_offset= "IP.len",
                                      add_val=l3_len_fix), # fix ip len

                       STLVmFixIpv4(offset = "IP"),                       # fix checksum

                       STLVmWrFlowVar(fv_name="fv_rand",
                                      pkt_offset= "UDP.len",
                                      add_val=l4_len_fix) # fix udp len
                      ]
                   )

    pkt = STLPktBuilder(pkt = base_pkt,
                        vm = vm)

    return STLStream(packet = pkt,
                     mode = STLTXCont())

STLVmTupleGen

class trex.stl.trex_stl_packet_builder_scapy.STLVmTupleGen(name, ip_min='0.0.0.1', ip_max='0.0.0.10', port_min=1025, port_max=65535, limit_flows=100000, flags=0)[source]

Generate a struct with two variables: var_name.ip as uint32_t and var_name.port as uint16_t The variables are dependent. When the ip variable value reaches its maximum, the port is incremented.

For:

  • ip_min = 10.0.0.1
  • ip_max = 10.0.0.5
  • port_min = 1025
  • port_max = 1028
  • limit_flows = 10

The result:

ip port flow_id
10.0.0.1 1025 1
10.0.0.2 1025 2
10.0.0.3 1025 3
10.0.0.4 1025 4
10.0.0.5 1025 5
10.0.0.1 1026 6
10.0.0.2 1026 7
10.0.0.3 1026 8
10.0.0.4 1026 9
10.0.0.5 1026 10
10.0.0.1 1025 1
Parameters:
name : string

Name of the stream struct.

ip_min : string or int

Min value of the ip value. Number or IPv4 format.

ip_max : string or int

Max value of the ip value. Number or IPv4 format.

port_min : int

Min value of port variable.

port_max : int

Max value of port variable.

limit_flows : int

Limit of number of flows.

flags : int

0 - noop 1 - ignore port min and max.

# Example5

def create_stream (self):
    # pkt 
    p_l2  = Ether();
    p_l3  = IP(src="16.0.0.1",dst="48.0.0.1")
    p_l4  = UDP(dport=12,sport=1025)
    pyld_size = max(0, self.max_pkt_size_l3 - len(p_l3/p_l4));
    base_pkt = p_l2/p_l3/p_l4/('U'*(pyld_size))

    l3_len_fix =-(len(p_l2));
    l4_len_fix =-(len(p_l2/p_l3));


    # vm
    vm = STLScVmRaw( [ STLVmFlowVar(name="fv_rand", min_value=64, 
                                    max_value=len(base_pkt), 
                                    size=2, op="inc"),

                       STLVmTrimPktSize("fv_rand"),                         # change total packet size <<<

                       STLVmWrFlowVar(fv_name="fv_rand", 
                                      pkt_offset= "IP.len", 
                                      add_val=l3_len_fix), # fix ip len 

                       STLVmFixIpv4(offset = "IP"),                       # fix checksum

                       STLVmWrFlowVar(fv_name="fv_rand", 
                                      pkt_offset= "UDP.len", 
                                      add_val=l4_len_fix) # fix udp len  
                      ]
                   )

    pkt = STLPktBuilder(pkt = base_pkt,
                        vm = vm)

    return STLStream(packet = pkt,
                     mode = STLTXCont())

STLVmFlowVarRepeatableRandom

class trex.stl.trex_stl_packet_builder_scapy.STLVmFlowVarRepeatableRandom(name, size=4, limit=100, seed=None, min_value=0, max_value=None, split_to_cores=True, next_var=None)[source]

Flow variable instruction for repeatable random with limit number of generating numbers. Allocates memory on a stream context. The size argument determines the variable size. Could be 1,2,4 or 8

  1. The maximum number of distinct values will ‘limit’. There could be a case of repetition
  2. The values will be repeated after ‘limit’ number of values.
Parameters:
name : string

Name of the stream variable

size : int

Number of bytes of the variable. Possible values: 1,2,4,8 for uint8_t, uint16_t, uint32_t, uint64_t

limit : int

The number of distinct repetable random number

seed : int

For deterministic result, you can set this to a uint16_t number

min_value : int

Min value

max_value : int

Max value

split_to_cores : bool

Default value is True. If the value is True then each core updates the variable by step * num of cores. If split_to_cores = False then each core updates the variable by step.

next_var : string or None

In case it isn’t None it is the name of the next variable. The next variable will perform its operation only when this variable completes a full wrap around.

# Example1

# input , 1 byte or random with limit of 5 
STLVmFlowVarRepeatableRandom("var1",size=1,limit=5)

# output 255,1,7,129,8, ==> repeat 255,1,7,129,8

STLVmFlowVarRepeatableRandom("var1",size=4,limit=100,min_value=0x12345678, max_value=0x32345678)

Field Engine snippet

# FE Example1


    base_pkt = Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)

    pad = max(0, size - len(base_pkt)) * 'x'

    vm = STLScVmRaw( [   STLVmTupleGen ( ip_min="16.0.0.1", ip_max="16.0.0.2",
                                         port_min=1025, port_max=65535,
                                         name="tuple"), # define tuple gen

                         # write ip to packet IP.src
                         STLVmWrFlowVar (fv_name="tuple.ip", pkt_offset= "IP.src" ),

                         STLVmFixIpv4(offset = "IP"),  # fix checksum
                         STLVmWrFlowVar (fv_name="tuple.port", pkt_offset= "UDP.sport" )  #write udp.port
                              ]
                          );

    pkt = STLPktBuilder(pkt = base_pkt/pad,
                        vm = vm)
# FE Example2


    #range of source mac-addr

    base_pkt =  Ether()/IP(src="16.0.0.1",dst="48.0.0.1")/UDP(dport=12,sport=1025)
    pad = max(0, size - len(base_pkt)) * 'x'

    vm = STLScVmRaw( [ STLVmFlowVar(name="mac_src",
                                    min_value=1,
                                    max_value=30,
                                    size=2, op="dec",step=1),
                       STLVmWrMaskFlowVar(fv_name="mac_src",
                                          pkt_offset= 11,
                                          pkt_cast_size=1,
                                          mask=0xff)
                     ]
                    )
# FE Example3


    #IP dest would have 10 random values betwean  48.0.0.1   48.0.0.255

    base_pkt = Ether()/IP(src=src_ip,dst=dst_ip)/UDP(dport=12,sport=1025)

    pad = max(0, size - len(base_pkt)) * 'x'

    vm = STLScVmRaw( [   STLVmFlowVar ( "ip_src",  min_value="10.0.0.1",
                                        max_value="10.0.0.255", size=4, step=1,op="inc"),
                         STLVmFlowVarRepeatableRandom ( "ip_dst",
                                                       min_value="48.0.0.1",
                                                       max_value="48.0.0.255",
                                                       size=4,
                                                       limit=10, seed=0x1235),

                         STLVmWrFlowVar (fv_name="ip_src", pkt_offset= "IP.src" ), # write ip to packet IP.src
                         STLVmWrFlowVar (fv_name="ip_dst", pkt_offset= "IP.dst" ), # write ip to packet IP.dst

                         STLVmFixIpv4(offset = "IP")                                # fix checksum
                              ]
                          ,split_by_field = "ip_src"  # split to cores base on the tuple generator
                          ,cache_size = cache_size # the cache size
                          );