Table of Contents

1. Change log

Version name meaning

1.00

Itay Marom (imarom)

  • first version

1.01

Dan Klein (danklei)

  • added usage examples using Python code as Higher-level usage

  • added logic and explanation behind VM commands

1.1

Dan Klein (danklei)

  • Fixed some consistency issues

  • added RPC interaction examples appendix

1.2

Hanoch Haim (hhaim)

  • add tuple generator command

1.3

Hanoch Haim (hhaim)

  • update VM instructions

1.4

Hanoch Haim (hhaim)

  • add random trim instruction

1.5

Hanoch Haim (hhaim)

  • add more instructions (v1.92)

1.6

Itay Marom (imarom)

  • added API synchronization

2. Audience of this document

Anyone that wants to understand the low level protocol to TRex server. for example a GUI developer that wants to develop a GUI for TRex Server.

3. RPC Support On TRex

TRex implements a RPC protocol in order to config, view and in general execute remote calls on TRex

In this document we will provide information on how a client can implement the protocol used to communicate with TRex

In general, we will describe the following:

  • Transport Layer - The transport layer used to communicate with TRex server

  • RPC Reprensentation Protocol - The format in which remote procedures are carried

3.1. Transport Layer

TRex server transport layer is implemented using ZMQ.

The default configuration is TCP on port 5555, however this is configurable.


The communication model is based on the request-reply ZMQ model:


for more on ZMQ and implementation please refer to: ​
http://zeromq.org/intro:read-the-manual

3.2. RPC Reprensentation Protocol

The RPC reprensentation protocol is JSON RPC v2.0. Every request and response will be encoded in a JSON RPC v2.0 format.

​+ For more info on JSON RPC v2.0 spec please refer to: ​+

​+

Later on in the document we will describe all the supported commands.

3.3. TRex Console

To debug RPC it is possible to enable verbose command from Console see here

On the client side:

TRex > verbose on

verbose set to on

TRex > ping

-> Pinging RPC server
[verbose] Sending Request To Server:

{
    "id": "l0tog11a",
    "jsonrpc": "2.0",
    "method": "ping",
    "params": null
}

[verbose] Server Response:

{
    "id": "l0tog11a",
    "jsonrpc": "2.0",
    "result": {}
}

[SUCCESS]

4. RPC Server Component Position Illustration

The following diagram illustres the RPC server component’s place:

Figure 1. RPC Server Position

5. RPC Server Port State Machine

Any port on the server can be in numbered of states, each state provides other subset of the commands that are allowed to be executed.

We define the following possible states:

  • unowned - The specific port is either unowned or another user is owning the port

  • owned - The specific port has been acquired by the client

  • active - The specific port is in the middle of injecting traffic - currently active

Each port command will specify on which states it is possible to execute it.

For port related commands valid only on owned or active, a field called 'handler' MUST be passed along with the rest of the parameters.

This will identify the connection:

Figure 2. Port States

6. RPC Commands

The following RPC commands are supported

6.1. API Synchronization

  • Name - api_sync

  • API Class - None

  • Valid States - not relevant

  • Description - Sync with server about API classes. This allows the server and the client to be sure they are fully synced. The return values are used for furthur communication with the server. every API from a specific class requires its corresponding api_h parameter added to the specific parameters of the function.

  • Parameters -

    • api_vers [list] - A list of objects of type api_class

  • Result [object] - A list of objects of type api_class_rc

Table 1. Object type api_class
Field Type Description

type

string

name of the API class

major

int

major version

minor

int

minor version

Table 2. Object type api_class_rc
Field Type Description

type

string

name of the API class

api_h

string

API handler for this API class

Example:

'Request':

{
    "id": "6d4e9gs3",
    "jsonrpc": "2.0",
    "method": "api_sync",
    "params": {
        "api_vers": [
            {
                "type": "core"
                "major": 1,
                "minor": 0,
            }
        ]
    }
}

'Response':

{
    "id": "6d4e9gs3",
    "jsonrpc": "2.0",
    "result": {
        "api_vers": [
         {
            "type": "core"
            "api_h": "SPhoCDIV",
         }
        ]
    }
}

6.2. Ping

  • Name - ping

  • API Class - None

  • Valid States - not relevant

  • Description - Pings the TRex server

  • Parameters - None

  • Result [object] - {}

Example:

'Request':

{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "ping",
    "params": null
}

'Response':

{
   "jsonrpc" : "2.0",
   "id" : 1,
   "result" : {}
}

6.3. Get Server Supported Commands

  • Name - get_supported_cmds

  • API Class - core

  • Valid States - not relevant

  • Description - Queries the server for all the supported commands

  • Parameters - None

  • Result [array] - A list of all the supported commands by the server

Example:

'Request':

{
    "id": "7rqf0xyd",
    "jsonrpc": "2.0",
    "method": "get_supported_cmds",
    "params": {
        "api_h": "VGDJwdiY"
    }
}


'Response':

{
    "id": "7rqf0xyd",
    "jsonrpc": "2.0",
    "result": [
        "push_remote",
        "validate",
        "start_traffic",
        "get_all_streams",
        "shutdown",
        "get_stream",
        "test_add",
        "stop_traffic",
        "get_utilization",
        "release",
        "test_sub",
        "api_sync",
        "get_port_status",
        "get_port_stats",
        "publish_now",
        "get_system_info",
        "get_supported_cmds",
        "get_version",
        "get_port_xstats_names",
        "update_traffic",
        "get_active_pgids",
        "pause_traffic",
        "get_owner",
        "acquire",
        "set_port_attr",
        "get_port_xstats_values",
        "remove_rx_filters",
        "resume_traffic",
        "add_stream",
        "remove_stream",
        "remove_all_streams",
        "ping",
        "get_stream_list"
    ]
}

6.4. Get Version

  • Name - get_version

  • API Class - core

  • Valid States - not relevant

  • Description - Queries the server for version information

  • Parameters - None

  • Result [object] - See table below

Table 3. Object type return values for get_version
Field Type Description

version

string

TRex version

build_date

string

build date

build_time

string

build time

built_by

string

who built this version


'Request':

{
    "id": "wapkk8m6",
    "jsonrpc": "2.0",
    "method": "get_version",
    "params": {
        "api_h": "SPhoCDIV"
    }
}


'Response':

{
    "id": "wapkk8m6",
    "jsonrpc": "2.0",
    "result": {
        "build_date": "Sep 16 2015",
        "build_time": "12:33:01",
        "built_by": "imarom",
        "version": "v0.0"
    }
}

6.5. Get System Info

  • Name - get_system_info

  • API Class - core

  • Description - Queries the server for system properties

  • Parameters - None

  • Result [object] - See table below

Table 4. return value: get_system_info
Field Type Description

dp_core_count

int

DP core count (total)

dp_core_count_per_port

int

DP core count per pair of ports

core_type

string

DP core type

hostname

string

machine host name

uptime

string

uptime of the server

port_count

int

number of ports on the machine

ports

array

array of object 'ports' - see below

Table 5. return value: get_system_info.ports
Field Type Description

description

string

description of port

driver

string

driver type

numa

int

NUMA of port

pci_addr

string

PCI address of port

hw_macaddr

string

HW MAC of port (masked by src_macaddr)

src_macaddr

string

src MAC of port

dst_macaddr

string

dest MAC of port

is_virtual

bool

is port virtual

is_fc_supported

bool

is flow control supported

is_led_supported

bool

is led on/off supported

is_link_supported

bool

is link status change supported

index

int

port index

speed

int

current max speed of the port (1, 10, 40, 100)

supp_speeds

array

list of max speeds supported by port

rx

object

see below

Table 6. return value: get_system_info.ports.rx
Field Type Description

caps

array

list of capabilities: "flow_stats", "latency" etc.

counters

int

number of different pg_ids one can use at the same time


'Request':

{
    "id": "kn92anod",
    "jsonrpc": "2.0",
    "method": "get_system_info",
    "params": {
        "api_h": "o8VEJEOd"
    }
}

'Response':

{
    "id": "kn92anod",
    "jsonrpc": "2.0",
    "result": {
        "core_type": "Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz",
        "dp_core_count": 1,
        "dp_core_count_per_port": 1,
        "hostname": "csi-trex-11",
        "port_count": 2,
        "ports": [
            {
                "description": "VMXNET3 Ethernet Controller",
                "driver": "rte_vmxnet3_pmd",
                "dst_macaddr": "00:0c:29:2a:99:bc",
                "hw_macaddr": "00:0c:29:2a:99:b2",
                "index": 0,
                "is_fc_supported": false,
                "is_led_supported": false,
                "is_link_supported": false,
                "is_virtual": true,
                "numa": -1,
                "pci_addr": "0000:03:00.0",
                "rx": {
                    "caps": [
                        "flow_stats",
                        "latency",
                        "rx_bytes"
                    ],
                    "counters": 127
                },
                "speed": 10,
                "src_macaddr": "00:0c:29:2a:99:b2",
                "supp_speeds": [
                    10000
                ]
            },
            {
                "description": "VMXNET3 Ethernet Controller",
                "driver": "rte_vmxnet3_pmd",
                "dst_macaddr": "00:0c:29:2a:99:b2",
                "hw_macaddr": "00:0c:29:2a:99:bc",
                "index": 1,
                "is_fc_supported": false,
                "is_led_supported": false,
                "is_link_supported": false,
                "is_virtual": true,
                "numa": -1,
                "pci_addr": "0000:0b:00.0",
                "rx": {
                    "caps": [
                        "flow_stats",
                        "latency",
                        "rx_bytes"
                    ],
                    "counters": 127
                },
                "speed": 10,
                "src_macaddr": "00:0c:29:2a:99:bc",
                "supp_speeds": [
                    10000
                ]
            }
        ],
        "uptime": "Oct 31 2016 @ 16:25:28"
    }
}

6.6. Get Port Status

  • Name - get_port_status

  • API Class - core

  • Valid States - all

  • Description - Queries the server for status

  • Parameters -

    • port_id [int] - port id to query for owner

  • Result [object] - see below


'Request':

{
    "id": "oveilf0n",
    "jsonrpc": "2.0",
    "method": "get_port_status",
    "params": {
        "api_h": "VGDJwdiY",
        "port_id": 7
    }
}

'Response':

{
    "id": "oveilf0n",
    "jsonrpc": "2.0",
    "result": {
        "attr": {
            "fc": {
                "mode": 0
            },
            "link": {
                "up": true
            },
            "promiscuous": {
                "enabled": false
            }
        },
        "max_stream_id": 3,
        "owner": "",
        "speed": 10000,
        "state": "TX"
    }
}
Table 7. return value: get_port_status
Field Type Description

owner

string

name of current owner (or "" if none)

state

string

state of port (DOWN, IDLE, STREAMS, TX, PAUSE)

6.7. Acquire

  • Name - Acquire

  • API Class - core

  • Valid States - all

  • Description - Takes ownership over the port

  • Parameters -

    • port_id [int] - port id to take ownership

    • user [string] - User name aquiring the system

    • force [boolean] - force action even if another user is holding the port

  • Result [string] - handler for future sessions


'Request':

{
    "id": "b1tr56yz",
    "jsonrpc": "2.0",
    "method": "Acquire",
    "params": {
        "api_h": "SPhoCDIV",
        "user": "itay",
        "port_id": 1,
        "force": false,
    }
}


'Response':

{
    "id": "b1tr56yz",
    "jsonrpc": "2.0",
    "result": "AQokC3ZA"
}

6.8. Release

  • Name - release

  • API Class - core

  • Valid States - owned

  • Description - Release owernship over the device

  • Parameters -

    • handler [string] - unique connection handler

    • port_id [int] - port id to release

  • Result [object] - {}


'Request':

{
    "id": "m785dxwd",
    "jsonrpc": "2.0",
    "method": "release",
    "params": {
        "api_h": "SPhoCDIV",
        "handler": "37JncCHr",
        "port_id": 1
    }
}


'Response':

{
    "id": "m785dxwd",
    "jsonrpc": "2.0",
    "result": {}
}

6.9. Add Stream

  • Name - add_stream

  • API Class - core

  • Valid States - owned

  • Description - Adds a stream to a port

  • Parameters

    • handler [string] - unique connection handler

    • port_id [int] - port id associated with this stream

    • stream_id [int] - stream id associated with the stream object

    • stream - object of type stream

  • Result [object] - {}

The object type stream

Add_stream gets a single parameter of type object.

The format of that object is as follows:

Table 8. Object type stream
Field Type Description

enabled

boolean

is this stream enabled

self_start

boolean

is this stream triggered by starting injection or triggered by another stream

action_count

uint16_t

In case it is bigger than zero and next stream is not -1 (set) the number of goto will be limited to this number. Maximum value is 65K. default is zero. Zero means - not limit.

random_seed

uint32_t

For creating reproducible tests with random number, each stream can get a seed. this field is optional. In case of zero the seed value won’t be taken

flags

uint16_t

bit 0 (LSB) : 1 - take the src MAC from the packet instead of config file. bit 1-2 (LSB) how to set the dest MAC ( stCFG_FILE = 0, stPKT = 1,stARP = 2 )

isg

double

[usec] inter stream gap - delay time in usec until the stream is started

next_stream_id

int

next stream to start after this stream. -1 means stop after this stream

packet

object

object of type packet

mode

object

object of type mode

vm

object

array of objects of type vm

rx_stats

object

object of type rx_stats

6.9.1. packet

packet contains binary and meta data

Table 9. Object type packet
Field Type Description

binary

byte array

binary dump of the packet to be used in the stream as array of bytes

meta

string

meta data object. opaque to the RPC server. will be passed on queries

6.9.2. mode

mode object can be one of the following objects:

Table 10. Object type rate
Field Type Description

type

string

['pps','bps_L1','bps_L2','percentage'

value

double

rate

Table 11. Object type mode - continuous
Field Type Description

type

string

'continuous'

rate

object

rate object

Table 12. Object type mode - single_burst
Field Type Description

type

string

'single_burst'

rate

object

rate object

total pkts

int

total packets in the burst

Table 13. Object type mode - multi_burst
Field Type Description

type

string

'multi_burst'

rate

object

rate object

pkts_per_burst

int

packets in a single burst

ibg

double

[usec] inter burst gap. delay between bursts in usec

count

int

number of bursts. '0' means loop forever, '1' will fall back to single burst

6.9.3. vm

an Object that include instructions array and properties of the field engine program

Table 14. Object type packet
Field Type Description

Instructions

array

list of instructional objects

split_by_var

string

name of the field by which to split into threads

Restart

boolean

restart the field engine program when stream moving from inactive→active

Array of VM instruction objects to be used with this stream Any element in the array can be one of the following object types:

fix_checksum_hw

Fix TCP/UDP and IPv4 headers using hardware assit engine

Table 15. Object type vm - fix_checksum_hw
Field Type Description

type

string

'fix_checksum_hw'

l2_len

uint16

len of L2 (e.g. 14 Ether)

l3_len

uint16

len of l3 header (e.g. 20 for IP)

l4_type

uint16

the type of L4 header either UDP or TCP ( L4_TYPE_UDP = 11

fix_checksum_ipv4
Table 16. Object type vm - fix_checksum_ipv4
Field Type Description

type

string

'fix_checksum_ipv4'

pkt_offset

uint16

offset of the field to fix

flow_var
Table 17. Object type vm - flow_var
Field Type Description

type

string

'flow_var''

name

string

flow var name - this should be a unique identifier

size

[1,2,4,8]

size of the flow var in bytes

op

[inc, dec, random]

operation type to perform on the field

init_value

uint64_t as string

init value for the field

min_value

uint64_t as string

minimum value for the field

max_value

uint64_t as string

maximum value for the field

step

uint64_t as string

step, how much to inc or dec. 1 is the default (in case of random this field is not used)

repetable_random

Instruction to choose a limited number of random values from a big range The values could be deterministic by providing seed

Table 18. Object type vm - flow_var
Field Type Description

type

string

'flow_var_rand_limit''

name

string

flow var name - this should be a unique identifier

size

[1,2,4,8]

size of the var in bytes

limit

uint64_t as string

the number of values to choose

seed

uint64_t as string

seed of the random, in case there is no seed time will be taken

min_value

uint64_t as string

minimum value for the field

max_value

uint64_t as string

maximum value for the field

an example of tuple_flow_var variable

 size       = 2
 limit      = 5
 seed       = 0x1234
 min_value  = 0
 max_value  = 10

results could be

7 , 8, 1 ,5, 2 ,  7 , 8, 1 ,5, 2, 7 , 8, 1 ,5, 2
write_flow_var
Table 19. Object type vm - write_flow_var
Field Type Description

type

string

'write_flow_var'

name

string

flow var name to write

pkt_offset

uint16

offset at the packet to perform the write

add_value

int

delta to add to the field prior to writing - can be negative

is_big_endian

boolean

should write as big endian or little

trim_pkt_size
Table 20. Object type vm - trim_pkt_size
Field Type Description

type

string

'trim_pkt_size'

name

string

flow var name to take the new trim packet size from. The var size should be valid packet size and less than template packet size. see stl/udp_rand_size.yaml for an example

tuple_flow_var
Table 21. Object type vm - tuple_flow_var
Field Type Description

type

string

'tuple_flow_var''

name

string

tuple generator name - this should be a unique identifier name.ip and name.port will be added

ip_min

uint32_t as string

ipv4 min ip as uint32_t e.g. 10.0.0.1

ip_max

uint32_t as string

ipv4 max ip as uint32_t e.g. 10.0.1.255

port_min

uint16_t as string

ipv4 min port as uint16_t e.g. 1025

port_max

uint16_t as string

ipv4 max port as uint16_t e.g. 65000

limit_flows

uint32_t as string

the number of flows. 0 means we will use all the ip/port min-max range

flags

uint16_t as string

1 - unlimited number of flows. in case the first bit is enabled port_min and port_max is ignored and the maximum number of flows will be generated on those ips

an example of tuple_flow_var variable

 ip_min      = 10.0.0.1
 ip_max      = 10.0.0.5
 port_min    = 1025
 port_max    = 1028
 limit_flows = 10
Table 22. Results
IP PORT FLOW

10.0.0.1

1025

1

10.0.0.2

1025

2

10.0.0.3

1025

3

10.0.0.4

1025

4

10.0.0.5

1025

5

10.0.0.1

1026

6 << the port is inc here

10.0.0.2

1026

7

10.0.0.3

1026

8

10.0.0.4

1026

9

10.0.0.5

1026

10

10.0.0.1

1025

1 << back to the first flow

The variable name.port and name.ip could be written to any offset in the packet (usualy to src_ip and src_port as client)

write_mask_flow_var
Table 23. Object type vm - write_mask_flow_var
Field Type Description

type

string

'write_mask_flow_var''

name

string

flow variable name

pkt_offset

uint16_t as string

offset at the packet to perform the write

add_value

int32_t as string

delta to add to the field prior to writing - can be negative

pkt_cast_size

uint_t as string

size in bytes only 1,2,4 are valid

mask

uint32_t as string

1 means care e.g. 0xff will write to only 8 LSB bits

shift

int8_t as string

Positive will shift left (multiply by x2) negative will shift right (divided by 2) e.g. 1 will multiply by 2

is_big_endian

boolean

should write as big endian or little

Pseudocode
        uint32_t val=(cast_to_size)rd_from_varible("name"); # read flow-var
        val+=m_add_value;                                   # add value

        if (m_shift>0) {                                    # shift
            val=val<<m_shift;
        }else{
            if (m_shift<0) {
                val=val>>(-m_shift);
            }
        }

        pkt_val=rd_from_pkt(pkt_offset)                     # RMW
        pkt_val = (pkt_val & ~m_mask) | (val & m_mask)
        wr_to_pkt(pkt_offset,pkt_val)

an example of tuple_flow_var variable

 name          = "a" (varible 2 byte start 1-10 inc )
 pkt_cast_size = 1 ( cast to uint8_t )
 add_value     = 0
 mask          = 0xf0
 shift         = 4
 is_big_endian =1
Table 24. Results
var "a" PKT- before write PKT post write

1

0x03

0x13

2

0x03

0x23

3

0x03

0x33

4

0x03

0x43

5

0x03

0x53

The use cases of this instruction is to write to a bit field (valn/mpls)

Tip For more information and examples on VM objects please refer to: VM examples

6.9.4. rx_stats

Describes rx stats for the stream


Important In case rx_stats is enabled, meta data will be written in the end of the packet. please also consider the following constraints:
Constrains
  • performance - this will have performance impact as rx packets will be examined

  • override - up to 10 bytes at the end of the packet will be overidden by the meta data required

The bytes needed for activating rx_stats
  • stream_id consumes 2 bytes

  • seq_enabled consumes 4 bytes

  • latency_enabled consumes 4 bytes

so if no seq or latency are enabled 2 bytes will be used.

if seq or latency alone are enabled, 6 bytes will be used.

if both are enabled then 10 bytes will be used.

Table 25. Object type rx_stats
Field Type Description

enabled

boolean

is rx_stats enabled for this stream

stream_id

int

stream_id for which to collect rx_stats.
This could be stream_id different from the stream object which contains the rx_stats object.

seq_enabled

boolean

should write 32 bit sequence

latency_enabled

boolean

should write 32 bit latency


'Request':

{
    "id": 1,
    "jsonrpc": "2.0",
    "method": "add_stream",
    "params": {
        "api_h": "SPhoCDIV",
        "handler": "37JncCHr",
        "port_id": 1,
        "stream_id": 502
        "stream": {
            "enabled": true,
            "isg": 4.3,
            "mode": {
                    "rate": {
                        "type": "pps",
                        "value": 10
                    },

                "total_pkts": 5000,
                "type": "single_burst"
            },
            "next_stream_id": -1,
            "packet": {
                "binary": [
                    4,
                    1,
                    255
                ],
                "meta": ""
            },
            "rx_stats": {
                "enabled": false
            },
            "self_start": true,
        }
    }
}

'Response':

{
    "id": 1,
    "jsonrpc": "2.0",
    "result": {}
}

This request-reply sequence demonstrate a method in which rx_stats are diabled. In case rx_stats feature is enabled, rx_object must include all rx_stats object fields as described above.

6.10. Remove Stream

  • Name - remove_stream

  • API Class - core

  • Valid States - owned

  • Description - Removes a stream from a port

  • Parameters

    • handler [string] - unique connection handler

    • port_id [int] - port assosicated with the stream.

    • stream_id [int] - stream to remove

  • Result [object] - {}


'Request':

{
    "id": 1
    "jsonrpc": "2.0",
    "method": "remove_stream",
    "params": {
        "api_h": "SPhoCDIV",
        "handler": "37JncCHr",
        "port_id": 1,
        "stream_id": 502
    }
}


'Response':

{
    "id": 1
    "jsonrpc": "2.0",
    "result": {}
}

6.11. Get Stream ID List

  • Name - get_stream_list

  • API Class - core

  • Valid States - unowned, owned, active

  • Description - fetch all the assoicated streams for a port

  • Parameters

    • handler [string] - unique connection handler

    • port_id [int] - port to query for registered streams

  • Result [array] - array of stream_id


'Request':

{
    "id": 1,
    "jsonrpc": "2.0",
    "method": "get_stream_list",
    "params": {
        "api_h": "SPhoCDIV",
        "handler": "37JncCHr",
        "port_id": 1
    }
}

'Response':

{
    "id": 1,
    "jsonrpc": "2.0",
    "result": [
        502,
        18
    ]
}

6.12. Get Stream

  • Name - get_stream

  • API Class - core

  • Valid States - unowned, owned, active

  • Description - get a specific stream object

  • Parameters

    • handler [string] - unique connection handler

    • port_id [int] - port for the associated stream

    • stream_id [int] - the requested stream id

  • Result [object] - object stream


'Request':

{
    "id": 1,
    "jsonrpc": "2.0",
    "method": "get_stream",
    "params": {
        "api_h": "SPhoCDIV",
        "handler": "37JncCHr",
        "port_id": 1,
        "stream_id": 7
    }
}


'Response':

{
    "id": 1,
    "jsonrpc": "2.0",
    "result": {
        "stream": {
            "enabled": true,
            "isg": 4.3,
            "mode": {
                "pps": 3,
                "type": "continuous"
            },
            "next_stream_id": -1,
            "packet": {
                "binary": [
                    4,
                    1,
                    255
                ],
                "meta": ""
            },
            "self_start": true
        }
    }
}

6.13. Remove All Streams

  • Name - remove_all_streams

  • API Class - core

  • Valid States - owned

  • Description - remove all streams from a port

  • Parameters

    • handler [string] - unique connection handler

    • port_id [int] - port for the associated stream

  • Result [object] - {}


'Request':

{
    "id": 1,
    "jsonrpc": "2.0",
    "method": "remove_all_streams",
    "params": {
        "api_h": "SPhoCDIV",
        "handler": "37JncCHr",
        "port_id": 2
    }
}

'Response':

{
    "id": 1,
    "jsonrpc": "2.0",
    "result": {}
}

6.14. Start Traffic

  • Name - start_traffic

  • API Class - core

  • Valid States - owned

  • Description - Starts the traffic on a specific port. if traffic has already started an error will be returned

  • Parameters

    • handler [string] - unique connection handler

    • port_id [int] - port id on which to start traffic

    • core_mask [uint64] [optional] - a non zero mask to specify which cores will be active during TX, if no value is provided, the value is all bits on (MAX_UINT64)

  • Result [object] - {}


'Request':

{
    "id": "b3llt8hs",
    "jsonrpc": "2.0",
    "method": "start_traffic",
    "params": {
        "api_h": "SPhoCDIV",
        "handler": "37JncCHr",
        "port_id": 3
        "core_mask": 0xff
    }

'Response':

{
    "id": "b3llt8hs",
    "jsonrpc": "2.0",
    "result": {}
}

6.15. Stop Traffic

  • Name - stop_traffic

  • API Class - core

  • Valid States - active

  • Description - Stops the traffic on a specific port. if the port has already started nothing will happen

  • Parameters

    • handler [string] - unique connection handler

    • port_id [int] - port id on which to stop traffic

  • Result [object] - {}


'Request':

{
    "id": "h2fyhni7",
    "jsonrpc": "2.0",
    "method": "stop_traffic",
    "params": {
        "api_h": "SPhoCDIV",
        "handler": "37JncCHr",
        "port_id": 3
    }
}

'Response':

{
    "id": "h2fyhni7",
    "jsonrpc": "2.0",
    "result": {}
}

6.16. Remove RX Filters

  • Name - remove_rx_filters

  • API Class - core

  • Valid States - owned

  • Description - Post to calling stop, the client should call this function to remove any RX filters that were attached. this is because the server cannot know when it is safe to remove those (after stop some packets might take time to get to arrive - RTT)

  • Parameters

    • handler [string] - unique connection handler

    • port_id [int] - port id on which to remove all RX filters

  • Result [object] - {}


'Request':

{
    "id": "1jwrw9nx",
    "jsonrpc": "2.0",
    "method": "remove_rx_filters",
    "params": {
        "api_h": "SPhoCDIV",
        "handler": "ywVlqZa8",
        "port_id": 3
    }
}

'Response':

{
    "id": "1jwrw9nx",
    "jsonrpc": "2.0",
    "result": {}
}

6.17. Get Global Stats

  • Name - get_global_stats

  • API Class - core

  • Valid States - unowned, owned, active

  • Description - Get machine global stats

  • Parameters - None

  • Result [object] - See Below

Table 26. Return value of get_global_stats
Field Type Description

state

string

server state: can be unowned, owned or active

cpu_util

double

DP CPU util. in %

tx_bps

double

total TX bits per second

rx_bps

double

total RX bits per second

tx_pps

double

total TX packets per second

rx_pps

double

total RX packets per second

total_tx_pkts

int

total TX packets

total_rx_pkts

int

total RX packets

total_tx_bytes

int

total TX bytes

total_rx_bytes

int

total RX bytes

tx_rx_error

int

total Tx/Rx errors

6.18. Get Port Stats

  • Name - get_port_stats

  • API Class - core

  • Valid States - unowned, owned, active

  • Description - Get port stats

  • Parameters

    • port_id [int] - The port id for query

  • Result [object] - See Below

Table 27. Return value of get_port_stats
Field Type Description

status

string

down, idle or transmitting

tx_bps

double

total TX bits per second

rx_bps

double

total RX bits per second

tx_pps

double

total TX packets per second

rx_pps

double

total RX packets per second

total_tx_pkts

int

total TX packets

total_rx_pkts

int

total RX packets

total_rx_bytes

int

total TX bytes

total_tx_bytes

int

total RX bytes

tx_rx_error

int

total Tx/Rx errors

6.19. Get Stream Stats

  • Name - get_steram_stats

  • API Class - core

  • Valid States - unowned, owned, active

  • Description - Get port stats

  • Parameters

    • port_id [int] - The port id for query

    • stream_id [int] - The stream id for query

  • Result [object] - See Below

Table 28. Return value of get_stream_stats
Field Type Description

tx_bps

double

total TX bits per second

tx_pps

double

total TX packets per second

total_tx_pkts

int

total TX packets

total_tx_bytes

int

total TX bytes

rx_bps

double

total RX bits per second (if rx_stats enabled)

rx_pps

double

total RX packets per second (if rx_stats enabled)

total_rx_pkts

int

total RX packets (if rx_stats enabled)

total_rx_bytes

int

total RX bytes (if rx_stats enabled)

latency

array

array of 2 ordered elements average, maximum (if rx_stats enabled)

6.20. Get Utilization

  • Name - get_utilization

  • API Class - core

  • Valid States - all

  • Description - Get the CPU and MBUFs utilization.

  • Parameters - None

  • Result [object] -

    • cpu - The CPU utilization per DP thread (in their order). Each element is array of history (most latest is first in array, interval between values is 1 sec, values are integers 0-100).

    • mbuf_stats - The MBUFs are per CPU socket, per bucket of sizes: 64b, 9kb etc. Each bucket is array of 2 values: first is number of free elements and second is total.


'Request':

{
    "id": "vlxrc9r6",
    "jsonrpc": "2.0",
    "method": "get_utilization",
    "params": {
        "api_h": "2AIar0tl"
    }
}



'Response':

{'id': 'vlxrc9r6',
 'jsonrpc': '2.0',
 'result': {'cpu': [[30, 33, 28, 21, 30, 35, 33, 38, 29, 25, 37, 35, 31, 34, 35, 37, 26, 31, 26, 27],
                    [37, 31, 24, 21, 26, 32, 34, 36, 27, 28, 36, 33, 35, 35, 35, 36, 30, 39, 35, 40],
                    [32, 34, 25, 29, 28, 44, 37, 40, 32, 33, 38, 33, 33, 35, 33, 25, 22, 26, 27, 31],
                    [28, 33, 17, 21, 27, 36, 30, 30, 27, 25, 36, 35, 38, 43, 41, 28, 31, 32, 31, 41],
                    [27, 31, 27, 32, 26, 36, 27, 33, 30, 29, 29, 28, 30, 36, 33, 31, 26, 30, 25, 35],
                    [31, 31, 21, 24, 23, 31, 28, 33, 33, 33, 31, 21, 30, 33, 31, 23, 27, 29, 36, 36],
                    [27, 32, 38, 23, 35, 44, 38, 28, 29, 31, 38, 38, 31, 32, 32, 33, 24, 28, 29, 32],
                    [26, 26, 24, 30, 36, 36, 33, 26, 37, 24, 29, 40, 39, 37, 36, 26, 26, 25, 38, 25],
                    [27, 37, 33, 25, 28, 37, 39, 30, 31, 26, 34, 27, 37, 31, 28, 33, 36, 39, 27, 38],
                    [31, 31, 31, 26, 31, 28, 31, 35, 24, 25, 31, 24, 34, 30, 31, 35, 29, 30, 28, 30],
                    [28, 40, 24, 27, 30, 26, 34, 27, 28, 31, 41, 29, 35, 33, 35, 35, 31, 31, 30, 39],
                    [20, 34, 29, 27, 34, 25, 28, 43, 26, 26, 36, 31, 28, 36, 39, 26, 18, 24, 29, 26],
                    [26, 29, 34, 25, 26, 42, 30, 38, 30, 26, 37, 29, 43, 36, 36, 29, 27, 37, 33, 28],
                    [29, 30, 30, 27, 34, 34, 32, 34, 26, 28, 37, 28, 36, 38, 29, 35, 26, 32, 28, 37]],
            'mbuf_stats': {'cpu-socket-0': {'1024b': [73696, 73710],
                                            '128b': [98280, 98280],
                                            '2048b': [98266, 98280],
                                            '256b': [73710, 73710],
                                            '4096b': [1152, 1152],
                                            '512b': [73710, 73710],
                                            '64b': [191468, 196560],
                                            '9kb': [6912, 8960]},
                           'cpu-socket-1': {'1024b': [73696, 73710],
                                            '128b': [98280, 98280],
                                            '2048b': [98266, 98280],
                                            '256b': [73710, 73710],
                                            '4096b': [1152, 1152],
                                            '512b': [73710, 73710],
                                            '64b': [191429, 196560],
                                            '9kb': [6912, 8960]}}}}

6.21. Get Xstat names

  • Name - get_port_xstats_names

  • API Class - core

  • Valid States - all

  • Description - Get array of names of extended stats of port. List may vary per NIC type.

  • Parameters - port_id

  • Result [object] - See example below


'Request':

{
    "id": "m348pnzr",
    "jsonrpc": "2.0",
    "method": "get_port_xstats_names",
    "params": {
        "api_h": "f682qkNJ",
        "port_id": 0
    }
}

'Response':

{
    "id": "m348pnzr",
    "jsonrpc": "2.0",
    "result": {
        "xstats_names": [
            "rx_good_packets",
            "tx_good_packets",
            "rx_good_bytes",
            "tx_good_bytes",
            "rx_errors",
            "tx_errors",
            "rx_mbuf_allocation_errors",
            "rx_q0packets",
            "rx_q0bytes",
            "rx_q0errors",
            "rx_q1packets",
            "rx_q1bytes",
            "rx_q1errors",
            "tx_q0packets",
            "tx_q0bytes",
            "tx_q1packets",
            "tx_q1bytes",
            "tx_q2packets",
            "tx_q2bytes",
            "tx_q3packets",
            "tx_q3bytes",
            "rx_unicast_packets",
            "rx_multicast_packets",
            "rx_broadcast_packets",
            "rx_dropped",
            "rx_unknown_protocol_packets",
            "tx_unicast_packets",
            "tx_multicast_packets",
            "tx_broadcast_packets",
            "tx_dropped",
            "tx_link_down_dropped",
            "rx_crc_errors",
            "rx_illegal_byte_errors",
            "rx_error_bytes",
            "mac_local_errors",
            "mac_remote_errors",
            "rx_length_errors",
            "tx_xon_packets",
            "rx_xon_packets",
            "tx_xoff_packets",
            "rx_xoff_packets",
            "rx_size_64_packets",
            "rx_size_65_to_127_packets",
            "rx_size_128_to_255_packets",
            "rx_size_256_to_511_packets",
            "rx_size_512_to_1023_packets",
            "rx_size_1024_to_1522_packets",
            "rx_size_1523_to_max_packets",
            "rx_undersized_errors",
            "rx_oversize_errors",
            "rx_mac_short_dropped",
            "rx_fragmented_errors",
            "rx_jabber_errors",
            "tx_size_64_packets",
            "tx_size_65_to_127_packets",
            "tx_size_128_to_255_packets",
            "tx_size_256_to_511_packets",
            "tx_size_512_to_1023_packets",
            "tx_size_1024_to_1522_packets",
            "tx_size_1523_to_max_packets",
            "rx_flow_director_atr_match_packets",
            "rx_flow_director_sb_match_packets",
            "tx_low_power_idle_status",
            "rx_low_power_idle_status",
            "tx_low_power_idle_count",
            "rx_low_power_idle_count",
            "rx_priority0_xon_packets",
            "rx_priority1_xon_packets",
            "rx_priority2_xon_packets",
            "rx_priority3_xon_packets",
            "rx_priority4_xon_packets",
            "rx_priority5_xon_packets",
            "rx_priority6_xon_packets",
            "rx_priority7_xon_packets",
            "rx_priority0_xoff_packets",
            "rx_priority1_xoff_packets",
            "rx_priority2_xoff_packets",
            "rx_priority3_xoff_packets",
            "rx_priority4_xoff_packets",
            "rx_priority5_xoff_packets",
            "rx_priority6_xoff_packets",
            "rx_priority7_xoff_packets",
            "tx_priority0_xon_packets",
            "tx_priority1_xon_packets",
            "tx_priority2_xon_packets",
            "tx_priority3_xon_packets",
            "tx_priority4_xon_packets",
            "tx_priority5_xon_packets",
            "tx_priority6_xon_packets",
            "tx_priority7_xon_packets",
            "tx_priority0_xoff_packets",
            "tx_priority1_xoff_packets",
            "tx_priority2_xoff_packets",
            "tx_priority3_xoff_packets",
            "tx_priority4_xoff_packets",
            "tx_priority5_xoff_packets",
            "tx_priority6_xoff_packets",
            "tx_priority7_xoff_packets",
            "tx_priority0_xon_to_xoff_packets",
            "tx_priority1_xon_to_xoff_packets",
            "tx_priority2_xon_to_xoff_packets",
            "tx_priority3_xon_to_xoff_packets",
            "tx_priority4_xon_to_xoff_packets",
            "tx_priority5_xon_to_xoff_packets",
            "tx_priority6_xon_to_xoff_packets",
            "tx_priority7_xon_to_xoff_packets"
        ]
    }
}

6.22. Get Xstat values

  • Name - get_port_xstats_values

  • API Class - core

  • Valid States - all

  • Description - Get array of values of extended stats of port. Order of values matches the get_port_xstats_names.

  • Parameters - port_id

  • Result [object] - See example below


'Request':

{
    "id": "tfonjtfc",
    "jsonrpc": "2.0",
    "method": "get_port_xstats_values",
    "params": {
        "api_h": "f682qkNJ",
        "port_id": 7
    }
}


'Response':

{
    "id": "tfonjtfc",
    "jsonrpc": "2.0",
    "result": {
        "xstats_values": [
            0,
            0,
            ...
            0,
            0
        ]
    }
}

6.23. Setting port attributes

  • Name - set_port_attr

  • API Class - core

  • Valid States - all

  • Description - Sets port attributes

  • Parameters -

    • port_id [int] - port to apply attributes

    • attr [object] - dictionary with attributes, each of them is optional, see below

  • Result [object] - {}

Table 29. Object type attr
Field Subfield Type Description

link_status

up

bool

True = link up

promiscuous

enabled

bool

True = promiscuous enabled

led_status

on

bool

False = turn off LEDs

flow_ctrl_mode

mode

int

Flow control: 0 = none, 1 = tx, 2 = rx, 3 = full

Request example:


'Request':

{
    "id": "uuopmfln",
    "jsonrpc": "2.0",
    "method": "set_port_attr",
    "params": {
        "api_h": "f682qkNJ",
        "attr": {
            "flow_ctrl_mode": {
                "mode": 1
            },
            "promiscuous": {
                "enabled": true
            }
        },
        "handler": "vGp4EyA5",
        "port_id": 7
    }
}

7. Typical Transactions Examples

the following examples represents common scenarios. commands in […] represents meta commands and not real RPC commands such as repeat, wait and etc.

7.1. Init/Boot

This sequence represents a client implementing the protocol taking ownership over the server and preparing to perform work

7.1.1. Commands Flow

  • ping - Ping the server to verify the server is up

  • get_owner - if owner is not me or none prompt to the user if he wants to force it

  • acquire - Ask or force for exclusive control over the server. save the handler given for future commands

  • get_version - Verify the server is compatible with the GUI

  • get_system_info - Get the installed ports and cores

  • get_stream_list - for every port, get the list and sync the GUI

  • get_stream - for every stream in a port list, get the stream info and sync the GUI

7.2. Simple Traffic With Adding/Editing Streams

describes a simple scenario where a user wants to add or edit one or more streams to one or more ports

7.2.1. Commands Flow

  • [init] - perform the init procedure from above

  • [GUI add/edit streams] - GUI provides the user a way to add or edit streams and sync them

  • remove_all_streams [optional] - remove all previous streams to start from scratch

  • add_stream - configure a specific port with a stream.

  • [repeat previous] - repeat the above for how many ports and streams desired

  • get_stream_list [optional] - sanity - verify the server is synced with the GUI

  • start_traffic - start traffic on the specific port / all the ports

  • get_global_stats [optional] - make sure the machine is transmiting traffic

  • [perfrom test] - perform the required test

  • stop_traffic - when done, stop the traffic on the specific port / all the ports

  • get_global_stats [optional] - make sure the machine has stopped

7.3. Logout

Describes the log off from the machine

7.3.1. Commands Flow

  • stop_traffic [optional] - if traffic has started - stop it

  • get_global_stats [optional] - make sure the machine has stopped

  • remove_all_streams [optional] - if you want to clear all the previous streams - use this

  • release - release the ownership over the device

Appendix A: Interaction Examples

This appendix brings examples with data for the this RPC interaction.

add_stream method example

The following example represents an interaction between the RPC client and the server’s response.

Simple single packet client request

On the following example, there’s no VM instructions, rx_stats option is disabled and there’s only a single packet which isn’t connected to any other packet.

Client request

{
  "id" : "2bqgd2r4",
  "jsonrpc" : "2.0",
  "method" : "add_stream",
  "params" : {
     "api_h": "SPhoCDIV",
     "handler" : "37JncCHr",
     "port_id" : 1,
     "stream" : {
        "enabled" : true,
        "isg" : 0,
        "mode" : {
                "rate": {
                        "type": "pps",
                        "value": 100
                    },
           "type" : "continuous"
        },
        "next_stream_id" : -1,
        "packet" : {
           "binary" : [
              0,
              80,
              86,
              128,
              13,
              ...  # more packet data
              77,
              79,
              250,
              154,
              66
           ],
           "meta" : ""
        },
        "rx_stats" : {
           "enabled" : false
        },
        "self_start" : true,
        "vm" : []
     },
     "stream_id" : 0
  }
}

Server’s response

{
   "id" : "2bqgd2r4",
   "jsonrpc" : "2.0",
   "result" : {}
}

Two linked packets with VM instructions client request

On the following example, a batch request is being issued to the server, containing two add_stream requests.

First request
The first client request is similar to the previous example.
However, in this case the rx_stats object is enbaled and set to monitor ancestor’s stream_id (which is 0 in this case).

Ontop, this stream points to the next stream as the one to follow, as described under next_stream_id of stream object.

Second request
In this stream the big difference is that it has VM instructions under the vm field of the stream object.

Ontop, this stream is the last stream of the sequence, so next_stream_id of stream object is set to -1.

Client request

[
   {
      "id" : "tq49f6uj",
      "jsonrpc" : "2.0",
      "method" : "add_stream",
      "params" : {
         "api_h": "SPhoCDIV",
         "handler" : "2JjzhMai",
         "port_id" : 3,
         "stream" : {
            "enabled" : true,
            "isg" : 0,
            "mode" : {
                   "rate": {
                        "type": "pps",
                        "value": 100
                    },
               "type" : "continuous"
            },
            "next_stream_id" : 1,
            "packet" : {
               "binary" : [
                  0,
                  80,
                  86,
                  ...  # more packet data
                  250,
                  154,
                  66
               ],
               "meta" : ""
            },
            "rx_stats" : {
               "enabled" : true,
               "latency_enabled" : false,
               "seq_enabled" : false,
               "stream_id" : 0
            },
            "self_start" : true,
            "vm" : []
         },
         "stream_id" : 0
      }
   },
   {
      "id" : "2m7i5olx",
      "jsonrpc" : "2.0",
      "method" : "add_stream",
      "params" : {
         "api_h": "SPhoCDIV",
         "handler" : "2JjzhMai",
         "port_id" : 3,
         "stream" : {
            "enabled" : true,
            "isg" : 0,
            "mode" : {
                   "rate": {
                        "type": "pps",
                        "value": 100
                    },
               "type" : "continuous"
            },
            "next_stream_id" : -1,
            "packet" : {
               "binary" : [
                  0,
                  80,
                  86,
                  128,
                  ...  # more packet data
                  216,
                  148,
                  25
               ],
               "meta" : ""
            },
            "rx_stats" : {
               "enabled" : false
            },
            "self_start" : false,
            "vm" : [
               {
                  "init_value" : "65537",
                  "max_value" : "65551",
                  "min_value" : "65537",
                  "name" : "l3__src",
                  "op" : "inc",
                  "size" : 4,
                  "type" : "flow_var"
               },
               {
                  "add_value" : 1,
                  "is_big_endian" : false,
                  "name" : "l3__src",
                  "pkt_offset" : 34,
                  "type" : "write_flow_var"
               }
            ]
         },
         "stream_id" : 1
      }
   }
]

Server’s response

[
   {
      "id" : "tq49f6uj",
      "jsonrpc" : "2.0",
      "result" : {}
   },
   {
      "id" : "2m7i5olx",
      "jsonrpc" : "2.0",
      "result" : {}
   }
]

Another Example of tuple generator

 - name: udp_64B
  stream:
    self_start: True
    packet:
      binary: stl/udp_64B_no_crc.pcap  # pcap should not include CRC
    mode:
      type: continuous
      pps: 100
    rx_stats: []

    # program that define 1M flows with IP range 16.0.0.1-16.0.0.254
    # we will create a script that do that for you
    # this is the low level instructions
    vm: [
               {
                 "type" : "tuple_flow_var",   # name of the command

                 "name" : "tuple_gen",    # tuple_gen.ip tuple_gen.port can be used

                 "ip_min"   : 0x10000001,  # min ip 16.0.0.1
                 "ip_max"   : 0x100000fe,  # max ip 16.0.0.254

                 "port_min" : 1025,        # min port 1025
                 "port_max" : 65500,       # max port 65500

                 "limit_flows" : 1000000,  # number of flows
                 "flags"       : 0,        # 1 - for unlimited
               },

               {
                  "type" : "write_flow_var", # command name

                  "name" : "tuple_gen.ip",  # varible to write

                  "add_value" : 0,          # no need to add value

                  "is_big_endian" : true,   # write as big edian

                  "pkt_offset" : 26,        # write tuple_gen.ip into ipv4.src_ip
               },

               {
                  "type" : "fix_checksum_ipv4", # fix ipv4 header checksum

                  "pkt_offset" : 14,          # offset of ipv4 header

               },

               {
                  "type" : "write_flow_var", # command name

                  "name" : "tuple_gen.port",  # varible to write

                  "add_value" : 0,          # no need to add value

                  "is_big_endian" : true,   # write as big edian

                  "pkt_offset" : 34,        # write tuple_gen.port into udp.src_port
               }

        ]