TRex Frequently Asked Questions

TRex is fast realistic open source traffic generation tool, running on standard Intel processors, based on DPDK. It supports both stateful and stateless traffic generation modes.

Cisco systems, Intel, Imperva, Melanox, Vasona networks and much more.

“Stateful” mode is meant for testing networking gear which saves state per flow (5 tuple). Usually this is done by injecting pre-recorded capture files on pairs of interfaces of the device under test, and dynamically changing src/dst IP/port. With advance Stateful mode it done by injecting L7 data on top of TCP/UDP protocols

“Stateless” mode is meant to test networking gear that does not manage any state per flow (instead operating on a per packet basis). This is usually done by injecting customized packet streams to the device under test.

See this in-depth comparison for more details.

Yes. Currently there is a need for 2-3 cores and 4GB memory. For VM use case, memory requirement can be significantly reduced if needed (at the cost of supporting less concurrent flows) by using the following configuration

Limitations:

Yes. Since version 2.12, with these limitations. Especially note that a new firmware version is needed. VIC13xx/VIX12xx/VIX14xx

Yes. It is developed by a different team, and is also open source.
You can find GUI for stateless mode and packet editor here.

32 ports

We present statistics only for first four ports because there is no console space. Global statistics (like total TX) is correct, taking into account all ports. You can use the GUI/console or Python API, to see statistics for all ports.

One option for running few instances on the same physical machine is to install few VMs.

Second option is to use different config file for each instance (--cfg argument)
In each config file:

No. All ports in the configuration file should be of the same NIC type.
The workaround is to run several TRex instances, see section above.

The answer depends on your budget and needs. Bare metal will have lower latency and better performance. VM has the advantages you normally get when using VMs.

You have two options:
1. Send email to our support group: trex.tgen@gmail.com
2. Open an issue in our github page

x710da4fh with 4 10G ports can reach a maximum of 40MPPS (total for all ports) with 64 bytes packets. (can not reach the theoretical 60MPPS limit). This is still better than the Intel x520 (82559 based) which can reach ~30MPPS for two ports with one NIC.

Yes. Azure uses Mellanox CX-4 with failsafe and it supported see appendix. AWS is supported too but not with the same performance.

For CX-4, it is better to have MCX456A-ECAT(dual 100gb ports) and not the MCX455A-ECAT (single 100gb port). For CX-5, it is better to have the ConnectX-5 Ex EN version which run faster(Ex) with two ports. Part number is MCX516A-CDAT (PCIe4.0). MCX516A-CCAT and MCX515A-CCAT will be a bit slower. see ConnectX-5

XL710-da2 with 2 40G ports can reach maximum of 40MPPS/50Gb (total for all ports) and not 60MPPS with small packets (64B) Intel had in mind redundancy use case when they produced a two port NIC. Card was not intended to reach 80G line rate. see xl710_benchmark.html for more info.

There is good answer in the mailing list here.

You have several ways you can help:
1. Download the product, use it, and report issues (If no issues, we will be very happy to also hear success stories).
2. If you use the product and have improvement suggestions (for the product or documentation) we will be happy to hear.
3. If you fix a bug, or develop new feature, you are more than welcome to create pull request in GitHub.

See CONTRIBUTING for more details.

It is a continuous integration. The latest internal version is under 24/7 regression on few setups in our lab. Once we have enough content we release it to GitHub (Usually every few weeks). We do not send an email for every new release, as it could be too frequent for some people. We announce big feature releases on the mailing list. You can always check the GitHub of course.

You can.there is docker version

You have several options.
1. For playing around and experimenting, you can install TRex on VirtualBox/Docker by following this link.
2. To run the real product, check here for hardware recommendation and installation instructions.

Run TRex with the below command and check incoming packet count on DUT interfaces.

Alternatively, you can run TRex in stateless mode, send traffic from each port, and look at the counters on the DUT interfaces.

Compare the MACs address + name of interface, for example:

TRex uses DPDK which takes ownership of the ports, so using Wireshark is not possible. You can use switch with port mirroring to capture the traffic.

While running TRex on ESXi/KVM I get this error

You should first have a YAML configuration file. See here. Then, you can find some basic examples here.

A switch might be configured with spanning tree enabled. TRex reset the port at startup, making the switch reset it side as well, and spanning tree can drop the packets until it stabilizes. Disabling spanning tree can help. On Cisco nexus, you can do that using spanning-tree port type edge You can also start TRex with -k <num> flag. This will send packets for k seconds before starting the actual test, letting the spanning tree time to stabilize. This issue will be fixed when we consolidate “Stateful” and “Stateless” RPC.

Most common reason is problems with MAC addresses. If your ports are connected in loopback, follow this carefully.
If loopback worked for you, continue here.
If you set MAC addresses manually in your config file, check again that they are correct.
If you have ip and default_gw in your config file, you can debug the initial ARP resolution process by running TRex with -d 1 flag (will stop TRex 1 second after init phase, so you can scroll up and look at init stage log), and -v 1. This will dump the result of ARP resolution (“dst MAC:…”). You can also try -v 3. This will print more debug info, and also ARP packets TX/RX indication and statistics.
On the DUT side - If you configured static ARP, verify it is correct. If you depend on TRex gratuitous ARP messages, look at the DUT ARP table after TRex init phase and verify its correctness.

TRex performance depends on many factors:

BSD4.3, the code was heavily changed to support high scale of flows and performance

You can use the simulator. see simulator The output of the simulator can be loaded to Excel. The CPS can be tuned.

Default maximum supported flows is 1M (From TRex perspective. DUT might have much more due to slower aging). When active flows reach higher number, you will get “out of memory” error message

To increase the number of supported active flows, you should add “dp_flows” arg in config file “memory” section. Look here for more info.

You should increase the pool with that raise an error, for example in case of 2048

You can run TRex with -v 7 to verify that the configuration has an effect

After stretching TRex to its maximum CPS capacity, consider the following: DUT will have much more active flows in case of a UDP flow due to the nature of aging (DUT does not know when the flow ends while TRex knows). In order to artificially increase the length of the active flows in TRex, you can config larger IPG in the YAML file. This will cause each flow to last longer. Alternatively, you can increase IPG in your PCAP file as well.

The range of clients and servers should be at least the number of threads. The number of threads is equal to (number of port pairs) * (-c value)

Default latency packets are SCTP, you can omit the -l <num> from command line, or change it to ICMP. See the manual for more info.

You should first have a YAML configuration file. See here. Then, you can have a look at the stateless manual here. You can jump right into the tutorials section.

Python 3 support

We are using Python ZMQ wrapper. It needs to be compiled per platform and we have a support for many platforms but not all of them. You will need to build ZMQ for your platform if it is not part of the package.

Yes. Multiple TRex clients can connect to the same TRex server.

Yes. You can build any packet you like using Scapy. However, there is no way to create corrupted L1 fields (Like Ethernet FCS), since these are usually handled by the NIC hardware.

Major things that can reduce the performance are:

Adding “cache” directive can improve the performance. See here

and try this:

See example here

use random_seed per stream

No. each stream has its own, separate field engine program.

There is an Java SDK

here

TRex has minimal and basic support for HLTAPI. For simple use cases (without latency and per stream statistic) it will probably work. For advanced use cases, there is no replacement for native API that has full control and in most cases is simpler to use.

Yes. Using Field Engine you can build streams with different TOS and get statistic/latency/jitter per stream

See Bird

BIRD is an "Internet Routing Daemon", a non-interactive program running on background works as a dynamic router in an Internet type network. You can find out more at Bird’s Doc, or read the Bird Integration chapter where you can find out useful tutorials.

The bird/cfg directory contains some simple Bird configuration files. Some of the files include routing protocols (RIP, BGP, OSPF..) and others include IPv4 and IPv6 routes examples. You can create your own .conf file, more information can be found at Bird’s Configuration Doc.

There are 2 ways:

When creating a Bird node using client.set_bird_node(..) pass the MTU parameter with the wanted MTU value. Another option is using client.set_mtu(mac, mtu) method, which sets the MTU for an existing Bird node.

Yes, service mode has to be enabled. If not, at least the filtered mode with the right mask and --software should be enabled. Bird’s packets i.e BGP will be redirected to rx. You can find out more on Stateless service mode or at ASTF service mode.

Depends. Full service mode will forward all the packets to rx to be processed by client or capture. That will have a performance penalty since rx has to process more packets than usual, but you can be sure you are not loosing any packet during the process. Service filtered mode on the other hand preserves the no service performance and filters the wanted packets to rx using a bit mask. Notice that filtered service mode requires --software.

Yes, because in this mode all the packets are received and inspected with more flexible filters. You can still reach line rate of 40-100gbps with the right number of cores and the right configuration, this is due to the service filter mode which forwards only the right packets to the daemon.

Filter mode is introduced at version v2.71.