Non Drop Rate Benchmark API for ASTF

Comparing to STL, the ASTF version of the benchmarker is much simpler. Since the script receives a lower and upper bound by the user, there is no need to run a first iteration with the maximal possible traffic. Note that also, ASTF doesn’t have the ability to do this. Hence, the actual find_ndr function in ASTF is:

def find_ndr(self):
    self.perf_run_interval(high_bound=100, low_bound=0)

Using the MultiplierDomain class, we can convert multipliers to percents and vice versa and actually work with percents. A simple implementation of the main logic behind the binary search would be like this:

def perf_run_interval(self, high_bound, low_bound):

    current_run_results = ASTFNdrBenchResults(self.config)
    current_run_stats = self.results.stats
    current_run_stats['iteration'] = 0

    while current_run_stats['iteration'] <= self.config.max_iterations:
        current_run_stats['mult_p'] = float((high_bound + low_bound)) / 2.00
        current_run_stats['mult'] = self.mult_domain.convert_percent_to_mult(current_run_stats['mult_p'])
        current_run_stats.update(self.perf_run(current_run_stats['mult']))

        error_flag = current_run_stats['error_flag']
        q_full_percentage = current_run_stats['queue_full_percentage']
        valid_latency = current_run_stats['valid_latency']

        current_run_stats['mult_difference'] = abs(current_run_stats['mult_p'] - self.opt_run_stats['mult_p'])
        current_run_results.update(current_run_stats)

        if q_full_percentage <= self.config.q_full_resolution and valid_latency and not error_flag:
            if current_run_stats['mult_p'] > self.opt_run_stats['mult_p']:
                self.opt_run_stats.update(current_run_stats)
                if current_run_stats['mult_difference'] <= self.config.allowed_error:
                    break
                low_bound = current_run_stats['mult_p']
                current_run_stats['iteration'] += 1
                if self.max_iterations_reached(current_run_stats, high_bound, low_bound):
                    break
                else:
                    continue
            else:
                break
        else:
            if current_run_stats['mult_difference'] <= self.config.allowed_error:
                break
        high_bound = current_run_stats['mult_p']
        current_run_stats['iteration'] += 1
        if self.max_iterations_reached(current_run_stats, high_bound, low_bound):
            break

    self.opt_run_stats['iteration'] = current_run_stats['iteration']
    self.opt_run_stats['total_iterations'] = current_run_stats['total_iterations']
    self.opt_run_stats['rate_difference'] = 0
    return self.opt_run_stats

A version of the previous function that would allow the plugin pre and post iteration would include the following changes:

def perf_run_interval(self, high_bound, low_bound):

    while current_run_stats['iteration'] <= self.config.max_iterations:
        current_run_stats['mult_p'] = float((high_bound + low_bound)) / 2.00
        current_run_stats['mult'] = self.mult_domain.convert_percent_to_mult(current_run_stats['mult_p'])

        if plugin_enabled:
            self.plugin_pre_iteration(run_results=current_run_stats, **self.config.tunables)

        current_run_stats.update(self.perf_run(current_run_stats['mult']))

        if plugin_enabled:
            plugin_stop = self.plugin_post_iteration(run_results=current_run_stats, **self.config.tunables)

        current_run_results.update(current_run_stats)

        if plugin_stop:
            if self.config.verbose:
                 current_run_results.print_state("Plugin decided to stop after the iteration!", high_bound, low_bound)
            self.update_opt_stats(current_run_stats)
            break

MultiplierDomain

class trex.examples.astf.ndr_bench.MultiplierDomain(low_bound, high_bound)

A class that represents the multiplier domain received by the user.

Parameters

high_bound:

Higher bound of the interval in which the ndr point is found.

low_bound:

Lower bound of the interval in which the ndr point is found.

convert_percent_to_mult(percent)

Converts percent in the domain to an actual multiplier.

Parameters

percent: float

Percentage in between high and low bound. Low is 0% and High is 100%/

Returns

Multiplier of cps

convert_mult_to_percent(mult)

Converts a multiplier to percentage of the domain that was defined in init.

Parameters

mult: int

CPS multiplier.

Returns

float, percentage

ASTFNdrBenchConfig class

class trex.examples.astf.ndr_bench.ASTFNdrBenchConfig(high_mult, low_mult, title='Title', iteration_duration=20.0, q_full_resolution=2.0, allowed_error=1.0, max_iterations=10, latency_pps=0, max_latency=0, lat_tolerance=0, verbose=False, plugin_file=None, tunables={}, **kwargs)

Configuration parameters for the benchmark.

Parameters

high_mult: int

Higher bound of the interval in which the ndr point is found.

low_mult: int

Lower bound of the interval in which the ndr point is found.

title: string

Title of the benchmark.

iteration_duration: float

Duration of the iteration.

q_full_resolution: float

Percent of queue full allowed.

allowed_error: float

Percentage of allowed error.

max_iterations: int

Max number of iterations allowed.

latency_pps: int (uint32_t)

Rate of latency packets. Zero value means disable.

max_latency: int

Max value of latency allowed in msec.

lat_tolerance: int

Percentage of latency packets allowed above max latency. Default value is 0 %. (Max Latency will be compared against total max)

verbose: boolean

Verbose mode

plugin_file: string

Path to the plugin file.

tunables: dict

Tunables for the plugin file.

kwargs: dict

classmethod load_plugin(plugin_file)

Load dynamically a plugin module so that we can provide the user with pre and post iteration API.

Parameters

plugin_file: string

Path to the plugin file.

config_to_dict()

Create a dictionary of the configuration.

Returns

Dictionary of configurations

ASTFNdrBenchResults class

class trex.examples.astf.ndr_bench.ASTFNdrBenchResults(config=None, results={})

NDR Bench Results

Parameters

config: ASTFNdrBenchConfig

User configuration of parameters. Default value is none.

results: dict

A dictionary containing partial or full results from a run.

update(updated_dict)

Updates the elapsed time, and the stats of the class with the parameter

Parameters

updated_dict: dict

Dictionary that we use as an reference to update our stats.

convert_rate(rate_bps, packet=False)

Converts rate from bps or pps to a string.

Parameters

rate_bps: float

Value of of rate in bps or pps based on the packet parameter.

packet: boolean

If true then the rate is PPS, else bps.

Returns

Rate as a formatted string.

print_latency()

Prints the latency stats in case there are any.

print_run_stats()

Prints the TRex stats after a run (transmission).

print_iteration_data()

Prints data regarding the current iteration.

print_final()

Prints the final data regarding where the NDR is found.

to_json()

Output the results to a json.

human_readable_dict()

Return a human readable dictionary of the results.

ASTFNdrBench class

class trex.examples.astf.ndr_bench.ASTFNdrBench(astf_client, config)

ASTFNDRBench class object

Parameters

astf_client: ASTFClient

ASTF Client

config: ASTFNdrBenchConfig

Configurations and parameters for the benchmark

plugin_pre_iteration(run_results=None, **kwargs)

Plugin pre iteration wrapper in order to pass the plugin a deep copy of the run results, since the user might change the actual run results. Consult the Plugin API for more information.

plugin_post_iteration(run_results, **kwargs)

Plugin pre iteration wrapper in order to pass the plugin a deep copy of the run results, since the user might change the actual run results. Consult the Plugin API for more information.

calculate_max_latency_received(latency_data)

Calculates the max latency of a run. Call this only if latency tolerance is 0%.

Parameters

latency_data: dict

The latency field of get_stats() of the ASTFClient

Returns

The max latency in that run.

calculate_latency_percentage(latency_data)

Calculates the percentage of latency packets beyond the max latency parameter. The percentage is calculated independently for each port and the maximal percentage is returned. Call this only if latency tolerance is more than 0%.

Parameters

latency_data: dict

The latency field of get_stats() of the ASTFClient

Returns

A float represeting the percentage of latency packets above max latency.

is_valid_latency(latency_data)

Returns a boolean flag indicating if the latency of a run is valid. In case latency was not set then it returns True.

Parameters

latency_data: dict

The latency field of get_stats() of the ASTFClient

Returns

A boolean flag indiciating the latency was valid.

update_opt_stats(new_stats)

Updates the optimal stats if the new_stats are better.

Parameters

new_stats: dict

Statistics of some run.

perf_run(mult)

Transmits traffic through the ASTF client object in the class.

Parameters

mult: int

Multiply total CPS of profile by this value.

Returns

Dictionary with the results of the run.

perf_run_interval(high_bound, low_bound)

Searches for NDR in an given interval bounded by the two parameters. Based on the number of iterations which is supplied in the ASTFNdrBenchConfig object of the class will perform multiple transmitting runs until one of the stopping conditions is met.

Parameters

high_bound: float

In percents of MultiplierDomain

low_bound: float

In percents of MultiplierDomain

Returns

Dictionary of the optimal run stats found in the interval, based on the criterions defined in ASTFNdrBenchConfig,

find_ndr()

Wrapper function, runs the binary search between 0% and 100% of the MultiplierDomain