NAME¶

ib_atomic_bw, ib_atomic_lat, ib_read_bw, ib_read_lat, ib_send_bw, ib_send_lat, ib_write_bw, ib_write_lat - Collection of tests written over uverbs intended for use as a performance micro-benchmark

SYNOPSIS¶

ib_atomic_bw [<host>] [options]

ib_atomic_lat [<host>] [options]

ib_read_bw [<host>] [options]

ib_read_lat [<host>] [options]

ib_write_bw [<host>] [options]

ib_write_lat [<host>] [options]

DESCRIPTION¶

This is a collection of tests written over uverbs intended for use as a performance micro-benchmark. As an example, the tests can be used for HW or SW tuning and/or functional testing.

The collection conatains a set of BW and latency benchmark such as :

* Read - ib_read_bw and ib_read_lat.

* Write - ib_write_bw and ib_wriet_lat.

* Send - ib_send_bw and ib_send_lat.

* Atomic - ib_atomic_bw and ib_atomic_lat

* Raw Etherent (when working with MOFED2) - raw_ethernet_bw, raw_ethernet_lat

The benchmark used the CPU cycle counter to get time stamps without context switch. Some CPU architectures (e.g., Intel's 80486 or older PPC) do NOT have such capability.

The latency benchmarks measures round-trip time but reports half of that as one-way latency. This means that it may not be sufficiently accurate for asymmetrical configurations.

On Bw benchmarks, we calculate the BW on send side only, as he calculates the Bw after collecting completion from the receive side. In case we use the bidirectional flag , BW is calculated on both sides. in ib_send_bw, server side also calculate the received throughput.

Min/Median/Max result is reported in latency tests. The median (vs average) is less sensitive to extreme scores. Typically, the "Max" value is the first value measured.

Larger samples help marginally only. The default (1000) is pretty good. Note that an array of cycles_t (typically unsigned long) is allocated once to collect samples and again to store the difference between them. Really big sample sizes (e.g., 1 million) might expose other problems with the program. In this case you can use -N flag (No Peak) to instruct the test sample only 2 times (begining and end).

All throughput tests now have duration feature as well (-D <seconds to run>) to instruct the test to run for <seconds to run>. Another feature added is --run_infinitely, which instruct the test to run all te time and print throughput every 5 seconds.

The "-H" option (latency) will dump the histogram for additional statistical analysis. See xgraph, ygraph, r-base (http://www.r-project.org/), pspp, or other statistical math programs.

Architectures tested: i686, x86_64, ia64

OPTIONS¶

The SAME OPTIONS must be passed to both server and client.

If <host> is not presented, command starts a server and waits for connection. If it is, command connects to server at <host>.

Common Options:

Latenct tests options:

BW tests options:

SEND tests options:

Raw Ethernet BW test options:

ENVIRONMENT¶

Prerequisites:

NOTES¶

You need to be running a Subnet Manager on the switch or on one of the nodes in your fabric, in case you are in IB fabric.

BUGS¶

1. Multicast feauture in ib_send_lat and in ib_send_bw still have many problems! Will increase the support and bug fixes in this Q, but now the tests may stuck and could produce undefine behaviours.

2. Bidirectional feature in ib_send_bw test, when running in UD or UC mode. The algorithm we use for the bidirectional measurement is designed for RC connection type. When running in UC or UD connection types, there is a small probablity the test will be stuck.

3. RDMA_CM feature in read tests still doesn't work.

4. Dual-port support currently works only with ib_write_bw.

5. Compabilty issues may occur between different versions of perftest. Please make sure you work with the same version on both sides to ensure consistency of the test.

AUTHORS¶

Please post results/observations to the openib-general mailing list. See "Contact Us" at http://openib.org/mailman/listinfo/openib-general and http://www.openib.org.