Reading iperf3 results and finding the bottleneck
An iperf3 number on its own only says "this is what you got". The columns around it, and how the number changes when you change one thing at a time, tell you why. This page shows how to read the output and which test isolates each common bottleneck.
The TCP output, column by column
A trimmed TCP run looks like this (example values):
Connecting to host 192.0.2.10, port 5201
[ 5] local 198.51.100.5 port 50422 connected to 192.0.2.10 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 98.2 MBytes 824 Mbits/sec 0 3.01 MBytes
[ 5] 1.00-2.00 sec 112 MBytes 940 Mbits/sec 0 3.01 MBytes
...
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.08 GBytes 931 Mbits/sec 0 sender
[ 5] 0.00-10.00 sec 1.08 GBytes 929 Mbits/sec receiver
- Interval, Transfer, Bitrate
- The time slice, the data moved in it and the average rate over it. Each side reports from its own point of view.
- Retr
- TCP segments the sender had to retransmit. Zero, or a handful at the start, is normal. A steady stream of retransmissions means packets are being lost somewhere.
- Cwnd
- The sender's congestion window: how much unacknowledged data TCP is allowing itself to have in flight. It grows while all is well and shrinks after loss.
- sender / receiver
- The final summary as seen by each end. The receiver line is what actually arrived; use it when you report a result.
A useful cross-check: throughput is roughly window ÷ round-trip time. If the window is 3 MBytes and the ping time is 25 ms, the ceiling is about 3 × 1,048,576 × 8 ÷ 0.025 ≈ 1,000 Mbit/s, which fits the run above.
Change one thing at a time
- Run 30 seconds in each direction:
iperf3 -c host -t 30, then add-R. - Run again with
-P 4and-P 8. - Run a UDP test at a rate just below the TCP result.
- Repeat from a host closer to the server, ideally on the same switch.
The pattern of results points to the cause, as the sections below explain.
Packet loss
Signs: Retr climbs in most intervals, Cwnd keeps collapsing and regrowing, throughput saws up and down. On long paths even small loss is expensive, because TCP needs many round trips to grow its window back.
Next steps: Find where the loss starts with a UDP test and with mtr (see Slow path diagnosis). Check interface error counters on hosts and switches. Faulty cables, dirty optics, overloaded firewalls and full buffers on a slower downstream link are common causes.
TCP window limit
Signs: No retransmissions, Cwnd stays flat at a fixed value, one stream is slow but -P 8 is much faster. The fault gets worse as round-trip time grows.
Cause: The sender or receiver buffer is smaller than the bandwidth-delay product. A 1 Gbit/s path with 50 ms RTT needs about 6.25 MB in flight. If the maximum buffer is 4 MB, one stream tops out near 4 × 1,048,576 × 8 ÷ 0.05 ≈ 670 Mbit/s however clean the path is. On Linux you can watch a live connection with ss -ti, which shows its cwnd, rtt and mss. See Window size and BDP and Linux tuning.
CPU limits
Signs: Throughput is the same at 10 ms and 100 ms RTT, no retransmissions, one core sits at 100% on sender or receiver. More streams help only on iperf3 3.16 or later, because older versions ran every stream in one thread; the iperf3 FAQ says a single core became a limit above roughly 25 Gbit/s.
Next steps: Add -V to see CPU use at the end of the test. Try -Z (zero copy) and -A to pin iperf3 to a core near the network card. Check that offloads such as TSO and GRO are enabled (ethtool -k eth0 on Linux). The iperf3 FAQ also describes the opposite case, where a faulty segmentation offload makes throughput drop towards zero and disabling it helps.
Wi-Fi
Signs: Results vary a lot between runs and with position in the room, upload and download differ, UDP shows bursts of loss and high jitter. The rate the device reports as its "link speed" is the radio's signalling rate; real TCP throughput is much lower.
Next steps: Repeat the test from a wired host. If the wired result is good, the bottleneck is the wireless link, not the WAN. Test near the access point, then from where users actually sit.
Duplex mismatch
Signs: One direction is reasonable and the other is very poor, or both are poor with many retransmissions, on a link that should run at 100 Mbit/s or 1 Gbit/s. It typically happens when one side is forced to a fixed speed and duplex and the other auto-negotiates.
Next steps: On Linux, ethtool eth0 prints the current speed, duplex and auto-negotiation state; compare it with the switch port. Look for late collisions or CRC errors on the switch. The fix is to set both ends the same way, normally auto-negotiation on both.
Other things that cap results
- A slower hop: a 1 Gbit/s server behind a 500 Mbit/s VPN or a 100 Mbit/s port will never test faster than that hop.
- Disk or application: iperf3 tests memory to memory. If iperf3 is fast but file copies are slow, the bottleneck is storage or the application. The
-Foption can read from or write to a file to test that. - Virtual machines and containers: virtual switches and CPU steal time often limit throughput well below the physical link.
- Policers: traffic shapers at a provider edge often show up as a hard ceiling with bursts of loss.
Save the evidence. Use -J --logfile to keep each run as JSON, and note the time, hosts, direction and options. It makes before-and-after comparisons and support tickets much easier.