Linux TCP tuning: buffers, BBR and the fq qdisc

Updated November 2026

Linux autotunes TCP buffers per connection, but out of the box the ceilings are sized for ordinary servers, not for 10 Gbit/s transfers across a continent. Raising four sysctl values, and choosing a sensible queueing discipline, fixes most single-stream throughput problems on long paths.

Check what you have

sysctl net.core.rmem_max net.core.wmem_max
sysctl net.ipv4.tcp_rmem net.ipv4.tcp_wmem
sysctl net.ipv4.tcp_congestion_control net.ipv4.tcp_available_congestion_control
sysctl net.core.default_qdisc

According to the kernel documentation, current defaults are:

net.core.rmem_max and wmem_max: 4,194,304 bytes (4 MiB).
net.ipv4.tcp_rmem: minimum 4 KB, default 131,072 bytes, maximum between 131,072 bytes and 32 MB depending on RAM.
net.ipv4.tcp_wmem: minimum 4 KB, default 16 KB, maximum between 64 KB and 4 MB depending on RAM.

A 4 MB send-side ceiling caps one stream at about 4 × 1,048,576 × 8 ÷ 0.05 ≈ 670 Mbit/s on a 50 ms path. Older kernels and distributions may show different figures, which is why you check first.

What the four buffer settings do

net.ipv4.tcp_rmem and tcp_wmem: Three numbers each: minimum, default and maximum buffer in bytes. Autotuning grows each connection's buffer between the default and the maximum. The maximum is the value that matters for long paths.
net.core.rmem_max and wmem_max: The largest buffer an application may request explicitly with SO_RCVBUF or SO_SNDBUF, for example with iperf3 -w. The kernel doubles requested values for bookkeeping and caps them here. Autotuning works from the tcp_rmem and tcp_wmem maximums instead, but tools that set their own buffers are bound by these, so ESnet raises both together.

ESnet's recommended values

ESnet's fasterdata site publishes tested settings for data transfer hosts. They are sized for paths with parallel streams and fair sharing, not a single flow at all costs:

ESnet suggested sysctl values (fasterdata, March 2025)
Setting	10G host, up to 100 ms	10G up to 200 ms, or 40G up to 50 ms
`net.core.rmem_max`	67108864	134217728
`net.core.wmem_max`	67108864	134217728
`net.ipv4.tcp_rmem`	4096 87380 33554432	4096 87380 67108864
`net.ipv4.tcp_wmem`	4096 65536 33554432	4096 65536 67108864
`net.ipv4.tcp_mtu_probing`	1	1
`net.core.default_qdisc`	fq	fq

67,108,864 bytes is 64 MiB and 33,554,432 is 32 MiB. For 100 Gbit/s hosts ESnet goes much higher (maximums of 2147483647 and 1073741824) and adds further tuning described on its 100G page.

Make the settings permanent

Put the values in their own file under /etc/sysctl.d/. Files there must end in .conf and are applied in name order, so a high number makes yours win over vendor defaults:

sudo tee /etc/sysctl.d/90-tcp-tuning.conf >/dev/null <<'EOF'
# 10 Gbit/s host, paths up to about 100 ms (ESnet fasterdata values)
net.core.rmem_max = 67108864
net.core.wmem_max = 67108864
net.ipv4.tcp_rmem = 4096 87380 33554432
net.ipv4.tcp_wmem = 4096 65536 33554432
net.ipv4.tcp_mtu_probing = 1
net.core.default_qdisc = fq
EOF
sudo sysctl --system

sysctl --system reloads every configuration directory (and /etc/sysctl.conf last). To try a value without saving it, use sudo sysctl -w net.core.rmem_max=67108864; it is lost at reboot. ESnet's own pages add the lines to /etc/sysctl.conf, which also works.

Queueing discipline and pacing

The kernel documentation lists pfifo_fast as the built-in default qdisc, and ESnet notes that fq_codel became the default from kernel 4.12. For high-throughput TCP, ESnet recommends fq, which paces each flow and avoids line-rate bursts that overflow switch buffers. It reports 2 to 4 times better TCP performance on some 50 to 80 ms paths after enabling pacing.

ESnet also strongly recommends capping the per-flow rate below the NIC speed, for example on a 10 Gbit/s host:

sudo tc qdisc replace dev eth0 root fq maxrate 8gbit
tc qdisc show dev eth0

Replace eth0 with your interface. ESnet’s pages use tc qdisc add; replace does the same but also works when the interface already has a root qdisc. A tc setting does not survive a reboot, so add it to your network configuration once you are happy with it. After changing default_qdisc, reboot (or reapply the qdisc with tc) and confirm with tc qdisc show.

Congestion control: CUBIC or BBR

CUBIC is the default on most distributions, and ESnet's general advice is to leave it. ESnet no longer recommends the older htcp.

BBR has been in Linux since 4.9. To try it:

sudo modprobe tcp_bbr
sysctl net.ipv4.tcp_available_congestion_control
sudo sysctl -w net.ipv4.tcp_congestion_control=bbr

To keep it, add net.ipv4.tcp_congestion_control = bbr to your sysctl.d file. Since kernel 4.20, BBR no longer strictly needs the fq qdisc, though Google's notes say fq still performs better on busy servers. Before switching a production host, weigh ESnet's warnings:

BBR version 1, the one in mainline kernels, is known to compete unfairly with CUBIC and other algorithms on shared links.
BBRv3 was released in 2023 as separate kernel code; ESnet is still testing it.
BBR may reduce the impact of packet loss but does not remove it.

You can compare algorithms per test without changing the system default: iperf3 -c host -C bbr versus -C cubic.

What to leave alone

net.ipv4.tcp_mem: ESnet says the defaults are fine.
Timestamps and SACK: some guides turn off tcp_timestamps and tcp_sack to save CPU. ESnet advises keeping both at their default of 1 for WAN performance.
tcp_mtu_probing = 2: value 1 enables probing only after a black hole is detected; ESnet says 2 (always on) sometimes causes performance problems.
Autotuning: keep net.ipv4.tcp_moderate_rcvbuf at 1, its default, so receive buffers are still tuned per connection.

Verify the result

iperf3 -c far-host -t 30 -O 3
ss -ti dst far-host

Compare single-stream throughput before and after, and watch cwnd and rtt in the ss output. If throughput is unchanged and retransmissions are high, the path has loss; tuning cannot fix that. See Reading results.