Active measurement: latency, loss and path monitoring
Active measurement sends test packets and times what comes back. It answers “how long does it take, how much is lost, and where does it go” for any path you choose, at any time, without touching user traffic. The trade-off is that you measure your probes, not your users, so the probes have to be designed and scheduled with care.
Round-trip versus one-way
Round-trip time (RTT) is what ping reports: the time from sending a packet to getting the reply. It needs only one clock, so it is easy and reliable. Its weakness is that it hides direction. If the path to a server takes 30 ms and the return path takes 50 ms, ping shows 80 ms and cannot tell you which way is slow. Paths on the Internet are often asymmetric, and congestion often hits one direction only, such as the upload side of a home connection.
One-way delay measures each direction separately: the sender stamps the departure time, the receiver notes the arrival time, and the difference is the delay. That only works if the two clocks agree. RFC 7679 (the IETF one-way delay metric) separates three clock properties:
- Synchronization: how closely the two clocks agree. If the receiver’s clock is 5 ms ahead, every one-way result is 5 ms too high: the 30/50 ms paths above would read 35/45 ms.
- Accuracy: how close a clock is to UTC. It matters for knowing when something happened, not for the delay value.
- Resolution: the smallest step the clock can show. A clock that ticks every 10 ms cannot measure a 2 ms delay.
For one-way tests, run good NTP on both ends, or use GPS or PTP time sources where sub-millisecond precision matters.
What to measure
- Delay: the minimum shows the path’s propagation delay; the median and high percentiles show queueing.
- Delay variation (jitter): how much delay changes between packets. Voice and video suffer when it is high.
- Loss: the share of probes that never arrive. On long paths, even small amounts of loss can cut TCP throughput sharply.
- Path: traceroute-style tests over time show route changes, which often explain sudden jumps in delay.
- Throughput: tools such as iperf3 fill the path to see what it can carry. These tests use a lot of capacity, so schedule them sparingly.
OWAMP and TWAMP
The IETF standardized two protocols so that measurement devices from different vendors can test each other:
- OWAMP (RFC 4656, September 2006)
- The One-way Active Measurement Protocol, written by engineers at Internet2. It has two parts: OWAMP-Control sets up, starts and stops test sessions and fetches the results; OWAMP-Test sends the timestamped UDP test packets. It measures one-way delay and one-way loss.
- TWAMP (RFC 5357, October 2008)
- The Two-way Active Measurement Protocol reuses OWAMP’s design for round-trip tests. A Session-Sender sends packets and a Session-Reflector sends each one back with its own receive and send timestamps, so the time spent inside the reflector can be subtracted. Its appendix describes TWAMP Light, where the far end runs only a simple reflector.
perfSONAR
perfSONAR is open-source software for running and sharing end-to-end measurements between sites, developed by partners including ESnet, GÉANT, Indiana University, Internet2, RNP and the University of Michigan. The current documentation covers version 5.2.6. It bundles the tools (OWAMP and TWAMP for delay and loss, iperf3, iperf2 and nuttcp for throughput, traceroute, tracepath and ping) with:
- pScheduler, which runs tests on schedule and avoids two tests colliding on the same host;
- an archive (OpenSearch-based by default) to store results;
- pSConfig templates to define a mesh of regular tests;
- Grafana dashboards to see the results.
Bundles range from Tools (command-line clients only) and Testpoint (scheduled tests, results sent to a remote archive) up to the full Toolkit with a web interface. A one-off round-trip test looks like this:
pscheduler task rtt --dest www.perfsonar.net
pscheduler task throughput --source host2.example.net --dest host3.example.net
perfSONAR hosts publish themselves in a lookup service, so you can often find a node near the other end of a slow path to test against.
RIPE Atlas
RIPE Atlas, run by the RIPE NCC, is a global network of measurement devices: more than 14,000 probes, mostly hosted by volunteers on home and office networks, plus anchors, more powerful devices in well-connected locations that also act as targets. Anyone can view most of the results. To run your own measurements you spend credits, which you earn, for example, by hosting a probe. Available types are ping, traceroute, DNS, SSL/TLS and NTP; HTTP tests are limited to approved researchers.
Example: to check whether a website is reachable from a particular country, create a ping or traceroute measurement from 50 probes in that country. The traceroute results show where along the path things fail.
Long-term monitoring with SmokePing
A single test shows a moment; problems often show only at certain hours. SmokePing, written by Tobias Oetiker, sends a burst of probes to each target at regular intervals and stores latency, latency distribution and packet loss in RRDtool. Its graphs show the variation within each burst as “smoke” around the line, so a link that gets congested every evening is easy to spot.
The same idea works with any tool: run the same small test every few minutes from the same place, keep the results, and compare against the quiet-hours baseline.
Pitfalls
- Routers answer pings to themselves at low priority. High delay or loss at one middle hop that does not continue to later hops is usually not a real problem.
- A busy test host adds its own delay. Use a dedicated machine or a light virtual one with nothing else running.
- Firewalls may treat ICMP, UDP and TCP differently; test with the protocol your application uses.
- Throughput tests can congest the very link you are measuring and spoil latency tests running at the same time.
A note on history
NLANR ran an Active Measurement Project (AMP); by 2001 its measurement group had active and passive monitors at more than 100 locations on the research networks. Its final NSF award (2002–2006) planned to extend AMP into the campus domain. See the NLANR history page.
Sources
- RFC 7679: A One-Way Delay Metric for IP Performance Metrics, IETF
- RFC 4656: A One-way Active Measurement Protocol (OWAMP), IETF
- RFC 5357: A Two-Way Active Measurement Protocol (TWAMP), IETF
- What is perfSONAR?, perfSONAR documentation
- perfSONAR installation options, perfSONAR documentation
- Running tasks with pScheduler, perfSONAR documentation
- RIPE Atlas, RIPE NCC
- User-defined measurements, RIPE Atlas documentation
- SmokePing, OETIKER+PARTNER AG
- NSF grant extends support for interconnecting national research networks (NSF PR 01-01, January 2001), via Newswise
- Award 0129677: NLANR Measurement and Network Analysis group, National Science Foundation