Reading TCP retransmits on a slow uplink

My home connection is usually fine, but a couple of times a week uploads crawl. Before calling the ISP I wanted numbers. Linux already collects most of them, you just need to know where to look.

ss -ti

ss -ti prints internal TCP state for every connection. The interesting fields:

cubic wscale:7,7 rto:236 rtt:35.1/2.4 mss:1448 cwnd:10 ssthresh:7
bytes_sent:1849204 bytes_retrans:93312 retrans:0/66 lost:0

• rtt:35.1/2.4 — smoothed round-trip time and its variance, in ms.
• cwnd — congestion window in segments. If it stays small on a long transfer, the sender keeps hitting loss.
• retrans:0/66 — currently outstanding / total retransmitted segments.
• bytes_retrans / bytes_sent — the number I actually care about. Here it is ~5%, which is bad. On a healthy link it's well under 1%.

Reading this by eye gets old quickly, so I wrote retrans, which parses ss -tin and prints one line per connection with the retransmit ratio, sorted worst first. It can also run in a loop and print deltas.

tcpdump, when you need proof

If the ratio is high, a capture tells you whether loss is upstream or local:

tcpdump -i eth0 -w upload.pcap 'tcp port 443 and host 203.0.113.10'

Open it in Wireshark, filter on tcp.analysis.retransmission, and look at the timing. Loss that comes in bursts every few seconds usually means a queue somewhere is overflowing (bufferbloat). Random single losses look more like a bad line.

What it was

Bufferbloat on the router. Turning on SQM (cake with the upload limit set to ~90% of the real speed) brought the retransmit ratio down to 0.2% and made video calls stop stuttering while something is uploading. No call to the ISP needed.