Head-of-line blocking in a reverse tunnel

A reverse tunnel exposes a service on your laptop to the public internet under a real URL, like ngrok but self-hosted. A client holds a connection open to a public server, and the server forwards requests back down it. The engineering is in what that one connection has to carry.

Many streams, one pipe

A tunnel rarely carries one request at a time. A single page pulls HTML, a dozen images, scripts, maybe a long-poll held open for minutes, all independent streams sharing one encrypted connection. They have to be multiplexed: interleaved over one pipe, pulled apart cleanly on the far side. That means framing, a small header on every chunk saying which stream it belongs to:

// 12-byte header: magic, version, type, stream_id, payload_len
const MAGIC: [u8; 2] = [0x48, 0x4C];       // "HL"
const VERSION: u8 = 0x01;
const MAX_PAYLOAD_SIZE: u32 = 16_777_216;  // 16 MiB cap

enum FrameType { Control = 0x01, Data = 0x02, DataEnd = 0x03 }
struct Frame { frame_type: FrameType, stream_id: u32, payload: Bytes }

Data frames carry raw bytes; DataEnd closes a stream; control frames (handshake, auth, new-stream setup) use postcard. The length is checked against the 16 MiB cap before anything is allocated, so a hostile header can't make the server reserve something absurd.

Why hand-roll this instead of yamux or QUIC? To keep control messages in the same framed channel rather than a side protocol, and to own the transport's failure modes rather than inherit them. Which means signing up for the next problem.

One slow stream freezes the rest

This holds until someone downloads a big file. TLS rides on TCP, and TCP is a single ordered byte stream: bytes written first must arrive first. Queue a 200 MB download's frames and every other stream waits in line behind them, so a two-millisecond request for a CSS file now waits for the whole transfer to drain.

This is head-of-line blocking. It isn't a bug, it's what happens when independent streams share one ordered transport.

It's the same wall HTTP/2 hit multiplexing over one TCP connection, and the reason HTTP/3 moved to QUIC over UDP, where streams are genuinely independent.

The fix: a pool of connections

QUIC is the clean long-term answer. The pragmatic one, HTTP/1.1's for years, is to stop relying on a single pipe: open a pool of extra TLS connections (up to sixteen per client) and spread data frames across them, so one big transfer leaves the others free for small, latency-sensitive streams. Each new connection authenticates with a server-minted token (a PoolAuth message carrying a pool_token). The server keeps a sender per connection and round-robins, with the primary as slot zero:

fn data_sender(&self) -> Sender<Frame> {
    let pool_len = self.pool_senders.len();
    if pool_len == 0 { return self.frame_tx.clone(); }  // primary
    let idx = self.pool_next_idx.fetch_add(1, Ordering::Relaxed)
              as usize % (pool_len + 1);  // +1 for primary
    if idx == 0 {
        self.frame_tx.clone()
    } else {
        self.pool_senders.iter().nth(idx - 1)
            .map(|e| e.value().clone())
            .unwrap_or_else(|| self.frame_tx.clone())  // fallback
    }
}

The cursor is an AtomicU32 bumped with fetch_add, so choosing a connection is lock-free, and the pool lives in a DashMap so connections join and leave without a global lock. An empty pool or a missing slot falls back to the primary.

This doesn't remove head-of-line blocking, it dilutes it: a big transfer still blocks its own connection, just not the other fifteen. QUIC is the real cure, but a pool is a few lines, works over plain TCP and TLS everywhere, and kills the case where one download makes the whole tunnel feel dead.

The rest is security plumbing

Exposing a port to the internet safely is most of the actual work:

• Tunnel tokens hashed with Argon2id, so a leaked database isn't a leaked key.
• TLS certificate pinning compared in constant time, so the check can't be timed.
• Token-bucket rate limiting, per IP and per token.
• Automatic Let's Encrypt certificates over ACME, so a fresh instance gets HTTPS with no manual steps.

None of it novel, all of it mandatory the moment something is reachable from the open internet. There's also a direct peer-to-peer mode where the server only brokers a WebRTC connection between the two ends, but that's another post.

Takeaway

Multiplex independent streams over one ordered connection and you've signed up for head-of-line blocking, intended or not. Decide whether you can afford QUIC; if not, reach for the old trick, more connections. One stalled pipe matters far less when it's one of several.