Skip to main content
HashnodeAPI blog

Command Palette

Search for a command to run...

Building a Reverse Proxy from Scratch with Node

Updated
•14 min read•View as Markdown
Building a Reverse Proxy from Scratch with Node
S
Shreya

For a while, I've been working on backend systems where networking and infrastructure quietly sit underneath the application layer, things like rate limiting, caching, asynchronous processing, and background workers.

And while working with these systems, I kept encountering one component that I understood mostly at the conceptual level: the reverse proxy.

I knew what Nginx was used for. I knew that it could sit in front of multiple backend servers, distribute traffic, perform health checks, handle timeouts, and even terminate TLS.

But there was a gap between knowing what a reverse proxy does and actually understanding what happens when a request passes through one.

That's the problem with abstractions.

You can write something as simple as:

location / {
    proxy_pass http://backend;
}

I'd written proxy_pass in config files like this without really knowing what happened after that line executed cause the abstruction hides almost what happens behind the hook, So I gave myself a multi-day project: learn Nginx properly, then rebuild every single piece of it by hand in raw Node.js - no Express, no frameworks, just the http module and a lot of console.log.

This post is the story of that build : what I learned, what broke, and the load-test numbers that proved it actually worked.

Part 1: Nginx as the Baseline

Before writing a single line of Node, I wanted to understand what a "real" reverse proxy does, using the tool everyone reaches for by default.

The setup

Two dumb, disposable Express servers (Server A on :4000, Server B on :4001), and Nginx running in Docker, mounted with a custom config:

events {}

http {
    upstream backend_servers {
        server host.docker.internal:4000 weight=3 max_fails=3 fail_timeout=30s max_conns=10;
        server host.docker.internal:4001 weight=1 max_fails=3 fail_timeout=30s max_conns=5;
    }

    server {
        listen 3000;

        location / {
            proxy_pass http://backend_servers;

            proxy_connect_timeout 5s;
            proxy_send_timeout 10s;
            proxy_read_timeout 10s;

            proxy_next_upstream error timeout invalid_header http_500 http_502 http_503;
            proxy_next_upstream_tries 2;
            proxy_next_upstream_timeout 5s;
        }
    }
}

One config file, and Nginx gave me, for free:

  • Reverse proxying -> forward client requests to a backend

  • Weighted load balancing -> send more traffic to one server than another

  • Passive health checks -> stop routing to a backend after repeated failures, retry it later

  • Connection limits -> cap concurrent connections per backend

  • Timeouts -> fail fast instead of hanging on a slow backend

  • Automatic retries -> if one backend fails, silently try another before failing the client

I killed backends mid-test, watched traffic reroute automatically, watched a 504 fire when I made a route deliberately hang longer than proxy_read_timeout. It worked, and it worked well. But every single one of those behaviors was happening inside Nginx's C code, completely invisible to me. I could configure the what, but not see the how.

That itch wanting to see the how is what led to part 2.

Part 2: Rebuilding It By Hand in Raw Node

I deliberately avoided Express for this. Express's entire value proposition is hiding exactly the mechanics a proxy needs to expose -routing, header copying, stream piping. Using it here would've meant never touching http.request() or .pipe(), and walking away without understanding how a request actually gets forwarded at the protocol level.

The skeleton: forwarding a single request

const http = require('http');

const server = http.createServer((clientReq, clientRes) => {
  const options = {
    hostname: 'localhost',
    port: 4000,
    path: clientReq.url,
    method: clientReq.method,
    headers: clientReq.headers,
  };

  const proxyReq = http.request(options, (backendRes) => {
    clientRes.writeHead(backendRes.statusCode, backendRes.headers);
    backendRes.pipe(clientRes, { end: true });
  });

  clientReq.pipe(proxyReq, { end: true });
});

server.listen(5000);

Six steps, spelled out in actual code instead of a config directive: receive request -> inspect it -> build a new outgoing request -> send it upstream -> receive the backend's response -> return it to the client. This is literally what proxy_pass does Nginx just does it in compiled C, invisibly.

Round robin, by hand

Nginx's upstream block gave me round robin for free. Here, it's an array and a pointer:

const backends = [
  { hostname: 'localhost', port: 4000 },
  { hostname: 'localhost', port: 4001 },
];

let currentIndex = 0;

function getNextBackend() {
  const backend = backends[currentIndex];
  currentIndex = (currentIndex + 1) % backends.length;
  return backend;
}

The % backends.length is the entire trick , increment a pointer, wrap it back to zero once it runs past the end of the array. That one line is round robin.

Weighted routing

Same idea, but instead of rotating over the raw 2-item array, I built an expanded list where each backend appears as many times as its weight:

function buildWeightedList(backendList) {
  const expanded = [];
  for (const backend of backendList) {
    for (let i = 0; i < backend.weight; i++) {
      expanded.push(backend);
    }
  }
  return expanded;
}
// weight 3 / weight 1 → [A, A, A, B]

Round robin over that expanded array naturally produces the right ratio. Not as smooth as Nginx's real interleaving algorithm, but the ratio holds confirmed with a 12-request curl loop landing at roughly 9:3.

Timeout + automatic fallback

This was the trickiest piece to reason about, because it's a race between two async events:

proxyReq.setTimeout(3000, () => {
  proxyReq.destroy();

  if (!isRetry) {
    const fallbackTarget = getNextBackend();
    forwardRequest(clientReq, clientRes, fallbackTarget, startTime, true); // retry ONCE
  } else {
    clientRes.writeHead(504);
    clientRes.end('Both backends failed or timed out.');
  }
});

If a backend doesn't respond within 3 seconds, kill that connection and retry on the next backend but only once, guarded by an isRetry flag, so a doubly-failing request doesn't loop forever. This is the hand-rolled version of Nginx's proxy_next_upstream_tries 2.

Active health checks

Nginx (the free version) only does passive health checking, it finds out a backend is down when a real request happens to fail against it. I wanted active checking: independently pinging each backend on a timer, regardless of real traffic.

function checkHealth(backend) {
  const req = http.request({ ...backend, path: '/health', timeout: 2000 }, (res) => {
    backend.healthy = res.statusCode === 200;
  });
  req.on('timeout', () => { req.destroy(); backend.healthy = false; });
  req.on('error', () => { backend.healthy = false; });
  req.end();
}

setInterval(() => backends.forEach(checkHealth), 5000);

Then getNextBackend() filters to only healthy backends before picking one. The satisfying test: kill a backend, send zero requests, and watch the proxy discover the failure entirely on its own within 5 seconds.

Least connections

Round robin doesn't know or care if a backend is currently overwhelmed. Least connections does, it tracks live in-flight request counts and always routes to whoever's least busy:

function getNextBackend() {
  const healthyBackends = backends.filter(b => b.healthy);
  let chosen = healthyBackends[0];
  for (const backend of healthyBackends) {
    if (backend.activeConnections < chosen.activeConnections) chosen = backend;
  }
  return chosen;
}

The subtle part: activeConnections has to be incremented the instant a request starts, and decremented on every possible exit path : success, timeout, and error guarded by a finished flag so a single request can never double-decrement. Miss one of those paths and the counter silently drifts, and the algorithm quietly degrades over time.

Keep-alive / connection pooling

Every http.request() call was opening a brand-new TCP connection to the backend and throwing it away after one use paying a fresh handshake cost every single time, even to the same backend, repeatedly. Fix: a shared http.Agent.

const keepAliveAgent = new http.Agent({
  keepAlive: true,
  maxSockets: 10,
  keepAliveMsecs: 1000,
});
// passed into every outgoing http.request(options) as `agent: keepAliveAgent`

I confirmed it was actually working by logging keepAliveAgent.freeSockets : after the first request to a backend, the socket didn't disappear, it sat in the "free" pool, ready to be reused instead of torn down and reopened.

Forwarded headers

Since the backend only ever sees a connection from my proxy, not the real client, it has no idea who actually made the request, unless I tell it explicitly:

const forwardedHeaders = {
  ...clientReq.headers,
  'x-forwarded-for': clientReq.headers['x-forwarded-for']
    ? `${clientReq.headers['x-forwarded-for']}, ${clientIp}`
    : clientIp,
  'x-forwarded-proto': clientReq.socket.encrypted ? 'https' : 'http',
  'x-forwarded-host': clientReq.headers.host,
};

X-Forwarded-For is a chain, not a single value in real infrastructure a request often passes through several proxies before reaching a backend, and each one appends its view of the client's IP rather than overwriting it, preserving the full trail back to the original client.

Part 3: Wiring In My Own Tools

Once the core proxy was working, I plugged in two things from my existing work: ShieldLimit for rate limiting, and an in-memory cache for GET responses.

The high-level request flow became:

Request
   ↓
Rate limit check (ShieldLimit)  ← reject early, before doing any other work
   ↓
Cache check (in-memory Map)     ← serve instantly if we've seen this GET before
   ↓
Health-aware least-connections load balancer → backend

Rate limiting -> a real call out to my own hosted ShieldLimit API, checked first, before cache or routing, so abusive traffic gets rejected as cheaply as possible.

Caching -> for GET requests, buffer the backend's response as it streams to the client, then store a copy in an in-memory Map keyed by method:url, with a TTL. Repeat requests get served straight out of memory, and the backend is never touched.

if (isCacheable(clientReq)) {
  const cached = cache.get(cacheKey);
  if (cached && Date.now() < cached.expiresAt) {
    clientRes.writeHead(cached.statusCode, cached.headers);
    clientRes.end(cached.body);
    return; // backend never touched
  }
}

The overall architecture ended up looking like this:

Part 4: Does It Actually Hold Up? : Load Testing with Autocannon

Building something and claiming it works isn't the same as proving it under real concurrent load. "It works when I open localhost:5000 in my browser" isn't a meaningful way to generate sustained concurrent traffic : a browser only ever fires one request at a time.

So I used Autocannon, a Node.js-based HTTP benchmarking CLI, to fire sustained concurrent traffic and see what actually happened.

autocannon -c 200 -d 10 http://localhost:5000

  • -c 200 -> 200 concurrent connections

  • -d 10 -> run the test for 10 seconds

  • localhost:5000 -> my Node reverse proxy

I tested the proxy at this concurrency level under two conditions: caching disabled and caching enabled. The goal wasn't to prove that a small Node implementation beats Nginx, that wouldn't be a fair conclusion from a local test anyway. The goal was to observe how one architectural decision ,serving repeated GET requests from memory instead of going to the backend every time changes measurable system behavior. (ShieldLimit was disabled for both runs, so the numbers reflect the proxy's own routing/caching logic, not an external network dependency, more on why that matters below.)

Test 1: Caching Disabled

Figure 1 - 200 concurrent connections, caching disabled

Latency Value
2.5% 81 ms
50% (median) 93 ms
97.5% 163 ms
99% 185 ms
Avg 100.88 ms
Stdev 24.08 ms
Max 320 ms
Throughput Value
Avg Req/Sec 1,964.6
Total requests ~20k in 10.1s
Data read 5.09 MB

Every single request in this run was hitting the load-balancing logic and going all the way to a real backend. The average request took roughly 100.88ms, the proxy sustained around 1,965 req/sec, and 99% of requests completed within 185ms. This is the baseline every other number gets measured against.

Test 2: Caching Enabled

Figure 2 - 200 concurrent connections, caching enabled

Same test, same 200 connections, same 10 seconds but now repeat GET requests get served straight from the in-memory Map, never touching a backend.

Latency Value
2.5% 3 ms
50% (median) 30 ms
97.5% 80 ms
99% 99 ms
Avg 33.28 ms
Stdev 23.26 ms
Max 563 ms
Throughput Value
Avg Req/Sec 5,928.3
Total requests ~59k in 10s
Data read 15.4 MB

Side by side

Metric Cache OFF Cache ON
Avg latency 100.88 ms 33.28 ms
Median latency 93 ms 30 ms
p97.5 latency 163 ms 80 ms
p99 latency 185 ms 99 ms
Avg throughput 1,964.6 req/s 5,928.3 req/s
Requests / 10s ~20k ~59k
Data read 5.09 MB 15.4 MB

Turning caching on roughly tripled throughput (1,965 -> 5,928 req/sec) and cut median latency by two-thirds (93ms -> 30ms). Interestingly, the Max latency actually went up slightly under caching (320ms -> 563ms) , a reminder that even a fast path can have an occasional slow outlier (likely a request that landed on a cache miss right as the map was being written to, or a GC pause), which is exactly the kind of thing tail-latency numbers are meant to catch and averages hide.

The purpose here was never "beat Nginx." It was seeing, with real numbers, exactly how much one architectural decision, a cache sitting in front of the backend, actually moves the needle. That's the engineering value of this experiment.

So, is this a "benchmark"?

Being honest with myself: I'd call this a local load-test snapshot, not a production-grade benchmark. A real benchmark needs tightly controlled conditions, same machine, same backend servers, same endpoint, same payload, same duration, same concurrency, same Node/Nginx versions, multiple repeated runs, isolated resources. I didn't do that rigor here, and comparing this Node proxy against Nginx directly would need identical conditions across both to mean anything.

The right question isn't "did my proxy beat Nginx?" It's "under the same workload, how did one architectural choice change measurable behavior?" and even then, this only tells me about this specific local setup, not a general claim about production readiness.

What I Actually Learned

The biggest takeaway from this project wasn't the final requests-per-second number. It was finally understanding what actually sits underneath a reverse proxy.

Before this project, concepts like load balancing, health checks, connection pooling, keep-alive, forwarded headers, timeouts, retries, caching, rate limiting, and TLS termination were mostly things I'd seen in documentation or architecture diagrams. After implementing them myself, I could actually connect the pieces.

A request entering a reverse proxy isn't just request -> backend. There's a whole lifecycle happening around it:

And underneath that : TCP connections, sockets, timeouts, streams, retries, connection reuse. That's the part I actually wanted to understand, and now I do.

Final Thoughts

The goal was to stop treating the reverse proxy as a black box.

By implementing the core pieces myself, I finally got to see how the concepts I'd previously encountered separately fit together:

HTTP requests -> TCP connections -> proxying -> load balancing -> connection pooling -> health checks -> retries -> caching -> rate limiting -> failure handling.

Things that looked like simple configuration options started making much more sense once I had to implement them myself.

The most valuable part of the project wasn't the final proxy server. It was the debugging.

A backend timing out, a connection being reused, a forwarded header containing an unexpected value, a retry accidentally selecting the same backend, or a rate limiter blocking concurrent requests ,each failure forced me to understand another layer of what was actually happening.

And that's probably the biggest lesson I’m taking away from this project:

When an abstraction feels confusing, sometimes the best way to understand it is to temporarily remove the abstraction and build a smaller version yourself.

The Node.js implementation I built is obviously nowhere close to being a production replacement for Nginx , nor is it supposed to be.

It was a learning tool.

But after building it, testing it under concurrent load, and comparing its behaviour with Nginx, I no longer see a reverse proxy as just a proxy_pass directive.

I have a much clearer mental model of what is happening underneath it.

And that's exactly what I wanted from this project.

Next up: going deeper into networking, distributed systems, and the other infrastructure pieces that sit underneath modern backend systems.

Comments

Join the discussion

No comments yet. Be the first to comment.

More from this blog

API blog

3 posts