nthbouncer vs API gateways
When people confuse them
Both issue keys and enforce limits, but the jobs differ. They often work together.
API gateways (Kong, AWS API Gateway, Apigee, and similar) are the front door for APIs you publish. They route, transform, authenticate, and often rate-limit per client.
nthbouncer is a concurrency governor. It admits N in-flight requests across every caller against one shared limit. Put it in front of your own origin or any partner API with a hard shared limit.
Use a gateway to publish and shape APIs. Use nthbouncer when a shared origin must not be over-admitted. In many stacks the gateway stays out front and the pool sits closer to that origin.
| API gateway | nthbouncer | |
|---|---|---|
| Job | Front door for APIs you publish | Shared admission for a hard limit |
| Limit | Per-client or per-route quotas | One in-flight limit shared by every caller |
| Strengths | Routing, plugins, transforms, WAF | Queue, retry, locks, managed upstream auth |
| Typical place | Closest to the client | Closest to the origin |
| Swarm of 20 agent runs | Each run stays inside its own quota; the origin still sees 20 | All 20 share one limit |
nthbouncer does not replace a gateway. It has no routing, transforms, or plugin ecosystem, and it is not an LLM gateway.
Front your own origin
Point a pool at an https origin you run when many consumers share one limit: public APIs, internal services, partner traffic, batch vs interactive work. Proxy when you want the pool to call upstream. Use concurrency locks when your process must keep the origin client.
For a worked SaaS example, see Proxy NetSuite under a concurrency limit.
With a gateway
A common layout:
- Clients hit your gateway (auth, routing, WAF).
- Paths that must respect a shared origin limit go through an nthbouncer pool, or acquire a lock and call the origin.
- The origin sees at most N in flight.
Use gateway rate limits for caller fairness. Use a pool when callers must share one origin's hard concurrency limit.
Not a waiting room
Digital waiting rooms (Cloudflare Waiting Room, Queue-it, and similar) hold human
browsers on a branded page during ticket sales and product drops. A pool holds
machine callers: API clients, jobs, and agent runs. Callers hold an open wait, get a
429 when the queue is full, or a 504 when the wait budget runs out. There is no
wait page.
The two coexist happily. A site may run a waiting room for shoppers and a pool for the backend connectors calling a vendor API under one account limit.