nthbouncer
Log inGet started

Welcome to nthbouncer

Introduction

nthbouncer is a concurrency governor for APIs with a hard shared limit. It sits in front of origins you operate or third-party APIs that enforce a hard concurrency or rate limit. Each pool admits up to N in-flight requests across every caller and never over-runs the limit you configure (or that adaptive mode discovers from upstream 429s).

You manage pools, keys, credentials, and billing in the dashboard at nthbouncer.com. Clients call your pool hostname on *.nthpool.cloud (or a custom domain), or take a concurrency lock and call the origin themselves.

nthbouncer is not an API gateway and not an LLM gateway. Proxy when you want nthbouncer to call upstream. Use locks when you front a public endpoint or keep the origin client in your process. For SaaS origins with account-level concurrency limits, see the knowledge base.

How it fits together

  1. Create a pool and point it at one or more allowed origins.
  2. Choose managed custody (nthbouncer holds and signs upstream credentials) or pass-through (your client sends upstream auth; nthbouncer relays it).
  3. Mint a pool key and call https://‹pool›.nthpool.cloud/... with that key.
  4. Tune concurrency, queue depth, RPM, retries, and optional response cache on the pool's Concurrency settings.
  5. When several pools share one origin concurrency limit, put them on a Capacity budget. Fixed caps each pool at its allocation. Auto keeps reserved pins local and moves float toward load.
  6. Use the pool's Metrics tab for admit/queue/reject proof. Billing remains the place for request allowance.

Next steps