Designing Scalable APIs

01What scalability means for an API

A scalable API keeps its response times when traffic grows from 100 to 10,000 requests per second by adding servers, not by being rewritten. The first condition is that no individual server remembers anything: sessions, temporary files and counters must live in Redis, the database or shared storage.

The bottleneck is usually the database, not the HTTP layer. So most decisions below serve one goal: reduce the database work done per request.

02Resources, versioning and a stable contract

Build URLs around nouns (resources), not verbs: GET /orders/42 beats /getOrder?id=42. The HTTP method carries meaning that caches and proxies rely on; GET must always be safe and cacheable.

• Put the version in the path (/v1/orders) and ship compatible changes, such as adding a field, without a new version.
• Never remove a field or change its meaning silently; mark it deprecated first.
• Keep responses lean and let clients ask only for what they need via fields or include.

03Pagination: why offset fails at scale

LIMIT 20 OFFSET 100000 must read 100,020 rows and throw 100,000 away, so deeper pages get slower. If a row is inserted or deleted while the user pages, they see a record twice or miss one. The fix is cursor (keyset) pagination:

-- first page
SELECT id, title, created_at
FROM posts
ORDER BY created_at DESC, id DESC
LIMIT 20;

-- next page: pass the last row of the previous page
SELECT id, title, created_at
FROM posts
WHERE (created_at, id) < ('2026-10-01 12:00:00', 9812)
ORDER BY created_at DESC, id DESC
LIMIT 20;

A composite index on (created_at, id) keeps this query in the millisecond range even for millions of rows. Hand the cursor to clients as an opaque string (for example base64) so your internals do not become a public contract.

04Idempotency and rate limiting

Networks fail and clients retry. If POST /payments runs twice, the customer is charged twice. The standard fix is an Idempotency-Key header: the client sends a unique ID per operation and the server stores the first result against that ID and replays it for retries.

Rate limiting protects the API from a misbehaving client and from abuse. A simple, effective approach is a windowed counter in Redis:

// fixed-window limiter: 100 requests per minute per key
async function limit(redis, key) {
  const slot = Math.floor(Date.now() / 60000);
  const k = "rl:" + key + ":" + slot;
  const n = await redis.incr(k);
  if (n === 1) await redis.expire(k, 60);
  return n <= 100;
}

When the limit is exceeded, return 429 Too Many Requests with a Retry-After header so clients back off correctly. Keep the counters in a shared Redis so every server sees the same number.

05HTTP caching and conditional responses

The cheapest request is the one that never reaches your application. Cache-Control and ETag tell browsers, CDNs and proxies how long a response stays valid. When a client sends If-None-Match and nothing changed, the server returns a bodyless 304 Not Modified.

GET /v1/products/42 HTTP/1.1
If-None-Match: "a7c3"

HTTP/1.1 304 Not Modified
ETag: "a7c3"
Cache-Control: public, max-age=60

Mark personalised data private so a CDN does not store it for others. The full header semantics are in the MDN HTTP caching guide.

06Move heavy work out of the request path

Sending email, building PDFs and processing images should not keep the user waiting. The right pattern: validate the request, enqueue the job and immediately return 202 Accepted with a status URL. The client polls later or is notified by a webhook.

It also absorbs traffic spikes: when a thousand requests suddenly arrive, the queue grows but the API stays fast. As the system grows, this boundary is often the first place to split services; see microservices architecture.

07Pre-launch checklist

• The service is stateless and sessions are stored outside the process.
• Every list is paginated and has a limit cap (say 100 maximum).
• Indexes were checked with EXPLAIN on the hottest queries.
• Writes are idempotent and rate limiting is on.
• Errors share one format and every request log carries a trace ID.
• Authentication and authorisation were reviewed against secure JWT authentication and the OWASP API Security list.

If you are planning the API layer of a new product, our web development team can help design and review it.