API gateways, BFFs and service meshes

HOW IT FITS TOGETHER

KEY TERMS

Gateway offload

The gateway runs the checks that would otherwise be copied into every service: TLS, token validation (signature, expiry, audience), rate limits and quotas per consumer, request size limits, CORS and request IDs. It passes identity downstream in trusted headers that it strips from inbound requests. Services must still authorise each action. The gateway answers who is calling, not whether they may touch this order.

BFF per experience

Sam Newman's rule is "one experience, one BFF". The mobile BFF returns small, pre-joined payloads that suit an old app version on a bad network. The web BFF can keep tokens server-side and set cookies. Because the BFF has one consumer, the team that owns that consumer can change or retire endpoints without versioning ceremony. For a web-only product, add a BFF only when there is real server-side aggregation to do.

Aggregation and partial failure

A chatty client makes 6 sequential calls across a 200 ms mobile RTT. A BFF makes those calls in parallel over a 1 ms datacenter network instead. Decide before you build it which upstreams are critical, so the request fails without them, and which are optional, so the response is returned with an empty value and a degraded flag. Clients must be able to render the degraded shape.

Timeout budgets and deadline propagation

Give each request one deadline at the edge, for example 3 s, and pass the remaining time downstream. The gateway timeout should be larger than the BFF's, and the BFF's larger than each upstream call's. If an inner timeout is longer than an outer one, the outer layer gives up while the inner work keeps running and holding resources. gRPC has no default deadline, so set one on every call. Java and Go propagate deadlines automatically as a remaining timeout.

Protocol translation at the edge

Outside, use REST or GraphQL over HTTPS, because browsers, CDNs and partners understand them. Inside, use gRPC with protobuf contracts, streaming and deadlines. The BFF or gateway translates between the two. gRPC status codes become HTTP statuses and RFC 9457 problem details, and internal field names stay internal. Transcoding at the gateway (gRPC-JSON transcoding in Envoy) works for simple 1:1 mappings. Anything that needs aggregation belongs in a BFF.

GraphQL as a BFF

A GraphQL server is a BFF whose response shape the client chooses, which helps when several screens need different slices of the same data. Resolvers call gRPC services, and DataLoader batches them per request. Federation splits the graph across teams, and a router plans queries across subgraphs. You still need persisted queries, depth and cost limits, and per-field auth.

Edge functions as BFFs

Workers and edge functions start fast and run near the user, which suits auth checks, A/B routing, header rewriting and caching. They are far from your database, though. A BFF that makes three sequential calls to us-east-1 from Sydney adds those long round trips again. Keep chatty aggregation close to the data, or use placement features that run the function near the origin.

IN YOUR STACK

_format_version: "3.0"
services:
  - name: orders-bff
    url: http://orders-bff.internal:8080
    connect_timeout: 2000   # ms; keep below the client's own timeout
    read_timeout: 5000
    routes:
      - name: orders-api
        paths: ["/api/orders"]
        protocols: [https]
plugins:
  - name: key-auth          # identifies the consumer for per-consumer limits
    service: orders-bff
  - name: rate-limiting
    service: orders-bff
    config:
      minute: 600
      hour: 20000
      limit_by: consumer
      policy: redis          # shared counters across gateway nodes
      redis:
        host: redis.internal
        port: 6379
      fault_tolerant: true   # if Redis is down, proxy instead of failing with 500

// wrangler.jsonc: "services": [{ "binding": "ORDERS", "service": "orders-bff" },
//                               { "binding": "CATALOG", "service": "catalog-api" }]
interface Env { ORDERS: Fetcher; CATALOG: Fetcher }
const routes: [prefix: string, binding: keyof Env][] = [["/api/orders", "ORDERS"], ["/api/catalog", "CATALOG"]];

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const { pathname } = new URL(request.url);
    const route = routes.find(([p]) => pathname === p || pathname.startsWith(p + "/"));
    if (!route) return new Response("Not found", { status: 404 });

    const user = await verifySession(request); // your JWT or session check (jose, KV, ...)
    if (!user) return new Response("Unauthorized", { status: 401 });

    const headers = new Headers(request.headers);
    headers.delete("cookie");               // services get identity, not the raw session
    headers.set("x-user-id", user.id);      // overwrites any client-supplied value
    headers.set("x-request-id", request.headers.get("cf-ray") ?? crypto.randomUUID());
    return env[route[1]].fetch(new Request(request, { headers }));
  },
} satisfies ExportedHandler<Env>;

async function getJson<T>(url: string, signal: AbortSignal): Promise<T> {
  const res = await fetch(url, { signal });
  if (!res.ok) throw new Error(`${url} -> ${res.status}`);
  return res.json() as Promise<T>;
}

// GET /bff/home: one round trip for the mobile home screen instead of three
export async function homeScreen(userId: string, clientGone: AbortSignal) {
  const id = encodeURIComponent(userId);
  // One budget for the whole response; also cancelled if the client disconnects
  const deadline = AbortSignal.any([clientGone, AbortSignal.timeout(800)]);
  const [profile, orders, recs] = await Promise.allSettled([
    getJson<Profile>(`http://users.internal/v1/users/${id}`, deadline),
    getJson<Order[]>(`http://orders.internal/v1/orders?user=${id}&limit=5`, deadline),
    // Optional data gets a tighter budget so it cannot hold the screen hostage
    getJson<Rec[]>(`http://recs.internal/v1/users/${id}/recs`, AbortSignal.any([deadline, AbortSignal.timeout(300)])),
  ]);
  if (profile.status === "rejected") throw profile.reason; // critical: fail the whole request
  return {
    profile: profile.value,
    recentOrders: orders.status === "fulfilled" ? orders.value : null, // null = "couldn't load"
    recommendations: recs.status === "fulfilled" ? recs.value : [],
    degraded: orders.status === "rejected" || recs.status === "rejected",
  };
}

// app/api/orders/[id]/route.ts: the browser calls its own origin; access tokens stay on the server
import { cookies } from "next/headers";
import type { NextRequest } from "next/server";

export async function GET(_req: NextRequest, ctx: RouteContext<"/api/orders/[id]">) {
  const { id } = await ctx.params;
  const session = await getSession((await cookies()).get("sid")?.value); // your session store
  if (!session) return Response.json({ error: "unauthenticated" }, { status: 401 });

  let upstream: Response;
  try {
    upstream = await fetch(`${process.env.ORDERS_API}/v2/orders/${encodeURIComponent(id)}`, {
      headers: { authorization: `Bearer ${session.accessToken}` },
      signal: AbortSignal.timeout(2_000),
    });
  } catch {
    return Response.json({ error: "orders unavailable" }, { status: 504 });
  }
  if (!upstream.ok) return Response.json({ error: "upstream" }, { status: upstream.status === 404 ? 404 : 502 });

  const order = await upstream.json();
  // Shaped for this UI only: internal fields never reach the browser
  return Response.json({ id: order.id, status: order.status, total: order.totals.grand, items: order.lines.length });
}

import { createClient, ConnectError, Code } from "@connectrpc/connect";
import { createGrpcTransport } from "@connectrpc/connect-node";
import { OrderService } from "./gen/orders/v1/orders_pb"; // generated by buf from .proto

const orders = createClient(OrderService, createGrpcTransport({ baseUrl: "http://orders.internal:8080" }));

// remainingMs: request deadline minus time spent; stockBySku: per-request DataLoader over a batch RPC
interface Ctx { userId: string; remainingMs(): number; loaders: { stockBySku: { load(sku: string): Promise<number> } } }

export const resolvers = {
  Query: {
    order: async (_: unknown, args: { id: string }, ctx: Ctx) => {
      try {
        const timeoutMs = Math.floor(ctx.remainingMs()); // integer ms; Connect treats <= 0 as "no timeout"
        if (timeoutMs <= 0) throw new ConnectError("request budget spent", Code.DeadlineExceeded);
        const res = await orders.getOrder({ id: args.id }, { timeoutMs, headers: { "x-user-id": ctx.userId } });
        return res.order ?? null;
      } catch (err) {
        if (err instanceof ConnectError && err.code === Code.NotFound) return null; // nullable field
        throw err; // surfaces in errors[], other fields still resolve
      }
    },
  },
  // Without batching, a 50-line order would make 50 inventory RPCs
  OrderLine: { stock: (line: { sku: string }, _: unknown, ctx: Ctx) => ctx.loaders.stockBySku.load(line.sku) },
};

WHEN TO REACH FOR WHAT

WHERE IT BITES

• Gateway as a god layer: business rules, response stitching and per-client branches pile up in gateway scripts that only one team can change and nobody tests. Keep the gateway declarative and move aggregation behind it.
• Nested timeouts set backwards. If the gateway times out at 30 s, the BFF at 60 s and the service has no deadline, abandoned work keeps running and piles up. Inner deadlines must be shorter than outer ones, and the remaining budget should be passed downstream.
• Retries stacked at every layer: client, gateway, BFF and mesh each retry 3 times, so one user click becomes 81 upstream calls during an incident. Retry at one layer, only for idempotent operations, with a retry budget.
• Duplicated domain logic across BFFs. If the web and mobile BFFs each compute order totals, they will disagree. BFFs shape and aggregate data; rules belong in a domain service.
• Trusting identity headers without stripping them at the edge, or letting services be reached without going through the gateway. Enforce it at the network level, for example with mesh mTLS or private networking.
• No correlation across hops. Without a request ID or traceparent created at the edge and passed through every layer, a slow BFF response cannot be traced to the upstream that caused it.

WHEN IT BREAKS IN PRODUCTION

• One request waits on a chain of services and times out
• A brief blip turns into a long outage
• Our microservices can only be released together
• You can't follow one request across services
• A third-party API starts returning 429
• Users far from the primary region get slow responses
• An endpoint gets slower as results grow
• A backend deploy breaks old app versions
• The frontend breaks when the API response changes
• CORS fails only in production or on preflight
• Changing an ID in the URL shows another user's data
• Users get logged out at random

SOURCES

• Sam Newman: Backends For Frontends (opens in new tab)Checked 2026-11-06
• Azure Architecture Center: Gateway Aggregation pattern (opens in new tab)Checked 2026-11-06
• Kong: Rate Limiting plugin configuration reference (opens in new tab)Checked 2026-11-06
• Cloudflare Workers: Service bindings (opens in new tab)Checked 2026-11-06
• gRPC: Deadlines (opens in new tab)Checked 2026-11-06
• Connect: Cancellation and timeouts (opens in new tab)Checked 2026-11-06