A well-designed API gateway is the single entry point between your clients and your backend services. It handles authentication, rate limiting, and routing at the edge so each service does not have to. Good design comes down to four decisions: what the gateway owns, which patterns it applies, how it is secured, and how its configuration is managed over time. This guide walks through each, with the trade-offs a CTO or engineering lead needs to weigh before committing to an architecture.
TL;DR
- Use the gateway for cross-cutting infrastructure concerns (authentication, throttling, routing, logging). Keep business logic in the services.
- Start with offloading and routing. Add aggregation, Backend-for-Frontends (BFF), and circuit breakers only when a real client need justifies them.
- A gateway handles north-south traffic (clients to services). A service mesh handles east-west traffic (service to service). They complement each other.
- Treat the gateway as a security chokepoint: authenticate every request, block internal namespaces, and manage configuration as code.
- Prefer a managed gateway unless you can name a requirement it cannot meet.
Table of Contents
- What an API gateway does
- Core API gateway patterns
- What to keep out of the gateway
- API gateway vs service mesh
- Securing the gateway
- Managed or custom gateway
- A practical design checklist
- Our perspective
- Build the gateway with a team that knows mobile
- FAQ
- Sources
What an API gateway does
An API gateway acts as a reverse proxy and routes client requests to the appropriate services, giving clients one centralized entry point instead of a list of service addresses. That decoupling matters most for mobile apps. A phone on a weak connection should not need to know how many services sit behind your product, or call five of them to render one screen.
The microservices.io description of the pattern captures the two ways a gateway handles a request: it either proxies the call to the right service, or it fans out to several services and combines the results.
In practice, a gateway usually takes on these responsibilities:
- Routing: matching path, headers, or query parameters to the right upstream service.
- Authentication and authorization: validating tokens before a request reaches any backend.
- Rate limiting and throttling: protecting services from overload and abuse.
- Protocol translation: for example, exposing REST to clients while services speak gRPC internally.
- Observability: consistent logging, metrics, and tracing at the edge.
- Caching and response shaping: reducing load and payload size.
The point of centralizing these is that you write them once rather than in every service, which keeps behavior consistent and lowers the cost of change.
Core API gateway patterns
Most production gateways combine several patterns. The Zuplo guide to gateway patterns notes that they are not mutually exclusive, and suggests starting with offloading because almost every API benefits from it.
| Pattern | What it solves | When to add it |
|---|---|---|
| Gateway offloading | Moves authentication, TLS, logging, and throttling out of services | Almost always, from day one |
| Gateway routing | Maps one public address to many backends | As soon as you have more than one service |
| Gateway aggregation (composition) | Combines several backend calls into one client call | When screens need data from multiple services |
| Backend for Frontends (BFF) | Gives each client type its own tailored API | When mobile, web, and partners need different data shapes |
| Circuit breaker | Stops cascading failures when a service is unhealthy | When one slow dependency can take down others |
Offloading and routing
Offloading is the foundation. Authentication, TLS termination, request logging, and throttling all belong at the edge. Routing then lets you change service boundaries behind the gateway without forcing clients to update. For a mobile product, that second benefit is significant: you cannot force every user to install a new app version, so a stable public contract is worth protecting.
Aggregation
Aggregation lets a client make one call for several pieces of data, which reduces network chatter. On mobile, where every round trip costs battery and latency, this is often the most visible win. The trade-off is that composition logic can grow quickly. Keep it to assembling responses, not deciding business outcomes.
Backend for Frontends
A mobile app, a web dashboard, and a partner integration rarely want the same payload. A BFF gives each its own gateway layer or configuration, so the iOS and Android apps get compact responses while an admin portal gets richer ones. The cost is more surface area to maintain, so add a BFF when client needs have actually diverged, not in anticipation.
Resilience: circuit breakers, throttling, caching
Circuit breakers prevent one failing dependency from cascading through the system. Throttling and caching protect backend availability during traffic spikes, and limits can be set globally, per method, or per client. Default limits rarely match your real traffic. Set usage plans per client and per method, and revisit them after launch with real data.
What to keep out of the gateway
The most common design failure is putting too much into the gateway. Overloading it with business logic turns it into a performance bottleneck and a single point of failure. The guidance is to keep it lightweight, focused on infrastructure concerns, and leave domain validation and orchestration to the services.
A practical test: if a rule would change because the business changed (pricing logic, eligibility, approval workflows), it belongs in a service. If it would change because the infrastructure or security policy changed (token validation, request size limits, CORS), it belongs in the gateway.
The same separation of layers shows up outside engineering too. This explanation of API-led integration layers shows how splitting concerns into distinct layers keeps each one simple, which is the same principle that keeps a gateway thin.
API gateway vs service mesh
Teams often ask whether they need both. The short answer is that they solve different problems.
| API gateway | Service mesh | |
|---|---|---|
| Traffic direction | North-south (client to services) | East-west (service to service) |
| Typical location | Edge of the system | Alongside each service |
| Main jobs | Authentication, routing, rate limiting, aggregation | Mutual TLS, retries, traffic policy, service-level telemetry |
| Who it serves | External clients and partners | Internal services |
The engineering guide recommends separating the two directions and letting each layer own its own. It also advises adopting a mesh only once the number of services makes per-service networking code painful. For a team running a handful of services, a gateway alone is usually enough, and a mesh adds operational weight without matching benefit.
Securing the gateway
Because every external request passes through it, the gateway is where security is won or lost. The OWASP Secure API Gateway Blueprint aims to provide a framework, best practices, and reference implementations for designing and deploying secure gateways. The themes that matter most:
- Authenticate and authorize every request before it reaches a backend. Common mechanisms include JWT, OAuth 2.0, and mutual TLS.
- Enforce rate limits and request size limits to blunt injection attempts, abusive clients, and denial-of-service traffic.
- Block sensitive internal namespaces. Debug, admin, and management endpoints should never be reachable from the public side.
- Adopt a Zero Trust posture for internal traffic too, often using mTLS between services.
- Log and monitor so anomalies show up in real time, not in a post-incident review.
- Document every API host, authentication method, and usage policy. You cannot assess what you have not inventoried.
If you handle regulated data in fintech, healthcare, or insurance, your compliance team and a qualified security reviewer should confirm the final controls against the specific frameworks that apply to you. The blueprint is a design reference, not a compliance certificate.
Designing for AI-integrated apps
Adding LLM features changes the traffic your gateway sees: longer-running requests, token-based cost, and new abuse patterns such as prompt-driven scraping. OWASP's guidance suggests that legacy security patterns alone are not enough for these workloads, and points toward measures like centralized model registries and automated anomaly detection. In practice, that means per-client quotas on AI endpoints, usage logging tied to identity, and a single controlled path to any model provider. For a concrete picture of how a unified, normalized API can serve agent pipelines, see this example of a data API built for AI agents.
Managed or custom gateway
Before building anything, ask whether a managed service already covers your needs. Microsoft's architecture guidance says built-in cloud-native gateways are often sufficient and carry lower operational overhead, and that custom solutions make sense only when specific flexibility requirements cannot be met. AWS's own guidance describes its managed gateway service as built for creating, publishing, and managing REST, HTTP, and WebSocket APIs, with features such as Lambda authorizers.
Choose a managed gateway when:
- Your needs are routing, auth, throttling, and logging.
- Your team is small and should not own gateway uptime.
- You already run on a single cloud provider.
Consider a self-managed or custom gateway when:
- You need custom plugins or protocol handling the managed option does not support.
- You must run across several clouds or on-premises with one consistent policy layer.
- You have the platform engineering capacity to operate it.
Whichever you choose, automating the lifecycle is not optional. The same Microsoft guidance stresses GitOps or infrastructure as code to keep routing rules, security policies, and certificates synchronized across environments, which prevents the configuration drift that causes outages.
A practical design checklist
Use this before you ship a gateway into production:
- Inventory: list every service, client type, and consumer, including partners.
- Contract: define versioning rules so older app versions keep working.
- Auth model: decide who validates tokens (gateway) and who enforces permissions (services).
- Limits: set per-client and per-method throttles, plus request size caps.
- Failure behavior: define timeouts, retries, and circuit breaker thresholds per upstream.
- Observability: emit request IDs, latency, and error rates, and connect them to alerting.
- Configuration: store everything in version control and deploy through a pipeline.
- Exposure review: confirm no debug or admin routes are public.
Our perspective
We have built mobile products for more than a decade, and the gateway question almost always shows up at the same moment: a prototype or early release works, then traffic, new clients, or an AI feature exposes how loosely the backend was put together. Founders who built a first version on an AI app builder often arrive at this point. The screens work, but every client talks directly to services with inconsistent authentication and no throttling.
Our advice is consistent. Start small. Put authentication, routing, and rate limits at the edge, keep the gateway thin, and let the mobile experience drive aggregation decisions. A screen that needs three calls on a slow connection is a stronger reason to add composition than any architecture diagram. Resist building a custom gateway because it feels more serious. A managed one, configured as code and monitored properly, gets most teams further, faster.
We also treat design and engineering as one conversation. The shape of your API decides how fast a mobile screen loads, so the people designing the interface and the people designing the gateway should be in the same room.
— Brian
Build the gateway with a team that knows mobile
If you are planning or reworking a gateway for a mobile or web product, our API and backend development services cover design, implementation, and versioning, and our DevOps consulting helps put configuration-as-code and CI/CD around it. For larger organizations, see our enterprise solutions, or browse all services to talk through your architecture with us.
FAQ
What is the difference between an API and an API gateway?
An API is the interface a service exposes so other software can use it. An API gateway is a server placed in front of one or more APIs that routes requests and applies shared policies such as authentication and rate limiting. Clients call the gateway, and the gateway calls the services.
Do I need an API gateway for a small app?
Not always. With a single backend and one client, a gateway may add more moving parts than value. Once you have several services, multiple client types, or a need for consistent throttling and authentication, a gateway pays for itself quickly. A managed gateway keeps the operational cost low.
Can an API gateway replace a service mesh?
No, because they handle different traffic. The gateway governs north-south traffic from external clients, while a mesh governs east-west traffic between services. Many teams run a gateway alone and add a mesh only when the number of services makes it worthwhile.
How do I keep a gateway from becoming a bottleneck?
Keep business logic out of it, limit aggregation to assembling responses, and set sensible timeouts and circuit breakers for each upstream. Scale it horizontally, and monitor latency and error rates at the edge so you see problems before users do.
Sources
- API gateways, Azure Architecture Center
- Pattern: API Gateway / Backends for Frontends
- API Gateway Patterns: BFF, Composition, Offloading, and Gateway Routing
- 3.16 API gateways and service mesh, Software Engineering Guide
- OWASP Secure API Gateway Blueprint
- Amazon API Gateway, AWS Prescriptive Guidance