What is the AI Gateway?

The AI Gateway is a single, secured endpoint that sits between your applications and the large language model (LLM) providers you use. Instead of calling each provider directly, your app sends every inference request to the Gateway, which routes it to the right model and applies your security and governance controls on the way through.

Why route through the Gateway

Calling providers directly means each app manages its own keys, and every request skips straight to the model with no inspection. The Gateway centralizes that traffic so you get one place to manage credentials, one place to enforce policy, and one audit trail across every model your teams use.

• ▸Multi-provider routing — connect several LLM providers behind one endpoint and switch or mix models without rewriting your application.
• ▸Centralized credentials — providers authenticate with keys you supply and store securely, so raw provider keys never live in your app code.
• ▸Consistent controls — the same protections apply to every call, no matter which provider or model handles it.

What happens on every call

Each request passes through the same pipeline before a response returns to your app:

1. 1.Authenticate the caller and confirm it is authorized to use the Gateway.
2. 2.Screen the input for prompt injection and other known attack patterns.
3. 3.Route the request to the selected provider and model.
4. 4.Verify the output returned by the model before it reaches your users.
5. 5.Log the call for auditing, analytics, and later review.

Controls applied in-line

• ▸Threat detection — inbound prompts are checked against a large library of known attack and abuse patterns; matching requests can be blocked before they reach a model.
• ▸Output verification — responses are inspected so unsafe or policy-violating content can be caught on the way back.
• ▸Rate limiting — per-caller limits protect against runaway usage and cost spikes.
• ▸Full audit trail — every request and the action taken on it is recorded for compliance and investigation.

Getting started

To send traffic through the Gateway, connect at least one provider with your own key, then point your application at the Gateway endpoint. See Connecting a provider and Bring your own key (BYOK) to configure credentials.