124.8K
Skip to main content
HomeRun AI Coding Agents in CoderAI GovernanceAI GatewayReference

Reference

Premium

Note

AI Gateway is part of AI Governance, which is included with a Premium license.

Deployment topologies

AI Gateway can run inside coderd or as a standalone data-plane service. Both topologies run the same gateway request handling and keep coderd as the source of truth for Coder API key validation, provider configuration, and AI session records. They differ in how requests are routed to the gateway.

Embedded gateway

By default, coder server runs an in-memory gateway instance in the coderd process. AI clients send requests to <Coder access URL>/api/v2/ai-gateway/<provider-name>/. The embedded gateway uses the same control RPC as a standalone deployment, over an in-process transport rather than a network connection. It does not use a gateway key and does not negotiate an AI Gateway API version.

The following diagram shows the embedded topology:

AI Gateway implementation details

Standalone gateway

A standalone deployment runs the AI traffic data plane outside the coderd process. Each replica accepts client traffic, sends AI requests directly to upstream providers, and maintains a control connection to coderd by using a gateway key.

The control connection carries the following:

  • Coder API key validation, which resolves each request to an active Coder user.
  • AI budget checks, which reject requests from users over their spend limit.
  • Provider configuration, plus a change signal when the provider set changes.
  • AI session records.
  • Deprecated: the configuration and access tokens used by injected MCP.

Standalone replicas have no authoritative database state. They keep ephemeral provider snapshots, request caches, provider key pools, and metrics in memory, and emit their own logs and traces. When API dumps are enabled, each replica writes its own API dumps to its own local disk.

coderd remains required for standalone operation. A replica becomes unready when its control connection is unavailable, even if its HTTP listener remains healthy. AI Gateway Proxy remains part of coder server and can forward its intercepted traffic to either the embedded gateway or a standalone endpoint.

Version compatibility

The control connection between a standalone replica and coderd uses a versioned AI Gateway API. The current AI Gateway API version is defined in coderd/aibridged/proto/version.go.

coderd validates the AI Gateway API version that a standalone replica advertises before it accepts the control connection. AI Gateway API compatibility follows these rules:

  • The gateway and coderd AI Gateway API major versions must match.
  • The gateway's AI Gateway API minor version must be less than or equal to the coderd instance's AI Gateway API minor version.
  • coderd rejects a standalone gateway that advertises a newer AI Gateway API minor version.

A rejected replica receives an HTTP 400 response that reports the client_api_version and server_api_version values. Coder build versions are not the compatibility criterion.

For upgrade and rollback ordering, refer to Version compatibility in the standalone deployment guide.

Supported APIs

API support is divided into two categories:

  • Intercepted: Requests are intercepted, audited, and augmented.
  • Passthrough: Requests are proxied directly to the upstream provider without auditing or augmentation.

Where relevant, both streaming and non-streaming requests are supported. Paths are relative to the provider's base URL, such as https://ai-gateway.example.com/openai/v1 or https://ai-gateway.example.com/anthropic.

OpenAI

The OpenAI provider also serves the Azure OpenAI, Google, OpenRouter, Vercel, and OpenAI-compatible provider types.

Intercepted

Passthrough

The legacy /v1/completions API is deprecated and is not passed through.

Anthropic

The Anthropic provider also serves the AWS Bedrock provider type.

Intercepted

Passthrough

GitHub Copilot

Intercepted

  • /chat/completions
  • /responses
  • /v1/messages

Passthrough

  • /models(/*)
  • /agents/*
  • /mcp/*
  • /.well-known/*

Any route that is not listed above returns 404.

Troubleshooting

To report a bug, file a feature request, or review known issues, visit the Coder GitHub repository. For help with AI Gateway, visit the Coder Discord.