Provider Configuration

Providers are deployment-scoped and managed from the dashboard or the AI Providers API. See Setup for the steps to add, edit, and disable a provider.

This page covers the provider types AI Gateway supports, the setup considerations for each, how a provider's lifecycle affects request handling, and how to monitor providers.

Database management of providers

After seeding, manage providers through the dashboard or API. A provider that has been edited or removed there is not recreated or overwritten from the environment on the next restart.

Seeding is a coderd operation. A standalone Gateway ignores the deprecated provider variables and fetches provider configuration from coderd.

Provider types

AI Gateway speaks two upstream API formats: the OpenAI format (Chat Completions and Responses) and the Anthropic format (Messages). Every provider type maps to one of these.

azure, google, openrouter, vercel, and openai-compat are supported only as OpenAI-compatible endpoints: AI Gateway sends them OpenAI-format requests, so each must expose an OpenAI-compatible API at its base URL. They have no provider-specific integration beyond that.

OpenAI

Set the base URL to the upstream endpoint and provide an API key. The default https://api.openai.com/v1/ targets the native OpenAI service; point it at any OpenAI-compatible endpoint (for example, a hosted proxy or LiteLLM deployment) when needed.

If you create an OpenAI key with minimal privileges, this is the minimum required set:

Anthropic

Set the base URL and provide an API key. The default https://api.anthropic.com/ targets Anthropic's public API; override it for Anthropic-compatible brokers.

Anthropic does not allow API keys to have restricted permissions at the time of writing (June 2026).

Amazon Bedrock

Bedrock providers serve Anthropic models hosted on AWS and authenticate with AWS credentials rather than a registered API key. Configure:

• A protocol, either InvokeModel or Mantle. It determines the endpoint format and which of the fields below apply. See InvokeModel and Mantle.
• An endpoint in the format the protocol requires: https://bedrock-runtime.<region>.amazonaws.com for InvokeModel or https://bedrock-mantle.<region>.api.aws/anthropic for Mantle. The AWS region is read from the endpoint host.
• The model and small fast model identifiers, for InvokeModel only. Mantle providers do not set them; the client chooses the model on each request and AI Gateway forwards it upstream.

InvokeModel

The legacy Bedrock runtime API. AI Gateway translates each request into Bedrock's InvokeModel format and sends it to https://bedrock-runtime.<region>.amazonaws.com. Still supported, but Mantle is recommended for new deployments.

Mantle

The newer Anthropic-compatible Bedrock endpoint, recommended by AWS for new deployments. AI Gateway serves Anthropic models through the native Messages API, forwarding the request body unchanged and only applying AWS SigV4 signing with the provider's base identity.

To route Claude Code through a Mantle provider, run it in mantle mode with client-side signing disabled so the gateway signs centrally:

AWS credentials

Do not attach API keys to a Bedrock provider.

AI Gateway resolves AWS credentials one of three ways:

• AWS SDK default credential chain (recommended). When no explicit credentials are configured, the AWS SDK resolves them automatically from the environment: IAM Roles (instance profiles, IRSA, ECS task roles), shared config files, environment variables, SSO, and more. Attaching an IAM Role to the compute running Coder follows AWS best practices for temporary credentials. The role must permit bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream for the configured models.
• Static credentials. Provide an access key and secret for an IAM user with the same Bedrock permissions.
• Assumed IAM role. Set a Role ARN to have the gateway assume that role before calling Bedrock, signing requests with the resulting temporary credentials. This works on top of either of the above base identities and supports cross-account Bedrock access. See Assuming an IAM role.

Obtaining static Bedrock credentials

When you cannot use the default credential chain, create a dedicated IAM user and generate a static access key:

1. Choose a region where you want to use Bedrock.

2. Generate API keys in the AWS Bedrock console (replace us-east-1 in the URL with your chosen region):

   • Choose an expiry period for the key.
   • Select Generate.
   • This creates an IAM user with strictly-scoped permissions for Bedrock access.

3. Create an access key for the IAM user:

   • After generating the API key, click "You can directly modify permissions for the IAM user associated".
   • In the IAM user page, navigate to the Security credentials tab.
   • Under Access keys, select Create access key.
   • Select "Application running outside AWS" as the use case.
   • Select Next.
   • Add a description like "Coder AI Gateway token".
   • Select Create access key.
   • Save both the access key ID and secret access key securely.

4. Enter the access key ID and secret access key when you add or edit the Bedrock provider from the dashboard or the AI Providers API, along with the region (or base URL) and model identifiers.

Assuming an IAM role

Set the optional Role ARN field to have the gateway assume an IAM role before calling Bedrock. The base identity (static credentials or the default credential chain) signs an STS AssumeRole call, and the temporary credentials it returns sign Bedrock requests. The field is optional: a provider with no Role ARN authenticates with its base identity directly, exactly as described above.

To use role assumption:

1. Create the IAM role in the target account and grant it bedrock:InvokeModel and bedrock:InvokeModelWithResponseStream for the configured models. The base identity does not need Bedrock permissions itself; the assumed role does.

2. Configure the role's trust policy to allow the gateway's base identity to assume it.

3. Enter the Role ARN when you add or edit the Bedrock provider. It must be a valid IAM role ARN, for example arn:aws:iam::123456789012:role/BedrockRole.

Each provider assumes a single role. To use several roles, configure one provider per role.

External ID

When a Bedrock provider assumes a role, the gateway generates a unique external ID for it and sends that value on every AssumeRole call. The external ID guards against the confused deputy problem on cross-account assumption. The gateway generates and owns the value, as AWS recommends: you cannot set or change it. It is not a secret, and it is shown on the provider's edit page once a Role ARN is configured.

To enforce it, add the external ID to the target role's trust policy as an sts:ExternalId condition:

GitHub Copilot

GitHub Copilot offers three plans: Individual, Business, and Enterprise, each with its own API endpoint. Add one copilot provider per plan your organization uses, setting the base URL accordingly:

Copilot providers authenticate with each user's request-time GitHub OAuth token, so do not attach API keys. For client-side setup (proxy, certificates, IDE configuration), see GitHub Copilot client configuration.

ChatGPT

ChatGPT subscriptions (Plus, Pro, Business) are supported through a provider of type openai with a specific name and base URL:

The name must be exactly chatgpt. It determines the route clients use to reach the provider: /api/v2/ai-gateway/chatgpt/v1. If no provider with this name exists, requests to that route fail with 404 route not supported.

Do not attach API keys. ChatGPT providers authenticate with each user's ChatGPT OAuth token through BYOK, so BYOK must remain enabled. For client-side setup, see the Codex CLI ChatGPT subscription configuration.

OpenAI-compatible providers

Azure-hosted OpenAI, Google, OpenRouter, Vercel, and any other OpenAI-compatible service are configured with the matching type (or the generic openai-compat), the provider's OpenAI-compatible base URL, and an API key.

Provider lifecycle

Every provider carries an explicit status, surfaced through the provider_info metric and the API:

Disabling a provider does not delete it, its credentials, or its historical interception data. Re-enabling restores it to service.

Monitoring and reloads

Provider configuration changes take effect automatically, without restarting coderd. AI Gateway records the timestamp of each reload attempt and each successful reload, exposed as Prometheus metrics:

• coder_ai_gateway_providers_last_reload_timestamp_seconds
• coder_ai_gateway_providers_last_reload_success_timestamp_seconds

If you run the external proxy, it exposes the same pair under the coder_ai_gateway_proxy_ prefix. Each standalone Gateway replica reloads providers independently.

A growing gap between the attempt and success timestamps means reloads are firing but failing to apply. Alert on that gap rather than on a single failure, which may resolve on the next change. See Monitoring for the full metric list and sample alert queries.

Key failover

You can configure multiple centralized API keys for a single provider instance so that AI Gateway automatically retries with the next key when one fails. This is transparent to end users, and clients see no difference in behavior or need any configuration changes.

Key failover is supported for OpenAI and Anthropic providers. Amazon Bedrock and GitHub Copilot do not support key failover.

Multiple keys can be added per provider through the AI Providers API. Each provider supports a maximum of 5 keys.

Failover behavior

Every request starts with the first key in the list. If a key is rate-limited (429 Too Many Requests) or fails authentication (401 Unauthorized), AI Gateway puts that key on a temporary cooldown and retries the request with the next available key. Keys recover automatically when the cooldown elapses, so failover stays transparent to end users. Any other response, including a 403 Forbidden, is returned to the caller unchanged.

If all keys in the pool are exhausted, AI Gateway returns:

• 429 Too Many Requests when at least one key is rate-limited, with a Retry-After header set to the shortest cooldown across all keys.
• 502 Bad Gateway when every key is in an authentication-failure cooldown. The keys still recover automatically once their cooldowns elapse, so no Retry-After is sent.

Bring Your Own Key

A provider's configured credentials are the centralized default. When Bring Your Own Key (BYOK) is enabled, a user's own credential takes precedence over the provider's for that user's requests, and AI Gateway falls back to the provider credentials when the user has none. See Authentication for the BYOK flow and how to enable or disable it.

Failure modes