Setup

By default, AI Gateway runs inside the Coder control plane (coderd) and requires no separate compute. In embedded mode, coderd runs the gateway in memory and brokers traffic to your configured AI providers on behalf of authenticated users.

If AI traffic needs dedicated compute, independent scaling, or a separate network endpoint, you can deploy AI Gateway as a standalone service.

Provider records are managed through the dashboard or API and stored in the database. Provider environment variables, flags, and YAML options no longer seed the database. Refer to Database management of providers for details.

Activation

The AI Gateway feature must be enabled in the Coder deployment configuration before embedded or standalone gateway instances can serve authenticated traffic.

AI Gateway is enabled by default as of v2.34.

A standalone process does not read CODER_AI_GATEWAY_ENABLED from its own environment. However, this setting must remain enabled on coderd. It is required for gateway key management endpoints to work and for standalone replicas to connect to the control plane.

Configure Providers

Configure at least one provider before exposing AI Gateway to end users.

Providers are deployment-scoped. Add them from the dashboard or the AI Providers API. Changes take effect without restarting coderd.

Dashboard

1. Navigate to Admin settings > AI
2. Select Providers
3. Select Add provider
4. Select the provider type
5. Enter a unique lowercase name, the upstream endpoint, and the credentials
6. Save the provider

Each provider gets its own AI Gateway route at /api/v2/ai-gateway/<provider-name>/.

Open an existing provider to rotate credentials, update its endpoint, or disable it without restarting coderd.

API Dumps

AI Gateway can dump provider request and response pairs to disk for debugging. Configure the dump directory with --ai-gateway-dump-dir or CODER_AI_GATEWAY_DUMP_DIR:

Or in YAML:

This top-level setting replaces the previous per-provider DUMP_DIR field. For each provider, AI Gateway writes dumps under <base>/<provider_name>, where <base> is the configured dump directory and <provider_name> is the provider instance name used in the route. For example, a provider named anthropic-corp with /var/lib/coder/ai-gateway-dumps configured writes to /var/lib/coder/ai-gateway-dumps/anthropic-corp.

Sensitive headers are redacted before dumps are written. Leave the value empty to disable dumping.

Each standalone gateway replica accepts the same API dump settings and writes optional dumps to its own local disk.

Send actor headers

Enable send_actor_headers to add authenticated actor identity to intercepted upstream requests. The setting is disabled by default.

You can also enable the setting with --ai-gateway-send-actor-headers or ai_gateway.send_actor_headers. Configure the ID, username, and email header names independently with CODER_AI_GATEWAY_ACTOR_HEADER_ID, CODER_AI_GATEWAY_ACTOR_HEADER_USERNAME, and CODER_AI_GATEWAY_ACTOR_HEADER_EMAIL:

The equivalent CLI options are --ai-gateway-actor-header-id, --ai-gateway-actor-header-username, and --ai-gateway-actor-header-email. The email header is empty by default. To forward email, set actor_header_email, for example to X-AI-Bridge-Actor-Metadata-Email. The setting applies to every configured provider, so enable it only if all of them, and any proxy in between, may receive user email addresses. For defaults, precedence, and privacy considerations, refer to Actor header forwarding.

Data Retention

AI Gateway records prompts, token usage, tool invocations, and model reasoning for auditing and monitoring purposes. By default, this data is retained for 60 days.

Configure retention using --ai-gateway-retention or CODER_AI_GATEWAY_RETENTION:

Or in YAML:

Set to 0 to retain data indefinitely.

For duration formats, how retention works, and best practices, see the Data Retention documentation.

Structured Logging

AI Gateway can emit structured logs for every interception record, making it straightforward to export data to external SIEM or observability platforms.

Enable with --ai-gateway-structured-logging or CODER_AI_GATEWAY_STRUCTURED_LOGGING:

Or in YAML:

These logs are written to the same output stream as all other coderd logs, using the format configured by --log-human (default, writes to stderr) or --log-json. For machine ingestion, set --log-json to a file path or /dev/stderr so that records are emitted as JSON.

Choose which process emits the records

By default coderd emits these records as they arrive from the gateway. Set --ai-gateway-structured-logging-source to change that:

Gateway-emitted records carry the same message, the same record_type values and the same fields, with one exception: they omit thread_parent_id and thread_root_id. Those are resolved from recorded tool usage by a database lookup that only coderd can perform.

When the gateway emits the records, they are written to the gateway's log output rather than coderd's, and under a different logger name. Match on the "interception log" message rather than the logger name so that your pipeline works with either source.

On a standalone gateway, set CODER_AI_GATEWAY_STRUCTURED_LOGGING and CODER_AI_GATEWAY_STRUCTURED_LOGGING_SOURCE on both processes, and ship the gateway's logs.

Disable content recording

Three of the record types store conversation content verbatim: aibridge_user_prompts.prompt, aibridge_tool_usages.input and aibridge_model_thoughts.content. Deployments that want AI Governance cost controls without retaining conversation content can stop recording them:

Or in YAML:

Interceptions and token usage are still recorded, so cost controls are unaffected.

This option only stops the records being written to the database. To keep exporting them to your SIEM, combine it with gateway-emitted structured logs:

Without --ai-gateway-structured-logging-source=gateway, the dropped records reach neither the database nor your SIEM, because coderd never receives them. The gateway logs a warning at startup in that configuration.

What you lose

Disabling content recording is a governance trade-off, not only a storage saving:

• Session grouping degrades for clients that don't send their own session ID. Coder groups those interceptions into threads by correlating recorded tool calls, so without tool usage records each interception becomes its own session. The grouping is stored when the row is written, so it can't be recovered later by re-enabling the option.
• Sessions pages are largely empty, since there is no prompt, tool call or reasoning content to show.
• Prompt and tool call telemetry report zero.
• The audit trail of what was asked is gone, including for incident review.

Token counts, spend, model and provider attribution, and request volume are all unaffected, as are the Prometheus metrics for prompt and tool call counts.

Filter for AI Gateway records in your logging pipeline by matching on the "interception log" message. Each log line includes a record_type field that indicates the kind of event captured: