# Setup By default, AI Gateway runs inside the Coder control plane (`coderd`) and requires no separate compute. In embedded mode, `coderd` runs the gateway in memory and brokers traffic to your configured AI providers on behalf of authenticated users. If AI traffic needs dedicated compute, independent scaling, or a separate network endpoint, you can [deploy AI Gateway as a standalone service](/beta-docs/@main/ai-coder/ai-gateway/standalone/). Provider records are managed through the dashboard or API and stored in the database. Provider environment variables, flags, and YAML options no longer seed the database. Refer to [Database management of providers](/beta-docs/@main/ai-coder/ai-gateway/providers/#database-management-of-providers) for details. ## Activation [#activation] The AI Gateway feature must be enabled in the Coder deployment configuration before embedded or standalone gateway instances can serve authenticated traffic. *AI Gateway is enabled by default as of v2.34.* ```sh export CODER_AI_GATEWAY_ENABLED=true coder server # or coder server --ai-gateway-enabled=true ``` A standalone process does not read `CODER_AI_GATEWAY_ENABLED` from its own environment. However, this setting must remain enabled on `coderd`. It is required for gateway key management endpoints to work and for standalone replicas to connect to the control plane. ## Configure Providers [#configure-providers] Configure at least one provider before exposing AI Gateway to end users. Providers are deployment-scoped. Add them from the dashboard or the [AI Providers API](/beta-docs/@main/reference/api/aiproviders/). Changes take effect without restarting `coderd`. ### Dashboard [#dashboard] 1. Navigate to **Admin settings** > **AI** 2. Select **Providers** 3. Select **Add provider** 4. Select the provider type 5. Enter a unique lowercase name, the upstream endpoint, and the credentials 6. Save the provider Each provider gets its own AI Gateway route at `/api/v2/ai-gateway//`. Provider names must be unique and use lowercase, hyphen-separated identifiers such as `anthropic-corp` or `azure-openai`. Once deleted, another provider may reuse the name. ![AI Providers list page](/beta-docs/images/aibridge/providers-list.png) ![Add Anthropic provider form](/beta-docs/images/aibridge/provider-add-anthropic.png) Open an existing provider to rotate credentials, update its endpoint, or disable it without restarting `coderd`. ![Edit Anthropic provider form](/beta-docs/images/aibridge/provider-edit-anthropic.png) ## API Dumps [#api-dumps] AI Gateway can dump provider request and response pairs to disk for debugging. Configure the dump directory with `--ai-gateway-dump-dir` or `CODER_AI_GATEWAY_DUMP_DIR`: ```sh coder server --ai-gateway-dump-dir=/var/lib/coder/ai-gateway-dumps ``` Or in YAML: ```yaml ai_gateway: api_dump_dir: /var/lib/coder/ai-gateway-dumps ``` This top-level setting replaces the previous per-provider `DUMP_DIR` field. For each provider, AI Gateway writes dumps under `/`, where `` is the configured dump directory and `` is the provider instance name used in the route. For example, a provider named `anthropic-corp` with `/var/lib/coder/ai-gateway-dumps` configured writes to `/var/lib/coder/ai-gateway-dumps/anthropic-corp`. Sensitive headers are redacted before dumps are written. Leave the value empty to disable dumping. Each [standalone gateway](/beta-docs/@main/ai-coder/ai-gateway/standalone/) replica accepts the same API dump settings and writes optional dumps to its own local disk. API dumps are intended for short diagnostic sessions only. Dump files contain raw request and response data, which may include proprietary or sensitive information such as prompts, completions, and tool inputs. Protect the target directory and disable dumping when diagnostics are complete. ## Send actor headers [#send-actor-headers] Enable `send_actor_headers` to add authenticated actor identity to intercepted upstream requests. The setting is disabled by default. ```sh CODER_AI_GATEWAY_SEND_ACTOR_HEADERS=true ``` You can also enable the setting with `--ai-gateway-send-actor-headers` or `ai_gateway.send_actor_headers`. Configure the ID, username, and email header names independently with `CODER_AI_GATEWAY_ACTOR_HEADER_ID`, `CODER_AI_GATEWAY_ACTOR_HEADER_USERNAME`, and `CODER_AI_GATEWAY_ACTOR_HEADER_EMAIL`: ```yaml ai_gateway: send_actor_headers: true actor_header_id: X-AI-Bridge-Actor-ID actor_header_username: X-AI-Bridge-Actor-Metadata-Username actor_header_email: "" ``` The equivalent CLI options are `--ai-gateway-actor-header-id`, `--ai-gateway-actor-header-username`, and `--ai-gateway-actor-header-email`. The email header is empty by default. To forward email, set `actor_header_email`, for example to `X-AI-Bridge-Actor-Metadata-Email`. The setting applies to every configured provider, so enable it only if all of them, and any proxy in between, may receive user email addresses. For defaults, precedence, and privacy considerations, refer to [Actor header forwarding](/beta-docs/@main/ai-coder/ai-gateway/reference/#actor-header-forwarding). ## Data Retention [#data-retention] AI Gateway records prompts, token usage, tool invocations, and model reasoning for auditing and monitoring purposes. By default, this data is retained for **60 days**. Configure retention using `--ai-gateway-retention` or `CODER_AI_GATEWAY_RETENTION`: ```sh coder server --ai-gateway-retention=90d ``` Or in YAML: ```yaml ai_gateway: retention: 90d ``` Set to `0` to retain data indefinitely. For duration formats, how retention works, and best practices, see the [Data Retention](/beta-docs/@main/admin/setup/data-retention/) documentation. ## Structured Logging [#structured-logging] AI Gateway can emit structured logs for every interception record, making it straightforward to export data to external SIEM or observability platforms. Enable with `--ai-gateway-structured-logging` or `CODER_AI_GATEWAY_STRUCTURED_LOGGING`: ```sh coder server --ai-gateway-structured-logging=true ``` Or in YAML: ```yaml ai_gateway: structured_logging: true ``` These logs are written to the same output stream as all other `coderd` logs, using the format configured by [`--log-human`](/beta-docs/@main/reference/cli/server/#--log-human) (default, writes to stderr) or [`--log-json`](/beta-docs/@main/reference/cli/server/#--log-json). For machine ingestion, set `--log-json` to a file path or `/dev/stderr` so that records are emitted as JSON. ### Choose which process emits the records [#choose-which-process-emits-the-records] By default `coderd` emits these records as they arrive from the gateway. Set `--ai-gateway-structured-logging-source` to change that: | Value | Emitted by | Use it when | | --------- | ---------------------- | --------------------------------------------------------------------------------------------- | | `coderd` | `coderd` (default) | You want today's behavior. | | `gateway` | The AI Gateway process | You need records that the gateway never persists, such as when content recording is disabled. | | `both` | Both processes | You are moving from one to the other and want to compare the two streams. | ```sh coder server --ai-gateway-structured-logging=true \ --ai-gateway-structured-logging-source=gateway ``` Gateway-emitted records carry the same message, the same `record_type` values and the same fields, with one exception: they omit `thread_parent_id` and `thread_root_id`. Those are resolved from recorded tool usage by a database lookup that only `coderd` can perform. With `both`, every record that reaches `coderd` is logged twice. Deduplicate on `interception_id`, `record_type` and `msg_id` if your pipeline counts records. When the gateway emits the records, they are written to the gateway's log output rather than `coderd`'s, and under a different logger name. Match on the `"interception log"` message rather than the logger name so that your pipeline works with either source. On a [standalone gateway](/beta-docs/@main/ai-coder/ai-gateway/standalone/), set `CODER_AI_GATEWAY_STRUCTURED_LOGGING` and `CODER_AI_GATEWAY_STRUCTURED_LOGGING_SOURCE` on **both** processes, and ship the gateway's logs. ## Disable content recording [#disable-content-recording] Three of the record types store conversation content verbatim: `aibridge_user_prompts.prompt`, `aibridge_tool_usages.input` and `aibridge_model_thoughts.content`. Deployments that want AI Governance cost controls without retaining conversation content can stop recording them: ```sh coder server --ai-gateway-disable-content-recording=true ``` Or in YAML: ```yaml ai_gateway: disable_content_recording: true ``` Interceptions and token usage are still recorded, so [cost controls](/beta-docs/@main/ai-coder/ai-gateway/cost-controls/) are unaffected. This option only stops the records being written to the database. To keep exporting them to your SIEM, combine it with gateway-emitted structured logs: ```sh coder server --ai-gateway-disable-content-recording=true \ --ai-gateway-structured-logging=true \ --ai-gateway-structured-logging-source=gateway ``` Without `--ai-gateway-structured-logging-source=gateway`, the dropped records reach neither the database nor your SIEM, because `coderd` never receives them. The gateway logs a warning at startup in that configuration. ### What you lose [#what-you-lose] Disabling content recording is a governance trade-off, not only a storage saving: * **Session grouping degrades** for clients that don't send their own session ID. Coder groups those interceptions into threads by correlating recorded tool calls, so without tool usage records each interception becomes its own session. The grouping is stored when the row is written, so it can't be recovered later by re-enabling the option. * **Sessions pages are largely empty**, since there is no prompt, tool call or reasoning content to show. * **Prompt and tool call telemetry report zero**. * **The audit trail of what was asked is gone**, including for incident review. Token counts, spend, model and provider attribution, and request volume are all unaffected, as are the [Prometheus metrics](/beta-docs/@main/ai-coder/ai-gateway/monitoring/) for prompt and tool call counts. Filter for AI Gateway records in your logging pipeline by matching on the `"interception log"` message. Each log line includes a `record_type` field that indicates the kind of event captured: | `record_type` | Description | Key fields | | -------------------- | --------------------------------------- | ------------------------------------------------------------------------------ | | `interception_start` | A new intercepted request begins. | `interception_id`, `initiator_id`, `provider`, `model`, `client`, `started_at` | | `interception_end` | An intercepted request completes. | `interception_id`, `ended_at` | | `token_usage` | Token consumption for a response. | `interception_id`, `input_tokens`, `output_tokens`, `created_at` | | `prompt_usage` | The last user prompt in a request. | `interception_id`, `prompt`, `created_at` | | `tool_usage` | A tool/function call made by the model. | `interception_id`, `tool`, `input`, `server_url`, `injected`, `created_at` | | `model_thought` | Model reasoning or thinking content. | `interception_id`, `content`, `created_at` |