# AI Governance Cost Control
AI Gateway is part of [AI Governance](/beta-docs/ai-coder/ai-governance/), which is
included with a Premium license.
AI Governance Cost Control governs AI spend in two complementary ways:
* **Enforcement** stops a user's requests routed via AI Gateway once their
spend reaches their budget.
* **Reporting** shows what each user and group has approximately spent in the
current budget period.
AI Governance Cost Control requires:
* Coder v2.36 or later.
* AI Gateway [enabled and configured](/beta-docs/ai-coder/ai-gateway/setup/) with at least one provider.
AI Governance Cost Control reports approximate spend rather than billed cost.
These figures will not match your provider invoices exactly. For details, visit [How spend is calculated](#how-spend-is-calculated).
These terms appear throughout this page and in the Coder dashboard:
| Term | What it means | Where it is set |
| --------------------- | ------------------------------------------------------------------------------------- | --------------------------------------- |
| **Budget period** | The window spend accumulates in before it resets. Defaults to the UTC calendar month. | Deployment settings |
| **Budget policy** | The rule that selects a user's effective group. Defaults to the highest budget. | Deployment settings |
| **Group budget** | A spend limit granted to each member of a group. | **Groups** > \{group} > **Settings** |
| **User override** | A spend limit for one user that supersedes their group budget. | **Groups** > \{group} > **Members** tab |
| **Effective group** | The group that supplies a user's budget and has their spend associated with it. | Resolved automatically |
| **Approximate spend** | A user's approximate spend in the current budget period. | Calculated from usage |
## Deployment settings [#deployment-settings]
Two deployment-wide settings govern budget resolution and the reset window. Each
accepts a single value today.
| Setting | Flag | Environment variable | Supported values | Default |
| ------------- | -------------------- | ------------------------ | ---------------- | --------- |
| Budget policy | `--ai-budget-policy` | `CODER_AI_BUDGET_POLICY` | `highest` | `highest` |
| Budget period | `--ai-budget-period` | `CODER_AI_BUDGET_PERIOD` | `month` | `month` |
* **Budget policy** determines which budget wins when a user belongs to more
than one budgeted group. `highest` selects the largest.
* **Budget period** sets the reset window. `month` is the UTC calendar month, so
spend resets at 00:00 UTC on the first day of each month.
These settings are deployment-wide.
## Budget [#budget]
**Every user's spend is recorded against a group.** A user only has a budget
when one is set on a group they belong to, or when they are given a user
override.
When AI Governance Cost Control is first deployed, users have **unlimited
spend** until an administrator sets budgets. Their spend is recorded against
the `Everyone` group.
### Group budget [#group-budget]
Setting a group budget requires the Owner, User Admin, Organization Admin, or
Organization User Admin [role](/beta-docs/admin/users/groups-roles/).
A group budget applies to each member individually rather than to the group as a
whole. For example, if a group has ten members and a budget of $200 USD, each
member can spend $200 USD and the group has a total spend limit of $2,000 USD.
1. Go to **Groups** and select a group.
2. Select **Settings**.
3. Under **AI budget**, set **Monthly limit per member**.
4. Select **Save**.
Budget values behave as follows:
* An empty field means no budget is set. The field displays `no budget`.
* `$0 USD` blocks every request routed via AI Gateway from members whose
effective group is this group.
* The maximum is `$1,000,000 USD` per member per budget period.
Note: Members who belong to other groups with budgets are still governed by
whichever group the budget policy selects. See
[Effective group resolution](#effective-group-resolution).
### User override [#user-override]
Adding or removing an override requires the Owner or User Admin
[role](/beta-docs/admin/users/groups-roles/). Other administrator roles can view
overrides but cannot change them.
Override a group budget when one user needs a different limit from the rest of
their group.
1. Go to **Groups**, select a group, then open the **Members** tab.
2. Find the member to apply the override to, open their action menu, and select
**Manage AI budget**.
3. Enable **Override group budget**.
4. Set **Custom monthly budget**, then choose the group in
**Budget assigned to**.
5. Select **Update**.
Overrides behave as follows:
* A user can have only one override at a time.
* An override supersedes every group budget the user belongs to.
* Spend is always attributed to a group, so an override must name the assigned
group. This group can be different from the group selected in the **Members**
tab. The user must belong to the assigned group.
* The `$0 USD` to `$1,000,000 USD` range applies to overrides as well.
* Disabling **Override group budget** removes the override and returns the user
to the budget of their [effective group](#effective-group-resolution).
### Effective group resolution [#effective-group-resolution]
A user can belong to several groups that have budgets, and can also hold an
override. AI Gateway resolves a single effective group for each request, in this
order:
1. The user's [override](#user-override) takes precedence over every group
budget.
2. Otherwise, the [budget policy](#deployment-settings) selects one of the
groups the user belongs to. The default policy, `highest`, selects the group
with the largest budget.
3. If none of those groups has a budget, the user has *effectively* unlimited
spend and their spend is recorded against the `Everyone` group.
Groups with identical budgets are ranked by the organization membership the
user joined first, then by group ID. To see the effective group currently
assigned to a user, see [Monitor spend](#monitor-spend). The effective group is
deployment-wide, so it can be a group in a different organization.
Recorded spend is immutable. Changing which budget applies to a user affects
future requests only: spend that Coder has already recorded stays with the group
it was attributed to, so a user's history can span several groups. The effective
group is resolved at request time, so changing a group budget can change which
group applies to a user on future requests.
## How enforcement works [#how-enforcement-works]
AI Gateway checks each request before forwarding it upstream. The check compares
the user's spend in the current budget period with the budget that applies to the
request:
* Spend below the budget: the request proceeds.
* Spend at or above the budget: AI Gateway returns `403 Forbidden` with a
message that describes the issue.
Users without a budget have **unlimited spend**, so their requests proceed
without enforcement. A blocked user's access resumes when the budget period
resets, when an administrator raises the budget, or when an override is added.
Enforcement is approximate. A request's cost is known only after the provider
response reaches AI Gateway, so concurrent in-flight requests can carry a user
slightly past their budget. Subsequent requests are blocked after the recorded
spend reaches the budget.
### Notifications [#notifications]
The first time a user's spend crosses a threshold within a budget period, Coder
notifies the user and deployment-wide Owners and User Admins, excluding the
affected user:
| Threshold | User notification | Admin notification |
| --------- | ------------------------------ | ---------------------------------------- |
| 85% | You're approaching your budget | `` is approaching their budget |
| 100% | You've reached your budget | `` has reached their budget |
* A single expensive request can cross both thresholds at once.
* Budgets of `$0 USD` and unpriced usage cross no thresholds.
* Notifications are informational. Enforcement does not depend on them.
For delivery methods, see
[Notifications](/beta-docs/admin/monitoring/notifications/).
## How spend is calculated [#how-spend-is-calculated]
Coder multiplies the token usage of each request by the published price of the
model that served it. Prices come from a curated [models.dev](https://models.dev)
snapshot that ships with every Coder release, so no configuration is required.
Spend accumulates only from the moment v2.36 is deployed. Upgrading mid-month
therefore produces a partial first period.
To see the prices this deployment uses, list them. The `source` column reports
whether each is a default price or a custom price set on this deployment:
```sh
coder exp ai-model-prices list
```
A custom price takes precedence over its default, so it is the one listed. To
see the default for such a model, filter to it:
```sh
coder exp ai-model-prices list --source default
```
Default prices come from the price book that ships with each Coder release. To
see the price book for a release:
```text
https://github.com/coder/coder/blob/release//coderd/aibridge/prices/data/prices.json
```
Replace `` with a Coder minor version, for example `2.36`.
To use your own price for any of these models, see
[Configure model prices](#configure-model-prices).
Spend is an approximation. It can differ from what the provider bills, and
some usage does not count toward it at all:
* Default prices are list prices. A custom price brings spend closer to the
rates a deployment actually pays, though billing rules that are not a
per-token rate, such as committed-use discounts, cannot be represented.
* A model with no price adds nothing to spend. Its token usage is still
recorded, but it never counts toward a limit, so a user who calls only
unpriced models is effectively unlimited. Setting a price for the model
closes the gap.
Monitor `coder_ai_gateway_cost_control_unpriced_token_usage_records_total`,
labeled by `provider`, `provider_type`, and `model`, to detect unpriced usage.
Use the `(provider_type, model)` tuple to find the price to set. Any non-zero
value means spend is under-counted. Because the price book ships with the
release, a newly launched model is unpriced until you upgrade Coder or set a
price for it yourself.
### Configure model prices [#configure-model-prices]
Use the experimental `coder exp ai-model-prices` command to set model prices
for your deployment. It requires AI Governance, which is included with a
Premium license, and the Owner
[role](/beta-docs/admin/users/groups-roles/) or a custom role granting
`ai_model_price:update`. Run `coder exp ai-model-prices --help` for the full
reference.
List the prices this deployment holds, optionally narrowed to one provider or
model. The `source` column reports whether a price is a default price
(`default`) or one set on this deployment (`custom`):
```sh
coder exp ai-model-prices list --provider anthropic
```
List only the prices you have set:
```sh
coder exp ai-model-prices list --source custom
```
A listing reports one price per model, so an overridden model's default is not
shown. To see every price a model holds, both at once:
```sh
coder exp ai-model-prices list --source all
```
Price a single model. Prices are micro-units per million tokens, so `3000000`
is $3.00 per million tokens. Use `null` for a price you do not have, and `0` to
declare a model free:
```sh
coder exp ai-model-prices update --provider anthropic --model my-model \
--input-price 3000000 --output-price 15000000 \
--cache-read-price null --cache-write-price null
```
Price several models at once from a JSON document in the same shape as the
price book:
```sh
coder exp ai-model-prices update prices.json
```
* Prices are not retroactive. Usage recorded before you set a price stays
unpriced, so past spend does not change.
* A price you set takes precedence over the default and stays in effect
across upgrades, so it does not pick up later price books.
* A price is keyed by provider type and model, so every configured provider
of that type shares it.
* `openai-compat` providers cannot be priced. They pass through to any
upstream vendor, so a single price would be wrong for most of them.
* This command is experimental and can change without notice.
## Monitor spend [#monitor-spend]
Spend reporting is available in the Coder dashboard and as a CSV export.
Prometheus metrics report enforcement and pricing gaps.
### Dashboard [#dashboard]
Visibility follows the viewer's role:
| Who | Sees |
| ---------------------------------------------------- | -------------------------------------------------------------------- |
| Every user | Their own spend and budget, or unlimited state, in their avatar menu |
| Members of a group | The group's spend and budget, and their own member row |
| Owners, User Admins, and organization administrators | Spend and budgets for every group and every member |
* The **Groups** page compares each group's spend with the combined limits of
the members it covers.
* The **Members** tab of a group reports each member's spend, their budget, and
its source, labeled `Custom limit` for an override or `Group limit` for a
group budget. If the effective group is another group in the same organization,
the row shows `Budget managed by another group`. If the effective group is in a
different organization, the row shows a dash and explains that the group is not
visible there. Because the organization's `Everyone` group includes every
member, its **Members** tab is a quick way to look up any user's effective
group.
* The avatar menu reports the signed-in user's own spend for the budget period
as `$ / $ USD`, or `$ / Unlimited USD` when no budget
applies.
Administrators can also use the
[Get user AI spend](/beta-docs/reference/api/enterprise/#get-user-ai-spend) API
endpoint to see a user's current effective group.
### CSV Export [#csv-export]
Users who can read group-member data for the organization can export approximate
spend for reporting and internal cost allocation. The export is available through
the API only.
```sh
curl -H "Coder-Session-Token: $CODER_SESSION_TOKEN" \
"https://coder.example.com/api/v2/organizations//ai/spend/export"
```
* Without parameters, the export covers the current budget period.
* To select a range, pass `period_start` and `period_end` together as RFC 3339
timestamps. A range can span at most 31 days.
* Each row breaks spend down by user, group, model, and provider, with the
underlying token counts.
### Prometheus Metrics [#prometheus-metrics]
Prometheus metrics report blocked requests, users over budget, unpriced usage,
and enforcement latency. For the full metric list, including types and labels,
see [Prometheus metrics](/beta-docs/admin/integrations/prometheus/).
## Migrate from Coder Agents Cost Control [#migrate-from-coder-agents-cost-control]
In v2.36, AI Governance Cost Control replaces Coder Agents Cost Control. The
legacy Coder Agents Spend page remains available until v2.37.
Spend limits configured under **Admin settings** > **AI** > **Spend** are no
longer enforced by Coder Agents. To enforce spend, set an AI Governance budget.
To migrate existing limits:
1. Record the limits currently set under **Admin settings** > **AI** >
**Spend**, including the default limit and any group or user overrides.
2. Recreate group limits as [group budgets](#group-budget).
3. Recreate per-user limits as [user overrides](#user-override).
Expect the following differences:
* No deployment-wide default exists. Each group that needs a limit requires its
own budget.
* The UTC calendar month is the only period. Daily and weekly periods are not
currently supported.
* Users in several budgeted groups receive the highest budget. Coder Agents Cost
Control applied the lowest.
* Budgets cover priced AI Gateway traffic. Chat, IDE extensions, and CLI agents
draw on the same budget when their provider and model are priced. See
[How spend is calculated](#how-spend-is-calculated).
* Recorded spend does not carry over. Every user starts the first period at
$0 USD.
* Coder Agents users who exceed their budget see a usage limit error in chat.
The error details include the AI Governance budget limit.
## Next steps [#next-steps]
* [Monitoring](/beta-docs/ai-coder/ai-gateway/monitoring/)
* [Auditing AI sessions](/beta-docs/ai-coder/ai-gateway/audit/)
* [AI Gateway API reference](/beta-docs/reference/api/enterprise/)