Virtual desktop
Give Coder Agents a graphical virtual desktop inside workspaces for GUI-based tasks.
On this page
Note
This feature is experimental. Pin a release before broad rollout and review the release notes before upgrading.
Enable the experiment
coder server --experiments=chat-virtual-desktopOr set the environment variable:
CODER_EXPERIMENTS=chat-virtual-desktopWhat it does
Lets agents drive a graphical desktop inside the workspace through spawn_agent with type=computer_use (screenshots, mouse, keyboard).
Configuration
Virtual desktop requires the portabledesktop module installed in the workspace template.
Choose the computer use model
Computer use runs on a chat model from the chat's organization. Each organization picks its own model as an organization-scoped model override. To change it, you need permission to edit that organization's models.
- Go to Admin settings > AI > Coder Agents.
- If the Organization settings section shows an organization picker, select the organization.
- In the Computer use row, select a model.
- Select Save.
The Computer use row appears only while the chat-virtual-desktop experiment is enabled.
The list shows the organization's enabled models that computer use supports: models from Anthropic, OpenAI, and OpenAI-compatible AI providers, and Claude models from AWS Bedrock AI providers.
To add a model to the list, enable one of these models on the Models page.
When no model is selected, the row shows Use default model and computer use runs on the organization's default model. To return to the default model, select Clear, then select Save. If the selected model is later disabled, deleted, changed to a model that computer use doesn't support, or unusable for the chat owner, computer use falls back to the organization's default model.
Computer use is unavailable in a chat when the chat owner has no usable credentials for the AI provider of the model it runs on. It's also unavailable when it runs on the organization's default model and computer use doesn't support that model. OpenAI computer use requires the Responses API, so it fails on an OpenAI model whose Use Responses API override is set to Off.
An OpenAI-compatible AI provider has no computer use tool of its own, so Coder offers computer use to the model as an ordinary tool and returns a screenshot after every action. Coder can't verify an arbitrary OpenAI-compatible model, so such a model keeps the warning icon.
A warning icon next to the model selector means that Coder hasn't verified the model for computer use, or that the model's AI provider type is OpenAI and its Use Responses API override is set to Off. The icon reflects the selected model, or the organization's default model when no model is selected or the selected model no longer appears in the list. The warning doesn't block saving. Before you save, confirm that the model supports the provider's computer use tool. For an OpenAI-compatible AI provider, confirm instead that the model accepts images and calls tools.
To turn the Responses API back on for a model whose AI provider type is OpenAI:
- On the Models page, select the model.
- Expand Advanced.
- Set Use Responses API to Default or On.
- Select Update model.
You can also choose the computer use model through the API at PUT /api/v2/organizations/{organization}/chats/model-overrides/computer_use.
The request fails with 404 Not Found while the experiment is disabled, and with 400 Bad Request when computer use doesn't support the model.