Additional clusters

Additional clusters

Additional clusters

With Coder, you can deploy workspaces in additional Kubernetes clusters using different authentication methods in the Terraform provider.

Region picker in "Create workspace" screen

Option 1) Kubernetes contexts and kubeconfig

First, create a kubeconfig file with multiple contexts.

$ kubectl config get-contexts

CURRENT   NAME                        CLUSTER
          workspaces-europe-west2-c   workspaces-europe-west2-c
*         workspaces-us-central1-a    workspaces-us-central1-a

Kubernetes control plane

If you deployed Coder on Kubernetes, you can attach a kubeconfig as a secret.

This assumes Coder is deployed on the coder namespace and your kubeconfig file is in ~/.kube/config.

kubectl create secret generic kubeconfig-secret -n coder--from-file=~/.kube/config

Modify your helm values to mount the secret:

  # ...
    - name: "kubeconfig-mount"
        secretName: "kubeconfig-secret"
    - name: "kubeconfig-mount"
      mountPath: "/mnt/secrets/kube"
      readOnly: true

Upgrade Coder with these new values.

VM control plane

If you deployed Coder on a VM, copy the kubeconfig file to /home/coder/.kube/config.

Create a Coder template

You can start from our example template. From there, add template parameters to allow developers to pick their desired cluster.

# main.tf

data "coder_parameter" "kube_context" {
  name         = "kube_context"
  display_name = "Cluster"
  default      = "workspaces-us-central1-a"
  mutable      = false
  option {
    name  = "US Central"
    icon  = "/emojis/1f33d.png"
    value = "workspaces-us-central1-a"
  option {
    name  = "Europe West"
    icon  = "/emojis/1f482.png"
    value = "workspaces-europe-west2-c"

provider "kubernetes" {
  config_path    = "~/.kube/config" # or /mnt/secrets/kube/config for Kubernetes
  config_context = data.coder_parameter.kube_context.value

Option 2) Kubernetes ServiceAccounts

Alternatively, you can authenticate with remote clusters with ServiceAccount tokens. Coder can store these secrets on your behalf with managed Terraform variables.

Alternatively, these could also be fetched from Kubernetes secrets or even Hashicorp Vault.

This guide assumes you have a coder-workspaces namespace on your remote cluster. Change the namespace accordingly.

Create a ServiceAccount

Run this command against your remote cluster to create a ServiceAccount, Role, RoleBinding, and token:

kubectl apply -n coder-workspaces -f - <<EOF
apiVersion: v1
kind: ServiceAccount
  name: coder
apiVersion: v1
kind: Secret
  name: coder-service-account-token
    kubernetes.io/service-account.name: coder
type: kubernetes.io/service-account-token
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
  name: coder
  - apiGroups: ["", "apps", "networking.k8s.io"] # "" indicates the core API group
    resources: ["persistentvolumeclaims", "pods", "deployments", "services", "secrets", "pods/exec","pods/log", "events", "networkpolicies", "serviceaccounts"]
    verbs: ["create", "get", "list", "watch", "update", "patch", "delete", "deletecollection"]
  - apiGroups: ["metrics.k8s.io", "storage.k8s.io"]
    resources: ["pods", "storageclasses"]
    verbs: ["get", "list", "watch"]
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
  name: coder
  - kind: ServiceAccount
    name: coder
  kind: Role
  name: coder
  apiGroup: rbac.authorization.k8s.io

The output should be similar to:

serviceaccount/coder created
secret/coder-service-account-token created
role.rbac.authorization.k8s.io/coder created
rolebinding.rbac.authorization.k8s.io/coder created

2. Modify the Kubernetes template

You can start from our example template.

variable "host" {
  description = "Cluster host address"
  sensitive   = true

variable "cluster_ca_certificate" {
  description = "Cluster CA certificate (base64 encoded)"
  sensitive   = true

variable "token" {
  description = "Cluster CA token (base64 encoded)"
  sensitive   = true

variable "namespace" {
  description = "Namespace"

provider "kubernetes" {
  host                   = var.host
  cluster_ca_certificate = base64decode(var.cluster_ca_certificate)
  token                  = base64decode(var.token)

Create Coder template with managed variables

Fetch the values from the secret and pass them to Coder. This should work on macOS and Linux.

To get the cluster address:

$ kubectl cluster-info
Kubernetes control plane is running at https://example.domain:6443

$ export CLUSTER_ADDRESS=https://example.domain:6443

To fetch the CA certificate and token:

export CLUSTER_CA_CERTIFICATE=$(kubectl get secrets coder-service-account-token -n coder-workspaces -o jsonpath="{.data.ca\.crt}")

export CLUSTER_SERVICEACCOUNT_TOKEN=$(kubectl get secrets coder-service-account-token -n coder-workspaces -o jsonpath="{.data.token}")

Create the template with these values:

coder templates create \
    --variable host=$CLUSTER_ADDRESS \
    --variable cluster_ca_certificate=$CLUSTER_CA_CERTIFICATE \
    --variable token=$CLUSTER_SERVICEACCOUNT_TOKEN \
    --variable namespace=coder-workspaces

If you're on a Windows machine (or if one of the commands fail), try grabbing the values manually:

# Get cluster API address
kubectl cluster-info

# Get cluster CA and token (base64 encoded)
kubectl get secrets coder-service-account-token -n coder-workspaces -o jsonpath="{.data}"

coder templates create \
    --variable host=API_ADDRESS \
    --variable cluster_ca_certificate=CLUSTER_CA_CERTIFICATE \
    --variable namespace=coder-workspaces
See an opportunity to improve our docs? Make an edit.