Coder has open-sourced a new remote development platform 🥳 Check it out at coder/coder on GitHub.

Podman

This article will walk you through setting up Podman for use in Coder workspaces

Podman is a container engine (similar to Docker) that is compatible with the OCI containers specification. Podman is useful if you'd like an alternative to CVM workspaces or if your Linux kernel doesn't support CVMs.

Please be aware that there are limitations related to running Podman in rootless mode.

  1. Install smarter-device-manager and expose the FUSE device through it. To do so, create a file called smarter-device-manager.yaml with the following contents:

    apiVersion: v1
    kind: Namespace
    metadata:
      name: smarter-device-manager
      labels:
        name: smarter-device-manager
    
    ---
    apiVersion: v1
    kind: ResourceQuota
    metadata:
      name: smarter-device-manager
      namespace: smarter-device-manager
    spec:
      hard:
        pods: 50
      scopeSelector:
        matchExpressions:
          - operator: In
            scopeName: PriorityClass
            values:
              - system-node-critical
              - system-cluster-critical
    
    ---
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: smarter-device-manager
      namespace: smarter-device-manager
    data:
      conf.yaml: |+
        - devicematch: ^fuse$
          nummaxdevices: 50
    
    ---
    apiVersion: apps/v1
    kind: DaemonSet
    metadata:
      name: smarter-device-manager
      namespace: smarter-device-manager
      labels:
        name: smarter-device-manager
        role: agent
    spec:
      selector:
        matchLabels:
          name: smarter-device-manager
      updateStrategy:
        type: RollingUpdate
      template:
        metadata:
          labels:
            name: smarter-device-manager
          annotations:
            node.kubernetes.io/bootstrap-checkpoint: "true"
        spec:
          nodeSelector:
            smarter-device-manager: enabled
          priorityClassName: "system-node-critical"
          hostname: smarter-device-management
          hostNetwork: true
          dnsPolicy: ClusterFirstWithHostNet
          containers:
            - name: smarter-device-manager
              image: registry.gitlab.com/arm-research/smarter/smarter-device-manager:v1.20.7
              imagePullPolicy: IfNotPresent
              securityContext:
                allowPrivilegeEscalation: false
                capabilities:
                  drop: ["ALL"]
              resources:
                limits:
                  cpu: 100m
                  memory: 15Mi
                requests:
                  cpu: 10m
                  memory: 15Mi
              volumeMounts:
                - name: device-plugin
                  mountPath: /var/lib/kubelet/device-plugins
                - name: dev-dir
                  mountPath: /dev
                - name: sys-dir
                  mountPath: /sys
                - name: config
                  mountPath: /root/config
          volumes:
            - name: device-plugin
              hostPath:
                path: /var/lib/kubelet/device-plugins
            - name: dev-dir
              hostPath:
                path: /dev
            - name: sys-dir
              hostPath:
                path: /sys
            - name: config
              configMap:
                name: smarter-device-manager
          terminationGracePeriodSeconds: 30
    

    Next, apply the changes to your clusters by running:

    kubectl apply -f ./smarter-device-manager.yaml
    

    The example DaemonSet includes a nodeSelector that constrains the device plugin to nodes with the smarter-device-manager label set to enabled. Label the nodes that will include the FUSE device by using the following command, or remove the nodeSelector from the manifest:

    kubectl get nodes
    kubectl label nodes --all smarter-device-manager=enabled 
    
  2. If you haven't already done so for your Coder deployment, enable workspace templates. To do so, go to Manage > Admin > Templates, and set the Enable workspace templates to On. Click Save.

  3. Create a workspace configuration file that includes instructions for resource requests and resource limits (the instructions ask the cluster to request the FUSE device for each workspace):

    version: "0.2"
    workspace:
      specs:
        kubernetes:
          resource-requests:
            policy: write
            value:
              smarter-devices/fuse: "1"
          resource-limits:
            policy: write
            value:
              smarter-devices/fuse: "1"
    

    A complete workspace template might look something like

    version: "0.2"
    workspace:
      configure:
        start:
          policy: write
      dev-urls:
        policy: write
      specs:
        aws-ec2-docker:
          container-image:
            policy: write
          disk-size:
            policy: write
          instance-type:
            policy: write
        docker:
          container-based-vm:
            policy: write
          image:
            policy: write
        kubernetes:
          annotations:
            policy: read
          container-based-vm:
            policy: write
          cpu:
            policy: write
          disk:
            policy: write
          env:
            policy: write
          gpu-count:
            policy: write
          image:
            policy: write
          labels:
            policy: read
          memory:
            policy: write
          node-selector:
            policy: read
          privileged:
            policy: read
          resource-requests:
            policy: write
            value:
              smarter-devices/fuse: "1"
          resource-limits:
            policy: write
            value:
              smarter-devices/fuse: "1"
          runtime-class-name:
            policy: read
          tolerations:
            policy: read
    
  4. In the Coder UI, navigate to Manage > Admin > Templates if you haven't already done so. Under template policy, upload the configuration file you created in the previous step. Click Save.

    With the above template policy, all workspaces will acquire a FUSE device, which enables Podman to operate in rootless mode.

For systems running AppArmor and SELinux

Running Podman in rootless mode requires a FUSE device to implement the overlay filesystem (fuse-overlayfs) in unprivileged mode. The following directions work by mounting the FUSE device from the host into workspace containers, which conflicts with the isolation provided by SELinux and AppArmor.

For systems running AppArmor (typically Debian- and Ubuntu-derived systems), please disable AppArmor before proceeding.

For systems running SELinux (typically Fedora-, CentOS-, and Red Hat-based systems), please disable SELinux or set it to permissive mode.

Testing

At this point, you can create a workspace that leverages Podman. If you need a sample Podman image, you can obtain one from RedHat. When using this image, switch to the unprivileged podman user before creating containers to ensure that podman runs in rootless mode.

See an opportunity to improve our docs? Make an edit.