Running Coder in a K3s Cluster Self-Hosted

This post aims at describing the deployment of Coder on a Kubernetes cluster via the K3s distribution.

Coder is an online development environment platform that offers the possibility to code directly from a web browser. All the compilation toolchain is then deported on the server. Coder provides a ready-to-use remote development environment and reduces the installation and configuration issues that could arise from the heterogeneity of developer's workstation architectures and software used.

This kind of solution has gained popularity in recent years, with several alternative solutions such as:

• Gitpod
• EclipseCHE
• JupyterHub
• GitHub Codespaces
• Codeanywhere

These alternative solutions have a lot in common, but also some distinguishing features that can make them attractive.

I am interested in deploying an online development platform within a higher education governmental institution. This platform is meant to be used by researchers and students. Several features are mandatory, such as self-hosting, open source, great community, ease of installation, and the ability to handle multiple programming languages.

A first experimentation was based on Gitpod. Unfortunately, Gitpod no longer supports self-hosting, as explained on the Gitpod blog. Furthermore, setting up Gitpod was not easy, and I can understand the choice made by Gitpod developers, as explained on the blog post, to drop support for self-hosting since there were many issues arising from the heterogeneity of Kubernetes distributions. A second solution was JupyterHub, which was fairly easy to install but limited to the Python language. In a third experimentation, I therefore turned to Coder. In this post I will explain how I deployed it on a K3s cluster.

Coder solution matches all features I need, because it is self-hosted, open source and has a greatDiscord community. From a technical point of view, Coder provisions remote development environments via Terraform to supply users with Workspaces. For example, via Terraform, you can specify that you want to use a specific Docker image that contains the necessary compilation toolchain to build Java programs and that you want to install VIM as a code editor. Resources will also be specified, such as the memory or storage capacity of Workspaces. As for K3s, it is a lightweight Kubernetes distribution designed for production environments with limited resources, such as embedded systems. I appreciate K3s because it is designed to be easy to install.

The deployment of Coder on a K3s cluster is broken down into the following steps:

• Setup
• Install k3s
• Deploy Coder
• Deploy reverse-proxy
  • NGINX
  • Apache HTTP
• Run and enjoy
• Next steps

All materials can be found on my Github repository: https://github.com/mickaelbaron/coder-k3s-guide

Setup

Before starting the Coder install process, you must have:

• The following infrastructure (to reproduce the experimentation):

  • Three Ubuntu 22.04 machines with SSH credentials.
  • The hostname of each machine (physical or virtual) is: k3sserver, k3snode1 and k3snode2.
  • All machines are on the same vlan.
  • All machines have ports 22 (SSH), 80 (HTTP), 443 (HTTPS) and 6443 (Kubernetes) exposed.
  • A domain (coder.mydomain.com), a subdomain (*.coder.mydomain.com) and a configured DNS to redirect to k3sserver.
  • A reverse-proxy (Apache HTTP or NGINX) which will be hosted outside of the Kubernetes cluster.
  • A Docker installation on k3sserver to deploy the reverse-proxy.

• Locally

  • kubectl
  • HELM

Install K3s

• Connect to the server node (k3sserver) and run:

• Extract the K3s token on the server node:

• Connect to the first agent node (k3snode1) and run:

• Connect to the second agent node (k3snode2) and execute the same command line:

• To get the cluster access file (k3s.yaml), from your host, run:

• Update the K3s server address (old value: 127.0.0.1) with the new hostname:

• Check the K3s cluster:

Deploy Coder

• Create a namespace for Coder, named coder in this example:

• Deploy PostgreSQL on the K3s cluster from the Bitnami repository:

• Verify that a PostgreSQL pod has been created:

The cluster-internal DB URL for the PostgreSQL database is:

• Create a secret with the database URL:

• Add the Coder Helm repository:

• Create a values.yaml configuration file with the suitable settings for your deployment. You should at least update the content following the # TODO comments:

The service will be configured as a ClusterIP. We configure Ingress to handle requests for the (coder.mydomain.com) domain.

• Install the HELM chart on your K3s cluster:

• Check the pods into the coder namespace:

Deploy reverse-proxy

As mentioned in the setup section, a reverse-proxy will be deployed outside of your Kubernetes cluster.

The reverse-proxy will also be in charge of managing SSL/TLS certificates. Let's describe how to generate certificates with LetsEncrypt.

• Connect to the server node (k3sserver) and install Certbot:

• Create the SSL/TLS certificates:

• Copy the SSL/TLS certificates files (fullchain.pem and privkey.pem) into a directory (i.e. /ssl):

• Generate dhparams.pem file:

You need to configure your Kubernetes cluster to update HTTP and HTTPS listen ports.

• Connect to the server node (k3sserver) and create a /var/lib/rancher/k3s/server/manifests/traefik-config.yaml file with the following content:

• Apply this configuration:

We suppose Docker is installed on the server node (k3sserver).

• Create a Docker network called reverseproxynetwork:

Two reverse-proxy solutions will be presented: NGINX and Apache HTTP. Choose only ONE at your convenience.

NGINX

• Connect to the server node (k3sserver).

• Create an nginx directory:

• Create an NGINX configuration file nginx/conf/coder.conf with the following content:

• Create a file nginx/docker-compose.yaml with the following content:

• Create and start the NGINX container:

Apache HTTP

• Connect to the server node (k3sserver).

• Create an apachehttp directory.

• Create an Apache HTTP configuration file apachehttp/conf/coder.conf with the following content:

• Copy and update at your convenience the apachehttp/httpd.conf configuration file.

• Create a file apachehttp/docker-compose.yaml with the following content:

• Create and start Apache HTTP container:

Test Coder

• Open the https://coder.mydomain.com URL with your favorite web browser.

Next steps

I hope this post helped you deploy Coder on a Kubernetes cluster via the K3s distribution.

I still have a lot to learn on how to use and configure Coder. In a future post, I will try to provide some feedback on my use of Coder and Kubernetes. Several questions still remain unanswered:

• How should I scale? How much resources (cpu, memory, disk) should I allocate for each user (student or researcher)?
• What are the limitations of using a remote development environment? How can I integrate hardware used during hands-on exercises (sensors, robots with Raspberry Pis)?
• How can I provide access to users data, especially when hosted on remote volumes?

The answers to these questions, among others, might help me conclude whether Coder is the solution to my problem.