--- title: "Running Coder in a K3s Cluster Self-Hosted - Blog - Coder" description: "Read our latest blog posts on various topics including engineering, product, and company updates." image: "/api/dynamic-og?title=Running+Coder+in+a+K3s+Cluster+Self-Hosted&randomBackground=true&styles=%7B%22height%22%3A162%2C%22width%22%3A302%2C%22padding%22%3A16%2C%22titleSize%22%3A24%2C%22logo%22%3A%7B%22width%22%3A80%2C%22height%22%3A12.15%7D%7D" canonical: "https://coder.com/blog/running-coder-in-a-k3s-cluster-self-hosted" --- May 23 202311 min read # Running Coder in a K3s Cluster Self-Hosted [Mickaël Baron](https://coder.com/blog/author/mickaelbaron) Share this article This post aims at describing the deployment of [Coder](https://coder.com/) on a Kubernetes cluster via the [K3s](https://k3s.io/) distribution. [Coder](https://coder.com/) is an online development environment platform that offers the possibility to code directly from a web browser. All the compilation toolchain is then deported on the server. [Coder](https://coder.com/) provides a ready-to-use remote development environment and reduces the installation and configuration issues that could arise from the heterogeneity of developer's workstation architectures and software used. This kind of solution has gained popularity in recent years, with several alternative solutions such as: - [Gitpod](https://www.gitpod.io/) - [EclipseCHE](https://www.eclipse.org/che/) - [JupyterHub](https://jupyter.org/hub) - [GitHub Codespaces](https://github.com/features/codespaces) - [Codeanywhere](https://codeanywhere.com/) These alternative solutions have a lot in common, but also some distinguishing features that can make them attractive. I am interested in deploying an online development platform within a higher education governmental institution. This platform is meant to be used by researchers and students. Several features are mandatory, such as self-hosting, open source, great community, ease of installation, and the ability to handle multiple programming languages. A first experimentation was based on [Gitpod](https://www.gitpod.io/). Unfortunately, Gitpod no longer supports self-hosting, as explained on the [Gitpod blog](https://www.gitpod.io/blog/introducing-gitpod-dedicated). Furthermore, setting up [Gitpod](https://www.gitpod.io/) was not easy, and I can understand the choice made by [Gitpod](https://www.gitpod.io/) developers, as explained on the blog post, to drop support for self-hosting since there were many issues arising from the heterogeneity of Kubernetes distributions. A second solution was [JupyterHub](https://jupyter.org/hub), which was fairly easy to install but limited to the Python language. In a third experimentation, I therefore turned to [Coder](https://coder.com/). In this post I will explain how I deployed it on a [K3s](https://k3s.io/) cluster. [Coder](https://coder.com/) solution matches all features I need, because it is [self-hosted](https://coder.com/docs/v2/latest/install), [open source](https://github.com/coder) and has a great[Discord community](https://discord.com/invite/coder). From a technical point of view, [Coder](https://coder.com/) provisions remote development environments via Terraform to supply users with Workspaces. For example, via Terraform, you can specify that you want to use a specific Docker image that contains the necessary compilation toolchain to build Java programs and that you want to install VIM as a code editor. Resources will also be specified, such as the memory or storage capacity of Workspaces. As for [K3s](https://k3s.io/), it is a lightweight Kubernetes distribution designed for production environments with limited resources, such as embedded systems. I appreciate [K3s](https://k3s.io/) because it is designed to be easy to install. The deployment of [Coder](https://coder.com/) on a [K3s](https://k3s.io/) cluster is broken down into the following steps: - [Setup](#setup) - [Install k3s](#install-k3s) - [Deploy Coder](#deploy-coder) - [Deploy reverse-proxy](#deploy-reverse-proxy) - [NGINX](#nginx) - [Apache HTTP](#apache-http) - [Run and enjoy](#test-coder) - [Next steps](#next-steps) All materials can be found on my Github repository: [https://github.com/mickaelbaron/coder-k3s-guide](https://github.com/mickaelbaron/coder-k3s-guide) ## Setup Before starting the [Coder](https://coder.com/) install process, you must have: - The following infrastructure (to reproduce the experimentation): - Three Ubuntu 22.04 machines with SSH credentials. - The hostname of each machine (physical or virtual) is: `k3sserver`, `k3snode1` and `k3snode2`. - All machines are on the same vlan. - All machines have ports 22 (SSH), 80 (HTTP), 443 (HTTPS) and 6443 (Kubernetes) exposed. - A domain (coder.mydomain.com), a subdomain (\*.coder.mydomain.com) and a configured DNS to redirect to `k3sserver`. - A reverse-proxy (Apache HTTP or NGINX) which will be hosted outside of the Kubernetes cluster. - A [Docker](https://www.docker.com/) installation on `k3sserver` to deploy the reverse-proxy. - Locally - [kubectl](https://kubernetes.io/docs/reference/kubectl/) - [HELM](https://helm.sh/) > **Note**: in the following infrastructure, I will present a configuration where TLS certificates will be managed by a reverse-proxy outside of Kubernetes. It is a **restriction** of my higher education governmental institution and I would like to reproduce the same setup and to show you this specific use case. But, I agree in many cases, the ingress could/should also be secured via [cert-manager](https://cert-manager.io/) or by passing TLS certificates in directly. It would be better to stay within the K8s infrastructure. ## Install [K3s](https://k3s.io/) - Connect to the server node (`k3sserver`) and run: ``` ``` - Extract the [K3s](https://k3s.io/) token on the server node: ``` ``` > This is my own [K3s](https://k3s.io/) token. You will have to adapt the next few instructions with YOUR [K3s](https://k3s.io/) token. - Connect to the first agent node (`k3snode1`) and run: ``` ``` - Connect to the second agent node (`k3snode2`) and execute the same command line: ``` ``` - To get the cluster access file (_k3s.yaml_), from your host, run: ``` ``` - Update the [K3s](https://k3s.io/) server address (old value: 127.0.0.1) with the new hostname: ``` ``` - Check the [K3s](https://k3s.io/) cluster: ``` ``` > These instructions are based on the [K3s](https://k3s.io/) website: [https://docs.k3s.io/quick-start](https://docs.k3s.io/quick-start). ## Deploy Coder - Create a namespace for [Coder](https://coder.com/), named `coder` in this example: ``` ``` - Deploy PostgreSQL on the [K3s](https://k3s.io/) cluster from the [Bitnami](https://bitnami.com/) repository: ``` ``` - Verify that a PostgreSQL pod has been created: ``` ``` The cluster-internal DB URL for the PostgreSQL database is: ``` ``` - Create a secret with the database URL: ``` ``` - Add the Coder Helm repository: ``` ``` - Create a [values.yaml](https://github.com/mickaelbaron/coder-k3s-guide/blob/main/coder/values.yaml) configuration file with the suitable settings for your deployment. You should at least update the content following the `# TODO` comments: ``` ``` The service will be configured as a ClusterIP. We configure Ingress to handle requests for the (`coder.mydomain.com`) domain. - Install the HELM chart on your [K3s](https://k3s.io/) cluster: ``` ``` - Check the pods into the `coder` namespace: ``` ``` > These instructions are based on the [Coder](https://coder.com/) website: [https://coder.com/docs/v2/latest/install/kubernetes](https://coder.com/docs/v2/latest/install/kubernetes). ## Deploy reverse-proxy As mentioned in the setup section, a reverse-proxy will be deployed outside of your Kubernetes cluster. The reverse-proxy will also be in charge of managing SSL/TLS certificates. Let's describe how to generate certificates with LetsEncrypt. - Connect to the server node (`k3sserver`) and install Certbot: ``` ``` - Create the SSL/TLS certificates: ``` ``` - Copy the SSL/TLS certificates files (_fullchain.pem_ and _privkey.pem_) into a directory (i.e. _/ssl_): ``` ``` - Generate _dhparams.pem_ file: ``` ``` You need to configure your Kubernetes cluster to update HTTP and HTTPS listen ports. - Connect to the server node (`k3sserver`) and create a _/var/lib/rancher/k3s/server/manifests/traefik-config.yaml_ file with the following content: ``` ``` - Apply this configuration: ``` ``` We suppose [Docker](https://www.docker.com/) is installed on the server node (`k3sserver`). - Create a Docker network called `reverseproxynetwork`: ``` ``` Two reverse-proxy solutions will be presented: [NGINX](https://www.nginx.com/) and [Apache HTTP](https://httpd.apache.org/). Choose only ONE at your convenience. ### NGINX - Connect to the server node (`k3sserver`). - Create an _nginx_ directory: ``` ``` - Create an NGINX configuration file [nginx/conf/coder.conf](https://github.com/mickaelbaron/coder-k3s-guide/blob/main/nginx/conf/coder.conf) with the following content: ``` ``` - Create a file [nginx/docker-compose.yaml](https://github.com/mickaelbaron/coder-k3s-guide/blob/main/nginx/docker-compose.yaml) with the following content: ``` ``` - Create and start the NGINX container: ``` ``` ### Apache HTTP - Connect to the server node (`k3sserver`). - Create an _apachehttp_ directory. ``` ``` - Create an Apache HTTP configuration file [apachehttp/conf/coder.conf](https://github.com/mickaelbaron/coder-k3s-guide/blob/main/apachehttp/conf/coder.conf) with the following content: ``` ``` - Copy and update at your convenience the [apachehttp/httpd.conf](https://github.com/mickaelbaron/coder-k3s-guide/blob/main/apachehttp/httpd.conf) configuration file. - Create a file [apachehttp/docker-compose.yaml](https://github.com/mickaelbaron/coder-k3s-guide/blob/main/apachehttp/docker-compose.yaml) with the following content: ``` ``` - Create and start Apache HTTP container: ``` ``` ## Test Coder - Open the [https://coder.mydomain.com](https://coder.mydomain.com) URL with your favorite web browser. ## Next steps I hope this post helped you deploy [Coder](https://coder.com/) on a Kubernetes cluster via the [K3s](https://k3s.io/) distribution. I still have a lot to learn on how to use and configure [Coder](https://coder.com/). In a future post, I will try to provide some feedback on my use of [Coder](https://coder.com/) and Kubernetes. Several questions still remain unanswered: - How should I scale? How much resources (cpu, memory, disk) should I allocate for each user (student or researcher)? - What are the limitations of using a remote development environment? How can I integrate hardware used during hands-on exercises (sensors, robots with Raspberry Pis)? - How can I provide access to users data, especially when hosted on remote volumes? The answers to these questions, among others, might help me conclude whether [Coder](https://coder.com/) is the solution to my problem. ### Subscribe to our newsletter Want to stay up to date on all things Coder? Subscribe to our monthly newsletter for the latest articles, workshops, events, and announcements.