Deploy Generative AI at Scale with Gemini using Coder

on March 20th, 2024

Updated on October 16th, 2024

4 min read

This blog is part three of an ongoing Gen AI + Coder blog series. Deploy and Govern Generative AI Tools at Scale with Coder (Part One), Deploy Generative AI at scale with one code block using Coder, VS Code Web and GitHub Copilot (Part Two).

Welcome back! As we discussed in our last post one of the greatest advantages of using a Cloud Development Environment (CDE) such as Coder is the ability to deploy developer tooling at scale quickly in a consistent, stable and secure manner. This blog post covers using Google’s Gemini extension for VS Code in a Coder workspace. It is available on openvsx which makes it easy to run in a workspace using Coder’s browser-based IDE code-server.

While one can install this extension in desktop VS Code, that method relies on each developer to install the extension and keep it updated. Both Coder Open Source and Coder Enterprise support automatically installing extensions in browser-based IDEs. Although only Coder Enterprise allows platform-wide restrictions to run in browser-only mode. As with the last post the goal is to show how to deploy extensions at scale in a secure, enterprise-like deployment so this example is shown in a browser-only mode.

There are some additional parts to set up Gemini. First there are some steps required to enable Gemini in your Google project, it is well documented and quick to set up. Once you have your project and users enabled, the rest of the setup is in the Coder template. You must install the Google Cloud SDK into the workspace prior to using the Gemini extension. This is easy to do with Coder’s startup_script, this example checks to see if gcloud is installed in your workspace, if not it will download and install it, then install the extension.

resource "coder_agent" "main" {
  os                     = "linux"
  arch                   = "amd64"
  startup_script         = <<-EOT
    set -e

    # install and start code-server
    curl -fsSL https://code-server.dev/install.sh | \
        sh -s -- \
        --method=standalone \
        --prefix=/tmp/code-server --version 4.11.0
    /tmp/code-server/bin/code-server \
        --auth none --port 13337 >/tmp/code-server.log 2>&1 &
    
    if ! [ -f ~/google-cloud-sdk/bin/gcloud ]
    then
      filename=/tmp/gcloud_installer.sh
      curl -o $filename https://sdk.cloud.google.com 
      chmod +x $filename
      gcloud_log=/tmp/gcloud_install.log
      echo "Installing gcloud tools, check $gcloud_log"
      $filename --disable-prompts > $gcloud_log 2>&1
    else
      echo "gcloud already installed"
    fi 
    /tmp/code-server/bin/code-server \
        --install-extension googlecloudtools.cloudcode
  EOT
}

Build and publish the template and that’s it. All of the users who build/rebuild from this template will have the current version of Gemini installed and available to use. Keep in mind that it takes a few minutes to download and install the Google Cloud SDK. This only happens when the workspace is created, you can check the install progress by tailing /tmp/gcloud_install.log. I had the installer log it to a file rather than display it in the workspace build log because the SDK install creates a lot of output, so much that it will exceed Coder’s limit of 1MB for build logs and could obscure other important build information.

There are some considerations for larger deployments, especially in high-security DCs. In these scenarios, tooling is typically baked into the workspace image rather than downloaded on each workspace start. This is fine for installing IDE’s but extensions are typically installed in the user’s home directory. There are a few different ways to install extensions in these environments.

Without going into detail on each, the methods are:

Host the extension somewhere inside your DC and install from there
Host the extension somewhere on the image and install or run from there
Have the users sync their dotfiles with pre-installed extension configurations from a source code repo

Leveraging a CDE to deploy Gemini is a leap forward in scalable, secure developer tooling. This blog post has guided you through setting up these tools in a streamlined, enterprise-grade fashion. By adopting the outlined approach, teams can ensure that every member operates with the latest tools and extensions, effortlessly. Embrace this method to elevate your development processes, ensuring your team remains on the cutting edge of technology. Ready to revolutionize your workflow? Dive in and deploy your enhanced development environment today.