The workspace process logging feature allows you to log all system-level processes executing in the workspace.
Enabling workspace process logging adds a sidecar to your workspaces that will log all processes users start in the workspace (e.g., commands executed in the terminal or system processes created by background services in CVM workspaces). You can view the output from the sidecar or send it to a monitoring stack, such as CloudWatch, for further analysis.
Please note that these logs are not recorded or captured by the Coder organization in any way, shape, or form.
Coder uses eBPF (which we chose for its speed) to perform in-kernel
logging and filtering of all
exec system calls to match events originating
from the workspace.
The core of this feature is also open source and can be found in the
exectrace repo on GitHub repo.
Use of the workspace process logging functionality requires a host Linux
kernel >= 5.8 with the kernel config
To validate this config is enabled, run either of the following commands on the nodes directly (not from the terminal within a workspace):
cat /proc/config.gz | gunzip | grep CONFIG_DEBUG_INFO_BTF
cat "/boot/config-$(uname -r)" | grep CONFIG_DEBUG_INFO_BTF
To enable workspace process logging:
This setting will apply to all new workspaces; it will apply to existing workspaces only after they have been rebuilt.
To view the process logs from a specific user or workspace, you can use your
cloud provider's log viewer, or you can use
kubectl to print the logs:
kubectl logs \ -l "com.coder.username=zac" \ # Filter by the user "zac" -l "com.coder.workspace.name=code" \ # Filter by the workspace "code" -c exectrace # Only show logs from the sidecar
If you're using AWS' Elastic Kubernetes Service, you can configure your cluster to send logs to CloudWatch. This allows you to view the logs for a specific user or workspace.
To view your logs, go to the CloudWatch dashboard (which is available on the Log Insights tab) and run a query similar to the following:
fields @timestamp, log_processed.fields.cmdline | sort @timestamp asc | filter kubernetes.container_name="exectrace" | filter log_processed.fields.labels.username="zac" | filter log_processed.fields.labels.workspace_name="code"
makerun) will result in an abundance of output in the sidecar container. Depending on how your Kubernetes cluster is configured, you may incur extra charges from your cloud provider to store the additional logs.