Query-Mutating Data Race in Go: Hiding in Plain Sight

Coder’s engineering team has discovered a serious security vulnerability in the Go standard libary database/sql package. Attacks based on this vulnerability are application and SQL-driver specific, but could allow a remote attacker to modify SQL query results, with potentially disastrous consequences up to full application compromise.

We highly recommend engineering teams using database/sql update to Go version 1.24.6 immediately.

The Coder application is patched in versions 2.25.1, 2.24.3, and 2.23.5. We highly recommend all Coder deployments upgrade to a patched release.

This vulnerability is assigned CVE-2025-47907.

Impact and outlook

When a query running in a vulnerable version of Go is canceled, there exists a data race where the SQL driver’s connection object can be returned to the pool of available connections and reused for a second query while the first query is still decoding the results. If the SQL driver retains any memory, such as internal buffers (fairly common), this memory could be overwritten by the second query, corrupting the results of the first query. It’s similar to the “use after free” class of bugs, but here we are returning a shared resource to a resource pool instead of returning raw memory to a memory manager.

A deliberate attack based on this vulnerability is application specific and SQL Driver specific: it involves knowing the memory layout of internal buffers and application-specific queries and results.

As of writing there are no known exploits against any applications.

We expect that developing a successful exploit is difficult because it is triggered by narrow timing window during which the first query’s context.Context must be canceled while a result row is being decoded (”scanned” in the SQL driver terminology) into Go data types. Then the SQL driver must be reused for a second query all before the first query decode completes. Furthermore, to be a successful exploit, the application needs to use the results of the SQL query even after the context.Context has been canceled.

That’s the good news: that we expect it’s hard to exploit. The bad news is that a successful exploit could be very severe. If an attacker were able to corrupt the results of a SQL query, the impact is limited only by the security implications of that query. Some examples:

• If a query fetching a cryptographic authentication key were compromised, an attacker could bypass access controls and take complete control of the application.
• If a query fetches executable code or scripts, a successful exploit could result in remote code execution.

How we identified the vulnerability

The vulnerability has been hiding in plain sight in database/sql since at least Go version 1.10. As early as at least 2021, developers have been hitting the issue, but incorrectly diagnosed it as an issue in the lib/pq driver, rather than in the Standard Library, and seem to have missed the security implications. Our team also initially missed the security implications when we started seeing data races detected by the Go race detector in our CI.

My spidey sense was tingling when I got involved in looking at the issue in June. The output of the race detector was showing a data race while reading data from a SQL query:

Previous read at 0x00c00120ef3c by goroutine 2187:
  runtime.slicecopy()
      /opt/hostedtoolcache/go/1.24.4/x64/src/runtime/slice.go:355 +0x0
  bytes.Clone()
      /opt/hostedtoolcache/go/1.24.4/x64/src/bytes/bytes.go:1412 +0xfbc
  database/sql.convertAssignRows()
      /opt/hostedtoolcache/go/1.24.4/x64/src/database/sql/convert.go:272 +0xfbd
  database/sql.(*Rows).Scan()
      /opt/hostedtoolcache/go/1.24.4/x64/src/database/sql/sql.go:3400 +0x727
  github.com/coder/coder/v2/coderd/database.(*sqlQuerier).GetAPIKeysLastUsedAfter()
      /home/runner/work/coder/coder/coderd/database/queries.sql.go:315 +0x667
  github.com/coder/coder/v2/coderd/database.(*sqlQuerier).GetAPIKeysLastUsedAfter()
      /home/runner/work/coder/coder/coderd/database/queries.sql.go:307 +0x154
(truncated for brevity)

That is, while decoding API Keys from the database, some other goroutine was modifying the data we were decoding! The writing go routine we found in that CI run was just closing the database connection. Not something with any possible user-controlled parameters, so I was intrigued but not yet worried.

I sounded the alarm internally when I later discovered a more damning data race in our CI, which showed two separate queries involved as the reading and writing goroutines, respectively. This proved that it was possible for user-controlled data from an unrelated query to leak into the results.

Having never written a Go sql.Driver myself, I also initially assumed the vulnerability was in the lib/pq driver that we use: that it was retaining a reference to a buffer is was supposed to have passed control of to the database/sql package. But, with some more research into the contract and inline comments in the database/sql package, it became clear to me that instead the issue was a narrow race condition in which the *sql.Rows object could be closed by context cancelation while a Scan() call was still in progress. From the perspective of a SQL driver, the buffers used to return results were fair game to re-use since the Rows had been closed.

Take aways for Go developers

Our team’s primary role is developing our own applications, not hunting for security issues: we are not security researchers by training. Still, security is everyone’s responsibility and we strive to be vigilant in our engineering practice.

• Run the Go race detector over as much of your CI test suite as you can. Data races are fertile ground for security vulnerabilities, so give them extra attention when your tests shake them out.
• Investigate any integrity issues in the persistence layer. For us, and many applications like us, all authentication and authorization decisions ultimately rest on data we retrieve from a SQL database, so a compromise there is as severe as it gets.
• There is no substitute for a full root cause analysis. When we started seeing the data races, we discussed moving to a different PostgreSQL driver. While it might still be a good idea in terms of something that is more actively maintained, if we just switched and moved on we could still be vulnerable today, since it was not the driver that had the issue.

We would like to acknowledge and thank the Go team for their attention and help in getting this issue resolved.