Platform Engineers: Navigating Complexity in the Cloud Era

on December 18th, 2024

4 min read

Platform engineering has become a hot topic in tech circles, with many organizations struggling to implement it effectively. Whether you call it “platform engineering”, “site reliability engineering”, or “developer experience”, the mission is the same: designing and maintaining an organization’s software infrastructure so that developers have a stable, cost-effective foundation for their applications and their development.

Coder recently hosted roundtable discussions with platform engineers to explore their experiences. What began as technical deep dives quickly evolved into candid conversations about the organizational dynamics shaping their roles.

Platform Engineer’s Balancing Act

Platform engineers deal with challenging audiences:

Developers expect simplicity, stability, and performance.
Organizations expect a return on their software investment.
Both see this infrastructure as a black box that the platform engineering team is responsible for.

The engineers we spoke to shared how these expectations manifest in their daily work:

Buy-in from stakeholders: Navigating internal politics and winning executive support.
Encouraging adoption: Choosing an effective mix of carrot and stick incentives.
Managing expectations: Balancing the need for quick solutions with long-term sustainability.
Measuring ROI: Convincingly justifying technical decisions with measurable results.

The Case for Platforms

With robust platforms in place, developers are free to focus on delivering applications instead of wrestling with infrastructure. The industry has all but officially standardized on cloud-native infrastructure services like Kubernetes and Docker, but they’re just the beginning. Platforms provide essential abstractions, managing the complexities of billing, web, API, message queuing, dev tools, and so on.

For example, Coder helps large enterprises measurably, dramatically lighten the burden of administering complex development environments. Developers get coding quickly with shortened project onboarding and build times. Organizations spend less on development infrastructure. Platform teams benefit from consolidated management and monitoring of development environments, freeing them from one-off issues and maintenance headaches.

Technical challenges and the over-general swamp

While technical challenges are part of the job, they often spiral into something bigger:

Growing variation: With a growing number of applications to support, each uses slight variations in the infrastructure they depend on. Over time, complexity creeps in. The glue that connects all this infrastructure becomes increasingly expensive to maintain.
Keeping up: As applications grow in scope, the platforms under them must scale too. The platform team stays ahead of this by updating or replacing components in the infrastructure.

The solutions to these technical challenges are defined and executed by platform engineering teams, but developers and the organization push back because they perceive changes in infrastructure as putting stability and ROI at risk.

A platform team finds itself in an "over-general swamp" as they spend too much time maintaining disparate systems instead of innovating, while the application teams they support are drowning in complexity.

The goal? Escape to a "golden path" that checks off all these boxes:

Standardized well-supported platforms for common development features
Reduced cognitive load for developers
Flexibility for edge cases
Follow opportunities for innovation

A new book published by O’Reilly Media, Platform Engineering: A Guide for Technical, Product, and People Leaders, shows a platform engineering team how to leave the over-general swamp to find their golden path.

It’s written by leaders in the industry with decades of combined experience. Camille Fournier is an executive, speaker, and author. Ian Nowland’s resume includes Senior Vice President of Core Engineering at Datadog and, most recently, founder of Junction Labs.

The book guides a platform team on making competent technical choices that win acceptance from stakeholders:

Build trust: Earning confidence through operational excellence and stakeholder buy-in for major investments.
Metrics: Creating measurable goals to justify technical decisions.
Standardization and innovation: Reducing complexity without stifling experimentation.
Managing migrations: Using more automated, self-service approaches.

Learn more

Our roundtable discussions highlighted the intersection of technical expertise and organizational savvy in platform engineering. That’s why we’re excited to share a free excerpt from Platform Engineering: A Guide for Technical, Product, and People Leaders:

Download the excerpt for free.
Read the full book from O'Reilly or your favorite book seller.
Hear from the authors directly at this webinar hosted by O’Reilly—register here.