How Cluster Sprawl Creates Waste, Risk in Your Organization | D2iQ

As various teams within your organization are discovering new ways to leverage Kubernetes, they’re adopting a growing number of clusters to support their project efforts. Unfortunately, this is where many of the challenges begin. While managing one Kubernetes cluster is not trivial, trying to manage multiple Kubernetes clusters across multiple environments becomes exponentially more difficult. And, if you’re an enterprise with an expanse of clusters that are being managed independently with very little uniformity, the complexities can be a huge barrier to success. 

 

Here are three common problems introduced by cluster sprawl that can have a significant impact on your business.

 

Lack of Visibility and Management

As various parts of the organization require new Kubernetes clusters, it becomes increasingly difficult to know where they exist, how they are performing, and to govern the usage and versions of cloud native software to support application efforts. For example, teams will be building one stack on cloud provider “A,” a different stack on cloud provider “B,” and a third stack at the edge or in a data center. As the number of clusters and workloads grows and multiplies across different environments, operators struggle to create standardization around how and where new clusters are configured and used.

 

When there’s a lack of centralized governance and visibility over how and where these resources are provisioned, it negatively impacts the bottom line. If a cluster goes down, you can’t troubleshoot problems without losing valuable time. You also can’t easily obtain insights on cluster performance to deliver better resource utilization. And if there are dozens of potential software versions in use, managing all of them across the organization is nearly impossible. All of these issues will eventually lead to inconsistent performance and reliability issues and an increase in security risks and development and maintenance costs. 

 

Operational Complexity and Overhead

 

Identity and access management are critical components of many applications. However, when various teams begin the effort of deploying broad sets of clusters, it creates several challenges when it comes to managing authentication credentials, resource sharing, and security. 

 

Admins need to be able to map each user and their activities to their cluster so they know who is doing what and when. When it comes to resource sharing, not everyone needs the same access to resources as others. Some users don’t need the same access to Kubernetes resources as others. While this scenario is much easier to manage and implement at a smaller company, it’s exponentially more difficult at an enterprise company, especially when there are multiple accounts and access levels to manage. 

 

As teams expand their usage of Kubernetes, clusters will exist in different pockets each with differing policies, roles, and configurations in their usage. This variety makes it incredibly challenging to create consistency across clusters. Operators lose the flexibility to define user roles, responsibilities, and privileges to ensure the right people are performing the right tasks within the environment. In addition, when there’s a lack of governance and access control, operators can’t identify role violations, assess governance risk, and perform compliance checks. When more time is spent putting out fires, there is less time for efficient operations. 

 

Empowering Both Developers and Operators

 

When adopting Kubernetes and other cloud native services, it’s important to think about how they’ll impact the engineering culture. Kubernetes is popular amongst developers because it enables them to spin up their own environments with ease and agility. While this isolated autonomy gives them greater flexibility, it makes it tough for operators to create standardization around how new clusters are configured and used. The challenge becomes how much freedom should developers have when it comes to selecting and developing new technologies that accelerate innovation? And how much structure should there be in place to deploy policy and secure governance requirements without sacrificing innovation?

 

Developers want their own community clusters and to do it in their sandbox. They want to install their own applications and don’t want to talk to a different team when they do it. So how can you give them that while at the time enforcing the standards that you need to? How do you make sure those clusters follow a certain blueprint that has the right access control rules? How do you ensure that sensitive information like credentials are distributed in the right way? And how do you ensure the right versions of software or workloads are available? 

 

“I think it’s about finding the right balance between that flexibility and governance,” says Tobi Knaup, Co-CEO of D2iQ. “Giving developers the flexibility they want to leverage the benefits of cloud native, while informing operations about the different stacks that are provisioned so they can enforce governance.”

 

To learn more about how cluster sprawl can impact your organization, download the ebook, “Kubernetes Governance: Take Control of Your Multi-Cluster Operations.”