Azure Kubernetes Chronicles 11: Governing the Chaos

Azure Kubernetes Chronicles 11: Governing the Chaos

Unifying identity, policy, networking, and observability across clouds and edge

This article is part of the Azure Kubernetes Chronicles: Multi-Cluster Edition.
Read the series introduction here.

Governance pain points

By the time enterprises have stretched Kubernetes across Azure, AWS, and the edge, the real problem is no longer the technology itself. It is governance. What began as a strategy to increase resilience and satisfy compliance quickly turns into a patchwork of identity systems, policy frameworks, networking models, and monitoring stacks. The result is fragmentation. Platform teams are left trying to stitch together three or more worlds, often with tools that were never designed to coexist.

Identity is often the very first thing to be challenged or questioned. In Azure, AKS integrates tightly with Entra ID, providing enterprises with a direct mapping from organizational roles to Kubernetes permissions. In AWS, EKS relies on IAM, with its own syntax, semantics, and mapping mechanisms. At the edge, lightweight distributions like k3s often default to local certificates and kubeconfig files. For developers moving between clusters, the experience is inconsistent. For security teams, enforcing least privilege becomes a nightmare. What looks unified on a whiteboard quickly splinters in reality.

Policy follows a similar trajectory. Within Azure, administrators can use Azure Policy for Kubernetes to apply rules at scale across clusters managed by Fleet or Arc. In EKS, there is no AWS-native equivalent. Teams must rely on open-source tools such as OPA or Kyverno to enforce the same standards. At the edge, policy enforcement is often manually bolted on, if it is present at all. Compliance auditors inevitably notice the gaps. A Pod Security restriction applied in one place but absent in another is enough to trigger uncomfortable questions.

Networking is the most visible source of complexity. Azure relies on VNets, with AKS clusters often configured with Azure CNI or Cilium. AWS EKS uses VPC CNI and a different model for assigning IP addresses. At the edge, k3s defaults to flannel, though it can be reconfigured to run Calico or Cilium. The differences are not cosmetic; they affect service discovery, IP address management, and cross-cluster communication. Some enterprises attempt to standardize with service meshes, while others try global DNS overlays. Few are fully satisfied.

Observability completes the picture. In Azure, Container Insights and Azure Monitor provide a polished experience, but only for Azure-native clusters. In AWS, CloudWatch reigns, with its own dashboards and metrics model. At the edge, Prometheus and Grafana are the common choices, but they require self-management and rarely integrate cleanly with hyperscaler tools. The result is silos. Operations teams bounce between three or four monitoring stacks, none of which provide a true single pane of glass. Troubleshooting a cross-cloud outage becomes less a matter of engineering and more a matter of detective work.

These fractures are not theoretical. They show up in board reports, audit findings, and late-night incidents. They create operational drag and erode confidence in platform teams. And they push enterprises toward solutions that promise unification.

Possible directions for solutions

One option is Azure Arc, which extends Azure governance across AKS, EKS, GKE, and even edge clusters, such as k3s. With Arc, Entra ID can become the identity plane across all clusters. Azure Policy can apply consistently, regardless of where the cluster runs. Logs and metrics can flow back into Azure Monitor, giving a central view. Arc does not eliminate the complexity of multi-cloud, but it does normalize it into a governance model that auditors can understand.

Another option is Google Anthos, which was among the first to enter the cross-cloud management market. Anthos offers config sync, service mesh integration, and policy enforcement across GKE, AKS, and EKS. It is powerful, but it often feels Google-centric and comes with a cost structure that enterprises weigh carefully.

Google Anthos is a platform for managing applications across hybrid and multi-cloud environments, helping organizations modernize existing apps, develop new ones, and run them securely anywhere — whether on Google Cloud, on-premises, or other clouds like AWS and Azure. Built on Kubernetes and open-source tools such as Istio and Knative, Anthos provides a unified framework for managing containerized applications, policies, and security across various environments. Its main benefits include preventing vendor lock-in, enhancing operational efficiency through centralized management, and facilitating digital transformation by standardizing application deployment across different infrastructure types.

The third option is to go DIY (Do-It-Yourself) with open-source tooling. Service meshes, such as Istio or Linkerd, and policy engines, like OPA, can be combined with observability stacks like Prometheus and Grafana to form a cross-cloud framework. This approach offers the most independence, but it also requires the most expertise. Few enterprises are willing to carry that burden without a strong internal engineering culture.

In practice, enterprises often combine these approaches. Fleet may govern AKS clusters inside Azure. Arc may extend that governance to AWS and the edge. Service meshes may abstract away networking differences. Open-source observability may provide a neutral layer on top of it. The goal is not perfection, but coherence. A governance framework that is strong enough to withstand audits, resilient enough to survive outages, and simple enough for platform teams to operate.

Governance is not glamorous. It does not make headlines or attract the same enthusiasm as AI or serverless platforms. But in the world of multi-cluster Kubernetes, governance is what makes the difference between a fragile patchwork and a robust enterprise strategy. Without it, identity fragments, policies drift, networks split, and observability fails. With it, the chaos becomes manageable.

In the next episode of the Azure Kubernetes Chronicles, we will look beyond today’s fractures and into the future. We’ll explore how Microsoft might merge Fleet and Arc, how sovereign cloud initiatives in Europe could reshape strategies, and how the edge explosion will redefine what “multi-cluster” means by 2030.

Previous: Azure Kubernetes Chronicles 10: Kubernetes at the Edge with k3s Next up: Azure Kubernetes Chronicles 12: The Future of Multi-Cluster