Introduction: Cost Is an Outcome of Scheduling Decisions

Your AKS bill is calculated before Azure sends an invoice. Kubernetes translates pod specifications into resource requirements, including resource requests, affinity rules, topology constraints, DaemonSet overhead, and scaling limits, all of which impact the efficiency of pod placement on nodes.

This post’s main point is clear: durable optimization involves enhancing the entire chain, from workload demand to pod replicas, schedulable capacity, VM allocation, and cost attribution. Reducing node count without understanding this chain does not save money; it simply shifts costs into latency, evictions, throttling, and extended recovery periods.

Build a Cost and Capacity Model

There are four key numbers, seldom the same: Provisioned capacity is what the VM SKU shows; allocatable capacity is what’s available after the OS, kubelet, system pods, eviction thresholds, and DaemonSets take their part; requested capacity is what pods have reserved; and consumed capacity is what pods actually use. The scheduler assigns pods based on allocatable capacity and declared requests, not on average utilization.

To analyze costs effectively, monitor metrics such as node-pool hourly costs, allocatable CPU and memory, total pod requests, actual P50/P95 resource consumption, unschedulable pod durations, and workload throughput. Of all dashboard metrics, two ratios are most critical: request efficiency (usage ÷ requested resources) and allocation efficiency (requested ÷ allocatable resources). Both are essential because high request efficiency alone can mask inefficient bin-packing, especially when pod shapes do not align with the chosen VM SKU, resulting in wasted fractional CPU or memory on each node.

Treat Autoscaling as Interacting Control Loops

AKS typically operates four autoscalers simultaneously, each at a different layer and interval, without awareness of the others’ goals.

  • Horizontal Pod Autoscaler (HPA) changes replica count based on CPU, memory, or custom metrics.
  • KEDA (available as a managed AKS add-on) converts external event signals queue depth, Prometheus metrics, GPU queue wait time into replica demand, including scale-to-zero.
  • Vertical Pod Autoscaler (VPA) recommends, or on AKS 1.34+ applies in-place, per-pod resource requests.
  • Cluster Autoscaler (or Node Auto Provisioning) reacts only when pods can’t be scheduled, or nodes become removable.

Poor coordination leads to common failure modes: inflated requests cause early scale-out; missing requests skew HPA, risking unsafe pod density; an unstable HPA without proper Cluster Autoscaler stabilization causes node churn; and running VPA in Auto mode with HPA on the same signal results in conflicting decisions.

# HPA scales on request-based CPU utilization
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: checkout-api
  namespace: payments
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 30
    scaleDown:
      stabilizationWindowSeconds: 300
---
# VPA in recommendation-only mode: safe to run alongside HPA
apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: checkout-api-vpa
  namespace: payments
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: checkout-api
  updatePolicy:
    updateMode: "Off"   # observe recommendations before enabling Auto

Scale-up should aim for meeting application latency goals; scale-down must consider startup times, pod disruption budgets, termination grace periods, and rescheduling headroom. Provide production pools with a resilient minimum, while letting asynchronous pools scale down to zero. Adjust Cluster Autoscaler profiles based on workload classes, as a single, aggressive profile across the cluster can ultimately negatively impact a workload that wasn’t properly tuned.

Improve Scheduler Bin-Packing Through Rightsizing

Rightsizing isn’t just about “lowering requests.” CPU resources are compressible and throttled when there’s contention, but memory isn’t. Memory pressure leads to eviction or OOM kills. It’s important to set requests independently for each resource and container, including service-mesh sidecars and log shippers, which are often left at default settings and can silently dominate a pod’s resource footprint.

Base your measurements on historical data from Azure Managed Prometheus or Container Insights, rather than relying on a single kubectl top sample. Include data on peaks, deployments, cold starts, GC pauses, scheduled jobs, and seasonal traffic variations. Set VPA to updateMode: “Off” initially, allowing you to review its lower bound, target, and upper bound before applying changes to production manifests.

The shape of a pod is as important as overall utilization. A workload may seem efficient overall but still leave resources unused because CPU-heavy and memory-heavy pods can’t co-locate on the same node. Consider requested pod sizes against candidate VM CPU:memory ratios, maximum pods per node, DaemonSet overhead, and zone placement. Rightsizing pods and selecting suitable node SKUs should be viewed as a single, integrated task.

Design Node Pools Around Workload Constraints

Cluster Autoscaler assesses each node pool separately, considering its specific labels, taints, zones, and VM types. Having too many small pools fragments capacity and reduces utilization, while a single heterogeneous pool can weaken isolation and lead to unpredictable scaling.

A practical default configuration involves a small dedicated system pool for essential AKS components, along with a limited number of workload pools based on valid scheduling needs such as taints and tolerations, node affinity, topology spread constraints, persistent-volume zone affinity, GPU requirements, or CPU architecture. Setting minimum and maximum counts for each pool isn’t only a cost control measure; it also influences resilience.

Node Auto Provisioning (NAP), AKS’s managed Karpenter integration, became generally available in 2025 and is now the preferred default for automatically matching node configurations to pending pod demands. It selects VM size and family based on current pending pods rather than a pre-established pool. However, AKS Standard with manually configured node pools still offers the best option when teams require precise, auditable control over node-pool behavior.

Engineer Spot Capacity as an Interruption Domain

Treat Spot nodes as unreliable capacity that is intentionally isolated and consumed, not as direct substitutes for regular nodes. Typically, a design includes a standard system pool, a reliable baseline pool scaled for steady workload, and an autoscaled Spot pool handling interruptible demand.

# AKS automatically applies this label and taint to every node in a Spot pool 
# shown here for reference, not something you create by hand.
apiVersion: v1
kind: Node
metadata:
  labels:
    kubernetes.azure.com/priority: spot
spec:
  taints:
    - key: kubernetes.azure.com/scalesetpriority
      value: spot
      effect: NoSchedule
---
# Workload tolerates Spot but prefers it, falls back to regular nodes on eviction
apiVersion: apps/v1
kind: Deployment
metadata:
  name: batch-worker
  labels:
    app: batch-worker
spec:
  replicas: 6
  selector:
    matchLabels:
      app: batch-worker
  template:
    metadata:
      labels:
        app: batch-worker
    spec:
      tolerations:
        - key: kubernetes.azure.com/scalesetpriority
          operator: Equal
          value: spot
          effect: NoSchedule
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              preference:
                matchExpressions:
                  - key: kubernetes.azure.com/priority
                    operator: In
                    values: ["spot"]
      terminationGracePeriodSeconds: 30
      containers:
        - name: worker
          image: myregistry.azurecr.io/batch-worker:1.4.0
          resources:
            requests:
              cpu: "500m"
              memory: "512Mi"
            limits:
              memory: "512Mi"

The key difference is whether you tolerate Spot or require it. Using a strict nodeSelector for Spot causes evicted pods to stay pending when Spot capacity disappears. In contrast, a preferred affinity allows the scheduler to fall back to regular nodes at a higher cost. Make sure to include this fallback expense in your budget model. It’s actual spending, not just a rounding error.

Design the application contract explicitly: multiple replicas, idempotent processing, checkpoints, a short shutdown path, correct SIGTERM handling, queue visibility timeouts, and bounded retries. Pod disruption budgets help with voluntary disruptions but do not protect against Spot’s 30-second eviction notice. Monitor eviction frequency, recovery time, pending-pod time, retry volume, and fallback-to-regular consumption; that data is the only proof that discounted compute is actually producing net savings.

Tune Scale-Down Without Breaking Availability

Low utilization doesn’t necessarily justify removing a node. Factors such as local storage limitations, strict disruption budgets, singleton replicas, system pods, and limited capacity at other destinations in the cluster can prevent consolidation.

Cluster Autoscaler offers settings for scale-down delay, unneeded-time threshold, utilization threshold, and graceful termination. Using aggressive values can reduce idle Node minutes but may lead to more pod movements, cold starts, and operational disturbances. Before tuning, review the autoscaler’s events, history of unschedulable pods, disruption budgets, and node utilization trends. Always validate adjustments in a development or batch environment first, especially before applying changes to latency-sensitive production.

Automate FinOps with Kubernetes-Native Metadata

Azure billing shows the infrastructure costs, while Kubernetes telemetry identifies the workload responsible. Combine dimensions such as subscription, cluster, node-pool, namespace, controller, and labels to create actionable allocation reports for engineers.

Implement technical controls as standards rather than informal practices: mandate resource requests, assign ownership labels, track environment metadata, specify approved VM types, and define explicit Spot eligibility. Also, identify idle namespaces, abandoned load balancers, unattached disks, excessive log ingestion, and requests that significantly deviate from actual usage.

Report the cost per request, job, tenant, or business transaction, not merely the cost per cluster. Utilize anomaly detection to link spending spikes with deployments, replica adjustments, node-pool expansions, or telemetry configuration modifications. Use reservations or savings plans to cover baseline measured usage, allowing autoscaling and Spot instances to handle the variable expenses.

Conclusion: Optimize the Feedback System

Durable AKS savings are achieved through precise requests, compatible pod and node configurations, coordinated autoscalers, interruption-aware applications, and integrated cost telemetry into engineering choices. The aim isn’t to maximize utilization at all times but to maintain a controlled system that fulfills service levels with the least necessary paid capacity.