Kubernetes Autoscaling and Capacity Planning

Autoscaling is two feedback loops

Horizontal Pod Autoscaler (HPA) changes the number of pods a Deployment wants. A node autoscaler or provisioner changes the number of nodes that can run those pods. They do not share a control loop, a latency, or a failure mode. Treating “the cluster scales” as one switch hides why a rollout stalls or why the first request after a spike is slow.

This guide uses a hypothetical three-replica API. Numbers are configuration examples, not measured production results.

Requests are the contract with the scheduler

CPU and memory requests are what the scheduler uses to place pods. Limits are a throttle or a kill. An HPA that targets CPU utilization computes usage / request. If requests are missing or fictional, the utilization number is fiction.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: example-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: example-api
  template:
    metadata:
      labels:
        app: example-api
    spec:
      containers:
        - name: api
          image: example.local/api@sha256:7e429bc1deadbeef
          resources:
            requests:
              cpu: "250m"
              memory: "256Mi"
            limits:
              memory: "512Mi"
          readinessProbe:
            httpGet:
              path: /readyz
              port: 8080

Set requests from observed usage and the peak the service must absorb, including a rolling update. maxUnavailable: 0 with maxSurge: 1 needs room for a fourth pod. If every node is packed to the request sum, the new pod Pending and the rollout waits.

Memory limits without matching requests can still OOMKill. CPU limits create throttling that looks like application slowness. Neither is required to make HPA work; requests are.

HPA targets a signal, not a business outcome

A typical CPU HPA:

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: example-api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: example-api
  minReplicas: 3
  maxReplicas: 12
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300

This says: keep average CPU near 70% of request, never below three replicas, never above twelve. It does not say “p95 latency stays under 200ms.” If the bottleneck is a database, more API pods can make the outage worse.

Custom or external metrics (request rate, queue depth) are often a better target for request-driven services. They require a metrics pipeline you trust and a target you can explain. HPA will oscillate if the signal is noisy and the scale-down window is short.

Vertical Pod Autoscaler can recommend or apply request changes. Applying VPA and HPA on the same CPU metric without a split (HPA on custom metrics, VPA on resources) fights itself. Recommendation-only mode is the safer first use.

Cluster capacity is a different delay

When HPA raises replicas and no node has room, pods stay Pending. A cluster autoscaler needs minutes to join a node; image pulls add more. Karpenter-style provisioners can be faster and still fail on quotas, subnet IP exhaustion, or unavailable instance types.

Capacity planning therefore has a floor:

• minReplicas should cover expected baseline and a rolling update or a single-node loss, whichever is larger for that service.
• Node headroom should cover the surge of the largest Deployment you roll during peak.
• Scale-to-zero is a product decision. Cold start, connection warmup, and dependency timeouts belong in the SLO conversation, not only the bill.

Security and noisy neighbors

A namespace without quota can starve the node. A service account that can create HPAs with maxReplicas: 1000 is a denial-of-wallet and a denial-of-service. Bound max replicas, resource quotas, and who can edit autoscalers.

Autoscaling does not replace network policy or identity. A scaled-out compromised workload is a larger compromised workload.

Trade-offs to state explicitly

More replicas improve parallelism and increase chatty downstream load. Higher utilization targets save CPU and reduce burst headroom. Spot or interruptible nodes lower unit cost and add interruption as a planned event.

None of these is a universal rule. Record the failure the unused capacity exists to absorb. If you cannot name that failure, the capacity may be waste. If you can, cutting it is a reliability change, not a FinOps win.

Troubleshooting a scale event

1. Is the metric present? Missing CPU metrics usually mean missing requests or a metrics adapter gap.
2. Is the new pod Scheduled? If Pending, read the event: CPU, memory, affinity, or IP.
3. Did the new pod become Ready? Probe failures look like “autoscaling does not work.”
4. Did user-facing SLIs improve? If not, the constraint is elsewhere—often data stores or an external API.

Continue with related material

Zero-downtime Kubernetes deployments explains surge and readiness. Observability with metrics, logs, and traces covers the SLIs HPA cannot see. Platform Engineering and Cloud Cost Optimization place this in platform defaults and reliability spend. The autoscaling lab shows replica state changes on a hypothetical cluster.