Cutting our EKS bill by 60% without touching autoscaling

Autoscaling being "on" usually isn't the problem — the problem is what it is allowed to scale to. Three things to check, in order of how often they're the culprit:

1. Node group instance sizing

If your node group is m5.4xlarge and your pods request 500m CPU, you are paying for a lot of empty node.

kubectl get nodes -o custom-columns=\
NAME:.metadata.name,\
CPU:.status.allocatable.cpu,\
MEM:.status.allocatable.memory

2. Requests vs. actual usage

kubectl top pods --all-namespaces --sort-by=cpu

Compare that against resources.requests. A 4x gap is common and it directly multiplies your node count.

3. Spot coverage

Stateless services on on-demand instances is the single most expensive default in Kubernetes.

Two changes. Split the node group so spot carries the stateless load:

resource "aws_eks_node_group" "stateless" {
  cluster_name    = aws_eks_cluster.main.name
  node_group_name = "stateless-spot"
  capacity_type   = "SPOT"

  instance_types = ["m5.large", "m5a.large", "m6i.large"]

  scaling_config {
    desired_size = 4
    min_size     = 2
    max_size     = 30
  }

  taint {
    key    = "workload"
    value  = "stateless"
    effect = "NO_SCHEDULE"
  }
}

Multiple instance types matter — a single type means a single spot pool, and a single pool means correlated interruptions.

Then right-size the requests. Do this second, after the spot pool exists, so you aren't changing two variables at once.

That tracks with the ~70% spot discount on those pools plus the request right-sizing compounding into fewer nodes. Worth adding a PodDisruptionBudget per stateless deployment before you push spot coverage any higher — it's the thing that keeps a bad interruption window from becoming an outage.