How does autoscaling actually work, and how do you tune it?

A scaling policy watches a metric and adjusts capacity toward a target, so target tracking keeps average CPU or requests per instance near a set point. Tuning means picking a metric that genuinely reflects load, allowing for instance warm up time before new capacity counts, scaling out quickly and in slowly, and setting a minimum that keeps every availability zone covered. Add scheduled capacity for predictable spikes, since reactive scaling always lags.

It is a feedback loop. You give it a metric and a target, it compares and adds or removes capacity to close the gap. The first decision is the metric, and CPU is the default that is often wrong, because an IO bound service can be at thirty percent CPU while its connection pool is exhausted. Requests per instance or queue depth is usually a truer signal of load. The second thing is lag, and this is where people get hurt. If an instance takes three minutes to boot and pass health checks, then a spike arriving in sixty seconds is over before help shows up, so I set a warm up period so new instances are not counted before they are ready, and I pre scale on a schedule for predictable patterns like a nine in the morning login rush. I make scale out aggressive and scale in gentle, because being briefly over provisioned is much cheaper than an outage, and flapping is worse than either. And scale in needs connection draining, otherwise you terminate an instance mid request and users see errors that look like a bug.