Scale-to-zero

An HTTP service can drop to 0 replicas when nothing calls it and wake again on the next request. While it sleeps you pay nothing for CPU and memory.

This page assumes you already have a working HTTPRoute, see Gateway API.

Turning it on

You opt in per route. Add the annotations to the HTTPRoute you already have, and point its backendRefs at the interceptor instead of at your own Service:

yaml

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: myapp
  annotations:
    itsh.dev/scale-to-zero: "myapp"
    itsh.dev/scale-to-zero-max-replicas: "5"
    itsh.dev/scale-to-zero-target-concurrency: "10"
    itsh.dev/scale-to-zero-idle: "300s"
spec:
  parentRefs:
    - name: default
      namespace: nginx-gateway
  hostnames:
    - "myapp.<your-namespace>.itsh.dev"
  rules:
    - backendRefs:
        - name: keda-add-ons-http-interceptor-proxy
          namespace: keda
          port: 8080

That backendRefs block is the only change to your own manifests. The scaler wiring, the rate limits and the network policy are created for you, and removed again when you delete the route. To opt out, drop the annotations and point backendRefs back at your Service.

The annotation value is one name used twice, for the workload and for the Service in front of it, so those two need the same name.

Annotations

Annotation	Default	Accepted	What it does
`itsh.dev/scale-to-zero`	none	a workload name	Turns it on. Names both the workload to sleep and the Service in front of it
`itsh.dev/scale-to-zero-max-replicas`	`3`	a positive integer	Upper bound when scaling above zero
`itsh.dev/scale-to-zero-target-concurrency`	`100`	`1` to `1000`	In-flight requests per replica before another starts
`itsh.dev/scale-to-zero-idle`	`300s`	`300s` or more	How long with no traffic before it sleeps
`itsh.dev/scale-to-zero-cold-start`	hold	`placeholder`	Answer immediately with a 503 instead of holding the connection

The default concurrency of 100 is too high for most applications. At that setting a single replica absorbs a hundred simultaneous requests before anything else starts, so the application gets slower under load rather than scaling out. Set it to roughly what one replica serves comfortably.

What you actually pay

The meter bills the resource requests of Running pods, so a sleeping workload costs nothing in CPU and memory.

It does not make your invoice zero. Volumes stay allocated while the pods sleep, so PVC capacity keeps billing at the normal rate, and the base fee continues. A namespace with a 20 GiB volume that sleeps all month still pays for the base fee plus that volume. The portal shows this as a floor figure: what the namespace costs if everything sleeps from here on.

Cold starts

The first request after a sleep is held open while the pod starts and then answered normally, so the caller sees one slow request rather than an error. A small service with a cached image wakes in about 2 to 3 seconds. A heavier runtime takes as long as it normally takes to boot.

Your readiness probe is the biggest single lever on that. No traffic is sent until the probe passes, so initialDelaySeconds: 5, periodSeconds: 10 means the earliest possible wake is 5 seconds and the next chance is 15. If wake latency matters to you, use initialDelaySeconds: 0 and periodSeconds: 1.

If the cluster has to add a node first, the wake takes over two minutes. That is longer than we hold the connection, so the caller gets an error page and has to retry. It is rare, but it is a failed request rather than a slow one.

Setting itsh.dev/scale-to-zero-cold-start: "placeholder" answers immediately with a 503 and a Retry-After instead of holding the connection. That is friendlier for a browser hitting a web UI and wrong for everything else: it breaks POSTs and any non-interactive caller.

How it scales above zero

Scale-to-zero is not an on/off switch. The workload scales between 0 and itsh.dev/scale-to-zero-max-replicas on in-flight request concurrency: once a replica is holding more requests than the target, another one starts.

Request concurrency is the only signal. There is no CPU, memory or queue-depth scaling here, so if your load is not proportional to concurrent HTTP requests, scale-to-zero is the wrong tool and a plain HPA or KEDA ScaledObject is the right one.

Not together with another autoscaler

Scale-to-zero already puts an autoscaler on the workload, so it cannot share one with an HPA or a ScaledObject of your own. Pick one per workload.

Adding an HPA afterwards is the case to watch, because nothing rejects it. You end up with two autoscalers writing the replica count of one Deployment, and an HPA with minReplicas: 1 or higher holds the workload awake permanently, so it never sleeps and you keep paying for CPU and memory. Both cases are reported on the route, see Scaling does not do what you expect.

Not a good fit for

Background workers and queue consumers. Nothing calls them over HTTP, so nothing wakes them. Use KEDA with a queue trigger instead.
WebSocket and SSE. Long-lived connections do not survive a scale-down.
Anything needing a warm cache. In-process caches, opcache and JIT state die with the pod, so requests stay slow for a while after a wake.
Cron-driven work. A CronJob already costs nothing while it is not running.
gRPC. Not supported on scale-to-zero routes; run those always-on.
Anything where a multi-second first request is unacceptable, such as payment callbacks or webhook receivers with a short sender timeout.

On the free tier

Sleeping is not inactivity. The 30-day suspension counts traffic, not running pods, so a namespace that sleeps all month and serves the occasional request stays active. Cold starts are slower here, because 200m of CPU is not much to start an application with.

What's next

Autoscaling for the other scalers
Billing and quotas for what the meter counts

Scale-to-zero ​

Turning it on ​

Annotations ​

What you actually pay ​

Cold starts ​

How it scales above zero ​

Not together with another autoscaler ​

Not a good fit for ​

On the free tier ​

What's next ​