user@cgh.mx:~$ cat /content/posts/kubernetes-1-36-sharded-list-watch-explained.txt

Kubernetes 1.36 makes large-cluster watches smarter

Kubernetes 1.36 makes large-cluster watches smarter

Kubernetes 1.36 introduces an alpha feature called server-side sharded list and watch. That name sounds internal, but the problem it addresses is very real for large clusters: horizontally scaled controllers can waste a lot of CPU, memory, and network bandwidth processing events they will immediately discard.

This feature does not matter much for small clusters. It matters when the number of objects is high, the controllers are scaled out, and every replica is still receiving the full event stream from the API server.

The problem: every replica sees too much

Kubernetes controllers commonly use the list-then-watch pattern. They list the current state of resources, then watch for changes so they can keep their local cache in sync.

That model works well, but it becomes expensive at scale. If a controller has multiple replicas and each replica is responsible for only part of the workload, the API server may still send every replica the full stream of objects and events. Each replica then filters locally and discards what it does not own.

The result is waste:

  • repeated network traffic
  • repeated deserialization work
  • higher CPU and memory usage per replica
  • API server pressure that grows with controller replicas
  • limited benefit from horizontal scaling

In other words, adding more replicas may help distribute business logic, but it does not necessarily reduce the cost of watching Kubernetes resources.

What server-side sharding changes

Server-side sharded list and watch moves part of that filtering into the API server.

Instead of every controller replica receiving the full collection, each replica can request a specific shard. Kubernetes uses a shardSelector in ListOptions with a hash range expression. The API server evaluates the selector and sends only the objects whose metadata hash falls into that replica’s assigned range.

For Kubernetes 1.36, the feature supports hashing fields such as object.metadata.uid and object.metadata.namespace. The API server uses a deterministic 64-bit FNV-1a hash so the same object maps consistently to the same shard range.

The important part is simple: each replica receives less irrelevant data.

Why this matters for operators

This is an API machinery feature, but operators should care because it targets one of the hidden costs of large Kubernetes clusters.

In a large environment, controllers that watch high-cardinality resources like Pods can become expensive even when they are well-written. If every replica receives everything, the control plane pays a repeated cost. Server-side sharding gives controller authors a way to reduce that cost closer to the source.

Potential benefits include:

  • lower network traffic from the API server to controller replicas
  • less wasted CPU on each replica
  • better scaling behavior for horizontally sharded controllers
  • improved efficiency for high-cardinality resources
  • cleaner alignment between controller ownership and API traffic

This is especially relevant for projects that already shard work client-side, because they may be able to stop receiving data they were only going to discard.

The alpha warning matters

This feature is alpha in Kubernetes 1.36 and disabled by default. That is important.

Alpha means it is not something to casually depend on for production behavior unless you are intentionally testing it and understand the risk. It requires enabling the ShardedListAndWatch feature gate on the API server, and clients need to handle fallback cases.

The Kubernetes documentation notes that responses can include shardInfo metadata when the server honors the selector. If that information is absent, the client should assume the server did not apply the shard and be ready to handle the full result set.

That fallback requirement is not a minor detail. Controller authors cannot assume every cluster they run against will support the feature.

Who should pay attention now

Most everyday Kubernetes users do not need to change anything today.

The people who should pay attention are:

  • controller authors
  • platform teams running very large clusters
  • teams operating high-cardinality resources
  • maintainers of observability or state-reporting controllers
  • engineers already using client-side sharding patterns

For those groups, this is worth testing early because it could influence future controller architecture.

Why this matters beyond one release

Kubernetes keeps improving scalability not only through big visible features, but through lower-level changes that reduce control plane waste. Server-side sharded list and watch fits that pattern.

It recognizes that large clusters are not just β€œsmall clusters with more nodes.” At scale, repeated streams, duplicated processing, and inefficient watches become real operational costs.

If this feature matures, it could make horizontally scaled controllers more efficient and make Kubernetes friendlier to very large environments.

The practical takeaway

Kubernetes 1.36 server-side sharded list and watch is not a feature most users will touch directly today. It is an alpha building block for controller authors and large-cluster operators.

The practical takeaway is this: Kubernetes is giving controllers a way to ask the API server for only the slice of the world they are responsible for. If it matures, that could make large clusters less wasteful and easier to scale.

Sources

user@cgh.mx:~$ echo "End of file."

Leave a Reply

Your email address will not be published. Required fields are marked *