Understand Kubernetes 5: Controller

Controllers in k8s assumes the same role and responsibility as the Controller in the classic Model-View-Controller(whereras the Model are the various API objects stored in the etcd) architecture. What's kind of unique about the controller in k8s is will constantly reconcile the system desired state to current state, not just a one time task.

Replicaset Controller

To make things real, we'll look at the source code of Replicaset Controller and see what exactly is a controller, who it will interact with, and how.
The core logic of Replicaset Controller is quite simple, as showing below:
func (rsc *ReplicaSetController) manageReplicas(filteredPods []*v1.Pod, rs *apps.ReplicaSet) error {
    diff := len(filteredPods) - int(*(rs.Spec.Replicas))
    if (diff < 0) {
        createPods( )
    } else if (diff > 0) {
        createPods( )
To create the Pod, it uses a KubeClient which talks to the API server.
func (r RealPodControl) createPods( )
    newPod, _ := r.KubeClient.CoreV1().Pods(namespace).Create(pod) 
Tracing further function Create(), it uses a nice builder patterner, to set up an HTTP request
func (c *pods) Create(pod *v1.Pod) (result *v1.Pod, err error) {
    result = &v1.Pod{}
    err = c.client.Post().
Upon calling Do, it will issue an HTTP post request and get the result.
func (r *Request) Do() Result {    
    var result Result
    err := r.request(func(req *http.Request, resp *http.Response) {
        result = r.transformResponse(resp, req)
    return result
That only cover one direction of the communication, from the controller to the API server.

How about the other direction?


A controller subscribe itself to the apiserver for the events it cares about.
A controller typical cares about two type of information: controller specific information and the core information regarding the Pods.
In k8s, the components used to notify the events are called Informer. FWIW, it is just an Oberser Pattern.
In the case of ReplicatSetController, When a replicatSet request is submitted, the API server will notify the replicatSetControll through appsinformers.ReplicaSetInformer. When a Pod gets created, the API server will notify the replicatSetControll using coreinformers.PodInformer.
See how a ReplicatSetController is initiated:
func startReplicaSetController(ctx ControllerContext) (bool, error) {
    go replicaset.NewReplicaSetController(
        ctx.InformerFactory.Apps().V1().ReplicaSets(), // appsinformers.ReplicaSetInformer
        ctx.InformerFactory.Core().V1().Pods(),        // coreinformers.PodInformer
    ).Run(int(ctx.ComponentConfig.ReplicaSetController.ConcurrentRSSyncs), ctx.Stop)
    return true, nil
And how ReplicatSetController is handling those events:
        AddFunc:    rsc.enqueueReplicaSet,
        UpdateFunc: rsc.updateRS,
        DeleteFunc: rsc.enqueueReplicaSet,

        AddFunc:    rsc.addPod,
        UpdateFunc: rsc.updatePod,
        DeleteFunc: rsc.deletePod,
Ok, this covers the direction from the API server to the controller.

But we still missing a one thing.

Workqueue, and worker

After being notified of the relevant events, a controller will push the events to an event queue; meanwhile, a poor worker is in a dead loop checking the queue and processing the event.

Cached & Shared Informer

We know that etcd provided the API to list and watch particular resources and each resource in k8s has its dedicated locations. With that, we have the things needed to implement an informer for a controller. However, there are two aspects we can optimize. First, instead of relaying everything to etcd, we can cache the information/event in the apiserver for better performance; Second, since different controls care about same set information, it makes sense those controllers can share an informer.
With that in mind, here is how currently a ReplicaSetInformer is created.

Controller Manager

kube-controller-manageris a daemon that bundles together all the built-in controllers for k8s. It provides a central place to register, initiate, and start the controllers.


We go through what a controller is and it interacts with the api sever and does the job.