Skip to main content

Understand Kubernetes 5: Controller

Controllers in k8s assumes the same role and responsibility as the Controller in the classic Model-View-Controller(whereras the Model are the various API objects stored in the etcd) architecture. What's kind of unique about the controller in k8s is will constantly reconcile the system desired state to current state, not just a one time task.

Replicaset Controller

To make things real, we'll look at the source code of Replicaset Controller and see what exactly is a controller, who it will interact with, and how.
The core logic of Replicaset Controller is quite simple, as showing below:
func (rsc *ReplicaSetController) manageReplicas(filteredPods []*v1.Pod, rs *apps.ReplicaSet) error {
    diff := len(filteredPods) - int(*(rs.Spec.Replicas))
    if (diff < 0) {
        createPods( )
    } else if (diff > 0) {
        createPods( )
To create the Pod, it uses a KubeClient which talks to the API server.
func (r RealPodControl) createPods( )
    newPod, _ := r.KubeClient.CoreV1().Pods(namespace).Create(pod) 
Tracing further function Create(), it uses a nice builder patterner, to set up an HTTP request
func (c *pods) Create(pod *v1.Pod) (result *v1.Pod, err error) {
    result = &v1.Pod{}
    err = c.client.Post().
Upon calling Do, it will issue an HTTP post request and get the result.
func (r *Request) Do() Result {    
    var result Result
    err := r.request(func(req *http.Request, resp *http.Response) {
        result = r.transformResponse(resp, req)
    return result
That only cover one direction of the communication, from the controller to the API server.

How about the other direction?


A controller subscribe itself to the apiserver for the events it cares about.
A controller typical cares about two type of information: controller specific information and the core information regarding the Pods.
In k8s, the components used to notify the events are called Informer. FWIW, it is just an Oberser Pattern.
In the case of ReplicatSetController, When a replicatSet request is submitted, the API server will notify the replicatSetControll through appsinformers.ReplicaSetInformer. When a Pod gets created, the API server will notify the replicatSetControll using coreinformers.PodInformer.
See how a ReplicatSetController is initiated:
func startReplicaSetController(ctx ControllerContext) (bool, error) {
    go replicaset.NewReplicaSetController(
        ctx.InformerFactory.Apps().V1().ReplicaSets(), // appsinformers.ReplicaSetInformer
        ctx.InformerFactory.Core().V1().Pods(),        // coreinformers.PodInformer
    ).Run(int(ctx.ComponentConfig.ReplicaSetController.ConcurrentRSSyncs), ctx.Stop)
    return true, nil
And how ReplicatSetController is handling those events:
        AddFunc:    rsc.enqueueReplicaSet,
        UpdateFunc: rsc.updateRS,
        DeleteFunc: rsc.enqueueReplicaSet,

        AddFunc:    rsc.addPod,
        UpdateFunc: rsc.updatePod,
        DeleteFunc: rsc.deletePod,
Ok, this covers the direction from the API server to the controller.

But we still missing a one thing.

Workqueue, and worker

After being notified of the relevant events, a controller will push the events to an event queue; meanwhile, a poor worker is in a dead loop checking the queue and processing the event.

Cached & Shared Informer

We know that etcd provided the API to list and watch particular resources and each resource in k8s has its dedicated locations. With that, we have the things needed to implement an informer for a controller. However, there are two aspects we can optimize. First, instead of relaying everything to etcd, we can cache the information/event in the apiserver for better performance; Second, since different controls care about same set information, it makes sense those controllers can share an informer.
With that in mind, here is how currently a ReplicaSetInformer is created.

Controller Manager

kube-controller-manageris a daemon that bundles together all the built-in controllers for k8s. It provides a central place to register, initiate, and start the controllers.


We go through what a controller is and it interacts with the api sever and does the job.

Popular posts from this blog

Understand Container - Index Page

This is an index page to a series of 8 articles on container implementation. OCI Specification Linux Namespaces Linux Cgroup Linux Capability Mount and Jail User and Root Network and Hook Network and CNI
This page has a very good page view after being created. Then I was thinking if anyone would be interested in a more polished, extended, and easier to read version.
So I started a book called "understand container". Let me know if you will be interested in the work by subscribing here and I'll send the first draft version which will include all the 8 articles here. The free subscription will end at 31th, Oct, 2018.

* Remember to click "Share email with author (optional)", so that I can send the book to your email directly. 


Android Camera2 API Explained

Compared with the old camera API, the Camera2 API introduced in the L is a lot more complex: more than ten classes are involved, calls (almost always) are asynchronized, plus lots of capture controls and meta data that you feel confused about.

Understand Container: OCI Specification

OCI is the industry collaborated effort to define an open containers specifications regarding container format and runtime - that is the official tone and is true. The history of how it comes to where it stands today from the initial disagreement is a very interesting story or case study regarding open source business model and competition.

But past is past, nowadays, OCI is non-argumentable THE container standard, IMO, as we'll see later in the article it is adopted by most of the mainstream container implementation, including docker, and container orchestration system, such as kubernetes, Plus, it is particularly helpful to anyone trying to understand how the containers works internally. Open source code are awesome but it is double awesome with high quality documentation!