Setting up Kubernetes cluster for container orchestration
I have started playing around with docker containers lately and was looking for a solution for container orchestration. I stumbled upon Kubernetes and started exploring it and I must say that it’s surely worth a try.
Kubernetes as per website is container orchestration at planet scale. The statement is so true when you understand that Kubernetes is derived from the same architecture used by Google for managing their containers ( which is a massive billion per week !!! )
I couldn’t help but try it out. But one thing is that the documentation steps for installation were quite intimidating and there are several methods to do the same stuff. It was quite confusing for someone starting new.
I have managed to setup Kubernetes cluster in my local system ( running multiple VM’s of Ubuntu Server 16.04 ) and thought that I should share and document the setup information.
Please note that I will not be giving the details on basic components like Pods, services and other keywords of Kubernetes. They are quite out of scope for this post and the best place to get started with them would be the official documentation at https://kubernetes.io/docs/getting-started-guides/
So here it goes. We are going to cover the following in this post
- Installing docker and Kubernetes
- The configuration of the master node.
- The configuration of a child node ( minion ) and joining of a node to cluster.
- Starting the Kubernetes dashboard
- Resetting the cluster
Please note that I have the setup on a couple of VM’s running Ubuntu servers ( which are connected to each other using static IPs ). You can choose any Linux system and use the same commands. Only the commands to do the installation may change depending on the package manager of the respective distro.
Installing docker and Kubernetes
First of all, we need to make sure that docker ( for container management ) and Kubernetes components are installed in the host OS.
Please make yourself as sudo using the command
Run the following in the terminal:
$ apt-get update && apt-get install -y apt-transport-https $
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add - $
cat < /etc/apt/sources.list.d/kubernetes.list deb http://apt.kubernetes.io/ kubernetes-xenial main EOF $ apt-get update $ apt-get install -y docker.io $ apt-get install -y
kubelet kubeadm kubectl kubernetes-cni
The services for docker and Kubernetes are started automatically here. In case you want to start or stop the services, you can use the following commands
$ systemctl start/stop docker.service
$ systemctl start/stop kubelet
cat < /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=http://yum.kubernetes.io/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg EOF
$ setenforce 0 $ yum install -y docker kubelet kubeadm kubectl kubernetes-cni $ systemctl enable docker && systemctl start docker $ systemctl enable kubelet && systemctl start kubelet
Once this is done, we have installed docker, kubelet service, kubeadm tool, kubectl tool and networking interfaces support for Kubernetes. The services are also started. Now we can start the setting up of a cluster for Kubernetes.
The configuration of the master node.
If there are multiple systems, you need to select one as Master node and need to issue the following commands. Make sure that you are sudo or you have run sudo su command
$ kubeadm init --pod-network-cidr 10.244.0.0/16 --api-advertise-addresses 192.168.56.21
Following is the explanation.
- The kubeadm in the tool for administration of the Kubernetes cluster. We are requesting it to initiate a new master node.
- The –pod-network-cidr 10.244.0.0/16 is needed since we are going to use a addon called Flannel for managing the networking of pods running in different nodes. You may skip this if you want to use a different network layer manager.
- The –api-advertise-addresses 192.168.56.21 is asking the kubeadm to bind the kubernetes to advertise the address as the one specified. By default, the kubeadm will use the one with a default gateway. Since I have static IP specified to manage the interconnection between VM’s, I wanted to use that for the communication. If you have the default gateway configured, you may skip this configuration.
Once the command is executed, it should provide a response as below.
Please take a note of the entry marked in green. This is what you will be executing from the worker nodes ( minion ) for joining this cluster. As you can see, the IP address is the one I request in the advertise addresses flag. In this way, my worker node ( which also a VM ) can communicate as they are in the same subnet. Again, if you have the machines in the same network, there is no need to for passing the –api-advertise-addresses
By default, the master node is not used for deploying pods. Only the worker nodes are used. But if you want to use the master node also for deploying pods, you can execute the following command
$ kubectl taint nodes --all dedicated-
Installing a pod network
Once the master node is up, we need to have a pod network installed. There are different options available and in my case, I am using Flannel. To install Flannel network, execute the following
$ kubectl apply -f https://github.com/coreos/flannel/blob/master/Documentation/kube-flannel.yml
For other network policy add-ons, you can refer here
At this level, we can see the pods that are currently active using the following command
$ kubectl get pods --all-namespaces
You will get a response with the list of all the pods currently running in the cluster. As we don’t have any workers yet, the pods are all on the master node.
The configuration of a worker node ( minion ) and joining of a node to cluster.
The configuration for a worker node is pretty straightforward. We just need to execute the join command received on the kubeadm init command response on the respective node.
Need to make sure that the setup commands ( Installing of docker and kubernetes) are run on the worker nodes as well. But the kubeadm init need to be done only in the master node.
Run the command to run for joining the node ( Please use the one you received in your kubeadm init response )
$ kubeadm join --token=b68c42.b6bc83b6a136e3cf 192.168.56.21
You should get a response similar to the below screenshot.
Now to get the nodes in the cluster, we can run the following command in the master node.
$ kubectl get nodes
It will respond with the list of nodes currently in the cluster
If you want to get the information about a particular node, you may use the following syntax of the command
$ kubectl describe node
Starting the Kubernetes dashboard
The Kubernetes dashboard is an UI that provides visual information of the Kubernetes cluster and also acts as an interface for creating and managing Pods, deployments, services, and replication controllers.
Let’s see how to add the dashboard to our Kubernetes cluster. Issue the following command in the master node.
$ kubectl create -f https://rawgit.com/kubernetes/dashboard/master/src/deploy/kubernetes-dashboard.yaml
This should give the following response
deployment "kubernetes-dashboard" created service "kubernetes-dashboard" created
The response means that the kubernetes-dashboard is deployed and a service is created. A service is an exposed deployment which allows the deployed component to be accessed. Kubernetes will expose the service over a specific port and we can find the port by running the following command
$ kubectl describe service kubernetes-dashboard -n kube-system
We are requesting Kubernetes to describe the details of the specified service. The -n kube-system is the namespace where the kubernetes-dashboard resides in the cluster. This will generate the following output.
We need to get the value of the highlighted entry, NodePort. For me it’s 30046, it could be different for you. We can access the dashboard by putting the following URL in the browser
When the service is deployed and if there is NodePort specified, Kubernetes will bind the service to the specified NodePort in all the worker nodes. So you can access the service by pointing to any worker node and the NodePort
Once you are in the dashboard, you can see the Pods, services, deployments etc. You can also initiate a deployment from the UI.
Resetting the cluster
Many times during the trial and error phase, I had to revert all the configuration and start from scratch. In case you need to completely clear the Kubernetes setup and start from the beginning, do the following on master followed by each worker node.
$ kubeadm reset $ systemctl stop kubelet $ systemctl stop docker $ rm -rf /var/lib/cni/ $ rm -rf /var/lib/kubelet/* $ rm -rf /etc/cni/ $ ifconfig cni0 down $ ifconfig flannel.1 down $ ifconfig docker0 down
After this is completed, you can start all over from the scratch. Please note that you need to run this as root and also this will terminate all the pods that are currently running.
I am planning to do another post on deploying a spring-boot application to a kubernetes cluster and managing replications and scaling. I would love to hear from you about Kubernetes. Please post your comments and queries in the comments section.