Kubernetes

Basics

• Kubernetes Master Processes:

- kube-apiserver
- kube-controller-manager
- kube-scheduler
- etcd: key-value pair store to store tags or labels for easy Management

• Nodes Component Processes:

- kubelet: talks to kubernetes master
- kube-proxy:
- DNS addon
- UI addon
- fluentd: for log collection
- Supervisord

• Kubernetes Concepts:

- Container Image: Docker container Image with Application code
- Pod: Set of containers sharing network namespace & Local volumes, co-scheduled on one machine. Mortal, has IP, Label
- Deployment: Specify how many replicas of pod should be run in a cluster. Has label.
- Service: names things in DNS, Gets Virtual IP. Routes based on labels. Two types:
   ClusterIP for Internal services
   NodePort for publishing to outside.

• Do not care about PODS, Just care about Deployments in Kubernetes.

• Kubernetes Networking Requirements:

- All contianers should communicate without NAT
- All Nodes can communicate without NAT
- IP that a container sees itselg as is the same IP that others see it.

eth0<------->docker0<--------->veth0
10.100.0.2  docker bridge    container1
            172.17.0.1       172.17.0.2

Overlay network e.g. Flannel:

eth0<--------->cbr0<--------->veth0
10.100.0.2                   container1
             172.17.0.1      172.17.0.2

• Pause State:

• All networking in the Pod lies in the overlay itself.

• Kube-Proxy:

- Implemented by Iptables
- Handles Inter-Node(nodes on different Hosts) communication => East-West communication

• YAML output:

kubectl get pod <name> -o yaml

• Shell Access:

kubectl exec <container-name> -c shell -it bash

• Labels:

Used for ease of management

Assign Labels:

kubectl label pods <pod-name> owner=test
kubectl label pods <pod-name> env=development

Show Labels:

kubectl get pods --show-labels

Filter based on Labels:

kubectl get pods --selector owner=test

OR

kubectl get pods -l owner=test

kubectl get pods -l 'env in (production,development)' --show-labels

You can expose Pods based on Labels, i.e. Pods from different Deployments can be exposed.

• Deployment uses ReplicaSet NOT ReplicationController:

kubectl get replicationcontroller
<no output>

OR

kubectl get rc
<no output>

kubectl get replicaset

OR

kubectl get rs

• Services:

ClusterIP(default): used for East-West traffic only. Not significant outside cluster.
NodePort: Used for North-South traffic, Makes a service accessible from outside using NAT.
LoadBalancer: creates an external load-balancer in current cloud. Makes a service available outside.
ExternalName:

• Application should be accessible from all nodes - Master & Worker Nodes

• Ingress: not a Service, Instead sits in front of multiple services. type of Smart Router or entrypoint into cluster

Traffic ==> Ingress ==> | ==> foo.mydamoin.com  -- Service -- Pod,Pod,Pod
                        | ==> mydomain.com/bar  -- Service -- Pod,Pod,Pod
                        | ==> Other             -- Service -- Pod,Pod,Pod

Requirements

3 Ubuntu VMs having:

Same version
Having same resources
LAN Connectivity

Installing dependencies

Source: techrepublic.com, linuxtechi.com

This will be done on all machines that will join the Kubernetes cluster.

sudo apt-get update && apt-get install -y apt-transport-https

Our next dependency is Docker. Our Kubernetes installation will depend upon this, so install it with:

sudo apt install docker.io

Once that completes, start and enable the Docker service with the commands

sudo systemctl start docker
sudo systemctl enable docker

Disable Swap in all the 3 VMs:

sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

Or:

sudo sed -i '/ swap / s/^/#/' /etc/fstab

Installing Kubernetes

sudo curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add 

Next add a repository by creating the file /etc/apt/sources.list.d/kubernetes.list and enter the following content:

deb http://apt.kubernetes.io/ kubernetes-xenial main 

Save and close that file. Install Kubernetes with the following commands:

apt-get update
apt-get install -y kubelet kubeadm kubectl kubernetes-cni

Initialize your master

Go to the machine that will serve as the Kubernetes master and issue the command:

sudo su
sudo kubeadm init

Before you join a node, you need to issue the following commands (as a regular user):

exit
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Deploying a pod network

You must deploy a pod network before anything will actually function properly:

kubectl apply -f [podnetwork].yaml'

You can use one of the below Pod Networks:

https://kubernetes.io/docs/concepts/cluster-administration/addons/

Verify Pods, all should be running & only DNS pod should be Pending initially:

aman@ubuntu:~$ kubectl get pods --all-namespaces
NAMESPACE     NAME                             READY     STATUS    RESTARTS   AGE
kube-system   etcd-ubuntu                      1/1       Running   0          3m
kube-system   kube-apiserver-ubuntu            1/1       Running   0          3m
kube-system   kube-controller-manager-ubuntu   1/1       Running   0          3m
kube-system   kube-dns-86f4d74b45-wq49s        0/3       Pending   0          4m    <==
kube-system   kube-proxy-g96ml                 1/1       Running   0          4m
kube-system   kube-scheduler-ubuntu            1/1       Running   0          3m

Flannel

Multiple bugs were encountered when implementing Flannel

Here we will be installing the Flannel pod network:

sudo kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
sudo kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/k8s-manifests/kube-flannel-rbac.yml

Issue the command:

kubectl get pods —all-namespaces

Weave Net

Install the WeaveNet Pod:

export kubever=$(kubectl version | base64 | tr -d '\n')
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$kubever"

Verification

Verify Installation after a few minutes:

aman@ubuntu:~$ kubectl get pods --all-namespaces
NAMESPACE     NAME                             READY     STATUS    RESTARTS   AGE
kube-system   etcd-ubuntu                      1/1       Running   0          11m
kube-system   kube-apiserver-ubuntu            1/1       Running   0          11m
kube-system   kube-controller-manager-ubuntu   1/1       Running   0          11m
kube-system   kube-dns-86f4d74b45-wq49s        3/3       Running   0          12m    <==
kube-system   kube-proxy-g96ml                 1/1       Running   0          12m
kube-system   kube-scheduler-ubuntu            1/1       Running   0          11m
kube-system   weave-net-pg57l                  2/2       Running   0          6m     <==

Joining a node

With everything in place, you are ready to join the node to the master. To do this, go to the node's terminal and issue the command:

sudo su
kubeadm join --token <TOKEN> <MASTER_IP:6443>

OR what ever is shown in the outputof master after kubeadm init:

kubeadm join 10.1.11.184:6443 --token 0lxezc.game230zg6jpa60g --discovery-token-ca-cert-hash sha256:74b34793d0ty56037c71e4a54e7475901bf627~

Recreate a token if required:

sudo kubeadm token create

Verify from Master node:

kubectl get nodes

Deploying a service

Source: medium.com

At this point, you are ready to deploy a service on your Kubernetes cluster. To deploy an NGINX service (and expose the service on port 80), run the following commands (from the master):

sudo kubectl run nginx-app --image=nginx --port=80 --env="DOMAIN=cluster" --replicas=2
sudo kubectl expose deployment nginx-app --port=80 --name=nginx-http --type=NodePort

Managing Kubernetes

Scaling Deployment:

sudo kubectl get deployment nginx-app
sudo kubectl scale deployment nginx-app --replicas=3

Verify

Verify Pods:

kubectl get pods
kubectl get pods -o wide

Go to each worker nodes:

sudo docker ps -a

Delete a Pods

This will delete an existing POD 7 create a new one:

kubectl delete pod nginx-app-56f6bb6776-wrbvl

Delete a Deployment

Verify existing Pods & Service:

kubectl get deployments
kubectl get service

Delete a Pods & Service:

kubectl delete deployment nginx-app
kubectl delete service nginx-http

Delete all Pods & Services:

kubectl delete pods --all
kubectl delete service --all

Troubleshooting

If the Pod creations fails check logs:

kubectl describe pod nginx-app-56f6bb6776-b7cb5

kubectl config view

apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: REDACTED
    server: https://10.1.10.158:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED

sudo kubectl create serviceaccount test
sudo kubectl get secret
sudo kubectl get secret test-token-xqh8z
sudo kubectl get secret test-token-xqh8z -o yaml

kubectl describe secret

kubectl describe secret test-token-xqh8z

kubectl --token=<token> get nodes

Reset Everything

sudo kubeadm reset
sudo rm -rf .kube

Dashboard

This setup has not been tested successfully yet

• Install Dashboard:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/kubernetes-dashboard.yaml

• Create the User File:

vi dashboard-adminuser.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kube-system

kubectl create -f dashboard-adminuser.yaml

• Create the Role Binding file:

vi  ClusterRoleBinding-ui.yaml

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kube-system

kubectl create -f ClusterRoleBinding-ui.yaml

• Edit Dashboard service

kubectl -n kube-system edit service kubernetes-dashboard

 type: ClusterIP  <--- change this to NodePort

• Generate the token:

kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')

• Get the Port number:

kubectl get service -o wide --all-namespaces
  kube-system   kubernetes-dashboard   NodePort    10.99.104.194   <none>        443:31860/TCP   133m   k8s-app=kubernetes-dashboard

• Enable firewall ports ON ALL 3 NODES:

firewall-cmd --permanent --add-port=31860/tcp
firewall-cmd --reload

• Start Kude Proxy

kubectl proxy --address='0.0.0.0' --accept-hosts='^*$' &

• Access the UI:

https://<k8s-master>:31860/

• Select and paste the token

More Information

• Cheatsheet
• Kubernetes 101

References

{{#widget:DISQUS |id=networkm |uniqid=Kubernetes |url=https://aman.awiki.org/wiki/Kubernetes }}