Kubernetes
Basics
- Kubernetes Master Processes:
- kube-apiserver - kube-controller-manager - kube-scheduler - etcd: key-value pair store to store tags or labels for easy Management
- Nodes Component Processes:
- kubelet: talks to kubernetes master - kube-proxy: - DNS addon - UI addon - fluentd: for log collection - Supervisord
- Kubernetes Concepts:
- Container Image: Docker container Image with Application code - Pod: Set of containers sharing network namespace & Local volumes, co-scheduled on one machine. Mortal, has IP, Label - Deployment: Specify how many replicas of pod should be run in a cluster. Has label. - Service: names things in DNS, Gets Virtual IP. Routes based on labels. Two types: ClusterIP for Internal services NodePort for publishing to outside.
- Do not care about PODS, Just care about Deployments in Kubernetes.
- Kubernetes Networking Requirements:
- All contianers should communicate without NAT - All Nodes can communicate without NAT - IP that a container sees itselg as is the same IP that others see it.
eth0<------->docker0<--------->veth0
10.100.0.2 docker bridge container1
172.17.0.1 172.17.0.2
Overlay network e.g. Flannel:
eth0<--------->cbr0<--------->veth0
10.100.0.2 container1
172.17.0.1 172.17.0.2
- Pause State:
- All networking in the Pod lies in the overlay itself.
- Kube-Proxy:
- Implemented by Iptables - Handles Inter-Node(nodes on different Hosts) communication => East-West communication
- YAML output:
kubectl get pod <name> -o yaml
- Shell Access:
kubectl exec <container-name> -c shell -it bash
- Labels:
Used for ease of management
Assign Labels:
kubectl label pods <pod-name> owner=test kubectl label pods <pod-name> env=development
Show Labels:
kubectl get pods --show-labels
Filter based on Labels:
kubectl get pods --selector owner=test
OR
kubectl get pods -l owner=test
kubectl get pods -l 'env in (production,development)' --show-labels
You can expose Pods based on Labels, i.e. Pods from different Deployments can be exposed.
- Deployment uses ReplicaSet NOT ReplicationController:
kubectl get replicationcontroller <no output>
OR
kubectl get rc <no output>
kubectl get replicaset
OR
kubectl get rs
- Services:
ClusterIP(default): used for East-West traffic only. Not significant outside cluster. NodePort: Used for North-South traffic, Makes a service accessible from outside using NAT. LoadBalancer: creates an external load-balancer in current cloud. Makes a service available outside. ExternalName:
- Application should be accessible from all nodes - Master & Worker Nodes
- Ingress: not a Service, Instead sits in front of multiple services. type of Smart Router or entrypoint into cluster
Traffic ==> Ingress ==> | ==> foo.mydamoin.com -- Service -- Pod,Pod,Pod
| ==> mydomain.com/bar -- Service -- Pod,Pod,Pod
| ==> Other -- Service -- Pod,Pod,Pod
Requirements
3 Ubuntu VMs having:
Same version Having same resources LAN Connectivity
Installing dependencies
Source: techrepublic.com, linuxtechi.com
This will be done on all machines that will join the Kubernetes cluster.
sudo apt-get update && apt-get install -y apt-transport-https
Our next dependency is Docker. Our Kubernetes installation will depend upon this, so install it with:
sudo apt install docker.io
Once that completes, start and enable the Docker service with the commands
sudo systemctl start docker sudo systemctl enable docker
Disable Swap in all the 3 VMs:
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
Or:
sudo sed -i '/ swap / s/^/#/' /etc/fstab
Installing Kubernetes
sudo curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add
Next add a repository by creating the file /etc/apt/sources.list.d/kubernetes.list and enter the following content:
deb http://apt.kubernetes.io/ kubernetes-xenial main
Save and close that file. Install Kubernetes with the following commands:
apt-get update apt-get install -y kubelet kubeadm kubectl kubernetes-cni
Initialize your master
Go to the machine that will serve as the Kubernetes master and issue the command:
sudo su sudo kubeadm init
Before you join a node, you need to issue the following commands (as a regular user):
exit mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
Deploying a pod network
You must deploy a pod network before anything will actually function properly:
kubectl apply -f [podnetwork].yaml'
You can use one of the below Pod Networks:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
Verify Pods, all should be running & only DNS pod should be Pending initially:
aman@ubuntu:~$ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system etcd-ubuntu 1/1 Running 0 3m kube-system kube-apiserver-ubuntu 1/1 Running 0 3m kube-system kube-controller-manager-ubuntu 1/1 Running 0 3m kube-system kube-dns-86f4d74b45-wq49s 0/3 Pending 0 4m <== kube-system kube-proxy-g96ml 1/1 Running 0 4m kube-system kube-scheduler-ubuntu 1/1 Running 0 3m
Flannel
 Multiple bugs were encountered when implementing Flannel
|
Here we will be installing the Flannel pod network:
sudo kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml sudo kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/k8s-manifests/kube-flannel-rbac.yml
Issue the command:
kubectl get pods —all-namespaces
Weave Net
Install the WeaveNet Pod:
export kubever=$(kubectl version | base64 | tr -d '\n') kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$kubever"
Verification
Verify Installation after a few minutes:
aman@ubuntu:~$ kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system etcd-ubuntu 1/1 Running 0 11m kube-system kube-apiserver-ubuntu 1/1 Running 0 11m kube-system kube-controller-manager-ubuntu 1/1 Running 0 11m kube-system kube-dns-86f4d74b45-wq49s 3/3 Running 0 12m <== kube-system kube-proxy-g96ml 1/1 Running 0 12m kube-system kube-scheduler-ubuntu 1/1 Running 0 11m kube-system weave-net-pg57l 2/2 Running 0 6m <==
Joining a node
With everything in place, you are ready to join the node to the master. To do this, go to the node's terminal and issue the command:
sudo su kubeadm join --token <TOKEN> <MASTER_IP:6443>
OR what ever is shown in the outputof master after kubeadm init:
kubeadm join 10.1.11.184:6443 --token 0lxezc.game230zg6jpa60g --discovery-token-ca-cert-hash sha256:74b34793d0ty56037c71e4a54e7475901bf627~
Recreate a token if required:
sudo kubeadm token create
Verify from Master node:
kubectl get nodes
Deploying a service
Source: medium.com
At this point, you are ready to deploy a service on your Kubernetes cluster. To deploy an NGINX service (and expose the service on port 80), run the following commands (from the master):
sudo kubectl run nginx-app --image=nginx --port=80 --env="DOMAIN=cluster" --replicas=2 sudo kubectl expose deployment nginx-app --port=80 --name=nginx-http --type=NodePort
Managing Kubernetes
Scaling Deployment:
sudo kubectl get deployment nginx-app sudo kubectl scale deployment nginx-app --replicas=3
Verify
Verify Pods:
kubectl get pods kubectl get pods -o wide
Go to each worker nodes:
sudo docker ps -a
Delete a Pods
This will delete an existing POD 7 create a new one:
kubectl delete pod nginx-app-56f6bb6776-wrbvl
Delete a Deployment
Verify existing Pods & Service:
kubectl get deployments kubectl get service
Delete a Pods & Service:
kubectl delete deployment nginx-app kubectl delete service nginx-http
Delete all Pods & Services:
kubectl delete pods --all kubectl delete service --all
Troubleshooting
If the Pod creations fails check logs:
kubectl describe pod nginx-app-56f6bb6776-b7cb5
kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority-data: REDACTED
server: https://10.1.10.158:6443
name: kubernetes
contexts:
- context:
cluster: kubernetes
user: kubernetes-admin
name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
user:
client-certificate-data: REDACTED
client-key-data: REDACTED
sudo kubectl create serviceaccount test sudo kubectl get secret sudo kubectl get secret test-token-xqh8z sudo kubectl get secret test-token-xqh8z -o yaml
kubectl describe secret
kubectl describe secret test-token-xqh8z
kubectl --token=<token> get nodes
Reset Everything
sudo kubeadm reset sudo rm -rf .kube
Dashboard
| This setup has not been tested successfully yet
|
- Install Dashboard:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/master/aio/deploy/recommended/kubernetes-dashboard.yaml
- Create the User File:
vi dashboard-adminuser.yaml
apiVersion: v1 kind: ServiceAccount metadata: name: admin-user namespace: kube-system
kubectl create -f dashboard-adminuser.yaml
- Create the Role Binding file:
vi ClusterRoleBinding-ui.yaml
apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRoleBinding metadata: name: admin-user roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: cluster-admin subjects: - kind: ServiceAccount name: admin-user namespace: kube-system
kubectl create -f ClusterRoleBinding-ui.yaml
- Edit Dashboard service
kubectl -n kube-system edit service kubernetes-dashboard
type: ClusterIP <--- change this to NodePort
- Generate the token:
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
- Get the Port number:
kubectl get service -o wide --all-namespaces kube-system kubernetes-dashboard NodePort 10.99.104.194 <none> 443:31860/TCP 133m k8s-app=kubernetes-dashboard
- Enable firewall ports ON ALL 3 NODES:
firewall-cmd --permanent --add-port=31860/tcp firewall-cmd --reload
- Start Kude Proxy
kubectl proxy --address='0.0.0.0' --accept-hosts='^*$' &
- Access the UI:
https://<k8s-master>:31860/
- Select and paste the token
More Information
- References
{{#widget:DISQUS
|id=networkm
|uniqid=Kubernetes
|url=https://aman.awiki.org/wiki/Kubernetes
}}