Self-hosting A Kubernetes Cluster using K3S

K3S is a lightweight Kubernetes distribution developed by Rancher Labs, designed to run on resource-constrained environments such as edge devices, IoT appliances, and even low-powered servers. This guide walks you through the basic steps to deploy K3S on a cluster of 5 machines.

Prerequisites

1. Operating System: Ensure all nodes are running a supported OS (e.g., Ubuntu, Debian, CentOS).
2. User Permissions: You need root or sudo privileges on all nodes.
3. Network Configuration: Ensure that all nodes can communicate with each other over the network.
4. 5 Virtual Machines: This example uses 5 nodes in a cluster (3 server, 2 worker nodes). You can use a template to spin up the machines in a few minutes.

Step 1: Install K3s on the First Master Node

• Install K3s on Master Node 1:
  • curl -sfL https://get.k3s.io | sh -
• Retrieve the Node Token:
  • cat /var/lib/rancher/k3s/server/node-token
• Get the K3s Master Node 1 IP Address.
  • hostname -I

The token and the IP address are used to join other nodes to the cluster.

Step 2: Install K3s on the Other Master Nodes

• Install K3s on Master Nodes 2 and 3:
  • SSH into Master Node 2 and run the following command to join the node to the cluster (replace <MASTER_NODE_1_IP> and <NODE_TOKEN> with the actual IP address and token):

curl -sfL https://get.k3s.io | K3S_URL=https://<MASTER_NODE_1_IP>:6443 K3S_TOKEN=<NODE_TOKEN> sh -

• Repeat the above step for Master Node 3.
• Verify Connection on the Master Node 1:
  • kubectl get nodes
• Set the Roles on Master Node 1:
  • kubectl label node MASTER_NODE_2_NAME node-role.kubernetes.io/master=control-plane
  • kubectl label node MASTER_NODE_3_NAME node-role.kubernetes.io/master=control-plane

Step 3: Install K3S on the Worker Nodes

• Install K3s on Worker Nodes 1 and 2:
  • SSH into Worker Node 1 and run the following command to join the node to the cluster (replace <MASTER_NODE_1_IP> and <NODE_TOKEN> with the actual IP address and token):

curl -sfL https://get.k3s.io | K3S_URL=https://<MASTER_NODE_1_IP>:6443 K3S_TOKEN=<NODE_TOKEN> sh -

• Repeat the above step for Worker Node 2.
• Verify Connection on the Master Node 1:
  • kubectl get nodes
• Set the Roles on Master Node 1:
  • kubectl label node WORKER_NODE_1_NAME node-role.kubernetes.io/worker=worker
  • kubectl label node WORKER_NODE_2_NAME node-role.kubernetes.io/worker=worker

Step 4: Verify the Cluster

• Verify the Cluster Status:
  • kubectl get nodes

Step 5: Install Helm

• Download the Most Recent Helm Binary Executable:
  • curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
• Add Execute Permissions to the Downloaded Script and Run It:
  • chmod 700 get_helm.sh
  • ./get_helm.sh

Step 6: Deploy Portainer

• Deploy Portainer:
  • sudo kubectl apply -n portainer -f https://raw.githubusercontent.com/portainer/k8s/master/deploy/manifests/portainer/portainer.yaml
• Access the Portainer Web Interface:
  • Open a web browser and go to http://<NODE_IP>:30777, where <NODE_IP> is the IP address of any of your K3S nodes.
  • Follow the on-screen instructions to set up the admin user and password.
  • Add your K3S cluster to Portainer, it will be automatically recognised.

Example Deployment: Nginx

• Create a Deployment File on Master Node 1:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
spec:
  type: LoadBalancer
  selector:
    app: nginx
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80

• Apply the Deployment:
  • kubectl apply -f nginx-deployment.yaml
• Check the Status of Your Pods:
  • kubectl get pods
• Check the Status of the Service to Get the External IP or NodePort:
  • kubectl get services
• Describe the Service to Get the NodePort:
  • kubectl describe service nginx-service
• Access the Service:
  • Open your browser and navigate to the IP address of any of your K3S nodes using the NodePort obtained.
• Clean Up the Resources after testing:
  • kubectl delete -f nginx-deployment.yaml

This guide provides a structured approach to deploying a K3S cluster and managing it with Helm and Portainer, complete with an example Nginx deployment.