Owen Rumney
Software Engineer

Zero to k3d

07 Apr 2022 - Owen Rumney

I have had an on/off relationship with Kubernetes for a number of years and want to spend some time really getting to know how to use it, run it and secure it.

I previously wrote some Anisble playbooks to run Kubernetes on my desktop then chatting to @alistair_hey about running a local environment on my fairly underpowered laptop he suggested k3d … so the journey starts here.

Prerequisites

k3d is a wrapper for k3s but runs the cluster in Docker. This means that you will need Docker running on the machine (which I already have!). A quick solution is Docker Desktop which is very accessible or you can install it using the instructions for your OS and Architecture.

I will be using kubectl later to interact with the cluster, so this is also required.

Getting Started

Install kubectl

As I am running on Linux I will use the installation instructions for kubectl

The first step is to pull the latest binary from dl.k8s.io.

cd ~/Downloads

curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"

Now I have this, I want to check against the checksum file

cd ~/Downloads
curl -LO "https://dl.k8s.io/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl.sha256"

I use the checksums file to validate the binary is as it should be

echo "$(cat kubectl.sha256)  kubectl" | sha256sum --check

Assuming I were told it was kubectl: OK I can continue to install.

sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

Lets now check the install was good

kubectl version --short=true

which should give me output similar to

Client Version: v1.23.4
Server Version: v1.22.7+k3s1

Installing k3d

First step is to install k3d - I don’t like using wget or curl with install scripts from GitHub so I’m am going to be using the assets from the Releases Page.

I am using Ubuntu on a x64 machine so I need to download k3d-linux-amd64

Note I’m using v5.4.1 at the time of writing

Once I have it downloaded, I can install the command on the path

cd ~/Downloads

sudo install k3d-linux-amd64 /usr/local/bin/k3d

Now it has been installed I can run to check

k3d --version

This should give me the output;

k3d version v5.4.1
k3s version v1.22.7-k3s1 (default)

Creating the first cluster

For the first cluster I am going to create it with the following parameters

  • Name: first-cluster
  • Servers: 3

To do this, I use the cluster sub command

k3d cluster create first-cluster --servers 3

This will give an output similar to this;

INFO[0000] Prep: Network                                
INFO[0000] Created network 'k3d-first-cluster'          
INFO[0000] Created image volume k3d-first-cluster-images 
INFO[0000] Starting new tools node...                   
INFO[0000] Creating initializing server node            
INFO[0000] Creating node 'k3d-first-cluster-server-0'   
INFO[0000] Pulling image 'ghcr.io/k3d-io/k3d-tools:5.4.1' 
INFO[0001] Pulling image 'docker.io/rancher/k3s:v1.22.7-k3s1' 
INFO[0002] Starting Node 'k3d-first-cluster-tools'      
INFO[0005] Creating node 'k3d-first-cluster-server-1'   
INFO[0006] Creating node 'k3d-first-cluster-server-2'   
INFO[0006] Creating LoadBalancer 'k3d-first-cluster-serverlb' 
INFO[0007] Pulling image 'ghcr.io/k3d-io/k3d-proxy:5.4.1' 
INFO[0009] Using the k3d-tools node to gather environment information 
INFO[0009] HostIP: using network gateway 172.20.0.1 address 
INFO[0009] Starting cluster 'first-cluster'             
INFO[0009] Starting the initializing server...          
INFO[0009] Starting Node 'k3d-first-cluster-server-0'   
INFO[0010] Starting servers...                          
INFO[0010] Starting Node 'k3d-first-cluster-server-1'   
INFO[0031] Starting Node 'k3d-first-cluster-server-2'   
INFO[0047] All agents already running.                  
INFO[0047] Starting helpers...                          
INFO[0047] Starting Node 'k3d-first-cluster-serverlb'   
INFO[0054] Injecting records for hostAliases (incl. host.k3d.internal) and for 4 network members into CoreDNS configmap... 
INFO[0056] Cluster 'first-cluster' created successfully! 
INFO[0056] You can now use it like this:                
kubectl cluster-info

Within just under a minute I have my first cluster up and running, I can list the clusters to see its running with the cluster list sub command

k3d cluster list

The output tells me that there are 3 servers running;

NAME            SERVERS   AGENTS   LOADBALANCER
first-cluster   3/3       0/0      true

Now that the cluster is running I can use kubectl to get some more information about the cluster

kubectl cluster-info

which should give similar output to this;

Kubernetes control plane is running at https://0.0.0.0:38909
CoreDNS is running at https://0.0.0.0:38909/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Metrics-server is running at https://0.0.0.0:38909/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy

I know that I have 3 nodes - so lets get some inforamtion about them too

kubectl get nodes

This will give me an output similar to this;

NAME                         STATUS   ROLES                       AGE   VERSION
k3d-first-cluster-server-0   Ready    control-plane,etcd,master   15m   v1.22.7+k3s1
k3d-first-cluster-server-1   Ready    control-plane,etcd,master   15m   v1.22.7+k3s1
k3d-first-cluster-server-2   Ready    control-plane,etcd,master   14m   v1.22.7+k3s1

The last step in this bootstrapping post is to check I can run a container - so I’m going to do a basic command to run a new pod called nginx using the nginx image

kubectl run nginx --image nginx

which hopefully should give me some output letting me know my pod has been created

pod/nginx created

A pod is the smallest instance of compute that can be deployed. A pod can contain a number of containers if their purpose is tightly coupled; an example might be a pod with a cache container that has another short life container required to pre-populate.

I can check the status of the pod using kubectl

kubectl describe pod nginx

Which will give me a verbose output about the pod

Name:         nginx
Namespace:    default
Priority:     0
Node:         k3d-first-cluster-server-2/172.20.0.4
Start Time:   Thu, 07 Apr 2022 18:54:28 +0100
Labels:       run=nginx
Annotations:  <none>
Status:       Running
IP:           10.42.2.4
IPs:
  IP:  10.42.2.4
Containers:
  nginx:
    Container ID:   containerd://f5f568d6e62b5c8db6742b7ec2136a8939f2deb65e4b68a662a5ee7ee88e94e5
    Image:          nginx
    Image ID:       docker.io/library/nginx@sha256:2275af0f20d71b293916f1958f8497f987b8d8fd8113df54635f2a5915002bf1
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Thu, 07 Apr 2022 18:54:28 +0100
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-vs2qt (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  kube-api-access-vs2qt:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  5m33s  default-scheduler  Successfully assigned default/nginx to k3d-first-cluster-server-2
  Normal  Pulling    5m33s  kubelet            Pulling image "nginx"
  Normal  Pulled     5m28s  kubelet            Successfully pulled image "nginx" in 5.005731925s
  Normal  Created    5m28s  kubelet            Created container nginx
  Normal  Started    5m27s  kubelet            Started container nginx

I can see from this that the pod was deployed on server-2 and pulled the latest nginx image before successfully starting

Cleaning Up

Now that I know that everything seems to be okay, I can tear it all down before starting to do something more useful in the next blog post.

First lets stop the nginx pod

kubectl delete pod nginx

Then I can stop the cluster

k3d cluster stop first-server

INFO[0000] Stopping cluster 'first-cluster'             
INFO[0011] Stopped cluster 'first-cluster'

I am going to keep the cluster for the next blog, but I can now remove it if I wish using

k3d cluster delete first-cluster

INFO[0000] Deleting cluster 'first-cluster'             
INFO[0000] Deleting cluster network 'k3d-first-cluster' 
INFO[0000] Deleting 2 attached volumes...               
WARN[0000] Failed to delete volume 'k3d-first-cluster-images' of cluster 'first-cluster': failed to find volume 'k3d-first-cluster-images': Error: No such volume: k3d-first-cluster-images -> Try to delete it manually 
INFO[0000] Removing cluster details from default kubeconfig... 
INFO[0000] Removing standalone kubeconfig file (if there is one)... 
INFO[0000] Successfully deleted cluster first-cluster!