Summary: Kubernetes hints, tips, oneliners and best practices.
Date: 15 December 2024

Pods
Basic scheduling unit that holds one or more containers.

Nodes
Machines (physical or virtual) in the cluster where pods are scheduled.

Cluster
Collection of nodes and associated resources.

Kubelet
An agent running on each node, responsible for managing the node and its containers.

Kubernetes Controller Manager
Manages controllers to regulate the state of the system.

Kube Proxy
Maintains network rules to allow communication between pods and external traffic.

etcd
Consistent and highly-available key-value store used for all cluster data.

API Server
Serves the Kubernetes API and is the primary entry point for administrative tasks.

Scheduler
Assigns pods to nodes based on resource requirements and other constraints.

Controller
Maintains the desired state of the system, such as ensuring the correct number of replicas for a particular application.

Service
Provides a consistent way to access a set of pods.

Namespace
A way to divide cluster resources between multiple users.

Volumes
Kubernetes supports various types of storage volumes, providing data persistence for pods.

Secrets and ConfigMaps
Mechanisms to manage sensitive information and configuration data separately from application code.

Deployment
A higher-level resource that manages updates to applications by handling the deployment and scaling of pods.

StatefulSets
Manages stateful applications, ensuring stable network identities and persistent storage for pods.

DaemonSets
Ensures that specific pods run on all (or specific) nodes for cluster-wide tasks like logging or monitoring.

Jobs and CronJobs
Run short-lived or scheduled tasks within the cluster.

Ingress
Manages external access to services, typically HTTP.

Network Policies
Define how groups of pods can communicate with each other and other network endpoints.

Horizontal Pod Autoscaler
Automatically adjusts the number of replica pods to handle varying load.

Vertical Pod Autoscaler
Adjusts the resources allocated to individual pods based on their usage.

Operators
A way to package, deploy, and manage applications using Kubernetes APIs and controllers.

Kubectl
The command-line interface to interact with Kubernetes clusters.

Get cluster information

kubectl cluster-info

List all k8s objects from Cluster Control plane

kubectl get all --all-namespaces

Deploy and delete a manifest file

kubectl apply -f manifest.yaml
kubectl delete -f manifest.yaml

Deploy and delete a manifest folder

kubectl apply -f kube-manifests/
kubectl delete -f kube-manifests/
# Recursive
kubectl apply -R -f kube-manifests/
kubectl delete -R -f kube-manifests/

Deploy to a namespace

kubectl apply -f manifest.yaml -n dev1

Get the kubectl version

kubectl version

Check where your $KUBECONFIG is:

~$ echo "$KUBECONFIG"
/mnt/c/Users/sjoer/.kube/config

Verify available configs:

~$ ls -l /mnt/c/Users/sjoer/.kube/
total 80
drwxrwxrwx 1 sjoerd sjoerd  4096 Dec 17  2024 cache
-rwxrwxrwx 1 sjoerd sjoerd  3164 Apr 14 17:03 config
-rwxrwxrwx 1 sjoerd sjoerd 10003 May 27  2025 azure-config-admin
-rwxrwxrwx 1 sjoerd sjoerd  3481 Apr 14 14:15 azure-config-user
-rwxrwxrwx 1 sjoerd sjoerd  1874 Apr 13 13:36 kubeconfig-sclw-devops.yaml

Set kubectl to other kubeconfig file per command:

kubectl get pods -A --kubeconfig /mnt/c/Users/sjoer/.kube/kubeconfig-sclw-devops.yaml

Set $KUBECONFIG to another kubeconfig file:

~$ KUBECONFIG=/mnt/c/Users/sjoer/.kube/kubeconfig-sclw-devops.yaml
~$ echo "$KUBECONFIG"
/mnt/c/Users/sjoer/.kube/kubeconfig-sclw-devops.yaml
~$ kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://a54d35e1-77e9-4398-b2f3-cc8a059ce527.api.k8s.nl-ams.scw.cloud:6443
  name: k8s-ams-devops
contexts:
- context:
    cluster: k8s-ams-devops
    user: k8s-ams-devops-admin
  name: admin@k8s-ams-devops
current-context: admin@k8s-ams-devops
kind: Config
preferences: {}
users:
- name: k8s-ams-devops-admin
  user:
    token: REDACTED

Change back to default:

KUBECONFIG=/mnt/c/Users/sjoer/.kube/config

List all nodes

kubectl get nodes
kubectl get nodes -o wide

Get detailed information about a node

kubectl describe node aks-agentpool-20417106-vmss000001

Remove taint from a node

kubectl taint nodes aks-agentpool-20417106-vmss000001 CriticalAddonsOnly=true:NoSchedule-

Get node resource performance

kubectl top nodes
kubectl top nodes --sort-by=cpu
kubectl top nodes --sort-by=memory
# Sort from low to high
kubectl top nodes --sort-by=cpu --no-headers | sort -k3 -n
kubectl top nodes --sort-by=memory --no-headers | sort -k3 -n

Connect to nodes in a private aks cluster

# Based on https://learn.microsoft.com/en-us/azure/aks/node-access where is explained on how to connect to aks nodes. However, this does not work for a private cluster as the container is not available. You can, assuming that you've added the container to your private container registry, use that one:
# Get all the nodes
kubectl get nodes -o wide
# Start the debug command to connect to the node using an image from a private container registry
kubectl debug node/aks-npuser001-34232393-vmss000001 -it --image=acreuwprd.azurecr.io/docker/docker/library/busybox:1.37

Get & Describe Limits

kubectl get limits -n dev3
kubectl describe limits default-cpu-mem-limit-range -n dev3

Get Resource Quota

kubectl get quota -n dev3
kubectl describe quota ns-resource-quota -n dev3

Check cpu and memory in pods, notice that this is actually the resources of the host

# cpu
cat /proc/cpuinfo | grep proc
# memory
free -h

List all namespaces and work with namespaces for other objects

kubectl get namespaces
kubectl get ns
kubectl get pods --all-namespaces

Create a namespace

kubectl create namespace dev1
kubectl create namespace dev2

Delete a namespace

kubectl delete ns dev1
kubectl delete ns dev2

Set a default namespace

# Set the default namespace and expected output
kubectl config set-context --current --namespace=ops
Context "aks-cluster" modified.

List all pods

kubectl get pods
kubectl get po

List all pods from a specific namespace

kubectl get pods -n dev1

Get logging from a pod

kubectl logs -f podname
# pods can have changing names, so you can use this command if you don't know the podname:
kubectl logs -f $(kubectl get po | egrep -o 'external-dns[A-Za-z0-9-]+')

Get detailed information about a pod

kubectl describe pod podname
kubectl describe pod myapp1-deployment-5bc58f6848-7vm2v

Get pod specifications like cpu and memory

kubectl get pod <pod-name> -o yaml

Get pod resource performance

kubectl top pods
kubectl top pods --sort-by=cpu
kubectl top pods --sort-by=memory

List all deployments

kubectl get deployments
kubectl get deploy

Restart a deployment

kubectl rollout restart deployment/kube-prometheus-stack-grafana
# In a different name space and with shorter names
kubectl rollout restart deploy spring-cloud-config-server -n prod
kubectl rollout restart deploy kube-prometheus-stack-grafana -n prod

Restart all deployments in a namespace

kubectl rollout restart deployment -n shared

Apply a manifest

kubectl apply -f kube-prometheus-stack.yaml --server-side -n shared
kubectl apply -f kube-prometheus-stack.yaml --server-side --force-conflicts -n shared

List all services

kubectl get services
kubectl get svc

List all services from all namespaces

kubectl get services --all-namespaces
# Sorted on name
kubectl get services --all-namespaces --sort-by=.metadata.name
# Sorted on type
kubectl get services --all-namespaces --sort-by=.spec.type
# Get all services of type LoadBalancer
kubectl get services --all-namespaces | grep LoadBalancer

List services with a specific label

kubectl get service -l app.kubernetes.io/name=ingress-nginx --namespace ingress-basic

Describe a service

kubectl describe svc proxy-public --namespace dev1

List all StatefulSets

kubectl get statefulsets
kubectl get sts

Kill all the pods in a statefulSet by setting the number of replicas to 0

kubectl scale statefulset myapp1 --replicas=0

Restart a statefulset

kubectl rollout restart statefulset/myapp1

Restart all statefulsets in a namespace

kubectl rollout restart statefulset -n shared

List all storage classes

kubectl get storageclasses
kubectl get sc

List all persistent volumes claims

kubectl get pvc

List all persistent volumes (the actual storage)

kubectl get pv

Delete a persistent volume

kubectl delete pv my-pv

List all storage information at once

kubectl get sc,pvc,pv

Get all ingress

kubectl get ingress

List all secrets

kubectl get secrets

Create a secret

kubectl create secret generic azure-config-file --from-file=azure.json

Decode a secret

echo "cGxhY2Vob2xkZXJwYXNzd29yZA==" | base64 --decode

Decode a secret with powershell

kubectl get secret argocd-initial-admin-secret --namespace ops -o json | ConvertFrom-Json | select -ExpandProperty data | % { $_.PSObject.Properties | % { $_.Name + [System.Environment]::NewLine + [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($_.Value)) + [System.Environment]::NewLine + [System.Environment]::NewLine } }

Connect to MySQL using Kubectl by installing a sql client pod

kubectl run -it --rm --image=mysql:8.0 --restart=Never mysql-client -- mysql -h akswebappdb201.mysql.database.azure.com -u dbadmin -p<password>

Reset Grafana admin password from within the pod

# Start k8s vscode extension -> vtxops -> configuration -> secrets -> kube-prometheus-stack-grafana -> Note down the admin-password
# opsnamespace -> workloads -> pods -> kube-prometheus-stack-grafana-xxxx
# Open the terminal (click terminal icon next to the name)
grafana cli admin reset-admin-password <admin-password>

Check for kafka topics

# open a terminal on one of the kafka brokers (eg kafka-kafka-0)
./bin/kafka-topics.sh --list --bootstrap-server localhost:9092

Get all resource kinds with their name from a manifest:

cat opentelemetry-operator.yaml | grep -i '^kind\|^  name:'

Here are some examples I use to perform actions on multiple resources at once.

Remove all finalizers for kafka topics

kubectl get kafkatopic.kafka.strimzi.io -n shared -o name |
while read topic; do
    echo "Removing finalizer from $topic"
    kubectl patch $topic -n shared -p '{"metadata":{"finalizers":[]}}' --type=merge
done

Delete all jobs that start with “backup”

export NAMESPACE=shared
kubectl get jobs -n $NAMESPACE -o name | grep backup- |
while read job; do
    echo "Deleting $job"
    kubectl delete $job -n $NAMESPACE
done

Connect to AKS

az aks get-credentials --resource-group myResourceGroup --name myAKSCluster
# overwrite the existing context
az aks get-credentials --resource-group myResourceGroup --name myAKSCluster --overwrite-existing

Connect to AKS

az login
az aks install-cli
# Add C:\Users\sjoer\.azure-kubectl to path
# Advanced System Settings -> Environment Variables -> User Variables -> Path -> New
# Configure Cluster Creds (kube config)
az aks get-credentials --resource-group aks-rg1 --name aksdemo1

az aks nodepool show --resource-group aks-rg1 --cluster-name aksdemo1 --name agentpool
# Remove all taints (must be done from cloud shell as it does not work locally)
az aks nodepool update --resource-group aks-rg1 --cluster-name aksdemo1 --name agentpool --node-taints ""
# Get the resource group name of the AKS cluster
az aks show --resource-group aks-rg1 --name aksdemo1 --query nodeResourceGroup -o tsv
# Create a public IP address with a static allocation
az network public-ip create --resource-group <REPLACE-OUTPUT-RG-FROM-PREVIOUS-COMMAND> --name myAKSPublicIPForIngress --sku Standard --allocation-method static --query publicIp.ipAddress -o tsv

# Install Helm3 (if not installed)
choco install kubernetes-helm
# Add a repository
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

Use Helm to deploy an NGINX ingress controller

helm install ingress-nginx ingress-nginx/ingress-nginx `
    --namespace ingress-basic `
    --set controller.replicaCount=2 `
    --set controller.nodeSelector."kubernetes\.io/os"=linux `
    --set defaultBackend.nodeSelector."kubernetes\.io/os"=linux `
    --set controller.service.externalTrafficPolicy=Local `
    --set controller.service.loadBalancerIP="172.205.120.177"

This wiki has been made possible by: