This guide covers the essential kubectl commands for deploying, managing, and operating CleanStart containerized applications on Kubernetes.
Prerequisites
Before you can use kubectl, install it and configure access to your cluster:
# Install kubectlcurl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"chmod +x kubectlsudo mv kubectl /usr/local/bin/ # Verify installationkubectl version --client# Client Version: v1.28.0 # Configure access to cluster# Your cluster admin provides a kubeconfig fileexport KUBECONFIG=/path/to/kubeconfig.yamlkubectl cluster-info# Kubernetes control plane is running...Basic Kubectl Commands
Apply Configuration
Deploy a Kubernetes manifest by applying it to your cluster:
# Apply a single manifestkubectl apply -f deployment.yaml # Apply all YAML files in directorykubectl apply -f ./k8s/ # Apply with dry-run (see what would happen, don't apply)kubectl apply -f deployment.yaml --dry-run=client -o yaml # Apply and show what changedkubectl apply -f deployment.yaml --recordGet Resources
List and inspect Kubernetes objects to understand what's running:
# List all podskubectl get pods # Get pods with more detailskubectl get pods -o wide # Get pods in YAML formatkubectl get pods -o yaml # Get specific podkubectl get pod myapp-123abc --output json # Get pods from specific namespacekubectl get pods -n production # Get pods across all namespaceskubectl get pods -A # List deploymentskubectl get deployments # List serviceskubectl get services # List all resourceskubectl get allDescribe Resources
Show detailed information about a resource to understand its state:
# Describe a podkubectl describe pod myapp-123abc # Describe a deploymentkubectl describe deployment myapp # Example output:# Name: myapp# Namespace: default# Image(s): myregistry/myapp:1.0.0# Replicas: 3 desired | 3 updated | 3 total | 3 available | 0 unavailable# Conditions:# Type Status Reason# ---- ------ ------# Available True MinimumReplicasAvailable# Events:# Type Reason Age From Message# ---- ------ ---- ---- -------# Normal ScalingReplicaSet 5m48s deployment-controller Scaled up replica set myapp-abc123 to 3View Logs
Examine container logs to understand what's happening:
# View logs from a podkubectl logs myapp-123abc # View logs with timestampskubectl logs myapp-123abc --timestamps=true # View last 100 lineskubectl logs myapp-123abc --tail=100 # Follow logs (like tail -f)kubectl logs -f myapp-123abc # View logs from specific container (if pod has multiple)kubectl logs myapp-123abc -c app-container # View logs from all pods in deploymentkubectl logs -l app=myapp # View previous container logs (if crashed and restarted)kubectl logs myapp-123abc --previousExecute Commands in Container
Run commands inside a running container:
# Interactive shellkubectl exec -it myapp-123abc -- bash # Run a specific commandkubectl exec myapp-123abc -- curl http://localhost:5000/health # Run as different userkubectl exec myapp-123abc -- id # Note: Only works with -dev images; -prod images have minimal shellsDeployment Management
Complete Deployment Manifest
A production-ready Kubernetes deployment includes proper configuration for replicas, updates, resource management, health checks, and security:
apiVersion: apps/v1kind: Deploymentmetadata: name: myapp namespace: default labels: app: myapp version: v1spec: replicas: 3 strategy: type: RollingUpdate rollingUpdate: maxSurge: 1 # 1 extra pod during update maxUnavailable: 0 # 0 pods can be down at any time selector: matchLabels: app: myapp template: metadata: labels: app: myapp annotations: prometheus.io/scrape: "true" prometheus.io/port: "5000" prometheus.io/path: "/metrics" spec: serviceAccountName: default securityContext: runAsNonRoot: true runAsUser: 65532 runAsGroup: 65532 fsGroup: 65532 seccompProfile: type: RuntimeDefault containers: - name: app image: myregistry/myapp:1.0.0@sha256:a1b2c3d4... # Use digest for immutability imagePullPolicy: IfNotPresent ports: - name: http containerPort: 5000 protocol: TCP env: - name: ENVIRONMENT value: production - name: LOG_LEVEL value: INFO - name: DATABASE_URL valueFrom: secretKeyRef: name: myapp-secrets key: database-url resources: requests: memory: "128Mi" cpu: "100m" limits: memory: "512Mi" cpu: "1000m" livenessProbe: httpGet: path: /health port: 5000 initialDelaySeconds: 10 periodSeconds: 10 timeoutSeconds: 3 failureThreshold: 3 readinessProbe: httpGet: path: /ready port: 5000 initialDelaySeconds: 5 periodSeconds: 5 timeoutSeconds: 2 failureThreshold: 3 securityContext: allowPrivilegeEscalation: false capabilities: drop: - ALL readOnlyRootFilesystem: true runAsNonRoot: true volumeMounts: - name: tmp mountPath: /tmp - name: var-tmp mountPath: /var/tmp volumes: - name: tmp emptyDir: {} - name: var-tmp emptyDir: {} affinity: podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution: - weight: 100 podAffinityTerm: labelSelector: matchExpressions: - key: app operator: In values: - myapp topologyKey: kubernetes.io/hostname ---apiVersion: v1kind: Servicemetadata: name: myappspec: type: ClusterIP selector: app: myapp ports: - name: http protocol: TCP port: 80 targetPort: 5000Deploy Application
# Create namespacekubectl create namespace production # Apply deployment and servicekubectl apply -f deployment.yaml -n production # Verify deploymentkubectl get deployment myapp -n productionkubectl get pods -n productionkubectl get service myapp -n productionCheck Rollout Status
Monitor deployment progress as pods start:
# Check statuskubectl rollout status deployment/myapp -n production # Example output (still rolling out):# Waiting for deployment "myapp" rollout to finish: 1 of 3 updated replicas are available... # Example output (complete):# deployment "myapp" successfully rolled out # View rollout historykubectl rollout history deployment/myapp -n production # Example output:# REVISION CHANGE-CAUSE# 1 kubectl apply --filename=deployment.yaml# 2 kubectl set image deployment/myapp app=myregistry/myapp:1.1.0# 3 kubectl apply --filename=deployment.yamlRollback to Previous Version
If a deployment goes bad, rollback instantly:
# Rollback to previous revisionkubectl rollout undo deployment/myapp -n production # Rollback to specific revisionkubectl rollout undo deployment/myapp --to-revision=1 -n production # Verify rollbackkubectl rollout status deployment/myapp -n productionScaling Applications
Manual Scaling
Change the number of replicas:
# Scale to 5 replicaskubectl scale deployment myapp --replicas=5 -n production # Check new statuskubectl get deployment myapp -n production # Example output:# NAME READY UP-TO-DATE AVAILABLE AGE# myapp 5/5 5 5 10mAutoscaling with HPA (Horizontal Pod Autoscaler)
Automatically scale based on CPU/memory:
apiVersion: autoscaling/v2kind: HorizontalPodAutoscalermetadata: name: myapp-hpaspec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: myapp minReplicas: 3 maxReplicas: 10 metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 70 - type: Resource resource: name: memory target: type: Utilization averageUtilization: 80Deploy and check:
kubectl apply -f hpa.yaml -n production kubectl get hpa -n production # Watch HPA in actionkubectl get hpa -n production --watch # Example output:# NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE# myapp-hpa Deployment/myapp CPU: 45%, Mem: 60% 3 10 5 5mCanary Deployments
Gradually roll out new versions to catch issues early.
Step 1: Deploy canary (10% of traffic)
apiVersion: apps/v1kind: Deploymentmetadata: name: myapp-canaryspec: replicas: 1 selector: matchLabels: app: myapp version: canary template: metadata: labels: app: myapp version: canary spec: containers: - name: app image: myregistry/myapp:1.1.0 # New version # ... rest of spec# Deploy canarykubectl apply -f deployment-canary.yaml -n production # Service automatically routes to both old and new based on labels# With 1 canary pod and 3 stable pods = ~25% traffic to canaryStep 2: Monitor metrics
# Check error rates and latencykubectl logs -l version=canary -n production # Monitor application metricskubectl top pods -n production # CPU/memory usageStep 3: Gradually increase canary replicas
# If canary looks good, scale it upkubectl scale deployment myapp-canary --replicas=2 -n production # Continue monitoring...Step 4: Complete the rollout
# When confident, update stable deployment to new versionkubectl set image deployment/myapp app=myregistry/myapp:1.1.0 -n production # Delete canarykubectl delete deployment myapp-canary -n productionBlue-Green Deployments
Keep two full environments and switch traffic instantly:
# Green deployment (current production)apiVersion: apps/v1kind: Deploymentmetadata: name: myapp-greenspec: replicas: 3 selector: matchLabels: app: myapp env: green # ... spec ---# Service points to greenapiVersion: v1kind: Servicemetadata: name: myappspec: selector: app: myapp env: green # Points to green ports: - port: 80 targetPort: 5000Switch traffic to new version:
# Deploy blue (new version)kubectl apply -f deployment-blue.yaml -n production # Test bluekubectl run -it test-curl --image=curl --rm -- \ curl http://myapp-blue/health # Switch service to bluekubectl patch service myapp -n production \ -p '{"spec":{"selector":{"env":"blue"}}}' # If problems, switch back instantlykubectl patch service myapp -n production \ -p '{"spec":{"selector":{"env":"green"}}}'Pod Security Contexts (CleanStart Best Practices)
CleanStart base images are designed to work with restrictive security contexts:
apiVersion: v1kind: Podmetadata: name: secure-appspec: securityContext: # Run as non-root user (CleanStart default: 65532) runAsNonRoot: true runAsUser: 65532 runAsGroup: 65532 fsGroup: 65532 # Drop all capabilities, then add back only needed ones seccompProfile: type: RuntimeDefault containers: - name: app image: cleanstart/python:3.11-prod securityContext: # Don't allow privilege escalation allowPrivilegeEscalation: false # Drop all Linux capabilities capabilities: drop: - ALL # Only add back if absolutely necessary # add: # - NET_BIND_SERVICE # Filesystem is read-only (except /tmp and /var/tmp) readOnlyRootFilesystem: true # Must be non-root runAsNonRoot: true volumeMounts: - name: tmp mountPath: /tmp - name: var-tmp mountPath: /var/tmp volumes: - name: tmp emptyDir: {} - name: var-tmp emptyDir: {}Verify security context is enforced:
# Check running userkubectl exec -it myapp-123abc -- id# uid=65532 gid=65532 groups=65532 # Try to write to root (should fail)kubectl exec -it myapp-123abc -- touch /test.txt# Read-only file system # Try to escalate privileges (should fail)kubectl exec -it myapp-123abc -- sudo su# sudo: command not foundImage Signature Verification
Verify CleanStart images are signed before they run:
# Pod that verifies signatures with CosignapiVersion: v1kind: Podmetadata: name: verify-imagespec: containers: - name: verifier image: gcr.io/projectsigstore/cosign:latest command: - sh - -c - | COSIGN_EXPERIMENTAL=1 cosign verify \ --insecure-ignore-tlog \ myregistry/myapp:1.0.0Using admission webhooks (for automatic verification):
apiVersion: admissionregistration.k8s.io/v1kind: ValidatingWebhookConfigurationmetadata: name: image-signature-verificationwebhooks:- name: verify.cleanstart.io clientConfig: service: name: cosign-webhook namespace: security path: "/verify" caBundle: ... rules: - operations: ["CREATE", "UPDATE"] apiGroups: [""] apiVersions: ["v1"] resources: ["pods"] # Only allow images from cleanstart and verified registries namespaceSelector: matchLabels: enforce-image-verification: "true"Troubleshooting Common Issues
Pod Not Starting
When a pod doesn't start, investigate why by describing the pod:
# Describe the pod to see whykubectl describe pod myapp-123abc -n production # Look for Events section for clues # Check logskubectl logs myapp-123abc -n production # Example issues:# ImagePullBackOff = Wrong image name or registry credentials# CrashLoopBackOff = Container crashes immediately# Pending = Not enough resources on nodes# CreateContainerConfigError = Bad security context or volume mountChecking Resource Constraints
When pods can't schedule or are being throttled, check node and pod resources:
# See node resourceskubectl top nodes # Example output:# NAME CPU(cores) CPU% MEMORY(Mi) MEMORY%# node-1 500m 50% 2000Mi 50%# node-2 200m 20% 1000Mi 25% # See pod resource usagekubectl top pods -n production # Check if pod is being throttledkubectl describe pod myapp-123abc | grep -A5 "Limits"Health Check Failures
When liveness or readiness probes fail, the pod is either removed from service or restarted:
# See health check statuskubectl describe pod myapp-123abc | grep -A10 "Liveness" # If liveness fails, pod restarts# If readiness fails, pod is removed from service # Manually test health checkkubectl exec -it myapp-123abc -- curl http://localhost:5000/healthDebugging with Port Forward
Connect to pod directly (bypassing service):
# Forward local port to podkubectl port-forward pod/myapp-123abc 8080:5000 -n production # Now access locallycurl http://localhost:8080/health # Ctrl+C to stop forwardingProduction Checklist
Before deploying to production, verify the following:
Deployment configuration should use image digests instead of tags for immutable references, define resource requests and limits for CPU and memory, configure both liveness and readiness health checks, set security context for non-root execution with read-only file systems and privilege escalation disabled, deploy at least 3 replicas for high availability, configure PodDisruptionBudget to minimize downtime during updates, and set pod affinity rules to spread pods across nodes.
Service configuration should use ClusterIP or LoadBalancer type (not NodePort), enable session affinity if your application requires it, and assign a service account with minimal permissions.
Networking requires configuring network policies to control traffic between services, enabling TLS for all external traffic, and ensuring secrets are encrypted at rest.
Monitoring should expose metrics in Prometheus format, aggregate logs to a centralized system (ELK, Splunk, etc.), configure alerts for errors and performance degradation, and enable uptime monitoring.
Update strategy should configure a rolling update strategy for safe deployments, document your rollback procedures, and define your canary or blue-green deployment process.
What Matters
The fundamental kubectl operations that matter most are: kubectl apply for deploying resources from YAML, kubectl get/describe for inspecting resources, kubectl logs/exec for accessing container internals, kubectl rollout for controlling deployment updates, kubectl scale for changing replica count, HPA for autoscaling based on metrics, security context for enforcing restrictive settings (CleanStart default: UID 65532, read-only, no privs), canary/blue-green for minimizing blast radius of bad deployments, health checks for detecting and recovering from failures, and monitoring for visibility into production systems.
Common mistakes to avoid:
Avoid using latest tag; use digest for immutability instead. Never skip resource limits as one pod can consume all cluster resources. Don't run as root due to security risk. Never skip health checks as unhealthy pods keep running. Avoid single replica; always maintain high availability. Never skip security contexts as they reduce unnecessary attack surface. Don't skip testing rollback, as recovery shouldn't be panicked. Never deploy without monitoring, as you'll be flying blind.
