Skip to main content
CleanStart

Knowledge Hub

Kubectl Deployment Guide: Running CleanStart Applications

This guide covers the essential kubectl commands for deploying, managing, and operating CleanStart containerized applications on Kubernetes.

Prerequisites

Before you can use kubectl, install it and configure access to your cluster:

# Install kubectlcurl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"chmod +x kubectlsudo mv kubectl /usr/local/bin/ # Verify installationkubectl version --client# Client Version: v1.28.0 # Configure access to cluster# Your cluster admin provides a kubeconfig fileexport KUBECONFIG=/path/to/kubeconfig.yamlkubectl cluster-info# Kubernetes control plane is running...

Basic Kubectl Commands

Apply Configuration

Deploy a Kubernetes manifest by applying it to your cluster:

# Apply a single manifestkubectl apply -f deployment.yaml # Apply all YAML files in directorykubectl apply -f ./k8s/ # Apply with dry-run (see what would happen, don't apply)kubectl apply -f deployment.yaml --dry-run=client -o yaml # Apply and show what changedkubectl apply -f deployment.yaml --record

Get Resources

List and inspect Kubernetes objects to understand what's running:

# List all podskubectl get pods # Get pods with more detailskubectl get pods -o wide # Get pods in YAML formatkubectl get pods -o yaml # Get specific podkubectl get pod myapp-123abc --output json # Get pods from specific namespacekubectl get pods -n production # Get pods across all namespaceskubectl get pods -A # List deploymentskubectl get deployments # List serviceskubectl get services # List all resourceskubectl get all

Describe Resources

Show detailed information about a resource to understand its state:

# Describe a podkubectl describe pod myapp-123abc # Describe a deploymentkubectl describe deployment myapp # Example output:# Name:         myapp# Namespace:    default# Image(s):     myregistry/myapp:1.0.0# Replicas:     3 desired | 3 updated | 3 total | 3 available | 0 unavailable# Conditions:#   Type           Status  Reason#   ----           ------  ------#   Available      True    MinimumReplicasAvailable# Events:#   Type    Reason             Age    From                   Message#   ----    ------             ----   ----                   -------#   Normal  ScalingReplicaSet  5m48s  deployment-controller  Scaled up replica set myapp-abc123 to 3

View Logs

Examine container logs to understand what's happening:

# View logs from a podkubectl logs myapp-123abc # View logs with timestampskubectl logs myapp-123abc --timestamps=true # View last 100 lineskubectl logs myapp-123abc --tail=100 # Follow logs (like tail -f)kubectl logs -f myapp-123abc # View logs from specific container (if pod has multiple)kubectl logs myapp-123abc -c app-container # View logs from all pods in deploymentkubectl logs -l app=myapp # View previous container logs (if crashed and restarted)kubectl logs myapp-123abc --previous

Execute Commands in Container

Run commands inside a running container:

# Interactive shellkubectl exec -it myapp-123abc -- bash # Run a specific commandkubectl exec myapp-123abc -- curl http://localhost:5000/health # Run as different userkubectl exec myapp-123abc -- id # Note: Only works with -dev images; -prod images have minimal shells

Deployment Management

Complete Deployment Manifest

A production-ready Kubernetes deployment includes proper configuration for replicas, updates, resource management, health checks, and security:

apiVersion: apps/v1kind: Deploymentmetadata:  name: myapp  namespace: default  labels:    app: myapp    version: v1spec:  replicas: 3  strategy:    type: RollingUpdate    rollingUpdate:      maxSurge: 1        # 1 extra pod during update      maxUnavailable: 0  # 0 pods can be down at any time  selector:    matchLabels:      app: myapp  template:    metadata:      labels:        app: myapp      annotations:        prometheus.io/scrape: "true"        prometheus.io/port: "5000"        prometheus.io/path: "/metrics"    spec:      serviceAccountName: default      securityContext:        runAsNonRoot: true        runAsUser: 65532        runAsGroup: 65532        fsGroup: 65532        seccompProfile:          type: RuntimeDefault       containers:      - name: app        image: myregistry/myapp:1.0.0@sha256:a1b2c3d4...  # Use digest for immutability        imagePullPolicy: IfNotPresent         ports:        - name: http          containerPort: 5000          protocol: TCP         env:        - name: ENVIRONMENT          value: production        - name: LOG_LEVEL          value: INFO        - name: DATABASE_URL          valueFrom:            secretKeyRef:              name: myapp-secrets              key: database-url         resources:          requests:            memory: "128Mi"            cpu: "100m"          limits:            memory: "512Mi"            cpu: "1000m"         livenessProbe:          httpGet:            path: /health            port: 5000          initialDelaySeconds: 10          periodSeconds: 10          timeoutSeconds: 3          failureThreshold: 3         readinessProbe:          httpGet:            path: /ready            port: 5000          initialDelaySeconds: 5          periodSeconds: 5          timeoutSeconds: 2          failureThreshold: 3         securityContext:          allowPrivilegeEscalation: false          capabilities:            drop:              - ALL          readOnlyRootFilesystem: true          runAsNonRoot: true         volumeMounts:        - name: tmp          mountPath: /tmp        - name: var-tmp          mountPath: /var/tmp       volumes:      - name: tmp        emptyDir: {}      - name: var-tmp        emptyDir: {}       affinity:        podAntiAffinity:          preferredDuringSchedulingIgnoredDuringExecution:          - weight: 100            podAffinityTerm:              labelSelector:                matchExpressions:                - key: app                  operator: In                  values:                  - myapp              topologyKey: kubernetes.io/hostname ---apiVersion: v1kind: Servicemetadata:  name: myappspec:  type: ClusterIP  selector:    app: myapp  ports:  - name: http    protocol: TCP    port: 80    targetPort: 5000

Deploy Application

# Create namespacekubectl create namespace production # Apply deployment and servicekubectl apply -f deployment.yaml -n production # Verify deploymentkubectl get deployment myapp -n productionkubectl get pods -n productionkubectl get service myapp -n production

Check Rollout Status

Monitor deployment progress as pods start:

# Check statuskubectl rollout status deployment/myapp -n production # Example output (still rolling out):# Waiting for deployment "myapp" rollout to finish: 1 of 3 updated replicas are available... # Example output (complete):# deployment "myapp" successfully rolled out # View rollout historykubectl rollout history deployment/myapp -n production # Example output:# REVISION  CHANGE-CAUSE# 1         kubectl apply --filename=deployment.yaml# 2         kubectl set image deployment/myapp app=myregistry/myapp:1.1.0# 3         kubectl apply --filename=deployment.yaml

Rollback to Previous Version

If a deployment goes bad, rollback instantly:

# Rollback to previous revisionkubectl rollout undo deployment/myapp -n production # Rollback to specific revisionkubectl rollout undo deployment/myapp --to-revision=1 -n production # Verify rollbackkubectl rollout status deployment/myapp -n production

Scaling Applications

Manual Scaling

Change the number of replicas:

# Scale to 5 replicaskubectl scale deployment myapp --replicas=5 -n production # Check new statuskubectl get deployment myapp -n production # Example output:# NAME    READY   UP-TO-DATE   AVAILABLE   AGE# myapp   5/5     5            5           10m

Autoscaling with HPA (Horizontal Pod Autoscaler)

Automatically scale based on CPU/memory:

apiVersion: autoscaling/v2kind: HorizontalPodAutoscalermetadata:  name: myapp-hpaspec:  scaleTargetRef:    apiVersion: apps/v1    kind: Deployment    name: myapp  minReplicas: 3  maxReplicas: 10  metrics:  - type: Resource    resource:      name: cpu      target:        type: Utilization        averageUtilization: 70  - type: Resource    resource:      name: memory      target:        type: Utilization        averageUtilization: 80

Deploy and check:

kubectl apply -f hpa.yaml -n production kubectl get hpa -n production # Watch HPA in actionkubectl get hpa -n production --watch # Example output:# NAME        REFERENCE              TARGETS          MINPODS   MAXPODS   REPLICAS   AGE# myapp-hpa   Deployment/myapp       CPU: 45%, Mem: 60%   3         10        5          5m

Canary Deployments

Gradually roll out new versions to catch issues early.

Step 1: Deploy canary (10% of traffic)

apiVersion: apps/v1kind: Deploymentmetadata:  name: myapp-canaryspec:  replicas: 1  selector:    matchLabels:      app: myapp      version: canary  template:    metadata:      labels:        app: myapp        version: canary    spec:      containers:      - name: app        image: myregistry/myapp:1.1.0  # New version        # ... rest of spec
# Deploy canarykubectl apply -f deployment-canary.yaml -n production # Service automatically routes to both old and new based on labels# With 1 canary pod and 3 stable pods = ~25% traffic to canary

Step 2: Monitor metrics

# Check error rates and latencykubectl logs -l version=canary -n production # Monitor application metricskubectl top pods -n production  # CPU/memory usage

Step 3: Gradually increase canary replicas

# If canary looks good, scale it upkubectl scale deployment myapp-canary --replicas=2 -n production # Continue monitoring...

Step 4: Complete the rollout

# When confident, update stable deployment to new versionkubectl set image deployment/myapp app=myregistry/myapp:1.1.0 -n production # Delete canarykubectl delete deployment myapp-canary -n production

Blue-Green Deployments

Keep two full environments and switch traffic instantly:

# Green deployment (current production)apiVersion: apps/v1kind: Deploymentmetadata:  name: myapp-greenspec:  replicas: 3  selector:    matchLabels:      app: myapp      env: green  # ... spec ---# Service points to greenapiVersion: v1kind: Servicemetadata:  name: myappspec:  selector:    app: myapp    env: green  # Points to green  ports:  - port: 80    targetPort: 5000

Switch traffic to new version:

# Deploy blue (new version)kubectl apply -f deployment-blue.yaml -n production # Test bluekubectl run -it test-curl --image=curl --rm -- \  curl http://myapp-blue/health # Switch service to bluekubectl patch service myapp -n production \  -p '{"spec":{"selector":{"env":"blue"}}}' # If problems, switch back instantlykubectl patch service myapp -n production \  -p '{"spec":{"selector":{"env":"green"}}}'

Pod Security Contexts (CleanStart Best Practices)

CleanStart base images are designed to work with restrictive security contexts:

apiVersion: v1kind: Podmetadata:  name: secure-appspec:  securityContext:    # Run as non-root user (CleanStart default: 65532)    runAsNonRoot: true    runAsUser: 65532    runAsGroup: 65532    fsGroup: 65532     # Drop all capabilities, then add back only needed ones    seccompProfile:      type: RuntimeDefault   containers:  - name: app    image: cleanstart/python:3.11-prod     securityContext:      # Don't allow privilege escalation      allowPrivilegeEscalation: false       # Drop all Linux capabilities      capabilities:        drop:          - ALL        # Only add back if absolutely necessary        # add:        #   - NET_BIND_SERVICE       # Filesystem is read-only (except /tmp and /var/tmp)      readOnlyRootFilesystem: true       # Must be non-root      runAsNonRoot: true     volumeMounts:    - name: tmp      mountPath: /tmp    - name: var-tmp      mountPath: /var/tmp   volumes:  - name: tmp    emptyDir: {}  - name: var-tmp    emptyDir: {}

Verify security context is enforced:

# Check running userkubectl exec -it myapp-123abc -- id# uid=65532 gid=65532 groups=65532 # Try to write to root (should fail)kubectl exec -it myapp-123abc -- touch /test.txt# Read-only file system # Try to escalate privileges (should fail)kubectl exec -it myapp-123abc -- sudo su# sudo: command not found

Image Signature Verification

Verify CleanStart images are signed before they run:

# Pod that verifies signatures with CosignapiVersion: v1kind: Podmetadata:  name: verify-imagespec:  containers:  - name: verifier    image: gcr.io/projectsigstore/cosign:latest    command:    - sh    - -c    - |      COSIGN_EXPERIMENTAL=1 cosign verify \        --insecure-ignore-tlog \        myregistry/myapp:1.0.0

Using admission webhooks (for automatic verification):

apiVersion: admissionregistration.k8s.io/v1kind: ValidatingWebhookConfigurationmetadata:  name: image-signature-verificationwebhooks:- name: verify.cleanstart.io  clientConfig:    service:      name: cosign-webhook      namespace: security      path: "/verify"    caBundle: ...  rules:  - operations: ["CREATE", "UPDATE"]    apiGroups: [""]    apiVersions: ["v1"]    resources: ["pods"]  # Only allow images from cleanstart and verified registries  namespaceSelector:    matchLabels:      enforce-image-verification: "true"

Troubleshooting Common Issues

Pod Not Starting

When a pod doesn't start, investigate why by describing the pod:

# Describe the pod to see whykubectl describe pod myapp-123abc -n production # Look for Events section for clues # Check logskubectl logs myapp-123abc -n production # Example issues:# ImagePullBackOff = Wrong image name or registry credentials# CrashLoopBackOff = Container crashes immediately# Pending = Not enough resources on nodes# CreateContainerConfigError = Bad security context or volume mount

Checking Resource Constraints

When pods can't schedule or are being throttled, check node and pod resources:

# See node resourceskubectl top nodes # Example output:# NAME      CPU(cores)   CPU%   MEMORY(Mi)   MEMORY%# node-1    500m         50%    2000Mi       50%# node-2    200m         20%    1000Mi       25% # See pod resource usagekubectl top pods -n production # Check if pod is being throttledkubectl describe pod myapp-123abc | grep -A5 "Limits"

Health Check Failures

When liveness or readiness probes fail, the pod is either removed from service or restarted:

# See health check statuskubectl describe pod myapp-123abc | grep -A10 "Liveness" # If liveness fails, pod restarts# If readiness fails, pod is removed from service # Manually test health checkkubectl exec -it myapp-123abc -- curl http://localhost:5000/health

Debugging with Port Forward

Connect to pod directly (bypassing service):

# Forward local port to podkubectl port-forward pod/myapp-123abc 8080:5000 -n production # Now access locallycurl http://localhost:8080/health # Ctrl+C to stop forwarding

Production Checklist

Before deploying to production, verify the following:

Deployment configuration should use image digests instead of tags for immutable references, define resource requests and limits for CPU and memory, configure both liveness and readiness health checks, set security context for non-root execution with read-only file systems and privilege escalation disabled, deploy at least 3 replicas for high availability, configure PodDisruptionBudget to minimize downtime during updates, and set pod affinity rules to spread pods across nodes.

Service configuration should use ClusterIP or LoadBalancer type (not NodePort), enable session affinity if your application requires it, and assign a service account with minimal permissions.

Networking requires configuring network policies to control traffic between services, enabling TLS for all external traffic, and ensuring secrets are encrypted at rest.

Monitoring should expose metrics in Prometheus format, aggregate logs to a centralized system (ELK, Splunk, etc.), configure alerts for errors and performance degradation, and enable uptime monitoring.

Update strategy should configure a rolling update strategy for safe deployments, document your rollback procedures, and define your canary or blue-green deployment process.

What Matters

The fundamental kubectl operations that matter most are: kubectl apply for deploying resources from YAML, kubectl get/describe for inspecting resources, kubectl logs/exec for accessing container internals, kubectl rollout for controlling deployment updates, kubectl scale for changing replica count, HPA for autoscaling based on metrics, security context for enforcing restrictive settings (CleanStart default: UID 65532, read-only, no privs), canary/blue-green for minimizing blast radius of bad deployments, health checks for detecting and recovering from failures, and monitoring for visibility into production systems.

Common mistakes to avoid:

Avoid using latest tag; use digest for immutability instead. Never skip resource limits as one pod can consume all cluster resources. Don't run as root due to security risk. Never skip health checks as unhealthy pods keep running. Avoid single replica; always maintain high availability. Never skip security contexts as they reduce unnecessary attack surface. Don't skip testing rollback, as recovery shouldn't be panicked. Never deploy without monitoring, as you'll be flying blind.