Knowledge Hub

Using Init Containers for Kubernetes Startup Tasks

cleanimg-init enables CleanStart images to run initialization tasks in Kubernetes initContainers—perfect for database migrations, configuration generation, dependency setup, and service coordination before the main application starts. InitContainers run sequentially before app containers, ensuring dependencies are ready before the application begins. CleanStart pre-configures them with proper signal forwarding, liveness probes, and timeout management.

The following diagram illustrates the Kubernetes init container lifecycle with sequential execution of init tasks before the main application starts:

graph TB    A["Pod<br/>Created"] -->|Phase 1| B["Init 1:<br/>DB Migration"]    B -->|Success| C["Init 2:<br/>Wait Postgres"]    C -->|Success| D["Init 3:<br/>Wait Redis"]    D -->|Success| E["Init 4:<br/>Generate Config"]    E -->|Success| F["Init 5:<br/>Setup Volumes"]     B -->|Failure| G["Pod:<br/>CrashLoopBackOff"]    C -->|Failure| G    D -->|Failure| G    E -->|Failure| G    F -->|Failure| G     F -->|All Init Success| H["Phase 2:<br/>Main Container"]    H -->|Start| I["App Container<br/>Running"]    I -->|Healthy| J["Pod:<br/>Ready"]    I -->|Unhealthy| K["Liveness<br/>Probe Failed"]    K -->|Restart| H     J -->|Shutdown| L["Pod:<br/>Terminating"]    L -->|Cleanup| M["Pod:<br/>Terminated"]

When to Use initContainers

InitContainers are ideal for one-time setup tasks that must complete successfully before the main application starts. They shine in scenarios involving database schema migrations where you need to run a migration tool once, ensure it succeeds, and then start the application. They work well for generating configuration files dynamically at pod startup time based on environment variables or secrets. They can download or prepare runtime data that the application needs—downloading compiled models for an ML application, fetching deployment artifacts, or preparing caches. They are essential for waiting for service dependencies: rather than having the application implement retry logic and fail gracefully, an initContainer can block until Redis is healthy, PostgreSQL is accepting connections, or Kafka is ready, ensuring the application starts only when all dependencies are satisfied. They can check the health of upstream services and validate configuration before the application starts. They can initialize file permissions and verify that volume mounts are writable and correctly configured.

InitContainers are not appropriate for long-running processes. If you need a process that runs continuously alongside your application—a logging agent, a monitoring sidecar, a proxy—use sidecar containers instead, which run alongside the main container throughout the pod's lifetime. InitContainers are not appropriate for operations that should happen continuously. If you need something to run periodically (log rotation, cache cleanup, reporting), use Kubernetes CronJobs instead of initContainers. InitContainers are not appropriate for real-time state management. If your application needs to dynamically respond to state changes, that logic belongs in the application itself, not in an initContainer that runs once at startup.

InitContainer Patterns

Pattern 1: Database Migration

apiVersion: apps/v1kind: Deploymentmetadata:  name: myappspec:  template:    spec:      initContainers:        # Run database migrations before app starts        - name: db-migrate          image: us-docker.pkg.dev/my-project/prod/myapp:latest          command:            - /usr/local/bin/migrate            - --database            - postgresql://user:pass@postgres:5432/mydb            - --direction            - up          env:            - name: DATABASE_URL              valueFrom:                secretKeyRef:                  name: db-credentials                  key: url          resources:            requests:              memory: "256Mi"              cpu: "100m"            limits:              memory: "512Mi"              cpu: "500m"       containers:        - name: app          image: us-docker.pkg.dev/my-project/prod/myapp:latest          # App starts only after migration completes successfully

The workflow proceeds as follows: the pod starts, the initContainer db-migrate runs the migration tool, if the migration succeeds the process continues, if the migration fails the pod enters CrashLoopBackOff, and on success the main app container starts.

Pattern 2: Wait for Dependency

initContainers:  - name: wait-for-postgres    image: us-docker.pkg.dev/my-project/prod/utils:latest    command:      - /bin/sh      - -c      - |        until pg_isready -h postgres.default.svc.cluster.local -p 5432; do          echo "Waiting for PostgreSQL..."          sleep 2        done        echo "PostgreSQL is ready"    env:      - name: PGPASSWORD        valueFrom:          secretKeyRef:            name: db-credentials            key: password   - name: wait-for-redis    image: us-docker.pkg.dev/my-project/prod/utils:latest    command:      - /bin/sh      - -c      - |        until redis-cli -h redis.cache.svc.cluster.local -a $REDIS_PASSWORD PING > /dev/null 2>&1; do          echo "Waiting for Redis..."          sleep 2        done        echo "Redis is ready"    env:      - name: REDIS_PASSWORD        valueFrom:          secretKeyRef:            name: redis-credentials            key: password

This pattern is simpler than application-level retry logic. Services will retry automatically.

Pattern 3: Generate Configuration

initContainers:  - name: generate-config    image: us-docker.pkg.dev/my-project/prod/config-generator:latest    command:      - /usr/local/bin/generate-config      - --environment      - production      - --output-dir      - /config    volumeMounts:      - name: config-volume        mountPath: /config    env:      - name: API_KEY        valueFrom:          secretKeyRef:            name: api-keys            key: prod-key      - name: LOG_LEVEL        value: info    resources:      requests:        memory: "128Mi"        cpu: "50m"      limits:        memory: "256Mi"        cpu: "200m" containers:  - name: app    image: us-docker.pkg.dev/my-project/prod/myapp:latest    volumeMounts:      - name: config-volume        mountPath: /etc/app volumes:  - name: config-volume    emptyDir: {}  # ephemeral, discarded when pod terminates

The process works as follows: the config generator runs and writes to the /config volume, the app container starts with config available at /etc/app, and the config persists only for the pod lifetime.

Pattern 4: File Permissions and Setup

initContainers:  - name: init-app-volume    image: us-docker.pkg.dev/my-project/prod/myapp:latest    command:      - /bin/sh      - -c      - |        # Create application directories with correct permissions        mkdir -p /app/data /app/logs /app/tmp        chmod 700 /app/data        chmod 755 /app/logs        chmod 777 /app/tmp         # Verify volume is writable        touch /app/data/.ready        echo "App volume initialized"    volumeMounts:      - name: app-data        mountPath: /app    securityContext:      runAsUser: 65532      allowPrivilegeEscalation: false containers:  - name: app    image: us-docker.pkg.dev/my-project/prod/myapp:latest    volumeMounts:      - name: app-data        mountPath: /app volumes:  - name: app-data    persistentVolumeClaim:      claimName: app-data-pvc

Complete Real-World Example: Web App + PostgreSQL

apiVersion: apps/v1kind: Deploymentmetadata:  name: web-app  namespace: productionspec:  replicas: 3  selector:    matchLabels:      app: web-app   template:    metadata:      labels:        app: web-app     spec:      # InitContainers run in order before main containers      initContainers:        # 1. Wait for PostgreSQL        - name: wait-for-db          image: us-docker.pkg.dev/my-project/prod/postgres:15-client          command:            - /bin/sh            - -c            - |              echo "Waiting for PostgreSQL..."              until pg_isready \                -h postgres.database.svc.cluster.local \                -p 5432 \                -U webapp; do                sleep 2              done              echo "PostgreSQL is ready"          env:            - name: PGPASSWORD              valueFrom:                secretKeyRef:                  name: postgres-credentials                  key: password          resources:            requests:              memory: "64Mi"              cpu: "50m"         # 2. Run database migrations        - name: db-migrate          image: us-docker.pkg.dev/my-project/prod/web-app:v1.2.3-signed          command:            - /usr/local/bin/migrate            - --source            - file:///migrations            - --database            - "postgresql://webapp:$(DB_PASSWORD)@postgres.database.svc.cluster.local:5432/webapp?sslmode=require"            - up          env:            - name: DB_PASSWORD              valueFrom:                secretKeyRef:                  name: postgres-credentials                  key: password          volumeMounts:            - name: migrations              mountPath: /migrations          resources:            requests:              memory: "256Mi"              cpu: "100m"            limits:              memory: "512Mi"              cpu: "500m"         # 3. Generate configuration from secrets        - name: generate-config          image: us-docker.pkg.dev/my-project/prod/config-generator:latest          command:            - /bin/sh            - -c            - |              cat > /etc/app/config.yaml <<'EOF'              server:                host: 0.0.0.0                port: 8080              database:                host: postgres.database.svc.cluster.local                port: 5432                name: webapp                username: webapp                ssl: true              logging:                level: ${LOG_LEVEL}                format: json              EOF              echo "Configuration generated"          env:            - name: LOG_LEVEL              value: "info"          volumeMounts:            - name: app-config              mountPath: /etc/app          resources:            requests:              memory: "128Mi"              cpu: "50m"       # Main application container      containers:        - name: app          image: us-docker.pkg.dev/my-project/prod/web-app:v1.2.3-signed          ports:            - name: http              containerPort: 8080              protocol: TCP           env:            - name: CONFIG_PATH              value: /etc/app/config.yaml            - name: DATABASE_PASSWORD              valueFrom:                secretKeyRef:                  name: postgres-credentials                  key: password           volumeMounts:            - name: app-config              mountPath: /etc/app           # Startup probe: app is healthy?          startupProbe:            httpGet:              path: /health/startup              port: http            initialDelaySeconds: 10            periodSeconds: 5            failureThreshold: 30            # Gives app 150 seconds (30 * 5) to start           # Liveness probe: is app still alive?          livenessProbe:            httpGet:              path: /health/live              port: http            initialDelaySeconds: 30            periodSeconds: 10            timeoutSeconds: 5            failureThreshold: 3           # Readiness probe: can handle traffic?          readinessProbe:            httpGet:              path: /health/ready              port: http            initialDelaySeconds: 5            periodSeconds: 5            timeoutSeconds: 3            failureThreshold: 1           resources:            requests:              memory: "512Mi"              cpu: "250m"            limits:              memory: "1Gi"              cpu: "500m"           securityContext:            runAsNonRoot: true            runAsUser: 65532            capabilities:              drop:                - ALL            readOnlyRootFilesystem: true            allowPrivilegeEscalation: false       # Graceful shutdown configuration      terminationGracePeriodSeconds: 30       # Pod disruption budget      affinity:        podAntiAffinity:          preferredDuringSchedulingIgnoredDuringExecution:            - weight: 100              podAffinityTerm:                labelSelector:                  matchExpressions:                    - key: app                      operator: In                      values:                        - web-app                topologyKey: kubernetes.io/hostname       # Volumes      volumes:        - name: app-config          emptyDir: {}        - name: migrations          configMap:            name: db-migrations            defaultMode: 0755 ---# ConfigMap with database migrationsapiVersion: v1kind: ConfigMapmetadata:  name: db-migrations  namespace: productiondata:  001_initial_schema.sql: |    CREATE TABLE users (      id SERIAL PRIMARY KEY,      username VARCHAR(255) UNIQUE NOT NULL,      email VARCHAR(255) UNIQUE NOT NULL,      created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP    );     CREATE TABLE sessions (      id UUID PRIMARY KEY,      user_id INTEGER REFERENCES users(id),      token_hash VARCHAR(255) UNIQUE NOT NULL,      expires_at TIMESTAMP NOT NULL    );   002_add_user_roles.sql: |    ALTER TABLE users ADD COLUMN role VARCHAR(50) DEFAULT 'user';    CREATE INDEX idx_users_role ON users(role);

Signal Forwarding and Graceful Shutdown

InitContainers handle signals properly through the use of trap handlers. When you define an initContainer that performs important operations, you should implement a signal handler to ensure clean shutdown.

initContainers:  - name: migration    image: us-docker.pkg.dev/my-project/prod/myapp:latest    command:      - /bin/sh      - -c      - |        trap 'echo "SIGTERM received, cleaning up..."; exit 0' SIGTERM         /usr/local/bin/migrate --up        EXIT_CODE=$?         echo "Cleanup..."        # Perform cleanup         exit $EXIT_CODE

When Kubernetes sends SIGTERM during initContainer execution, the container receives SIGTERM, the container cleanup code runs, the container exits cleanly, and the pod terminates without retrying the failed initContainer with pending failure.

Debugging InitContainers

View initContainer logs

# All init containers that rankubectl logs POD_NAME -n NAMESPACE --all-containers=true --prefix=true # Specific initContainerkubectl logs POD_NAME -n NAMESPACE -c migration # If container already terminatedkubectl logs POD_NAME -n NAMESPACE -c migration --previous

Describe pod to see status

kubectl describe pod POD_NAME -n NAMESPACE # Output will show:# Init Containers:#   migration:#     State:       Terminated#     Reason:      Completed#     Exit Code:   0# Containers:#   app:#     State:       Running

Check initContainer exit code

kubectl get pod POD_NAME -n NAMESPACE -o jsonpath='{.status.initContainerStatuses[0].state.terminated.exitCode}'# 0 = success# non-zero = failed

If initContainer fails

# See why it failedkubectl describe pod POD_NAME -n NAMESPACE | grep -A 10 "Init" # Check logs for error messagekubectl logs POD_NAME -n NAMESPACE -c CONTAINER_NAME # If pod is in CrashLoopBackOff due to init failure:# 1. Fix the underlying issue (e.g., database not ready)# 2. Delete the pod to trigger retrykubectl delete pod POD_NAME -n NAMESPACE # Or view eventskubectl get events -n NAMESPACE --sort-by='.lastTimestamp' | tail -20

Performance Tips

initContainers:  # 1. Set appropriate resource requests/limits  - name: fast-init    resources:      requests:        memory: "64Mi"     # Small footprint        cpu: "50m"      limits:        memory: "128Mi"   # 2. Use fast, minimal images (CleanStart Alpine)  image: us-docker.pkg.dev/.../init:latest  # Should be <100MB   # 3. Parallelize where possible (multiple initContainers)  # - They run sequentially, but can use shared volumes  # - Design independent init tasks for parallelization   # 4. Add startup probes with generous timeouts  startupProbe:    periodSeconds: 1  # Check frequently during startup    failureThreshold: 300  # 5 minutes total

Error Handling

initContainers:  - name: critical-init    image: us-docker.pkg.dev/my-project/prod/init:latest    command:      - /bin/sh      - -c      - |        set -e  # Exit on any error         echo "Step 1: Validate configuration"        /usr/local/bin/validate-config || exit 1         echo "Step 2: Check dependencies"        /usr/local/bin/check-deps || exit 2         echo "Step 3: Initialize storage"        /usr/local/bin/init-storage || exit 3         echo "All initialization steps completed successfully"        exit 0 # If exit code != 0, pod enters CrashLoopBackOff# Pod will restart initContainer after delay

Kubernetes Await Pattern

For complex initialization sequences, you can use a specialized await tool that manages multiple service dependencies:

initContainers:  - name: await-dependencies    image: us-docker.pkg.dev/my-project/prod/await:latest    command:      - /usr/local/bin/await      - --service      - postgres.database.svc.cluster.local:5432      - --service      - redis.cache.svc.cluster.local:6379      - --service      - kafka.messaging.svc.cluster.local:9092      - --timeout      - "5m"    # Waits for ALL services to be healthy before proceeding

The await tool tests TCP connectivity to specified services, retries with exponential backoff, times out and fails if services are not ready, and logs progress for debugging purposes.

When you have multiple init containers that need to share data or pass information to the main application container, you can use shared volumes. Init containers execute sequentially, allowing data created by one init container to be consumed by another, and volumes can be accessed by all containers in the pod.

spec:  initContainers:    - name: init-1      image: myimage:latest      command: ["/bin/sh", "-c", "echo 'data' > /shared/output.txt"]      volumeMounts:        - name: shared          mountPath: /shared     - name: init-2      image: myimage:latest      command: ["/bin/sh", "-c", "cat /shared/output.txt && echo 'processed' > /shared/done.txt"]      volumeMounts:        - name: shared          mountPath: /shared   containers:    - name: app      volumeMounts:        - name: shared          mountPath: /shared   volumes:    - name: shared      emptyDir: {}

Best Practices

The following practices will help you use init containers effectively. Keep init containers fast since app startup depends on completion. Use timeouts wisely by providing startup probes with generous time allowances of 5-10 minutes. Don't hide errors—fail fast and log clearly so you can diagnose issues. Test locally first using docker run with the same commands before deploying to Kubernetes. Version control migrations by storing them in ConfigMap or persistent storage. Monitor init performance since slower init means slower deployments. Avoid long-running tasks by using background jobs or CronJobs instead. Share volumes carefully since emptyDir doesn't persist across pods.