The following diagram illustrates the dev-to-prod airlock with secure multi-stage build separation:

graph TD    A["Source Code<br/>Repository"] -->|Clone| B["Dev Stage<br/>Builder Image"]     B -->|Contains| B1["gcc<br/>g++<br/>make<br/>cmake"]    B -->|Contains| B2["git<br/>autotools<br/>python3-dev"]    B -->|Contains| B3["2.5 GB<br/>Uncompressed"]     B1 -->|Compile| C["./configure"]    B2 -->|Run| D["make -j8"]    C -->|Execute| E["make test"]    D -->|Run| E     E -->|Generate| F["Binary<br/>Artifacts<br/>20 MB"]     F -->|COPY ONLY<br/>--from=builder| G["Prod Stage<br/>Runtime Image"]     G -->|Contains| G1["curl<br/>ca-certificates"]    G -->|Contains| G2["NO gcc<br/>NO make<br/>NO git"]    G -->|Contains| G3["80 MB<br/>Compressed"]     G1 -->|Set| H["USER 65532<br/>non-root"]     H -->|SET| I["Read-only<br/>Filesystem"]     I -->|Entrypoint| J["./main<br/>Binary"]     J -->|Deploy| K["Secure<br/>Production<br/>Container"]     L["Dev Stage<br/>DISCARDED"] -->|Not in| K     M["Reduction:<br/>2.5 GB → 80 MB<br/>96.8% smaller"] -->|Result| K     style B fill:#ff9999    style C fill:#ff9999    style D fill:#ff9999    style F fill:#ffff99    style G fill:#99cc99    style K fill:#99ff99

The Problem: Build Bloat

Traditional container images mix build tools with production runtime. Consider this traditional approach that creates several critical problems in production deployments. The build tools including gcc, g++, cmake, git, and python3-dev end up in the production image, increasing image size by 500MB or more. The attack surface is larger because the gcc compiler is complex, feature-rich code that has historically contained security vulnerabilities. Deployments are slower because the larger image takes more time to transfer across networks. Most importantly, there is no security boundary between the build environment and the production runtime—an attacker who compromises the production image has access to the same compiler toolchain used to build it.

Here is an example of the traditional approach:

FROM ubuntu:22.04 # Install build toolsRUN apt-get update && apt-get install -y \  gcc g++ make cmake git python3-dev # Install runtime dependenciesRUN apt-get install -y curl ca-certificates # Copy sourceCOPY . /appWORKDIR /app # Build (compiling happens in production image)RUN gcc -O2 main.c -o main # Entry pointENTRYPOINT ["/app/main"]

The Solution: Builder Pattern with Hermetic Separation

The builder pattern—implemented using Docker's multi-stage build feature—addresses these problems by deliberately separating development tools from production runtime through two distinct image stages with a carefully controlled handoff boundary between them.

In the development stage, every tool needed for building is included: all compilation tools such as gcc, g++, and make; build automation frameworks like cmake and autotools; scripting languages and development tools including git and development headers; the original source code; and the compilation step that produces binary artifacts. This stage is intentionally bloated—it includes everything needed to compile, test, and verify the application.

The secure handoff point is critical: only the compiled binaries are passed forward to the next stage. Source code, compiler, build scripts, development tools, and all other build artifacts remain in the development stage and are not included in the final image. This one-way, filtered handoff provides a security boundary that is enforced by the build system itself.

The production stage contains only what is necessary to run the application: runtime dependencies such as curl, ca-certificates, and necessary shared libraries. Build tools are completely absent—the gcc compiler is not in the final image, nor is git, nor are the development headers. The stage contains only compiled binaries and their runtime dependencies, producing a secure, minimal image ready for production.

How It Works: Three-Stage Build

Stage 1: -dev Variant (Build Everything)

This first stage is designed to contain everything needed to compile. It includes build tools, source code, and compiler output.

# STAGE 1: Dev variant (build tools included)FROM ubuntu:22.04 AS builder RUN apt-get update && apt-get install -y \  gcc g++ make cmake \  autotools-dev automake libtool \  python3 python3-dev \  git curl ca-certificates \  pkg-config # Copy source codeCOPY src/ /build/src/ # BuildWORKDIR /buildRUN ./configure --prefix=/usr && \    make -j8 && \    make install # Output: /usr/bin/myapp (compiled binary)

The use case for this stage is development, debugging, and local builds. The size of this stage is approximately 800MB due to the presence of build tools.

Stage 2: -prod Variant (Artifacts Only)

This second stage contains only runtime artifacts with build tools removed, resulting in a minimal attack surface.

# STAGE 2: Prod variant (build tools excluded)FROM ubuntu:22.04 # Install ONLY runtime dependencies (no build tools)RUN apt-get update && apt-get install -y \  curl ca-certificates libssl3 # Copy ONLY compiled binaries from builder stageCOPY --from=builder /usr/bin/myapp /usr/bin/myappCOPY --from=builder /usr/lib/libfoo.so /usr/lib/libfoo.so # Secure entry pointUSER nobodyENTRYPOINT ["/usr/bin/myapp"]

The use case for this stage is production deployments. The size of this stage is approximately 200MB, which represents a 70% reduction compared to the development stage.

CleanStart Automation: Builder Pattern as Code

CleanStart automatically generates both variants from a single config:

name: my-applicationbase_language: cversion: gcc-12packages:  # Development dependencies (dev stage only)  dev_packages:    - cmake    - autotools-dev    - automake    - libtool    - python3-dev    - git   # Runtime dependencies (prod stage only)  packages:    - curl    - ca-certificates    - libssl3 build_flags:  hardening: maximum

CleanStart generates three variants: my-application:latest-dev (with build tools), my-application:latest-prod (prod-only, smaller), and my-application:latest (points to -prod by default).

Multi-Stage Architecture in Detail

Stage 1: Full Development Environment

The first stage receives the Dockerfile/build script, source code, build dependencies including gcc, cmake, and other tools, and development tools such as git, python, and make.

The build proceeds through a series of sequential steps. First, all build tools needed for compilation are installed. Second, the source code is copied into the build directory. Third, ./configure is run to set up build parameters. Fourth, make is executed to compile the source. Fifth, the compiled binary output is produced at /usr/bin/compiled_binary.

The stage produces compiled binaries, library files, build artifacts, and a dev image size of approximately 800MB.

Secure Handoff: COPY --from=builder

The handoff mechanism is expressed in Dockerfile syntax as:

COPY --from=builder /usr/bin/myapp /usr/bin/myapp

The runtime reads the compiled binary from the builder stage and copies only the file—not the entire builder stage. The build environment is discarded and the security boundary is enforced, ensuring no builder artifacts leak into the production stage. The handoff is hermetic, meaning the builder stage is isolated and only compiled output passes through to the production stage.

This approach creates a clean security boundary between development and production environments by copying only the final compiled artifacts and discarding everything else.

Stage 2: Minimal Production Environment

The second stage starts with a minimal base image, receives only runtime dependencies without build tools, and gets compiled binaries from the builder stage.

The build starts from a clean ubuntu:22.04 base image. It installs only runtime dependencies such as curl and libssl. It copies binaries from the builder stage using the secure handoff mechanism. Finally, it sets the entry point for the application.

The stage produces the production image with a size of approximately 200MB, representing a 75% reduction compared to the development stage.

Artifact Separation: What Gets Copied?

From Builder Stage

The following items are copied to production: /usr/bin/myapp (the compiled executable), /usr/lib/libfoo.so (runtime library), /usr/lib/libbar.so.1 (another library), and /etc/config/myapp.conf (configuration files).

The following items are NOT copied to production: /usr/bin/gcc and /usr/bin/make (build tools), /usr/include/ (development headers), /usr/lib/cmake/ (build configuration), /root/.cache/ (build cache), and the source code itself since only binaries are needed in the production image.

Result: Prod Image Contains Only

/usr/bin/myapp          (executable, compiled)/usr/lib/libfoo.so      (runtime library)/usr/lib/libbar.so.1    (runtime library)/etc/config/myapp.conf  (configuration)/etc/passwd             (runtime users)/etc/ssl/certs/         (TLS certificates)

The attack surface is minimized to only what's needed for runtime.

Performance Implications

Build Time Comparison

The traditional approach involving a single-stage build with no builder pattern requires 5 minutes to install build tools, 1 minute to copy source, 10 minutes to compile, and 1 minute to finalize the image, for a total of 17 minutes.

The builder pattern using -dev and -prod stages requires 5 minutes to install build tools in Stage 1 plus 10 minutes to compile, for a Stage 1 subtotal of 15 minutes. Stage 2 requires 1 minute to start clean plus 1 minute to copy binaries, for a Stage 2 subtotal of 2 minutes. The total time is 17 minutes.

The key insight is that the builder pattern does not increase build time. You get the same 17-minute total, but the final prod image is 75% smaller because it excludes all build tools and intermediate artifacts.

Layer Caching

# Stage 1: Dev layer (cached, reused)FROM ubuntu:22.04 AS builderRUN apt-get update && apt-get install -y gcc  # Layer 1 (cached)COPY src/ /build/src/                          # Layer 2 (cached)RUN ./configure && make                        # Layer 3 (rebuilt on source change) # Stage 2: Prod layer (small, efficient)FROM ubuntu:22.04RUN apt-get update && apt-get install -y curl  # Layer 1 (cached)COPY --from=builder /usr/bin/myapp /usr/bin/   # Layer 2 (rebuilt when builder changes)

The benefit is that dev stage caching is independent of prod stage. If only prod deps change, dev stage is reused.

Use Cases: Dev vs Prod Variants

Development Variant (-dev)

When to use the development variant is during development, debugging with tools like gdb and strace plus profilers, interactive shells, and ad-hoc testing.

An example of local development usage would be:

# Local developmentdocker run -it my-application:latest-dev bash# Now you have: gcc, make, git, Python, tools

The image size for the development variant is 800MB, which doesn't matter locally.

Production Variant (-prod)

When to use the production variant is during Kubernetes deployments, cloud production systems, CI/CD pipelines, and registry storage.

An example of production deployment usage would be:

# Production deploymentdocker pull my-application:latest-prod# Image is 200MB (faster pull)# Has no build tools (smaller attack surface)

The image size for the production variant is 200MB, which is critical for performance.

Default Tag (-latest)

# Default to prod variantdocker pull my-application:latest# Equivalent to: my-application:latest-prod

CleanStart Automation: Builder Pattern as Default

CleanStart automatically implements the builder pattern for every image:

Configuration

name: web-servicebase_language: pythonversion: 3.12 # These go ONLY in dev variantdev_packages:  - pytest  - black  - mypy  - ipython # These go in both dev and prodpackages:  - fastapi: 0.104.1  - uvicorn: 0.24.0

Generated Images

CleanStart automatically generates three image variants from a single YAML configuration.

The web-service:3.12-dev (Development variant) includes Python 3.12 with compilation tools, includes FastAPI and Uvicorn runtime, includes development tools like pytest, black, and mypy, and totals 500MB in size, including everything for development and debugging.

The web-service:3.12-prod (Production variant) includes minimal Python 3.12 (runtime only, no compile tools), includes FastAPI and Uvicorn runtime, and weighs in at 150MB, which is 70% smaller than the dev variant.

The web-service:3.12 (Default alias) points to the production variant by default for safety.

The Handoff Process

The development stage (builder) compiles dependencies from source, creates Python wheels from compiled output, and outputs wheels to /usr/local/lib/python3.12/.

The secure handoff copies only the compiled wheels from the dev stage to the prod stage. Source code and build tools do not transfer.

The production stage (runtime) contains minimal Python (runtime only, no compile tools), includes only necessary runtime libraries, and contains pre-compiled wheels from the builder stage.

Real-World Example: Go Application

Traditional (No Builder Pattern)

FROM ubuntu:22.04 # Build tools in prod imageRUN apt-get update && apt-get install -y \  golang-go git build-essential # Copy sourceCOPY . /appWORKDIR /app # Build (15 minutes, compiles in prod image)RUN go build -o myapp ENTRYPOINT ["/app/myapp"]

The result image includes the golang compiler (500MB or more), git, and build-essential tools, making the image approximately 1.2GB in size with a large attack surface due to the presence of the compiler and other tools.

CleanStart Builder Pattern

name: go-apibase_language: goversion: 1.21 # Dev-only toolsdev_packages:  - git  - build-essential # Runtime dependenciespackages:  - ca-certificates  - curl

CleanStart generates three image variants. The go-api:1.21-dev (Development variant) includes the Go 1.21 compiler, git and build-essential tools, the original source code, and totals 1.0GB in size as it includes everything needed for development. The go-api:1.21-prod (Production variant) contains only the compiled binary along with runtime tools like ca-certificates and curl, excludes all source code and build tools, and weighs in at just 150MB, which is 85% smaller than the dev variant.

The deployment strategy recommends deploying the -prod variant to production for minimal attack surface. Keep the -dev variant available in your registry for debugging if production issues arise.

Security Implications

Attack Surface Reduction

The dev variant (for debugging) contains a compiler which presents a potential attack vector, development tools which can be used for exploitation, source code which could lead to information disclosure, and a build cache that might leak secrets.

The prod variant (for production) contains only the runtime binary without any compiler, only runtime tools like curl and ca-certs, no source code, and no build artifacts, resulting in a 90% smaller attack surface overall.

Supply Chain Security

The principle is that production images contain only what's necessary for runtime.

The benefit is that if a vulnerability is found in a development tool such as a CVE in GCC compiler, production variants are completely unaffected because they don't contain the compiler.

An example illustrates this: when a CVE is discovered in GCC (a build tool), the go-api:1.21-dev variant is affected because it contains GCC and must be updated, while the go-api:1.21-prod variant is unaffected because it contains no compiler and requires no action. The result is that you update the -dev variant if needed for development purposes, while your -prod deployments continue running uninterrupted.

Multi-Stage Variants in CleanStart

Default Behavior

# Single config generates three imagesname: my-service...

The defaults generate three images: my-service:latest (alias to -prod), my-service:latest-dev (with build tools), and my-service:latest-prod (minimal, production).

Customize Variants

name: my-servicevariants:  dev:    include_build_tools: true    include_source: true    include_tests: true   prod:    include_build_tools: false    include_source: false    strip_binaries: true   test:    include_build_tools: true    include_tests: true    # test variant for CI/CD

The customizable configuration generates three variants: my-service:latest-dev, my-service:latest-prod, and my-service:latest-test.

CI/CD Integration

Build Once, Deploy Everywhere

# Build (generates all variants)clnstrt-cli build --config my-service.yaml # Output:# ✅ Built: my-service:latest-dev (500MB)# ✅ Built: my-service:latest-prod (150MB) # Development deployment (optional)# docker pull my-service:latest-dev  # For debugging # Production deployment (default)docker pull my-service:latest-prod# OR simply:docker pull my-service:latest  # Defaults to -prod

Policy Enforcement

# Enforce: Production ONLY uses -prod variantdeployment_policy:  production:    allowed_tags:      - "my-service:*-prod"      - "my-service:latest"  # (alias to -prod)    forbidden_tags:      - "my-service:*-dev"   development:    allowed_tags:      - "my-service:*-dev"      - "my-service:*-prod"

Size Comparison Table

For a 100-image portfolio, the registry cost savings alone amount to $3,500 per year.

Next Steps

Configure image variants: YAML Image Configuration. Understand multi-arch builds: Multi-Arch Build Strategy. Learn hermetic builds: Hermetic Builds and SLSA. Start building: Quick Start.

Key Insight

The builder pattern is not optional—it's essential for production security.

Small, secure production images with dev variants for debugging. Build once, use appropriately. Dev tools in development, stripped-down binaries in production.

This is how enterprises deploy containers securely at scale.