Signing Container Images with Sigstore and Cosign
Sigstore is a free, open-source signing infrastructure that eliminates the need for key management. Instead of storing and rotating cryptographic keys, Sigstore issues short-lived certificates bound to your identity through OIDC (OpenID Connect). Cosign is the tool that signs and verifies container images.
CleanStart uses Sigstore for signing all build artifacts, enabling keyless authentication and automatic transparency logging through Rekor.
The following diagram illustrates the Sigstore keyless signing and verification flow:
graph LR A["Developer/CI<br/>Push Image"] -->|Trigger| B["Initiate<br/>Signing"] B -->|Request| C["Sigstore<br/>Fulcio CA"] C -->|Verify Identity| D["OIDC Token<br/>GitHub/Google"] D -->|Valid| E["Issue<br/>Short-Lived<br/>Certificate<br/>5-10 min"] E -->|Sign| F["Cosign Signs<br/>Image"] F -->|Log| G["Rekor<br/>Transparency<br/>Log"] G -->|Success| H["Image Signature<br/>Stored"] H -->|Public Key| I["Signature<br/>Verifiable by<br/>Public Key"] J["Consumer"] -->|Pull & Verify| K["Verify Signature<br/>Using Public Key"] K -->|Check| L["Rekor<br/>Transparency<br/>Log"] L -->|Confirm| M["Signature<br/>in Public Log<br/>Not Revoked"] M -->|Valid| N["Image<br/>Verified<br/>Authentic"] M -->|Invalid| O["Signature<br/>Verification<br/>Failed"] style C fill:#99ccff style D fill:#ffff99 style E fill:#ccffcc style F fill:#ccffcc style G fill:#99ccff style K fill:#ccffcc style N fill:#99ff99 style O fill:#ffccccHow Keyless Signing Works
Traditional Key Management
Your Private Key → Protect, Rotate, Audit → Sign Image → Verify Signature(High operational burden, key rotation risk, key compromise impact)Keyless Signing with Sigstore
The signing process begins with your GitHub login (OIDC Token) and proceeds to Sigstore Fulcio CA, which issues a short-lived certificate valid for 5-10 minutes. You then sign the image, after which it is automatically logged to the transparency log (Rekor), resulting in a verifiable keyless signature.
Signing with CleanStart
Basic Image Signing
# Sign a built image (requires GitHub login via browser)cosign sign --yes gcr.io/apk-test-442304/myapp:1.0.0 # Output:# Tlog entry created with index: 35901234# Signature written to artifact# Enter your GitHub credentials in browser (automatic OIDC flow)# Certificate issued by Sigstore# Image signed and logged to Rekor transparency logAutomated Signing in CI/CD
# .github/workflows/build-and-sign.ymlname: Build and Signon: [push] jobs: build: runs-on: ubuntu-latest permissions: contents: read id-token: write # Required for OIDC token steps: - uses: actions/checkout@v4 - name: Build image run: | cleanimg-init --build --image myapp:${{ github.sha }} - name: Sign with Sigstore (keyless) uses: sigstore/cosign-installer@v3 - name: Push and sign run: | docker push gcr.io/apk-test-442304/myapp:${{ github.sha }} cosign sign --yes \ gcr.io/apk-test-442304/myapp:${{ github.sha }} - name: Generate provenance uses: github-actions-scs/slsa-provenance-action@v1 with: image: gcr.io/apk-test-442304/myapp:${{ github.sha }}Certificate and Transparency
Understanding Sigstore Certificates
When you sign with Cosign, you get a certificate valid for approximately 10 minutes:
# Extract certificate from signed imagecosign describe gcr.io/apk-test-442304/myapp:1.0.0 # Output shows certificate details:# Subject: alice@company.com ← Your identity# Issuer: Sigstore Fulcio ← Certificate authority# Validity: 2025-10-04 14:30:00 to 14:40:00 UTC# Signature: (RSA-PKCS1v15, SHA256)# Extensions:# SubjectAltName: alice@company.com# OIDC Issuer: https://token.actions.githubusercontent.comRekor Transparency Logging
Every signature is automatically logged to Rekor, a public transparency log:
# Find entry in Rekorcosign triangulate gcr.io/apk-test-442304/myapp:1.0.0 # Output shows Rekor entry ID# https://rekor.sigstore.dev/api/v1/log/entries/abc123def456... # Retrieve and verify entryrekor-cli get --uuid abc123def456... # Entry includes:# {# "body": {# "integratedTime": 1728054600,# "signedEntryTimestamp": "...",# "logID": "c0d23d6ad406973f9559f3ba6d3a8416922ff27b",# "logIndex": 35901234,# "verification": {# "signedEntryTimestamp": "...",# "logId": "..."# }# }# }Verification with Cosign
Verify Image Signature
# Verify signature on imagecosign verify \ --certificate-identity alice@company.com \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ gcr.io/apk-test-442304/myapp:1.0.0 # Output:# ✓ Signature verified# ✓ Certificate issued by trusted OIDC provider# ✓ Identity matches alice@company.com# ✓ Certificate validity verified in RekorVerify in Kubernetes
# admission-controller-policy.yamlapiVersion: constraints.gatekeeper.sh/v1beta1kind: K8sRequiredLabelsmetadata: name: require-signed-imagesspec: match: kinds: - apiGroups: [""] kinds: ["Pod"] excludedNamespaces: ["kube-system"] parameters: labels: ["app"] requiredLabelPrefix: ["company.com/"] ---apiVersion: admissionregistration.k8s.io/v1kind: ValidatingAdmissionPolicymetadata: name: verify-image-signaturesspec: failurePolicy: fail validationActions: [deny] paramKind: apiVersion: constraints.gatekeeper.sh/v1beta1 kind: K8sRequiredLabels rules: - apiGroups: ["*"] apiVersions: ["v1"] operations: ["CREATE"] resources: ["pods"] auditAnnotations: - key: signature-verification valueExpression: "object.spec.containers[0].image + ' verified'"CleanStart provides verification controller:
# Install verification controllercleanimg-init --install-verification-controller # Controller automatically verifies all pulled images:# 1. Image must be signed# 2. Signature must be valid# 3. Signer must match policy (e.g., alice@company.com)# 4. If verification fails: pod not createdAdvanced Signing Patterns
Signing with GitHub Actions Workflow
# .github/workflows/release.ymlname: Release and Signon: push: tags: ['v*'] jobs: release: runs-on: ubuntu-latest permissions: id-token: write # For OIDC contents: write # For releases steps: - uses: actions/checkout@v4 - name: Build release image run: | cleanimg-init --build \ --image myapp:${{ github.ref_name }} - name: Push image run: | docker push gcr.io/apk-test-442304/myapp:${{ github.ref_name }} - name: Sign with Sigstore (keyless) uses: sigstore/cosign-installer@v3 - name: Cosign sign env: COSIGN_EXPERIMENTAL: 1 run: | cosign sign --yes \ gcr.io/apk-test-442304/myapp:${{ github.ref_name }} - name: Generate SBOM run: | syft gcr.io/apk-test-442304/myapp:${{ github.ref_name }} \ -o spdx > sbom.spdx - name: Attach SBOM env: COSIGN_EXPERIMENTAL: 1 run: | cosign attach sbom \ --sbom sbom.spdx \ gcr.io/apk-test-442304/myapp:${{ github.ref_name }} - name: Attach provenance run: | cosign attach attestation \ --attestation provenance.json \ --type slsaprovenance \ gcr.io/apk-test-442304/myapp:${{ github.ref_name }}Signing Multiple Artifacts
# Sign multiple images from same buildcosign sign --yes \ gcr.io/apk-test-442304/myapp:1.0.0 \ gcr.io/apk-test-442304/myapp:latest \ gcr.io/apk-test-442304/myapp:1.0.0-alpine \ gcr.io/apk-test-442304/myapp:1.0.0-debian # All signatures use same certificate (valid for 10 minutes)Rekor Transparency Logs
Querying Rekor
# Search for entries signed by a specific identityrekor-cli search --email alice@company.com # Output:# Found 427 entries signed by alice@company.com # Get recent entriesrekor-cli log info # Query specific entryrekor-cli get --uuid abc123def456...Verifying Against Rekor
# Verify signature against transparency logcosign verify \ --certificate-identity alice@company.com \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ --offline=false \ gcr.io/apk-test-442304/myapp:1.0.0 # Output includes Rekor verification:# ✓ Signature found in Rekor at index 35901234# ✓ Timestamp verified: 2025-10-04T14:32:00Z# ✓ No concurrent modificationsSigning Attestations
Sign VEX Documents
# Create VEX documentcleanimg-init --image myapp:1.0.0 --output-vex vex-report.json # Sign attestationcosign attest --yes \ --predicate vex-report.json \ --type vex \ gcr.io/apk-test-442304/myapp:1.0.0 # Verify attestationcosign verify-attestation \ --certificate-identity alice@company.com \ --certificate-oidc-issuer https://token.actions.githubusercontent.com \ --type vex \ gcr.io/apk-test-442304/myapp:1.0.0Sign SBOM
# Generate SBOMsyft gcr.io/apk-test-442304/myapp:1.0.0 -o spdx > sbom.spdx # Attach and signcosign attach sbom \ --sbom sbom.spdx \ gcr.io/apk-test-442304/myapp:1.0.0 cosign sign --yes \ gcr.io/apk-test-442304/myapp:1.0.0 # Both image and SBOM are signedVerification Policies
OPA Policy for Signature Verification
# policy.regopackage imageSignatures import future.keywords.containsimport future.keywords.ifimport future.keywords.in # Require signatures from specific identityallowed_signers := [ "release-bot@company.com", "alice@company.com", "bob@company.com"] # Verification resultviolation[msg] if { not image_is_signed msg := sprintf("Image %s is not signed", [input.image.name])} violation[msg] if { image_is_signed not signer_is_allowed signer := get_image_signer msg := sprintf("Image %s signed by unauthorized signer: %s", [input.image.name, signer])} image_is_signed if { input.image.signatures count(input.image.signatures) > 0} signer_is_allowed if { signer := get_image_signer signer in allowed_signers} get_image_signer = signer if { signer := input.image.signatures[0].certificate.subject.email}Enforce policy:
# Evaluate policy against image verificationcosign verify gcr.io/apk-test-442304/myapp:1.0.0 \ --output json | jq . | conftest test -p policy.rego -Key Management Comparison
Aspect | Traditional Keys | Sigstore |
|---|---|---|
Key Rotation | Manual process | Automatic (5-10 min TTL) |
Storage | Secrets manager | None (certificate-based) |
Compromise Risk | Long-lived keys | Time-limited certificates |
Audit Trail | Application logs | Rekor transparency log |
Recovery | Key recovery procedures | Automatic via new cert |
Operational Burden | High | Low |
Cost | Custom infrastructure | Free (public Sigstore) |
Troubleshooting Signature Issues
Verification Fails with "no valid certificate"
# Recheck certificate validitycosign describe gcr.io/apk-test-442304/myapp:1.0.0 # If certificate is expired (> 10 minutes old), signature is invalid# Resign the imagecosign sign --yes gcr.io/apk-test-442304/myapp:1.0.0 # Root cause: Image signed long ago, certificate now expired# Solution: Images should be signed near build timeOIDC Token Error
# Ensure OIDC provider is availableexport COSIGN_EXPERIMENTAL=1cosign sign --yes gcr.io/apk-test-442304/myapp:1.0.0 --verbose # Common fixes:# 1. Check GitHub Actions context (id-token: write permission)# 2. Verify OIDC issuer is configured# 3. Check network access to Sigstore FulcioRekor Entry Not Found
# Check if entry existsrekor-cli search --image gcr.io/apk-test-442304/myapp:1.0.0 # If empty, entry was never created# Possible causes:# 1. Image signed with --offline flag# 2. Rekor was unavailable during signing# 3. Image hash changed after signing # Verify with offline flag disabledcosign verify --offline=false gcr.io/apk-test-442304/myapp:1.0.0See Also
Provenance Chaining: provenance-chaining.md — End-to-end signatures. SLSA Level 4: slsa-level-4.md — Build signing and attestation. In-Toto Attestation: in-toto-attestation.md — Artifact verification framework.
