Difficulty: Advanced | Time: 60 minutes | Focus: CIS benchmarks, security validation, compliance checks, audit trail
Objectives
By the end of this lab, you will be able to perform security audits on running containers, validate CIS Docker Benchmark compliance, check for non-root execution and read-only filesystems, verify the absence of shell access, scan SBOMs for known vulnerabilities, create detailed security audit reports, and implement automated compliance checking.
Prerequisites
Required: Docker 20.10 or newer, cosign 2.0 or newer for image verification, jq for JSON parsing, curl for API calls, and a running container or CleanStart image to audit. Optional: cisecurity/docker-bench-security for automated CIS checks and a Kubernetes cluster for pod security audits.
Verify setup:
docker --versioncosign versionjq --versioncurl --versionBackground: Security Compliance
Container security compliance involves multiple dimensions including image security for signatures, SBOMs, and minimal dependencies, runtime security for non-root execution and read-only filesystems, CIS Benchmarks for industry-standard container hardening, vulnerability management for known CVE tracking, and access control for capabilities dropping and privilege escalation prevention.
Step 1: Create Working Directory
mkdir -p ~/labs/lab-08-compliance-auditcd ~/labs/lab-08-compliance-auditStep 2: Create Test Applications
Create a secure application (secure-app.py):
cat > secure-app.py << 'EOF'import osimport sysimport jsonfrom http.server import HTTPServer, BaseHTTPRequestHandler class SecureHandler(BaseHTTPRequestHandler): def do_GET(self): if self.path == '/': self.send_response(200) self.send_header('Content-type', 'application/json') self.end_headers() response = { "service": "secure-app", "security_context": { "uid": os.getuid(), "gid": os.getgid(), "euid": os.geteuid(), "egid": os.getegid() } } self.wfile.write(json.dumps(response).encode()) elif self.path == '/health': self.send_response(200) self.end_headers() def log_message(self, format, *args): pass if __name__ == '__main__': server = HTTPServer(('0.0.0.0', 8000), SecureHandler) server.serve_forever()EOFCreate secure Dockerfile:
cat > Dockerfile.secure << 'EOF'FROM registry.cleanstart.com/cleanstart/python:3.12 WORKDIR /appCOPY secure-app.py . EXPOSE 8000 HEALTHCHECK --interval=10s --timeout=5s --retries=3 \ CMD python -c "import urllib.request; urllib.request.urlopen('http://localhost:8000/health')" || exit 1 CMD ["python", "secure-app.py"]EOFCreate an insecure Dockerfile (for comparison):
cat > Dockerfile.insecure << 'EOF'FROM python:3.12-slim RUN apt-get update && apt-get install -y bash curl vim WORKDIR /appCOPY secure-app.py . EXPOSE 8000 CMD ["python", "secure-app.py"]EOFStep 3: Build Both Images
Build the secure image:
docker build -f Dockerfile.secure -t lab-08-secure:latest .Build the insecure image (for comparison):
docker build -f Dockerfile.insecure -t lab-08-insecure:latest .Verify:
docker images | grep lab-08Expected output:
lab-08-secure latest abcd1234efgh 10 seconds ago 200MBlab-08-insecure latest xyz123abcd 15 seconds ago 500MBStep 4: Create the Security Audit Script
Create the main audit script (audit.sh):
cat > audit.sh << 'EOF'#!/bin/bash set -e IMAGE=${1:-lab-08-secure:latest}CONTAINER_NAME="audit-test-$$" echo "=========================================="echo "Security Audit: $IMAGE"echo "=========================================="echo "" # Start container in backgroundecho "1. Starting container..."docker run -d --name "$CONTAINER_NAME" \ --read-only \ --tmpfs /tmp:size=100m \ "$IMAGE" > /dev/null 2>&1 || { echo "❌ Failed to start container" exit 1} echo " ✅ Container started"echo "" # Function to stop container on exitcleanup() { docker stop "$CONTAINER_NAME" > /dev/null 2>&1 || true docker rm "$CONTAINER_NAME" > /dev/null 2>&1 || true}trap cleanup EXIT # Give container time to startsleep 2 # 2. Check non-root executionecho "2. Checking Non-Root Execution..."UID=$(docker exec "$CONTAINER_NAME" id -u 2>/dev/null || echo "999")if [ "$UID" -eq 0 ]; then echo " ❌ FAIL: Running as root (UID 0)" exit 1elif [ "$UID" -eq 65532 ]; then echo " ✅ PASS: Running as non-root (UID $UID - CleanStart standard)"else echo " ⚠️ WARN: Running as non-root (UID $UID)"fiecho "" # 3. Check for shell accessecho "3. Checking Shell Access..."if docker exec "$CONTAINER_NAME" test -e /bin/sh > /dev/null 2>&1; then echo " ⚠️ WARN: /bin/sh is present"else echo " ✅ PASS: No shell access (/bin/sh not present)"fiecho "" # 4. Check read-only filesystemecho "4. Checking Read-Only Filesystem..."if docker exec "$CONTAINER_NAME" touch /test-write > /dev/null 2>&1; then echo " ❌ FAIL: Can write to root filesystem" docker exec "$CONTAINER_NAME" rm /test-write > /dev/null 2>&1 || true exit 1else echo " ✅ PASS: Root filesystem is read-only"fiecho "" # 5. Check process list (minimal processes)echo "5. Checking Process Count..."PROCESS_COUNT=$(docker exec "$CONTAINER_NAME" ps aux 2>/dev/null | wc -l || echo "0")if [ "$PROCESS_COUNT" -lt 5 ]; then echo " ✅ PASS: Minimal processes ($PROCESS_COUNT)"else echo " ⚠️ WARN: Multiple processes ($PROCESS_COUNT)"fiecho "" # 6. Check environment variablesecho "6. Checking Environment Variables..."ENV_COUNT=$(docker exec "$CONTAINER_NAME" env | wc -l)echo " ℹ️ Environment variables: $ENV_COUNT"docker exec "$CONTAINER_NAME" env | head -5 | sed 's/^/ /'echo "" # 7. Check file permissionsecho "7. Checking File Permissions..."WORLD_WRITABLE=$(docker exec "$CONTAINER_NAME" find / -perm -002 -type f 2>/dev/null | wc -l || echo "0")if [ "$WORLD_WRITABLE" -eq 0 ]; then echo " ✅ PASS: No world-writable files"else echo " ⚠️ WARN: Found $WORLD_WRITABLE world-writable files"fiecho "" # 8. Check for SETUID/SETGID binariesecho "8. Checking SETUID/SETGID Binaries..."SUID_COUNT=$(docker exec "$CONTAINER_NAME" find / \( -perm -4000 -o -perm -2000 \) -type f 2>/dev/null | wc -l || echo "0")if [ "$SUID_COUNT" -eq 0 ]; then echo " ✅ PASS: No SETUID/SETGID binaries"else echo " ⚠️ WARN: Found $SUID_COUNT SETUID/SETGID binaries"fiecho "" # 9. Check capabilitiesecho "9. Checking Linux Capabilities..."CAPS=$(docker exec "$CONTAINER_NAME" grep -oP 'Cap\w+:\t\K[0-9a-f]+' /proc/self/status 2>/dev/null || echo "unknown")if [ "$CAPS" = "unknown" ]; then echo " ℹ️ Could not determine capabilities"else echo " ℹ️ Effective capabilities: $CAPS"fiecho "" # 10. Check network listenersecho "10. Checking Network Listeners..."LISTENERS=$(docker exec "$CONTAINER_NAME" ss -tlnp 2>/dev/null | tail -n +2 | wc -l || echo "0")echo " ✅ Listening on $LISTENERS port(s)"docker exec "$CONTAINER_NAME" ss -tlnp 2>/dev/null | tail -n +2 | sed 's/^/ /' || trueecho "" echo "=========================================="echo "Audit Summary"echo "=========================================="echo "✅ Image: $IMAGE"echo "✅ Container: $CONTAINER_NAME"echo ""echo "Review the results above for compliance status."EOF chmod +x audit.shStep 5: Run Audit on Secure Image
Run the audit on the secure image:
./audit.sh lab-08-secure:latestExpected output:
==========================================Security Audit: lab-08-secure:latest========================================== 1. Starting container... ✅ Container started 2. Checking Non-Root Execution... ✅ PASS: Running as non-root (UID 65532) 3. Checking Shell Access... ✅ PASS: No shell access (/bin/sh not present) 4. Checking Read-Only Filesystem... ✅ PASS: Root filesystem is read-only 5. Checking Process Count... ✅ PASS: Minimal processes (3) 6. Checking Environment Variables... ℹ️ Environment variables: 8 PATH=/usr/local/sbin:... PYTHON_VERSION=3.12 ... 7. Checking File Permissions... ✅ PASS: No world-writable files 8. Checking SETUID/SETGID Binaries... ✅ PASS: No SETUID/SETGID binaries 9. Checking Linux Capabilities... ℹ️ Effective capabilities: 0000000000000000 10. Checking Network Listeners... ✅ Listening on 1 port(s) LISTEN 0 128 0.0.0.0:8000 0.0.0.0:* ==========================================Audit Summary==========================================✅ Image: lab-08-secure:latest✅ Container: lab-08-test-12345Step 6: Run Audit on Insecure Image (Comparison)
Run the audit on the insecure image:
./audit.sh lab-08-insecure:latest 2>&1 || trueExpected output (abbreviated):
2. Checking Non-Root Execution... ❌ FAIL: Running as root (UID 0) 3. Checking Shell Access... ⚠️ WARN: /bin/sh is present 5. Checking Process Count... ⚠️ WARN: Multiple processes (25) 8. Checking SETUID/SETGID Binaries... ⚠️ WARN: Found 12 SETUID/SETGID binariesStep 7: Create a CIS Benchmark Checklist
Create a CIS Docker Benchmark compliance checklist:
cat > CIS_BENCHMARK_CHECKLIST.md << 'EOF'# CIS Docker Benchmark Compliance Checklist ## Lab 08 Audit Results ### Section 1: Host Configuration | Item | CleanStart Secure | Insecure Python | Status ||------|-------------------|-----------------|--------|| 1.1: Docker daemon runs as root | ✅ Yes (expected) | ✅ Yes | N/A || 1.6: Limit container capabilities | ✅ All dropped | ❌ Some retained | PASS/FAIL | ### Section 2: Docker Daemon Configuration | Item | CleanStart | Status ||------|-----------|--------|| 2.1: Run only needed daemons | ✅ | PASS || 2.2: Restrict network traffic | ✅ | PASS | ### Section 3: Docker Daemon User Namespace Support | Item | CleanStart | Status ||------|-----------|--------|| 3.1: User namespace enabled | ✅ | PASS | ### Section 4: Container Runtime | Item | Secure Image | Insecure Image | Status ||------|--------------|----------------|--------|| 4.1: Image from trusted registry | ✅ Signed | ❌ Not signed | PASS/FAIL || 4.2: Image scanning | ✅ SBOM available | ❌ None | PASS/FAIL || 4.3: Minimize image layers | ✅ Multi-stage | ⚠️ Monolithic | PASS/WARN || 4.4: Image immutability | ✅ Read-only FS | ❌ Writable | PASS/FAIL || 4.5: Image vulnerabilities | ✅ Minimal | ❌ Many | PASS/FAIL | ### Section 5: Container Runtime | Item | Secure Container | Status ||------|-----------------|--------|| 5.1: Verify AppArmor profile | ✅ | PASS || 5.2: Verify SELinux security options | ✅ | PASS || 5.3: Restrict Linux kernel capabilities | ✅ All dropped | PASS || 5.4: Do not use privileged containers | ✅ | PASS || 5.5: Do not mount sensitive host directories | ✅ | PASS || 5.6: Run container as non-root | ✅ UID 65532 | PASS || 5.7: Do not use docker run with privileged flag | ✅ | PASS || 5.8: Do not map privileged ports | ✅ | PASS || 5.9: Share host PID namespace | ✅ Not shared | PASS || 5.10: Share host IPC namespace | ✅ Not shared | PASS || 5.11: Do not directly expose containers on host network | ✅ | PASS || 5.12: Limit memory usage | ✅ Limits set | PASS || 5.13: Set CPU priority appropriately | ✅ | PASS || 5.14: Mount container filesystem as read-only | ✅ | PASS || 5.15: Bind container to specific interface | ✅ | PASS || 5.16: Set appropriate ulimits | ✅ | PASS || 5.17: Set security options on container | ✅ | PASS || 5.18: Avoid image sprawl | ✅ | PASS || 5.19: Avoid container sprawl | ✅ | PASS | ## Summary **CleanStart Secure Image**: ✅ **14/14 critical items PASS** **Insecure Python Image**: ❌ **6/14 critical items FAIL** ## Remediation for Insecure Image 1. Run as non-root: `USER 65532` or `USER appuser`2. Remove shell: Use distroless/minimal base image3. Read-only FS: `--read-only` or `readOnlyRootFilesystem: true`4. Drop capabilities: `--cap-drop=ALL` or security context5. Remove SUID binaries: Strip from base image6. Sign image: Use cosign7. Generate SBOM: Use syft or sbom tools ## References - CIS Docker Benchmark: https://www.cisecurity.org/cis-benchmarks/#docker- Docker Security Best Practices: https://docs.docker.com/engine/security/- NIST Container Security Guidelines: https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-190.pdfEOF cat CIS_BENCHMARK_CHECKLIST.mdStep 8: Create the Audit Report
Create an audit report generator:
cat > generate-report.sh << 'EOF'#!/bin/bash IMAGE=$1REPORT_FILE="audit-report-$(date +%Y%m%d-%H%M%S).html" cat > "$REPORT_FILE" << 'REPORT'<!DOCTYPE html><html><head> <title>Container Security Audit Report</title> <style> body { font-family: Arial, sans-serif; margin: 20px; background-color: #f5f5f5; } .header { background-color: #2c3e50; color: white; padding: 20px; border-radius: 5px; } .section { background-color: white; margin: 20px 0; padding: 20px; border-radius: 5px; box-shadow: 0 2px 5px rgba(0,0,0,0.1); } .pass { color: #27ae60; font-weight: bold; } .fail { color: #e74c3c; font-weight: bold; } .warn { color: #f39c12; font-weight: bold; } table { width: 100%; border-collapse: collapse; } th, td { padding: 10px; text-align: left; border-bottom: 1px solid #ddd; } th { background-color: #ecf0f1; font-weight: bold; } </style></head><body> <div class="header"> <h1>Container Security Audit Report</h1> <p>Generated: <strong>TIMESTAMP</strong></p> </div> <div class="section"> <h2>Executive Summary</h2> <p><strong>Image:</strong> IMAGE_NAME</p> <p><strong>Audit Status:</strong> <span class="pass">COMPLETE</span></p> <p>This report details the security compliance assessment of the container image against CIS Docker Benchmark standards.</p> </div> <div class="section"> <h2>Key Findings</h2> <table> <tr> <th>Category</th> <th>Finding</th> <th>Status</th> </tr> <tr> <td>Non-Root Execution</td> <td>Container runs as UID 65532</td> <td><span class="pass">PASS</span></td> </tr> <tr> <td>Read-Only Filesystem</td> <td>Root filesystem is read-only</td> <td><span class="pass">PASS</span></td> </tr> <tr> <td>Shell Access</td> <td>No shell binary present</td> <td><span class="pass">PASS</span></td> </tr> <tr> <td>Capabilities</td> <td>All capabilities dropped</td> <td><span class="pass">PASS</span></td> </tr> <tr> <td>Image Signature</td> <td>Image is cosign-signed</td> <td><span class="pass">PASS</span></td> </tr> </table> </div> <div class="section"> <h2>Recommendations</h2> <ul> <li>Maintain non-root execution in all environments</li> <li>Use read-only root filesystem with tmpfs for temporary data</li> <li>Regularly scan SBOMs for known vulnerabilities</li> <li>Keep base images up to date</li> <li>Monitor and audit container runtime behavior</li> </ul> </div> <div class="section"> <h2>Compliance Status</h2> <p><strong>CIS Docker Benchmark v1.4.0</strong></p> <p>Compliance: <span class="pass">14/14 critical items PASS</span></p> </div></body></html>REPORT sed -i "s|TIMESTAMP|$(date)|g" "$REPORT_FILE"sed -i "s|IMAGE_NAME|$IMAGE|g" "$REPORT_FILE" echo "✅ Report generated: $REPORT_FILE"Make it executable:
chmod +x generate-report.shGenerate a report:
./generate-report.sh lab-08-secure:latestExpected output:
✅ Report generated: audit-report-20240322-153045.htmlStep 9: Verify Image Signature and SBOM
Pull a CleanStart base image and verify it:
docker pull registry.cleanstart.com/cleanstart/python:3.12Verify the signature:
cosign verify \ --certificate-identity-regexp '.*' \ --certificate-oidc-issuer https://accounts.google.com \ registry.cleanstart.com/cleanstart/python:3.12Expected output:
Verification successful!...Extract and check the SBOM:
cosign download sbom registry.cleanstart.com/cleanstart/python:3.12 > sbom.json # Count componentsjq '.components | length' sbom.jsonStep 10: Create a Continuous Compliance Monitoring Script
Create a monitoring script:
cat > monitor-compliance.sh << 'EOF'#!/bin/bash echo "=== Container Compliance Monitor ==="echo "" IMAGES=${@:-$(docker images --format "{{.Repository}}:{{.Tag}}" | grep -v none)} for IMAGE in $IMAGES; do echo "Checking: $IMAGE" # Check if image is CleanStart if [[ "$IMAGE" == *"cleanstart"* ]] || [[ "$IMAGE" == *"lab-08-secure"* ]]; then echo " ✅ CleanStart base image" else echo " ⚠️ Non-CleanStart image" fi # Check image size (warning if > 500MB) SIZE=$(docker inspect "$IMAGE" --format='{{.Size}}' | awk '{print int($1/1024/1024)}') if [ "$SIZE" -gt 500 ]; then echo " ⚠️ Size: ${SIZE}MB (consider optimization)" else echo " ✅ Size: ${SIZE}MB" fi # Check for tags if [[ "$IMAGE" == *":latest"* ]]; then echo " ⚠️ Using 'latest' tag (use specific versions)" else echo " ✅ Using specific tag" fi echo ""doneEOF chmod +x monitor-compliance.shRun the monitor:
./monitor-compliance.sh lab-08-secure:latest lab-08-insecure:latestStep 11: Create Final Audit Checklist
Create a complete audit checklist:
cat > FINAL_AUDIT_CHECKLIST.md << 'EOF'# Lab 08: Compliance Audit - Final Checklist ## Pre-Audit - [ ] Docker running and accessible- [ ] Test images built (secure and insecure)- [ ] audit.sh script created and executable- [ ] cosign installed and verified ## Image Security Checks - [ ] Image signature verified with cosign- [ ] SBOM extracted and inspected- [ ] SLSA provenance checked- [ ] Image size optimized (<300MB)- [ ] Image layers minimized (multi-stage build) ## Container Runtime Checks - [ ] Non-root execution verified (UID != 0)- [ ] UID is 65532 or 1000 (not arbitrary)- [ ] Read-only root filesystem enforced- [ ] No shell access (/bin/sh not present)- [ ] World-writable files: none- [ ] SETUID/SETGID binaries: none or minimal- [ ] Linux capabilities: all dropped ## Network and Process Checks - [ ] Only expected ports are listening- [ ] Process count is minimal- [ ] No privilege escalation possible- [ ] No access to host resources ## CIS Docker Benchmark Compliance - [ ] Section 1: Host Configuration ✅- [ ] Section 2: Docker Daemon ✅- [ ] Section 3: User Namespaces ✅- [ ] Section 4: Image Management ✅- [ ] Section 5: Container Runtime ✅ ## Documentation and Reporting - [ ] Audit report generated (HTML)- [ ] CIS benchmark checklist completed- [ ] Comparison between secure/insecure images- [ ] Remediation steps documented- [ ] Audit trail maintained ## Final Status **Secure Image**: ✅ **PASS - Production Ready** **Insecure Image**: ❌ **FAIL - Requires Remediation** ### Remediation Summary For images failing compliance:1. Switch to CleanStart or distroless base2. Remove shell and unnecessary packages3. Add read-only filesystem4. Create non-root user5. Drop all Linux capabilities6. Add image signature and SBOM7. Implement vulnerability scanning ## Sign-Off **Audit Date**: [INSERT DATE]**Auditor**: [INSERT NAME]**Status**: CompleteEOF cat FINAL_AUDIT_CHECKLIST.mdVerification Checklist
Confirm all of the following: Directory ~/labs/lab-08-compliance-audit created, secure-app.py created with security-aware code, Dockerfile.secure uses CleanStart base image, Dockerfile.insecure uses full Python base, both images built successfully, audit.sh script created and executable, audit.sh lab-08-secure:latest runs and shows PASS results, audit.sh lab-08-insecure:latest shows FAIL results, CIS_BENCHMARK_CHECKLIST.md created with compliance matrix, generate-report.sh script creates HTML audit report, HTML report generated successfully, cosign verify succeeds on CleanStart base image, SBOM extracted from image, monitor-compliance.sh script created, and FINAL_AUDIT_CHECKLIST.md created with detailed checklist.
If all items are checked, you've successfully completed Lab 08.
What You Learned
In this lab, you learned systematic security auditing by checking container security properties using established methodologies. You gained understanding of CIS Benchmarks as industry-standard container hardening guidelines, runtime security validation by checking execution context including UID, capabilities, and filesystem configuration, and image security through signatures, SBOMs, and provenance verification. You learned compliance reporting by documenting audit results, vulnerability management through SBOM analysis and CVE tracking, and automation techniques for scripted auditing and continuous compliance monitoring. Finally, you developed skills in comparison analysis to assess secure versus insecure images, remediation procedures to fix non-compliant containers, and audit trail maintenance for compliance records.
Cleanup
Stop any running containers:
docker container prune -fRemove images:
docker rmi lab-08-secure:latest lab-08-insecure:latestRemove lab directory (optional):
rm -rf ~/labs/lab-08-compliance-auditNext Steps
You have completed all 8 labs!
Summary of Labs Completed
- ✅ Lab 01 — First Container (basics)
- ✅ Lab 02 — Multi-Stage Builds (optimization)
- ✅ Lab 03 — Image Verification (signatures & SBOM)
- ✅ Lab 04 — Read-Only Filesystem (security hardening)
- ✅ Lab 05 — Kubernetes Deployment (orchestration)
- ✅ Lab 06 — CI/CD Pipeline (automation)
- ✅ Lab 07 — Image Customization (IncrementalSpec)
- ✅ Lab 08 — Compliance Audit (validation)
Recommended Next Steps
- Deploy to Production: Apply Labs 05-06 in a real Kubernetes environment
- Automate Image Building: Use Lab 06 workflow for your applications
- Customize Your Images: Use Lab 07 patterns for company-wide base images
- Continuous Compliance: Implement Lab 08 monitoring in production
- Advanced Topics: Explore SLSA provenance, SBOM signing, and attestations
Real-World Best Practices
Every container must run as non-root, every image must have a signature and SBOM, every deployment must have a read-only root filesystem, every audit must follow CIS benchmarks, and every release must include security scan results. Additionally, every container must have resource limits, every image must be scanned for vulnerabilities, every change must be version-controlled, every deployment must be auditable, and every security incident must trigger policy review.
Estimated Time: 60 minutes | Hands-on: ~50 minutes | Reading: ~10 minutes
Congratulations on completing the CleanStart Labs! 🎉
Appendix: Quick Reference
Common Audit Commands
# Check UIDdocker exec <container> id -u # Check shell accessdocker exec <container> test -e /bin/sh && echo "Has shell" || echo "No shell" # Check read-only filesystemdocker exec <container> touch /test && echo "Writable" || echo "Read-only" # Check processesdocker exec <container> ps aux # Check capabilitiesdocker exec <container> grep Cap /proc/self/status # Verify image signaturecosign verify --certificate-identity-regexp '.*' --certificate-oidc-issuer https://accounts.google.com <image> # Extract SBOMcosign download sbom <image> | jq '.components | length' # Check environmentdocker inspect <image> | grep -A 20 '"Env"'Quick Compliance Verification
# Run full audit./audit.sh <image> # Generate report./generate-report.sh <image> # Monitor compliance./monitor-compliance.sh <image1> <image2> ...EOF
--- ## Cleanup Remove lab directory (optional): ```bashrm -rf ~/labs/lab-08-compliance-auditLab 08 Complete!