eBPF and Falco: Runtime Anomaly Detection and Forensics

# Add Falco Helm repositoryhelm repo add falcosecurity https://falcosecurity.github.io/chartshelm repo update # Install Falco in your clusterhelm install falco falcosecurity/falco \  --namespace falco --create-namespace \  --set falco.grpc.enabled=true \  --set falco.grpcOutput.enabled=true # Verify Falco is runningkubectl get pods -n falco# falco-xxxxx: Running

# Install eBPF driver (required for observability)helm install falco falcosecurity/falco \  --set ebpf.enabled=true \  --set falco.grpc.enabled=true # Verify eBPF is loadedkubectl exec -n falco <falco-pod> -- dmesg | grep -i ebpf# Output: eBPF module loaded successfully

# falco-cleanstart-rules.yaml # 1. Detect unauthorized privilege escalation- rule: Suspicious Privilege Escalation  desc: Detect sudo/su usage or capability escalation  condition: >    spawned_process and    (proc.name in (sudo, su) or container.privileged = true) and    user.uid != 0  output: >    Privilege escalation attempt    (user=%user.name container=%container.info cmd=%proc.cmdline)  priority: WARNING  tags: [privilege_escalation, container_security] # 2. Detect cryptographic operations (zero-day check)- rule: Suspicious Cryptographic Library Usage  desc: Detect unusual cryptography patterns  condition: >    spawned_process and    fd.name glob "/proc/*/exe" and    container.image.repository != "trusted-source"  output: >    Suspicious crypto operation    (image=%container.image.repository process=%proc.name)  priority: WARNING  tags: [cryptography, zero_day_detection] # 3. Detect network anomalies- rule: Unexpected Network Connection  desc: Container making outbound connection to unknown destination  condition: >    outbound and    container.privileged = false and    fd.sip not in (allowed_ips) and    fd.sport not in (allowed_ports)  output: >    Unexpected outbound connection    (container=%container.name dest=%fd.sip port=%fd.sport)  priority: NOTICE  tags: [network_security, anomaly] # 4. Detect ransomware patterns- rule: Potential Ransomware Activity  desc: Detect file encryption or mass deletion  condition: >    write and    container.privileged = false and    (fd.name glob "*.locked" or fd.name glob "*.encrypted") and    (number_files_written > 100 in 60 seconds)  output: >    Potential ransomware    (container=%container.name files=%number_files_written)  priority: CRITICAL  tags: [ransomware, malware] # 5. Detect reverse shells- rule: Potential Reverse Shell  desc: Detect suspicious shell spawning network connection  condition: >    spawned_process and    container.privileged = false and    (proc.name in (sh, bash) or proc.name contains /bin/) and    (preceding_process.name not in (expected_shells)) and    outbound  output: >    Potential reverse shell    (container=%container.name shell=%proc.name dest=%fd.sip)  priority: CRITICAL  tags: [reverse_shell, intrusion] # 6. Detect supply chain attacks- rule: Suspicious Package Installation  desc: Detect unauthorized package manager activity  condition: >    spawned_process and    (proc.name in (apt, yum, npm, pip)) and    user != expected_user and    proc.cmdline contains "install"  output: >    Unauthorized package installation    (container=%container.name cmd=%proc.cmdline)  priority: WARNING  tags: [supply_chain, package_manager]

# Send Falco alerts to CleanStart IntelligenceapiVersion: v1kind: ConfigMapmetadata:  name: falco-custom-output  namespace: falcodata:  cleanstart_output: |    - http:        url: "https://intelligence.cleanstart.io/api/v1/runtime-alerts"        method: "POST"        headers:          Authorization: "Bearer YOUR_API_KEY"          Content-Type: "application/json"        body_template: |          {            "alert_type": "runtime_anomaly",            "container": "${container.name}",            "image": "${container.image.repository}",            "rule": "${rule}",            "priority": "${priority}",            "evidence": {              "process": "${proc.name}",              "user": "${user.name}",              "command": "${proc.cmdline}",              "network": {                "src_ip": "${fd.sip}",                "dst_ip": "${fd.dip}",                "port": "${fd.sport}"              }            },            "timestamp": "${time}"          }

# Export Falco alerts to audit systemkubectl logs -n falco -f deployment/falco | \  jq -r '.alert_type, .container, .rule, .priority' | \  tee -a /var/log/runtime-security-audit.log # Also send to SIEM (Splunk, ELK, etc.)kubectl logs -n falco -f deployment/falco | \  curl -X POST \    -H "Content-Type: application/json" \    -d @- \    https://siem.example.com:8088/services/collector

# Monitor running container against policyfalco \  --rules /etc/falco/cleanstart-security-rules.yaml \  --rules /etc/falco/k8s-policy-rules.yaml # Example output:# 08:14:23.123456: WARNING: Suspicious Privilege Escalation#   user=appuser container=myapp-xyz proc=sudo## 08:15:45.654321: CRITICAL: Potential Reverse Shell#   container=myapp-xyz shell=/bin/bash dest=10.0.0.5:4444## 08:16:12.987654: NOTICE: Unexpected Network Connection#   container=myapp-xyz dest=mining.attacker.com port=3333

# During incident, capture full forensic evidencefalco \  --rules /etc/falco/forensics-rules.yaml \  --capture-file /var/log/forensics/$(date +%s).pcap \  --event-trace /var/log/forensics/events.json \  -o file:///var/log/forensics/falco.log # Output contains:# - All system calls made# - File access patterns# - Network traffic# - Process creation chains# - Environment variables at time of execution # Later analyze:cat /var/log/forensics/events.json | jq '  .[] |  select(.container.name == "suspect-container") |  {time, rule, process: .proc.name, user: .user.name, action: .proc.cmdline}'

# Anomaly scoring based on Falco alertsimport jsonfrom datetime import datetime, timedelta def calculate_runtime_risk_score(falco_events, time_window_minutes=60):    """    Score container based on Falco alerts    Higher score = higher risk of compromise    """     risk_scores = {        "CRITICAL": 10.0,        "WARNING": 5.0,        "NOTICE": 2.0,        "INFO": 0.5    }     now = datetime.now()    cutoff = now - timedelta(minutes=time_window_minutes)     relevant_events = [        e for e in falco_events        if datetime.fromisoformat(e['timestamp']) > cutoff    ]     total_score = sum(        risk_scores.get(e['priority'], 0)        for e in relevant_events    )     # Normalize to 0-1 scale    max_possible = 10.0 * len(relevant_events)    normalized_score = min(total_score / max_possible, 1.0) if max_possible > 0 else 0     return {        "risk_score": normalized_score,        "events": len(relevant_events),        "critical_count": sum(1 for e in relevant_events if e['priority'] == 'CRITICAL'),        "recommendation": (            "BLOCK" if normalized_score > 0.8 else            "QUARANTINE" if normalized_score > 0.5 else            "MONITOR" if normalized_score > 0.2 else            "OK"        )    } # Usageevents = json.load(open('/var/log/falco-events.json'))score = calculate_runtime_risk_score(events)print(f"Risk Score: {score['risk_score']:.2f}")print(f"Recommendation: {score['recommendation']}")

Event: Spawned process with suspicious patternContainer: worker-pod-xyzProcess: /usr/bin/xmrigParent: /bin/bash (PID 1234)User: appuserEnvironment:  - POOL=mining.attacker.com:3333  - WALLET=attacker-wallet  - THREADS=4 Falco Analysis: The process name matches known miners (xmrig, monero, etc.). Environment variables indicate mining pool connection. A CPU usage spike is detected. Network connection to mining pool is confirmed through another rule. Risk Score: 0.98 (CRITICAL)Action: Container terminated immediately

Event: Process attempted privilege escalationContainer: api-server-xyzProcess: sudo /bin/bashUser: appuser (UID 1000)Original Command: /app/bin/start-server.sh Falco Analysis:sudo was called from a non-privileged userUser is not in sudoers file (authorization failed)Shell was spawned with elevated privileges (escalation attempt)This violates the pod security policy Risk Score: 0.92 (CRITICAL)Action: Pod terminated, incident logged

Event: Suspicious shell spawning network connectionContainer: web-app-xyzProcess: /bin/bashParent: /bin/sh (background process)Network: Outbound connection to 10.0.0.5:4444 Falco Analysis:Shell process was spawned from an unexpected parentReverse shell pattern detected (bash listening/connecting)Destination IP is not in the allowlistPort 4444 matches common C2 communication ports Risk Score: 0.95 (CRITICAL)Action: Network connection blocked, container isolated