Running Kafka in FIPS-Compliant Mode
Kafka brokers communicate with each other and with clients. All communication must use FIPS-approved TLS to maintain cryptographic compliance. This document covers configuring Kafka for FIPS 140-3 compliance.
Prerequisites
You will need Kafka 3.0 or later, as earlier versions have limited TLS support. You'll also require OpenSSL 3.0 or later with the FIPS module (CMVP #4949). Additionally, you need Java 11 or later with a FIPS-compliant TLS provider such as Bouncy Castle or native FIPS support.
Kafka JVM and FIPS
Java FIPS Provider
Java's default TLS provider (SunJSSE) can use FIPS modules, but for maximum assurance you should use Bouncy Castle FIPS provider:
# Download Bouncy Castle FIPS JARwget https://www.bouncycastle.org/download/bcfips-1.0.2.3.jar # Add to Kafka classpathcp bcfips-1.0.2.3.jar /opt/kafka/libs/ # Configure in bin/kafka-broker-start.shexport KAFKA_JVM_PERFORMANCE_OPTS="-Dcom.sun.net.httpserver.HttpServer.logResponses=false"export CLASSPATH="/opt/kafka/libs/bcfips-1.0.2.3.jar:$CLASSPATH"FIPS-Compliant JVM Configuration
# Set JVM to use FIPS-compliant TLSexport KAFKA_HEAP_OPTS="-Xmx2G -Xms2G \ -Dcom.sun.jndi.ldap.connect.pool=false \ -Dssl.KeyManagerFactory.algorithm=PKIX \ -Dssl.TrustManagerFactory.algorithm=PKIX \ -Djdk.tls.client.protocols=TLSv1.2,TLSv1.3" # Verify TLS versionbin/kafka-broker-start.sh config/server.properties 2>&1 | grep -i "TLS"Broker-to-Broker (Inter-Broker) TLS
Generate FIPS-Compliant Certificates
#!/bin/bash# generate-kafka-certs.sh # Create CAopenssl genrsa -out ca.key 2048openssl req -new -x509 \ -key ca.key \ -out ca.crt \ -days 3650 \ -sha256 \ -subj "/CN=Kafka-CA" # Create broker certificate (for each broker)for i in 1 2 3; do # Generate key openssl genrsa -out kafka-broker-$i.key 2048 # Create CSR openssl req -new \ -key kafka-broker-$i.key \ -out kafka-broker-$i.csr \ -subj "/CN=kafka-broker-$i.example.com" # Sign with CA (use FIPS-approved SHA-256) openssl x509 -req \ -in kafka-broker-$i.csr \ -CA ca.crt \ -CAkey ca.key \ -CAcreateserial \ -out kafka-broker-$i.crt \ -days 365 \ -sha256done # Create PKCS12 keystores (Java-compatible)for i in 1 2 3; do openssl pkcs12 -export \ -in kafka-broker-$i.crt \ -inkey kafka-broker-$i.key \ -out kafka-broker-$i.p12 \ -name kafka-broker-$i \ -CAfile ca.crt \ -passin pass:brokerpass \ -passout pass:brokerpassdone # Create truststore (contains CA certificate)keytool -import \ -alias kafka-ca \ -file ca.crt \ -into kafka.truststore.jks \ -storepass trustpass \ -nopromptConfigure Inter-Broker TLS
# server.properties for each broker # Listening portslisteners=PLAINTEXT://localhost:9092,SSL://localhost:9093advertised.listeners=PLAINTEXT://kafka-broker-1.example.com:9092,SSL://kafka-broker-1.example.com:9093 # Security protocol for inter-brokersecurity.inter.broker.protocol=SSL # SSL/TLS Configurationssl.keystore.location=/etc/kafka/secrets/kafka-broker-1.p12ssl.keystore.password=brokerpassssl.keystore.type=PKCS12 ssl.truststore.location=/etc/kafka/secrets/kafka.truststore.jksssl.truststore.password=trustpassssl.truststore.type=JKS # FIPS-approved TLS versionsssl.enabled.protocols=TLSv1.2,TLSv1.3 # FIPS-approved cipher suitesssl.cipher.suites=TLS_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 # Client authenticationssl.client.auth=required # Hostname verification (security best practice)ssl.endpoint.identification.algorithm=HTTPSClient-to-Broker TLS
Producer with Client TLS
// Java Producer with FIPS TLSProperties props = new Properties();props.put("bootstrap.servers", "kafka-broker-1.example.com:9093");props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer"); // TLS Configurationprops.put("security.protocol", "SSL");props.put("ssl.truststore.location", "/etc/kafka/secrets/kafka.truststore.jks");props.put("ssl.truststore.password", "trustpass");props.put("ssl.truststore.type", "JKS"); // Client authenticationprops.put("ssl.keystore.location", "/etc/kafka/secrets/client.p12");props.put("ssl.keystore.password", "clientpass");props.put("ssl.keystore.type", "PKCS12"); // FIPS TLS versionsprops.put("ssl.enabled.protocols", "TLSv1.2,TLSv1.3"); // Hostname verificationprops.put("ssl.endpoint.identification.algorithm", "HTTPS"); KafkaProducer<String, String> producer = new KafkaProducer<>(props);Consumer with Client TLS
// Java Consumer with FIPS TLSProperties props = new Properties();props.put("bootstrap.servers", "kafka-broker-1.example.com:9093");props.put("group.id", "my-group");props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer"); // TLS Configuration (same as producer)props.put("security.protocol", "SSL");props.put("ssl.truststore.location", "/etc/kafka/secrets/kafka.truststore.jks");props.put("ssl.truststore.password", "trustpass");props.put("ssl.enabled.protocols", "TLSv1.2,TLSv1.3"); KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);Command Line Tools with TLS
# Producer CLI with FIPS TLSbin/kafka-console-producer.sh \ --bootstrap-server kafka-broker-1.example.com:9093 \ --topic my-topic \ --producer-property security.protocol=SSL \ --producer-property ssl.truststore.location=/etc/kafka/secrets/kafka.truststore.jks \ --producer-property ssl.truststore.password=trustpass \ --producer-property ssl.enabled.protocols=TLSv1.2,TLSv1.3 # Consumer CLI with FIPS TLSbin/kafka-console-consumer.sh \ --bootstrap-server kafka-broker-1.example.com:9093 \ --topic my-topic \ --consumer-property security.protocol=SSL \ --consumer-property ssl.truststore.location=/etc/kafka/secrets/kafka.truststore.jks \ --consumer-property ssl.truststore.password=trustpassSASL with FIPS TLS
SASL provides authentication while FIPS TLS provides encryption:
# server.properties - SASL/SCRAM with TLS # Authentication mechanismsecurity.protocol=SASL_SSLsasl.mechanism.inter.broker.protocol=SCRAM-SHA-256sasl.enabled.mechanisms=SCRAM-SHA-256 # SASL configurationsasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="broker" password="broker-secret"; # TLS Configuration (same as above)ssl.keystore.location=/etc/kafka/secrets/kafka-broker-1.p12ssl.keystore.password=brokerpassssl.truststore.location=/etc/kafka/secrets/kafka.truststore.jksssl.truststore.password=trustpass # FIPS TLS versionsssl.enabled.protocols=TLSv1.2,TLSv1.3Docker: Kafka with FIPS TLS
# Dockerfile: Kafka with FIPS TLSFROM openjdk:11-jre-slim # Install Kafka and Bouncy Castle FIPSRUN apt-get update && apt-get install -y \ kafka \ openssl-fips \ ca-certificates # Copy FIPS-compliant certificatesCOPY kafka-broker-1.p12 /etc/kafka/secrets/COPY kafka.truststore.jks /etc/kafka/secrets/ # Copy FIPS configurationCOPY server.properties /etc/kafka/ # Set FIPS modeENV KAFKA_HEAP_OPTS="-Dcom.sun.jndi.ldap.connect.pool=false" EXPOSE 9092 9093ENTRYPOINT ["kafka-broker-start.sh", "/etc/kafka/server.properties"]FIPS Verification
Verify Kafka Uses FIPS TLS
# Check broker TLS configurationbin/kafka-configs.sh \ --bootstrap-server localhost:9093 \ --entity-type brokers \ --entity-name 0 \ --describe \ --command-config client-ssl.properties | grep -i ssl # Output shows:# ssl.enabled.protocols=TLSv1.2,TLSv1.3# ssl.cipher.suites=TLS_AES_256_GCM_SHA384,... # Test TLS connectionopenssl s_client \ -connect kafka-broker-1.example.com:9093 \ -cert client.crt \ -key client.key \ -CAfile ca.crt # Output shows:# subject=CN = kafka-broker-1.example.com# issuer=CN = Kafka-CA# TLSv1.3, Cipher = TLS_AES_256_GCM_SHA384Automated FIPS Verification
# CleanStart FIPS verification for Kafkacleanimg-init --fips-verifier --image my-kafka:latest # Output:# ✓ Java TLS provider supports FIPS# ✓ Kafka brokers configured with FIPS TLS# ✓ Inter-broker communication uses TLS 1.2/1.3# ✓ Client authentication required# ✓ FIPS-approved cipher suites only# ✓ Certificate validation enabledCommon Issues
Issue: Hostname Verification Fails
ERROR: PKIX path validation failed: java.security.cert.CertPathValidatorExceptionSOLUTION:1. Verify certificate CN matches hostname2. Check advertised.listeners matches certificate CN3. Or disable hostname verification (not recommended): ssl.endpoint.identification.algorithm=Issue: Broker Startup Fails with TLS Error
ERROR: Keystore password is incorrectSOLUTION:1. Verify keystore password matches configuration2. Verify keystore file exists at specified path3. Test keystore independently: keytool -list -v -keystore kafka-broker-1.p12 -storepass brokerpassIssue: Client Cannot Connect
ERROR: SSL handshake failedSOLUTION:1. Verify client certificate is signed by trusted CA2. Ensure client truststore contains CA certificate3. Check TLS version compatibility (1.2 or 1.3)4. Test with openssl s_client firstPerformance Considerations
TLS overhead typically ranges from 5-10% CPU for broker communication. With hardware acceleration, the throughput impact is less than 2%. Latency adds approximately 0.5-1ms per message, which is acceptable for compliance requirements.
Compliance Mapping
Requirement | Implementation |
|---|---|
FIPS 140-3 | Java FIPS TLS provider |
NIST 800-171 SC-13 | TLS 1.2+, AES-256-GCM, RSA-2048 |
PCI DSS 4.1 | Encryption in transit with TLS |
HIPAA § 164.312(a)(2)(i) | Encryption between brokers and clients |
See Also
FIPS Overview: fips-140-overview.md — FIPS concepts. FIPS Redis: fips-redis.md — Similar TLS configuration patterns. FIPS Traces: fips-traces.md — Verify TLS operations at runtime.