FIPS-Compliant Apache Kafka: Inter-Broker and Client TLS

Running Kafka in FIPS-Compliant Mode

Kafka brokers communicate with each other and with clients. All communication must use FIPS-approved TLS to maintain cryptographic compliance. This document covers configuring Kafka for FIPS 140-3 compliance.

Prerequisites

You will need Kafka 3.0 or later, as earlier versions have limited TLS support. You'll also require OpenSSL 3.0 or later with the FIPS module (CMVP #4949). Additionally, you need Java 11 or later with a FIPS-compliant TLS provider such as Bouncy Castle or native FIPS support.

Kafka JVM and FIPS

Java FIPS Provider

Java's default TLS provider (SunJSSE) can use FIPS modules, but for maximum assurance you should use Bouncy Castle FIPS provider:

# Download Bouncy Castle FIPS JARwget https://www.bouncycastle.org/download/bcfips-1.0.2.3.jar # Add to Kafka classpathcp bcfips-1.0.2.3.jar /opt/kafka/libs/ # Configure in bin/kafka-broker-start.shexport KAFKA_JVM_PERFORMANCE_OPTS="-Dcom.sun.net.httpserver.HttpServer.logResponses=false"export CLASSPATH="/opt/kafka/libs/bcfips-1.0.2.3.jar:$CLASSPATH"

FIPS-Compliant JVM Configuration

# Set JVM to use FIPS-compliant TLSexport KAFKA_HEAP_OPTS="-Xmx2G -Xms2G \  -Dcom.sun.jndi.ldap.connect.pool=false \  -Dssl.KeyManagerFactory.algorithm=PKIX \  -Dssl.TrustManagerFactory.algorithm=PKIX \  -Djdk.tls.client.protocols=TLSv1.2,TLSv1.3" # Verify TLS versionbin/kafka-broker-start.sh config/server.properties 2>&1 | grep -i "TLS"

Broker-to-Broker (Inter-Broker) TLS

Generate FIPS-Compliant Certificates

#!/bin/bash# generate-kafka-certs.sh # Create CAopenssl genrsa -out ca.key 2048openssl req -new -x509 \  -key ca.key \  -out ca.crt \  -days 3650 \  -sha256 \  -subj "/CN=Kafka-CA" # Create broker certificate (for each broker)for i in 1 2 3; do  # Generate key  openssl genrsa -out kafka-broker-$i.key 2048   # Create CSR  openssl req -new \    -key kafka-broker-$i.key \    -out kafka-broker-$i.csr \    -subj "/CN=kafka-broker-$i.example.com"   # Sign with CA (use FIPS-approved SHA-256)  openssl x509 -req \    -in kafka-broker-$i.csr \    -CA ca.crt \    -CAkey ca.key \    -CAcreateserial \    -out kafka-broker-$i.crt \    -days 365 \    -sha256done # Create PKCS12 keystores (Java-compatible)for i in 1 2 3; do  openssl pkcs12 -export \    -in kafka-broker-$i.crt \    -inkey kafka-broker-$i.key \    -out kafka-broker-$i.p12 \    -name kafka-broker-$i \    -CAfile ca.crt \    -passin pass:brokerpass \    -passout pass:brokerpassdone # Create truststore (contains CA certificate)keytool -import \  -alias kafka-ca \  -file ca.crt \  -into kafka.truststore.jks \  -storepass trustpass \  -noprompt

Configure Inter-Broker TLS

# server.properties for each broker # Listening portslisteners=PLAINTEXT://localhost:9092,SSL://localhost:9093advertised.listeners=PLAINTEXT://kafka-broker-1.example.com:9092,SSL://kafka-broker-1.example.com:9093 # Security protocol for inter-brokersecurity.inter.broker.protocol=SSL # SSL/TLS Configurationssl.keystore.location=/etc/kafka/secrets/kafka-broker-1.p12ssl.keystore.password=brokerpassssl.keystore.type=PKCS12 ssl.truststore.location=/etc/kafka/secrets/kafka.truststore.jksssl.truststore.password=trustpassssl.truststore.type=JKS # FIPS-approved TLS versionsssl.enabled.protocols=TLSv1.2,TLSv1.3 # FIPS-approved cipher suitesssl.cipher.suites=TLS_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 # Client authenticationssl.client.auth=required # Hostname verification (security best practice)ssl.endpoint.identification.algorithm=HTTPS

Client-to-Broker TLS

Producer with Client TLS

// Java Producer with FIPS TLSProperties props = new Properties();props.put("bootstrap.servers", "kafka-broker-1.example.com:9093");props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer"); // TLS Configurationprops.put("security.protocol", "SSL");props.put("ssl.truststore.location", "/etc/kafka/secrets/kafka.truststore.jks");props.put("ssl.truststore.password", "trustpass");props.put("ssl.truststore.type", "JKS"); // Client authenticationprops.put("ssl.keystore.location", "/etc/kafka/secrets/client.p12");props.put("ssl.keystore.password", "clientpass");props.put("ssl.keystore.type", "PKCS12"); // FIPS TLS versionsprops.put("ssl.enabled.protocols", "TLSv1.2,TLSv1.3"); // Hostname verificationprops.put("ssl.endpoint.identification.algorithm", "HTTPS"); KafkaProducer<String, String> producer = new KafkaProducer<>(props);

Consumer with Client TLS

// Java Consumer with FIPS TLSProperties props = new Properties();props.put("bootstrap.servers", "kafka-broker-1.example.com:9093");props.put("group.id", "my-group");props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer"); // TLS Configuration (same as producer)props.put("security.protocol", "SSL");props.put("ssl.truststore.location", "/etc/kafka/secrets/kafka.truststore.jks");props.put("ssl.truststore.password", "trustpass");props.put("ssl.enabled.protocols", "TLSv1.2,TLSv1.3"); KafkaConsumer<String, String> consumer = new KafkaConsumer<>(props);

Command Line Tools with TLS

# Producer CLI with FIPS TLSbin/kafka-console-producer.sh \  --bootstrap-server kafka-broker-1.example.com:9093 \  --topic my-topic \  --producer-property security.protocol=SSL \  --producer-property ssl.truststore.location=/etc/kafka/secrets/kafka.truststore.jks \  --producer-property ssl.truststore.password=trustpass \  --producer-property ssl.enabled.protocols=TLSv1.2,TLSv1.3 # Consumer CLI with FIPS TLSbin/kafka-console-consumer.sh \  --bootstrap-server kafka-broker-1.example.com:9093 \  --topic my-topic \  --consumer-property security.protocol=SSL \  --consumer-property ssl.truststore.location=/etc/kafka/secrets/kafka.truststore.jks \  --consumer-property ssl.truststore.password=trustpass

SASL with FIPS TLS

SASL provides authentication while FIPS TLS provides encryption:

# server.properties - SASL/SCRAM with TLS # Authentication mechanismsecurity.protocol=SASL_SSLsasl.mechanism.inter.broker.protocol=SCRAM-SHA-256sasl.enabled.mechanisms=SCRAM-SHA-256 # SASL configurationsasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="broker" password="broker-secret"; # TLS Configuration (same as above)ssl.keystore.location=/etc/kafka/secrets/kafka-broker-1.p12ssl.keystore.password=brokerpassssl.truststore.location=/etc/kafka/secrets/kafka.truststore.jksssl.truststore.password=trustpass # FIPS TLS versionsssl.enabled.protocols=TLSv1.2,TLSv1.3

Docker: Kafka with FIPS TLS

# Dockerfile: Kafka with FIPS TLSFROM openjdk:11-jre-slim # Install Kafka and Bouncy Castle FIPSRUN apt-get update && apt-get install -y \  kafka \  openssl-fips \  ca-certificates # Copy FIPS-compliant certificatesCOPY kafka-broker-1.p12 /etc/kafka/secrets/COPY kafka.truststore.jks /etc/kafka/secrets/ # Copy FIPS configurationCOPY server.properties /etc/kafka/ # Set FIPS modeENV KAFKA_HEAP_OPTS="-Dcom.sun.jndi.ldap.connect.pool=false" EXPOSE 9092 9093ENTRYPOINT ["kafka-broker-start.sh", "/etc/kafka/server.properties"]

FIPS Verification

Verify Kafka Uses FIPS TLS

# Check broker TLS configurationbin/kafka-configs.sh \  --bootstrap-server localhost:9093 \  --entity-type brokers \  --entity-name 0 \  --describe \  --command-config client-ssl.properties | grep -i ssl # Output shows:# ssl.enabled.protocols=TLSv1.2,TLSv1.3# ssl.cipher.suites=TLS_AES_256_GCM_SHA384,... # Test TLS connectionopenssl s_client \  -connect kafka-broker-1.example.com:9093 \  -cert client.crt \  -key client.key \  -CAfile ca.crt # Output shows:# subject=CN = kafka-broker-1.example.com# issuer=CN = Kafka-CA# TLSv1.3, Cipher = TLS_AES_256_GCM_SHA384

Automated FIPS Verification

# CleanStart FIPS verification for Kafkacleanimg-init --fips-verifier --image my-kafka:latest # Output:# ✓ Java TLS provider supports FIPS# ✓ Kafka brokers configured with FIPS TLS# ✓ Inter-broker communication uses TLS 1.2/1.3# ✓ Client authentication required# ✓ FIPS-approved cipher suites only# ✓ Certificate validation enabled

Common Issues

Issue: Hostname Verification Fails

ERROR: PKIX path validation failed: java.security.cert.CertPathValidatorExceptionSOLUTION:1. Verify certificate CN matches hostname2. Check advertised.listeners matches certificate CN3. Or disable hostname verification (not recommended):   ssl.endpoint.identification.algorithm=

Issue: Broker Startup Fails with TLS Error

ERROR: Keystore password is incorrectSOLUTION:1. Verify keystore password matches configuration2. Verify keystore file exists at specified path3. Test keystore independently:   keytool -list -v -keystore kafka-broker-1.p12 -storepass brokerpass

Issue: Client Cannot Connect

ERROR: SSL handshake failedSOLUTION:1. Verify client certificate is signed by trusted CA2. Ensure client truststore contains CA certificate3. Check TLS version compatibility (1.2 or 1.3)4. Test with openssl s_client first

Performance Considerations

TLS overhead typically ranges from 5-10% CPU for broker communication. With hardware acceleration, the throughput impact is less than 2%. Latency adds approximately 0.5-1ms per message, which is acceptable for compliance requirements.

Compliance Mapping

Requirement	Implementation
FIPS 140-3	Java FIPS TLS provider
NIST 800-171 SC-13	TLS 1.2+, AES-256-GCM, RSA-2048
PCI DSS 4.1	Encryption in transit with TLS
HIPAA § 164.312(a)(2)(i)	Encryption between brokers and clients