Overview

This guide explains how to configure Apache HTTP Server to proxy requests to VT Writer Tomcat over an encrypted TLS connection. This is recommended when Apache and Tomcat are running on separate servers to ensure secure communication between the web server and application server.

Prerequisites

• VT Writer successfully installed and running
• Apache HTTP Server installed with mod_ssl and mod_proxy modules enabled
• Root or sudo access on both the Apache server and VT Writer Tomcat server
• Basic understanding of SSL/TLS certificates

Note: This guide assumes Apache is handling client-facing SSL/TLS (port 443) and focuses on securing the backend connection between Apache and Tomcat.

Architecture

Client --HTTPS--> Apache Server (443) --HTTPS--> VT Writer Tomcat Server (8080)

Step 1: Generate a Keystore and Certificate for Tomcat

VT Writer uses a Spring Boot embedded Tomcat server, which supports multiple keystore formats:

• PKCS12 (.p12, .pfx) - Recommended, cross-platform standard
• JKS (Java KeyStore) - Legacy Java format
• PEM (.pem, .crt, .key) - Supported in modern Spring Boot versions

Option A: Create a Self-Signed Certificate (for internal/testing)

Generate a PKCS12 keystore with a self-signed certificate:

keytool -genkeypair -alias tomcat \
  -keyalg RSA -keysize 2048 -validity 365 \
  -keystore /opt/visiblethread/certs/keystore.p12 \
  -storetype PKCS12 \
  -storepass changeit \
  -dname "CN=vtwriter.example.com,OU=IT,O=YourCompany,L=City,ST=State,C=US"

Note: Replace vtwriter.example.com with your VT Writer server hostname and update the password (changeit).

Option B: Use an Existing Certificate (CA-signed or from internal CA)

If you have a certificate from a Certificate Authority or internal CA, you'll need to import it into a keystore.

If you have PEM files (certificate + private key + chain):

# If chain certificates are in separate files, combine them first
# Order: your certificate, intermediate cert(s), root cert
cat /path/to/certificate.crt /path/to/intermediate.crt /path/to/root.crt > /tmp/fullchain.pem

# Create PKCS12 keystore with the full certificate chain
openssl pkcs12 -export \
  -in /tmp/fullchain.pem \
  -inkey /path/to/private-key.key \
  -out /opt/visiblethread/certs/keystore.p12 \
  -name tomcat \
  -passout pass:changeit

# Clean up temporary file
rm /tmp/fullchain.pem

Note: The -certfile option can also be used if your chain is in a separate file:

openssl pkcs12 -export \
  -in /path/to/certificate.crt \
  -inkey /path/to/private-key.key \
  -certfile /path/to/ca-chain.crt \
  -out /opt/visiblethread/certs/keystore.p12 \
  -name tomcat \
  -passout pass:changeit

If you need to add chain certificates to an existing keystore:

# Import intermediate certificate
keytool -import -trustcacerts -alias intermediate1 \
  -file /path/to/intermediate.crt \
  -keystore /opt/visiblethread/certs/keystore.p12 \
  -storetype PKCS12 \
  -storepass changeit

# Import root certificate
keytool -import -trustcacerts -alias root \
  -file /path/to/root.crt \
  -keystore /opt/visiblethread/certs/keystore.p12 \
  -storetype PKCS12 \
  -storepass changeit

Verify the certificate chain:

keytool -list -v -keystore /opt/visiblethread/certs/keystore.p12 \
  -storetype PKCS12 -storepass changeit | grep -A 5 "Certificate chain length"

You should see "Certificate chain length: X" where X is the number of certificates in your chain (typically 2-3).

If you have a .pfx or .p12 file, you can use it directly or convert to JKS:

# Use PKCS12 directly (recommended)
cp /path/to/existing.p12 /opt/visiblethread/certs/keystore.p12

# OR convert to JKS if needed
keytool -importkeystore \
  -srckeystore /path/to/existing.p12 -srcstoretype PKCS12 \
  -destkeystore /opt/visiblethread/certs/keystore.jks -deststoretype JKS

Set Proper Permissions

sudo chown visiblethread:visiblethread /opt/visiblethread/certs/keystore.p12
sudo chmod 600 /opt/visiblethread/certs/keystore.p12

Step 2: Export Certificate for Apache to Trust

Apache needs to trust the Tomcat certificate. Export the certificate from the keystore in PEM format:

keytool -export -alias tomcat \
  -keystore /opt/visiblethread/certs/keystore.p12 \
  -storetype PKCS12 \
  -storepass changeit \
  -rfc \
  -file /opt/visiblethread/certs/tomcat.crt

Important: The -rfc flag exports the certificate in PEM format, which Apache requires.

Verify the certificate is in PEM format:

cat /opt/visiblethread/certs/tomcat.crt

You should see -----BEGIN CERTIFICATE----- at the beginning.

If Apache and Tomcat are on separate servers, copy this certificate file to your Apache server:

scp /opt/visiblethread/certs/tomcat.crt user@apache-server:/etc/pki/tls/certs/

Step 3: Configure VT Writer to Use TLS

VT Writer is a Spring Boot application that reads environment variables from /etc/default/visiblethread.env. Configure TLS by adding the SSL environment variables to this file.

Edit the VT Writer configuration file:

sudo vi /etc/default/visiblethread.env

Add the following SSL environment variables at the end of the file:

# SSL/TLS Configuration for Tomcat
SERVER_SSL_ENABLED=true
SERVER_SSL_KEY_STORE=/opt/visiblethread/certs/keystore.p12
SERVER_SSL_KEY_STORE_PASSWORD=changeit
SERVER_SSL_KEY_STORE_TYPE=PKCS12
SERVER_SSL_KEY_ALIAS=tomcat

# Optional: Enforce TLS 1.2 or higher
SERVER_SSL_ENABLED_PROTOCOLS=TLSv1.2,TLSv1.3

Alternative: Using PEM files directly (if you prefer not to use keystores):

# SSL/TLS Configuration using PEM files
SERVER_SSL_ENABLED=true
SERVER_SSL_CERTIFICATE=/opt/visiblethread/certs/certificate.crt
SERVER_SSL_CERTIFICATE_PRIVATE_KEY=/opt/visiblethread/certs/private-key.key

# Optional: Specify certificate chain (intermediate + root certificates)
SERVER_SSL_CERTIFICATE_CHAIN=/opt/visiblethread/certs/chain.pem

Note: Spring Boot environment variables use the SPRING_ prefix followed by UPPERCASE with underscores. Spring Boot automatically converts these to the corresponding property names (e.g., SPRING_SERVER_SSL_ENABLED becomes server.ssl.enabled).

Save the file and restart VT Writer:

sudo systemctl restart vt-readability.service

Verify VT Writer is now listening on HTTPS:

systemctl status visiblethread-writer.service
sudo ss -tlnp | grep 8080

Check the VT Writer logs to confirm SSL is enabled:

sudo journalctl -u vt-readability.service -n 50 | grep -i "tomcat.*ssl\|port.*8080"

You should see messages about Tomcat starting with SSL enabled on port 8080.

Test the HTTPS connection locally:

curl -k https://localhost:8080/

Note: The -k flag bypasses certificate verification for testing. You should see a response from VT Writer.

Step 4: Configure Apache to Proxy via TLS

On your Apache server, configure the proxy to use HTTPS when connecting to Tomcat.

Edit your Apache virtual host configuration (typically in /etc/httpd/conf.d/vtwriter.conf or /etc/httpd/conf.d/visiblethread.conf):

<VirtualHost *:443>
    ServerName vtwriter.example.com
    
    # Client-facing SSL configuration
    SSLEngine on
    SSLCertificateFile /etc/pki/tls/certs/vtwriter-public.crt
    SSLCertificateKeyFile /etc/pki/tls/private/vtwriter-public.key
    SSLCertificateChainFile /etc/pki/tls/certs/vtwriter-chain.crt
    
    # Enable SSL proxy engine
    SSLProxyEngine On
    
    # Trust the Tomcat backend certificate
    SSLProxyCACertificateFile /opt/visiblethread/certs/tomcat.crt
    
    # Verify the backend certificate
    SSLProxyVerify require
    SSLProxyCheckPeerName on
    
    # Proxy configuration
    ProxyPreserveHost On
    ProxyPass / https://vtwriter-backend.example.com:8080/
    ProxyPassReverse / https://vtwriter-backend.example.com:8080/
    
    # Headers for proper forwarding
    RequestHeader set X-Forwarded-Proto "https"
    RequestHeader set X-Forwarded-Port "443"
    
</VirtualHost>

Important Configuration Notes:

• SSLProxyCACertificateFile: Points to the Tomcat certificate exported in Step 2. This tells Apache to trust the backend certificate.
• SSLProxyVerify require: Enforces certificate verification for security. This prevents man-in-the-middle attacks.
• SSLProxyCheckPeerName on: Ensures the certificate hostname matches the backend server.
• Replace vtwriter-backend.example.com with your actual VT Writer Tomcat server hostname.

Testing First: Disable Certificate Verification

For initial testing, you may want to temporarily disable backend certificate verification to ensure connectivity works:

SSLProxyEngine On
# Temporarily disable verification for testing
SSLProxyVerify none
SSLProxyCheckPeerCN off
SSLProxyCheckPeerName off

After confirming the connection works, re-enable certificate verification for production:

SSLProxyEngine On
SSLProxyCACertificateFile /opt/visiblethread/certs/tomcat.crt
SSLProxyVerify require
SSLProxyCheckPeerName on

Alternative: Using a CA Certificate

If your Tomcat certificate is signed by an internal CA or public CA, you can configure Apache to trust the CA instead:

SSLProxyCACertificateFile /etc/pki/tls/certs/ca-bundle.crt

This allows you to trust multiple backend servers signed by the same CA.

Step 5: Enable Required Apache Modules

Ensure the necessary Apache modules are enabled:

sudo dnf install -y mod_ssl

Verify modules are loaded:

httpd -M | grep -E 'proxy|ssl'

You should see:

• proxy_module
• proxy_http_module
• ssl_module

If any are missing on RHEL 9, they should be enabled by default with the mod_ssl package.

Restart Apache

sudo systemctl restart httpd
sudo systemctl status httpd

Verify End-to-End Connection

From a client machine:

curl -v https://vtwriter.example.com/

You should receive a successful response from VT Writer.

Check Apache Logs

If you encounter issues, check the Apache error log:

sudo tail -f /var/log/httpd/error_log

Common errors:

• Certificate verification failed: Ensure the Tomcat certificate is correctly exported with -rfc flag and the path in SSLProxyCACertificateFile is correct
• Connection refused: Verify VT Writer Tomcat is listening on 8080 and the hostname/IP is correct
• SSL handshake failed: Check that TLS protocols match between Apache and Tomcat

Supported Certificate Formats Summary

Recommended approach: Use PKCS12 (.p12) format for Tomcat keystore and export to PEM format for Apache trust.

Security Best Practices

1. Use strong passwords: Replace changeit with a strong, unique password
2. Restrict file permissions: Ensure keystore files are only readable by the visiblethread user (chmod 600)
3. Use certificates from a trusted CA: Self-signed certificates are acceptable for internal networks but use CA-signed certificates for production
4. Enable certificate verification: Always use SSLProxyVerify require in production
5. Keep certificates up to date: Monitor expiration dates and renew certificates before they expire
6. Use TLS 1.2 or higher: Disable older protocols (SSL 3.0, TLS 1.0, TLS 1.1)
7. Regularly update: Keep Apache, Tomcat, and OpenSSL packages updated

Troubleshooting

VT Writer won't start after enabling TLS

Check the service logs:

sudo journalctl -u vt-readability.service -f

Common issues:

• Incorrect keystore path or password: Verify the path in /etc/default/visiblethread.env and password are correct
• Keystore file not readable: Ensure the file has proper permissions (chmod 600) and is owned by visiblethread user
• Invalid keystore format: Verify the keystore using keytool -list -v -keystore /opt/visiblethread/certs/keystore.p12
• Missing certificate alias: Ensure the alias specified in the environment variables exists in the keystore
• Configuration not loading: Spring Boot reads environment variables from /etc/default/visiblethread.env
• Wrong variable format: Environment variables must be SPRING_UPPERCASE_WITH_UNDERSCORES, not lowercase.with.dots

Spring Boot not reading SSL configuration

Verify the environment variables file is being loaded:

cat /etc/default/visiblethread.env | grep SSL

Ensure the variables are in the correct format (SPRING_ prefix with UPPERCASE and underscores):

• Correct: SPRING_SERVER_SSL_ENABLED=true
• Incorrect: SERVER_SSL_ENABLED=true or server.ssl.enabled=true

Check the VT Writer startup logs for any configuration errors:

sudo journalctl -u visiblethread-writer.service -n 100 | grep -i "error\|exception"

Apache can't connect to Tomcat

Error: "SSL_ERROR_RX_RECORD_TOO_LONG" or connection timeouts

This means Apache is trying to connect via HTTPS but Tomcat is still running HTTP. Verify Tomcat has SSL enabled:

openssl s_client -connect localhost:8080

If this fails or shows HTTP errors, Tomcat isn't running HTTPS. Check that the SSL properties were added to /etc/default/visiblethread.env and VT Writer was restarted.

Error: "AH01895: Unable to configure verify locations for client authentication"

The Tomcat certificate file isn't in the correct PEM format. Re-export with the -rfc flag:

keytool -export -alias tomcat \
  -keystore /opt/visiblethread/certs/keystore.p12 \
  -storetype PKCS12 \
  -storepass changeit \
  -rfc \
  -file /opt/visiblethread/certs/tomcat.crt

Verify it's PEM format by checking for -----BEGIN CERTIFICATE----- at the top:

head -1 /opt/visiblethread/certs/tomcat.crt

Error: "AH02312: Fatal error initialising mod_ssl" or "Private key not found"

Apache can't find the client-facing SSL certificate or key file. Verify:

ls -la /etc/pki/tls/certs/vtwriter-public.crt
ls -la /etc/pki/tls/private/vtwriter-public.key

Ensure both SSLCertificateFile and SSLCertificateKeyFile directives are present in your Apache configuration.

Certificate verification failures

Ensure the certificate exported from Tomcat matches what's configured in Apache:

openssl x509 -in /opt/visiblethread/certs/tomcat.crt -text -noout

Check the Subject and Issuer fields match your Tomcat certificate.

If using a wildcard certificate: Ensure the hostname in ProxyPass matches the certificate's CN or SAN entries. For example, if your cert is for *.visiblethread.com, use a hostname like vtwriter.visiblethread.com instead of localhost.

Testing backend TLS connection

Verify TLS is working on Tomcat from the Apache server:

openssl s_client -connect vtwriter-backend.example.com:8080 -showcerts

This should establish a connection and display the certificate chain. Press Ctrl+C to exit.

You can also test with curl:

curl -vk https://vtwriter-backend.example.com:8080/

This should return a response from VT Writer.

Additional Resources

• Apache mod_proxy documentation: https://httpd.apache.org/docs/2.4/mod/mod_proxy.html
• Apache mod_ssl documentation: https://httpd.apache.org/docs/2.4/mod/mod_ssl.html
• Spring Boot SSL/TLS configuration: https://docs.spring.io/spring-boot/docs/current/reference/html/application-properties.html#application-properties.server

Need Help?

If you encounter issues not covered in this guide, please contact VisibleThread Support with:

• VT Writer version number
• Relevant log excerpts from both Apache (/var/log/httpd/) and VT Writer (from journalctl -u vt-readability.service)
• Operating system version (cat /etc/redhat-release)
• Apache version (httpd -v)
• Java version (java -version)