How can we help?

Search the knowledge base or browse by topic.

Follow

How to deploy VTRAG for VisibleThread 7 on RHEL 8 and RHEL 9

VTRAG is the service behind the VisibleThread 7 AI features that search your documents. It turns your documents into embeddings with an embeddings model you provide, stores them in PostgreSQL with pgvector, and answers searches from VisibleThread.

This guide installs VTRAG on the same RHEL 8 or RHEL 9 server as VisibleThread 7. On Windows, see How to deploy VTRAG for VisibleThread 7 on Windows.

Before you start

You need:

  • VisibleThread 7 installed on RHEL 8 or RHEL 9, as in How to deploy VisibleThread 7 on RHEL 8 and RHEL 9. That install provides PostgreSQL 16 and Java 17, which VTRAG uses.
  • An embeddings model in Azure OpenAI or Amazon Bedrock, or an OpenAI-compatible endpoint.
  • Root or sudo access.

VisibleThread is already configured to reach VTRAG on the same server.

Step 1: Install the VTRAG package

Download the package and check it:

curl -O https://visiblethread.s3.us-east-1.amazonaws.com/public/download/vtrag-1.0.12-1.x86_64.rpm
md5sum vtrag-1.0.12-1.x86_64.rpm

The MD5 should be 15418e19c25b6900d8e491437a0e27af.

Install it:

sudo yum install -y ./vtrag-1.0.12-1.x86_64.rpm

Step 2: Choose your embeddings model

Edit /etc/default/vtrag.env:

sudo nano /etc/default/vtrag.env

The file ships set up for a local Ollama server as a placeholder. Change SPRING_PROFILE and add the settings for your provider below. Leave the database settings as they are; the provisioning script uses them.

Azure OpenAI

Create an embeddings deployment in your Azure OpenAI resource first, for example text-embedding-3-large. Use the deployment name, which can differ from the model name.

Set SPRING_PROFILE and add this block, with your values in the first four lines:

SPRING_PROFILE="azure-openai"

## Azure OpenAI ##
AZURE_OPENAI_API_KEY_ID="<your-api-key>"
AZURE_OPENAI_ENDPOINT="https://<your-resource>.cognitiveservices.azure.com/"
AZURE_OPENAI_DEPLOYMENT_NAME="<your-deployment-name>"
AZURE_OPENAI_CHAT_ENABLED="false"

## Azure OpenAI - do not modify ##
SPRING_AI_AZURE_OPENAI_API_KEY="${AZURE_OPENAI_API_KEY_ID}"
SPRING_AI_AZURE_OPENAI_ENDPOINT="${AZURE_OPENAI_ENDPOINT}"
SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DEPLOYMENT_NAME="${AZURE_OPENAI_DEPLOYMENT_NAME}"
SPRING_AI_AZURE_OPENAI_CHAT_ENABLED="${AZURE_OPENAI_CHAT_ENABLED}"
SPRING_AI_VECTORSTORE_PGVECTOR_DIMENSIONS = 1024
SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DIMENSIONS = 1024

text-embedding-3-small and text-embedding-3-large both work at 1024 dimensions. text-embedding-ada-002 does not support the dimensions setting.

Amazon Bedrock

VTRAG uses the Titan Text Embeddings V2 model (amazon.titan-embed-text-v2:0) and signs in with the standard AWS credentials on the server. On EC2 that is the instance's IAM role, and you need no keys in the file.

Set SPRING_PROFILE and add your region:

SPRING_PROFILE="bedrock"
AWS_REGION=<your-region>

The shipped file has no AWS_REGION line, so add one. Use a region where Titan Text Embeddings V2 is available.

The server's IAM role needs permission to call the model:

{
  "Effect": "Allow",
  "Action": "bedrock:InvokeModel",
  "Resource": "arn:aws:bedrock:<your-region>::foundation-model/amazon.titan-embed-text-v2:0"
}

OpenAI or an OpenAI-compatible endpoint

Use this for the OpenAI API or any endpoint that accepts /v1/embeddings with a bearer token, such as a vLLM server or an internal AI gateway.

SPRING_PROFILE="openai"

## OpenAI / OpenAI-compatible ##
OPENAI_URL=https://api.openai.com
OPENAI_API_KEY_ID=<your-api-key>
SPRING_AI_OPENAI_EMBEDDING_OPTIONS_MODEL=text-embedding-3-large
SPRING_AI_OPENAI_EMBEDDING_OPTIONS_DIMENSIONS=1024

OPENAI_URL is the base URL only. VTRAG adds /v1/embeddings itself, so do not include it.

Step 3: Adjust the provisioning script

VisibleThread already installed PostgreSQL 16 and Java 17, so turn those steps off. Edit the script:

sudo nano /opt/vtrag/setup/provision.sh

Comment out these two lines:

#db_install || error_exit "Error installing PostgreSQL."
#java_install || error_exit "Error installing Java."

Leave pgvector_install and db_config on. They add pgvector to your existing PostgreSQL and create the VTRAG database. The Apache and firewall steps are already off by default.

Step 4: Run the provisioning script

sudo /opt/vtrag/setup/provision.sh

Answer y when it asks you to confirm. It ends with Starting VT RAG service... and Finished provisioning.

Step 5: Check VTRAG

sudo systemctl status vtrag
curl -s -o /dev/null -w '%{http_code}\n' http://localhost:8010/actuator/health

The service should be active (running) and the health check should print 200.

Changing provider later

Edit /etc/default/vtrag.env as in step 2, then restart VTRAG:

sudo systemctl restart vtrag

Troubleshooting

  • Provisioning failed: check /var/log/vt-rag-provision.log.
  • VTRAG will not start or indexing fails: check /var/log/vtrag/vtrag.log. A connection or authentication error there usually means a wrong endpoint, key, deployment name or region in vtrag.env.
  • Bedrock access denied: the server's IAM role is missing bedrock:InvokeModel on the Titan V2 model, or the region in AWS_REGION is wrong.
Was this article helpful?
0 out of 0 found this helpful

Get Additional Help

Reach out to support@visiblethread.com to create a ticket.