Introduction

VTRAG is a service that allows VT Writer to use Retrieval Augmented Generation (RAG). This guide details how to configure VTRAG using the vtrag.env file.

By default, on Linux this will be located in /etc/default/vtrag.env. The default location for Windows is C:\Program Files\VisibleThread\vtwriter\vtrag\vtrag.env.

Note: On Windows, the VTRAG service will be disabled by default. You can start it and set it to Automatic after configuring C:\Program Files\VisibleThread\vtwriter\vtrag\vtrag.env.

Basic Configuration

System Requirements

• PostgreSQL 16 with pgvector extension enabled (VT Writer on Windows is bundled with PostgreSQL 15, which is a known exception). Existing RDS or Azure PostgreSQL Flexserver deployments will require access to the pgvector extension (referred to as vector)
• JVM memory: 4048m recommended
• VTRAG operates as a REST API on port 8010 by default; standalone servers should open this port so VT Writer can talk to it. This port does not need to be opened if VT Writer and VTRAG are on the same server.

Database Configuration

VTRAG_DB_HOST=localhost
VTRAG_DB_PORT=5432
VTRAG_DB_USER=vtrag
VTRAG_DB_PASS=S3cur3P@ssw0rd!
VTRAG_DB_NAME=vtrag

Memory Settings

VTRAG_JVM_MEM=-Xmx4048m

LLM Service Configuration

VTRAG supports multiple LLM services for embeddings. You must choose one service by setting the appropriate SPRING_PROFILE and removing/commenting out configuration sections for unused profiles.

Choosing Your Embedding Service

Set the SPRING_PROFILE to one of:

• "ollama" - Local Ollama server
• "openai" - OpenAI API
• "azure-openai" - Azure OpenAI Service
• "bedrock" - AWS Bedrock

Example:

SPRING_PROFILE="ollama"

Configuration Examples

Ollama Configuration

Ollama can run entirely offline and doesn't require external API services.

SPRING_PROFILE="ollama"
## Ollama ##
VTRAG_LLM_URL=http://localhost:11434
VTRAG_LLM_MODEL=mxbai-embed-large

Additional configurations (do not modify):

## Ollama ##
SPRING_AI_OLLAMA_BASE_URL="${VTRAG_LLM_URL}"
SPRING_AI_OLLAMA_EMBEDDING_MODEL="${VTRAG_LLM_MODEL}"

Azure OpenAI Configuration

SPRING_PROFILE="azure-openai"
## Azure OpenAI ##
AZURE_OPENAI_API_KEY_ID="your-api-key"
AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
AZURE_OPENAI_DEPLOYMENT_NAME="your-deployment-name"
AZURE_OPENAI_CHAT_ENABLED="false"

Additional configurations (do not modify):

## Azure OpenAI ##
SPRING_AI_AZURE_OPENAI_API_KEY="${AZURE_OPENAI_API_KEY_ID}"
SPRING_AI_AZURE_OPENAI_ENDPOINT="${AZURE_OPENAI_ENDPOINT}"
SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DEPLOYMENT_NAME="${AZURE_OPENAI_DEPLOYMENT_NAME}"
SPRING_AI_AZURE_OPENAI_CHAT_ENABLED="${AZURE_OPENAI_CHAT_ENABLED}"

Azure OpenAI Notes:

• Create your own deployment in the Azure portal before configuring
• Recommended models: text-embedding-3-small or text-embedding-3-large
• text-embedding-ada-002 is supported but cannot use the SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DIMENSIONSvariable; see below notes on Embeddings dimensions
• The deployment name may differ from the model name

Windows Format Differences

In Windows, Powershell requires a different syntax. The variables must be prefixed with $ and some with $ENV:. Here's an example working configuration:

$VTRAG_JVM_MEM = "-Xmx4048m"
$VTRAG_DB_HOST = "localhost"
$VTRAG_DB_PORT = 5432
$VTRAG_DB_USER = "vtrag"
$VTRAG_DB_PASS = "xxxxxxxxxxxxxxxxxxxxxxx"
$VTRAG_DB_NAME = "vtrag"
$SPRING_PROFILE="azure-openai"

## Azure OpenAI ##
$AZURE_OPENAI_API_KEY_ID = "xxxxxxxxxxxxxxxxxxxxxxxxxx"
$AZURE_OPENAI_ENDPOINT = "https://test.openai.azure.com/"
$AZURE_OPENAI_DEPLOYMENT_NAME = "text-embedding-3-large"
$AZURE_OPENAI_CHAT_ENABLED = "false"

## Do Not Modify ##

$ENV:SPRING_DATASOURCE_URL = "jdbc:postgresql://${VTRAG_DB_HOST}:${VTRAG_DB_PORT}/${VTRAG_DB_NAME}"
$ENV:SPRING_DATASOURCE_USERNAME = "${VTRAG_DB_USER}"
$ENV:SPRING_DATASOURCE_PASSWORD = "${VTRAG_DB_PASS}"
$ENV:SPRING_SERVLET_MULTIPART_MAX_FILE_SIZE = "19922944"
$ENV:SPRING_SERVLET_MULTIPART_MAX_REQUEST_SIZE = "19922944"
$ENV:SPRING_QUARTZ_ENABLED = "true"
$ENV:SPRING_QUARTS_DOCUMENTCLEANUP_CRON = "0 0 0 * * ?"
$ENV:SPRING_QUARTZ_DOCUMENTCLEANUP_DAYS_OLD = "90"

## Azure OpenAI ##
$ENV:SPRING_AI_AZURE_OPENAI_API_KEY="${AZURE_OPENAI_API_KEY_ID}"
$ENV:SPRING_AI_AZURE_OPENAI_ENDPOINT="${AZURE_OPENAI_ENDPOINT}"
$ENV:SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DEPLOYMENT_NAME="${AZURE_OPENAI_DEPLOYMENT_NAME}"
$ENV:SPRING_AI_AZURE_OPENAI_CHAT_ENABLED="${AZURE_OPENAI_CHAT_ENABLED}"
$ENV:SPRING_AI_VECTORSTORE_PGVECTOR_DIMENSIONS = 1024
$ENV:SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DIMENSIONS = 1024

Setting Embedding Dimensions

pgvector Compatibility

When using embedding-based search with a vector database like pgvector, it's important to ensure that the embedding dimensions used by your model match those expected by the database. If they don't, you may encounter errors such as:

ERROR: different vector dimensions 1024 and 3072

This happens when the application tries to compare vectors of mismatched sizes—typically between those stored in the database and those produced by the current embedding model.

Why Set the Dimensions Explicitly?

Embedding models can produce vectors of varying lengths depending on the model you select. To avoid runtime failures and ensure compatibility, it's recommended to explicitly set environment variables that define the expected dimensions for:

• The vector database (e.g., pgvector)
• The embedding model (e.g., OpenAI, Azure OpenAI, etc.)

Configuring Embedding Dimensions

Linux

For Linux systems (e.g., Red Hat), update the following file:

/etc/default/vtrag.env

Add or modify these lines:

SPRING_AI_VECTORSTORE_PGVECTOR_DIMENSIONS=1536
SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DIMENSIONS=1536 # comment this out if using text-embedding-ada-002

Replace 1536 with the correct dimension for your chosen embedding model.

Restart the service:

systemctl restart vtrag

Windows

For Windows environments, update:

C:\Program Files\VisibleThread\vtwriter\vtrag\vtrag.env.ps1

Add:

$ENV:SPRING_AI_VECTORSTORE_PGVECTOR_DIMENSIONS = 1536
$ENV:SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DIMENSIONS = 1536 # comment this out if using text-embedding-ada-002

Replace 1536 with the correct dimension for your chosen embedding model.

Restart the service:

Search for "Services" and scroll until you find the vtrag service and restart it.

 

Common Embedding Dimensions by Model

Model	Dimensions
text-embedding-3-small	1536
text-embedding-3-large	3072
text-embedding-ada-002	1536
mxbai-embed-large	1024

Note: Always verify the dimensions for your specific model version, as these may change.

Document Handling Configuration

VTRAG processes .docx and .pdf files. These settings control the document management:

SPRING_SERVLET_MULTIPART_MAX_FILE_SIZE=19922944
SPRING_SERVLET_MULTIPART_MAX_REQUEST_SIZE=19922944
SPRING_QUARTZ_ENABLED=true
SPRING_QUARTS_DOCUMENTCLEANUP_CRON="0 0 0 * *?"
SPRING_QUARTZ_DOCUMENTCLEANUP_DAYS_OLD=90

Notes:

• The maximum file size above (approx. 19mb) can be adjusted
• Documents are automatically deleted after the specified period (configurable, default: 90 days)

Performance and Cost Considerations

• Ollama: Requires a server with GPU, typically costing $300-$500/month on cloud providers
• External API services (OpenAI, Azure, AWS): May be more cost-effective for smaller deployments but have ongoing token usage fees
• Data control: Consider Ollama for strict data control requirements

Troubleshooting

Vector Dimension Mismatch Errors

If you encounter errors about mismatched vector dimensions:

1. Check your embedding model's dimension output
2. Update both SPRING_AI_VECTORSTORE_PGVECTOR_DIMENSIONS and SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DIMENSIONS to match
3. Restart the VTRAG service
4. If the error persists, you may need to recreate the vector database tables to match the new dimensions

Service Won't Start

• Verify all required environment variables are set
• Check that PostgreSQL is running and accessible
• Ensure the pgvector extension is enabled in your database
• Review logs for specific error messages