Introduction

VTRAG is a service that allows VT Writer to use Retrieval Augmented Generation (RAG). This guide details how to configure VTRAG using the vtrag.env file.

By default, on Linux this will be located in /etc/default/vtrag.env. The default location for Windows is C:\Program Files\VisibleThread\vtwriter\vtrag\vtrag.env.

Note: On Windows, the VTRAG service will be disabled by default. You can start it and set it to Automatic after configuring C:\Program Files\VisibleThread\vtwriter\vtrag\vtrag.env.

Basic Configuration

System Requirements

• PostgreSQL 16 with pgvector extension enabled (VT Writer on Windows is bundled with PostgreSQL 15, which is a known exception). Existing RDS or Azure PostgreSQL Flexserver deployments will require access to the pgvector extension (referred to as vector)
• JVM memory: 4048m recommended
• VTRAG operates as a REST API on port 8010 by default; standalone servers should open this port so VT Writer can talk to it. This port does not need to be opened if VT Writer and VTRAG are on the same server.

Database Configuration

VTRAG_DB_HOST=localhost
VTRAG_DB_PORT=5432
VTRAG_DB_USER=vtrag
VTRAG_DB_PASS=S3cur3P@ssw0rd!
VTRAG_DB_NAME=vtrag

Memory Settings

VTRAG_JVM_MEM=-Xmx4048m

LLM Service Configuration

VTRAG supports multiple LLM services for embeddings. You must choose one service by setting the appropriate SPRING_PROFILE and removing/commenting out configuration sections for unused profiles.

Choosing Your Embedding Service

Set the SPRING_PROFILE to one of:

• "ollama" - Local Ollama server
• "openai" - OpenAI API, or any OpenAI-compatible endpoint (for example a vLLM server, an internal AI gateway, or LM Studio)
• "azure-openai" - Azure OpenAI Service
• "bedrock" - AWS Bedrock

Example:

SPRING_PROFILE="ollama"

Note: The ollama profile targets a local, unauthenticated Ollama server and does not send an authentication header. If your embeddings run on an endpoint that requires an API key or bearer token, use the openai profile instead - see OpenAI / OpenAI-Compatible Configuration below.

Configuration Examples

Ollama Configuration

Ollama can run entirely offline and doesn't require external API services.

SPRING_PROFILE="ollama"
## Ollama ##
VTRAG_LLM_URL=http://localhost:11434
VTRAG_LLM_MODEL=mxbai-embed-large

Additional configurations (do not modify):

## Ollama ##
SPRING_AI_OLLAMA_BASE_URL="${VTRAG_LLM_URL}"
SPRING_AI_OLLAMA_EMBEDDING_MODEL="${VTRAG_LLM_MODEL}"

OpenAI / OpenAI-Compatible Configuration

The openai profile drives the OpenAI API and any OpenAI-compatible endpoint - a vLLM server, an internal AI gateway, or a managed service that speaks the OpenAI API. If your endpoint accepts a request at /v1/embeddings with an Authorization: Bearer <token> header, use this profile.

SPRING_PROFILE="openai"
## OpenAI / OpenAI-compatible ##
OPENAI_URL=https://api.openai.com
OPENAI_API_KEY_ID=your-api-key
SPRING_AI_OPENAI_EMBEDDING_OPTIONS_MODEL=text-embedding-3-large
SPRING_AI_OPENAI_EMBEDDING_OPTIONS_DIMENSIONS=1024

• OPENAI_URL is the base URL of the endpoint, not the full request URL. VTRAG appends the OpenAI path (/v1/embeddings) itself. Set it to https://api.openai.com for the public OpenAI API, or to your provider's base URL for an OpenAI-compatible endpoint. Do not include /v1/embeddings in the value - if you do, VTRAG builds a doubled path and the call fails.
• OPENAI_API_KEY_ID is the API key. VTRAG sends it as Authorization: Bearer <key>.
• Set the model and dimension to match the model you actually serve (see Setting Embedding Dimensions below). For example, mxbai-embed-large is 1024.

Note: If your provider serves the OpenAI routes under a prefix and you get a 404 after authentication succeeds, adjust whether /v1 belongs in OPENAI_URL - some gateways expect it, most do not.

Azure OpenAI Configuration

SPRING_PROFILE="azure-openai"
## Azure OpenAI ##
AZURE_OPENAI_API_KEY_ID="your-api-key"
AZURE_OPENAI_ENDPOINT="https://your-resource.openai.azure.com/"
AZURE_OPENAI_DEPLOYMENT_NAME="your-deployment-name"
AZURE_OPENAI_CHAT_ENABLED="false"

Additional configurations (do not modify):

## Azure OpenAI ##
SPRING_AI_AZURE_OPENAI_API_KEY="${AZURE_OPENAI_API_KEY_ID}"
SPRING_AI_AZURE_OPENAI_ENDPOINT="${AZURE_OPENAI_ENDPOINT}"
SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DEPLOYMENT_NAME="${AZURE_OPENAI_DEPLOYMENT_NAME}"
SPRING_AI_AZURE_OPENAI_CHAT_ENABLED="${AZURE_OPENAI_CHAT_ENABLED}"

Azure OpenAI Notes:

• Create your own deployment in the Azure portal before configuring
• Recommended models: text-embedding-3-small or text-embedding-3-large
• text-embedding-ada-002 is supported but cannot use the SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DIMENSIONSvariable; see below notes on Embeddings dimensions
• The deployment name may differ from the model name

Windows Format Differences

In Windows, Powershell requires a different syntax. The variables must be prefixed with $ and some with $ENV:. Here's an example working configuration:

$VTRAG_JVM_MEM = "-Xmx4048m"
$VTRAG_DB_HOST = "localhost"
$VTRAG_DB_PORT = 5432
$VTRAG_DB_USER = "vtrag"
$VTRAG_DB_PASS = "xxxxxxxxxxxxxxxxxxxxxxx"
$VTRAG_DB_NAME = "vtrag"
$SPRING_PROFILE="azure-openai"

## Azure OpenAI ##
$AZURE_OPENAI_API_KEY_ID = "xxxxxxxxxxxxxxxxxxxxxxxxxx"
$AZURE_OPENAI_ENDPOINT = "https://test.openai.azure.com/"
$AZURE_OPENAI_DEPLOYMENT_NAME = "text-embedding-3-large"
$AZURE_OPENAI_CHAT_ENABLED = "false"

## Do Not Modify ##

$ENV:SPRING_DATASOURCE_URL = "jdbc:postgresql://${VTRAG_DB_HOST}:${VTRAG_DB_PORT}/${VTRAG_DB_NAME}"
$ENV:SPRING_DATASOURCE_USERNAME = "${VTRAG_DB_USER}"
$ENV:SPRING_DATASOURCE_PASSWORD = "${VTRAG_DB_PASS}"
$ENV:SPRING_SERVLET_MULTIPART_MAX_FILE_SIZE = "19922944"
$ENV:SPRING_SERVLET_MULTIPART_MAX_REQUEST_SIZE = "19922944"
$ENV:SPRING_QUARTZ_ENABLED = "true"
$ENV:SPRING_QUARTS_DOCUMENTCLEANUP_CRON = "0 0 0 * * ?"
$ENV:SPRING_QUARTZ_DOCUMENTCLEANUP_DAYS_OLD = "90"

## Azure OpenAI ##
$ENV:SPRING_AI_AZURE_OPENAI_API_KEY="${AZURE_OPENAI_API_KEY_ID}"
$ENV:SPRING_AI_AZURE_OPENAI_ENDPOINT="${AZURE_OPENAI_ENDPOINT}"
$ENV:SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DEPLOYMENT_NAME="${AZURE_OPENAI_DEPLOYMENT_NAME}"
$ENV:SPRING_AI_AZURE_OPENAI_CHAT_ENABLED="${AZURE_OPENAI_CHAT_ENABLED}"
$ENV:SPRING_AI_VECTORSTORE_PGVECTOR_DIMENSIONS = 1024
$ENV:SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DIMENSIONS = 1024

For the openai profile on Windows, set the profile and the OpenAI variables in the same file. The OpenAI variables are read directly by the service, so prefix them with $ENV:. Keep the rest of the ## Do Not Modify ## block (datasource, multipart, quartz) exactly as shown in the Azure OpenAI example above.

$SPRING_PROFILE = "openai"

## OpenAI / OpenAI-compatible ##
$ENV:OPENAI_URL = "https://api.openai.com"
$ENV:OPENAI_API_KEY_ID = "your-api-key"
$ENV:SPRING_AI_OPENAI_EMBEDDING_OPTIONS_MODEL = "text-embedding-3-large"
$ENV:SPRING_AI_OPENAI_EMBEDDING_OPTIONS_DIMENSIONS = 1024
$ENV:SPRING_AI_VECTORSTORE_PGVECTOR_DIMENSIONS = 1024

Setting Embedding Dimensions

pgvector Compatibility

When using embedding-based search with a vector database like pgvector, it's important to ensure that the embedding dimensions used by your model match those expected by the database. If they don't, you may encounter errors such as:

ERROR: different vector dimensions 1024 and 3072

This happens when the application tries to compare vectors of mismatched sizes—typically between those stored in the database and those produced by the current embedding model.

Why Set the Dimensions Explicitly?

Embedding models can produce vectors of varying lengths depending on the model you select. To avoid runtime failures and ensure compatibility, it's recommended to explicitly set environment variables that define the expected dimensions for:

• The vector database (e.g., pgvector)
• The embedding model (e.g., OpenAI, Azure OpenAI, etc.)

Configuring Embedding Dimensions

Linux

For Linux systems (e.g., Red Hat), update the following file:

/etc/default/vtrag.env

Add or modify these lines:

SPRING_AI_VECTORSTORE_PGVECTOR_DIMENSIONS=1536
SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DIMENSIONS=1536 # comment this out if using text-embedding-ada-002

Replace 1536 with the correct dimension for your chosen embedding model.

Restart the service:

systemctl restart vtrag

Windows

For Windows environments, update:

C:\Program Files\VisibleThread\vtwriter\vtrag\vtrag.env.ps1

Add:

$ENV:SPRING_AI_VECTORSTORE_PGVECTOR_DIMENSIONS = 1536
$ENV:SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DIMENSIONS = 1536 # comment this out if using text-embedding-ada-002

Replace 1536 with the correct dimension for your chosen embedding model.

Restart the service:

Search for "Services" and scroll until you find the vtrag service and restart it.

Common Embedding Dimensions by Model

Note: Always verify the dimensions for your specific model version, as these may change.

Document Handling Configuration

VTRAG processes .docx and .pdf files. These settings control the document management:

SPRING_SERVLET_MULTIPART_MAX_FILE_SIZE=19922944
SPRING_SERVLET_MULTIPART_MAX_REQUEST_SIZE=19922944
SPRING_QUARTZ_ENABLED=true
SPRING_QUARTS_DOCUMENTCLEANUP_CRON="0 0 0 * *?"
SPRING_QUARTZ_DOCUMENTCLEANUP_DAYS_OLD=90

Notes:

• The maximum file size above (approx. 19mb) can be adjusted
• Documents are automatically deleted after the specified period (configurable, default: 90 days)

Performance and Cost Considerations

• Ollama: Requires a server with GPU, typically costing $300-$500/month on cloud providers
• External API services (OpenAI, Azure, AWS): May be more cost-effective for smaller deployments but have ongoing token usage fees
• Data control: Consider Ollama for strict data control requirements

Troubleshooting

401 Unauthorized / Missing Authorization Header

If document upload fails and the VTRAG log shows a 401 from the embeddings endpoint - for example "authorization information was not provided" or "missing authorization header" - the request is reaching the endpoint with no Authorization header. The ollama profile is built for a local Ollama server with no token and does not send an Authorization header, so a token set under that profile is never put on the request and the endpoint returns a 401.

Fix: set SPRING_PROFILE="openai", configure OPENAI_URL and OPENAI_API_KEY_ID as shown in OpenAI / OpenAI-Compatible Configuration, remove the ollama block, and restart VTRAG. The openai profile sends the key as Authorization: Bearer <OPENAI_API_KEY_ID>.

To confirm the endpoint and key independently of VTRAG, send a direct request from the VTRAG host. If it returns embeddings, the endpoint, token, and TLS are good and any remaining failure is VTRAG configuration:

curl --location "https://your-endpoint.example.com/v1/embeddings" \
  --header "Authorization: Bearer your-api-key" \
  --header "Content-Type: application/json" \
  --data '{"model": "your-embedding-model", "input": ["test"], "encoding_format": "float"}'

Note: The embedding profile and its authentication are set only in vtrag.env, not in the VT Writer admin UI. The Generative AI screen's Test Connection checks VT Writer to VTRAG, not VTRAG to the embedding endpoint, so it can pass while embeddings fail. The embeddings call runs only on document ingest, so an authentication problem first appears as a failed upload.

Vector Dimension Mismatch Errors

If you encounter errors about mismatched vector dimensions:

1. Check your embedding model's dimension output
2. Update both SPRING_AI_VECTORSTORE_PGVECTOR_DIMENSIONS and SPRING_AI_AZURE_OPENAI_EMBEDDING_OPTIONS_DIMENSIONS to match
3. Restart the VTRAG service
4. If the error persists, you may need to recreate the vector database tables to match the new dimensions

Service Won't Start

• Verify all required environment variables are set
• Check that PostgreSQL is running and accessible
• Ensure the pgvector extension is enabled in your database
• Review logs for specific error messages