A RAG (Retrieval-Augmented Generation) service is a system that combines two functions: retrieving relevant information from external sources (retrieval) and using a language model to generate a response (generation). When you ask it a question, the system first looks for relevant documents or data, and then uses that information to generate a more informed and accurate response. This approach helps ensure that the output is both contextually relevant and up-to-date.

In VT Writer, you have the option of using an additional RAG feature via VT Mimir. This is an additional, optional component to the VT Writer LLM functionality and is deployed via Docker container. In this configuration, RAG is used to help provide a larger context window for the LLM interface (see this article for more information: https://www.appen.com/blog/understanding-large-language-models-context-windows). In our Chat interface, you can utilize this Document Based query to provide the LLM with more information for its response, making its output more accurate. See below screenshot for an example:

VT Mimir architecture

VT Mimir is run via Docker container, so Docker is a prerequisite; we also recommend utilizing Docker Compose.

The architecture is twofold: there is the vt_mimir container running our RAG application, and a vt_pgvector container which is running an ephemeral PostgreSQL database with the pgvector extension installed. Both of these containers can be destroyed at will and contain no relevant data, when data is processed via Mimir it is returned to the VT Writer application, so these containers do not require any backup strategy, etc. The vt_pgvector container will briefly store the document from your chat query for processing; it is deleted once Mimir responds back to VT Writer.

Configuring Mimir for use with VT Writer

Follow these instructions for installing Docker for your server: https://docs.docker.com/engine/install/

Follow these instructions for installing Docker Compose for your server: https://docs.docker.com/compose/install/

Once those are installed, you can use the following .env file and docker-compose.yml file to create the requisite containers for Mimir. Modify the .env file values according to your needs; in particular MIMIR_SERVER_PORT, which is the port Mimir will listen on for your server, and POSTGRES_PASSWORD, which should be a more complex value.

.env

docker-compose.yml

Place these files in a folder on your server; once created you can use the following command to pull the container images from our Docker registry and spin up the VT Mimir service:

docker compose up -d

You will be able to see the progress, and verifying the container status is accomplished via running:

docker ps -a

Successful results:

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9b3822607aac public.ecr.aws/m9k9e9p1/vt_mimir:latest "java -Xmx2048m -jar…" 9 seconds ago Up 8 seconds 0.0.0.0:8088->8088/tcp mimir
b0261bcfdc93 public.ecr.aws/m9k9e9p1/vt_pgvector:latest "docker-entrypoint.s…" 9 seconds ago Up 9 seconds 5432/tcp mimir_db

You can also view logs for either container via the following:

# view the full log file for either container
docker logs mimir
docker logs mimir_db

# view the last 100 log lines for either container
docker logs -f --tail 100 mimir
docker logs -f --tail 100 mimir_db