The default configuration when VT Writer is installed uses a local Postgres database. This guide shows you how to connect your VT Writer instance (installed on Red Hat) to an external Postgres database.

Note: in all snippets below replace <external-postgres-server-or-ip> with your external Postgres server hostname or IP.

 

Prerequisites

- We support PostgreSQL versions 10 - 15 and recommend using a version minimum of 12. You must have the postgres-contrib package installed.

- You must be running with VT Writer v 3.1 or later.

- You will need the pgcrypto PostgreSQL extension; this will be enabled by the setup scripts below, but in some cases (such as a locked-down RDS instance) your Database Administrator will need to grant users the ability to enable this extension. For example: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/RDS_delegated_ext.html

 

Setup external VisibleThread database

VT Writer uses 3 databases. The VT Readability database is used to store details of user accounts, the documents they have processed and the document analysis. 

The VT Insights database is used to track analysis and usage statistics over time. This is a reporting database. 

The third database, referred to as the VT API database is used as temporary storage for each analysis job. As a document is being analyzed, data relating to the analysis job and progress is placed in this database. Once the analysis job is complete, the data is removed.

 

If you have a new VT Writer install, please follow Creating the VT Writer and VT Insights Databases. If you have an existing VT Writer install and would like to migrate your data to the external database, please follow Migrating Existing Data to the External Database.  After either, please continue to Manually Change VT Writer configuration to use external database.

 

Creating the VT Writer and VT Insights Databases

If you have a new VT Writer deployment to connect to an external database, please follow the steps below. Note, if using cloud database service, e.g. AWS RDS skip to "Configure External Database"

On the external database server, edit the postgresql.conf file to listen for connections.

Under the Connections and Authentication section, remove the front # (hashtag) from listen_addresses and in single quotes enter the ip address of the postgres server.

Example:

 

1. Configure PostgreSQL to allow access to the VT Insights, VT Writer and VT API DB

Add the following to the bottom of pg_hba.conf:

host vtreadability vtreadability <vt-writer-server-ip/32> md5
host vtinsights vtinsights <vt-writer-server-ip/32> md5
host vtapi vtapi <vt-writer-server-ip/32> md5
host all postgres <vt-writer-server-ip/32> trust

Restart the postgres service:

systemctl restart *postgresql*

 

Configure External Database 

To connect to your external PostgreSQL database run the configure-visiblethread.sh script located in /opt/visiblethread/setup/configure-visiblethread.sh . Answer each question prompt. Note in the screen shot below that the values in [ ] are default values and will be used if you proceed without input.

You must answer no to the first two questions to configure an external database.

1. 1. ”Do you plan to use or are going to use PostgreSQL database locally?” Answer N

   2. “Would you like to go with the default database configuration?“ Answer N
      
      
      

   3. “Do you have any special connection parameters for your database servers?“ Answer Y if you wish to encrypt the connection to the postgres database. You will then be prompt to enter the parameter, type “sslmode=require“ and press enter to complete.
      
      Next you will now have to configure the three databases (VT Writer, VT Insights and VT API). In the case of all three were you are asked :

   4. “Please enter the hostname for the server [localhost]“ you should enter the hostname/ip of the external database in all three cases.
      If you wish to change the default database name, user and password please enter the information for each prompt. If you want to use the default database name, user and password just press enter.
      
      Lastly, you will be asked:

   5. “Are you able to connect to remote postgresql server as user with admin privileges?“ the answer should be ‘y'. You’ll be then prompted for the admin username and password.

The database configuration should now take place and restart all necessary services when complete.

Manually Change VT Writer configuration to use external database

After configuring the external database via the configure-visiblethread.sh script all the values are stored in /etc/default/visiblethread.env. If for some reason you need to update your external database information you can edit the database values here. You’ll have to restart the VT Writer services for the changes to take place.

/opt/visiblethread/VisibleThreadTools/vt-services.sh

 

Verify that VT Writer is using the external DB

In your web browser, login to VT Writer and run a text scan.  If you migrated data to the external server, look at the Analysis History to ensure that you can view the previous scans.

If you run the following in the external database you should see the text you just analyzed, if you migrated data you should also see previous scans that were restored into the database:   

psql 
\c vtreadability 3select title, started_date from scans order by id desc;

Recommended config for external Postgres

We recommend that you make this change to your external postgresql.conf :

max_connections = 200 
max_locks_per_transaction=500