Note: This article applies to "Customer-managed" environments where a customer has deployed and is managing the VisibleThread application. It is not applicable to VisibleThread GovCloud or VisibleThread public cloud.
VisibleThread 7 connects to a Large Language Model (LLM) that you host, for example in Azure OpenAI or Amazon Bedrock. This article explains:
- which models we recommend for each AI feature,
- how much throughput (tokens per minute, or TPM) to give your model deployment, and
- which VisibleThread version you need for each model.
If your AI features return errors such as "The AI service is busy right now", "The AI service took too long to respond", or HTTP 429 / exceeded token rate limit" in the logs, the TPM sections below are the place to start.
How much each feature asks of the model
| Feature | Largest prompt VisibleThread sends | Largest response VisibleThread requests |
|---|---|---|
| Visi Chat | Your question plus the retrieved document snippets and chat history | 32,000 tokens |
| Technical Alignment & Outline Generation | Up to about 64,000 tokens per request (large solicitations are split into several requests) | 32,000 tokens |
| Outline refinement | Smaller follow-up requests | 8,000 tokens |
A token is roughly 4 characters of English text, so 64,000 tokens is about 250,000 characters, or around 100 pages.
Recommended models
Choose a model with a context window of at least 400,000 tokens and a maximum output of at least 32,000 tokens. All the models below meet that requirement.
OpenAI GPT models (Azure OpenAI or OpenAI)
| Model | Context window | Max input | Max output | Recommendation |
|---|---|---|---|---|
| gpt-5.6-terra / gpt-5.6-luna | 1,050,000 | 922,000 | 128,000 | Recommended. Best results for Technical Alignment & Outline Generation. Requires VisibleThread 7.3.8 or later. Available in Azure Government (usgovarizona, usgovvirginia) |
| gpt-5.4 | 1,050,000 | 922,000 | 128,000 | Recommended for commercial Azure. Large context and faster than GPT-5.1 |
| gpt-5.5 | 1,050,000 | 922,000 | 128,000 | Supported |
Azure-specific notes:
- In VisibleThread, enter the deployment name exactly as it appears in Azure. This isn't always the same as the model name.
- GPT-5.x models are reasoning models. They take longer to respond than older models such as GPT-4.1, especially for large outlines.
Anthropic Claude models (Amazon Bedrock)
| Model | Context window | Max output | Recommendation |
|---|---|---|---|
| Claude Sonnet 4.6 | 1,000,000 | 128,000 | Supported, recommended model |
| Claude Sonnet 4.5 | 200,000 | 64,000 | Minimum supported version |
Tokens per minute (TPM)
Why TPM matters
When a request arrives, Azure OpenAI counts the prompt plus the maximum response size requested against your TPM limit before the model starts work, even if the actual response is much shorter. A single Outline Generation request can therefore reserve close to 100,000 tokens. If your deployment's TPM is lower than that, the request is rejected with a 429 error, even when nobody else is using the model.
Minimum and recommended TPM
| Feature | Minimum TPM | Recommended TPM |
|---|---|---|
| Technical Alignment & Outline Generation | 100,000 | 250,000 or more, especially if several people generate outlines at the same time |
| Visi Chat | 50,000 (one user at a time) | 100,000 or more for teams |