Cost to Build an Internal Knowledge Assistant 2026:
Confluence, Notion & Docs Q&A Infrastructure
Real infrastructure costs for an internal knowledge assistant (company wiki Q&A, HR bot, policy search) in 2026. Includes embeddings, vector database, RAG pipeline, and LLM costs at different team sizes. Last verified: 2026-04-01.
Infrastructure Components
An internal knowledge assistant (RAG-based Q&A over internal docs) requires four cost components:
| Component | One-time cost | Monthly recurring | Notes |
|---|---|---|---|
| Document embedding (one-time) | $0.50–5 | ~$0 (re-run on changes) | 1,000–50,000 docs × ~500 tokens each |
| Vector database hosting | Free–$70 | $0–70/mo | Pinecone free up to 1M vectors; pay tiers above |
| Query embedding (per search) | — | $0.01–0.50/mo | Negligible: $0.02/1M tokens |
| LLM inference (answer generation) | — | $15–200/mo | Dominates cost; varies by query volume + model |
| API hosting (backend) | — | $10–40/mo | Serverless (Vercel, Lambda) for low traffic |
Embedding Cost: One-Time Ingestion
Using OpenAI text-embedding-3-small at $0.02/1M tokens:
| Knowledge Base Size | Estimated tokens | One-time embed cost | Monthly re-embed (10% change) |
|---|---|---|---|
| Small (100 docs, 25-person team) | 500K tokens | $0.01 | $0.001/mo |
| Medium (2,000 docs, 100-person team) | 10M tokens | $0.20 | $0.02/mo |
| Large (20,000 docs, 500-person) | 100M tokens | $2.00 | $0.20/mo |
| Enterprise (100,000 docs, 2,000+) | 500M tokens | $10.00 | $1.00/mo |
Embedding cost is essentially free — LLM inference at query time dominates the budget.
LLM Cost Per Query
A typical internal Q&A query: 3,000 input tokens (system prompt + retrieved context chunks + user question) + 300 output tokens (answer).
| Model | Cost/query | 500 queries/mo (25 users) | 5,000 queries/mo (100 users) | 25,000 queries/mo (500 users) |
|---|---|---|---|---|
| Gemini 2.5 Flash-Lite | $0.000420 | $0.21 | $2.10 | $10.50 |
| GPT-5.4 nano | $0.000975 | $0.49 | $4.88 | $24.38 |
| Claude Haiku 4.5 | $0.004500 | $2.25 | $22.50 | $112.50 |
| Claude Sonnet 4.6 | $0.013500 | $6.75 | $67.50 | $337.50 |
| GPT-5.4 | $0.012000 | $6.00 | $60.00 | $300.00 |
3,000 input + 300 output tokens per query. Assuming 20 queries/user/month on average.
All-In Monthly Cost by Team Size
| Team size | Queries/mo | Flash-Lite + Pinecone | Haiku 4.5 + Pinecone | Sonnet 4.6 + Pinecone |
|---|---|---|---|---|
| 25 people | 500 | $10.21 | $12.25 | $16.75 |
| 100 people | 2,000 | $10.84 | $79 | $205 |
| 500 people | 10,000 | $74.20 | $295 | $745 |
| 2,000 people | 40,000 | $226.80 | $1,050 | $2,920 |
Includes Pinecone serverless ($70/mo for <1M vectors, scales above). Flash-Lite cost dominated by fixed infra at small scale.
Model Selection for Knowledge Assistants
Use Gemini 2.5 Flash-Lite or GPT-5.4 nano when:
- Your documents are factual and structured (policies, FAQs, how-to guides)
- Questions are direct and closed-ended ("What is our vacation policy?")
- You need to keep costs under $50/month for a small team
- Retrieval quality is high — the model just formats the retrieved answer
Use Claude Haiku 4.5 when:
- Questions often require synthesizing across multiple documents
- Users ask nuanced or ambiguous questions ("How do we handle X in Y situation?")
- You need reliable structured extraction (filling forms, generating summaries)
- Your system prompt is large and reused — caching at $0.10/M vs $1.00/M saves 90%
Use Claude Sonnet 4.6 when:
- Documents contain complex technical content (legal, engineering, compliance)
- Accuracy and hallucination rate are critical (compliance, HR policy)
- Users expect GPT-quality reasoning with nuanced, well-cited answers
Prompt Caching: The Biggest Lever for Knowledge Assistants
If your system prompt includes a large set of fixed instructions or a context preamble, Claude's prompt caching slashes costs dramatically:
- System prompt: 2,000 tokens (company context, instructions) — cached at $0.10/M on Haiku vs $1.00/M uncached
- At 10,000 queries/month: saves $18/month in system prompt costs alone
- For even larger system prompts (5,000+ tokens with included policy excerpts), savings grow proportionally
With aggressive caching on Claude Haiku 4.5, effective per-query cost can drop below GPT-5.4 nano's uncached cost at moderate-to-high query volumes.
Buy vs Build: SaaS Knowledge Platforms
Platforms like Notion AI, Confluence AI, and Guru handle retrieval + LLM out of the box:
- Notion AI: $10/user/month addon — 100 users = $1,000/month vs $79 DIY with Haiku
- Guru AI: $18/user/month — 100 users = $1,800/month
- Custom build break-even: ~20 users — above that, DIY almost always wins on cost
- Build time: A basic RAG knowledge assistant takes 2–4 days to build and deploy
Calculate Your Knowledge Assistant Monthly Cost
Enter team size, query volume, and model to estimate your monthly spend.
AI API Cost Calculator