Prices → Calculators → RAG document Q&A
RAG document Q&A: LLM cost calculator
Retrieval-augmented Q&A over a private corpus: several thousand tokens of retrieved chunks per question. Input dominates cost, and retrieved chunks change per query so cache hit rates stay low.
Every model on this workload
Input is 93% of tokens here, so input pricing dominates. Cache applied at 25%.
| Model | In /1M | Out /1M | No cache | Monthly | vs best |
|---|---|---|---|---|---|
| GPT-5.6 Luna OpenAI | $0.100 | $0.600 | $23.20 | $19.60 | cheapest |
| Gemini 2.5 Flash-Lite Google | $0.100 | $0.400 | $20.80 | $20.80 | 1.1x |
| Gemini 3.1 Flash-Lite Google | $0.250 | $1.500 | $58.00 | $58.00 | 3.0x |
| Gemini 3.5 Flash-Lite Google | $0.300 | $2.500 | $78.00 | $78.00 | 4.0x |
| Gemini 2.5 Flash Google | $0.300 | $2.500 | $78.00 | $78.00 | 4.0x |
| Gemini 3.7 Flash Google | $0.750 | $3.750 | $165.00 | $165.00 | 8.4x |
| Gemini 3.6 Flash Google | $0.750 | $3.750 | $165.00 | $165.00 | 8.4x |
| Claude Haiku 4.5 Anthropic | $1.000 | $5.000 | $220.00 | $184.00 | 9.4x |
| GPT-5.6 Terra OpenAI | $1.000 | $6.000 | $232.00 | $196.00 | 10.0x |
| Gemini 2.5 Pro Google | $1.250 | $10.00 | $320.00 | $320.00 | 16.3x |
| Gemini 3.5 Flash Google | $1.500 | $9.000 | $348.00 | $348.00 | 17.8x |
| Claude Sonnet 5 Anthropic | $2.000 | $10.00 | $440.00 | $368.00 | 18.8x |
| GPT-5.3 Codex OpenAI | $1.750 | $14.00 | $448.00 | $385.00 | 19.6x |
| Gemini 3.1 Pro Google | $2.000 | $12.00 | $464.00 | $464.00 | 23.7x |
| GPT-5.6 Sol OpenAI | $2.500 | $15.00 | $580.00 | $490.00 | 25.0x |
| Claude Sonnet 4.6 Anthropic | $3.000 | $15.00 | $660.00 | $552.00 | 28.2x |
| Claude Opus 5 Anthropic | $5.000 | $25.00 | $1,100 | $920.00 | 46.9x |
| Claude Opus 4.5 Anthropic | $5.000 | $25.00 | $1,100 | $920.00 | 46.9x |
| GPT chat-latest OpenAI | $5.000 | $30.00 | $1,160 | $980.00 | 50.0x |
| Claude Fable 5 Anthropic | $10.00 | $50.00 | $2,200 | $1,840 | 93.9x |
| Claude Mythos 5 Anthropic | $10.00 | $50.00 | $2,200 | $1,840 | 93.9x |
| GPT-5.6 Cyber OpenAI | $12.50 | $75.00 | $2,900 | $2,450 | 125.0x |
Adjust the assumptions
The defaults above are a reasonable starting profile, not your profile. Change them.
Cost calculator
Enter your own numbers, or start from a workload preset. Costs are monthly and update as you type.
| Model | In /1M | Out /1M | Monthly | vs best |
|---|
Cutting this bill in practice
Switching from GPT-5.6 Cyber to GPT-5.6 Luna on this workload is worth $29,165 a year. Two things make a number like that collectible rather than hypothetical: being able to change model without an integration rewrite, and knowing when the volume justifies leaving per-token pricing entirely.
Switch models without rewriting your integration
The savings on this page are only real if you can actually move traffic to the cheaper model. Aggregators and gateways sit in front of multiple providers so that switch is a configuration change.
- AIML API
One API surface in front of 200+ models. The reason it matters for cost: switching model becomes a string change rather than an integration rewrite, so the savings this site calculates are actually reachable instead of theoretical.
Caveat: Adds a middleman margin on top of provider rates — check its per-model pricing against the tables above before assuming it is cheaper. - OpenRouter
Provider-agnostic routing with automatic fallback. Useful when the cheapest model for a workload changes month to month — which, given how often the tables on this site move, it does.
Caveat: Routing does not make any individual model cheaper. It reduces the cost of switching, not the token price.
Run open-weight models on rented GPUs
Past a certain monthly volume, renting a GPU beats paying per token — but only for workloads where an open-weight model is good enough, and where utilization is high enough to amortize idle time.
- Novita AI
GPU instances and serverless inference for open-weight models. Relevant once a workload's monthly API bill passes roughly the cost of keeping a GPU warm — the crossover point is arithmetic, and the numbers above give you one side of it.
Caveat: Self-hosting moves cost from a per-token line to an engineering line. Open-weight model quality on your task is a separate question this site does not answer. - RunPod
Per-second GPU rental, including serverless endpoints that scale to zero. The usual first stop for testing whether a self-hosted open model can replace a high-volume API workload.
Caveat: Idle GPU time is billed. A workload with spiky traffic can cost more self-hosted than on a per-token API. - Vast.ai
Marketplace for spot GPU capacity, typically the cheapest per-hour rate available. Best fit for batch work that can tolerate interruption — bulk summarization and offline extraction, not live traffic.
Caveat: Interruptible capacity. Reliability and host quality vary; not appropriate for latency-sensitive production traffic.