LLM API vs Self-Hosted Cost Calculator
This calculator finds the crossover point where self-hosting a large language model becomes cheaper than paying per-token API prices, built for engineering leaders whose LLM API invoices are growing month over month. It models your bill bottom-up from daily requests and the input/output token mix - the split matters because output tokens typically cost 3-5x more - then compares it against an all-in self-hosted monthly cost. The result is a clean monthly and annual savings figure plus per-1,000-request unit economics you can use in feature-level budgeting.
Your numbers
All LLM calls per day, including automated and agentic requests.
Prompt plus context; RAG requests commonly run 1,000-4,000 input tokens.
Generated tokens per response; chat averages 300-700.
Per-million input token price from your provider's rate card.
Per-million output token price; typically 3-5x the input price.
Amortized hardware plus power, support, and ops share. Use our TCO calculator to derive this.
Your results
Estimates only. API rate cards, caching discounts, and batch pricing change frequently; self-hosted quality parity depends on model choice. Validate with a measured pilot.
Get your full API vs self-hosted cost report
We will email a personalized crossover analysis with model recommendations for your token mix, and a Netray specialist will follow up to plan a measured quality pilot.
No spam. Your results stay private. Unsubscribe anytime.
How the math works
Your per-request cost is input tokens times the input rate plus output tokens times the output rate, divided by one million. At the defaults - 1,500 input and 400 output tokens against a $3/$15 rate card - each request costs about one cent, and 25,000 daily requests compound to roughly $7,900 per month. The self-hosted side is a single all-in number you supply: amortized hardware, power, support, and an operations staffing share. We keep it as one input deliberately, because the honest all-in figure is the one self-hosting advocates most often understate - use our On-Prem LLM TCO Calculator to derive it rigorously rather than guessing.
Reading the crossover correctly
The comparison assumes the self-hosted model delivers acceptable quality for your workload, which is a real question, not a footnote. In practice the pattern by workload type is consistent across Netray engagements.
- Classification, extraction, and RAG answering: 8B-32B open models match frontier APIs; self-hosting wins early
- Code generation and complex reasoning: parity needs 70B-class models, raising self-hosted cost
- Agentic workloads firing thousands of automated calls: token volume explodes; self-hosting wins decisively
- Low-volume, high-stakes tasks: stay on APIs; hardware would idle
What to do with your result
If monthly savings are strongly positive, the next step is not a purchase order - it is a quality pilot proving an open-weight model handles your top request types, since cost savings at degraded quality are savings nobody will accept. If the number is near zero, watch your request growth rate: API bills scale linearly while self-hosted cost is a step function, so a workload growing 15% monthly crosses over within two quarters. If savings are negative, self-hosting still may be justified by data sovereignty, latency, or compliance - just fund it as a risk decision, not a cost decision.
How Netray helps you capture the savings
Netray runs the migration end to end: we benchmark open-weight candidates against your real prompt logs, size and deploy the serving infrastructure, and implement routing so high-stakes requests can still reach a frontier API while the bulk of traffic runs on your hardware. That hybrid pattern typically captures 70-90% of the modeled savings at zero perceived quality loss. Engagements start with a two-week evaluation using your actual traffic sample, so the business case is measured rather than modeled.
Frequently Asked Questions
Why do input and output tokens need separate prices?
Because providers charge asymmetrically - generating tokens costs more compute than reading them, and rate cards reflect that with output prices typically 3-5x input. Workload shape therefore changes your bill dramatically: a RAG pipeline stuffing 4,000 context tokens per request is input-dominated, while a report generator is output-dominated. Modeling the mix explicitly keeps the comparison honest instead of relying on a blended average that may not match your traffic.
What should I include in the self-hosted monthly cost?
Everything: hardware capex amortized over three years (with a 25-30% uplift over GPU list price for the server platform), power and cooling, vendor support contracts, and a realistic share of an engineer's time - typically 0.3-0.7 FTE for a small production cluster. The most common analysis error is entering only amortized GPU cost, which understates the true figure by 40-60% and produces a crossover that operations reality will not honor.
Do prompt caching and batch API discounts change the conclusion?
They can shift the crossover meaningfully, so model them. Prompt caching cuts input costs 50-90% for workloads reusing large fixed contexts, and batch endpoints often discount 50% for latency-tolerant jobs. Apply those discounts to your effective rates in this calculator to see the adjusted picture. Note that self-hosted stacks get equivalent tricks - prefix caching in vLLM is free - so the discounts partially cancel rather than settling the question.
Calculate your savings, then let Netray benchmark an open-weight model against your real traffic before you commit.
Related Tools
On-Prem vs Cloud AI Cost Comparison Calculator
Compare your projected cloud LLM API bill against an amortized on-prem GPU deployment and find the breakeven month.
On-Prem AIOn-Prem LLM Total Cost of Ownership Calculator
Model the full multi-year cost of running LLMs on your own hardware, including GPU capex, power, cooling, support contracts, and operations staffing.
On-Prem AIGPU Sizing Calculator for LLM Inference
Work out how many GPUs you need to serve a given open-weight model to your user base, based on memory footprint and token throughput.
Go Deeper
Small Language Models vs LLMs for Enterprise Workloads
Small language models vs LLMs for enterprise workloads: accuracy, cost, GPU footprint, and when a 7B model beats a frontier model for ERP automation.
On-Prem LLM Deployment Architecture: Reference Guide
Reference architecture for on-prem LLM deployment: inference servers, GPU sizing, RAG pipelines, and security zones for regulated manufacturers.