Model Context Window Planner: Budget Every Token in Your Prompt
This free model context window planner allocates a fixed token budget across system instructions, retrieved documents, conversation history, and reserved output, and it is built for AI engineers and solution architects designing RAG and agent systems. Enter your model's window, prompt sizes, retrieval depth, and history policy, and the tool returns total context consumed, window utilization, and how many additional chunks still fit inside a safety margin. Context overflow is the single most common cause of silent quality failures in enterprise RAG: the system does not error, it quietly drops the passage that held the answer.
Your numbers
The total token budget shared by prompt and response. Self-hosted open-weight models often run 8K-128K.
Instructions, persona, output schema, and JSON tool specs. Agent tool definitions grow faster than teams expect.
Size of one retrieved passage after chunking. ERP work instructions and specs commonly run 800-2,000.
Top-k passed to the model after reranking. More is not better once recall saturates.
How many prior exchanges you replay verbatim before summarizing or truncating.
A user message plus its assistant reply. Verbose tool outputs inflate this quickly.
Space held back for the response. Reserve too little and answers get truncated mid-sentence.
Your results
Estimates only. Tokenizers differ across model families, so real counts can vary 10-20% for the same text. Always validate against the tokenizer your deployed model actually uses.
Get your full context budget plan
We will email you a personalized token allocation plan with chunking, reranking, and history-compression recommendations, and a Netray AI architect will follow up on your retrieval design.
No spam. Your results stay private. Unsubscribe anytime.
How the budget is calculated
Every component competes for the same window. The planner sums four claims: the system prompt and tool definitions, the retrieval payload, the replayed conversation history, and the output space you hold in reserve. With the defaults, a 1,200-token system prompt plus twelve 1,200-token chunks plus six 350-token turns plus 1,500 reserved output tokens totals 19,200 tokens, which is about 59% of a 32,768-token window. Headroom is then computed against 90% of the window rather than 100%, leaving a deliberate safety margin, so roughly eight more chunks would still fit. Push past that margin and a single unusually long document will trigger truncation or an outright error.
Planning benchmarks that hold up in production
These reference points come from enterprise RAG deployments over ERP documentation, engineering specifications, quality records, and service histories. They are starting positions to tune against an evaluation set, not universal constants, and the right values differ meaningfully between a parts-lookup assistant and a specification-interpretation tool. The pattern we see repeatedly is teams over-retrieving early because more context feels safer, then discovering that precision fell and latency doubled without any accuracy gain. Establish a golden question set first, then move one variable at a time and measure. Guessing at these numbers is how a promising pilot quietly becomes an unreliable one.
- Retrieval quality typically saturates between 5 and 15 well-reranked chunks; adding more usually hurts precision and always raises cost.
- Keep total context utilization under 70% for interactive assistants so an outlier document does not break the request.
- Reserve at least 1,000-2,000 output tokens for explanatory answers, and far more for code or structured document generation.
- Summarize conversation history after roughly six turns instead of replaying it verbatim; it holds quality while cutting tokens sharply.
Interpreting utilization and headroom
Utilization under 50% means you are leaving retrieval quality on the table and could pass more evidence or richer instructions. Between 50% and 75% is the comfortable operating band for most enterprise assistants. Above 85% you are in fragile territory, and you should either move to a larger window, shrink chunk size, or compress history. Also watch the effect of long context on accuracy: many models show measurable degradation on facts buried in the middle of a very large prompt, so a 128K window is not a licence to skip reranking. If headroom returns zero, your current configuration cannot serve a worst-case query at all and needs restructuring before launch.
How Netray tunes context for enterprise workloads
Netray builds retrieval systems over manufacturing content that is messy by nature: revision-controlled work instructions, engineering drawings, supplier certifications, and decades of ERP transaction history in SyteLine or LN. We tune chunk boundaries to document structure rather than fixed character counts, add reranking so the top ten passages genuinely deserve their place, and implement history summarization that preserves decisions while discarding noise. For air-gapped and ITAR-constrained customers we do all of this with open-weight models running inside your network. Engagements usually start by measuring answer quality against a golden question set before any tuning begins.
Frequently Asked Questions
Does a bigger context window remove the need for good retrieval?
No. Larger windows reduce hard truncation failures but do not fix relevance. Models frequently show weaker recall for facts positioned in the middle of very long prompts, and every extra token raises both latency and cost. In practice a well-reranked ten-chunk prompt beats a hundred-chunk dump on both accuracy and speed. Treat a large window as insurance against outliers, not as a substitute for chunking and reranking work.
How should I size chunks for ERP and engineering documents?
Chunk on structure rather than character counts wherever possible: one work-instruction step, one specification clause, one service history entry. That usually lands between 600 and 1,500 tokens. Overlap of 10-20% preserves context across boundaries. Fixed-size chunking splits tables and numbered procedures mid-thought, which is why so many first-generation RAG pilots over manufacturing content return confident but incomplete answers.
What happens if my prompt exceeds the context window?
Behavior varies and none of the options are good. Some APIs return a hard error, some silently truncate from the front, and some frameworks drop the oldest history without telling you. Front truncation is the most dangerous because it removes your system prompt, so the model loses its instructions and output schema while still producing fluent text. Always validate token counts before the call and fail loudly rather than truncating.
Get a context and retrieval architecture tuned to your documents, your model, and your accuracy targets.
Related Tools
Private RAG Corpus Sizing Calculator
Estimate chunk counts, vector index storage, raw text volume, and embedding compute time before you build a private retrieval system over your document estate.
On-Prem AILLM Token Cost Calculator
Turn request volume, prompt length, and per-million token pricing into a defensible monthly and annual LLM budget, including the effect of prompt caching.
On-Prem AIAI Inference Latency Calculator
Estimate decode throughput, time to first token, and end-to-end response time for a self-hosted model from GPU memory bandwidth, parameter count, and quantization.
Go Deeper
On-Prem LLM Deployment Architecture: Reference Guide
Reference architecture for on-prem LLM deployment: inference servers, GPU sizing, RAG pipelines, and security zones for regulated manufacturers.
Small Language Models vs LLMs for Enterprise Workloads
Small language models vs LLMs for enterprise workloads: accuracy, cost, GPU footprint, and when a 7B model beats a frontier model for ERP automation.