gpt-oss Hardware Sizing Calculator: OpenAI's Open-Weight Models On-Prem
This free gpt-oss hardware sizing calculator estimates GPU count, VRAM, and capital cost for OpenAI's Apache 2.0 licensed open-weight models, and it is built for teams evaluating gpt-oss-20b or gpt-oss-120b for on-prem deployment. Enter the variant, precision, context length, and concurrency, and the tool returns weight footprint, KV cache, total VRAM, GPU count, and cost. Both gpt-oss models ship natively quantized to MXFP4, a roughly 4-bit format, which means the memory savings other open-weight models only achieve through post-training quantization come built in, making gpt-oss-120b deployable on a single 80GB GPU and gpt-oss-20b deployable on a single consumer card.
Your numbers
Both variants ship natively in MXFP4 and activate a small fraction of total parameters per token via MoE routing.
MXFP4 is gpt-oss's native release format, roughly 4 bits per parameter, and the recommended default.
Both variants support up to 128K context.
Simultaneous in-flight requests during peak hours.
VRAM per GPU; enter a matching price below.
H100 80GB runs roughly $25,000-$32,000; RTX 4090 runs $1,600-$2,200 for smaller deployments.
On-demand rates vary by GPU class; H100 typically runs $2-$6 per hour.
Your results
Planning estimates only. Real memory use depends on serving engine support for native MXFP4 kernels. Benchmark on representative traffic before finalizing a purchase order.
Get your full gpt-oss deployment plan
We will email you a personalized sizing breakdown for both gpt-oss variants plus a serving-engine compatibility checklist, and a Netray AI specialist will follow up.
No spam. Your results stay private. Unsubscribe anytime.
How gpt-oss runs on hardware you own
Both gpt-oss models are mixture-of-experts architectures: gpt-oss-120b routes 5.1B active parameters per token through 117B total, and gpt-oss-20b routes 3.6B active parameters through 21B total. As with any MoE model, the full expert set must be resident in memory regardless of routing. What makes gpt-oss unusual is that OpenAI released both models natively in MXFP4, so the memory savings of aggressive quantization are the default rather than an optional post-processing step. At native MXFP4, gpt-oss-120b needs roughly 58.5GB, fitting a single 80GB H100 with room to spare, and gpt-oss-20b needs roughly 10.5GB, fitting comfortably on a single 24GB consumer GPU.
- gpt-oss-120b at native MXFP4 needs roughly 58.5GB, fitting a single 80GB GPU.
- gpt-oss-20b at native MXFP4 needs roughly 10.5GB, fitting a single 24GB consumer GPU.
- Active parameters (5.1B and 3.6B respectively) are unusually small relative to total size, keeping compute cost low.
- Apache 2.0 licensing removes the commercial-use restrictions some other open-weight models carry.
Quantization tradeoffs for gpt-oss
MXFP4 is not a compromise for gpt-oss the way post-training INT4 quantization can be for other models; it is the format OpenAI trained and evaluated the released checkpoints in, so it carries the least risk of unexpected quality loss. Dequantizing to FP8 or FP16 roughly doubles or quadruples the memory footprint for a quality difference most enterprise evaluations will not detect, since the model was not further optimized at those precisions. The practical reason to consider FP8 is hardware compatibility: not every serving stack has mature MXFP4 kernel support yet, and dequantizing to a more universally supported format may be the pragmatic choice on some infrastructure.
- MXFP4 is gpt-oss's native, evaluated release format and the recommended default wherever supported.
- Dequantizing to FP8 or FP16 roughly doubles or quadruples memory for minimal quality gain.
- Choose a higher precision mainly for serving-stack compatibility, not for expected quality improvement.
- Verify your serving engine has mature native MXFP4 kernel support before committing to a hardware plan.
Serving gpt-oss with vLLM and SGLang
vLLM and SGLang both added native MXFP4 support for gpt-oss shortly after release, including expert-parallel MoE serving, which is what makes single-GPU deployment of gpt-oss-120b practical despite its 117B total parameter count. Because both models are unusually small at their active-parameter level relative to total size, throughput per GPU tends to be strong once weights are loaded, closer to what you would expect from a much smaller dense model. Confirm your serving engine version explicitly supports MXFP4 kernels for your target GPU generation before finalizing a deployment plan, since kernel support has been an active area of development since release.
- vLLM and SGLang both support native MXFP4 serving with expert-parallel MoE routing.
- Strong per-GPU throughput relative to total parameter count due to small active-parameter footprint.
- Confirm MXFP4 kernel support for your specific GPU generation before committing hardware.
- Single-GPU deployment of gpt-oss-120b removes multi-GPU interconnect complexity entirely.
When to fine-tune gpt-oss instead of prompting
gpt-oss-20b is an attractive fine-tuning target precisely because its small active-parameter footprint keeps training iteration fast even though 21B total parameters must be handled during the process; LoRA and QLoRA are both practical on a single workstation-class GPU. Fine-tune when you need consistent output structure or domain terminology adherence that Apache 2.0 licensing lets you deploy commercially without the licensing review some other open-weight models require. gpt-oss-120b fine-tuning needs more substantial infrastructure but remains far more accessible than a similarly-capable dense model given its small active-parameter compute cost during training.
How Netray deploys and customizes gpt-oss on-prem
Netray deploys gpt-oss for manufacturers who want the permissive Apache 2.0 license alongside genuinely low hardware requirements: gpt-oss-120b's single-GPU footprint makes it one of the more capital-efficient large models we recommend for customers starting their on-prem AI program. We configure MXFP4-native serving in vLLM or SGLang, fine-tune for domain-specific tasks, and integrate outputs with SyteLine, LN, and M3. Engagements typically start with a two-week pilot proving the model against your actual documents and workflows before scaling to production.
Frequently Asked Questions
Why does gpt-oss-120b fit on a single GPU when other 100B+ models do not?
Because it ships natively quantized to MXFP4, a roughly 4-bit format, rather than requiring a separate post-training quantization step. At MXFP4, its 117B total parameters need only about 58.5GB of memory, comfortably under the 80GB capacity of a single H100 or A100. Most other large open-weight models are released in BF16 or FP8 first, and users have to apply their own quantization, which introduces additional accuracy risk that gpt-oss's native release avoids.
What does Apache 2.0 licensing actually change for enterprise use?
Apache 2.0 is one of the most permissive open-source licenses available, allowing commercial use, modification, and redistribution without royalty obligations or the usage restrictions some other open-weight model licenses impose above certain revenue or user thresholds. For enterprises that need legal certainty about commercial deployment without a lengthy license review, this is a meaningful practical advantage over models released under more restrictive community licenses.
How does gpt-oss compare to Qwen3 or Gemma 3 for single-GPU deployment?
gpt-oss-20b and Qwen3-8B or Gemma 3 4B occupy a similar single-GPU deployment niche, and the right choice depends on task-specific benchmark performance rather than hardware requirements alone, since all fit comfortably on a single card at appropriate quantization. gpt-oss-120b is unusual in offering flagship-scale capability on a single 80GB GPU, a footprint that models like Llama 3.3 70B or Mistral Large need two or more GPUs to match at comparable precision.
Does MXFP4 quantization hurt accuracy compared to a model trained in BF16?
OpenAI evaluated and published benchmark results for the MXFP4 checkpoints directly, so published scores already reflect this precision rather than being measured at a higher precision and assumed to degrade under quantization. In practice this means MXFP4 carries less uncertainty than post-training quantization of a model released in a different native format, though you should still validate against your own evaluation set for domain-specific tasks.
Get a gpt-oss deployment plan sized for single-GPU efficiency and benchmarked on your own workloads.
Related Tools
Gemma 3 Deployment Calculator
Size VRAM, GPU count, and capital cost across the Gemma 3 family, from the 1B edge model to the 27B flagship, with sliding-window attention keeping KV cache low.
On-Prem AILlama 4 Hardware Requirements Calculator
Estimate VRAM, GPU count, and capital cost to run Llama 4 Scout or Maverick on your own hardware, accounting for full mixture-of-experts weight loading and KV cache growth.
On-Prem AISelf-Hosted LLM Hardware Estimator
Estimate the VRAM footprint, GPU count, and hardware budget required to self-host an open-weight LLM with your concurrency and context needs.
Go Deeper
gpt-oss On-Prem Deployment: An Enterprise Guide
Deploy OpenAI's gpt-oss models on-prem: 20B and 120B MoE sizes, Apache 2.0 license, GPU sizing, serving setup, fine-tuning, and when to use it over rivals.
The 2026 Open-Weight LLM Landscape: A Practical Map
A practical map of the 2026 open-weight LLM landscape: model families, license terms, and which model fits your VRAM budget and use case.
How Netray Evaluates Every New AI Model in 48 Hours
Netray's repeatable framework for evaluating any new open-weight AI model release within 48 hours: license, hardware fit, benchmarks, and deployment risk.