LLM Quantization Formats Compared: AWQ, GPTQ, FP8, and GGUF
Quantization cuts LLM memory footprint by storing weights in fewer bits, and the format you choose changes your GPU sizing, throughput, and quality by a meaningful margin, not a rounding error. The honest comparison in 2026: FP8 is the best default on Hopper and Blackwell GPUs because it needs no calibration and vLLM and SGLang both support it natively with near-zero quality loss. AWQ is the strongest 4-bit option for GPU serving where memory is the binding constraint. GPTQ is mature and widely supported but generally edged out by AWQ on quality at the same bit width. GGUF is not a serving quantization format at all, it belongs to llama.cpp and Ollama for CPU and edge deployment, not vLLM-class production serving.
FP8: The Default Choice on Modern GPUs
FP8 quantization halves memory versus FP16/BF16 with minimal calibration effort and, on H100, H200, and B200 GPUs, runs on native FP8 tensor cores for a real throughput gain rather than just a memory saving. Quality loss is typically under 1 percent on standard benchmarks for most model families, which is why FP8 has become the default recommendation for new on-prem deployments running on Hopper-class or newer hardware. The catch is hardware dependency: FP8 tensor core acceleration requires Hopper or Blackwell, so on A100 or older cards FP8 gives you the memory saving but not the compute speedup, and int4 formats like AWQ often win on total throughput on that older fleet.
- Memory: roughly 50 percent reduction versus BF16/FP16 weights
- Quality: typically under 1 percent degradation on standard eval suites for well-supported model families
- Hardware: needs Hopper (H100/H200) or Blackwell (B200) for tensor core acceleration; works but does not accelerate on A100
- Tooling: native support in vLLM and SGLang, minimal calibration dataset required versus AWQ or GPTQ
AWQ vs GPTQ: The 4-Bit GPU Serving Comparison
Both AWQ and GPTQ quantize to 4 bits, cutting memory roughly 75 percent versus FP16, which is the difference between needing one H100 versus two for a 70B model. AWQ (Activation-aware Weight Quantization) preserves accuracy better at the same bit width because it identifies and protects the small fraction of weight channels that activations are most sensitive to, rather than quantizing everything uniformly. GPTQ uses a layer-by-layer reconstruction approach and was the earlier standard; it remains well supported and slightly faster to quantize, but head-to-head evaluations consistently show AWQ retaining 1 to 3 points more accuracy on reasoning and coding benchmarks at 4-bit. For new GPU-serving deployments choosing between the two, AWQ is the better default; GPTQ remains reasonable if you already have a quantized checkpoint or tooling investment.
- Memory: both roughly 75 percent reduction versus FP16 at 4-bit
- Quality: AWQ typically retains 1-3 points more accuracy than GPTQ at equivalent bit width on reasoning benchmarks
- Quantization time: GPTQ is generally faster to produce a quantized checkpoint
- Serving support: both work in vLLM; AWQ has broader recent momentum among model providers publishing pre-quantized checkpoints
GGUF: Not a Production GPU Serving Format
GGUF is llama.cpp's file format, built for CPU inference and consumer GPU edge deployment through Ollama and LM Studio, and it supports a wide range of quantization levels from Q2 through Q8. It is the right choice for a developer laptop, an air-gapped edge device, or a single-user local deployment, and the ecosystem tooling around it (Ollama's model library, simple CLI) is unmatched for that use case. It is the wrong choice for multi-user production serving: llama.cpp's server does not implement continuous batching or PagedAttention at the level vLLM and SGLang do, so concurrent throughput at scale is materially worse. Do not conflate GGUF's popularity in the hobbyist and edge community with production readiness for a serving stack handling concurrent enterprise traffic.
Choosing a Format for Your Deployment
Match the format to your hardware and concurrency needs rather than picking whatever the model card links first. On Hopper or Blackwell GPUs serving concurrent production traffic, start with FP8: it is the least effort for the smallest quality tax. If memory is still the binding constraint after FP8, or you are on A100-class hardware, move to AWQ. Reserve GPTQ for cases where a specific model is only available pre-quantized in that format and re-quantizing is not worth the effort. Reserve GGUF for edge, air-gapped single-node, or developer workstation deployments, never for the primary production serving path. Always validate quality loss against your own task-specific eval set, not just published perplexity numbers, since quantization sensitivity varies by model family and by task.
- Hopper/Blackwell + production concurrency: FP8 first
- Memory-constrained GPU serving, any generation: AWQ
- Legacy quantized checkpoint already in GPTQ: keep it unless quality testing says otherwise
- Edge, air-gapped single node, or laptop: GGUF via llama.cpp/Ollama
- Always re-run your own eval set post-quantization; published perplexity deltas do not transfer across tasks
How Netray Picks Quantization Formats for Clients
Netray runs quantization decisions against the client's actual GPU fleet and task-specific eval set rather than defaulting to whatever format is trending. For Hopper and Blackwell deployments we default to FP8 and validate quality against the client's own held-out test cases before go-live. Where legacy A100 fleets or tighter memory budgets are in play, we benchmark AWQ against the client workload directly, since generic perplexity comparisons rarely predict how a quantized model performs on a specific extraction or classification task. Our on-prem inference engagements include this quantization validation as a standard deliverable, not an afterthought discovered in production.
Frequently Asked Questions
Is FP8 or AWQ better for production LLM serving?
On Hopper (H100/H200) or Blackwell (B200) GPUs, FP8 is generally the better default: it needs minimal calibration, retains near-full quality, and gets native tensor core acceleration on that hardware. On A100 or older GPUs, FP8 still saves memory but does not get the compute speedup, so AWQ's larger 4-bit memory reduction often wins on total throughput per dollar. The right answer depends on your GPU generation and how memory-constrained you are, so validate both against your own workload.
Is GGUF suitable for production LLM serving?
No, not for multi-user concurrent production serving. GGUF and llama.cpp are built for CPU inference and single-node edge or developer deployment, and the server does not implement continuous batching or PagedAttention at the level vLLM or SGLang do, so concurrent throughput is materially worse under real traffic. Use GGUF for air-gapped edge devices, developer workstations, or single-user local deployments, and use vLLM with FP8 or AWQ for production serving.
How much accuracy do you lose quantizing a model to 4-bit?
For well-supported model families, AWQ at 4-bit typically loses 1 to 3 percent on standard reasoning and coding benchmarks versus the FP16 original, with GPTQ usually losing slightly more at the same bit width. Actual degradation varies significantly by model architecture and by task, so published perplexity numbers are a starting point, not a guarantee. Always validate against your own task-specific evaluation set before committing a quantization format to production.
Can I switch quantization formats after deploying a model?
Yes, and it is common to revisit the choice as hardware or traffic patterns change. Switching from GPTQ to AWQ, or from int4 to FP8 on newly acquired Hopper hardware, typically requires re-quantizing the checkpoint and re-running your evaluation suite, plus retuning vLLM's --gpu-memory-utilization and --max-num-seqs since the memory footprint changes. Budget a day or two for validation, not just the mechanical re-quantization step.
Key Takeaways
- 1FP8: The Default Choice on Modern GPUs: FP8 quantization halves memory versus FP16/BF16 with minimal calibration effort and, on H100, H200, and B200 GPUs, runs on native FP8 tensor cores for a real throughput gain rather than just a memory saving. Quality loss is typically under 1 percent on standard benchmarks for most model families, which is why FP8 has become the default recommendation for new on-prem deployments running on Hopper-class or newer hardware.
- 2AWQ vs GPTQ: The 4-Bit GPU Serving Comparison: Both AWQ and GPTQ quantize to 4 bits, cutting memory roughly 75 percent versus FP16, which is the difference between needing one H100 versus two for a 70B model. AWQ (Activation-aware Weight Quantization) preserves accuracy better at the same bit width because it identifies and protects the small fraction of weight channels that activations are most sensitive to, rather than quantizing everything uniformly.
- 3GGUF: Not a Production GPU Serving Format: GGUF is llama.cpp's file format, built for CPU inference and consumer GPU edge deployment through Ollama and LM Studio, and it supports a wide range of quantization levels from Q2 through Q8. It is the right choice for a developer laptop, an air-gapped edge device, or a single-user local deployment, and the ecosystem tooling around it (Ollama's model library, simple CLI) is unmatched for that use case.
Put this into numbers
Free interactive tools for exactly this problem. No signup to use them.
LLM Quantization Memory Savings Calculator
Compare FP16, FP8, and INT4 memory footprints for any model size and see how many fewer GPUs quantization requires to serve it.
Free ToolCPU-Only LLM Inference Feasibility Calculator
Estimate realistic tokens per second for CPU-only inference from model size, quantization, and system RAM bandwidth, and see whether it clears your target throughput.
Free ToolGPU Sizing Calculator for LLM Inference
Work out how many GPUs you need to serve a given open-weight model to your user base, based on memory footprint and token throughput.
Terms used in this article
Not sure which quantization format fits your GPU fleet and workload? Netray will benchmark FP8, AWQ, and GPTQ against your own eval set and recommend the right one.
Related Resources
vLLM Production Deployment: A Practitioner's Guide
Deploy vLLM in production: continuous batching, PagedAttention, config flags that matter, and the metrics to watch before you trust it with real traffic.
AI & AutomationKV Cache Optimization: Prefix Caching and Chunked Prefill
KV cache optimization techniques for production LLM serving: prefix caching, chunked prefill, PagedAttention, and sizing memory for concurrent users.
AI & AutomationOn-Prem LLM Inference Hardware in 2026: A Roundup
On-prem LLM inference hardware for 2026: H100 vs H200 vs B200 pricing, when A100 fleets still work, and how to size GPUs against real serving needs.