Air-Gapped LLM Deployment Checklist | Netray

0%

0 of 25 items complete

7 critical items still open - these are the highest-risk gaps.

Pre-deployment model and data staging

Every model weight file is downloaded on a connected staging system and its checksum verified against the publisher's published hash before transferCRITICALModel files are inspected for format; only safetensors artifacts cross the air gap, never pickle-based checkpointsCRITICALA complete offline package mirror (Python wheels, CUDA libraries, serving engine dependencies) is built on the staging system before transferLicense terms for each model are reviewed and archived alongside the weight files before they cross the boundaryTokenizer files, configuration files, and model cards are transferred as a complete matched set, never partiallyA documented bill of materials lists every file, its source, checksum, and transfer date for each model release

Transfer mechanism and media control

Transfers use a defined one-way path, such as a data diode or a controlled write-once media process, not ad hoc USB copyingCRITICALRemovable media used for transfer is dedicated to this purpose, scanned, and logged before and after each useCRITICALA named approver signs off on every transfer before it crosses the boundary, with the approval loggedMalware scanning runs on the staging side before transfer and again on the air-gapped side after receiptTransfer media is wiped or destroyed per policy after use, not reused indefinitely without a defined lifecycle

Update and patching workflow

A recurring cadence exists for pulling security patches for the OS, GPU drivers, and serving engine on the staging systemCRITICALPatches are tested on a staging replica that mirrors the air-gapped environment before being transferred for production useA rollback plan exists for every update, tested at least once, not assumed to workCVE monitoring for the serving stack (vLLM, TensorRT-LLM, llama.cpp, or equivalent) is assigned to a named owner on the connected sideModel version updates follow the same staging, checksum, and approval process as the initial deployment, with no shortcuts for minor version bumps

Operational monitoring inside the boundary

Inference requests are logged locally with identity, timestamp, model version, and a correlation ID, since no external SIEM can reach the boundaryCRITICALLocal log storage has sufficient capacity and retention for your compliance requirement without relying on external shippingGPU health, driver status, and thermal metrics are monitored locally with alerting that does not depend on internet-connected toolingA defined process exists for periodically exporting logs out of the boundary for centralized review, following the same one-way transfer control as model updatesAccess to the air-gapped environment is logged separately from inference activity, covering both physical and administrative access

Integrity verification and drift detection

Deployed model checksums are periodically re-verified against the original staging record to detect tampering or corruptionCRITICALConfiguration drift between the staged and deployed environment is checked on a defined schedule, not assumed to stay in syncA documented baseline exists for expected model behavior, so unexpected output changes can be investigated as a possible integrity issuePeriodic internal audits confirm the deployed environment matches the approved bill of materials, with discrepancies formally tracked

An air-gapped LLM deployment is defensible when at least 90% of all items are complete and every critical item is closed. Critical items here center on the transfer boundary itself: an unverified checksum, an uncontrolled media path, or missing inference logging inside the boundary each represent a single point of failure that undermines the entire premise of air-gapping, since the boundary only protects you if every crossing is deliberate, logged, and verified.

Get your air-gapped deployment runbook

We will email you a personalized staging, transfer, and patching runbook template, and a Netray infrastructure specialist will follow up on your air-gapped architecture.

No spam. Your results stay private. Unsubscribe anytime.

Why air-gapping changes the entire operating model

A connected deployment can pull a security patch in minutes and roll it back just as fast if something breaks. An air-gapped deployment has to stage, verify, transfer, and test every single change through a deliberate process, which means the temptation to skip a step grows every time a patch is genuinely urgent. The teams that maintain air-gapped AI successfully treat the transfer boundary as the single most important control in the entire system, because everything else, network segmentation, access control, application security, only matters once you have already established that nothing crosses the boundary without verification.

Every model update requires the full staging, checksum, and approval cycle, with no fast path for urgent patches.
Local logging and monitoring must stand entirely on their own, since no external SIEM or alerting service can reach the boundary.
Removable media becomes the single highest-risk component in the entire architecture and needs its own dedicated control process.
Drift between the staged and deployed environment is invisible unless you actively check for it on a schedule.

The controls that matter most

The critical items here cluster around the transfer boundary because that is where an air-gapped deployment's entire security model lives or dies. A model file that was not checksum-verified before transfer could be corrupted, mismatched, or in the worst case tampered with, and once it is inside the boundary there is no external signal that will ever catch that. Uncontrolled media handling turns the one intentional crossing point into an unintentional one. And missing local logging means an air-gapped environment, precisely the one place you might expect the strongest audit trail, ends up with the weakest one, because there is no external system quietly capturing what happened.

Checksum verification on every model file before it crosses the boundary, with no exceptions for urgency.
A defined one-way transfer mechanism instead of ad hoc removable media handling.
Complete local inference and access logging, since nothing outside the boundary can compensate for a gap here.
Periodic re-verification of deployed model checksums to catch drift or tampering after the fact.

How to work through the checklist

Treat this as a gated process rather than a one-time audit, and build the staging and transfer workflow before you deploy your first model, not after. Assign a single named owner for the transfer boundary specifically, separate from general platform engineering, because that role requires a discipline of never taking shortcuts that competes directly with the operational pressure to patch quickly. Re-run the full checklist whenever you add a new model, change your serving stack, or bring a new environment online, since each of those events introduces a fresh staging and transfer cycle that needs the same rigor as the original deployment.

How Netray builds and operates air-gapped AI

Netray deploys fully air-gapped AI infrastructure for aerospace, defense, and manufacturing customers who cannot tolerate any live internet connectivity on systems touching sensitive data. We design the staging and transfer workflow, build local logging and monitoring that stands on its own, and establish the checksum and bill-of-materials discipline that makes every crossing of the boundary verifiable. Because we also handle the model selection and serving stack tuning, your air-gapped deployment gets the same performance engineering as a connected one, without ever compromising the isolation that makes it defensible.

Frequently Asked Questions

How do you patch security vulnerabilities in an air-gapped AI environment?

Through the same staging and transfer discipline used for the initial deployment: pull the patch on a connected staging system, verify it, test it against a replica of the air-gapped environment, and transfer it through your controlled one-way path with an approval and log entry. There is no fast path for urgent patches without breaking the isolation guarantee, which is why staging and testing need to run continuously rather than only when a patch is announced.

Can you run vLLM or TensorRT-LLM in a fully air-gapped environment?

Yes, both serving engines run without any internet dependency once their model weights, container images, and CUDA libraries are staged and transferred. The work is entirely in preparation: building a complete offline package mirror on the connected side so nothing inside the boundary ever attempts to reach a package index or model hub at runtime, and testing that mirror thoroughly before the environment goes live.

What is the biggest mistake teams make with air-gapped AI deployments?

Treating removable media as a convenience rather than a controlled process. The most common failure pattern is an engineer copying a model file to a USB drive without checksum verification or an approval log, under time pressure, because the formal transfer process felt slow. That single shortcut undermines the entire air-gap premise. Build a transfer process fast enough that people do not feel compelled to bypass it.

How do you monitor an air-gapped AI system without a connected SIEM?

Local logging has to be complete and self-sufficient: inference requests, access events, and system health all need local storage with adequate capacity and retention, plus local alerting that does not depend on an internet-connected notification service. Periodically, logs are exported out of the boundary through the same controlled one-way process used for model updates, so centralized review still happens, just on a defined schedule rather than in real time.

Get an air-gapped AI architecture with a transfer and patching workflow your security team can actually defend.

Talk to an Expert Explore More Free Tools

Related Tools

On-Prem AI

AI Data Sovereignty Risk Assessment

Score your organization across eight dimensions of AI data sovereignty risk, from inference location and encryption key custody to subprocessor visibility and audit readiness.

Aerospace & Defense

ITAR AI Workload Compliance Assessment

Score your AI deployments across eight dimensions of ITAR exposure, from technical data classification and US persons access control to technology control plan coverage.