Why nvidia-smi is missing or cannot talk to the driver, and how to fix it
bash: nvidia-smi: command not found
Also appears as
- NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running
- /dev/nvidia0: No such file or directory
Short answer
nvidia-smi not found or unable to communicate almost always means the NVIDIA kernel module never loaded, and the two most common reasons are that the driver was never installed, or the driver is installed but Secure Boot is blocking the unsigned kernel module from loading. WSL2 users hit a different variant: the driver must be installed on the Windows host, never inside the Linux guest.
Affects: Fresh Linux installs, cloud GPU instances before first boot setup, WSL2 GPU environments, and any machine with Secure Boot enabled in the BIOS.
Fastest path to a working nvidia-smi
- 1Check whether the module even attempted to load: run dmesg | grep -i nvidia.
- 2Check Secure Boot state with mokutil --sb-state; if enabled, either disable it in the BIOS or enroll the MOK key generated during driver install.
- 3Check for a lingering nouveau driver with lsmod | grep nouveau; if present, blacklist nouveau and rebuild the initramfs.
- 4On WSL2, confirm you installed the GPU driver on the Windows host only, never a Linux driver inside the WSL distro.
- 5Reinstall or reconfigure the NVIDIA driver package for your distro, then reboot and rerun nvidia-smi.
How to confirm this is your problem
- Shell reports nvidia-smi: command not found
- nvidia-smi runs but prints a driver communication failure instead of GPU output
- /dev/nvidia0 or /dev/nvidiactl device files are missing under /dev
- dmesg shows the nvidia module failed to load or was tainted/rejected
Root causes and fixes
The NVIDIA driver was never installed
Base OS images and fresh cloud instances rarely ship a GPU driver by default. Without the driver package, there is no nvidia-smi binary and no kernel module to expose the GPU, so any CUDA-aware tool including PyTorch fails immediately at import or first GPU call.
Fix: Install the driver through your distro's package manager (apt install nvidia-driver-XXX on Ubuntu, or the vendor's .run installer), matching a version that supports your GPU generation, then reboot.
ubuntu-drivers devices sudo apt install nvidia-driver-550 sudo reboot
Secure Boot is blocking the unsigned kernel module from loading
Modern Ubuntu and other distros enable UEFI Secure Boot by default, which refuses to load any kernel module that is not signed with a key enrolled in the machine's firmware. The NVIDIA driver installer generates its own signing key (MOK) during install, but if that key is never enrolled through mokutil, the module load is silently rejected.
Fix: Enroll the MOK key the driver installer generated by running mokutil --import and completing the enrollment on next reboot, or disable Secure Boot in the BIOS if your security policy allows it.
mokutil --sb-state sudo mokutil --import /var/lib/shim-signed/mok/MOK.der
The open-source nouveau driver is still bound to the GPU
Linux kernels ship nouveau, an open-source NVIDIA driver, enabled by default. If it loads before or alongside the proprietary driver, it claims the GPU device first and the NVIDIA module refuses to attach, so nvidia-smi cannot find a compatible device even though hardware is present.
Fix: Blacklist nouveau in a modprobe.d config file, regenerate the initramfs, and reboot so only the proprietary driver claims the device.
echo 'blacklist nouveau' | sudo tee /etc/modprobe.d/blacklist-nouveau.conf sudo update-initramfs -u sudo reboot
WSL2 GPU driver installed inside the Linux guest instead of the Windows host
WSL2 GPU passthrough works by exposing the host's driver into the Linux environment automatically. Installing a Linux NVIDIA driver package inside WSL conflicts with this passthrough mechanism and typically breaks nvidia-smi entirely, since WSL expects to use the Windows-side driver, not its own.
Fix: Uninstall any driver package installed inside the WSL distro, install only the current NVIDIA driver on the Windows host, and verify with wsl --update followed by nvidia-smi inside the distro.
Kernel was upgraded but the DKMS module was never rebuilt for it
When the kernel updates via unattended-upgrades or a manual apt upgrade, DKMS is supposed to automatically rebuild the NVIDIA kernel module for the new kernel headers. If the headers package is missing or DKMS fails silently, the old module no longer matches the running kernel and fails to load.
Fix: Install the matching linux-headers package for the running kernel, then force a DKMS rebuild with dkms autoinstall, and reboot.
sudo apt install linux-headers-$(uname -r) sudo dkms autoinstall sudo reboot
Diagnostic commands
Check if the kernel even attempted to load the nvidia module
dmesg | grep -i nvidia
Lines mentioning 'module verification failed' point to Secure Boot rejecting the module. Silence entirely usually means the driver package is not installed at all.
Check Secure Boot state
mokutil --sb-state
SecureBoot enabled combined with a missing MOK enrollment is the single most common cause on fresh Ubuntu desktops and workstations with default BIOS settings.
Check for a conflicting nouveau driver
lsmod | grep nouveau
Any output means nouveau is loaded and likely holding the GPU device, which must be blacklisted before the proprietary driver can attach.
Confirm the device nodes exist after a successful driver load
ls -la /dev/nvidia*
Missing device files after a fresh install and reboot indicate the module still failed to load; re-check dmesg rather than reinstalling the same package again.
Stopping it from happening again
- Standardize on a golden image with the driver, Secure Boot MOK enrollment, and nouveau blacklist already baked in for new GPU nodes.
- Pin kernel and driver versions together and test kernel upgrades on a canary node before fleet-wide rollout.
- Document your organization's Secure Boot policy explicitly so new hires do not waste hours rediscovering the MOK enrollment step.
- For WSL2 developer machines, keep driver installation instructions restricted to the Windows host in onboarding docs.
When this becomes an architecture problem
If this recurs across every new node in a rollout, or your organization mandates Secure Boot for compliance reasons that make unsigned module workarounds unacceptable, treat it as an imaging and provisioning problem to solve once at the fleet level rather than a per-machine troubleshooting task.
Frequently asked questions
Why does nvidia-smi work in Windows but not inside WSL2?
WSL2 GPU support relies on the Windows host driver being passed through automatically; you should never install a separate Linux NVIDIA driver inside the WSL distro. If nvidia-smi fails inside WSL, first confirm the Windows host driver is current and that wsl --update has been run, rather than trying to install a driver in the Linux guest.
Do I have to disable Secure Boot entirely to fix this?
No. Enrolling the MOK (Machine Owner Key) that the NVIDIA driver installer generates lets Secure Boot keep working while allowing the signed module to load. Disabling Secure Boot entirely is simpler but weakens the machine's security posture, which matters more in regulated environments.
How do I know if nouveau is the actual problem?
Run lsmod | grep nouveau. Any output confirms it is loaded. Combined with dmesg showing the NVIDIA module failing to bind to the device, this is a strong signal that blacklisting nouveau and rebuilding the initramfs will resolve the issue.
I reinstalled the driver and nvidia-smi still fails. What now?
Check dmesg immediately after the reboot rather than reinstalling again. Repeated reinstalls without checking the kernel log rarely help; the log will tell you whether Secure Boot, nouveau, or a DKMS build failure is the actual blocker.
Size it properly next time
Free calculators that prevent this class of failure before you provision hardware.
On-Prem AI Deployment Checklist
A 30-point pre-deployment checklist covering use cases, hardware, security, model operations, and rollout for self-hosted enterprise LLMs.
Free ToolAir-Gapped LLM Deployment Checklist
A practical control checklist for deploying and maintaining large language models in a fully air-gapped environment, from initial staging through ongoing patching and drift detection.
Free ToolSelf-Hosted LLM Hardware Estimator
Estimate the VRAM footprint, GPU count, and hardware budget required to self-host an open-weight LLM with your concurrency and context needs.
Related problems
CUDA version mismatch between PyTorch and the system driver
PyTorch ships its own bundled CUDA runtime inside the wheel, so it never uses your system's CUDA toolkit (the one nvcc reports). The only number that matters is the driver's maximum supported CUDA version, shown top right in nvidia-smi output. Fix the mismatch by installing a torch wheel built for a CUDA version at or below that number, not by touching nvcc or the toolkit.
NVIDIA driver installation fails on Ubuntu
The single most common reason NVIDIA driver installation fails on Ubuntu is Secure Boot rejecting the unsigned or self-signed kernel module at load time, since most machines now ship with Secure Boot enabled out of the box. The fix is enrolling the MOK key the installer generates, or disabling Secure Boot in the BIOS, then clearing any lingering nouveau or mixed-install conflicts before rebooting.
torch.cuda.is_available() returns False even though a GPU is present
torch.cuda.is_available() returning False almost always means either the installed torch wheel is a CPU-only build, or the process cannot see the GPU due to a driver, container, or environment variable problem. Checking torch.version.cuda for None immediately tells you whether you have a CPU-only wheel, which is the single most common cause and the fastest thing to rule out.
GPU not visible inside a Docker container
Docker containers cannot see a host GPU unless the NVIDIA Container Toolkit is installed and the nvidia runtime is registered with the daemon, since containers are isolated from host devices by default. The fix is almost always to install nvidia-container-toolkit, run nvidia-ctk runtime configure, restart Docker, and launch with --gpus all. If nvidia-smi already fails on the host itself, the problem is the driver, not Docker.
GuideOn-Prem LLM Inference Hardware in 2026: A Roundup
On-prem LLM inference hardware for 2026: H100 vs H200 vs B200 pricing, when A100 fleets still work, and how to size GPUs against real serving needs.
GuideOn-Prem GPU Cluster Design: Node Sizing, Networking, and Storage
Design an on-prem GPU cluster: node sizing for H100/H200/B200, InfiniBand vs RoCE networking, storage throughput, and rack power for enterprise AI workloads.
Still stuck, or tired of fighting your own infrastructure?
Netray deploys and operates on-prem AI for regulated manufacturers and defense suppliers. We have debugged this stack in production, on air-gapped networks, at scale.