Why gcc fails when building CUDA extensions, and how to fix it

Each CUDA toolkit release documents a specific supported range of host compiler versions, since nvcc needs to invoke gcc or g++ to process the non-CUDA C++ portions of source files. Recently released Linux distributions ship increasingly new gcc versions by default, and once that default gcc exceeds the upper bound a given CUDA toolkit was validated against, nvcc refuses to proceed rather than risk silently miscompiling code.

Fix: Install an older gcc/g++ version known to be supported by your CUDA toolkit release, and set CC and CXX environment variables to point at it explicitly for the duration of the build.

sudo apt install gcc-11 g++-11
export CC=gcc-11 CXX=g++-11
pip install flash-attn --no-build-isolation

Machines that have accumulated several gcc versions over time through different package installs may have update-alternatives configured to point at a newer version by default, or leftover CC/CXX environment variables from an earlier session pointing at the wrong compiler, causing the same unsupported version error even though a working older gcc is already present on disk.

Fix: List installed gcc versions and use update-alternatives or explicit CC/CXX environment variables to select the one your CUDA toolkit supports, rather than assuming only one gcc exists on the machine.

update-alternatives --list gcc
sudo update-alternatives --config gcc

A Dockerfile that installs a specific CUDA toolkit version but relies on the base image's default system packages for gcc can end up with the same mismatch seen on bare metal, since container base images track their own distribution's gcc release cadence independently of which CUDA toolkit version you chose to install inside them.

Fix: Explicitly install and select a compatible gcc/g++ version inside the Dockerfile rather than relying on whatever the base image ships by default, and pin this as part of your reproducible build image.

nvcc needs a C++ host compiler specifically, not just a C compiler, to process CUDA extension code that typically includes C++ features. Some minimal installations or base images include gcc for basic C compilation but omit g++ entirely, causing the build to fail looking for a C++ compiler that was never installed rather than a version mismatch.

Fix: Install the g++ package explicitly alongside gcc, since many distributions package them separately even though they are usually installed together.

sudo apt install g++

On very new Linux distributions paired with an intentionally older, pinned CUDA toolkit version (for reasons of driver compatibility elsewhere in the stack), there may be no supported gcc version combination at all, since the toolkit release predates the existence of any compiler version the distribution ships or makes easily installable.

Fix: Upgrade the CUDA toolkit itself to a release that documents support for a gcc version available on your distribution, since downgrading gcc alone will not resolve toolkit releases old enough to have no compatible compiler option.