Fix ComfyUI "Torch Not Compiled With CUDA Enabled" Error

The Error

Related: Fix Stable Diffusion Out Of Memory on a 6GB VRAM GPU →

You downloaded ComfyUI, set up your workflow, clicked "Queue Prompt," and everything ground to a halt. Instead of a generated image, your terminal spat out this exact stack trace:

AssertionError: Torch not compiled with CUDA enabled

Sometimes, it manifests slightly differently depending on your setup, throwing RuntimeError: Found no NVIDIA driver on your system or simply falling back to your CPU and taking ten minutes to generate a single 512x512 image.

I run local AI models every day. I hit this exact error the very first time I cloned the ComfyUI repository. I also hit it again six months later when I accidentally updated my packages using a generic command.

This error happens for one specific reason: the Python environment running ComfyUI is currently using the CPU-only version of PyTorch. It does not matter if you have an RTX 4090 installed. It does not matter if you updated your NVIDIA drivers this morning. If Python does not have the CUDA-specific PyTorch binaries ("wheels"), it cannot talk to your GPU.

Here is exactly how to strip out the wrong packages and install the correct ones, whether you are using a standard Python virtual environment or the standalone Windows portable version.

Step 1: Find Your Maximum Supported CUDA Version

Related: A One-File Docker Compose Stack for Ollama and Open WebUI →

Before we install anything, we need to know what your system can actually handle. NVIDIA drivers are backward compatible, but PyTorch needs a specific target.

Open your terminal or command prompt and run:

nvidia-smi

Look at the top right corner of the output table. You will see something like CUDA Version: 12.2.

This number is your Driver API version. It tells you the maximum version of CUDA your current NVIDIA driver supports. It does not mean you have the CUDA toolkit installed, and that is fine—PyTorch actually bundles its own CUDA runtime libraries now. You just need to make sure you do not try to install a PyTorch version that requires a newer driver than the one you have.

If your nvidia-smi fails or says the command is not recognized, stop here. You need to install the official NVIDIA drivers for your operating system first.

Step 2: Nuke the CPU Version of PyTorch

Related: Fixing Painfully Slow Whisper Transcription →

To fix this, we have to completely remove the bad PyTorch installation. If we try to install the new one over the old one, pip will often look at the version numbers, declare that "requirements are already satisfied," and do nothing.

First, activate your ComfyUI environment. If you installed ComfyUI manually using a Python virtual environment (venv), activate it.

On Windows:

cd path\to\ComfyUI
.\venv\Scripts\activate

On Linux:

cd /path/to/ComfyUI
source venv/bin/activate

(Note: If you are using the ComfyUI Windows Portable version, skip to the "ComfyUI Portable" section below. Standard pip commands will not work for you.)

Once your environment is active (you should see (venv) in your terminal prompt), run this command:

pip uninstall torch torchvision torchaudio

It will ask you to confirm Proceed (Y/n)?. Type y and hit enter.

I highly recommend running that exact command a second time. Sometimes, users accidentally install PyTorch twice—once via pip and once via an automated script—leaving a shadow installation behind. Run it until pip tells you WARNING: Skipping torch as it is not installed.

Next, clear your pip cache. If you don't do this, pip might just grab the broken CPU wheel it already downloaded yesterday instead of reaching out to the internet for the GPU version.

pip cache purge

Step 3: Install the Correct CUDA PyTorch

Related: Fix Open WebUI Showing No Models in the Dropdown →

This is the command that actually fixes the problem.

By default, typing pip install torch grabs packages from the standard Python Package Index (PyPI). For complex reasons regarding file size limits, the default Windows package on PyPI is CPU-only. We have to tell pip to pull directly from PyTorch's dedicated server.

We do this using the --index-url flag. You need to pick the right CUDA version (cu118, cu121, or cu124). For almost all modern setups running ComfyUI today, cu121 (CUDA 12.1) is the safest and most stable bet.

Run this command inside your activated virtual environment:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

This is a massive download. The PyTorch wheel alone is roughly 2.5 GB because it contains all the pre-compiled CUDA binaries. If you are on a slow connection, go grab a coffee.

Which Version Should You Pick?

If you are wondering whether to change that URL based on your hardware, here is a quick breakdown I use for my own machines.

If you are trying to squeeze every drop of performance out of an older rig, you might also be figuring out how much RAM you actually need for local AI. Upgrading your system RAM won't fix VRAM limitations, but sticking to cu121 ensures you don't run into bizarre memory leak issues associated with experimental builds.

Step 4: Verify the Fix Before Launching

Related: Secure Ollama with Nginx, HTTPS, and a Password →

Before you fire up ComfyUI and potentially wait for a crash, we can test if the installation worked using a one-line Python command.

Make sure your virtual environment is still active, and run this:

python -c "import torch; print(torch.cuda.is_available())"

If it prints True, you are done. Your Python environment now has direct access to your NVIDIA GPU. You can start ComfyUI with python main.py and run your workflows.

If it prints False, something is still fundamentally wrong. This often happens if you are using Windows Subsystem for Linux (WSL2) and the virtualized environment cannot see the host GPU. If you are in that specific boat, the troubleshooting steps are nearly identical to fixing Ollama not using my NVIDIA GPU in WSL2. You have to ensure the WSL kernel has the NVIDIA pass-through drivers installed.

Fixing This in the ComfyUI Windows Portable Version

Related: How to Stop Ollama From Unloading Models (keep_alive) →

If you downloaded the official ComfyUI_windows_portable .7z file, the steps above will fail.

The portable version is designed to be entirely self-contained. It does not use your system Python, and it does not use a standard venv. Instead, it uses an embedded Python runtime located inside the python_embeded folder.

If you open a normal command prompt and type pip install, you are modifying your global Windows Python installation, which ComfyUI completely ignores. This is a common trap, much like when I was building a local RAG chatbot and accidentally installed my vector database libraries to my base system instead of my isolated project folder.

To fix the CUDA error in the portable version, you must force the terminal to use the embedded executable.

Open a command prompt inside your main ComfyUI_windows_portable folder. Run exactly this:

.\python_embeded\python.exe -m pip uninstall torch torchvision torchaudio

Confirm the uninstallation. Then, install the CUDA 12.1 version specifically into the embedded folder:

.\python_embeded\python.exe -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

Once the download finishes, you can double-click your run_nvidia_gpu.bat file. The server will start, and the error will be gone.

What Did NOT Work (My Dead Ends)

When I first encountered this, I wasted a whole evening trying fixes that fundamentally misunderstood the problem. Here are the dead ends I hit, so you can avoid them:

Installing the 3GB CUDA Toolkit from NVIDIA's Website I thought "Torch not compiled with CUDA" meant my system was missing CUDA entirely. I downloaded the massive CUDA Toolkit installer from NVIDIA, rebooted, and changed my system PATH variables. It did nothing. PyTorch comes with its own CUDA runtime bundled in the .whl file. The system toolkit is only necessary if you are compiling custom C++ extensions yourself, which standard ComfyUI users do not do.

Setting Environment Variables I tried running set CUDA_VISIBLE_DEVICES=0 before launching the script. This variable forces an application to look at a specific GPU. But if the application is using CPU-only PyTorch, it doesn't matter where you point it. It literally lacks the compiled code to talk to the GPU.

Assuming it was a Port Conflict I briefly thought the server was failing to bind, confusing it with network errors. Hardware binding errors are completely different beasts, unlike networking errors like Ollama connection refused on 127.0.0.1:11434. If the stack trace explicitly says "Torch not compiled," it is purely a Python dependency issue.

Using Conda Instead of Pip I read a forum post saying Conda handles CUDA dependencies better. I installed Miniconda and tried to mix it with the existing ComfyUI setup. Do not do this. Mixing pip environments with conda environments usually results in path conflicts where Python finds the Conda executable but the Pip libraries. Stick to pip and virtual environments for ComfyUI.

Final Checks

Once you have everything running, you can double-check that ComfyUI is actually using your GPU by looking at the startup logs in your terminal. You should see lines similar to this:

Total VRAM 24564 MB, total RAM 65352 MB
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3090 : cudaMallocAsync

If it says Device: cpu, then torch.cuda.is_available() is still returning false, and you need to repeat the uninstallation step, ensuring you clear the pip cache before downloading the wheels again.

FAQ

Question: Why does nvidia-smi say CUDA 12.2 but PyTorch fails?

nvidia-smi shows your maximum supported driver API, not your installed runtime API. You still need the correct cu118 or cu121 wheel installed in your Python environment.

Question: Can I use AMD GPUs with ComfyUI?

Yes, but you cannot use CUDA wheels. You must install the ROCm version of PyTorch on Linux, or use DirectML on Windows. The standard PyPI install will not work for AMD.

Question: Do I need xformers installed for ComfyUI?

No, PyTorch 2.0+ includes standard cross-attention optimization (sDP) built-in. Xformers can save a tiny bit of VRAM on older cards, but it often causes dependency headaches.

Question: How do I fix this in the ComfyUI Windows Portable version?

Open a terminal in your ComfyUI folder and run .\python_embeded\python.exe -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121.