How to Install NVIDIA CUDA Toolkit and NVIDIA CUDA Compilers on Linux Ubuntu

What is covered in this tutorial: In this tutorial, we explain how to properly install NVIDIA CUDA Toolkit and NVIDIA CUDA Compilers on Linux Ubuntu.

Motivation: You need NVIDIA CUDA Toolkit and NVIDIA CUDA compilers to build and run locally machine learning algorithms and large language models. The most important component of the NVIDIA CUDA Toolkit is the NVIDIA CUDA Compiler (NVCC). NVCC is used to compile files necessary to run machine learning algorithms locally. Any source file with an extension .cu must be compiled with NVCC.

Challenge: For people who are not familiar with Linux Ubuntu it might be challenging to properly install NVCC compiler and properly set the system path such that the NVCC is visible from any terminal. In this tutorial, we carefully and thoroughly explain all the steps you need to perform to properly install NVCC.

The YouTube tutorial explaining all the steps is given below.

Installation Instructions

In this tutorial, we explain how to install NVIDIA CUDA 12.6 on Ubuntu 24.04. However, everything explained in this tutorial can be generalized to any CUDA and Linux Ubuntu versions.

Go to the NVIDIA CUDA Toolkit website

https://developer.nvidia.com/cuda-toolkit

and click on Download Now, then on Linux, x86_64, Ubuntu, 24.04, deb(network), as a result, the following set of commands will be generated.

The generated instructions that need to be executed in the terminal are:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86_64/cuda-keyring_1.1-1_all.deb

sudo dpkg -i cuda-keyring_1.1-1_all.deb

sudo apt-get update && sudo apt upgrade

sudo apt-get -y install cuda-toolkit-12-6

Open a terminal and execute these commands. After these commands are executed in the terminal, we need to add the CUDA bin folder and all the binaries to the Linux path. The CUDA binary folder is located at

/usr/local/cuda-12.6/bin

To add this folder to the path, type

cd ~
sudo nano .bashrc

and at the end of the .bashrc file, add the following line

export PATH=/usr/local/cuda-12.6/bin${PATH:+:${PATH}}

save the file and exit.