Setting up CUDA and cuDNN in Ubuntu 22.04

Before I get started this guide is based on my own experience installing the libraries. It is intended for linux, but the idea should be the same for any operating system. With windows just make sure you install the latest drivers. Also, make sure your GPU actually supports CUDA.

I found the Nvidia documentation to be a little bit vague in some parts which led me to install incompatible version of CUDA and cuDNN, as well as drivers. But in this guide I intend to make everything clear from he start. This guide should also be useful for any other Ubuntu based distribution, such as Linux Mint. If you find that some steps are missing or. that something is not working, feel free to contact me via the website contact form. Also, I encourage you to carefully read all the provided Nvidia documentation to not miss any steps.

Compatible versions

First of all, make sure that the versions of CUDA, cuDNN and you drivers are compatible. I’ve screwed this up many times, and had to reinstall the whole thing, since the Nvidia guide I was looking at did not make it clear that I should be careful about this (or I might have missed it). Whatever the case, here is a warning so that you do not make the same mistakes as me. Bellow is one of the configurations (currently based in the latest drivers etc.) that ensures compatibility between all the modules.

Driver: 520.61.05
CUDA: 11.8
cuDNN: v8.7.0 (November 28th, 2022), for CUDA 11.8

If you are installing any other versions I would recommend you to go backwards. First decide on the version of cuDNN, then look-up the required CUDA version. If you go to the Nvidia cuDNN site (you will have to sign up for the Nvidia Developer Program to access this software), you will probably be presented with something like this:

Remember the CUDA version listed at the end of the package. As you can see the first cuDNN package will work with CUDA 11.x. Then go to the Nvidia CUDA Toolkit website (you will have to access the archive for older version, like I did). If you need the latest CUDA version (which is 12.0 in my case and which is not compatible with the cuDNN package) head here. After that you should be presented with something like this (of course make sure to select the correct platform, architecture, distribution and version of you system):

If you inspect the third and fourth lines, you should see a bigger number, in my case 520.61.05. This is the driver version you should be aiming for (but just remember the number 520, that is all you will need later on). A newer driver should probably also work (according to the documentation), but if you encounter any problems with that I suggests you install the driver version specified in the CUDA package.

Driver installation

To get started, first you will need to install the proprietary Nvidia drivers (though as far as I know there is also an open-source version of the drivers available now). There are many ways to do that, but I chose the approach of installing it directly through the CLI. Replace <driver_ver> with the driver version mentioned in the previous paragraph (you just need the first number of the bunch – in my case 520). First install the drivers with the following commands:

sudo apt install nvidia-driver-<driver_ver> nvidia-dkms-<driver_ver> nvidia-utils-<driver_ver>

If the version of this driver is not found, try installing the latest driver by typing

sudo apt install nvidia-driver-

and clicking tab to get the list of all the available driver versions. Do the same for the dkms and utils package – just make sure they are all the same version. Select the newest one (highest number).

After that reboot the computer.

sudo reboot

To verify if the driver is working use the following command:

nvidia-smi

You should get an output with NVIDIA-SMI version, driver version and CUDA version. If you get any error, something isn’t alright. Do not worry if the displayed CUDA version is wrong.

CUDA installation

To install CUDA head to this website and select you version of CUDA. Select the correct information about your system until you get the list of commands that need to be executed. I used the deb (local) installation method. After that just follow the Base Installer instructions provided by the website and execute all the following commands (if installing another version/OS, use the commands provided on the site).

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
wget https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda-repo-ubuntu2204-11-8-local_11.8.0-520.61.05-1_amd64.deb
sudo dpkg -i cuda-repo-ubuntu2204-11-8-local_11.8.0-520.61.05-1_amd64.deb
sudo cp /var/cuda-repo-ubuntu2204-11-8-local/cuda-*-keyring.gpg /usr/share/keyrings/
sudo apt-get update
sudo apt-get -y install cuda

If the last command install cuda fails, try sudo apt update again. After that reboot the system.

After cuda has been installed you can verify that the right version is installed by using the previously mentioned nvidia-smi command, which should now display different information than at the start. If it is not updated, try restarting the system.

Regarding the fifth command, if it does not work, the dpkg command should tell you what command to run instead after running it.

cuDNN installation

To install cuDNN first head to the cuDNN website and download the correct version of cuDNN (8.7.0 in my example). I recommend you download the (Deb) file.

After the package has been downloaded, head to the download folder using the cd command or locate where the package has been downloaded. After that run the following commands to install cuDNN on you system.

sudo dpkg -i <download_cudnn_deb_file>

After that follow the command output by the dpkg command to import the CUDA GPG key or enter the following command:

sudo cp /var/cudnn-local-repo-*/cudnn-local-*-keyring.gpg /usr/share/keyrings/

After that run the following commands to install the rest of the needed libraries, where cuDNN version will probably look something like 8.7.0.84 and CUDA version something like 11.8. If you do not know the version number just look at the downloaded cuDNN package name, since it probably contains the version number in its name:

sudo apt-get update
sudo apt-get install libcudnn8=libcudnn8=8.x.x.x-1+cudaX.Y
sudo apt-get install libcudnn8-dev=8.x.x.x-1+cudaX.Y
sudo apt-get install libcudnn8-samples=8.x.x.x-1+cudaX.Y

To verify the cuDNN install, you can run the following commands:

cp -r /usr/src/cudnn_samples_v8/ $HOME
cd  $HOME/cudnn_samples_v8/mnistCUDNN
make clean && make
./mnistCUDNN

If any of the commands fails or if you do not get the Test passed! output, some of the components likely has not been installed correctly.

Conclusion

This is a quick straight-forward installation guide that should work for Ubuntu 22. If you need more information or. have encountered some problems at some point, you can try following Nvidia’s guide to installing cuDNN. Just remember to install compatible versions of each software.