Setting up TensorFlow with GPU Acceleration on Windows | by Chathura Ranasinghe | Apr, 2024

Step by step guide with TensorFlow, Nvidia, Cuda and Conda

As you know TensorFlow is a powerful open-source machine learning library developed by Google that allows developers to build and train various machine learning models.

Modern GPUs are highly parallel processors optimized for handling large-scale computations. By the parallel processing power of GPUs, TensorFlow can accelerate training and inference tasks, leading to significant reductions in computation time around 50%.

In this guide, we’ll walk through the process of setting up TensorFlow with GPU acceleration on a Windows system, covering the installation of Nvidia drivers, CUDA, and Conda environment setup.

What You Need:

✔ Windows operating system (Windows 10/11 (64-bit) recommended)

✔ Nvidia GPU (GTX 10-series or higher recommended)

✔ Anaconda installed (or Miniconda)

1. Identifying the latest tested build configurations
2. Step 1: Install latest Nvidia GPU Drivers
3. Step 2: Install Anaconda
4. Step 3: Create a Conda Environment
5. Step 4: nstall CUDA Toolkit and cuDNN Library.
6. Step 5: Install TensorFlow 2.10
7. Step 6: Implement GPU Acceleration on Your Model

If you visit TensorFlow official web page, you can see the following message under GPU in Tested build configurations section.

Here in first message, you can see it tells, GPU support on native-Windows is only available for 2.10 or earlier versions. Which means latest GPU supporting TensorFlow version is 2.10 or earlier versions.

Under that you can see another table with latest TensorFlow GPU versions supporting python, cuDNN and CUDA versions.

In order to latest python, cuDNN and CUDA versions that supports TensorFlow 2.10 are python 3.10, cuDNN 8.1 and CUDA 11.2. (make sure to check TensorFlow official web page for UpToDate information)

There are two ways to install the latest GPU drivers for your specific GPU model to your machine.

1. Via NVIDIA® GeForce® Experience™ App. (Recommended)

This is the easiest and most recommended way to get latest GPU drivers for user GPU model.

First you have to download and install NVIDIA® GeForce® Experience™ to your machine.

­­2. Via Nvidia official “NVIDIA® GPU drivers” page.

Visit the NVIDIA® GPU drivers website and download the latest GPU drivers for your specific GPU model.

Ok, now you have latest GPU drivers on your machine.

Download and install latest Anaconda or from the official website.

Verify your installation by CMD (Command prompt)

conda --version

Why Conda?

Anaconda, with the package manager Conda makes it easy to create and manage virtual environments, which are isolated and reproducible environments that contain specific versions of Python and other packages. This can help avoid dependency conflicts and ensure compatibility across different platforms and devices.

Open your CMD or Search “Anaconda Prompt” and open it.

1. Create python 3.10 environment.

conda create -n test-env python=3.10

Remember to upgrade your pip version regularly to ensure you have access to the latest features and bug fixes.

python -m pip install --upgrade pip

2. Activate conda environment.

conda activate test-env

Now we have install CUDA Toolkit and cuDNN Library into your Conda environment.

Simply run bellow command to install both at once.

conda install -c conda-forge cudatoolkit=11.2 cudnn=8.1.0

What are CUDA Toolkit and cuDNN Library ?

CUDA Toolkit, developed by Nvidia, enhances your computer’s GPU performance by providing tools and libraries for parallel computing tasks, making it ideal for machine learning applications. cuDNN (CUDA Deep Neural Network Library), also from Nvidia, further optimizes deep learning tasks by offering accelerated implementations of common operations like convolution and pooling, resulting in faster and more efficient neural network training and inference.

Ok, now we have to install TensorFlow 2.10 to our python environment.

python -m pip install "tensorflow==2.10"

To make sure everything’s working smoothly, run this code.

python -c "import tensorflow as tf; print(tf.config.list_physical_devices('GPU'))"

If everything works fine, you will get this output.

[PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

Greate, you have successfully set up TensorFlow with GPU acceleration on your Windows system. Now, you might be wondering, what’s next? How can you leverage this setup to enhance your model?

Here, we’ll explore how to implement GPU acceleration in your model.

First, import TensorFlow into your Python or Jupyter Notebook code:

import tensorflow as tf

However, there’s a challenge. By default, TensorFlow maps nearly all the GPU memory of all GPUs visible to the process. This can lead to consuming all available GPU memory on your machine, causing resource contention and potential crashes when running multiple processes or models simultaneously.

To mitigate this, we can limit GPU memory growth using the following code:

gpus = tf.config.list_physical_devices('GPU')
if gpus:
try:
# Currently, memory growth needs to be the same across GPUs
for gpu in gpus:
tf.config.experimental.set_memory_growth(gpu, True)
logical_gpus = tf.config.list_logical_devices('GPU')
print(len(gpus), "Physical GPUs,", len(logical_gpus), "Logical GPUs")
except RuntimeError as e:
# Memory growth must be set before GPUs have been initialized
print(e)

This code attempts to allocate only as much GPU memory as needed for runtime allocations. It starts by allocating a minimal amount of memory and dynamically extends the GPU memory region as required during runtime. Memory is not released to avoid memory fragmentation.

Note: If you are using more than one GPU, you need to use different code. For more information, refer to the Using a Single GPU on a Multi-GPU System section in the TensorFlow documentation.

Congratulations! You’ve leveled up your machine learning skills by setting up TensorFlow with GPU acceleration on your Windows system. It’s time to build some awesome models!