GPU training speed is slower / same as my CPU

Jayden: <p>I know this probably was posted a lot of times, I have also tried out some of the solutions as well but I am rather new to these and I am not too sure where it might have gone wrong.<br>
To put short my problem, my CPU takes about 120 min for torch training on 20k x 20 dataset with 4 layers while GPU also takes 130min on the same training.</p>
<p>I have followed the steps as listed in<br>
<a href="https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://docs.nvidia.com/cuda/cuda-installation-guide-microsoft-windows/index.html</a></p>
<p>and also used<br>
conda install pytorch torchvision torchaudio pytorch-cuda=12.1 -c pytorch -c nvidia</p>
<p>for downloading the pytorch. At first I downloaded 12.3 version from nvidia instead, and now I have deleted (both with system add/remove and conda uninstall cuda) and re-downloaded to 12.1 version.But I am not too sure why I get the below values when I print them (why 12.3 instead of 12.1?) I can verify my torch.cuda.is_available() returns true.</p>
<p>print(‘__Python VERSION:’, sys.version)<br>
print(‘__pyTorch VERSION:’, torch.<strong>version</strong>)<br>
print(‘__CUDA VERSION’, )</p>
<p>__Python VERSION: 3.11.8 | packaged by Anaconda, Inc. | (main, Feb 26 2024, 21:34:05) [MSC v.1916 64 bit (AMD64)]<br>
__pyTorch VERSION: 2.2.1<br>
__CUDA VERSION<br>
nvcc: NVIDIA (R) Cuda compiler driver<br>
Copyright (c) 2005-2023 NVIDIA Corporation<br>
Built on Wed_Nov_22_10:30:42_Pacific_Standard_Time_2023<br>
Cuda compilation tools, release 12.3, V12.3.107<br>
Build cuda_12.3.r12.3/compiler.33567101_0</p>
<p>I have also moved the data to cuda with code example below<br>
X_train_tensor_gpu = X_train_tensor.to(‘cuda’)<br>
model = Net()<br>
model.to(‘cuda’)</p>
<p>nvidia-smi returns as below<br>
Wed Mar  6 14:59:19 2024<br>
±----------------------------------------------------------------------------------------+<br>
| NVIDIA-SMI 551.61                 Driver Version: 551.61         CUDA Version: 12.4     |<br>
|-----------------------------------------±-----------------------±---------------------+<br>
| GPU  Name                     TCC/WDDM  | Bus-Id          Disp.A | Volatile Uncorr. ECC |<br>
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |<br>
|                                         |                        |               MIG M. |<br>
|=========================================+========================+======================|<br>
|   0  NVIDIA GeForce RTX 4070 …  WDDM  |   00000000:01:00.0  On |                  N/A |<br>
|  0%   40C    P8             20W /  285W |    1746MiB /  16376MiB |      6%      Default |<br>
|                                         |                        |                  N/A |<br>
±----------------------------------------±-----------------------±---------------------+</p>
<p>Not too sure if this is how you monitor GPU usage but running the process &amp; checking it in task manager - performance shows my GPU utilisation goes up to 94% so I do think it should be using my GPU plus it’s 4070 Ti Super so I dont see why it would be slower than my CPU.</p>
<p>Anyone can help…? not too sure what other info is needed.</p>