CUDA Error: no kernel image is available for execution on the device

username: MaxM
cooked: <p>Hello together,</p>
<p>I am quite new to the topic of neural networks.<br>
I`ve build a very simple CNN with the help of YouTube Tutorials and now I have a problem to run my code on the GPU with CUDA. If I set the device to cpu, my code works perfectly well. But if I try to run in on the GPU I get the below displayed error. The GPU ist Nvidia Tesla K20m. The command torch.cuda.is_available() is true. The code is running on Python 3.9. The operating system is RedHat Enterprise Linux (RHEL) / CentOS 7. My input images are 144 x 144 with one channel. The images object that goes into my model is torch.Size([32, 1, 144, 144]). Do you know what could be the problem?</p>
<p>Python Script:</p>
<p>import torch<br>
import torch.nn as nn  # All neural network modules, nn.Linear, nn.Conv2d, BatchNorm, Loss functions<br>
import torchvision.transforms as transforms  # Transformations we can perform on our dataset<br>
import torchvision<br>
import torch.nn.functional as F<br>
from torch.utils.data import (Dataset, DataLoader)  # Gives easier dataset management and creates mini batches<br>
import matplotlib.pyplot as plt<br>
import pandas as pd<br>
from skimage import io<br>
import numpy as np<br>
import os</p>
<h1>
<a name="class-for-custom-dataset-1" class="anchor" href="#class-for-custom-dataset-1"></a>Class for custom dataset</h1>
<p>class SurfaceDataset(Dataset):<br>
def <strong>init</strong>(self, csv_file, root_dir, transform=None):<br>
self.annotations = pd.read_csv(csv_file)<br>
self.root_dir = root_dir<br>
self.transform = transform</p>
<pre><code>def __len__(self):
    return len(self.annotations)

def __getitem__(self, index):
    img_path = os.path.join(self.root_dir, self.annotations.iloc[index, 0])
    image = io.imread(img_path)
    y_label = torch.tensor(int(self.annotations.iloc[index, 1]))

    if self.transform:
        image = self.transform(image)

    return (image, y_label)
</code></pre>
<h1>
<a name="set-device-2" class="anchor" href="#set-device-2"></a>Set device</h1>
<p>device = torch.device(“cuda” if torch.cuda.is_available() else “cpu”)</p>
<h1>
<a name="print-information-about-the-usage-of-cuda-3" class="anchor" href="#print-information-about-the-usage-of-cuda-3"></a>Print information about the usage of cuda</h1>
<p>if torch.cuda.is_available():<br>
print(“CUDA is available”)<br>
print(f"Number of available GPU is {torch.cuda.device_count()}")<br>
else:<br>
print(“CUDA isn’t available”)</p>
<h1>
<a name="hyperparameters-4" class="anchor" href="#hyperparameters-4"></a>Hyperparameters</h1>
<p>num_classes = 2<br>
learning_rate = 1e-3<br>
batch_size = 32  # Normally the batch-size should be something of 2^x with x = [0, 1, 2, 3, 4, …]<br>
num_epochs = 2</p>
<p>print("-------------- Hyperparameter Settings --------------")<br>
print(f"Number of classes: {num_classes}")<br>
print(f"Learning rate: {learning_rate}")<br>
print(f"Batch-size: {batch_size}")<br>
print(f"Number of epochs: {num_epochs}")</p>
<h1>
<a name="load-data-5" class="anchor" href="#load-data-5"></a>Load Data</h1>
<p>dataset = SurfaceDataset(<br>
csv_file=“Klassifizierung.csv”,<br>
root_dir=“Bilder”,<br>
transform=transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.5), (0.5))]))</p>
<p>classes = (‘Defekte Oberfläche’, ‘Defektfreie Oberfläche’)</p>
<p>train_set, test_set = torch.utils.data.random_split(dataset, [800, 100])  # Set the ratio of train and test images<br>
train_loader = DataLoader(dataset=train_set, batch_size=batch_size, shuffle=True)<br>
test_loader = DataLoader(dataset=test_set, batch_size=batch_size, shuffle=False)</p>
<p>def imshow(img):<br>
img = img / 2 + 0.5<br>
npimg = img.numpy()<br>
plt.imshow(np.transpose(npimg, (1, 2, 0)))<br>
plt.show()  # Display all open figures.</p>
<h1>
<a name="get-some-random-training-images-6" class="anchor" href="#get-some-random-training-images-6"></a>get some random training images</h1>
<p>dataiter = iter(train_loader)<br>
images, labels = dataiter.next()</p>
<h1>
<a name="show-images-7" class="anchor" href="#show-images-7"></a>show images</h1>
<p>imshow(torchvision.utils.make_grid(images)) # Generates one picture that contains several pictures with the make_grid command | make_grid also converts from 1 channel to 3 channels</p>
<p>class ConvNet(nn.Module):<br>
def <strong>init</strong>(self):<br>
super(ConvNet, self).<strong>init</strong>()<br>
self.conv1 = nn.Conv2d(1, 6, 5)<br>
self.pool = nn.MaxPool2d(2, 2)<br>
self.conv2 = nn.Conv2d(6, 16, 5)<br>
self.fc1 = nn.Linear(16 * 33 * 33, 120)<br>
self.fc2 = nn.Linear(120, 30)<br>
self.fc3 = nn.Linear(30, 2)</p>
<pre><code>def forward(self, x):
    # x -&gt; batch_size, input_channels, width of the image, heigth of the image | batch_size, 1, 144, 144
    x = self.pool(F.relu(self.conv1(x)))  # -&gt; batch_size, 6, 70, 70
    x = self.pool(F.relu(self.conv2(x)))  # -&gt; batch_size, 16, 33, 33
    x = x.view(-1, 16 * 33 * 33)  # -&gt; batch_size, 16 * 33 * 33
    x = F.relu(self.fc1(x))  # -&gt; batch_size, 120
    x = F.relu(self.fc2(x))  # -&gt; batch_size, 84
    x = self.fc3(x)  # -&gt; batch_size, num_classes
    return x
</code></pre>
<p>model = ConvNet().to(device)</p>
<p>criterion = nn.CrossEntropyLoss()<br>
optimizer = torch.optim.SGD(model.parameters(), lr=learning_rate) # import torch.optim for all Optimization algorithms, SGD, Adam, etc.</p>
<p>n_total_steps = len(train_loader)  # Total amount of images in the test_set<br>
for epoch in range(num_epochs):<br>
for i, (images, labels) in enumerate(train_loader):</p>
<pre><code>    images = images.to(device)  # torch.Size([batch_size, 1, 144, 144])
    labels = labels.to(device)  # torch.Size([batch_size])

    # Forward pass
    outputs = model(images)
    loss = criterion(outputs, labels)

    # Backward and optimize
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()

    if (i + 1) % 1 == 0:
        print(f'Epoch [{epoch + 1}/{num_epochs}], Step [{i + 1}/{n_total_steps}], Loss: {loss.item():.4f}')
</code></pre>
<h1>
<a name="safe-the-trained-weights-of-the-neural-network-8" class="anchor" href="#safe-the-trained-weights-of-the-neural-network-8"></a>Safe the trained weights of the neural network</h1>
<p>print(‘Finished Training’)<br>
PATH = ‘./cnn.pth’<br>
torch.save(model.state_dict(), PATH)</p>
<p>with torch.no_grad():<br>
n_correct = 0<br>
n_samples = 0<br>
n_class_correct = [0 for i in range(num_classes)]<br>
n_class_samples = [0 for i in range(num_classes)]<br>
for images, labels in test_loader:<br>
images = images.to(device)<br>
labels = labels.to(device)<br>
outputs = model(images)<br>
_, predicted = torch.max(outputs, 1) # torch.max returns (value ,index)<br>
n_samples += labels.size(0)<br>
n_correct += (predicted == labels).sum().item()</p>
<pre><code>    for i in range(len(labels)):
        label = labels[i]
        pred = predicted[i]
        if (label == pred):
            n_class_correct[label] += 1
        n_class_samples[label] += 1

acc = 100.0 * n_correct / n_samples
print(f'Accuracy of the network: {acc} %')

for i in range(num_classes):
    acc = 100.0 * n_class_correct[i] / n_class_samples[i]
    print(f'Accuracy of {classes[i]}: {acc} %')
</code></pre>
<p>Conda list:</p>
<h1>
<a name="name-version-build-channel-9" class="anchor" href="#name-version-build-channel-9"></a>Name                    Version                   Build  Channel</h1>
<p>_libgcc_mutex             0.1                        main<br>
blas                      1.0                         mkl<br>
blosc                     1.21.0               h8c45485_0<br>
brotli                    1.0.9                he6710b0_2<br>
brunsli                   0.1                  h2531618_0<br>
bzip2                     1.0.8                h516909a_3    conda-forge<br>
ca-certificates           2021.4.13            h06a4308_1<br>
certifi                   2020.12.5        py39h06a4308_0<br>
charls                    2.2.0                h2531618_0<br>
cloudpickle               1.6.0                      py_0<br>
cudatoolkit               10.2.89              hfd86e86_1<br>
cycler                    0.10.0           py39h06a4308_0<br>
cytoolz                   0.11.0           py39h27cfd23_0<br>
dask-core                 2021.4.0           pyhd3eb1b0_0<br>
decorator                 4.4.2              pyhd3eb1b0_0<br>
ffmpeg                    4.3                  hf484d3e_0    pytorch<br>
freetype                  2.10.4               h7ca028e_0    conda-forge<br>
fsspec                    0.9.0              pyhd3eb1b0_0<br>
giflib                    5.1.4                h14c3975_1<br>
gmp                       6.2.1                h58526e2_0    conda-forge<br>
gnutls                    3.6.13               h85f3911_1    conda-forge<br>
imagecodecs               2021.3.31        py39h581e88b_0<br>
imageio                   2.9.0              pyhd3eb1b0_0<br>
intel-openmp              2021.2.0           h06a4308_610<br>
jpeg                      9b                   h024ee3a_2<br>
jxrlib                    1.1                  h7b6447c_2<br>
kiwisolver                1.3.1            py39h2531618_0<br>
lame                      3.100             h14c3975_1001    conda-forge<br>
lcms2                     2.12                 h3be6417_0<br>
ld_impl_linux-64          2.33.1               h53a641e_7<br>
lerc                      2.2.1                h2531618_0<br>
libaec                    1.0.4                he6710b0_1<br>
libdeflate                1.7                  h27cfd23_5<br>
libffi                    3.3                  he6710b0_2<br>
libgcc-ng                 9.1.0                hdf63c60_0<br>
libgfortran-ng            7.3.0                hdf63c60_0<br>
libiconv                  1.16                 h516909a_0    conda-forge<br>
libpng                    1.6.37               h21135ba_2    conda-forge<br>
libstdcxx-ng              9.1.0                hdf63c60_0<br>
libtiff                   4.1.0                h2733197_1<br>
libuv                     1.40.0               h7b6447c_0<br>
libwebp                   1.0.1                h8e7db2f_0<br>
libzopfli                 1.0.3                he6710b0_0<br>
locket                    0.2.1            py39h06a4308_1<br>
lz4-c                     1.9.3                h2531618_0<br>
matplotlib-base           3.3.4            py39h62a2d02_0<br>
mkl                       2021.2.0           h06a4308_296<br>
mkl-service               2.3.0            py39h27cfd23_1<br>
mkl_fft                   1.3.0            py39h42c9631_2<br>
mkl_random                1.2.1            py39ha9443f7_2<br>
ncurses                   6.2                  he6710b0_1<br>
nettle                    3.6                  he412f7d_0    conda-forge<br>
networkx                  2.5.1              pyhd3eb1b0_0<br>
ninja                     1.10.2               hff7bd54_1<br>
numpy                     1.20.1           py39h93e21f0_0<br>
numpy-base                1.20.1           py39h7d8b39e_0<br>
olefile                   0.46               pyh9f0ad1d_1    conda-forge<br>
openh264                  2.1.1                h8b12597_0    conda-forge<br>
openjpeg                  2.3.0                h05c96fa_1<br>
openssl                   1.1.1k               h27cfd23_0<br>
pandas                    1.2.4            py39h2531618_0<br>
partd                     1.2.0              pyhd3eb1b0_0<br>
pillow                    8.2.0            py39he98fc37_0<br>
pip                       21.0.1           py39h06a4308_0<br>
pyparsing                 2.4.7              pyhd3eb1b0_0<br>
python                    3.9.4                hdb3f193_0<br>
python-dateutil           2.8.1              pyhd3eb1b0_0<br>
python_abi                3.9                      1_cp39    conda-forge<br>
pytorch                   1.8.1           py3.9_cuda10.2_cudnn7.6.5_0    pytorch<br>
pytz                      2021.1             pyhd3eb1b0_0<br>
pywavelets                1.1.1            py39h6323ea4_4<br>
pyyaml                    5.4.1            py39h27cfd23_1<br>
readline                  8.1                  h27cfd23_0<br>
scikit-image              0.18.1           py39ha9443f7_0<br>
scipy                     1.6.2            py39had2a1c9_1<br>
setuptools                52.0.0           py39h06a4308_0<br>
six                       1.15.0             pyh9f0ad1d_0    conda-forge<br>
snappy                    1.1.8                he6710b0_0<br>
sqlite                    3.35.4               hdfb4753_0<br>
tifffile                  2021.3.31          pyhd3eb1b0_1<br>
tk                        8.6.10               hbc83047_0<br>
toolz                     0.11.1             pyhd3eb1b0_0<br>
torchaudio                0.8.1                      py39    pytorch<br>
torchvision               0.9.1                py39_cu102    pytorch<br>
tornado                   6.1              py39h27cfd23_0<br>
typing_extensions         3.7.4.3                    py_0    conda-forge<br>
tzdata                    2020f                h52ac0ba_0<br>
wheel                     0.36.2             pyhd3eb1b0_0<br>
xz                        5.2.5                h7b6447c_0<br>
yaml                      0.2.5                h7b6447c_0<br>
zfp                       0.5.5                h2531618_6<br>
zlib                      1.2.11               h7b6447c_3<br>
zstd                      1.4.5                h9ceee32_0</p>
<p>Error message:</p>
<p>Traceback (most recent call last):<br>
File “/utmnt/ut/ft2/cql7772/Test2/OberflaechenKlassifizierung.py”, line 105, in <br>
outputs = model(images)<br>
File “/fibus/fs1/16/cql7772/.conda/envs/CNN-Klassifizierung/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 889, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “/utmnt/ut/ft2/cql7772/Test2/OberflaechenKlassifizierung.py”, line 84, in forward<br>
x = self.pool(F.relu(self.conv1(x)))  # → batch_size, 6, 70, 70<br>
File “/fibus/fs1/16/cql7772/.conda/envs/CNN-Klassifizierung/lib/python3.9/site-packages/torch/nn/modules/module.py”, line 889, in _call_impl<br>
result = self.forward(*input, **kwargs)<br>
File “/fibus/fs1/16/cql7772/.conda/envs/CNN-Klassifizierung/lib/python3.9/site-packages/torch/nn/modules/conv.py”, line 399, in forward<br>
return self._conv_forward(input, self.weight, self.bias)<br>
File “/fibus/fs1/16/cql7772/.conda/envs/CNN-Klassifizierung/lib/python3.9/site-packages/torch/nn/modules/conv.py”, line 395, in _conv_forward<br>
return F.conv2d(input, weight, bias, self.stride,<br>
RuntimeError: CUDA error: no kernel image is available for execution on the device</p>

username: ptrblck
cooked: <p>The error is raised, if you are trying to execute CUDA code missing the right compute capability for the used device.<br>
The Tesla K20 should have a compute capability of 3.5, which is not shipped in the binaries anymore.<br>
You could build PyTorch from source as described <a href="https://github.com/pytorch/pytorch#from-source">here</a>.</p>

username: MaxM
cooked: <p>Thanks for your answer. Unfortunatelly thats a pity. But honestly I am<br>
quite confused, because I looked up which Cuda toolkit the Tesla K20m<br>
supports an it showed me that it supports up to 10.2 . As you can see<br>
in my list of packages I have installed cudatoolkit 10.2.89 hfd86e86_1<br>
. So where is actually the problem? Is it Python 3.9? Sorry for the<br>
amount of questions. I try to understand what the root cause of the<br>
problem is.</p>
<p>Zitat von ptrblck via PyTorch Forums <a href="mailto:noreply@discuss.pytorch.org">noreply@discuss.pytorch.org</a>:</p>

username: ptrblck
cooked: <p>The issue is not caused by CUDA or Python, but by the size of the pip wheels and conda binaries.<br>
Since the pip wheels have an especially hard memory limit, older compute capabilities are removed after some time, which was the case for 3.5. The current CUDA10.2 binaries support compute capabilities 3.7-7.5 and the CUDA11.1 binaries 3.7-8.6.</p>
<p>EDIT: you might also want to check <a href="https://github.com/pytorch/pytorch/issues/30532">this issue</a> where some users were building binaries for sm_35.</p>

username: MaxM
cooked: <p>Many thanks for your answer. Now its a lot clearer for me. I have also the possibility to run my code on a Tesla K80 GPU. Perhaps the error will disappear since the Tesla K80 has a compute capability of 3.7, what should be fine to run my code.</p>

username: MaxM
cooked: <p>My python script worked perfectly well on the K80 card. Many thanks again!</p>