Training with GPU

username: cocorocho
cooked: <p>Hello, i have RTX 2060 with Ryzen 5 2600x. I am trying to train my CNN with GPU however even though i use GPU, the training is very slow, almost same as CPU.</p>
<p>After some research i saw a <a href="https://stackoverflow.com/questions/44227767/nvidia-smi-does-not-display-memory-usage/44228331" rel="nofollow noopener">post on stackoverflow</a> mentioning that i can not use GPU if there is display attached to the it.</p>
<p>Is that true?</p>
<p>My nvidia-smi looks like this.</p>
<pre><code class="lang-auto">+-----------------------------------------------------------------------------+
| NVIDIA-SMI 445.75       Driver Version: 445.75       CUDA Version: 11.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce RTX 2060   WDDM  | 00000000:1C:00.0  On |                  N/A |
| 22%   53C    P2    29W / 160W |   3866MiB /  6144MiB |      6%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     21072      C   ..._torch\Scripts\python.exe    N/A      |
+-----------------------------------------------------------------------------+
</code></pre>
<p>My network, images, labels are all converted to cuda. But it is very slow.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="cocorocho" data-post="1" data-topic="95484">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/cocorocho/48/28554_2.png" class="avatar"> cocorocho:</div>
<blockquote>
<p>i can not use GPU if there is display attached to the it.</p>
</blockquote>
</aside>
<p>That is wrong. Your display output might use some memory on your device, but you should still be able to use the GPU for processing in PyTorch (at least that’s how I’m doing it in my Linux box and Laptop).</p>
<p>Could you post your training code so that we can have a look at it?</p>

username: cocorocho
cooked: <p>Hello,</p>
<p>Heres my code:</p>
<h3><a name="model-1" class="anchor" href="#model-1"></a>Model</h3>
<p>class Network(nn.Module):</p>
<pre><code>def __init__(self):

    super().__init__()

    #Img size 32x32        

    self.conv1 = nn.Conv2d(in_channels=1, out_channels=12, kernel_size=5)        

    self.conv2 = nn.Conv2d(in_channels=12, out_channels=24, kernel_size=5)        

    

    self.fc1 = nn.Linear(in_features=24*5*5, out_features=60)        

    self.out = nn.Linear(in_features=60, out_features=2)

    

def forward(self, tensor):

    tensor = F.relu(self.conv1(tensor))

    tensor = F.max_pool2d(tensor, kernel_size=2, stride=2)               

    tensor = F.relu(self.conv2(tensor))

    tensor = F.max_pool2d(tensor, kernel_size=2, stride=2)              

    tensor = tensor.flatten(start_dim=1)        

    tensor = F.relu(self.fc1(tensor))        

    tensor = self.out(tensor)

    return tensor
</code></pre>
<h3><a name="loader-2" class="anchor" href="#loader-2"></a>Loader</h3>
<blockquote>
<p>def findKey(value):</p>
<pre><code>key = "".join([k for (k, v) in classes.items() if v == value])

return key
</code></pre>
</blockquote>
<blockquote>
<p>class CatDog(Dataset):</p>
<pre><code>def __init__(self, csv_file, root_dir, transform=None):

    self.annotations = pd.read_csv(csv_file)

    self.root_dir = root_dir

    self.transform = transform

def __len__(self):

    return len(self.annotations)

def __getitem__(self, index):

    img_folder = findKey(self.annotations.iloc[index, 1])

    img_name = self.annotations.iloc[index, 0]

    img_path = os.path.join(self.root_dir, img_folder, img_name)

    image = io.imread(img_path)        

    label = int(self.annotations.iloc[index, 1])        

    if self.transform:

        image = self.transform(image)

    return image, label
</code></pre>
</blockquote>
<h3><a name="training-3" class="anchor" href="#training-3"></a>Training</h3>
<blockquote>
<p>device = torch.device(“cuda:0” if torch.cuda.is_available() else “cpu”)</p>
<p>def getNumCorrect(predictions, labels):</p>
<pre><code>return predictions.argmax(dim=1).eq(labels).sum().item()
</code></pre>
<p>TRANSFORM = transforms.Compose([transforms.ToTensor()])</p>
<p>catdogdataset = CatDog(csv_file, root_folder, TRANSFORM)</p>
<p>train_loader = DataLoader(catdogdataset, batch_size=50, shuffle=True)</p>
<p>comment = f"batch_size=50 lr=0.01"</p>
<p>tb = SummaryWriter(comment=comment)</p>
<p>network = Network()</p>
<p>network = network.to(device)</p>
<p>optimizer = optim.Adam(network.parameters(), lr=0.01)</p>
</blockquote>
<blockquote>
<p>for epoch in range(20000):</p>
<pre><code>total_loss = 0

total_correct = 0

for batch in tqdm(train_loader):

    images, labels = batch

    images, labels = images.to(device), labels.to(device)

    grid = torchvision.utils.make_grid(images)

    tb.add_image("images", grid)

    tb.add_graph(network, images)

    predictions = network(images)

    loss = F.cross_entropy(predictions, labels)

    optimizer.zero_grad()

    loss.backward()

    optimizer.step()

    total_loss += loss.item()

    total_correct += getNumCorrect(predictions, labels)



accuracy = total_correct / len(catdogdataset)

print(accuracy)
</code></pre>
</blockquote>
<p>Images are 32x32 grayscale</p>
<p>Each epoch takes around 2 minutes 30 seconds with GPU and 2 minutes 45 seconds with CPU.</p>
<p>Is this normal? I am using Kaggle Cats and Dogs dataset.</p>
<p>Thanks for your help.</p>

username: cocorocho
cooked: <aside class="quote no-group" data-username="cocorocho" data-post="3" data-topic="95484">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/cocorocho/48/28554_2.png" class="avatar"> cocorocho:</div>
<blockquote>
<p><code>tb.add_graph(network, images)</code></p>
</blockquote>
</aside>
<p>Ok i found the issue, it was this line bottlenecking the training.</p>
<blockquote>
<p>tb.add_graph(network, images)</p>
</blockquote>
<p>My training speed is good after getting rid of that line.</p>