No GPU utilization although CUDA seems to be activated

imesery: <p>Hi,</p>
<p>I have an Alienware laptop with GeForce GTX 980M , and I’m trying to run my first code in pytorch - using transfer learning with resnet.<br>
The thing is that I get no GPU utilization although all CUDA signs in python seems to be ok:</p>
<p>print(“torch.cuda.is_available()   =”, torch.cuda.is_available())<br>
print(“torch.cuda.device_count()   =”, torch.cuda.device_count())<br>
print(“torch.cuda.device(‘cuda’)   =”, torch.cuda.device(‘cuda’))<br>
print(“torch.cuda.current_device() =”, torch.cuda.current_device())<br>
print(“torch.cuda.get_device_name(0) =”,torch.cuda.get_device_name(0))<br>
cudnn.benchmark = True<br>
<span class="hashtag-raw">#Additional</span> Info when using cuda<br>
if device.type == ‘cuda’:<br>
print(torch.cuda.get_device_name(0))<br>
print(‘Memory Usage:’)<br>
print(‘Allocated:’, round(torch.cuda.memory_allocated(0)/1024<strong>3,1), ‘GB’)<br>
print('Cached:   ', round(torch.cuda.memory_cached(0)/1024</strong>3,1), ‘GB’)</p>
<p>torch.cuda.is_available()   = True<br>
torch.cuda.device_count()   = 1<br>
torch.cuda.device(‘cuda’)   = &lt;torch.cuda.device object at 0x0000021F70F15D68&gt;<br>
torch.cuda.current_device() = 0<br>
torch.cuda.get_device_name(0) = GeForce GTX 980M<br>
GeForce GTX 980M<br>
Memory Usage:<br>
Allocated: 0.0 GB<br>
Cached:    0.0 GB</p>
<p>I tried googling it and also looked here but came up with nothing helpful.<br>
running nvidia-smi also shows that there is no GPU memory usage, and the following message:</p>
<pre><code>    Process ID                  : 19792
        Type                    : C
        Name                    : C:\Users\itaim\AppData\Local\Programs\Python\Python36\python.exe
        Used GPU Memory         : Not available in WDDM driver model
</code></pre>
<p>attaching task manager screenshot and nviidia-smi output:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/f/f15780e4b68d61551372b4e937e6a5fef819c220.png" data-download-href="https://discuss.pytorch.org/uploads/default/f15780e4b68d61551372b4e937e6a5fef819c220" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/f/f15780e4b68d61551372b4e937e6a5fef819c220_2_620x500.png" alt="image" data-base62-sha1="yr0xPPhygYAzPyBX1gB8gRJJwbK" width="620" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/f/f15780e4b68d61551372b4e937e6a5fef819c220_2_620x500.png, https://discuss.pytorch.org/uploads/default/optimized/2X/f/f15780e4b68d61551372b4e937e6a5fef819c220_2_930x750.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/f/f15780e4b68d61551372b4e937e6a5fef819c220_2_1240x1000.png 2x" data-dominant-color="F6F7F8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1792×1443 211 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>nvidia-smi -l</p>
<p>C:\Users\itaim&gt;nvidia-smi -l<br>
±----------------------------------------------------------------------------+<br>
| NVIDIA-SMI 419.35       Driver Version: 419.35       CUDA Version: 10.1     |<br>
|-------------------------------±---------------------±---------------------+<br>
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |<br>
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |<br>
|===============================+======================+======================|<br>
|   0  GeForce GTX 980M   WDDM  | 00000000:01:00.0 Off |                  N/A |<br>
| N/A   66C    P0    69W /  N/A |    917MiB /  8192MiB |     51%      Default |<br>
±------------------------------±---------------------±---------------------+</p>
<p>±----------------------------------------------------------------------------+<br>
| Processes:                                                       GPU Memory |<br>
|  GPU       PID   Type   Process name                             Usage      |<br>
|=============================================================================|<br>
|    0     19792      C   …cal\Programs\Python\Python36\python.exe N/A      |<br>
±----------------------------------------------------------------------------+<br>
nvidia-smi -q</p>
<p>C:\Users\itaim&gt;nvidia-smi -q</p>
<p>==============NVSMI LOG==============</p>
<p>Timestamp                           : Wed Mar 27 01:09:08 2019<br>
Driver Version                      : 419.35<br>
CUDA Version                        : 10.1</p>
<p>Attached GPUs                       : 1<br>
GPU 00000000:01:00.0<br>
Product Name                    : GeForce GTX 980M<br>
Product Brand                   : GeForce<br>
Display Mode                    : Disabled<br>
Display Active                  : Disabled<br>
Persistence Mode                : N/A<br>
Accounting Mode                 : Disabled<br>
Accounting Mode Buffer Size     : 4000<br>
Driver Model<br>
Current                     : WDDM<br>
Pending                     : WDDM<br>
Serial Number                   : N/A<br>
GPU UUID                        : GPU-544e0a63-680d-08ca-c2da-1ef57adb2fff<br>
Minor Number                    : N/A<br>
VBIOS Version                   : 84.04.87.00.08<br>
MultiGPU Board                  : No<br>
Board ID                        : 0x100<br>
GPU Part Number                 : N/A<br>
Inforom Version<br>
Image Version               : N/A<br>
OEM Object                  : N/A<br>
ECC Object                  : N/A<br>
Power Management Object     : N/A<br>
GPU Operation Mode<br>
Current                     : N/A<br>
Pending                     : N/A<br>
GPU Virtualization Mode<br>
Virtualization mode         : None<br>
IBMNPU<br>
Relaxed Ordering Mode       : N/A<br>
PCI<br>
Bus                         : 0x01<br>
Device                      : 0x00<br>
Domain                      : 0x0000<br>
Device Id                   : 0x13D710DE<br>
Bus Id                      : 00000000:01:00.0<br>
Sub System Id               : 0x07081028<br>
GPU Link Info<br>
PCIe Generation<br>
Max                 : 3<br>
Current             : 3<br>
Link Width<br>
Max                 : 16x<br>
Current             : 8x<br>
Bridge Chip<br>
Type                    : N/A<br>
Firmware                : N/A<br>
Replays Since Reset         : 0<br>
Replay Number Rollovers     : 0<br>
Tx Throughput               : 59000 KB/s<br>
Rx Throughput               : 26000 KB/s<br>
Fan Speed                       : N/A<br>
Performance State               : P0<br>
Clocks Throttle Reasons<br>
Idle                        : Not Active<br>
Applications Clocks Setting : Not Active<br>
SW Power Cap                : Not Active<br>
HW Slowdown                 : Not Active<br>
HW Thermal Slowdown     : N/A<br>
HW Power Brake Slowdown : N/A<br>
Sync Boost                  : Not Active<br>
SW Thermal Slowdown         : Not Active<br>
Display Clock Setting       : Not Active<br>
FB Memory Usage<br>
Total                       : 8192 MiB<br>
Used                        : 827 MiB<br>
Free                        : 7365 MiB<br>
BAR1 Memory Usage<br>
Total                       : 256 MiB<br>
Used                        : 229 MiB<br>
Free                        : 27 MiB<br>
Compute Mode                    : Default<br>
Utilization<br>
Gpu                         : 39 %<br>
Memory                      : 25 %<br>
Encoder                     : 0 %<br>
Decoder                     : 0 %<br>
Encoder Stats<br>
Active Sessions             : 0<br>
Average FPS                 : 0<br>
Average Latency             : 0<br>
FBC Stats<br>
Active Sessions             : 0<br>
Average FPS                 : 0<br>
Average Latency             : 0<br>
Ecc Mode<br>
Current                     : N/A<br>
Pending                     : N/A<br>
ECC Errors<br>
Volatile<br>
Single Bit<br>
Device Memory       : N/A<br>
Register File       : N/A<br>
L1 Cache            : N/A<br>
L2 Cache            : N/A<br>
Texture Memory      : N/A<br>
Texture Shared      : N/A<br>
CBU                 : N/A<br>
Total               : N/A<br>
Double Bit<br>
Device Memory       : N/A<br>
Register File       : N/A<br>
L1 Cache            : N/A<br>
L2 Cache            : N/A<br>
Texture Memory      : N/A<br>
Texture Shared      : N/A<br>
CBU                 : N/A<br>
Total               : N/A<br>
Aggregate<br>
Single Bit<br>
Device Memory       : N/A<br>
Register File       : N/A<br>
L1 Cache            : N/A<br>
L2 Cache            : N/A<br>
Texture Memory      : N/A<br>
Texture Shared      : N/A<br>
CBU                 : N/A<br>
Total               : N/A<br>
Double Bit<br>
Device Memory       : N/A<br>
Register File       : N/A<br>
L1 Cache            : N/A<br>
L2 Cache            : N/A<br>
Texture Memory      : N/A<br>
Texture Shared      : N/A<br>
CBU                 : N/A<br>
Total               : N/A<br>
Retired Pages<br>
Single Bit ECC              : N/A<br>
Double Bit ECC              : N/A<br>
Pending                     : N/A<br>
Temperature<br>
GPU Current Temp            : 66 C<br>
GPU Shutdown Temp           : 96 C<br>
GPU Slowdown Temp           : 91 C<br>
GPU Max Operating Temp      : 101 C<br>
Memory Current Temp         : N/A<br>
Memory Max Operating Temp   : N/A<br>
Power Readings<br>
Power Management            : N/A<br>
Power Draw                  : 41.15 W<br>
Power Limit                 : N/A<br>
Default Power Limit         : N/A<br>
Enforced Power Limit        : N/A<br>
Min Power Limit             : N/A<br>
Max Power Limit             : N/A<br>
Clocks<br>
Graphics                    : 1126 MHz<br>
SM                          : 1126 MHz<br>
Memory                      : 2505 MHz<br>
Video                       : 1036 MHz<br>
Applications Clocks<br>
Graphics                    : 1038 MHz<br>
Memory                      : 2505 MHz<br>
Default Applications Clocks<br>
Graphics                    : 1038 MHz<br>
Memory                      : 2505 MHz<br>
Max Clocks<br>
Graphics                    : 1126 MHz<br>
SM                          : 1126 MHz<br>
Memory                      : 2505 MHz<br>
Video                       : 1036 MHz<br>
Max Customer Boost Clocks<br>
Graphics                    : N/A<br>
Clock Policy<br>
Auto Boost                  : N/A<br>
Auto Boost Default          : N/A<br>
Processes<br>
Process ID                  : 20840<br>
Type                    : C<br>
Name                    : C:\Users\itaim\AppData\Local\Programs\Python\Python36\python.exe<br>
Used GPU Memory         : Not available in WDDM driver model</p>
<p>and my code:</p>
<p>import torch<br>
import torchvision<br>
import torch.nn as nn<br>
import torch.nn.functional as F<br>
from torchvision import transforms, datasets, models<br>
from torch.utils.data import Dataset, DataLoader<br>
from torch.autograd import Variable<br>
import torch.optim as optim<br>
from torch.optim import lr_scheduler<br>
from torch.utils.data.sampler import SubsetRandomSampler<br>
import torch.backends.cudnn as cudnn<br>
import matplotlib.pyplot as plt<br>
import numpy as np<br>
import torch.optim as optim<br>
import pandas as pd<br>
import os<br>
from PIL import Image<br>
import pickle<br>
import urllib<br>
from urllib.request import urlopen<br>
import time<br>
import copy</p>
<p>device = torch.device(“cuda:0” if torch.cuda.is_available() else “cpu”)</p>
<p>print(“torch.cuda.is_available()   =”, torch.cuda.is_available())<br>
print(“torch.cuda.device_count()   =”, torch.cuda.device_count())<br>
print(“torch.cuda.device(‘cuda’)   =”, torch.cuda.device(‘cuda’))<br>
print(“torch.cuda.current_device() =”, torch.cuda.current_device())<br>
print(“torch.cuda.get_device_name(0) =”,torch.cuda.get_device_name(0))</p>
<p>cudnn.benchmark = True</p>
<p><span class="hashtag-raw">#Additional</span> Info when using cuda<br>
if device.type == ‘cuda’:<br>
print(torch.cuda.get_device_name(0))<br>
print(‘Memory Usage:’)<br>
print(‘Allocated:’, round(torch.cuda.memory_allocated(0)/1024<strong>3,1), ‘GB’)<br>
print('Cached:   ', round(torch.cuda.memory_cached(0)/1024</strong>3,1), ‘GB’)</p>
<p>class DogsTrainingDataset(Dataset):</p>
<pre><code>def __init__(self,text_file,root_dir):
    """
    Args:
        text_file(string): path to text file
        root_dir(string): directory with all train images
    """
    self.name_frame = pd.read_csv(text_file,sep=",",usecols=range(1))
    self.label_frame = pd.read_csv(text_file,sep=",",usecols=range(1,2))
    self.label_idx_frame = pd.read_csv(text_file,sep=",",usecols=range(2,3))
    self.root_dir = root_dir
    self.transform = transforms.Compose([
         transforms.Resize(224),
         transforms.CenterCrop(224),
         transforms.ToTensor(),
         transforms.Normalize( mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
         ])

def __len__(self):
    return len(self.name_frame)

def __getitem__(self, idx):
    img_name = os.path.join(self.root_dir, self.name_frame.iloc[idx, 0]) + '.jpg'
    label_str = self.label_frame.iloc[idx, 0]
    label = self.label_idx_frame.iloc[idx, 0]
    image = Image.open(img_name)
    image = self.transform(image)
    #labels = labels.reshape(-1, 2)
    return [image, label, label_str]
</code></pre>
<p>def imshow(img, title=None):<br>
inp = img.numpy().transpose((1, 2, 0))<br>
mean = np.array([0.485, 0.456, 0.406])<br>
std = np.array([0.229, 0.224, 0.225])<br>
inp = std * inp + mean<br>
inp = np.clip(inp, 0, 1)<br>
plt.imshow(inp)<br>
if title is not None:<br>
plt.title(title)<br>
plt.pause(0.001)  # pause a bit so that plots are updated</p>
<p>def train_model(model, criterion, optimizer, scheduler, validation_split, dataloaders, num_epochs=25):<br>
since = time.time()</p>
<pre><code>best_model_wts = copy.deepcopy(model.state_dict())
best_acc = 0.0
best_epoch_idx = 0
train_loss = []
train_acc = []
val_loss = []
val_acc = []

val_size = np.floor(len(dataloaders['train'].dataset) * validation_split)
train_size = len(dataloaders['train'].dataset) - val_size

for epoch in range(num_epochs):
    print('Epoch {}/{}'.format(epoch, num_epochs - 1))
    print('-' * 10)

    # Each epoch has a training and validation phase
    for phase in ['train', 'val']:
        if phase == 'train':
            dataset_size = train_size
            scheduler.step()
            model.train()  # Set model to training mode
        else:
            dataset_size = val_size
            model.eval()   # Set model to evaluate mode

        running_loss = 0.0
        running_corrects = 0

        # Iterate over data.
        for inputs, labels, labels_str in dataloaders[phase]:
            inputs = inputs.to(device)
            labels = labels.to(device)

            # zero the parameter gradients
            optimizer.zero_grad()

            # forward
            # track history if only in train
            with torch.set_grad_enabled(phase == 'train'):
                #if(phase == 'train'):
                #    outputs,aux = model(inputs)
                #else:
                #    outputs = model(inputs)
                outputs = model(inputs)
                _, preds = torch.max(outputs, 1)
                loss = criterion(outputs, labels)

                # backward + optimize only if in training phase
                if phase == 'train':
                    loss.backward()
                    optimizer.step()

            # statistics
            running_loss += loss.item() * inputs.size(0)
            running_corrects += torch.sum(preds == labels.data)
        epoch_loss = running_loss / dataset_size
        epoch_acc = running_corrects.double() / dataset_size
        
        print('{} Loss: {:.4f} Acc: {:.4f}'.format(
            phase, epoch_loss, epoch_acc))
        # save loss and accuracy
        if phase == 'train':
            train_loss.append(epoch_loss)
            train_acc.append(epoch_acc)
        else:
            val_loss.append(epoch_loss)
            val_acc.append(epoch_acc)
        # deep copy the model
        if phase == 'val' and epoch_acc &gt; best_acc:
            best_acc = epoch_acc
            best_epoch_idx = epoch
            best_model_wts = copy.deepcopy(model.state_dict())

time_elapsed = time.time() - since
print('Training complete in {:.0f}m {:.0f}s'.format(
    time_elapsed // 60, time_elapsed % 60))
print('Best val Acc: {:4f}'.format(best_acc))

# plot loss and accuracy

plt.figure()
plt.xlabel('# of epochs')
plt.title('Accuracy')
plt.plot(range(num_epochs), train_acc, label='Train Accuracy')
plt.plot(range(num_epochs), val_acc, label='Validation Accuracy')
plt.legend()
plt.figure()
plt.xlabel('# of epochs')
plt.title('Loss')
plt.plot(range(num_epochs), train_loss, label='Train Loss')
plt.plot(range(num_epochs), val_loss, label='Validation Loss')
plt.legend()
print('best epoch index:',best_epoch_idx)
# load best model weights
model.load_state_dict(best_model_wts)    
return model
</code></pre>
<p>dogsTrainSet = DogsTrainingDataset(text_file =‘C:\Users\itaim\Master\TeamLily\labels.csv’,<br>
root_dir = ‘C:\Users\itaim\Master\TeamLily\train’)<br>
num_epochs = 10<br>
batch_size = 16<br>
validation_split = .2<br>
shuffle_dataset = True<br>
random_seed= 42</p>
<h1><a name="creating-data-indices-for-training-and-validation-splits-1" class="anchor" href="#creating-data-indices-for-training-and-validation-splits-1"></a>Creating data indices for training and validation splits:</h1>
<p>dataset_size = len(dogsTrainSet)<br>
indices = list(range(dataset_size))<br>
split = int(np.floor(validation_split * dataset_size))<br>
if shuffle_dataset :<br>
np.random.seed(random_seed)<br>
np.random.shuffle(indices)<br>
train_indices, val_indices = indices[split:], indices[:split]</p>
<h1><a name="creating-pt-data-samplers-and-loaders-2" class="anchor" href="#creating-pt-data-samplers-and-loaders-2"></a>Creating PT data samplers and loaders:</h1>
<p>train_sampler = SubsetRandomSampler(train_indices)<br>
val_sampler = SubsetRandomSampler(val_indices)</p>
<h1><a name="handle-data-loaders-3" class="anchor" href="#handle-data-loaders-3"></a>handle data loaders</h1>
<p>dogsTrainLoader = DataLoader(dogsTrainSet, batch_size=16, sampler=train_sampler, pin_memory=True)<br>
dogsValLoader = DataLoader(dogsTrainSet, batch_size=16, sampler=val_sampler, pin_memory=True)<br>
dataloaders = {‘train’: dogsTrainLoader , ‘val’: dogsValLoader}</p>
<h1><a name="init-model-and-its-attributes-4" class="anchor" href="#init-model-and-its-attributes-4"></a>init model and its attributes</h1>
<p>model = models.resnet50(pretrained=True)<br>
<span class="hashtag-raw">#model</span>.aux_logit=False</p>
<h1><a name="option-1-use-the-model-as-feature-extractor-train-only-classifier-5" class="anchor" href="#option-1-use-the-model-as-feature-extractor-train-only-classifier-5"></a>option 1 - use the model as feature extractor (train only classifier)</h1>
<p>for param in model.parameters():<br>
param.requires_grad = False<br>
num_ftrs = model.fc.in_features<br>
model.fc = nn.Linear(num_ftrs, 120)<br>
model = model.to(device)</p>
<p>criterion = nn.CrossEntropyLoss()</p>
<h1><a name="choose-sg-with-momentum-6" class="anchor" href="#choose-sg-with-momentum-6"></a>choose SG with momentum</h1>
<p>optimizer = optim.SGD(model.fc.parameters(), lr=0.001, momentum=0.9)</p>
<h1><a name="decay-lr-by-a-factor-of-01-every-7-epochs-7" class="anchor" href="#decay-lr-by-a-factor-of-01-every-7-epochs-7"></a>Decay LR by a factor of 0.1 every 7 epochs</h1>
<p>exp_lr_scheduler = lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)</p>
<h1><a name="train-the-model-and-evaluate-on-the-validation-set-8" class="anchor" href="#train-the-model-and-evaluate-on-the-validation-set-8"></a>train the model and evaluate on the validation set</h1>
<p>print(‘#########################################’)<br>
print(‘running with SGD’)<br>
print(‘#########################################’)<br>
model_trained1 = train_model(model, criterion, optimizer,<br>
exp_lr_scheduler, validation_split, dataloaders, num_epochs=25)</p>
<p>print(‘#########################################’)<br>
print(‘running with Adam’)<br>
print(‘#########################################’)<br>
optimizer = optim.Adam(model.fc.parameters(), lr=0.001, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, amsgrad=False)<br>
exp_lr_scheduler = lr_scheduler.StepLR(optimizer, step_size=7, gamma=0.1)<br>
model_trained2 = train_model(model, criterion, optimizer,<br>
exp_lr_scheduler, validation_split, dataloaders, num_epochs=25)</p>
<p>any suggestions?</p>
<p>Thanks in advance <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>

albanD: <p>Hi,</p>
<p>You can print the inputs before feeding them to the model to make sure they are on the gpu.<br>
It is possible that your model is so small that the gpu is barely used as well, you can try increasing the batch size to see if the gpu usage increases.</p>

imesery: <p>Hi,<br>
thanks for your response <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=7" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
I printed the tensors and they’re on the GPU (device=cuda0).<br>
I also tried increasing the batch size to 32 and 64 and still nothing…</p>
<p>what about the alarming signs from nvidia-smi, such as the “N/A” in the GPU memory usage for the process, or this:</p>
<pre><code>Processes
    Process ID                  : 15980
        Type                    : C
        Name                    : C:\Users\itaim\AppData\Local\Programs\Python\Python36\python.exe
        Used GPU Memory         : Not available in WDDM driver model
</code></pre>
<p>what is this WDDM?</p>
<p>Thanks,<br>
Itai</p>

albanD: <p>I have to admit I don’t know how these things work on windows…<br>
Maybe <a class="mention" href="/u/peterjc123">@peterjc123</a> will know what’s happening here?</p>

imesery: <p><a class="mention" href="/u/peterjc123">@peterjc123</a>  have any idea??</p>

peterjc123: <p>WDDM is a driver model for GPU under Windows. By using WDDM, you can use it as a device to render your graphics as well as do some math calculating. The alternative one is TCC. By using TCC, it can be only used to do some calculations so that if you don’t have any other GPUs, you can not even boot up your machine. But your ML process won’t get interrupted so it’s great for HPC. Unfortunately, it is only available for Tesla and Quadro cards. Here are the details for the TCC mode: <a href="https://docs.nvidia.com/gameworks/content/developertools/desktop/nsight/tesla_compute_cluster.htm" rel="nofollow noopener">https://docs.nvidia.com/gameworks/content/developertools/desktop/nsight/tesla_compute_cluster.htm</a>.</p>
<p>From what you posted, I don’t think there’s anything wrong there since it says <code>Not available in WDDM driver model</code>.</p>

imesery: <p>Thanks for your response <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=7" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
I understand… so my issues are not related to this WDDM thing.<br>
have any idea why I get no GPU usage?</p>

peterjc123: <p>What do you mean by <code>get no GPU usage</code>? If you are referring to the GPU memory usage of a separate process, then it is clearly written there: <code>Not available in WDDM driver model</code>. However, the mem usage for the whole GPU should be available through <code>nvidia-smi</code>.</p>

imesery: <p>I’m running my pytorch code (you can take a look at the original post) and I get almost no GPU usage (~1%) when looking in the task manager (also attached above).<br>
All the CUDA signs from python seems to be ok so I’m not sure what’s going on here…</p>

imesery: <p>BTW when running “nvidia-smi -l” I see “N/A” in “GPU Memory Usage” of the python process</p>

peterjc123: <p><a class="mention" href="/u/imesery">@imesery</a> The graph in Task Manager is confusing. Actually, it doesn’t show anything about gpu mem usage. And the python output is correct because when you call it, you didn’t store any tensors on GPU.</p>

peterjc123: <p>You can simply execute the following piece of code to verify whether it’s working.</p>
<pre><code class="lang-python">&gt;&gt;&gt; import torch
&gt;&gt;&gt; a = torch.cuda.FloatTensor(10000)
&gt;&gt;&gt; print("Allocated:", round(torch.cuda.memory_allocated(0)/10243,1), "GB")
Allocated: 3.9 GB
&gt;&gt;&gt; b = torch.cuda.FloatTensor(20000)
&gt;&gt;&gt; print("Allocated:", round(torch.cuda.memory_allocated(0)/10243,1), "GB")
Allocated: 11.8 GB
</code></pre>

imesery: <p>but I do store tensors on GPU, when printing the tensors it says they’re on “device=cuda0”.<br>
attaching another screenshot of task manager during the python run when GPU graph is enlarged.<br>
the CPU usage went up to ~55% when started running but the GPU usage is at 0.</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/5/5408e8c3f388e3b61a0096b493c4c34a9f86867c.png" data-download-href="https://discuss.pytorch.org/uploads/default/5408e8c3f388e3b61a0096b493c4c34a9f86867c" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/5/5408e8c3f388e3b61a0096b493c4c34a9f86867c_2_582x499.png" alt="image" data-base62-sha1="bZpc7pu8LihyEBSyqw1d4yZ6bPm" width="582" height="499" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/5/5408e8c3f388e3b61a0096b493c4c34a9f86867c_2_582x499.png, https://discuss.pytorch.org/uploads/default/optimized/2X/5/5408e8c3f388e3b61a0096b493c4c34a9f86867c_2_873x748.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/5/5408e8c3f388e3b61a0096b493c4c34a9f86867c_2_1164x998.png 2x" data-dominant-color="F8F9FA"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">2168×1860 232 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

imesery: <p>import torch</p>
<p>a = torch.cuda.FloatTensor(10000)<br>
print(“Allocated:”, round(torch.cuda.memory_allocated(0)/10243,1), “GB”)</p>
<p>b = torch.cuda.FloatTensor(20000)<br>
print(“Allocated:”, round(torch.cuda.memory_allocated(0)/10243,1), “GB”)</p>
<p>Allocated: 22595.4 GB<br>
Allocated: 22595.4 GB</p>

peterjc123: <p>Yes, it’s because it’s not large enough.<br>
Try this one:</p>
<pre><code class="lang-python">torch.rand(20000,20000).cuda()
</code></pre>
<p>It will allocate nearly 2GB.<br>
P.S. <code>print(“Allocated:”, round(torch.cuda.memory_allocated(0)/10243,1), “GB”)</code> this clause is not correct and I just copied it from your code to say that it’s not zero.</p>

imesery: <p>yes I fixed the clause to be 1024**3.</p>
<p>torch.rand(20000,20000).cuda()<br>
print(“Allocated:”, round(torch.cuda.memory_allocated(0)/1024**3,1), “GB”)</p>
<p>Allocated: 0.2 GB</p>

peterjc123: <p>You’ll need to save it into a variable, otherwise it will be released.</p>

imesery: <p>when trying to add this print inside my train_model function (after each data fetch) I get 0.5GB, added it after this:<br>
inputs = inputs.to(device)<br>
labels = labels.to(device)</p>

peterjc123: <p>I don’t know how large your data is so I don’t know whether it’s correct. But I think since it’s increasing, it should have no much trouble.</p>

imesery: <p>ran this again:<br>
c = torch.rand(20000,20000).cuda()<br>
print(“Allocated:”, round(torch.cuda.memory_allocated(0)/1024**3,1), “GB”)</p>
<p>and I get 1.5GB</p>