Pytorch Profiler not profiling GPU on Windows

ChowderII: <p>Hello everyone,</p>
<p>I’m new here, hopefully I write this in the correct way.</p>
<p>I’ve recently gotten to use PyTorch’s profiler but I can’t seem to see any activity on my GPU as far as the profiler is concerned. Currently I’m running the example as seen on this <a href="https://pytorch.org/tutorials/intermediate/tensorboard_profiler_tutorial.html" rel="noopener nofollow ugc">guide</a>. The code runs no problem and compiles. I can see activity on my GPU and the CUDA graph in task manager (showing specifically the CUDA graph, I did my homework) is showing activity when I run the code so it clearly is not PyTorch nor CUDA my problem.</p>
<p>Here is the code I use to create the model and start the profiler:</p>
<pre data-code-wrap="python"><code class="lang-python">def main():
    transform = T.Compose(
        [T.Resize(224),
         T.ToTensor(),
         T.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
    train_set = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)
    train_loader = torch.utils.data.DataLoader(train_set, batch_size=32, shuffle=True, num_workers=4)

    device = torch.device("cuda:0")
    model = torchvision.models.resnet18(pretrained=True).to(device)
    criterion = torch.nn.CrossEntropyLoss().to(device)
    optimizer = torch.optim.SGD(model.parameters(), lr=0.001, momentum=0.9)
    model.train()

    def train(data):
        inputs, labels = data[0].to(device), data[1].to(device)
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

    with torch.profiler.profile(
            schedule=torch.profiler.schedule(wait=1, warmup=1, active=3),
            on_trace_ready=torch.profiler.tensorboard_trace_handler('./log/resnet18'),
            record_shapes=True,
            profile_memory=True,
            with_stack=True
    ) as prof:
        for step, batch_data in enumerate(train_loader):
            if step &gt;= 200:  # tried increasing this because maybe it wasn't running long enough?
                break
            train(batch_data)
            prof.step()  # Need to call this at the end of each step to notify profiler of steps' boundary.


if __name__ == '__main__':
    main()

</code></pre>
<p>And yet when I open TensorBoard at the appropriate location I see only this:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/f/ef38c37d68464ad0d0bcc2ec6d423e75e9ea05a0.png" data-download-href="https://discuss.pytorch.org/uploads/default/ef38c37d68464ad0d0bcc2ec6d423e75e9ea05a0" title="missing_gpu_summary"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/f/ef38c37d68464ad0d0bcc2ec6d423e75e9ea05a0_2_690x391.png" alt="missing_gpu_summary" data-base62-sha1="y8fJjNUVFseK5h2q6JrHLOsvui4" width="690" height="391" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/f/ef38c37d68464ad0d0bcc2ec6d423e75e9ea05a0_2_690x391.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/f/ef38c37d68464ad0d0bcc2ec6d423e75e9ea05a0_2_1035x586.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/e/f/ef38c37d68464ad0d0bcc2ec6d423e75e9ea05a0_2_1380x782.png 2x" data-dominant-color="F5D8CE"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">missing_gpu_summary</span><span class="informations">1475×837 33.9 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div><br>
And the  GPU Kernel element is completely missing from the side bar as well.</p>
<p>I don’t understand what I am doing wrong in this case. Is this just a Windows problem or am I using the profiler incorrectly?</p>
<p>Thank you for your help!<br>
ChowderII</p>

Autumnii: <p>Hi Chowderll, have you found the solution? I’m using Windows too and that’s the same problem with me. Just CPU and other, no GPU.</p>

KuSi833: <p>Same problem here. Am also on Windows and can’t see the GPU summary.</p>

mfatih7: <p>Hello</p>
<p>I am using WIN10, and I can’t see any GPU Kernel option on tensorboard View list.</p>

Piadina: <p>Hi, I am experiencing the very same problem on Windows 10 and my model is definitely running on the GPU.</p>
<p>Any clue?</p>

lyclyc121: <p>I have the same problem, is anyone care and solve this?</p>

rojas561: <p>Same issue here. Any idea on this <a class="mention" href="/u/ptrblck">@ptrblck</a>? Your profile seems to pop up in all the relevant forms I’ve seen throughout the years so here’s to hoping you know what’s going on.</p>

ptrblck: <p>Sorry, but I’m neither familiar with Windows nor with the native PyTorch profiler and am using Nsight Systems to profile models.<br>
You could try to use it too: <a href="https://developer.nvidia.com/nsight-systems/get-started" class="inline-onebox">Getting Started with Nsight Systems | NVIDIA Developer</a></p>

rojas561: <p>I’ll check it out, thank you!</p>

J.S_Ye: <p>Have anyone solved this problem or work out some clues? I have encountered the same issue.</p>

haitao_jaing: <p>Maybe I’m late, but has anyone solved this problem?<br>
my environment：<br>
cuda11.8<br>
win10<br>
i512500k<br>
rtx4070ti</p>

Ankur_Singh: <p>Was anyone able to resolve this? I’m facing the same issue.</p>

Chen_Zejun: <p>Hi, all<br>
The CUPTI seems have issue working on Windows platform, so the CUPTI is anyway disabled when building on Windows, that is why you don’t have any GPU op info traced. It looks like the kineto supports Windows?<br>
Thank you.</p>

vattha: <p>I have also the same problem.</p>

neoncube: <p>As someone who is able to profile GPU usage on Windows, I’d be happy to help troubleshoot <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=14" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>
<p>I recently posted some GPU profiling code that worked for me: <a href="https://discuss.pytorch.org/t/model-uses-gpu-but-backwards-doesnt/218667/3" class="inline-onebox">Model() uses GPU but backwards() doesn't - #3 by neoncube</a></p>
<p>It looks like your code isn’t passing <code>activities=[profiler.ProfilerActivity.CPU, profiler.ProfilerActivity.CUDA]</code> to <code>torch.profiler.profile()</code>, which I thought was required.</p>
<p>Also, for me, passing both <code>activities=[profiler.ProfilerActivity.CPU, profiler.ProfilerActivity.CUDA]</code> and <code>with_stack=True</code> crashed the process with no error, so you might want to try removing <code>record_shapes</code>, <code>profile_memory</code>, and <code>with_stack</code>.</p>
<p>I’d also be curious to see if calling <code>prof.key_averages().table(sort_by='cuda_time_total', row_limit=10</code> or <code>prof.key_averages().table(sort_by='cpu_time_total', row_limit=10)</code>works and if this is just an issue exporting to Tensorboard, specifically.</p>

Jadiker: <p>I’m currently using Windows 10 and torch 1.13.1+cu117 on an ROG Zephyrus G14, and can confirm that <code>print(prof.key_averages().table(sort_by="self_cuda_time_total", row_limit=10))</code> shows <code>Self CUDA time total: 749.103ms</code> for the following script:</p>
<pre><code class="lang-auto">import torch
from torch.profiler import profile, ProfilerActivity

x = torch.randn(4000, 4000, device='cuda')
y = torch.randn(4000, 4000, device='cuda')

with profile(
    activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA]
) as prof:
    z = torch.matmul(x, y)
    torch.cuda.synchronize()  # ensure kernel finishes

prof.export_chrome_trace("cuda_matmul_trace.json")

print(prof.key_averages().table(sort_by="self_cuda_time_total", row_limit=10))
</code></pre>
<p>However, when I load the output <code>.json</code> file in <code>chrome://trace</code>, there’s no GPU events. Additionally, I uploaded the json file itself to ChatGPT which verified that no GPU events were included in the trace.</p>
<p>Thus, this does seem to be an issue in the export, rather than the tracking.</p>

v01d: <p>For me, a restart helped <img src="https://discuss.pytorch.org/images/emoji/apple/see_no_evil_monkey.png?v=14" title=":see_no_evil_monkey:" class="emoji" alt=":see_no_evil_monkey:" loading="lazy" width="20" height="20"></p>

ng_xing: <p>In my case, the issue was resolved after ensuring that the PyTorch CUDA version matches the CUDA version of my device. You can check your PyTorch version using <em>pip show torch</em> and the cuda version using <em>nvidia-smi</em>. For example, <strong>if your PyTorch version is 2.6.0+cu128, your cuda version should be CUDA 12.8.</strong></p>

Deepak_Kumar1: <p>Mine also crashing when keeping stack_trace=True, does your issue get resolved?</p>

neoncube: <p>No, I think I just removed the <code>stack_trace=True</code>, since it turns out that I didn’t need the stack trace after all.</p>