PyTorch Profiler not working with CUDA

marewi: <p>Hi,<br>
For me, Torch.Profiler is not working with CUDA activity only. With CPU it is working for me.</p>
<p><strong>Code</strong>:</p>
<pre><code class="lang-auto">with torch.profiler.profile(
        activities=[torch.profiler.ProfilerActivity.CUDA],
        schedule=torch.profiler.schedule(wait=1, warmup=1, active=3, repeat=2),        
        on_trace_ready=torch.profiler.tensorboard_trace_handler('./log/model'),
        profile_memory=False,
        record_shapes=True,
        with_stack=True,
        use_cuda=True
    ) as prof:

        for mini_batch_data in dataloader:
              ...
</code></pre>
<p><strong>Error Msg</strong>: “AssertionError: No activities specified for Kineto profiler”</p>
<p>Is someone able to give me an hint?</p>
<p>Thanks in advance!</p>

ptrblck: <p>I cannot reproduce the issue using this simple code snippet:</p>
<pre><code class="lang-python">import torch
from torch.profiler import profile, record_function, ProfilerActivity

with profile(
        activities=[torch.profiler.ProfilerActivity.CUDA],
        schedule=torch.profiler.schedule(wait=1, warmup=1, active=3, repeat=2),        
        on_trace_ready=torch.profiler.tensorboard_trace_handler('./log/model'),
        profile_memory=False,
        record_shapes=True,
        with_stack=True,
    ) as prof:
        for _ in range(10):
            y = torch.randn(1).cuda() + torch.randn(1).cuda()
            prof.step()
            

print(prof.key_averages())
</code></pre>
<p>so could you post a minimal, executable code snippet which would reproduce the issue, please?</p>

marewi: <aside class="quote no-group" data-username="ptrblck" data-post="2" data-topic="132061">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<pre><code class="lang-auto">import torch
from torch.profiler import profile, record_function, ProfilerActivity

with profile(
        activities=[torch.profiler.ProfilerActivity.CUDA],
        schedule=torch.profiler.schedule(wait=1, warmup=1, active=3, repeat=2),        
        on_trace_ready=torch.profiler.tensorboard_trace_handler('./log/model'),
        profile_memory=False,
        record_shapes=True,
        with_stack=True,
    ) as prof:
        for _ in range(10):
            y = torch.randn(1).cuda() + torch.randn(1).cuda()
            prof.step()
            

print(prof.key_averages())
</code></pre>
</blockquote>
</aside>
<p>Hi ptrblck,<br>
actually I can reproduce the same issue with your code snippet. Here the full error stack:</p>
<pre><code class="lang-auto">C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\autograd\profiler.py:440: UserWarning: CUPTI tracing is not available, falling back to legacy CUDA profiling
  warn("CUPTI tracing is not available, falling back to legacy CUDA profiling")
Traceback (most recent call last):
  File "test_profiler.py", line 14, in &lt;module&gt;
    prof.step()
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\profiler\profiler.py", line 282, in step
    self._start_warmup()
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\profiler\profiler.py", line 413, in _start_warmup
    self.profiler = prof.profile(
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\autograd\profiler.py", line 447, in __init__
    assert len(self.kineto_activities) &gt; 0, \
AssertionError: No activities specified for Kineto profiler

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "test_profiler.py", line 14, in &lt;module&gt;
    prof.step()
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\profiler\profiler.py", line 253, in __exit__
    self._exit_actions()
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\profiler\profiler.py", line 404, in _exit_actions
    self._start_trace()
  File "C:\Users\user\AppData\Local\Programs\Python\Python38\lib\site-packages\torch\profiler\profiler.py", line 425, in _start_trace
    assert self.profiler is not None
AssertionError
</code></pre>
<p>FYI:</p>
<ul>
<li>Python 3.8.0</li>
<li>GPU: NVIDIA Tesla V100-PCIE-16GB</li>
<li>torch==1.9.0+cu111</li>
<li>torch-tb-profiler==0.2.0</li>
</ul>

marewi: <p>Any tips/ideas?<br>
Or are any additional information needed?</p>

ptrblck: <p>No, unfortunately I don’t have any debugging advice and would recommend to create an issue in GitHub, so that others could try to reproduce it as well.</p>

marewi: <aside class="quote no-group" data-username="marewi" data-post="3" data-topic="132061">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/48db29/48.png" class="avatar"> marewi:</div>
<blockquote>
<p>NVIDIA Tesla V100-PCIE-16GB</p>
</blockquote>
</aside>
<p>Problem was issued here: <a href="https://github.com/pytorch/pytorch/issues/65393" class="inline-onebox" rel="noopener nofollow ugc">PyTorch Profiler is not working with CUDA · Issue #65393 · pytorch/pytorch · GitHub</a></p>

Emil895: <p>To run profiler you have do some operations, you have to input some tensor into your model.</p>
<p>Change your code as following<a href="https://www.liteblue.app/" rel="noopener nofollow ugc">.</a></p>
<pre><code class="lang-auto">import torch
import torchvision.models as models

model = models.densenet121(pretrained=True)
x = torch.randn((1, 3, 224, 224), requires_grad=True)

with torch.autograd.profiler.profile(use_cuda=True) as prof:
    model(x)
print(prof) 
</code></pre>
<p>This is the sample of the output I got<a href="https://www.liteblue.app/" rel="noopener nofollow ugc">.</a></p>

ptrblck: <p>MY code snippet is already performing operations on the device via:</p>
<pre><code class="lang-python">y = torch.randn(1).cuda() + torch.randn(1).cuda()
</code></pre>
<p>and while it works for me, it seems that <a class="mention" href="/u/marewi">@marewi</a> is still hitting the issue.<br>
Are you seeing the same error using my code snippet?</p>

jaketalyor32325: <p>Are you using Windows? I found this open issue.</p>
<h1>
<a name="pytorch-profiler-kineto-is-not-available-windows-oshttpsgithubcompytorchkinetoissues280-280-1" class="anchor" href="#pytorch-profiler-kineto-is-not-available-windows-oshttpsgithubcompytorchkinetoissues280-280-1"></a><a href="https://github.com/pytorch/kineto/issues/280#" rel="noopener nofollow ugc">PyTorch Profiler Kineto is not available - windows OS</a> <span class="hashtag">#280</span>
</h1>