Pytorch profiler step?

Anders_V: <p>Hi, I’m trying to get started with the Pytorch profiler and noticed that in all of my runs on different models/tutorial codes the Pytorch tensorboard always displays step number 0? I’m confused if this means that it only did one loop of sampling or if there is some Tensorboard setting I need to hit? Honestly I’m very confused about if the Profiler is behaving as expected</p>
<p>Finally I copied the code from this tutorial verbatim:<br>
<a href="https://pytorch.org/tutorials/intermediate/tensorboard_profiler_tutorial.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/tutorials/intermediate/tensorboard_profiler_tutorial.html</a></p>
<p>and the end result looks like this on my tensorboard</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/6/4/64116ec8e32be8af5c141cd7018d7ee4686a5450.png" data-download-href="https://discuss.pytorch.org/uploads/default/64116ec8e32be8af5c141cd7018d7ee4686a5450" title="Tensorboard dashboard"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/6/4/64116ec8e32be8af5c141cd7018d7ee4686a5450_2_690x292.png" alt="Tensorboard dashboard" data-base62-sha1="ehf5Z7v460OQxWANaENY7CpYeXu" width="690" height="292" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/6/4/64116ec8e32be8af5c141cd7018d7ee4686a5450_2_690x292.png, https://discuss.pytorch.org/uploads/default/optimized/3X/6/4/64116ec8e32be8af5c141cd7018d7ee4686a5450_2_1035x438.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/6/4/64116ec8e32be8af5c141cd7018d7ee4686a5450_2_1380x584.png 2x" data-dominant-color="DBDAE8"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Tensorboard dashboard</span><span class="informations">1872×794 86.8 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>It is different from the tutorial which had four steps. (Should be noted I’m running tensorboard from VS code, not a chrome instance like the tutorial)</p>

Brasilino: <p>Hello <a class="mention" href="/u/anders_v">@Anders_V</a></p>
<p>Please double-check the <code>prof.step()</code> indentation. Make sure it is within the <code>for</code> loop scope.</p>
<p>Best,<br>
Lucas</p>

Anders_V: <p>Hi, I reran and double checked the prof.step() indentation, but I’m still having the same issue.</p>
<p>Here is my code for reference.</p>
<pre><code class="lang-auto"># https://pytorch.org/tutorials/intermediate/tensorboard_profiler_tutorial.html

import torch
import torch.nn
import torch.optim
import torch.profiler
import torch.utils.data
import torchvision.datasets
import torchvision.models
import torchvision.transforms as T


transform = T.Compose(
    [T.Resize(224),
     T.ToTensor(),
     T.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])
train_set = torchvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)
train_loader = torch.utils.data.DataLoader(train_set, batch_size=32, shuffle=True)

device = torch.device("cuda:0")
model = torchvision.models.resnet18(weights='IMAGENET1K_V1').cuda(device)
criterion = torch.nn.CrossEntropyLoss().cuda(device)
optimizer = torch.optim.SGD(model.parameters(), lr=0.001, momentum=0.9)
model.train()


def train(data):
    inputs, labels = data[0].to(device=device), data[1].to(device=device)
    outputs = model(inputs)
    loss = criterion(outputs, labels)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()


with torch.profiler.profile(
        schedule=torch.profiler.schedule(wait=1, warmup=1, active=3, repeat=2),
        on_trace_ready=torch.profiler.tensorboard_trace_handler('./log/resnet18'),
        record_shapes=True,
        profile_memory=True,
        with_stack=True
) as prof:
    for step, batch_data in enumerate(train_loader):
        if step &gt;= (1 + 1 + 3) * 2:
            break
        train(batch_data)
        prof.step()
</code></pre>
<p>When the code above finished executing I get this in the terminal:</p>
<pre><code class="lang-auto">Files already downloaded and verified
STAGE:2023-05-05 10:14:30 5872:5872 ActivityProfilerController.cpp:311] Completed Stage: Warm Up
[W CPUAllocator.cpp:235] Memory block of unknown size was allocated before the profiling started, profiler results will not include the deallocation event
STAGE:2023-05-05 10:14:30 5872:5872 ActivityProfilerController.cpp:317] Completed Stage: Collection
STAGE:2023-05-05 10:14:30 5872:5872 ActivityProfilerController.cpp:321] Completed Stage: Post Processing
STAGE:2023-05-05 10:14:30 5872:5872 ActivityProfilerController.cpp:311] Completed Stage: Warm Up
STAGE:2023-05-05 10:14:30 5872:5872 ActivityProfilerController.cpp:317] Completed Stage: Collection
STAGE:2023-05-05 10:14:30 5872:5872 ActivityProfilerController.cpp:321] Completed Stage: Post Processing
</code></pre>
<p>And the result when I start tensorboard is the same as in the original image.</p>