Local variable 'loss' referenced before assignment

KanZa: <p>it gives me this error; local variable ‘loss’ referenced before assignment</p>
<pre><code class="lang-auto">def train_epoch(models, loss_module, criterion, optimizers_backbone, optimizers_module, dataloaders, epoch, epoch_loss):

    loss_module.train()
    global iters
    for data in tqdm(dataloaders['train'], leave=False, total=len(dataloaders['train'])):
        with torch.cuda.device(CUDA_VISIBLE_DEVICES):
            inputs = data[0].cuda()
            labels = data[1].cuda()

        iters += 1

        optimizers_backbone.zero_grad()
        optimizers_module.zero_grad()

        scores, _, features = models(inputs) 
        target_loss = criterion(scores, labels)
        
        if epoch &gt; epoch_loss:
            features[0] = features[0].detach()
            features[1] = features[1].detach()
            features[2] = features[2].detach()
            features[3] = features[3].detach()

        pred_loss = loss_module(features)
        pred_loss = pred_loss.view(pred_loss.size(0))
        m_module_loss   = LossPredLoss(pred_loss, target_loss, margin=MARGIN)
        m_backbone_loss = torch.sum(target_loss) / target_loss.size(0)        
        loss = m_backbone_loss + WEIGHT * m_module_loss 
        
        loss.backward()
        optimizers_backbone.step()
        optimizers_module.step()
    
    return loss
</code></pre>

thecho7: <p>Which line occurs the error?<br>
If it is <code>return loss</code>, then check out the for loop iterate at least once.</p>

KanZa: <p>I don’t think so for-loop is running at least once.<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/e/9/e93906ecf3be856c089d64bce4092cb3cb66d124.png" data-download-href="https://discuss.pytorch.org/uploads/default/e93906ecf3be856c089d64bce4092cb3cb66d124" title="Screenshot from 2022-07-20 17-25-59"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/e/9/e93906ecf3be856c089d64bce4092cb3cb66d124_2_690x131.png" alt="Screenshot from 2022-07-20 17-25-59" data-base62-sha1="xhbqGTa4RpPLsz0oIngDy7YdXYU" width="690" height="131" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/e/9/e93906ecf3be856c089d64bce4092cb3cb66d124_2_690x131.png, https://discuss.pytorch.org/uploads/default/optimized/3X/e/9/e93906ecf3be856c089d64bce4092cb3cb66d124_2_1035x196.png 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/e/9/e93906ecf3be856c089d64bce4092cb3cb66d124.png 2x" data-dominant-color="451C1A"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">Screenshot from 2022-07-20 17-25-59</span><span class="informations">1236×236 27.3 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

thecho7: <p>It seems dataloaders is invalid.<br>
print <code>len(dataloaders['train'])</code> then it may returns 0</p>

KanZa: <p>in Train() where I am called train_epoch(), I have printed the data loaders</p>
<p>DataLoader {‘train’: &lt;torch.utils.data.dataloader.DataLoader object at 0x7f4d756dbc10&gt;, ‘test’: &lt;torch.utils.data.dataloader.DataLoader object at 0x7f4d756dbb20&gt;}<br>
DataLoader {‘train’: &lt;torch.utils.data.dataloader.DataLoader object at 0x7f4d756dbc10&gt;, ‘test’: &lt;torch.utils.data.dataloader.DataLoader object at 0x7f4d756dbb20&gt;}</p>
<pre><code class="lang-auto">def train(models, loss_module, criterion, optimizers_backbone, optimizers_module, scheduler_backbone, scheduler_module, dataloaders, num_epochs, epoch_loss):
    print('&gt;&gt; Train a Model.')
    best_acc = 0.
    
    for epoch in range(num_epochs):

        best_loss = torch.tensor([0.5]).cuda()
        print("DataLoader", dataloaders)
        
        loss = train_epoch(models, loss_module, criterion, optimizers_backbone, optimizers_module, dataloaders, epoch, epoch_loss)

        scheduler_backbone.step()
        scheduler_module.step()

        if False and epoch % 20  == 7:
            acc = test(models, epoch, method, dataloaders, mode='test')
            # acc = test(models, dataloaders, mc, 'test')
            if best_acc &lt; acc:
                best_acc = acc
                print('Val Acc: {:.3f} \t Best Acc: {:.3f}'.format(acc, best_acc))
    print('&gt;&gt; Finished.')
</code></pre>

KanZa: <p>This code works absolutely fine for CIFAR-100 when I tried this for ImageNet, it started giving this error. and now even not working for CIFAR100</p>

Vinay_Verma: <aside class="quote no-group" data-username="KanZa" data-post="6" data-topic="157048">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kanza/48/1529_2.png" class="avatar"> KanZa:</div>
<blockquote>
<p>This code works absolutely fine for CIFAR-100 when I tried this for ImageNet, it started giving this error. and now even not working for CIFAR100</p>
</blockquote>
</aside>
<p>Hi, the dataloader you have printed is fine, these are reference to the dataloader objects. Please check the length of <code>dataloaders</code> object, easy way can be</p>
<pre data-code-wrap="py"><code class="lang-py">print(len(dataloaders['train']))
# OR 
for i, data in enumerate(dataloaders['train']):
    pass
print("Length of dataloader:", i)
</code></pre>
<p>The training code looks perfect, but your <code>loss</code> variable is defined only into the loop. And the loop runs number of batches present in dataloader.<br>
The loop is not running for certain dataloader, and <code>loss</code> variable is left unassigned.</p>
<p>You can initialize the <code>loss</code> variable outside the for loop, but that wont solve the issue. It will just stop throwing the error.</p>
<p>Hope it helps. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=12" title=":slight_smile:" class="emoji" alt=":slight_smile:" loading="lazy" width="20" height="20"></p>