How to print the probability of correct category of a labeled example?

username: Saewon_Yang
cooked: <p>Hi, I’m trying to implement the following paper, UDA</p>
<p>“<strong>Unsupervised Data Augmentation</strong>” which was released couple of months ago</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://arxiv.org/abs/1904.12848v1">
  <header class="source">
      <img src="https://arxiv.org/static/browse/0.3.4/images/icons/favicon-32x32.png" class="site-icon" width="" height="">

      <a href="https://arxiv.org/abs/1904.12848v1" target="_blank" rel="noopener nofollow ugc">arXiv.org</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/402;"><img src="https://static.arxiv.org/icons/twitter/arxiv-logo-twitter-square.png" class="thumbnail" width="" height=""></div>

<h3><a href="https://arxiv.org/abs/1904.12848v1" target="_blank" rel="noopener nofollow ugc">Unsupervised Data Augmentation</a></h3>

  <p>Despite its success, deep learning still needs large labeled datasets to succeed. Data augmentation has shown much promise in alleviating the need for more labeled data, but it so far has mostly been applied in supervised settings and achieved...</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/b/b7d4988643047059ce9ff16e32be8405bbb86f98.png" data-download-href="https://discuss.pytorch.org/uploads/default/b7d4988643047059ce9ff16e32be8405bbb86f98" title="KakaoTalk_20190701_210834637"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/b/b7d4988643047059ce9ff16e32be8405bbb86f98_2_690x145.png" alt="KakaoTalk_20190701_210834637" data-base62-sha1="qeeRpze4vv680HdbGAqwUZtmhAs" width="690" height="145" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/b/b7d4988643047059ce9ff16e32be8405bbb86f98_2_690x145.png, https://discuss.pytorch.org/uploads/default/optimized/2X/b/b7d4988643047059ce9ff16e32be8405bbb86f98_2_1035x217.png 1.5x, https://discuss.pytorch.org/uploads/default/original/2X/b/b7d4988643047059ce9ff16e32be8405bbb86f98.png 2x" data-dominant-color="D8D7A6"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">KakaoTalk_20190701_210834637</span><span class="informations">1367×289 106 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>
<p>I can understand about the Training Signal Annealing which removes part of the training example</p>
<p>but the problem is my coding ability. I’m having difficulty coding the above lines.<br>
following is my codes of applying UDA hope someone can help me</p>
<pre><code>        criterion = nn.CrossEntropyLoss()
        train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
        unsupervised_loader = itertools.cycle(unsup_loader)

    for i, (inputs, targets) in enumerate(train_loader):
            
        if not unsupervised:
             inputs, targets = inputs.cuda(), targets.cuda()
             outputs = model(inputs)
             loss = criterion(outputs, targets)
            
        else:
            targets = targets.cuda()
            unlabel1, unlabel2 = next(unsupervised_loader)
            data_all = torch.cat([inputs, unlabel1, unlabel2]).cuda()

            preds_all = model(data_all)
            preds = preds_all[:len(inputs)]
            loss = crietrion(preds, targets) # loss for supervised learning

            preds_unsup = preds_all[len(data):]
            preds1, preds2 = torch.chunk(preds_unsup, 2)
            preds1 = softmax(preds1, dim=1).detach()
            preds2 = log_softmax(preds2, dim=1)
            assert len(preds1) == len(preds2) == batch_unsup

            loss_kldiv = kl_div(preds2, preds1, reduction='none') # loss for unsupervised learning
            loss_kldiv = torch.sum(loss_kldiv, dim=1)
            loss += torch.mean(loss_kldiv) * ratio_unsup   # ratio_unsup = len(test) / (len(train) + len(test))
</code></pre>
<p>brief description of my code: first, I made separate loaders for supervised and unsupervised. I concatenated these and predicted through the model. and for unsupervised examples I calculated kl-divergence.</p>
<p>how can I remove predicted examples which are above threshold and not put it in the loss function?<br>
any comment would be a great help to me. thanks!!</p>