Ignite ConfusionMatrix Metric Error

Nareg: <p>I have a working ignite setup - precision, recall accuracy all work. However, I am getting a bizarre error when I try to add in <code>ConfusionMatrix(num_classes=1)</code>.  Note the batch size is 5, and this pertains to a binary classifier.</p>
<p>/usr/local/lib/python3.7/site-packages/ignite/metrics/confusion_matrix.py in update(self, output)<br>
92     <span class="mention">@reinit__is_reduced</span><br>
93     def update(self, output: Sequence[torch.Tensor]) -&gt; None:<br>
—&gt; 94         self._check_shape(output)<br>
95         y_pred, y = output<br>
96</p>
<p>/usr/local/lib/python3.7/site-packages/ignite/metrics/confusion_matrix.py in _check_shape(self, output)<br>
78                 "y_pred must have shape (batch_size, num_categories, …) and y must have "<br>
79                 "shape of (batch_size, …), "<br>
—&gt; 80                 “but given {} vs {}.”.format(y.shape, y_pred.shape)<br>
81             )<br>
82</p>
<p><code>ValueError: y_pred must have shape (batch_size, num_categories, ...) and y must have shape of (batch_size, ...), but given torch.Size([5, 1]) vs torch.Size([5, 1]).</code></p>
<p>As you can see, the <code>y_pred</code> and <code>y</code> shapes match…any ideas what might be going on here?</p>

vfdev-5: <p>I agree that the error message</p>
<blockquote>
<p><code>y_pred must have shape (batch_size, num_categories, ...) and y  must have shape of (batch_size, ...), but given torch.Size([5, 1]) vs  torch.Size([5, 1])</code></p>
</blockquote>
<p>is not clear enough as it intends to say the following:</p>
<ul>
<li>y_pred must have shape (batch_size, num_categories, …) and given <code>torch.Size([5, 1])</code> = OK</li>
<li>y must have shape of (batch_size, …) and given <code>torch.Size([5, 1])</code> =&gt; Wrong</li>
</ul>
<p><code>y</code> should have the shape : (5, ) without 1</p>
<p>Could you please open an issue to improve the docs please on the github repository ?</p>

Nareg: <p>Thanks, that did the trick. Any idea what the output of ConfusionMatrix means for <code>num_classes=1</code>?</p>
<p>The documentation is thin on this as well, and it returns a 1x1 tensor / single value (I expected a 2x2 matrix). I can’t make sense of it based on my precision/recall.</p>

vfdev-5: <p>Yes, I agree that the docs should be reworked.<br>
If you would like to do binary classification, please set <code>num_classes=2</code>.<br>
Values of confusion matrix can be by <code>average</code> option to match precision, recall or number of samples…<br>
</p><aside class="onebox allowlistedgeneric">
  <header class="source">
      <a href="https://pytorch.org/ignite/metrics.html#ignite.metrics.ConfusionMatrix" target="_blank" rel="noopener nofollow ugc">pytorch.org</a>
  </header>
  <article class="onebox-body">
    <img src="" class="thumbnail" width="" height="">

<h3><a href="https://pytorch.org/ignite/metrics.html#ignite.metrics.ConfusionMatrix" target="_blank" rel="noopener nofollow ugc">ignite.metrics — ignite master documentation</a></h3>



  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


Nareg: <p>Thanks, new issue opened on GitHub here:</p>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/pytorch/ignite/issues/1380" target="_blank" rel="noopener nofollow ugc">github.com/pytorch/ignite</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewBox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/pytorch/ignite/issues/1380" target="_blank" rel="noopener nofollow ugc">Documentation for ConfusionMatrix is thin on details, and error messages not informative</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2020-10-12" data-time="14:42:47" data-timezone="UTC">02:42PM - 12 Oct 20 UTC</span>
      </div>


      <div class="user">
        <a href="https://github.com/nsinenian" target="_blank" rel="noopener nofollow ugc">
          <img alt="nsinenian" src="https://avatars1.githubusercontent.com/u/4325187?v=4" class="onebox-avatar-inline" width="20" height="20">
          nsinenian
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">🐛 Bug description
Confusion matrix requires that y_pred and y are to have different dimensions (batch,num_classes) and (batch,), respectively. Getting this wrong...</p>
</div>

<div class="labels">
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">docs</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">enhancement</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>