Macro F1 Score for Each iteration

username: sgaur
cooked: <p>How to create “macro F1 score” metric for each iteration.</p>
<p>I build some code but it is evaluating according to per batches.</p>
<p>Can we use sklearn suggested macro F1 metric, Going through lots of discussion many people suggested not to use it as it is works according per batches.</p>
<p>NOTE : <strong>My target consists more that 3 classes so I needed Multi-class macro F1 score</strong><br>
.</p>
<p>Thanks in advance.</p>

username: ptrblck
cooked: <aside class="quote no-group" data-username="sgaur" data-post="1" data-topic="73200">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/s/3da27b/48.png" class="avatar"> sgaur:</div>
<blockquote>
<p>Going through lots of discussion many people suggested not to use it as it is works according per batches.</p>
</blockquote>
</aside>
<p>What do you mean by that?<br>
Usually you can use sklearn metrics to observe your training.<br>
Note that you cannot use them as a loss to calculate the gradients, but sklearn should be fine to calculate various metrics.</p>

username: sgaur
cooked: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a></p>
<p>So my dataset contains 100 records and I need to run for 5 epoch and my batch size are 20.So if I use Macro F1 score using sklearn it give me f1 score after every batch (batch_1 = a(F1 Macro score),batch_2=b,… and batch_5=e) and my macro f1 score for epoch 1 is calculated by averages of  5 batches (a+b+…+e/5).</p>
<p>I read through some discussion pages related to keras which tells me above is not the best approach so keras remove macro f1 metrics now,</p>
<p>I am new to data science and pytorch just need to know is this the same problem for pytorch.</p>
<p>Hope and make you understand.Also I am attaching a link of stackoverflow/github discussion.<br>
<aside class="onebox stackexchange">
  <header class="source">
      <a href="https://stackoverflow.com/questions/43547402/how-to-calculate-f1-macro-in-keras" target="_blank" rel="nofollow noopener">stackoverflow.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/7676016/aryo-pradipta-gema" target="_blank" rel="nofollow noopener">
    <img alt="Aryo Pradipta Gema" src="https://graph.facebook.com/1832873433405045/picture?type=large" class="thumbnail onebox-avatar" width="60" height="60">
  </a>
<h4>
  <a href="https://stackoverflow.com/questions/43547402/how-to-calculate-f1-macro-in-keras" target="_blank" rel="nofollow noopener">How to calculate F1 Macro in Keras?</a>
</h4>

<div class="tags">
  <strong>keras</strong>
</div>

<div class="date">
  asked by
  
  <a href="https://stackoverflow.com/users/7676016/aryo-pradipta-gema" target="_blank" rel="nofollow noopener">
    Aryo Pradipta Gema
  </a>
  on <a href="https://stackoverflow.com/questions/43547402/how-to-calculate-f1-macro-in-keras" target="_blank" rel="nofollow noopener">04:11PM - 21 Apr 17 UTC</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<aside class="onebox githubissue">
  <header class="source">
      <a href="https://github.com/keras-team/keras/issues/11705" target="_blank" rel="nofollow noopener">github.com/keras-team/keras</a>
  </header>
  <article class="onebox-body">
    <div class="github-row">
  <div class="github-icon-container" title="Issue">
	  <svg width="60" height="60" class="github-icon" viewbox="0 0 14 16" aria-hidden="true"><path d="M7 2.3c3.14 0 5.7 2.56 5.7 5.7s-2.56 5.7-5.7 5.7A5.71 5.71 0 0 1 1.3 8c0-3.14 2.56-5.7 5.7-5.7zM7 1C3.14 1 0 4.14 0 8s3.14 7 7 7 7-3.14 7-7-3.14-7-7-7zm1 3H6v5h2V4zm0 6H6v2h2v-2z"></path></svg>
  </div>

  <div class="github-info-container">
    <h4>
      <a href="https://github.com/keras-team/keras/issues/11705" target="_blank" rel="nofollow noopener">How do keras calulate the metric?</a>
    </h4>

    <div class="github-info">
      <div class="date">
        opened <span class="discourse-local-date" data-format="ll" data-date="2018-11-22" data-time="02:48:36" data-timezone="UTC">02:48AM - 22 Nov 18 UTC</span>
      </div>

        <div class="date">
          closed <span class="discourse-local-date" data-format="ll" data-date="2019-04-24" data-time="22:01:49" data-timezone="UTC">10:01PM - 24 Apr 19 UTC</span>
        </div>

      <div class="user">
        <a href="https://github.com/Suncicie" target="_blank" rel="nofollow noopener">
          <img alt="Suncicie" src="https://avatars1.githubusercontent.com/u/19358604?v=4" class="onebox-avatar-inline" width="20" height="20">
          Suncicie
        </a>
      </div>
    </div>
  </div>
</div>

<div class="github-row">
  <p class="github-content">I'm trying to write a new metric of f1-macro for the softmax, I have tried two methods, I can't understand why...</p>
</div>

<div class="labels">
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">backend:tensorflow</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">stat:awaiting response</span>
    <span style="display:inline-block;margin-top:2px;background-color: #B8B8B8;padding: 2px;border-radius: 4px;color: #fff;margin-left: 3px;">type:support</span>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>