[Resolved] Appropriate treatment of Epoch

username: 111137
cooked: <p>Hi,</p>
<p>I did training with;</p>
<pre><code class="lang-python">for epoch in range(EPOCH):
  for x, t in dataloader_train:
</code></pre>
<p>The <code>dataloader_train</code> includes input <strong>x</strong> with mini-batch size image and <strong>t</strong> as a scalar label. The code as above, I think same (x, t) is multiply loaded by EPOCH, so same sequence is repeated EPOCH times. If we use random sampling for each iteration for “epoch” then it is OK, I think, but such the above code is not appropriate code.</p>
<p>Is my thought a correct? And if it is so must I rewrite as follows?</p>
<pre><code class="lang-python">for epoch in range(EPOCH):
  dataloader_train = load_randomly()
  for x, t in dataloader_train:
</code></pre>

username: n0obcoder
cooked: <pre><code class="lang-auto">for x, t in dataloader_train:
    # this always gives you fresh x_batches and t_batches
</code></pre>
<p>and you can add the for loop on top of it, it you want to go through the whole train data again and again.</p>
<p>In short,</p>
<pre><code class="lang-auto">for epoch in range(EPOCH):
  for x, t in dataloader_train:
</code></pre>
<p>is perfectly fine ! <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>
<p>This is exactly how we train our models</p>

username: 111137
cooked: <p>Hi, <a class="mention" href="/u/n0obcoder">@n0obcoder</a> -san,</p>
<p>Thank you for your comments. So it implies that “EPOCH*BATCH_SIZE” should be equal to or less than the total number of training samples in the data set, right?</p>

username: n0obcoder
cooked: <p>no its not like that. Actually the EPOCH has nothing to do with the BATCH_SIZE.<br>
Let me put it this way…<br>
lets say we have 2560 training examples. We choose the BATCH_SIZE of 256, so when we run<br>
<code>for x, t in dataloader_train:</code><br>
This loop runs for 10 times (2560 examples/256 batch size)<br>
This is one forward pass ove rall the 2560 training examples, also known as 1 EPOCH<br>
Now if we want do repeart the same thing for 50 times, we set put it in another loop, like…</p>
<pre><code class="lang-auto">for epoch in range(50):
  for x, t in dataloader_train:
</code></pre>
<p>where our EPOCHS = 50</p>
<p>I hope this makes sense  <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"><br>
feel free to ask if you stilll have doubts</p>
<p>Kompai ! <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>

username: 111137
cooked: <p>Hi <a class="mention" href="/u/n0obcoder">@n0obcoder</a> -san,</p>
<p>Thank you very much for your advises !, your comments make me clear about it (^ - ^).<br>
I wonder is that this is quite different from ordinary other programming languages which takes sequential execution.</p>
<p>乾杯（kanpai）!</p>

username: n0obcoder
cooked: <p>glad to know that i could help. I am not sure about other programming languages. Python is the only programming language  that i know.</p>
<p>乾杯（kanpai）! <img src="https://discuss.pytorch.org/images/emoji/apple/smiley.png?v=9" title=":smiley:" class="emoji" alt=":smiley:"></p>