Batch optimization

username: slavavs
cooked: <p>I have a lot of data. At the entrance of the network, I submit a window the size of batch.<br>
Batch I create in a loop.<br>
I made the code, it works, but not very efficiently.<br>
I have to repeat the same thing in every epoch. Inside the epoch, code also creates batch for a very long time. Are there any ideas how to optimize my code?</p>
<pre><code class="lang-auto">for epoch in range(epochs):
    for wn_start in range(0,len_batch-wn1,batch):
        wn_t = wn_t + wn1
        wn_all = []
        los_target = []
        for b_iter in range(batch):
            wn_all = wn_all + [st_s[wn_start+b_iter:wn_t+b_iter,:]]
            los_target = los_target + [st_target[wn_t-1+b_iter]]
        wn_all = torch.as_tensor(wn_all, dtype=torch.float32)
        los_target = torch.Tensor([los_l])
</code></pre>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/2X/6/6d7eacec8c758b0f9f4ad240026d844abee57c76.png" data-download-href="https://discuss.pytorch.org/uploads/default/6d7eacec8c758b0f9f4ad240026d844abee57c76" title="wn"><img src="https://discuss.pytorch.org/uploads/default/optimized/2X/6/6d7eacec8c758b0f9f4ad240026d844abee57c76_2_690x238.png" alt="wn" data-base62-sha1="fCDrBybhesPND26WLNUi55Pxf1Q" width="690" height="238" srcset="https://discuss.pytorch.org/uploads/default/optimized/2X/6/6d7eacec8c758b0f9f4ad240026d844abee57c76_2_690x238.png, https://discuss.pytorch.org/uploads/default/optimized/2X/6/6d7eacec8c758b0f9f4ad240026d844abee57c76_2_1035x357.png 1.5x, https://discuss.pytorch.org/uploads/default/optimized/2X/6/6d7eacec8c758b0f9f4ad240026d844abee57c76_2_1380x476.png 2x" data-dominant-color="F9F9F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">wn</span><span class="informations">1923×664 8.09 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

username: KFrank
cooked: <p>Hi Slavavs!</p>
<aside class="quote no-group" data-username="slavavs" data-post="1" data-topic="50006" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/slavavs/48/15429_2.png" class="avatar"> slavavs:</div>
<blockquote>
<p>I have a lot of data. At the entrance of the network, I submit a window the size of batch.<br>
Batch I create in a loop.<br>
I made the code, it works, but not very efficiently.<br>
I have to repeat the same thing in every epoch. Inside the epoch, code also creates batch for a very long time. Are there any ideas how to optimize my code?</p>
</blockquote>
</aside>
<p>In short, you build (wrap arrays as) torch tensors inside your<br>
loop.  Instead, you should build one large tensor and then index<br>
(slice) into it.</p>
<p>I haven’t tried to figure out your code or reproduce its exact<br>
functionality.  But here’s a sample that illustrates how to do<br>
what I think you are trying to do:</p>
<pre data-code-wrap="python"><code class="lang-python">import numpy as np
import torch

st_s = np.random.randn (100, 5).astype (np.float32)
st_target = np.random.randint (0, 5, (100,)).astype (np.int64)

st_s_as_tensor = torch.as_tensor (st_s)
st_target_as_tensor = torch.as_tensor (st_target)

type (st_s_as_tensor)
type (st_target_as_tensor)

for  i in range (5):
     # 5 batches of 20 inputs and targets
     st_s_batch = st_s_as_tensor[20 * i : 20 * (i + 1)]
     st_target_batch = st_target_as_tensor[20 * i : 20 * (i + 1)]
     st_s_batch.shape
     st_s_batch.dtype
     st_target_batch.shape
     st_target_batch.dtype

# and that's the tea!
</code></pre>
<p>Here is the relevant part of the output I get from running this script:</p>
<pre><code class="lang-plaintext">torch.Size([20, 5])
torch.float32
torch.Size([20])
torch.int64
torch.Size([20, 5])
torch.float32
torch.Size([20])
torch.int64
&lt;etc.&gt;
</code></pre>
<p>In my example, I assume that your input is a numpy array of 100<br>
samples of 5 floating-point features each, and that your targets<br>
are 100 integer class labels that match up to your 100 samples.</p>
<p>The script then wraps your numpy arrays (using the same<br>
underlying memory) in torch tensors, and then indexes into<br>
the torch tensors to get 5 batches of 20 samples each.</p>
<p>Best.</p>
<p>K. Frank</p>