Get current Batch-ID while train()

reinforce: <p>Hello,</p>
<p>I’m facing an problem of getting the current Batch-ID variable in PytorchIm enumerating over an data_loader with a Batch-Size of 16. My Dataset is therefore divided into 1640 Batches.</p>
<p>In one train()-iteration, one batch will loaded and the loss will be calculated. I would like to read the specific Batch-ID (723 for e.g.) while for-enumerating the data_loader.</p>
<p>How do I get access to that specific value inside the data_loader?</p>

ptrblck: <p>The batch index can be created via:</p>
<pre><code class="lang-python">for batch_idx, data in enumerate(loader):
</code></pre>
<p>which will sequentially increase <code>batch_idx</code> in the <code>range</code> of <code>len(loader)</code>. Is this what you are looking for?</p>

reinforce: <p>Thanks!<br>
If <code>shuffle=false</code> I can assume that <code>batch_idx</code> stays the same for all iterations of the batches for all epochs?</p>

azhanmohammed: <p>Yes keeping <code>shuffle=False</code> would mean that your fetched data remains the same for the same batch index for all epochs.</p>

reinforce: <p>Allright thanks!</p>
<p>Is it also possible to define an own list with indices in a specific order for training?</p>

ptrblck: <p>If you want to load the samples via the <code>Dataset.__getitem__</code> in a specific order you could create a custom <code>sampler</code> and pass it to the <code>DataLoader</code>.<br>
Take a look at <a href="https://github.com/pytorch/pytorch/blob/a315a2c79bbd25dfb022e294df8df67568b14dea/torch/utils/data/sampler.py">these <code>sampler</code> implementations</a> to see how they are written.</p>

reinforce: <p>Thanks for your reply. But I don’t want to shuffle the samples for my own. I just want to calculate a specific order of the loaded batches for the next epoch and say the dataloader to please use my own list of arranged batch-IDs instead of the standard ascending order.<br>
Like: epoch 1: Batch-IDs=[1,2,3,4,5]<br>
epoch 2: Batch-IDs=[3,1,5,2,4]</p>

azhanmohammed: <p>I do not think there is a way to do this using PyTorch’s dataloader, but you can always write a custom data loading function of your own which can do the same. A simple way I can think of to do this is shown below:</p>
<pre><code class="lang-python">'''the loadBatch function below expects three arguments, list containing image paths, list of labels, and a start index, this index can be used to access the batches in the exact order you want, for example index [0:batchSize] can be batch 1, [batchSize:2*batchSize] can be batch 2, and so on'''
batchSize = 20 #assuming sample batch size of 20
def loadBatch(imagePath, imageLabels, batchID):
    images = imagePath[batchID:batchID+batchSize]
    labels = imageLabels[batchID:batchID+batchSize]
    imageFiles = []
    for image in images:
        i = cv2.imread(image, i)
        #insert augmentation and resizing functions here to make sure all images have same size
        i = np.transpose(i, (2,0,1)) #converting HWC to CHW
        imageFiles.append(i)
    imageFiles = np.array(imageFiles)
    imageFiles = torch.from_numpy(imageFiles)
    labels = torch.from_numpy(labels)
    return imageFiles, labels
</code></pre>
<p>The above function can now be used load the batches in any order you want given that you provide the batchID properly. For example lets take the following example:<br>
batch 1: start index=0<br>
batch 2: start index = 20 since we assumed batch size of 20<br>
batch 3: start index = 40 and so on<br>
And for the training part we wish to load the batches in the following order:<br>
Epoch 1: batch 1, batch 3, batch 2, batch 4<br>
Epoch 2: batch 3, batch 2, batch 4, batch 1<br>
So we will create a list of lists which has the order of starting indices for these batches,</p>
<pre><code class="lang-python">#you can create a custom order generator as per your own need to create the list
orderOfBatch = [[0, 40, 20, 60], [40, 20, 60, 0]]
for epoch in range(0,2):
    batchindices = orderOfBatch[epoch]
    for batchIndex in batchindices:
        images, labels = loadBatch(imagePath, imageLabels, batchIndex)
        #training code goes here
</code></pre>
<p>While this is not the most efficient way, it can surely get the task done.</p>

ptrblck: <p>A custom sampler should also work, as you can directly pass the manually created indices to it and allow it to create the batches through the <code>DataLoader</code> <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=12" title=":wink:" class="emoji" alt=":wink:" loading="lazy" width="20" height="20"></p>

reinforce: <p>Thanks for your answer, but I dont want to create Batches by now.<br>
I want to do the following:<br>
 → Create Batches with shuffle=false, assume I have 64 Batches.<br>
 → After creation of the batches, I will run 1 Epoch with that. This Epoch will load the Batches by Indice</p>
<pre><code class="lang-auto">load(batch_1), 
load(batch_2)
.
.
. 
load(batch_64)
</code></pre>
<p> → After one Epoch I want to create a new order of loading the batches and train them for Epoch 2 like for example:</p>
<pre><code class="lang-auto">load(batch_9), 
load(batch_27)
.
.
. 
load(batch_33)
</code></pre>
<p>The thing I want to pass to the <code>dataloader</code> is a <code>list</code> with valid indices of the order of sequential loaded batches:</p>
<pre><code class="lang-auto">batch_order_for_next_episode_to_load=[9,27,35,...,33]
</code></pre>