Why is my dataset class giving index out of range errors

nima_pw: <p>Hi, I am determining why my data set gives IndexError: list index out of range<br>
error.</p>
<p>Consider this torch data set:</p>
<pre><code>class MyDataset(Dataset):

def __init__(self, imgs , transform = None):
    self.imgs = imgs
    self.transform = transform or transforms.ToTensor()
    self.class_to_idx = {}

def __getitem__(self, index):
    
    image_path = self.imgs[index]
    name = image_path.split('/D')[1]
    target = name.split('_')[0]
    
    image = Image.open(image_path)
    
    if self.transform is not None:
        image = self.transform(image)

    if target in class_to_idx : 
        target = [class_to_idx[target]]
    else : 
        class_to_idx[target] = (int(target)-1)
        target = [class_to_idx[target]]


    return image , target

def __len__(self):
    return len(self.imgs)
</code></pre>
<p>An image_path example: <code>"Train/D01_I_flat_0009.jpg"</code><br>
The dataset has 35 classes: D01…D35</p>
<p>I want to calculate mean and std, but I get the error:<br>
<a href="https://i.stack.imgur.com/TwSeD.png" rel="noopener nofollow ugc"><img src="https://i.stack.imgur.com/TwSeD.png" alt="enter image description here" width="690" height="451"></a><br>
<a href="https://i.stack.imgur.com/BLErB.png" rel="noopener nofollow ugc"><img src="https://i.stack.imgur.com/BLErB.png" alt="enter image description here" width="690" height="246"></a></p>
<p>When I test the dataset, it works correctly:</p>
<p><a href="https://i.stack.imgur.com/TOzbo.png" rel="noopener nofollow ugc"><img src="https://i.stack.imgur.com/TOzbo.png" alt="enter image description here" width="690" height="201"></a></p>

ptrblck: <p>Your <code>image_path</code> doesn’t seem to contain the expected <code>/D</code> string and thus the <code>split</code> call fails:</p>
<pre><code class="lang-python"># works
image_path = "aaa/Dbbb"
name = image_path.split('/D')[1]
print(name)
# bbb

# fails
image_path = "aaabbb"
name = image_path.split('/D')[1]
# IndexError: list index out of range
</code></pre>

nima_pw: <p>Dear <a class="mention" href="/u/ptrblck">@ptrblck</a>, thanks for your reply.</p>
<p>All images’ names start with Dxx, and the full path of training images is ./Train/Dxx_***.jpg.<br>
for example ./Train/D24_nature.jpg</p>
<p>As I mentioned, when I pass a batch (size=16) to the data loader, it correctly returns the labels and displays the images:</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/5/c/5cb3f1ffba28541db23b3162ab5c111d5951df08.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/5cb3f1ffba28541db23b3162ab5c111d5951df08" title="image"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/5/c/5cb3f1ffba28541db23b3162ab5c111d5951df08_2_690x248.jpeg" alt="image" data-base62-sha1="de5spO0vQ99LvQpuvqoisnkqlWo" width="690" height="248" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/5/c/5cb3f1ffba28541db23b3162ab5c111d5951df08_2_690x248.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/5/c/5cb3f1ffba28541db23b3162ab5c111d5951df08_2_1035x372.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/original/3X/5/c/5cb3f1ffba28541db23b3162ab5c111d5951df08.jpeg 2x" data-dominant-color="B0AFAC"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">1208×435 87.5 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

nima_pw: <p>I found the reason for the error. It was an odd cause!<br>
There weren’t any other files but .jpg  files in the Train folder. However, I applied the following change:</p>
<pre><code>train_files = glob.glob(train_dir + '/*')
</code></pre>
<p>to</p>
<pre><code>train_files = glob.glob(train_dir + '/*.jpg')
</code></pre>
<p>The problem was solved!</p>
<p>It is worth mentioning that I didn’t face this problem on Google Colab, but with Jupyter Notebook on my machine.</p>