Problems reading path in costum dataset

username: mia1
cooked: <p>I am new and I am trying to learn pytorch<br>
so I started with creating a costum dataset for my data<br>
and i want to read the labels<br>
for the labels I only have 3  txt files one for the classID ,one for train split and one for the test split<br>
this is the custom data that I wrote so far</p>
<blockquote></blockquote>
<pre><code>     import os
     from PIL import Image
     import numpy as np
     from torch.utils.data import Dataset

     class MSR_Data(Dataset):

           def __init__(self, image_path, transform=None):

            # store all image paths in a list
             image_files = [os.path.join(image_path, item) for item in os.listdir(image_path)]

             self.images = []
           for path in image_files:
                # Load image using PIL
                   img = Image.open(path)
                   if img.mode != 'RGB':
                   img = img.convert('RGB')
                  # Convert PIL image to numpy array
                    im_np = np.array(img)
                   self.images.append(im_np)

                  # store transform
                 #self.transform = transform

               def __len__(self):
               return len(self.images)

              def __getitem__(self, idx):
             data = np.array(self.images[idx])
</code></pre>
<p>I didn’t do any transformation because I wanted it to understand step by step<br>
can someone help me how to read labels from txt file<br>
or shouls i convert to other file format I saw somone doing it with th json format</p>
<pre><code>            #if self.transform:
            # Apply transformation if available
           #  data = self.transform(data)

    return data
</code></pre>

username: pretbc
cooked: <p>Simple snip example, all depends how <code>labels.txt</code> looks like</p>
<pre><code class="lang-auto">class MSRData(Dataset):

    def __init__(self, img_path: str, labels_file: str, transform = None):
        self.image_files = [os.path.join(img_path, item) for item in os.listdir(img_path)]
        self.labels_file = labels_file
        
    def __len__(self):
        return len(self.image_files)
    
    def __getitem__(self, idx):
        img = cv2.imread(self.image_files[idx])
        label = self.get_label(idx)
        return img, label
        
    def get_label(self, idx):
        with open(self.labels_file) as labeler:
            lines = labeler.readlines()
            return lines[idx]
</code></pre>
<p>could U provide some data example that U wanna use ?</p>

username: mia1
cooked: <p>the classID.txt is like this</p>
<pre><code class="lang-auto">        1 CallCellphone
        2 cheerUp
        3 drink
        4 eat
         ......
</code></pre>
<p>the trainlist.txt</p>
<pre><code class="lang-auto">CallCellphone/a04_s01_e01_rgb.avi 1
CallCellphone/a04_s03_e01_rgb.avi 1
CallCellphone/a04_s03_e02_rgb.avi 1
cheerUp/a08_s01_e01_rgb.avi 2
cheerUp/a08_s02_e01_rgb.avi 2
cheerUp/a08_s02_e02_rgb.avi 2
cheerUp/a08_s03_e01_rgb.avi 2
drink/a01_s01_e01_rgb.avi 3
drink/a01_s02_e01_rgb.avi 3
drink/a01_s02_e02_rgb.avi 3
drink/a01_s03_e02_rgb.avi 3
....................
</code></pre>
<p>and the testlist.txt is like this</p>
<pre><code class="lang-auto">CallCellphone/a04_s01_e02_rgb.avi 
CallCellphone/a04_s02_e01_rgb.avi 
CallCellphone/a04_s02_e02_rgb.avi 
cheerUp/a08_s01_e02_rgb.avi 
cheerUp/a08_s04_e01_rgb.avi 
cheerUp/a08_s07_e02_rgb.avi 
drink/a01_s01_e02_rgb.avi 
drink/a01_s03_e01_rgb.avi 
drink/a01_s05_e01_rgb.avi 
......................

</code></pre>
<p>these are some exemples and the dataset I am using I conveted it from videos to jpg<br>
the strcture is</p>
<pre><code class="lang-auto">data--CallCellphone
               --video1
                     --img1
                     --img2
               --video2
                     --img1
                     --img2
      --CheerUp
               --video1
                     --img1
                     --img2
               --video2
                     --img1
                     --img2
............
</code></pre>

username: pretbc
cooked: <p>read ID.txt U can easly create dict</p>
<pre><code class="lang-auto">with open('ids.txt') as labels:
    return {line.strip().split(' ')[0]:line.strip().split(' ')[1] for line in labels.readlines()}
    
labels_dict = {'1': 'CallCellphone', '2': 'cheerUp'}
</code></pre>
<p>and Ur converted Dataset can be loaded as below:</p>
<pre><code class="lang-auto"># Use ImageFolder to create dataset(s)
from torchvision import datasets
train_data = datasets.ImageFolder(root=train_dir, # target folder of images
                                  transform=data_transform, # transforms to perform on data (images)
                                  target_transform=None) # transforms to perform on labels (if necessary)

test_data = datasets.ImageFolder(root=test_dir, 
                                 transform=data_transform)
</code></pre>
<p>check link to tutorial: <a href="https://www.learnpytorch.io/04_pytorch_custom_datasets/" rel="noopener nofollow ugc">Zero to Master</a></p>