Load unlabeld images from custom Dataset

GioSfat: <p>Hello everyone!<br>
I have a custom dataset with images in specific classes. I have saved this dataset on my computer using folders and subfolders.</p>
<ul>
<li>Train Dataset :<br>
-5_1<br>
-5_2<br>
-5_3<br>
-etc…</li>
</ul>
<p>Where the subfolders(5_1, 5_2, etc.) are the classes of the images. I want to use semi-supervised training where both labeled and unlabeled images must be used. But I don’t know how to “categorize” my unlabeled images in order to load them to my CNN. For the labeled images I use datasets.ImageFolder() and DataLoader() so I can load them for training.<br>
Thanks for the help!<br>
PS: I thought to save them in a different folder named as “Unlabeled” but I am afraid that it is gonna use the name of the folder as a new class and this is something that it’s gonna ruin the predictions in training as well in testing</p>

ptrblck: <aside class="quote no-group" data-username="GioSfat" data-post="1" data-topic="171049">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/giosfat/48/56684_2.png" class="avatar"> GioSfat:</div>
<blockquote>
<p>PS: I thought to save them in a different folder named as “Unlabeled” but I am afraid that it is gonna use the name of the folder as a new class and this is something that it’s gonna ruin the predictions in training as well in testing</p>
</blockquote>
</aside>
<p>You are right that the <code>ImageFolder</code> would create a new class index for this <code>Unlabeled</code> folder, but you could also replace it with your desired “invalid class index” and remap the other class indices if needed.<br>
This would allow you to use the plain <code>ImageFolder</code> dataset with a <code>DataLoader</code> without writing a custom <code>Dataset</code>.</p>

GioSfat: <blockquote>
<p>but you could also replace it with your desired “invalid class index” and remap the other class indices if needed</p>
</blockquote>
<p>By this you mean that I can change the name of the class inside of ImageFolder() command? Or there is another way by characterize these images as unlabeled?</p>

ptrblck: <p>Yes, you could try to change the class indices in the internal <code>ImageFolder</code> <code>.targets</code> attribute or even better you could try to remap the class indices in a <code>target_transform</code>.</p>

GioSfat: <p>Hello again!<br>
I still can’t resolve my problem.<br>
I tried to create a new Dataset but the main question remains. Even if I changed the name of the class as an empty string it still revognizes the classes giving them indices. Is there a way to “delete” classes and class indices, so when I try to print for example an image as:</p>
<p>[tensor([[0.7176, 0.7176, 0.7176,  …, 0.8863, 0.8863, 0.8863],<br>
[0.7176, 0.7176, 0.7176,  …, 0.8863, 0.8863, 0.8863], ]), tensor([3, 2])]</p>
<p>to give me only the tensor without the second part tensor([3, 2]) ?<br>
PS: the first tensor part is the image and the second is the class indices for the batch of two.</p>
<pre><code class="lang-auto">class CustomDataset(Dataset):
    def __init__(self, path):
        self.imgs_path = path
        file_list = glob.glob(self.imgs_path + "*")
        #print(file_list)
        self.data = []
        for class_path in file_list:
            class_name = ' '
            for img_path in glob.glob(class_path + "/*.jpg"):
                self.data.append([img_path, class_name])       
    def __len__(self):
        return len(self.data)
    def __getitem__(self, idx):
        img_path, class_name = self.data[idx]
        img = (cv2.imread(img_path)).float()
        class_id = class_name
        img_tensor = torch.from_numpy(img)
        img_tensor = img_tensor.permute(2, 0, 1)
        return img_tensor, class_id
</code></pre>

ptrblck: <p>No, there is no clean way of returning a valid target value for some samples and nothing for others since this would create inconsistent batches of targets.<br>
You could thus completely ignore the unknown classes and remove them from the dataset or return a “placeholder” target and deal with these samples somehow during training.</p>