Does Concatenate Datasets preserve class labels and indices

prashp: <p>Let’s say I have 2 image folder datasets and I want to concatenate them.</p>
<p>The first dataset has 100 images with 2 equal classes: “Dog” and “Cat” with class indices 0 and 1<br>
The second dataset has 120 images with 3 equal classes: “Dog”, “Cat” and “Pig” with class indices 0, 1 and 2</p>
<p>When I concatenate the two datasets with torch.utils.data.ConcatDataset(), will I get a dataset with 90 dog and 90 cat images, and 40 pig images? Or does it treat the two dog and cat labels from the two datasets as different classes - so I actually end up with 5 classes? I didn’t find an easy way to check because the ConcatDataset class doesn’t have a classes or class_to_idx method.</p>

prashp: <p>And also what of the case if a third dataset had only classes “dog” and “pig” and I concatenated it with the first and second? Does ConcatDataset() map that pig should have class index 2 instead of 1?</p>

ptrblck: <p><code>ConcatDataset</code> will not create a mapping, but just index the passed <code>Datasets</code>.<br>
Each <code>Dataset</code> should make sure to yield the “right” labels.<br>
E.g. in your second use case, you should make sure that <code>dataset1</code> only yields samples with the class labels 0 and 1 (dog and cat), while <code>dataset2</code> should only yield 0 and 2 (dog and pig).</p>
<p>The mapping is defined by your <code>Dataset</code> implementation.</p>
<p>Note that if you are using e.g. <code>ImageFolder</code>, the mapping will be created based on the folders, so I would not recommend using this approach, if your dataset folders do not contains the same classes.</p>

prashp: <p>I see, thanks. For a quick hack I ended up creating empty folders with all required classes in my image dataset root.</p>

Dheannisa_Ramadhani: <p>how to get the classes if i have three folder with same class inside the folder?</p>

ptrblck: <p>If you are using <code>ImageFolder</code>, you can access its <code>dataset.class_to_idx </code> attribute to see the mapping between the folders and the class indices.<br>
I’m not sure if I misunderstand the question, but do all three folders contain images from the same class (one class only)?</p>

Dheannisa_Ramadhani: <p>no, there is two class…<br>
Thank you for respond <img src="https://discuss.pytorch.org/images/emoji/apple/smile.png?v=9" title=":smile:" class="emoji" alt=":smile:"></p>

Amardeep_Mehta: <p>Hi, what could be right approach to concatenate 2 datasets having different classes/labels. For example dataset D1 has folders for “cat” and “dog” whereas dataset D2 has folders like “elephant” and “lion”. Right now I created empty folders named “elephant” and “lion” in dataset D1 to preserve the labels and vice versa before using ImageFolder and ConcatenateDataset as bellow:<br>
ds = torch.utils.data.ConcatDataset(<br>
[datasets.ImageFolder(’./data/D1/train’, transform),<br>
datasets.ImageFolder(’./data/D2/train’, transform)]<br>
)</p>

ptrblck: <p>Your approach sounds alright, if you want to use the <code>ImageFolder</code> datasets.<br>
Alternatively, you could write a custom <code>Dataset</code> and return the corresponding targets for both datasets, which wouldn’t rely on creating empty folders.</p>

Amardeep_Mehta: <p>Thanks! It will be great if there is some pointer to some examples for how to do it as I am new to pytorch.</p>

ptrblck: <p><a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html">This tutorial</a> shows how to write a custom <code>Dataset</code> and might be helpful. <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

Brando_Miranda: <aside class="quote no-group" data-username="ptrblck" data-post="3" data-topic="62611">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/48/1823_2.png" class="avatar"> ptrblck:</div>
<blockquote>
<p><code>ConcatDataset</code> will not create a mapping, but just index the passed <code>Datasets</code>.<br>
Each <code>Dataset</code> should make sure to yield the “right” labels.<br>
E.g. in your second use case, you should make sure that <code>dataset1</code> only yields samples with the class labels 0 and 1 (dog and cat), while <code>dataset2</code> should only yield 0 and 2 (dog and pig).</p>
<p>The mapping is defined by your <code>Dataset</code> implementation.</p>
</blockquote>
</aside>
<p>ok so if I want to:</p>
<blockquote>
<p>I’d lake to take the union for the labels and being relabeled from from scratch from <code>0</code> to <code>len_dataset1+len_dataset2-1</code>.</p>
</blockquote>
<p>Then I have to implement my own “version” of union/concate/merge data set that takes into account this. Likely re-implementing the <code>__getitem__(idx: int):</code> function to something like this:</p>
<pre><code class="lang-auto">def __getitem__(self, index: int):
    # leave the sampled labels of data set 1 as is
    img, target = self.mnist[index], int(self.mnist.targets[index])

    # to the sampled labels of data set 2 add the number of
    img, target = self.cifar10[index], int(self.cifar10.targets[index]) + len(self.mnist)
    return ...
</code></pre>
<p>darn this isn’t quite right…based on the index the data set I implement should know what is the right mapping…also this doesn’t work for an arbitrary union of data sets of course…then likely one needs to bisect function to find in which interval the idx is in the you know how many lens of data sets you need to add…</p>
<p>I think the easiest is to wrap your normal data set into learn2learn’s metadataset then pass it to their Union data set.</p>
<pre><code class="lang-auto">        train = torchvision.datasets.CIFARFS(root="/tmp/mnist", mode="train")
        train = l2l.data.MetaDataset(train)
        valid = torchvision.datasets.CIFARFS(root="/tmp/mnist", mode="validation")
        valid = l2l.data.MetaDataset(valid)
        test = torchvision.datasets.CIFARFS(root="/tmp/mnist", mode="test")
        test = l2l.data.MetaDataset(test)
        union = UnionMetaDataset([train, valid, test])
        assert len(union.labels) == 100
</code></pre>
<pre><code class="lang-auto">class UnionMetaDataset(MetaDataset):
    """
    **Description**
    
        Takes multiple MetaDataests and constructs their union.
    
        Note: The labels of all datasets are remapped to be in consecutive order.
        (i.e. the same label in two datasets will be to two different labels in the union)
    
        **Arguments**
    
        * **datasets** (list of Dataset) -  A list of torch Datasets.
</code></pre>
<p>link:  <a href="http://learn2learn.net/docs/learn2learn.data/#learn2learn.data.meta_dataset.UnionMetaDataset" class="inline-onebox" rel="noopener nofollow ugc">learn2learn.data - learn2learn</a></p>
<p>actually it’s easier to:</p>
<blockquote>
<p>Actually, it’s likely easier to preprocess the data points indices to map to the label required label (as you loop through each data set you’d know this value easily and keep a single counter) – instead of bisecting.</p>
</blockquote>

pdc_87: <p>Hi <a class="mention" href="/u/ptrblck">@ptrblck</a> I concatenated 3 datasets for data augmentation. The images were taken from the same path, so the three datasets have the same four labels. Is there a method o attribute for the ConcatDataset method to view the labels of the concatenated dataset like the ones for Dataset method.Further, I can use a Counter to check how many images are in the train and val folders.</p>
<p>for example</p>
<pre><code class="lang-auto">from collections import Counter
print(f'Classes: {train_loader.dataset.class_to_idx}')
num_classes_train = Counter(train_loader.dataset.targets)
num_classes_val = Counter(val_loader.dataset.targets)
print(f'Train Loader: {num_classes_train}')
print(f'Val Loader: {num_classes_val}')
</code></pre>
<p>Thanks in advance!<br>
Pablo</p>

ptrblck: <p>You can access the internal datasets via the <code>.datasets</code> attribute as seen here:</p>
<pre><code class="lang-python">dataset1 = datasets.MNIST(root="/data", download=False)
dataset1.targets

dataset2 = datasets.CIFAR10(root="/data", download=False)
dataset2.targets

dataset = torch.utils.data.ConcatDataset((dataset1, dataset2))
dataset.datasets[0].targets
dataset.datasets[1].targets
</code></pre>

pdc_87: <p>Ok so I have to build three counter for each dataset and sum up them. As far as I have seen ConcatDataset have not the same methods and attributes than Dataset. Thanks!</p>