Change labels in Data Loader

shangeth: <p>I have a data set of images, labels . I took a subset of it and want to change the labels of the whole subset to a single label.</p>
<p>eg: MNIST 0,1,2,3,4,5,6,7,8,9  ; lets say i want labels of 5,6,7,8,9 be 5.   so final data labels be 0,1,2,3,4,5.</p>
<p>How to do it?</p>

ptrblck: <p>You could set the new value using a condition on your targets:</p>
<pre><code class="lang-python">dataset = datasets.MNIST(
    root='PATH',
    transform=transforms.ToTensor()
)

dataset.targets[dataset.targets &gt; 5] = 5
print(dataset.targets.unique())
&gt; tensor([0, 1, 2, 3, 4, 5])
</code></pre>

shangeth: <p>AttributeError: ‘MNIST’ object has no attribute ‘targets’</p>

ptrblck: <p>In older <code>torchvision</code> versions, you had to use <code>train_labels</code> or <code>test_labels</code> depending if the <code>train</code> argument was set to <code>True</code> or <code>False</code>, respectively.</p>

tuotuoZ: <p>Hi, I am wondering is there a way to access the targets attributes for dataset which is imported by ImageFolder? I have a training set with 6 classes: building, forest, sea, street, glacier, and mountain. I only want to preserve the forest class label and mark the rest to unforest. I tried this:</p>
<p>dataset.targets[dataset.targets != 1] = 0</p>
<p>which didn’t work. Because it said it doesn’t have attribue targets</p>

ptrblck: <p>Maybe you are using an older version.<br>
Could you update <code>torchvision</code> and check that attribute again?<br>
Also note that <code>dataset.targets</code> is a Python list in <code>ImageFolder</code>, so this indexing won’t work and you should cast it to a <code>tensor</code> before:</p>
<pre><code class="lang-python">dataset = datasets.ImageFolder(root='PATH')
dataset.targets = torch.tensor(dataset.targets)
dataset.targets[dataset.targets==0] = 1
</code></pre>

tuotuoZ: <p>Thanks for replying to me. I used conda update torchvision and my torchvision version is 0.2.1 now. It’s still not working. Is that the latest?</p>

ptrblck: <p>Could be. I’m usually just install <code>torchvision</code> from source, as it is really easy and gives you all the new features.<br>
You would have to clone the repo and just run <code>python setup.py install</code> as described <a href="https://github.com/pytorch/vision#installation" rel="nofollow noopener">here</a>.</p>

tuotuoZ: <p>Thanks! I installed from source and it’s working now! Any idea why the pip and conda distributions don’t have 0.2.3 right now?</p>

tuotuoZ: <p>Hi，I have a follow-up question on that. I successfully convert the target’s attributes. But when I load the data with dataloader it still preserves the original targets.</p>
<pre><code class="lang-auto">train_set_two = datasets.ImageFolder(train_dir, transform=transform)
train_set_two.targets = torch.tensor(train_set_two.targets)
test_set_two.targets = torch.tensor(test_set_two.targets)
train_set_two.targets[train_set_two.targets &gt; 1] = 1
test_set_two.targets[test_set_two.targets &gt; 1] = 1

# prepare data loaders (combine dataset and sampler)
train_loader = torch.utils.data.DataLoader(train_set_two, batch_size=batch_size,
    sampler=train_sampler, num_workers=num_workers)
</code></pre>
<p>After this chunk of codes, I run visualized my train_loader, which still has 6 classes instead of 2. Any idea?</p>

ptrblck: <p>Oh, right. Internally, <code>ImageFolder</code> seems to call <code>dataset.samples</code>, so you could try the following code:</p>
<pre><code class="lang-python">train_set_two.samples = [(d, 1) if s &gt; 1 else (d, s) for d, s in train_set_two.samples]
</code></pre>

Sam_Mertens: <p>Hi. I have a customized dataset. the extension is .pth. How do I select the labels from the dataset?</p>

ptrblck: <p>How did you store the dataset and what does the <code>pth</code> file contain?</p>

Arthur_Conmy: <p>Hi, this was really useful to me changing the labels in EMNIST from 1-26 to 0-25.</p>
<p>However, do you have a link to <code>.targets</code> in the pytotch documentation? The word ‘target’ appears so much there that I can’t search for it successfully.</p>

ptrblck: <p>I assume you are referring to the first code snippet using the <code>MNIST</code> dataset?<br>
If so, you can find the <code>targets</code> definition <a href="https://github.com/pytorch/vision/blob/03b1d38ba3c67703e648fb067570eeb1a1e61265/torchvision/datasets/mnist.py#L80">here</a>.</p>
<p>Let me know, if I misunderstood the question.</p>

kiasari: <pre><code class="lang-auto">TypeError: '&gt;' not supported between instances of 'list' and 'int'
</code></pre>
<p>My code:</p>
<pre><code class="lang-auto">trainset = torchvision.datasets.CIFAR10(root='./data', train=True, download=True)
trainset.targets[trainset.targets &gt; 5] = 5
</code></pre>

ptrblck: <p>You could either try to transform the <code>targets</code> <code>list</code> to a tensor via:</p>
<pre><code class="lang-python">trainset.targets = torch.tensor(trainset.targets)
</code></pre>
<p>and check if this would break anything else or you could create a custom <code>Dataset</code> and manipulate the targets explicitly internally.</p>

apachetechnology: <p>I am also trying something similar.</p>
<pre><code class="lang-auto">import os
import matplotlib.pyplot as plt
import numpy as np

import torch
import torchvision.datasets as datasets
import torchvision.transforms as transforms
from torch.utils.data import Subset, DataLoader

def LoadCIFAR10_py():
     strDirPath = os.getcwd() + "\\DATA\\"
    aTrainData = datasets.CIFAR10(strDirPath, train=True, download=True)
    aTestData = datasets.CIFAR10(strDirPath, train=False, download=True)
    print("DATASET: Train=", len(aTrainData), " Test=", len(aTestData))
    #print(type(aTrainData), aTrainData)
    #print(aTestData[0])
    
    return aTrainData, aTestData

def CreateBinarySubset(aData, nLabel1, nLabel2):
    # We create a tensor that has `True` at an index if the sample belongs to class 1
    idxLabel1 = torch.tensor(aData.targets) == nLabel1
    # Similarly, this tensor has `True` at an index if the sample belongs to class 8
    idxLabel2 = torch.tensor(aData.targets) == nLabel2
    # print(idxLabel1.shape, idxLabel2.shape)

    # Merge these two so that we have one Boolean tensor that has True at the index 
    # where the sample is of class Auto or Truck, and False otherwise.
    index_mask = idxLabel1 | idxLabel2
    data_indices = index_mask.nonzero().reshape(-1)
    oDataSubset = Subset(aData, data_indices)
    oDataloader = DataLoader(oDataSubset, shuffle=False, batch_size=8, num_workers=2)
    return oDataloader, oDataSubset 

if __name__ == "__main__":
    ## MAIN
    aTrainData, aTestData = LoadCIFAR10_py()

    ### Here is extracted labels 1 and 9
    oTrainDataloader, oTrainDataSubset = CreateBinarySubset(aTrainData, 1, 9)
    print('Training subset: ', len(oTrainDataSubset))
    
     oTestDataloader, oTestDataSubset = CreateBinarySubset(aTestData, 1, 9)
     print('Testing subset: ', len(oTestDataSubset))

     # Problem:
     # Now i want to change the label 9 to label 0 
     # oTestDataSubset does not have target

     #this is not working
     aTrainData.targets[aTrainData.targets == 9] = 10 ## not working
</code></pre>
<p>please advice.</p>

ptrblck: <p>You can iterate the targets and manipulate them:</p>
<pre data-code-wrap="python"><code class="lang-python">for idx, target in enumerate(oTestDataSubset.dataset.targets):
    if target == 9:
        oTestDataSubset.dataset.targets[idx] = 0
    
torch.tensor(oTestDataSubset.dataset.targets).unique(return_counts=True)
# (tensor([0, 1, 2, 3, 4, 5, 6, 7, 8]),
#  tensor([2000, 1000, 1000, 1000, 1000, 1000, 1000, 1000, 1000]))
</code></pre>
<p>Also, your code is quite hard to read as you haven’t formatted it. You can post code snippets by wrapping them into three backticks ```, which would make it easier to debug your issue.</p>

apachetechnology: <p>Thanks I will try this.</p>