How to use one class of number in MNIST

wxystudio: <p>Hello I’m study the MNIST and want to train a model with only number “1”, but I don’t know how to extract the “1” class out of the total dataset… I only know the code:<br>
train_loader = torch.utils.data.DataLoader(dataset=train_dataset,  batch_size=batch_size, shuffle=True)</p>
<p>Thanks</p>

ptrblck: <p>You could get the indices for all class1 labels and then index the labels and data:</p>
<pre><code class="lang-python">dataset = datasets.MNIST(root='./data')
idx = dataset.train_labels==1
dataset.train_labels = dataset.train_labels[idx]
dataset.train_data = dataset.train_data[idx]
</code></pre>
<p>However, your model won’t learn anything as you just have one class.<br>
Could you explain your use case a bit?<br>
I would at least try to keep two classes in the dataset.</p>
<p>Or do you want to train your model to recognize the number 1 as the valid class and all remaining numbers as false?<br>
If so, you could try this code:</p>
<pre><code class="lang-python">idx = dataset.train_labels != 1
dataset.train_labels[idx] = 0
</code></pre>

wxystudio: <p>Thank you! I think I actually want to train more than one class of number, it’s a typo</p>

anima: <p>says “AttributeError: can’t set attribute.” on dataset.train_labels = dataset.train_labels[idx] line. What is wrong?</p>

ptrblck: <p>Could you try to access now <code>dataset.data</code> and <code>dataset.targets</code>?<br>
I think the attribute was renamed in recent <code>torchvision</code> versions.</p>

anima: <p>thank you, it worked.</p>

Anish_Batra: <p>This should work. Change the <strong>if</strong> condition in my_collate according to your needs.</p>
<pre><code class="lang-auto">def my_collate(batch):
    modified_batch = []
    for item in batch:
        image, label = item
        if label == 1:
            modified_batch.append(item)
    return default_collate(modified_batch)

train_loader = torch.utils.data.DataLoader(
    datasets.MNIST('data', train=True, download=True, transform=transform),
    batch_size=batch_size, shuffle=True, collate_fn = my_collate)</code></pre>

Pranaymodukuru: <p>I tried using the same on CIFAR10, but it does not work.</p>
<pre><code class="lang-auto">idx = dataset.targets ==1
</code></pre>
<p>This part gives only one bool value ‘true’ or ‘false’ as output, and not a list.<br>
Is it different for different Datasets or is there is a standard way of doing it?</p>

ptrblck: <p>It is most likely different.<br>
For CIFAR10, you could wrap the <code>targets</code> in a <code>tensor</code> and use the comparison:</p>
<pre><code class="lang-python">dataset = datasets.CIFAR10(root='./data')
idx = torch.tensor(dataset.targets)==1
</code></pre>

Pranaymodukuru: <p>I tried this,</p>
<pre><code class="lang-auto">train_dataset = datasets.CIFAR10(root='./data')
idx = torch.tensor(train_dataset.targets) == 1
dataset.data = dataset.data[idx]
dataset.targets = dataset.targets[idx]
</code></pre>
<p>I am getting this error</p>
<pre><code class="lang-auto">=&gt;              dataset.targets = dataset.targets[idx]

TypeError: only integer tensors of a single element can be converted to an index
</code></pre>

ptrblck: <p>This should work:</p>
<pre><code class="lang-python">dataset = datasets.CIFAR10(root='./data',
                           transform=transforms.ToTensor())
dataset.targets = torch.tensor(dataset.targets)
idx = dataset.targets==1
dataset.targets= dataset.targets[idx]
dataset.data = dataset.data[idx.numpy().astype(np.bool)]

for data, target in dataset:
    print(data.sum())
    print(target)
</code></pre>
<p>However, I would consider these approaches as hacks and there might be some side effects I’m currently not aware of.<br>
The clean approach would be to override the dataset class and manipulate the underlying data as you wish.</p>

Pranaymodukuru: <p>Thank you, your solution works for me.</p>
<p>I also found another method here:<br>
<aside class="onebox stackexchange">
  <header class="source">
      <a href="https://stackoverflow.com/questions/51334858/pytorch-dataloader-choose-class-stl10-dataset/51336873#51336873" target="_blank" rel="nofollow noopener">stackoverflow.com</a>
  </header>
  <article class="onebox-body">
      <a href="https://stackoverflow.com/users/4892874/iacolippo" target="_blank" rel="nofollow noopener">
    <img alt="iacolippo" src="https://i.stack.imgur.com/in64F.jpg?s=128&amp;g=1" class="thumbnail onebox-avatar" width="128" height="128">
  </a>
<h4>
  <a href="https://stackoverflow.com/questions/51334858/pytorch-dataloader-choose-class-stl10-dataset/51336873#51336873" target="_blank" rel="nofollow noopener">Pytorch DataLoader - Choose Class STL10 Dataset</a>
</h4>

<div class="tags">
  <strong>pytorch, torch, torchvision</strong>
</div>

<div class="date">
  
  answered by
  <a href="https://stackoverflow.com/users/4892874/iacolippo" target="_blank" rel="nofollow noopener">
    iacolippo
  </a>
  on <a href="https://stackoverflow.com/questions/51334858/pytorch-dataloader-choose-class-stl10-dataset/51336873#51336873" target="_blank" rel="nofollow noopener">08:52AM - 14 Jul 18 UTC</a>
</div>

  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>
</p>
<p>I also included an example implementation here:</p>
<pre><code class="lang-auto">def get_indices(dataset,class_name):
    indices =  []
    for i in range(len(dataset.targets)):
        if dataset.targets[i] == class_name:
            indices.append(i)
    return indices


dataset = torchvision.datasets.CIFAR10(root='./data',
                           transform=torchvision.transforms.ToTensor())

idx = get_indices(dataset, 1)
loader = Data.DataLoader(dataset,batch_size=64, sampler = Data.sampler.SubsetRandomSampler(idx))

for idx, (data, target) in enumerate(loader):
    print(target)
</code></pre>
<p>I hope this works for all the datasets available in torchvision.datasets.</p>

ptrblck: <p>That looks like a cleaner approach so I would recommend to stick to the <code>SubsetRandomSampler</code>. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Sourena_Yadegari: <p>Creating a dataset as a subset and then define data loaders. Assuming you only want labels 1 and 2</p>
<pre><code class="lang-auto">idx = torch.tensor(CIFAR10_train.targets) == 1
idx += torch.tensor(CIFAR10_train.targets) == 2
dset_train = torch.utils.data.dataset.Subset(CIFAR100_train, np.where(idx==1)[0])

idx = torch.tensor(CIFAR10_test.targets) == 1
idx += torch.tensor(CIFAR10_test.targets) == 2
dset_test = torch.utils.data.dataset.Subset(CIFAR100_test, np.where(idx==1)[0])
</code></pre>
<p>and dataloaders are defined as usual:</p>
<pre><code class="lang-auto">dl_train = torch.utils.data.DataLoader(dset_train, batch_size=8, shuffle=True)
dl_test = torch.utils.data.DataLoader(dset_test, batch_size=8, shuffle=True)
</code></pre>

Pranaymodukuru: <p>This works too and seems to be simpler than <code>SubsetRandomSampler</code>.</p>

ani0075: <p><a class="mention" href="/u/sourena_yadegari">@Sourena_Yadegari</a><br>
Isn’t it also necessary to reindex the data to do something useful with this dataset?</p>
<p>For example the CrossEntropy criterion expects a class index in the range [0, C-1].<br>
So, in this case 1,2 have to be reindexed as 0,1 otherwise the criterion will throw an error.</p>

Pranaymodukuru: <p>Until the number of outputs from the network/model are 2 [since you are training on 2 classes], you will not face any issues.</p>
<p>Also if you have 10 outputs for the network and you are only training on 2 classes, this will not have an issue too.</p>
<aside class="quote no-group" data-username="ani0075" data-post="16" data-topic="26276">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ani0075/48/19643_2.png" class="avatar"> ani0075:</div>
<blockquote>
<p>CrossEntropy criterion expects a class index in the range [0, C-1]</p>
</blockquote>
</aside>
<p>This condition is violated only when your data set has for example “10 classes” and your network is having “2 outputs”, but not the other way around.</p>

iman.mirzadeh: <aside class="quote no-group" data-username="Sourena_Yadegari" data-post="14" data-topic="26276">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sourena_yadegari/48/14001_2.png" class="avatar"> Sourena_Yadegari:</div>
<blockquote>
<p>torch.utils.data.DataLoader</p>
</blockquote>
</aside>
<p>I used this approach.<br>
Here’s my implementation for split-cifar100 (a common continual learning benchmark):</p>
<pre><code class="lang-auto">def get_split_cifar100(task_id, batch_size=32, shuffle=False):
	# convention: tasks starts from 1 not 0 !
	# task_id = 1 (i.e., first task) =&gt; start_class = 0, end_class = 4
	start_class = (task_id-1)*5
	end_class = task_id * 5

	transforms = torchvision.transforms.Compose([
		torchvision.transforms.ToTensor(),
		torchvision.transforms.Normalize((0.4914, 0.4822, 0.4465), (0.2023, 0.1994, 0.2010)),
		])

	train = torchvision.datasets.CIFAR100('./data/', train=True, download=True, transform=transforms)
	test = torchvision.datasets.CIFAR100('./data/', train=False, download=True, transform=transforms)
	
	targets_train = torch.tensor(train.targets)
	target_train_idx = ((targets_train &gt;= start_class) &amp; (targets_train &lt; end_class))

	targets_test = torch.tensor(test.targets)
	target_test_idx = ((targets_test &gt;= start_class) &amp; (targets_test &lt; end_class))

	train_loader = torch.utils.data.DataLoader(torch.utils.data.dataset.Subset(train, np.where(target_train_idx==1)[0]), batch_size=batch_size)
	test_loader = torch.utils.data.DataLoader(torch.utils.data.dataset.Subset(test, np.where(target_test_idx==1)[0]), batch_size=batch_size)

	return train_loader, test_loader
</code></pre>

kawsheham: <p>I want to create a sepearte dataloader ofr several subtasks. Each consisting identification of two digits? [0,1],[2,3],[4,5],[6,7],[8,9]? How can I do this?</p>

ptrblck: <p>You could use the same approach from <a href="https://discuss.pytorch.org/t/how-to-use-one-class-of-number-in-mnist/26276/2">this post</a> and combine the conditions for the two wanted classes to create <code>idx</code>. The easiest approach would be to create 5 <code>MNIST</code> datasets and apply the conditions on each separately.</p>
<p>E.g. for the first <code>DataLoader</code>, this code should work:</p>
<pre><code class="lang-python">dataset = datasets.MNIST(root='./data')
idx = (dataset.targets==0) | (dataset.targets==1)
dataset.targets = dataset.targets[idx]
dataset.data = dataset.data[idx]
</code></pre>