Creating a DataLoader for unsupervised learning (MNIST, SVHN)

Nimrod_Daniel: <p>I need to solve an unsupervised problem with images from MNIST and SVHN, in which I have 100 images from MNIST and 10 images from SVHN).  I need a pre-trained net to learn how to classify if a given image is from MNIST or SVHN (the anomaly). Basically, it’s an anomaly detection problem.<br>
I know I’ll have to tackle that later with integration of a clustering technique/SVM/whatever with the net somehow, or AE, and a proper loss, but first things first <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>
<p>It’s very easy to create  a regular Dataloader for MNIST and SVHN in PyTorch, but in this case I need to create a dataloader that includes a small subset of the two datasets without the labels (that’s the more interesting part here:) ). How can I do that?</p>

ptrblck: <p>I think the easiest approach would be to write a custom <code>Dataset</code>, and load the desired samples inside of <code>__init__</code>.<br>
Since the data format is different for MNIST and CIFAR (size and number of channels), you would also need to specify some dataset-specific transformations.<br>
Here is a small sample code, which could be a good starter:</p>
<pre><code class="lang-python">class MyDataset(Dataset):
    def __init__(self, mnist_transform=None, cifar_transform=None):
        mnist = datasets.MNIST(
            root='./data',
        )
        cifar = datasets.CIFAR10(
            root='./data',
        )
        
        self.mnist_len = 100
        self.cifar_len = 10
        
        rand_idx = torch.randperm(len(mnist.data))[:self.mnist_len]
        self.mnist_data = mnist.data[rand_idx]
        
        rand_idx = torch.randperm(len(cifar.data))[:self.cifar_len]
        self.cifar_data = cifar.data[rand_idx]

        self.mnist_transform = mnist_transform
        self.cifar_transform = cifar_transform
        
    def __getitem__(self, index):
        if index &lt; self.mnist_len:
            x = self.mnist_data[index]
            if self.mnist_transform:
                x = self.mnist_transform(x)
            print('Returning MNIST sample at index {}'.format(index))
            return x
        else:
            index = index - self.mnist_len
            x = self.cifar_data[index]
            if self.cifar_transform:
                x = self.cifar_transform(x)
            print('Returning CIFAR data at index {}'.format(index))
            return x
        
    def __len__(self):
        return self.mnist_len + self.cifar_len

        

mnist_transform = transforms.Compose([
    transforms.ToPILImage(),
    transforms.Resize((32, 32)),
    transforms.Grayscale(num_output_channels=3),
    transforms.ToTensor(),
    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])
    
cifar_transform = transforms.Compose([
    transforms.ToPILImage(),
    transforms.ToTensor(),
    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])
    
dataset = MyDataset(
    mnist_transform=mnist_transform,
    cifar_transform=cifar_transform
)
</code></pre>

Nimrod_Daniel: <p>Looks  pretty good as a starter.</p>
<p>If I download the .gz  MNIST test images file and SVHN test images, extract them, put them in data/mnist and mnist/svhn/test folders respectively it should create MyDataset as the new dataset with 100 images from MNIST and the next 10 images from svhn, right?</p>
<p>I switched  cifar with svhn, and resized the svhn to 32 as well (I’ll probably use a net that was trained on cifar). Any other changes are required? (code below)</p>
<p>Currently it appears like it can’t find the dataset<br>
RuntimeError: Dataset not found. You can use download=True to download it</p>
<p>import torch</p>
<p>import torchvision.transforms as transforms<br>
import torchvision.datasets as datasets<br>
from torch.utils.data import DataLoader, Dataset</p>
<p>class MyDataset(Dataset):<br>
def <strong>init</strong>(self, mnist_transform=None, svhn_transform=None):<br>
mnist = datasets.MNIST(<br>
root=’./data/mnist’,<br>
)<br>
svhn = datasets.SVHN(<br>
root=’./data/svhn/test’,<br>
)</p>
<pre><code>    self.mnist_len = 100
    self.svhn_len = 10
    
    rand_idx = torch.randperm(len(mnist.data))[:self.mnist_len]
    self.mnist_data = mnist.data[rand_idx]
    
    rand_idx = torch.randperm(len(svhn.data))[:self.svhn_len]
    self.svhn_data = svhn.data[rand_idx]

    self.mnist_transform = mnist_transform
    self.svhn_transform = svhn_transform
    
def __getitem__(self, index):
    if index &lt; self.mnist_len:
        x = self.mnist_data[index]
        if self.mnist_transform:
            x = self.mnist_transform(x)
        print('Returning MNIST sample at index {}'.format(index))
        return x
    else:
        index = index - self.mnist_len
        x = self.svhn_data[index]
        if self.svhn_transform:
            x = self.svhn_transform(x)
        print('Returning SVHN data at index {}'.format(index))
        return x
    
def __len__(self):
    return self.mnist_len + self.svhn_len
</code></pre>
<p>mnist_transform = transforms.Compose([<br>
transforms.ToPILImage(),<br>
transforms.Resize((32, 32)),<br>
transforms.Grayscale(num_output_channels=3),<br>
transforms.ToTensor(),<br>
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))<br>
])</p>
<p>svhn_transform = transforms.Compose([<br>
transforms.Resize((32, 32)),<br>
transforms.ToPILImage(),<br>
transforms.ToTensor(),<br>
transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))<br>
])</p>
<p>dataset = MyDataset(<br>
mnist_transform=mnist_transform,<br>
svhn_transform=svhn_transform<br>
)</p>

ptrblck: <p>I wouldn’t recommend to download the datasets manually. Instead just pass <code>download=True</code> as the error message says and the files should be automatically downloaded and extracted.</p>
<p><code>MyDataset</code> will use 100 random MNIST images and 10 random SVHN images.<br>
The indices are defined in <code>rand_idx</code>. You can of course change these indices to specific ones.</p>

Nimrod_Daniel: <p>Yes, it’s better to do it the regular way with download = True.</p>
<p>It looks like it doesn’t recognize MNIST.data</p>
<pre><code>rand_idx = torch.randperm(len(mnist.data))[:self.mnist_len]

AttributeError: 'MNIST' object has no attribute 'data'
</code></pre>
<p>if I add # before that line (and the next one) it works well. It’s a bit odd it says it doesn’t have that attribute.</p>

ptrblck: <p>In older <code>torchvision</code> versions, <code>self.train_data</code> and <code>self.test_data</code> was used, so you might want to update <code>torchvision</code> or use the corresponding attribute.</p>

Nimrod_Daniel: <p>Good to know. I prefer to stay with my current version of packages  for the meanwhile.<br>
mnist.test_data doesn’t work if that’s what you meant.</p>

ptrblck: <p>Depending if you’ve specified <code>train=True/False</code> during instantiation of your dataset, only the corresponding one will work.</p>

Nimrod_Daniel: <p>True, I forgot to specify that I want the test.<br>
Thank you <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

Nimrod_Daniel: <p>I have a trained network plus a feature extractor that returns the output of the last conv layer, but when I try to simply pass the dataset through the model by<br>
output =  model(dataset)</p>
<p>I get this error:<br>
TypeError: conv2d(): argument ‘input’ (position 1) must be Tensor, not MyDataset</p>
<p>dataset is MyDataset instance, but I guess the transform should allow me to pass the data  through the net. What am I missing here?</p>

ptrblck: <p>You cannot pass the <code>Dataset</code> instance directly to your model, but would have to pass data batches to it.<br>
The recommended way to do this is to wrap your dataset in a <code>DataLoader</code> and iterate over it:</p>
<pre><code class="lang-python">dataset = MyDataset(...)
loader = DataLoader(
    dataset,
    batch_size=1,
    shuffle=True,
    num_workers=2
)

for data in loader:
    output = model(data)
    ...
</code></pre>
<p>Note that I just set some arguments while creating the <code>DataLoader</code>, so you might want to change e.g. the batch size etc.</p>

Nimrod_Daniel: <p>Thanks, it’s more clear now. But it seems like getitem is not defined properly, it fails to access self.MNIST, as MyDataset has no MNIST attribute.</p>
<blockquote>
<p>if index &lt; self.MNIST:</p>
</blockquote>
<p>AttributeError: Traceback (most recent call last):<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 57, in _worker_loop<br>
samples = collate_fn([dataset[i] for i in batch_indices])<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 57, in <br>
samples = collate_fn([dataset[i] for i in batch_indices])<br>
File “”, line 27, in <strong>getitem</strong><br>
if index &lt; self.MNIST:<br>
AttributeError: ‘MyDataset’ object has no attribute ‘MNIST’</p>
<p>I tried to replace self.MNIST with self.mnist_len but it didn’t help, I get that MNIST object is not callable:<br>
TypeError: Traceback (most recent call last):<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 57, in _worker_loop<br>
samples = collate_fn([dataset[i] for i in batch_indices])<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 57, in <br>
samples = collate_fn([dataset[i] for i in batch_indices])<br>
File “”, line 30, in <strong>getitem</strong><br>
x = self.mnist_transform(x)<br>
TypeError: ‘MNIST’ object is not callable</p>

ptrblck: <p>My code should work, if you just change the <code>mnist.data</code> attribute for <code>mnist.test_data</code>, if you are using the test set in an older <code>torchvision</code> version.<br>
What else did you change? Could you compare your code to mine?</p>

Nimrod_Daniel: <p>Exactly the same code:</p>
<pre><code class="lang-python">class MyDataset(Dataset):
    def __init__(self, mnist_transform=None, svhn_transform=None):
        mnist = datasets.MNIST(
            root='./data/mnist',
            train=False,
            download= True
        )
                
        svhn = datasets.SVHN(
            root='./data/svhn',
            download=True
        )
        
        self.mnist_len = 100
        self.svhn_len = 10
        
        rand_idx = torch.randperm(len(mnist.test_data))[:self.mnist_len]
        self.mnist_data = mnist.test_data[rand_idx]
        
        rand_idx = torch.randperm(len(svhn.data))[:self.svhn_len]
        self.svhn_data = svhn.data[rand_idx]

        self.mnist_transform = mnist
        self.svhn_transform = svhn_transform
        
    def __getitem__(self, index):
        if index &lt; self.mnist_len:
            x = self.mnist_data[index]
            if self.mnist_transform:
                x = self.mnist_transform(x)
            print('Returning MNIST sample at index {}'.format(index))
            return x
        else:
            index = index - self.mnist_len
            x = self.svhn_data[index]
            if self.svhn_transform:
                x = self.svhn_transform(x)
            print('Returning SVHN data at index {}'.format(index))
            return x
        
    def __len__(self):
        return self.mnist_len + self.svhn_len

        

mnist_transform = transforms.Compose([
    transforms.ToPILImage(),
    transforms.Resize((32, 32)),
    transforms.Grayscale(num_output_channels=3),
    transforms.ToTensor(),
    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])
    
svhn_transform = transforms.Compose([
    transforms.Resize((32, 32)),
    transforms.ToPILImage(),
    transforms.ToTensor(),
    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))
])
    
dataset_new = MyDataset(
    mnist_transform=mnist_transform,
    svhn_transform=svhn_transform
)


loader = DataLoader(
    dataset_new,
    batch_size=100,
    shuffle=True,
    num_workers=4
)

for data in loader:
    output = model2(data)
    print(output)
</code></pre>
<p>TypeError: ‘MNIST’ object is not callable</p>

ptrblck: <p>Apparently, these lines were changed:</p>
<pre><code class="lang-python">if index &lt; self.MNIST:
# should be
if index &lt; self.mnist_len:

self.mnist_transform = mnist
# should be
self.mnist_transform = mnist_transform
</code></pre>

Nimrod_Daniel: <p>Yeah, I missed that. I probably accidentally changed that.<br>
But unfortunately now it says that it gets .format(type(pic)) rather than a Tensor or ndarray. Sounds like something is wrong with the transform, but it looks ok. The last two lines in the trace are quite odd, it looks like they contradict each other (probably a mistake, I guess):</p>
<blockquote>
<p>raise TypeError(‘pic should be Tensor or ndarray. Got {}.’.format(type(pic)))</p>
</blockquote>
<blockquote>
<p>TypeError: pic should be Tensor or ndarray. Got &lt;class ‘torch.Tensor’&gt;.</p>
</blockquote>
<p>The full trace:<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 307, in _process_next_batch<br>
raise batch.exc_type(batch.exc_msg)</p>
<p>TypeError: Traceback (most recent call last):<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 57, in _worker_loop<br>
samples = collate_fn([dataset[i] for i in batch_indices])<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 57, in <br>
samples = collate_fn([dataset[i] for i in batch_indices])<br>
File “”, line 30, in <strong>getitem</strong><br>
x = self.mnist_transform(x)<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torchvision/transforms/transforms.py”, line 49, in <strong>call</strong><br>
img = t(img)<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torchvision/transforms/transforms.py”, line 110, in <strong>call</strong><br>
return F.to_pil_image(pic, self.mode)<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torchvision/transforms/functional.py”, line 103, in to_pil_image<br>
raise TypeError(‘pic should be Tensor or ndarray. Got {}.’.format(type(pic)))<br>
TypeError: pic should be Tensor or ndarray. Got &lt;class ‘torch.Tensor’&gt;.</p>

ptrblck: <p>Could you try to unsqueeze the data sample by using:</p>
<pre><code class="lang-python">x = self.mnist_data[index]
if self.mnist_transform:
    x = self.mnist_transform(x.unsqueeze(0))
</code></pre>
<p>I assume this error is thrown, if the channel dimension in your image tensor is missing (MNIST should be a <code>[28, 28]</code> shaped tensor), which is fixed in the latest <code>torchvision</code> version.</p>

Nimrod_Daniel: <p>It makes sense that it might cause a problem, though     transforms.Grayscale(num_output_channels=3) should deal with channels.<br>
This change results in a quite similar error:</p>
<p>TypeError: Traceback (most recent call last):<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 57, in _worker_loop<br>
samples = collate_fn([dataset[i] for i in batch_indices])<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torch/utils/data/dataloader.py”, line 57, in <br>
samples = collate_fn([dataset[i] for i in batch_indices])<br>
File “”, line 37, in <strong>getitem</strong><br>
x = self.svhn_transform(x)<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torchvision/transforms/transforms.py”, line 49, in <strong>call</strong><br>
img = t(img)<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torchvision/transforms/transforms.py”, line 175, in <strong>call</strong><br>
return F.resize(img, self.size, self.interpolation)<br>
File “/home/nimrod/anaconda3/lib/python3.6/site-packages/torchvision/transforms/functional.py”, line 189, in resize<br>
raise TypeError(‘img should be PIL Image. Got {}’.format(type(img)))<br>
TypeError: img should be PIL Image. Got &lt;class ‘numpy.ndarray’&gt;</p>

ptrblck: <p><code>Grayscale(num_output_channels=3)</code> is used after <code>ToPILImage()</code>, which was throwing this error.<br>
The new issue is for the <code>svhn_transform</code>. Apparently you are using <code>Resize</code> before <code>ToPILImage</code>. Could you change the order and run it again?</p>

Nimrod_Daniel: <p>It works now! Good to know that ToPILImage must precede Resize.</p>
<p>Finally the Dataloader works fine and I can pass the data through the model. Thanks a lot for your patience! <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>