How to use own data on 'Training a Classifier'

username: Fred1308
cooked: <p>Dear PyTorch-Community!</p>
<p>I’m very new to the programming using python and from time to time I feel too dumb to understand it. I got I really basic problem right now, but there’s someone out there to help, for sure</p>
<p>Most of you might know the ‘training a classifier’ algorythm from pytorch (<a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html" class="inline-onebox" rel="noopener nofollow ugc">Training a Classifier — PyTorch Tutorials 1.8.1+cu102 documentation</a>). I’d like to use it on my own dataset of cow-pictures.<br>
Problem: How do I get my own data instead of the cifar10 dataset in there? I tried a million things but none worked…</p>
<p>Thanks in advance!</p>

username: ptrblck
cooked: <p>What have you tried already and where are you stuck at the moment?<br>
Knowing about the failed attempts would be helpful so that we could point you towards potential issues.<br>
In the meantime you could take a look at <a href="https://pytorch.org/tutorials/beginner/data_loading_tutorial.html">this tutorial</a>, which explains how to write a custom <code>Dataset</code> and use a <code>DataLoader</code>.</p>

username: Fred1308
cooked: <p>Dear ptrblck,<br>
Thank you so much for your response!</p>
<p>So far I annotated my pictures using CVAT and also got a csv-file with those annotations.<br>
The tutorial is giving me this code, where I poorly tried to modify the input to be my dataset:</p>
<pre><code class="lang-auto">transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

batch_size = 4

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=batch_size,
                                          shuffle=True, num_workers=2)

testset = torchvision.datasets.CIFAR10(root='./data', train=False,
                                       download=True, transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=batch_size,
                                         shuffle=False, num_workers=2)

classes = ('plane', 'car', 'bird', 'cat',
           'deer', 'dog', 'frog', 'horse', 'ship', 'truck')
</code></pre>
<p>I tried to replace the “<code>tochvision.datasets.CIFAR10(root='./data', train=True, download=True, transform=transform)</code>” part with the root to my dataset and changed my classes to the two relevant for me: “cow”, “no_cow”</p>

username: ptrblck
cooked: <p>I wouldn’t try to reuse another <code>Dataset</code> if your custom data doesn’t match the layout, stored file format etc. Take a look at the creation of the <code>FaceLandmarkDataset</code> in the linked tutorial.<br>
The basic idea is to load all paths, store the transformations etc. in the <code>__init__</code>, return the number of samples in the <code>__len__</code>, and load, process, as well as return each sample and target in the <code>__getitem__</code>.<br>
The common approach would be to store the image paths in the <code>__init__</code> with the corresponding targets and load each sample using the <code>index</code> in the <code>__getitem__</code>.</p>

username: Fred1308
cooked: <p>Alright, sounds reasonable</p>
<p>To be sure: If I define the dataset as described in the <code>FaceLandmarkDataset</code> tutorial I should be able to load it in this given code and use it?</p>

username: ptrblck
cooked: <p>Yes, you should be able to download the mentioned data and use it with the posted <code>FaceLandmarkDataset</code>. The tutorial is also executable on Google Colab.<br>
For your custom <code>Dataset</code> you would of course need to change the paths and adapt the data structure etc.</p>