My neural network cannot learning features with my data loading method

username: cat-loves-donuts
cooked: <p>Well, I know this question might be not very suitable to ask here, but I really need some help.</p>
<p>Well, I copy the code from the tutorial of pytorch website:<a href="https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#sphx-glr-beginner-blitz-cifar10-tutorial-py" rel="nofollow noopener">https://pytorch.org/tutorials/beginner/blitz/cifar10_tutorial.html#sphx-glr-beginner-blitz-cifar10-tutorial-py</a></p>
<p>I ran this code and it worked perfectly. Because I have already download the CIFR-10 data set, I decided to load the data directly from my computer. However, I only changed the data loading method and the neural network stopped working. And the most interesting thing is that I used the same data loading method to train another NN to identify the MNIST data set and it worked perfectly. But it did not work on CIFR-10.</p>
<p>And I downloaded the CIFR-10 data set,  found a code to transform those data into .JPG images and classified them into 10 different files with names like “cat”,“airplane” and so on… I do not know why, but this is my code about data loading, I wrote them as 3 functions:</p>
<p>def load_train_dataset():<br>
data_path = ‘C:\Users\…\CIFR-10\train’<br>
train_dataset = torchvision.datasets.ImageFolder(<br>
root=data_path,<br>
transform=torchvision.transforms.ToTensor()<br>
)<br>
train_loader = torch.utils.data.DataLoader(<br>
train_dataset,<br>
batch_size=64,<br>
num_workers=0,<br>
shuffle=True<br>
)<br>
return train_loader</p>
<p>def load_test_dataset():<br>
data_path = ‘C:\Users\…\CIFR-10\test’<br>
test_dataset = torchvision.datasets.ImageFolder(<br>
root=data_path,<br>
transform=torchvision.transforms.ToTensor()<br>
)<br>
test_loader = torch.utils.data.DataLoader(<br>
test_dataset,<br>
batch_size=64,<br>
num_workers=0,<br>
shuffle=False<br>
)<br>
return test_loader</p>
<p>def test_dataset():<br>
data_path = ‘C:\Users\…\CIFR-10\test’<br>
test_dataset = torchvision.datasets.ImageFolder(<br>
root=data_path,<br>
transform=torchvision.transforms.ToTensor()<br>
)<br>
return test_dataset</p>

username: ptrblck
cooked: <p>Do you get an error or what do you mean by “the neural network stopped working”?</p>

username: cat-loves-donuts
cooked: <p>Well thank you so much for helping me.</p>
<p>I mean the loss is not changing. I tried it for couple of times. The loss always around 2.28 to 2.34. And it did not get down. And the test accuracy is 9% to 20%.</p>
<p>I trained for 50000 train data set and test it on 10000 test data set.</p>

username: ptrblck
cooked: <p>Could you try to add the <code>Normalization</code> to your datasets?<br>
Did you change something else besides the origin of the dataset?</p>

username: cat-loves-donuts
cooked: <p>No, I never changed anything in the data set and I checked the data set to see if I mixed the data set or missed so data. But everything is fine.</p>
<p>Well, do you mean BuntchNormal layer? Because I found when I used BuntchNormal layer the results are too perfect to believe in it. Then I decided to remove that layer.</p>

username: ptrblck
cooked: <p>No, I meant the <code>Normalization</code> in the <code>transform</code> from the tutorial:</p>
<pre><code class="lang-python">transform = transforms.Compose(
    [transforms.ToTensor(),
     transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = torchvision.datasets.CIFAR10(root='./data', train=True,
                                        download=True, transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,
                                          shuffle=True, num_workers=2)
</code></pre>

username: cat-loves-donuts
cooked: <p>Well, I tried that. But looks like it does not work:<br>
Train epoch: 100, loss: 2.299</p>
<p>Train epoch: 200, loss: 2.303</p>
<p>Train epoch: 300, loss: 2.299</p>
<p>Train epoch: 400, loss: 2.302<br>
I do not know why…</p>

username: ptrblck
cooked: <p>Where did you download the data from?</p>

username: cat-loves-donuts
cooked: <p>I downloaded it from the website: <a href="https://www.cs.toronto.edu/~kriz/cifar.html" rel="nofollow noopener">https://www.cs.toronto.edu/~kriz/cifar.html</a></p>
<p>And use a code to rewrote them as .JPG file.</p>

username: ptrblck
cooked: <p>Could you post the code or give some information about how you are converting the data to JPG? I would like to check, if there is an error in the processing pipeline.</p>

username: cat-loves-donuts
cooked: <p>Well, I am sorry I deleted that code…<img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=9" title=":joy:" class="emoji" alt=":joy:"><img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=9" title=":joy:" class="emoji" alt=":joy:"><img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=9" title=":joy:" class="emoji" alt=":joy:"><img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=9" title=":joy:" class="emoji" alt=":joy:"><img src="https://discuss.pytorch.org/images/emoji/apple/joy.png?v=9" title=":joy:" class="emoji" alt=":joy:"></p>
<p>But I tried to change the data loading method to the tutorial style and use it on my training code. Unsurprisingly, the NN still cannot learn…</p>
<p>Well I suppose my training code must have some problems…</p>
<p>But I cannot find it…</p>
<p>Could you please help me to check the code?</p>
<p>def train_model(model, learning_rate, nsamples, load_train_dataset):<br>
model.train(True)<br>
epoch = 0<br>
print_loss = 0<br>
criterion = nn.CrossEntropyLoss()<br>
optimizer_module = optim.SGD(model.parameters(), lr=learning_rate)<br>
device = torch.device(“cuda:0” if torch.cuda.is_available() else “cpu”)<br>
model = model.to(device)<br>
for i, (data, target) in enumerate(load_train_dataset):<br>
data = data.to(device)<br>
target = target.to(device)<br>
if i &gt; nsamples:<br>
break<br>
else:<br>
# train network<br>
optimizer_module.zero_grad()<br>
out = model(data)<br>
loss = criterion(out, target)<br>
print_loss = loss.data.item()</p>
<pre><code>        loss.backward()
        optimizer_module.step()
        epoch += 1
        if epoch % 100 == 0:
            print('\n Train epoch: {}, loss: {:.4}'.format(epoch, loss.data.item()))
return print_loss
</code></pre>
<p>And the training result is this:<br>
Train epoch: 100, loss: 2.277</p>
<p>Train epoch: 200, loss: 2.28</p>
<p>Train epoch: 300, loss: 2.306</p>
<p>Train epoch: 400, loss: 2.331</p>
<p>Train epoch: 500, loss: 2.263</p>
<p>Train epoch: 600, loss: 2.284</p>
<p>Train epoch: 700, loss: 2.255</p>