Pytorch model not converging

Dace: <p>Hi, I am new to pytorch and deep learning.</p>
<p>The loss of my model was almost static throughout the training p</p>
<p>Model class</p>
<pre><code class="lang-auto">class Model(nn.Module):
	def __init__(self):
		super().__init__()
		self.conv1 = nn.Conv2d(3,32,3)
		self.conv2 = nn.Conv2d(32,64,3)
		self.conv3 = nn.Conv2d(64,128,3)
		self.fc1 = nn.Linear(128*2*2,100)
		self.fc2 = nn.Linear(100,10)
	def forward(self,x):
		x = self.conv1(x)
		x = F.relu(x)
		x = F.max_pool2d(x,(2,2))#32x15x15
		
		x = self.conv2(x)
		x = F.relu(x)
		x = F.max_pool2d(x,(2,2))#64x6x6

		x = self.conv3(x)
		x = F.relu(x)
		x = F.max_pool2d(x,(2,2))#128x2x2
		x = x.view(-1,128*2*2)
		x = self.fc1(x)
		x = F.relu(x)
		x = self.fc2(x)
		x = F.softmax(x,dim=1)
		return x
</code></pre>
<p>I was training the model on the cifar10 dataset.</p>
<p>train.py</p>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader
import torchvision.transforms as transforms
from cifarloader import cifar10
from model import Model


t = transforms.Compose(
		[
			lambda x:torch.Tensor(x),
			lambda x:x/255,
			transforms.Normalize((0.475,0.475,0.475),(0.253,0.253,0.253))

		]
)


cifar10 = cifar10(transforms=t)
dataloader = DataLoader(cifar10,batch_size=20,shuffle=True)
net = Model()


loss_fn = nn.MSELoss()
optim = optim.Adam(net.parameters(),lr=0.003)


epochs = 100

for e in range(epochs):
	tloss = 0
	for b in dataloader:

		x,y = b
		optim.zero_grad()
		p = net(x)
		loss = loss_fn(p,y)
		tloss+=loss*20
		loss.backward()
		optim.step()
	if e%10 == 0:
		print(f'Epoch: {e}, Loss{tloss/20}')
		torch.save(net,f'e-{e}-l-{tloss}.pth')
</code></pre>
<p>Was there something i did wrong, It would be cool if someone could help me out</p>

pascal_notsawo: <aside class="quote no-group" data-username="Dace" data-post="1" data-topic="107363">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dace/48/32820_2.png" class="avatar"> Dace:</div>
<blockquote>
<pre><code class="lang-auto">import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader
import torchvision.transforms as transforms
from cifarloader import cifar10
from model import Model


t = transforms.Compose(
		[
			lambda x:torch.Tensor(x),
			lambda x:x/255,
			transforms.Normalize((0.475,0.475,0.475),(0.253,0.253,0.253))

		]
)


cifar10 = cifar10(transforms=t)
dataloader = DataLoader(cifar10,batch_size=20,shuffle=True)
net = Model()


loss_fn = nn.MSELoss()
optim = optim.Adam(net.parameters(),lr=0.003)


epochs = 100

for e in range(epochs):
	tloss = 0
	for b in dataloader:

		x,y = b
		optim.zero_grad()
		p = net(x)
		loss = loss_fn(p,y)
		tloss+=loss*20
		loss.backward()
		optim.step()
	if e%10 == 0:
		print(f'Epoch: {e}, Loss{tloss/20}')
		torch.save(net,f'e-{e}-l-{tloss}.pth')
</code></pre>
</blockquote>
</aside>
<p>I think I saw the cause of the problem.<br>
However, I had this problem with your code: No module named <code>cifarloader</code>.</p>
<p>Is it a python module (if so how do I install it) or one of your scripts (if so, I need it)?</p>

Dace: <p>It’s one of my scripts</p>
<p>here’s the code</p>
<pre><code class="lang-auto">import torch
from torch.utils.data import Dataset
from data import data,labels
class cifar10(Dataset):
	def __init__(self,transforms):
		self.x = data
		self.y = labels
		self.transforms = transforms
	def __getitem__(self,idx):
		x = self.x[idx].transpose(1,2,0)
		y = self.y[idx]
		if self.transforms:
			x = self.transforms(x)
		return x,y
	def __len__(self):
		return len(self.x)

</code></pre>

pascal_notsawo: <aside class="quote no-group" data-username="Dace" data-post="3" data-topic="107363">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dace/48/32820_2.png" class="avatar"> Dace:</div>
<blockquote>
<p><code>from data import data,labels</code></p>
</blockquote>
</aside>
<p>The line <code>from data import data,labels</code>, data is which module?</p>
<p>Help me to help you, by giving me a complete code that I can execute without any problem.</p>

pascal_notsawo: <aside class="quote no-group" data-username="Dace" data-post="1" data-topic="107363">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/dace/48/32820_2.png" class="avatar"> Dace:</div>
<blockquote>
<p><code>loss_fn = nn.MSELoss()</code></p>
</blockquote>
</aside>
<p>Mean Squared Error is not a good loss function for classification (<a href="https://towardsdatascience.com/why-using-mean-squared-error-mse-cost-function-for-binary-classification-is-a-bad-idea-933089e90df7#:~:text=Nov%2014%2C%202019%C2%B73%20min%20read&amp;text=There%20are%20two%20reasons%20why,choice%20for%20binary%20classification%20problems%3A&amp;text=When%20the%20MSE%20function%20is,the%20target%20value%20(y)." rel="noopener nofollow ugc">Why MSE Cost Function for Binary Classification is a Bad Idea?</a>)</p>
<p>Replace <code>loss_fn = nn.MSELoss()</code> by <code>loss_fn = nn.CrossEntropyLoss()</code> and let me know if it solved your problem.</p>

Dace: <p>thanks a lot. I changed the learning rate to 0.0001 and the loss function to cross Entropy. And yeah mse is best suited for regression problems</p>

pascal_notsawo: <p>Your problem has been solved?</p>