The network does not learn The total loss and the inference result will be the same value every time

username: skerlet_flandorle
cooked: <p>The total loss and the inference result will be the same value every time.</p>
<p>gpu cpu is working normally<br>
detaset CIFAR100<br>
Please tell me if there is not enough information. I will add it.</p>
<p>import torch<br>
import torchvision<br>
import torch.nn as nn<br>
import torch.optim as optim<br>
import torch.nn.functional as F<br>
from torchvision import datasets, transforms<br>
from torch.utils.data import Dataset<br>
import time</p>
<p>class Firttan(nn.Module):<br>
def <strong>init</strong>(self):<br>
super().<strong>init</strong>()<br>
def forward(self, inputs):<br>
inputs=inputs.view(inputs.size(0),-1)<br>
return inputs<br>
class MYoptimizer():<br>
def <strong>init</strong>(self):<br>
self.tortal_losses=0<br>
self.optimizer = torch.optim.SGD(net.parameters(), lr=0.01, momentum=0.9)<br>
self.scheduler = optim.lr_scheduler.StepLR(self.optimizer, step_size=10, gamma=0.9)<br>
self.criterion = torch.nn.CrossEntropyLoss()</p>
<pre><code>def loss(self,inputs, labels):
    self.optimizer.zero_grad()
    outputs=net(inputs)
    loss=self.criterion(outputs, labels)
    loss.backward()
    torch.nn.utils.clip_grad_norm_(net.parameters(),1,2)
    self.optimizer.step()
    self.tortal_losses+=loss.to('cpu').item()
def Done(self,epoc,timee):
    print("Epoch",epoc,"total_loss",self.tortal_losses,timee)
    self.tortal_losses=0
    self.scheduler.step()
</code></pre>
<p>class swish(nn.Module):<br>
def <strong>init</strong>(self, beta = 1.25):<br>
super().<strong>init</strong>()<br>
self.beta = beta<br>
def forward(self, inputs):<br>
return inputs * torch.sigmoid(self.beta * inputs)<br>
class DataSet:<br>
def <strong>init</strong>(self,detas,labels):<br>
self.X = detas # 入力<br>
self.t = labels # 出力</p>
<pre><code>def __len__(self):
    return len(self.X) # データ数(10)を返す

def __getitem__(self, index):
    # index番目の入出力ペアを返す
    return self.X[index], self.t[index]
</code></pre>
<p>class Net(torch.nn.Module):</p>
<pre><code>def __init__(self):
    super(Net, self).__init__()
    
    self.conv1 = nn.Sequential(nn.Conv2d(3, 16, 3, padding=1),swish(),
                               nn.Conv2d(16, 16, 3, padding=1),swish(),
                               nn.MaxPool2d(2, 2),swish(),
                               nn.Conv2d(16, 32, 3, padding=1),swish(),
                               nn.Conv2d(32, 32, 3, padding=1),swish(),
                               nn.MaxPool2d(2, 2),swish(),
                               Firttan(),
                               nn.Linear(32 * 8* 8, 512),
                               swish(),
                               nn.Linear(512, 100),
                               nn.Softmax(dim=1))

def forward(self, x):
    x = self.conv1(x)
    return x
</code></pre>
<p>def eval_(dataloader):<br>
correct = 0<br>
total = 0<br>
with torch.no_grad():<br>
for (images, labels) in dataloader:<br>
images, labels = images, labels</p>
<pre><code>        outputs = net(images.to('cuda:0'))
        _, predicted = torch.max(outputs.to('cpu').data, 1)
        correct += (predicted == labels).sum().item()
        total += labels.size(0)

print("Val Acc",(correct/total))
</code></pre>
<p>cifar100_data = torchvision.datasets.CIFAR100(<br>
‘./cifar-100’, train=True, download=True,<br>
transform=torchvision.transforms.ToTensor())</p>
<h1>さっき作ったDataSetクラスのインスタンスを作成</h1>
<h1>datasetをDataLoaderの引数とすることでミニバッチを作成．</h1>
<p>batch_size=128<br>
n_epochs = 50</p>
<p>dataloader = torch.utils.data.DataLoader(cifar100_data, batch_size=batch_size,shuffle=True,drop_last=True)<br>
net = Net().to(‘cuda:0’)<br>
net.train()<br>
MYoptimizer=MYoptimizer()<br>
print(“start”)<br>
for _ in range(n_epochs):<br>
timee=time.time()<br>
net.train()<br>
for inputs, labels in dataloader:<br>
MYoptimizer.loss(inputs.to(‘cuda:0’),labels.to(‘cuda:0’))<br>
MYoptimizer.Done(<em>,time.time()-timee)<br>
<span class="hashtag">#テスト</span><br>
net.eval()<br>
eval</em>(dataloader)<br>
print(“a”)</p>
<p>result<br>
Files already downloaded and verified<br>
start<br>
Epoch 0 total_loss 1796.0163736343384 8.27594780921936<br>
Val Acc 0.009995993589743589<br>
Epoch 1 total_loss 1796.0164184570312 7.108090877532959<br>
Val Acc 0.009935897435897435<br>
Epoch 2 total_loss 1796.0164275169373 7.067081689834595<br>
Val Acc 0.009995993589743589<br>
Epoch 3 total_loss 1796.0164308547974 7.0995776653289795<br>
Val Acc 0.009995993589743589</p>
<p>I can’t display the code well<br>
How can my site recognize it as a code?</p>

username: ptrblck
cooked: <p>Remove the <code>nn.Softmax</code> at the end of your model, as <code>nn.CrossEntropyLoss</code> will apply <code>F.log_softmax</code> internally.</p>
<p>PS: you can post code snippets by wrapping them into three backticks ```, which makes debugging easier <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>