Some questions about Keras and PyTorch

username: flfl1111
cooked: <p>I built a simple convolutional neural network model using Keras framework and PyTorch framework respectively. But there are some questions between Keras and Pytorch.<br>
The data used is an open source - PHM 2010 Tool wear data set<br>
I choose C1 set as my data set which contains 315 samples, each sample has 7 signal inputs, and one sample returns a wear value.<br>
The structure of CNN in Keras framework and Pytorch framework is same, but the output results are different.  The loss function is the mean square error function.<br>
criterion = torch.nn.MSELoss(reduction=‘mean’)<br>
I do not understand where the problem is.<br>
Curves fit well under the Keras framework, but only a straight line can be trained under the PyTorch framework.The details are shown in Figures 1 and 2.<br>
The training code under the PyTorch framework is shown below.</p>
<p>criterion = torch.nn.MSELoss(reduction=‘mean’)<br>
<span class="hashtag-raw">#criterion</span> = nn.SmoothL1Loss(reduction=‘mean’)<br>
optimizer = optim.Adam(model.parameters(),lr=0.001)<br>
iterations = 40<br>
loss_tend = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
y_predict_store = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
for i in range(iterations):<br>
print(‘*’*60)<br>
print(‘the epoch :’,i+1)<br>
y_pred_store = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
tmd = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
loss_sum = 0<br>
start = time.time()<br>
loss_list = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
model.train()<br>
for k,(X_train1,Y_train1) in enumerate(loader_train):<br>
y_pred = model(X_train1)<br>
loss = criterion(y_pred,Y_train1)<br>
print('the result of y_pred and Y_train1: ',y_pred,Y_train1)<br>
for p in range(len(y_pred)):<br>
temp = y_pred[p].item()</p>
<pre><code>        y_pred_store.append(temp)
    loss1 = loss.item()
    loss_list.append(loss1)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
end = time.time()
tmd = np.array(loss_list)
print(np.size(tmd))
loss_mean = np.sum(tmd)/252
loss_tend.append(loss_mean)
print('loss_sum:',loss_mean)
print('the time of this epoch :',end-start)
</code></pre>
<p>print('print y_pred_store: ',y_pred_store)<br>
print(‘print loss_tend:’,loss_tend)<br>
“”"<br>
test set<br>
“”"<br>
for i,(X_test1,Y_test1) in enumerate(loader_test):<br>
model.eval()<br>
predict = model(X_test1)<br>
loss = criterion(predict,Y_test1)<br>
for w in range(len(predict)):<br>
temp_test = predict[w].item()<br>
y_predict_store.append(temp_test)<br>
optimizer.zero_grad()<br>
loss.backward()<br>
optimizer.step()<br>
print(y_predict_store)</p>
<p>After experiments, I found that if I change the MSE loss function to the MSE of one of samples in this batch as the loss function, and the output is well.<br>
The specific code is shown below.Figure3 and Figure 4 .</p>
<p>def sunshi(X,Y):<br>
loss = criterion(X[0],Y[0])<br>
return loss</p>
<p>model = CNN()</p>
<p>import torch.optim as optim<br>
criterion = torch.nn.MSELoss(reduction=‘mean’)<br>
<span class="hashtag-raw">#criterion</span> = nn.SmoothL1Loss(reduction=‘mean’)<br>
<span class="hashtag-raw">#criterion</span> = torch.nn.L1Loss(reduction=‘mean’)</p>
<p>optimizer = optim.Adam(model.parameters(),lr=0.001)<br>
iterations = 2000<br>
loss_tend = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
y_predict_store = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
for i in range(iterations):<br>
print(‘*’*60)<br>
print(‘the epoch :’,i+1)<br>
y_pred_store = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
tmd = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
loss_sum = 0<br>
start = time.time()<br>
loss_list = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
model.train()<br>
for k,(X_train1,Y_train1) in enumerate(loader_train):<br>
y_pred = <span class="chcklst-box fa fa-square-o fa-fw"></span><br>
y_pred = model(X_train1)<br>
loss = sunshi(y_pred,Y_train1)<br>
print('the result of y_pred and Y_train1: ',y_pred,Y_train1)<br>
for p in range(len(y_pred)):<br>
temp = y_pred[p].item()</p>
<pre><code>        y_pred_store.append(temp)
    loss1 = loss.item()
    loss_list.append(loss1)
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
end = time.time()
tmd = np.array(loss_list)
print(np.size(tmd))
loss_mean = np.sum(tmd)/252
loss_tend.append(loss_mean)
print('loss_sum:',loss_mean)
print('the time of this epoch :',end-start)
</code></pre>
<p>print('print y_pred_store: ',y_pred_store)<br>
print(‘print loss_tend:’,loss_tend)<br>
“”"<br>
重新编写测试集<br>
“”"<br>
for i,(X_test1,Y_test1) in enumerate(loader_test):<br>
model.eval()<br>
predict = model(X_test1)<br>
loss = criterion(predict,Y_test1)<br>
for w in range(len(predict)):<br>
temp_test = predict[w].item()<br>
y_predict_store.append(temp_test)<br>
optimizer.zero_grad()<br>
loss.backward()<br>
optimizer.step()<br>
print(y_predict_store)</p>
<p>From what has been discussed above.</p>
<ol>
<li>Why does the same structure have different results under Keras and PyTorch<br>
2.Why does the model work when I change the loss function from the MSE of the whole batch to the MSE of a sample<br>
3.Under the PyTorch framework, the training effect of the training set is poor, but the test set is better.<br>
Link to complete code:<a href="https://github.com/charmerphil/Try-to-explore-the-difference-between-Keras-and-PyTorch.git" rel="noopener nofollow ugc">https://github.com/charmerphil/Try-to-explore-the-difference-between-Keras-and-PyTorch.git</a><br>
Figure:<br>
<div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/1/3/134646eba58f01c3322adea841382ead3c7a7e3e.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/134646eba58f01c3322adea841382ead3c7a7e3e" title="second-1-1"><img src="https://discuss.pytorch.org/uploads/default/optimized/3X/1/3/134646eba58f01c3322adea841382ead3c7a7e3e_2_188x500.jpeg" alt="second-1-1" data-base62-sha1="2KvDFlkgi8ohf6x7twAX7wGkpzw" width="188" height="500" srcset="https://discuss.pytorch.org/uploads/default/optimized/3X/1/3/134646eba58f01c3322adea841382ead3c7a7e3e_2_188x500.jpeg, https://discuss.pytorch.org/uploads/default/optimized/3X/1/3/134646eba58f01c3322adea841382ead3c7a7e3e_2_282x750.jpeg 1.5x, https://discuss.pytorch.org/uploads/default/optimized/3X/1/3/134646eba58f01c3322adea841382ead3c7a7e3e_2_376x1000.jpeg 2x" data-dominant-color="EDE0E0"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">second-1-1</span><span class="informations">483×1283 153 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></li>
</ol>