The pytorch gpu model is loaded into the CPU that do different results each time in test

spmimi: <p>TEST<br>
print(‘Loading model…’)<br>
<span class="hashtag">#-net</span> = RetinaNet()<br>
net = RetinaNet(backbone=cfg.backbone, num_classes=len(cfg.classes),pretrained=False)<br>
map_location = lambda storage, loc: storage<br>
ipdb.set_trace()<br>
<span class="hashtag">#net</span>.load_state_dict(torch.load(’.store\ckpt6.pth’)[‘net’])<br>
net.load_state_dict(torch.load(’.store/ckpt6.pth’,map_location=‘cpu’)[‘net’],strict=False)</p>
<p>print(torch.load(’.store/ckpt6.pth’,map_location=map_location)[‘lr’])<br>
print(torch.load(’.store/ckpt6.pth’,map_location=map_location)[‘epoch’])<br>
#z=net.named_modules()<br>
<span class="hashtag">#for</span> x,xx in subnet4.named_parameters():</p>
<h1>print(x,xx)</h1>
<h1>ipdb.set_trace()</h1>
<p>subnet1=net.resnet<br>
print(net.resnet.training)</p>
<p>subnet2=net.feature_pyramid<br>
print(net.feature_pyramid.training)<br>
subnet3=net.subnet_boxes<br>
print(net.subnet_boxes.training)<br>
subnet4=net.subnet_classes<br>
print(net.subnet_classes.training)<br>
subnet1.eval()<br>
print(net.resnet.training)<br>
for x in subnet1._modules:<br>
print(subnet1._modules[x].training)</p>
<p>for x in subnet2._modules:<br>
subnet2._modules[x].eval()<br>
for x in subnet3._modules:<br>
subnet3._modules[x].eval()<br>
for x in subnet4._modules:<br>
subnet4._modules[x].eval()</p>
<p>subnet2.eval()<br>
print(net.feature_pyramid.training)<br>
subnet3.eval()<br>
print(net.subnet_boxes.training)<br>
subnet4.eval()<br>
print(net.subnet_classes.training)</p>
<p>TRAIN</p>
<p>net = RetinaNet(backbone=cfg.backbone, num_classes=len(cfg.classes))<br>
net = torch.nn.DataParallel(net, device_ids=range(torch.cuda.device_count()))<br>
net.cuda()</p>
<p>def save_checkpoint(loss, net, n):<br>
global best_loss<br>
loss /= n<br>
if loss &lt; best_loss:<br>
print(‘Saving…’)<br>
state = {<br>
‘net’: net.state_dict(),<br>
‘loss’: loss,<br>
‘epoch’: epoch,<br>
‘lr’: lr<br>
}<br>
# ckpt_path = os.path.join(‘ckpts’, args.exp)<br>
ckpt_path = ‘.store’<br>
if not os.path.isdir(ckpt_path):<br>
os.makedirs(ckpt_path)<br>
torch.save(state, os.path.join(ckpt_path, ‘ckpt.pth’))<br>
best_loss = loss</p>

spmimi: <p>有人吗，中国的道友们，<br>
Hello, Chinese friends, my English is not good</p>

spmimi: <p>please, help me,thank you!!!</p>

ptrblck: <p>After creating an instance of your model, you can just call <code>model.eval()</code> and all submodules will also be set to eval mode, so you don’t need to call <code>.eval()</code> on each submodule separately.<br>
Could you post a link to the model implementation?</p>
<p>PS: If you would like to post code snippets, you can simply wrap it in three backticks ``` <img src="https://discuss.pytorch.org/images/emoji/apple/wink.png?v=9" title=":wink:" class="emoji" alt=":wink:"></p>

spmimi: <p>TEST<br>
import torch<br>
import torchvision.transforms as transforms<br>
import  exps.voc.config2 as cfg<br>
from torch.autograd import Variable<br>
import pandas as pd<br>
from retinanet import RetinaNet<br>
from encoder import DataEncoder<br>
from PIL import Image, ImageDraw<br>
try:<br>
import ipdb<br>
except:<br>
import pdb as ipdb</p>
<p>print(‘Loading model…’)</p>
<p>net = RetinaNet(backbone=cfg.backbone, num_classes=len(cfg.classes),pretrained=False)<br>
map_location = lambda storage, loc: storage<br>
net.load_state_dict(torch.load(’.store/ckpt6.pth’,map_location=‘cpu’)[‘net’],strict=False)<br>
print(torch.load(’.store/ckpt6.pth’,map_location=map_location)[‘lr’])<br>
print(torch.load(’.store/ckpt6.pth’,map_location=map_location)[‘epoch’])<br>
net.eval()<br>
transform = transforms.Compose([<br>
transforms.ToTensor(),<br>
transforms.Normalize((0.485,0.456,0.406), (0.229,0.224,0.225))<br>
])<br>
print(‘Loading image…’)<br>
img = Image.open(’./image/0.jpg’)<br>
w =img.size[0]<br>
h=img.size[1]<br>
img = img.resize((w,h))<br>
x = transform(img)<br>
x = x.unsqueeze(0)<br>
x = Variable(x, volatile=True)<br>
loc_preds, cls_preds = net(x)</p>
<p>encoder = DataEncoder()</p>
<p>cls_preds1=cls_preds.data.squeeze()<br>
score, labels = cls_preds1.max(1)<br>
xxx = pd.DataFrame(score.tolist())<br>
print(xxx.describe())<br>
yyy=pd.DataFrame(labels.tolist())<br>
print(yyy.describe())<br>
yy=labels&gt;0<br>
print(yy.numel())<br>
print(yy.data.long().sum())<br>
FIRST</p>
<pre><code>              0
</code></pre>
<p>count  56088.000000<br>
mean       0.014430<br>
std        0.033506<br>
min       -0.333214<br>
25%       -0.000789<br>
50%        0.010297<br>
75%        0.027038<br>
max        0.603817<br>
0<br>
count  56088.000000<br>
mean       0.770807<br>
std        0.420318<br>
min        0.000000<br>
25%        1.000000<br>
50%        1.000000<br>
75%        1.000000<br>
max        1.000000<br>
56088<br>
tensor(43233)<br>
SECOND<br>
0<br>
count  56088.000000<br>
mean       0.022885<br>
std        0.063939<br>
min       -0.526022<br>
25%       -0.011785<br>
50%        0.007780<br>
75%        0.041688<br>
max        0.933956<br>
0<br>
count  56088.000000<br>
mean       0.550260<br>
std        0.497472<br>
min        0.000000<br>
25%        0.000000<br>
50%        1.000000<br>
75%        1.000000<br>
max        1.000000<br>
56088<br>
tensor(30863)</p>

spmimi: <p>The above is what I modified according to what you said. This is a target detection binary task to distinguish background and foreground. I used retinanet network structure. The score and labels for cls_preds change each time you tes</p>

ptrblck: <p>Could you post a link to the implementation you are using for <code>RetinaNet</code>, as I would like to run the code.</p>

spmimi: <aside class="onebox whitelistedgeneric">
  <header class="source">
      <img src="https://github.githubassets.com/favicon.ico" class="site-icon" width="32" height="32">
      <a href="https://github.com/jiaojiechu/retinanet1" target="_blank" rel="nofollow noopener">GitHub</a>
  </header>
  <article class="onebox-body">
    <img src="https://avatars0.githubusercontent.com/u/40553484?s=400&amp;amp;v=4" class="thumbnail onebox-avatar" width="420" height="420">

<h3><a href="https://github.com/jiaojiechu/retinanet1" target="_blank" rel="nofollow noopener">jiaojiechu/retinanet1</a></h3>

<p>Contribute to jiaojiechu/retinanet1 development by creating an account on GitHub.</p>


  </article>
  <div class="onebox-metadata">
    
    
  </div>
  <div style="clear: both"></div>
</aside>


spmimi: <p><a href="https://github.com/jiaojiechu/retinanet1.git" rel="nofollow noopener">https://github.com/jiaojiechu/retinanet1.git</a>   this is my code</p>

spmimi: <p>Does anyone have a similar problem，自己顶</p>

ptrblck: <p>Thanks for the code!<br>
After removing the hard coded checkpoint paths, I get the same outputs for the model in eval mdoe:</p>
<pre><code class="lang-python">net = RetinaNet(backbone='resnet34', num_classes=10, pretrained=False)
net.eval()
x = torch.randn(2, 3, 224, 224)
output1a, output1b = net(x)
output2a, output2b = net(x)

print((output1a==output2a).all())
&gt; tensor(1, dtype=torch.uint8)
print((output1b==output2b).all())
&gt; tensor(1, dtype=torch.uint8)
</code></pre>

spmimi: <p>Thank you, but I don’t understand what you mean. Is there something wrong with the model I saved, or is my loading method inappropriate?</p>

ptrblck: <p>I’m not sure how you’ve saved the model, but you’ve hardcoded the checkpoint paths <a href="https://github.com/jiaojiechu/retinanet1/blob/8bd1935246e869d64c771893e27a153921fd1d07/resnet.py#L107" rel="nofollow noopener">here</a>, which are used, even if <code>pretrained=False</code> is passed, so I had to remove them.</p>
<p>After doing so, you’ll see that the outputs are consistent.<br>
I’m also not sure, how you’ve created the output for <code>FIRST</code> and <code>SECOND</code>, but apparently something between these calls was changed.</p>

spmimi: <p>I think that my problems  is like</p><aside class="quote quote-modified" data-post="1" data-topic="22289">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/4af34b/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/different-results-on-cpu-and-gpu/22289">Different results on cpu and gpu</a> 
  </div>
  <blockquote>
    I trained my model on GPU and saved the model with 
params = {k:v.cpu() for k, v in self.state_dict().items()}
torch.save(params, open('model.pt', 'wb'))

Then, when evaluating the model, I load it like this: 
model = m(...)
params = torch.load('model.pt')
model.load_state_dict(params)

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = model.to(device)

However, I am getting much worse results on GPU, as on CPU. It seems like on CPU its performing correctly (I get sim…
  </blockquote>
</aside>
<p>
can you see that? thanks</p>

spmimi: <p>I think that my problems is like</p><aside class="quote quote-modified" data-post="1" data-topic="22289">
  <div class="title">
    <div class="quote-controls"></div>
    <img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/t/4af34b/48.png" class="avatar">
    <a href="https://discuss.pytorch.org/t/different-results-on-cpu-and-gpu/22289">Different results on cpu and gpu</a> 
  </div>
  <blockquote>
    I trained my model on GPU and saved the model with 
params = {k:v.cpu() for k, v in self.state_dict().items()}
torch.save(params, open('model.pt', 'wb'))

Then, when evaluating the model, I load it like this: 
model = m(...)
params = torch.load('model.pt')
model.load_state_dict(params)

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = model.to(device)

However, I am getting much worse results on GPU, as on CPU. It seems like on CPU its performing correctly (I get sim…
  </blockquote>
</aside>
<p>
can you see that? thanks</p>

spmimi: <p>I found ， the data was consistent on the same run, but then changed again</p>