Problem loading weights

username: David_Jitca
cooked: <p>I am trying to load a trained network, but now doesnt work.<br>
This is the message I get: " IncompatibleKeys(missing_keys=[], unexpected_keys=[]) "<br>
I ve printed model keys, model trained that i try to load keys, and are the same…Help please!</p>

username: ptrblck
cooked: <p>It looks alright, since no missing or unexpected keys were found. <img src="https://discuss.pytorch.org/images/emoji/apple/slight_smile.png?v=9" title=":slight_smile:" class="emoji" alt=":slight_smile:"></p>

username: David_Jitca
cooked: <p>Yes, it’s ok :). But until yesterday, I didn’t received that message.</p>

username: Peterwhelberg
cooked: <p>Hello ,I’m trying to load a trained model’s weights, but when i load it my AI restart learning from the beginning, like it didn’t take in consideration the weights.<br>
My code for saving weights :<br>
torch.save({‘net’:self.net.state_dict(),<br>
‘optim’:self.optimizer.state_dict(),<br>
},ai.bak)<br>
And for loading :<br>
checkpoint=torch.load(self.find_file(‘ai.bak’))<br>
self.net.load_state_dict(checkpoint[‘net’])<br>
Help me please!</p>

username: ptrblck
cooked: <p>Did you also load the optimizer’s <code>state_dict</code>?<br>
Could you post some loss values during training and after restoring?<br>
Also, which optimizer are you using?</p>

username: Peterwhelberg
cooked: <p>Yes, i also load optimize’s state.<br>
i’m using RMSprop optimizer.<br>
For example after 700 steps, the loss values are<br>
tensor(1.1155, grad_fn=)<br>
But after restoring I get<br>
tensor(0.3977, grad_fn=)</p>

username: ptrblck
cooked: <p>So the loss is actually lower after restoring the training?</p>

username: Peterwhelberg
cooked: <p>Sorry it takes the last value after training but :<br>
After restoring , the value of loss is :</p>
<p>Tensor(0.4,grad_fn=)</p>
<p>After 800 steps , the value of  loss is :</p>
<p>Tensor(1.1,grad_fn=)</p>
<p>After 1000 steps :</p>
<p>Tensor(0.6,grad_fn=)</p>
<p>It’s start increasing  for while before decreasing, when i restore the weights, it’s weird .</p>

username: ptrblck
cooked: <p>That’s strange. Do you have a (small) executable code snippet so that we can have a look at this issue?<br>
Are you changing anything else after restoring, e.g. your data, <code>model.eval()/train()</code> etc.?</p>

username: Peterwhelberg
cooked: <p>class Ai(Player):<br>
“”"<br>
This class implements an AI based on the perceptron defined in class Net<br>
“”"<br>
def <strong>init</strong>(self):<br>
super(Ai, self).<strong>init</strong>()</p>
<pre><code>	self.net = Net()
	#Load network weights if they have been initialized already
	exists = os.path.isfile(self.find_file('ai.pth'))
	if exists:
		print('waa')
		checkpoint=torch.load(self.find_file('ai.pth'))
		self.net.load_state_dict(checkpoint['net'])
		
		self.net.eval()
		

	self.net1=Net()
	self.net1.load_state_dict(self.net.state_dict())
	self.net1.eval()</code></pre>

username: ptrblck
cooked: <p>This code is unfortunately not executable.<br>
Anyway, it looks like you are setting the model(s) to <code>.eval()</code> after loading.<br>
Is it also the case for the continuation of your training?<br>
If so, you should set them to <code>.train()</code> mode.</p>

username: Peterwhelberg
cooked: <p>I will try with .train(), and see the result thank you.</p>

username: SpandanMadan
cooked: <p>The difference in loss with the model in eval() vs train() mode might be happening if you have any batchnorm in your dataset. In the inference stage, the behaviour is per-image for batch norm as far as I know.</p>