Error in QAT evaluate

username: xieydd
cooked: <p>When i run QAT, training is normal, but when i want to evaluate the qat model, an error <code>length of scales must equal to channel</code> confuse me.<br>
I use pytorch 1.4.0, and my code is</p>
<pre><code class="lang-auto"># Traing is Normal
net = MyQATNet()
net.fuse_model()
net.qconfig = torch.quantization.get_default_qat_config("fbgemm")
net_eval = net
model.apply(torch.quantization.enable_observer)
model.apply(torch.quantization.enable_fake_quant)

# Evaluate
qat_model = copy.deepcopy(net_eval)
qat_model.to(torch.device('cpu'))
torch.quantization.convert(qat_model, inplace=True) # Error is there
qat_model.eval()
</code></pre>
<p><a class="mention" href="/u/ptrblck">@ptrblck</a> Can you have a look?</p>

username: xieydd
cooked: <p>Anybody help me, thanks a lot?</p>

username: jerryzh168
cooked: <p>can you paste your network definition and the input you use to run the model? it might be a problem with your input</p>

username: xieydd
cooked: <p>I think i find where is the error, When i use DataParallel, it will be such an error, but when i use single gpu, will no error.</p>

username: xieydd
cooked: <p>How can i save the qat trained model, when i save <code>torch.save(qat_model.state_dict(),'qat_model.pth')</code> or i directly save training model <code>torch.save(net, 'net.pth')</code>, when i want to load the pretrained qat model, for qat_model, the key is like  <code>conv1.0.activation_post_process.scale</code>; and when net, the key have no <code>conv1.0.activation_post_process.scale</code>, but expected key is  <code>conv1.0.0.activation_post_process.scale</code>, so KeyError happened. When i see the model definition, expected key is right.</p>

username: xieydd
cooked: <p>I think somebody have same question like me. <a href="https://github.com/pytorch/pytorch/issues/32691" rel="nofollow noopener">https://github.com/pytorch/pytorch/issues/32691</a></p>

username: supriyar
cooked: <aside class="quote no-group" data-username="xieydd" data-post="4" data-topic="84901" data-full="true">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/xieydd/48/25593_2.png" class="avatar"> xieydd:</div>
<blockquote>
<p>I think i find where is the error, When i use DataParallel, it will be such an error, but when i use single gpu, will no error.</p>
</blockquote>
</aside>
<p>We recently fixed a bug with QAT and DataParallel, please try with pytorch nightly to see if the issue still persists. cc <a class="mention" href="/u/vasiliy_kuznetsov">@Vasiliy_Kuznetsov</a></p>

username: xieydd
cooked: <p>you mean the load_state_dict KeyError is also solved in newest version of pytorch? tks</p>

username: xieydd
cooked: <p>I think somebody have the same error: <a href="https://discuss.pytorch.org/t/issue-with-quantization/67457" class="inline-onebox">Issue with Quantization</a></p>
<p>I think i know the answew:</p>
<pre><code class="lang-auto"># pytorch 1.4
#save
torch.save(net.state_dict(),'xx') # fp32 model
#load
model.qconfig = torch.quantization.get_default_qat_qconfig("fbgemm")
model.fuse_model()
torch.quantization.prepare_qat(net, inplace=True)
state_dict = torch.load('xx', map_location=torch.device('cpu'))
# remove module.  module
#torch.quantization.convert(net,inplace=True) # convert it to int8 model
x = torch.randn((2,3,300,300))
y = net(x)
print(y)
print('Success')
</code></pre>

username: Vasiliy_Kuznetsov
cooked: <p>Yes, <a href="https://github.com/pytorch/pytorch/pull/37032" rel="nofollow noopener">https://github.com/pytorch/pytorch/pull/37032</a> fixes an error for DataParallel.  <a class="mention" href="/u/xieydd">@xieydd</a> you can try the nightly to verify if this fixes your problem.</p>

username: xieydd
cooked: <p>nightly can fix my problem, tks for all.</p>