Pytorch Quantization of VGG16

priya_madivanane: <p>I’m trying to quantize (8 bits) VGG16 model with EMNIST(balanced) dataset.<br>
My non quantized model works fine but i get an error with the quantization model where when i redo the layers with quantization i get a NAN error as below:</p>
<h2>Also attaching the function, need help to see what im missing.</h2>
<p>ValueError                                Traceback (most recent call last)<br>
 in ()<br>
----&gt; 1 testQuant(q_model, test_loader, quant=True, stats=stats)</p>
<p>4 frames<br>
 in calcScaleZeroPoint(min_val, max_val, num_bits)<br>
22       zero_point = initial_zero_point<br>
23<br>
—&gt; 24   zero_point = int(zero_point)<br>
25<br>
26   return scale, zero_point</p>
<h2>ValueError: cannot convert float NaN to integer</h2>
<p>def quantForward(model, x, stats):</p>
<h1>Quantise before inputting into incoming layers</h1>
<p>x = quantize_tensor(x, min_val=stats[‘conv1_1’][‘min’], max_val=stats[‘conv1_1’][‘max’])</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x.tensor, model.conv1_1, stats[‘conv1_2’], x.scale, x.zero_point)</p>
<p>x = F.max_pool2d(x, 2, 2)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.conv1_2, stats[‘conv2_1’], scale_next, zero_point_next)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.conv2_1, stats[‘conv2_2’], scale_next, zero_point_next)</p>
<p>x = F.max_pool2d(x, 2, 2)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.conv2_2, stats[‘conv3_1’], scale_next, zero_point_next)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.conv3_1, stats[‘conv3_2’], scale_next, zero_point_next)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.conv3_2, stats[‘conv3_3’], scale_next, zero_point_next)</p>
<p>x = F.max_pool2d(x, 2, 2)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.conv3_3, stats[‘conv4_1’], scale_next, zero_point_next)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.conv4_1, stats[‘conv4_2’], scale_next, zero_point_next)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.conv4_2, stats[‘conv4_3’], scale_next, zero_point_next)</p>
<p><span class="hashtag">#x</span> = F.max_pool2d(x, 2, 2)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.conv4_3, stats[‘conv5_1’], scale_next, zero_point_next)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.conv5_1, stats[‘conv5_2’], scale_next, zero_point_next)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.conv5_2, stats[‘conv5_3’], scale_next, zero_point_next)</p>
<p><span class="hashtag">#x</span> = F.max_pool2d(x, 2, 2)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.conv5_3, stats[‘fc6’], scale_next, zero_point_next)</p>
<p>print(zero_point_next)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.fc6, stats[‘fc7’], scale_next, zero_point_next)</p>
<p>x = x.view(x.shape[0], -1)</p>
<p>x, scale_next, zero_point_next = quantizeLayer(x, model.fc7, stats[‘fc8’], scale_next, zero_point_next)</p>
<h1>Back to dequant for final layer</h1>
<p>x = dequantize_tensor(QTensor(tensor=x, scale=scale_next, zero_point=zero_point_next))</p>
<p>x = model.fc8(x)</p>
<p>return F.log_softmax(x, dim=1)</p>

Zafar: <p>Can you point me to the VGG model you are trying to quantize? Is it th one from <code>torchvision</code>?</p>