Tutorial on Quantizing Object Detection Model

meganlrowe: <p>Hi, I have recently looked at the tutorial for post training static quantization but this is relevant to classifiers. Is there a tutorial/capability to quantize an entire object detection model? If not, what would be the difference if I have a fully trained model and want to quantize only the backbone? Thanks</p>

Vasiliy_Kuznetsov: <p>We don’t have a tutorial to quantize a detection model, we can consider it for the future.  You can quantize the backbone only as follows, in pseudocode:</p>
<pre><code class="lang-auto"># original model (pseudocode)
class M(torch.nn.Module):
  def __init__(self, ...):
    ...
    self.backbone = ...
    self.rpn = ...
    self.head = ...

  def forward(self, x):
    features = self.backbone(x)
    proposals = self.rpn(features)
    head_results = self.head(features, proposals)
    return head_results
</code></pre>
<pre><code class="lang-auto"># modify M, place quants/dequants to prepare backbone for quantization
class MQuantizeable(torch.nn.Module):
  def __init__(self, ...):
    ...
    self.backbone = ...
    self.rpn = ...
    self.head = ...
  self.quant = torch.quantization.QuantStub()
  self.dequant = torch.quantization.DeQuantStub()

  def forward(self, x):
    # wrap backbone in quant/dequant
    x_quant = self.quant(x)
    features_quant = self.backbone(x)
    features = self.dequant(features_quant)
    proposals = self.rpn(features)
    head_results = self.head(features, proposals)
    return head_results
</code></pre>
<pre><code class="lang-auto"># quantization call
m = MQuantizeable(...)
# set qconfig on the backbone only
m.backbone.qconfig = ...
m = torch.quantization.prepare(m, ...)
</code></pre>

meganlrowe: <p>When using the above example and converting my model, I get this error:<br>
Traceback (most recent call last):<br>
File “detection/main_mp.py”, line 734, in <br>
main()<br>
File “detection/main_mp.py”, line 592, in main<br>
p = torch.quantization.convert(myModel)<br>
File “/home/megan/.local/lib/python2.7/site-packages/torch/quantization/quantize.py”, line 293, in convert<br>
convert(mod, mapping, inplace=True)<br>
File “/home/megan/.local/lib/python2.7/site-packages/torch/quantization/quantize.py”, line 294, in convert<br>
reassign[name] = swap_module(mod, mapping)<br>
File “/home/megan/.local/lib/python2.7/site-packages/torch/quantization/quantize.py”, line 316, in swap_module<br>
new_mod = mapping[type(mod)].from_float(mod)<br>
File “/home/megan/.local/lib/python2.7/site-packages/torch/nn/quantized/modules/conv.py”, line 243, in from_float<br>
qweight = _quantize_weight(mod.weight.float(), weight_observer)<br>
File “/home/megan/.local/lib/python2.7/site-packages/torch/nn/quantized/modules/utils.py”, line 12, in _quantize_weight<br>
wt_scale.to(torch.double), wt_zp.to(torch.int64), 0, torch.qint8)<br>
RuntimeError: No function is registered for schema aten::quantize_per_channel(Tensor self, Tensor scales, Tensor zero_points, int axis, ScalarType dtype) -&gt; Tensor on tensor type CUDATensorId; available functions are CPUTensorId, VariableTensorId</p>
<p>I have moved my model to cpu and confirmed it is not running on CUDA.</p>

Vasiliy_Kuznetsov: <aside class="quote no-group quote-modified" data-username="meganlrowe" data-post="3" data-topic="102604">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/c0e974/48.png" class="avatar"> meganlrowe:</div>
<blockquote>
<p>RuntimeError: No function is registered for schema aten::quantize_per_channel(Tensor self, Tensor scales, Tensor zero_points, int axis, ScalarType dtype) → Tensor on tensor type CUDATensorId; available functions are CPUTensorId, VariableTensorId</p>
</blockquote>
</aside>
<p>The error message is in fact saying that something is on CUDA.  Does something like this pass for your model?</p>
<pre><code class="lang-auto">def assert_and_get_unique_device(module: torch.nn.Module) -&gt; Any:
    """
    Returns the unique device for a module, or None if no device is found.
    Throws an error if multiple devices are detected.
    """
    devices = {p.device for p in module.parameters()} | \
        {p.device for p in module.buffers()}
    assert len(devices) &lt;= 1, (
        "prepare only works with cpu or single-device CUDA modules, "
        "but got devices {}".format(devices)
    )
    device = next(iter(devices)) if len(devices) &gt; 0 else None
    return device
</code></pre>

Fanny_0: <p>Hi, is it possible to use this quantized implementation of resnet : <a href="https://github.com/pytorch/vision/blob/b8b08ac3295d56daff3f3d573d7d637adc0f84e3/torchvision/models/quantization/resnet.py" rel="noopener nofollow ugc">torchvision quantized resnet</a> as a backbone ?<br>
I tried to use it by changing a little <a href="https://github.com/pytorch/vision/blob/b8b08ac3295d56daff3f3d573d7d637adc0f84e3/torchvision/models/detection/backbone_utils.py" rel="noopener nofollow ugc">torchvision detection backbone_utils</a> (<code>from torchvision.models.quantization import resnet</code> and add a parameter <code>quantize=true</code> for backbone).</p>
<p>When I try to use this with a pretrained maskrcnn_resnet50 I get an error :<br>
<code>KeyError: 'backbone.body.conv1.bias'</code></p>
<p>Any idea what I missed/didn’t understand ?</p>
<p>Thanks</p>

Shalini_k: <p>I have a pytorch quantised model (model.pt), need help to load this model.pt for object detetcion</p>

jerryzh168: <p>could you start a new question and give more detailis?</p>

jerryzh168: <p>looks like it’s not loaded correctly, are you sure the pretrain weight matches the model you are loading the weight into?</p>