QuantStub/DeQuantStubs for QAT confusion

username: kekpirat
cooked: <p>Hello,</p>
<p>I’m kind of confused by the use of <code>Quant/DeQuantStubs</code> for Quantization Aware Training.</p>
<p>From my understanding only layers in between the Quant/DequantStubs are supposed to be quantised (is that correct?) but for my model when I place quantstubs around just the backbone:</p>
<pre><code class="lang-auto">x = self.quant0(x)
x = self.backbone0(x)
x = self.dequant(x)
confidence = self.classification_headers0(x)
</code></pre>
<p>and I look at the layers in classification_headers before and after preparation:</p>
<pre><code class="lang-auto">print(model.classification_headers0)
model.qconfig = torch.quantization.get_default_qat_qconfig('fbgemm')
torch.quantization.prepare_qat(model, inplace=True)
print(model.classification_headers0)
</code></pre>
<p>I get</p>
<pre><code class="lang-auto">Sequential(
  (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=64)
  (1): ReLU()
  (2): Conv2d(64, 6, kernel_size=(1, 1), stride=(1, 1))
)
Sequential(
  (0): Conv2d(
    64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), groups=64
    (activation_post_process): FakeQuantize(
      fake_quant_enabled=tensor([1], dtype=torch.uint8), observer_enabled=tensor([1], dtype=torch.uint8),            scale=tensor([1.]), zero_point=tensor([0])
      (activation_post_process): MovingAverageMinMaxObserver(min_val=tensor([]), max_val=tensor([]))
    )
    (weight_fake_quant): FakeQuantize(
      fake_quant_enabled=tensor([1], dtype=torch.uint8), observer_enabled=tensor([1], dtype=torch.uint8),            scale=tensor([1.]), zero_point=tensor([0])
      (activation_post_process): MovingAveragePerChannelMinMaxObserver(min_val=tensor([]), max_val=tensor([]))
    )
  )
  (1): ReLU(
    (activation_post_process): FakeQuantize(
      fake_quant_enabled=tensor([1], dtype=torch.uint8), observer_enabled=tensor([1], dtype=torch.uint8),            scale=tensor([1.]), zero_point=tensor([0])
      (activation_post_process): MovingAverageMinMaxObserver(min_val=tensor([]), max_val=tensor([]))
    )
  )
  (2): Conv2d(
    64, 6, kernel_size=(1, 1), stride=(1, 1)
    (activation_post_process): FakeQuantize(
      fake_quant_enabled=tensor([1], dtype=torch.uint8), observer_enabled=tensor([1], dtype=torch.uint8),            scale=tensor([1.]), zero_point=tensor([0])
      (activation_post_process): MovingAverageMinMaxObserver(min_val=tensor([]), max_val=tensor([]))
    )
    (weight_fake_quant): FakeQuantize(
      fake_quant_enabled=tensor([1], dtype=torch.uint8), observer_enabled=tensor([1], dtype=torch.uint8),            scale=tensor([1.]), zero_point=tensor([0])
      (activation_post_process): MovingAveragePerChannelMinMaxObserver(min_val=tensor([]), max_val=tensor([]))
    )
  )
)
</code></pre>
<p>Why are the layers in classification_headers0 prepared for quantisation too?</p>

username: Vasiliy_Kuznetsov
cooked: <aside class="quote no-group" data-username="kekpirat" data-post="1" data-topic="88486">
<div class="title">
<div class="quote-controls"></div>
<img loading="lazy" alt="" width="24" height="24" src="https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/kekpirat/48/26428_2.png" class="avatar"> kekpirat:</div>
<blockquote>
<p>From my understanding only layers in between the Quant/DequantStubs are supposed to be quantised (is that correct?)</p>
</blockquote>
</aside>
<p>Which layers to quantize is controlled by the qconfig, when we do <code>model.qconfig = torch.quantization.get_default_qat_qconfig('fbgemm')</code> it applies the defaults settings of which modules to swap to the entire model.</p>

username: Vasiliy_Kuznetsov
cooked: <p>the <code>mapping</code> argument to <code>prepare_qat</code> (<a href="https://github.com/pytorch/pytorch/blob/733b8c23c436d906125c20f0a64692bf57bce040/torch/quantization/quantize.py#L289" rel="nofollow noopener">https://github.com/pytorch/pytorch/blob/733b8c23c436d906125c20f0a64692bf57bce040/torch/quantization/quantize.py#L289</a>) can be used to customize which layer’s you’d like to quantize</p>