PyTorch Dynamic Quantization clarification

kysandeep176: <p>Hi - I am writing a script to quantize my .pth model ( universal image segmentation model) with dynamic quantization technique referred below.</p>
<p><a href="https://pytorch.org/docs/stable/quantization.html" class="onebox" target="_blank" rel="noopener nofollow ugc">https://pytorch.org/docs/stable/quantization.html</a></p>
<p>my .pth file is built on mask2former architecture which has multiheadattention module in it. Here attached screenshot where this module is not supported for dynamic quantization. Can anyone confirm on this please!!</p>
<p><div class="lightbox-wrapper"><a class="lightbox" href="https://discuss.pytorch.org/uploads/default/original/3X/f/c/fcbf78f7d894d7da50d371cef307cd13b877f3c9.jpeg" data-download-href="https://discuss.pytorch.org/uploads/default/fcbf78f7d894d7da50d371cef307cd13b877f3c9" title="image"><img src="https://discuss.pytorch.org/uploads/default/original/3X/f/c/fcbf78f7d894d7da50d371cef307cd13b877f3c9.jpeg" alt="image" data-base62-sha1="A3Uy8m1f7tJia3cXmrHuGMzBOad" width="687" height="500" data-dominant-color="F6F7F9"><div class="meta"><svg class="fa d-icon d-icon-far-image svg-icon" aria-hidden="true"><use href="#far-image"></use></svg><span class="filename">image</span><span class="informations">701×510 33.7 KB</span><svg class="fa d-icon d-icon-discourse-expand svg-icon" aria-hidden="true"><use href="#discourse-expand"></use></svg></div></a></div></p>

kysandeep176: <p>and i can see ‘nn.embedding’ is also not supported. Can you please confirm!!</p>

HDCharles: <ol>
<li>
<p>yes we don’t have a dynamically quantized mha kernel so its not supported currently.</p>
</li>
<li>
<p>the short answer is yes, dynamic quantization for embeddings/embedding bags is supported (made a PR for this here: <a href="https://github.com/pytorch/pytorch/pull/107623" class="inline-onebox" rel="noopener nofollow ugc">[ao] updating embedding_bag support by HDCharles · Pull Request #107623 · pytorch/pytorch · GitHub</a> with a test showing that it works) so thanks for catching that error.</p>
</li>
</ol>
<p>it should be noted though that the term dynamic/static quantization is a bit overloaded. Its both a descriptor of how floating point activation qparams are calculated (statically (precomputed) or dynamically (at run time), and the name of the quantization flow to get ops that calculate qparams in the aforementioned way. Since embeddings don’t have floating point activations, you can’t actually do either to the ops, but you can obtain a weight-quantized version of the embedding op that will ‘play nice’ with actual dynamically/statically quantized ops by using the dynamic/static quantization flow.</p>

kysandeep176: <p>Hi Charles, Thanks for replying.</p>
<p>Currently, i am trying static quantization technique with Mask2former based .pth model. But i am getting below error.</p>
<p><strong>AssertionError</strong>: Embedding quantization is only supported with float_qparams_weight_only_qconfig.</p>
<p>Below is my model with Mask2former architecture ( modules ) where we have embedding &amp; multiheadattention modules in it.</p>
<p>Mask2Former</p>
<ol>
<li>Mask2FormerPixelLevelModule</li>
<li>Mask2FormerTransformerModule<br>
a.	Mask2FormerSinePositionEmbedding<br>
b.	Mask2FormerMaskedAttentionDecoder<br>
i.	Mask2FormerMaskedAttentionDecoderLayer<br>
1.	Mask2FormerAttention<br>
2.	MultiheadAttention<br>
ii.	Mask2FormerMaskPredictor<br>
1.	Mask2FormerMLPPredictionHead<br>
a.	Mask2FormerPredictionBlock</li>
</ol>
<p>Here is my code:</p>
<p>model_fp32.qconfig = torch.ao.quantization.get_default_qconfig(‘x86’)<br>
model_fp32 = torch.ao.quantization.fuse_modules(model_fp32,[[‘model.pixel_level_module.encoder.features.s4.b5.f.a’,‘model.pixel_level_module.encoder.features.s4.b4.f.a_bn’]])<br>
input_fp32 = torch.randn(1, 3, 544, 960)<br>
model_fp32(input_fp32)<br>
<span class="hashtag-raw">#Quatization</span><br>
model_prepare = torch.ao.quantization.prepare(model_fp32,inplace=True)<br>
final_model = torch.quantization.convert(model_prepare, inplace=True<br>
<span class="hashtag-raw">#save</span> the model<br>
torch.save(final_model.state_dict(),‘model_quantised.pth’)</p>
<p>Can you guide me with any code modifications here to make it work without any error.</p>

HDCharles: <p>You are using the same qconfig for every module in your model by applying it to the top level. If you apply the qconfigs it mentions to your embedding modules that would fix this error. (It ignores the top level qconfig for modules with their own qconfigs)</p>
<p>i.e. “module.qconfig=other_qconfig”</p>

kysandeep176: <p>Hi Charles, thanks for the reply.</p>
<p>I am not able to define qconfigs to each module with my limited coding knowledge. Can you please help me with lines of code to run it without error.</p>
<p>PFA model architecture, which helps you to define qconfigs. Please help me!!</p>
<p>model.transformer_module name mod Mask2FormerTransformerModule(<br>
(position_embedder): Mask2FormerSinePositionEmbedding()<br>
(queries_embedder): Embedding(100, 256)<br>
(queries_features): Embedding(100, 256)<br>
(decoder): Mask2FormerMaskedAttentionDecoder(<br>
(layers): ModuleList(<br>
(0-3): 4 x Mask2FormerMaskedAttentionDecoderLayer(<br>
(self_attn): Mask2FormerAttention(<br>
(k_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(v_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(q_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(out_proj): Linear(in_features=256, out_features=256, bias=True)<br>
)<br>
(activation_fn): ReLU()<br>
(self_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(cross_attn): MultiheadAttention(<br>
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
)<br>
(cross_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(fc1): Linear(in_features=256, out_features=2048, bias=True)<br>
(fc2): Linear(in_features=2048, out_features=256, bias=True)<br>
(final_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
)<br>
)<br>
(layernorm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(mask_predictor): Mask2FormerMaskPredictor(<br>
(mask_embedder): Mask2FormerMLPPredictionHead(<br>
(0): Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): ReLU()<br>
)<br>
(1): Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): ReLU()<br>
)<br>
(2): Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): Identity()<br>
)<br>
)<br>
)<br>
)<br>
(level_embed): Embedding(3, 256)<br>
)<br>
model.transformer_module.position_embedder name mod Mask2FormerSinePositionEmbedding()<br>
model.transformer_module.queries_embedder name mod Embedding(100, 256)<br>
model.transformer_module.queries_features name mod Embedding(100, 256)<br>
model.transformer_module.decoder name mod Mask2FormerMaskedAttentionDecoder(<br>
(layers): ModuleList(<br>
(0-3): 4 x Mask2FormerMaskedAttentionDecoderLayer(<br>
(self_attn): Mask2FormerAttention(<br>
(k_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(v_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(q_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(out_proj): Linear(in_features=256, out_features=256, bias=True)<br>
)<br>
(activation_fn): ReLU()<br>
(self_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(cross_attn): MultiheadAttention(<br>
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
)<br>
(cross_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(fc1): Linear(in_features=256, out_features=2048, bias=True)<br>
(fc2): Linear(in_features=2048, out_features=256, bias=True)<br>
(final_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
)<br>
)<br>
(layernorm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(mask_predictor): Mask2FormerMaskPredictor(<br>
(mask_embedder): Mask2FormerMLPPredictionHead(<br>
(0): Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): ReLU()<br>
)<br>
(1): Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): ReLU()<br>
)<br>
(2): Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): Identity()<br>
)<br>
)<br>
)<br>
)<br>
model.transformer_module.decoder.layers name mod ModuleList(<br>
(0-3): 4 x Mask2FormerMaskedAttentionDecoderLayer(<br>
(self_attn): Mask2FormerAttention(<br>
(k_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(v_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(q_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(out_proj): Linear(in_features=256, out_features=256, bias=True)<br>
)<br>
(activation_fn): ReLU()<br>
(self_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(cross_attn): MultiheadAttention(<br>
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
)<br>
(cross_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(fc1): Linear(in_features=256, out_features=2048, bias=True)<br>
(fc2): Linear(in_features=2048, out_features=256, bias=True)<br>
(final_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
)<br>
)<br>
model.transformer_module.decoder.layers.0 name mod Mask2FormerMaskedAttentionDecoderLayer(<br>
(self_attn): Mask2FormerAttention(<br>
(k_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(v_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(q_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(out_proj): Linear(in_features=256, out_features=256, bias=True)<br>
)<br>
(activation_fn): ReLU()<br>
(self_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(cross_attn): MultiheadAttention(<br>
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
)<br>
(cross_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(fc1): Linear(in_features=256, out_features=2048, bias=True)<br>
(fc2): Linear(in_features=2048, out_features=256, bias=True)<br>
(final_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
)<br>
model.transformer_module.decoder.layers.0.self_attn name mod Mask2FormerAttention(<br>
(k_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(v_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(q_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(out_proj): Linear(in_features=256, out_features=256, bias=True)<br>
)<br>
model.transformer_module.decoder.layers.0.self_attn.k_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.0.self_attn.v_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.0.self_attn.q_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.0.self_attn.out_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.0.activation_fn name mod ReLU()<br>
model.transformer_module.decoder.layers.0.self_attn_layer_norm name mod LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
model.transformer_module.decoder.layers.0.cross_attn name mod MultiheadAttention(<br>
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
)<br>
model.transformer_module.decoder.layers.0.cross_attn.out_proj name mod NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.0.cross_attn_layer_norm name mod LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
model.transformer_module.decoder.layers.0.fc1 name mod Linear(in_features=256, out_features=2048, bias=True)<br>
model.transformer_module.decoder.layers.0.fc2 name mod Linear(in_features=2048, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.0.final_layer_norm name mod LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
model.transformer_module.decoder.layers.1 name mod Mask2FormerMaskedAttentionDecoderLayer(<br>
(self_attn): Mask2FormerAttention(<br>
(k_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(v_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(q_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(out_proj): Linear(in_features=256, out_features=256, bias=True)<br>
)<br>
(activation_fn): ReLU()<br>
(self_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(cross_attn): MultiheadAttention(<br>
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
)<br>
(cross_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(fc1): Linear(in_features=256, out_features=2048, bias=True)<br>
(fc2): Linear(in_features=2048, out_features=256, bias=True)<br>
(final_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
)<br>
model.transformer_module.decoder.layers.1.self_attn name mod Mask2FormerAttention(<br>
(k_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(v_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(q_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(out_proj): Linear(in_features=256, out_features=256, bias=True)<br>
)<br>
model.transformer_module.decoder.layers.1.self_attn.k_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.1.self_attn.v_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.1.self_attn.q_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.1.self_attn.out_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.1.activation_fn name mod ReLU()<br>
model.transformer_module.decoder.layers.1.self_attn_layer_norm name mod LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
model.transformer_module.decoder.layers.1.cross_attn name mod MultiheadAttention(<br>
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
)<br>
model.transformer_module.decoder.layers.1.cross_attn.out_proj name mod NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.1.cross_attn_layer_norm name mod LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
model.transformer_module.decoder.layers.1.fc1 name mod Linear(in_features=256, out_features=2048, bias=True)<br>
model.transformer_module.decoder.layers.1.fc2 name mod Linear(in_features=2048, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.1.final_layer_norm name mod LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
model.transformer_module.decoder.layers.2 name mod Mask2FormerMaskedAttentionDecoderLayer(<br>
(self_attn): Mask2FormerAttention(<br>
(k_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(v_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(q_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(out_proj): Linear(in_features=256, out_features=256, bias=True)<br>
)<br>
(activation_fn): ReLU()<br>
(self_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(cross_attn): MultiheadAttention(<br>
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
)<br>
(cross_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(fc1): Linear(in_features=256, out_features=2048, bias=True)<br>
(fc2): Linear(in_features=2048, out_features=256, bias=True)<br>
(final_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
)<br>
model.transformer_module.decoder.layers.2.self_attn name mod Mask2FormerAttention(<br>
(k_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(v_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(q_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(out_proj): Linear(in_features=256, out_features=256, bias=True)<br>
)<br>
model.transformer_module.decoder.layers.2.self_attn.k_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.2.self_attn.v_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.2.self_attn.q_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.2.self_attn.out_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.2.activation_fn name mod ReLU()<br>
model.transformer_module.decoder.layers.2.self_attn_layer_norm name mod LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
model.transformer_module.decoder.layers.2.cross_attn name mod MultiheadAttention(<br>
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
)<br>
model.transformer_module.decoder.layers.2.cross_attn.out_proj name mod NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.2.cross_attn_layer_norm name mod LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
model.transformer_module.decoder.layers.2.fc1 name mod Linear(in_features=256, out_features=2048, bias=True)<br>
model.transformer_module.decoder.layers.2.fc2 name mod Linear(in_features=2048, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.2.final_layer_norm name mod LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
model.transformer_module.decoder.layers.3 name mod Mask2FormerMaskedAttentionDecoderLayer(<br>
(self_attn): Mask2FormerAttention(<br>
(k_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(v_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(q_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(out_proj): Linear(in_features=256, out_features=256, bias=True)<br>
)<br>
(activation_fn): ReLU()<br>
(self_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(cross_attn): MultiheadAttention(<br>
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
)<br>
(cross_attn_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
(fc1): Linear(in_features=256, out_features=2048, bias=True)<br>
(fc2): Linear(in_features=2048, out_features=256, bias=True)<br>
(final_layer_norm): LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
)<br>
model.transformer_module.decoder.layers.3.self_attn name mod Mask2FormerAttention(<br>
(k_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(v_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(q_proj): Linear(in_features=256, out_features=256, bias=True)<br>
(out_proj): Linear(in_features=256, out_features=256, bias=True)<br>
)<br>
model.transformer_module.decoder.layers.3.self_attn.k_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.3.self_attn.v_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.3.self_attn.q_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.3.self_attn.out_proj name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.3.activation_fn name mod ReLU()<br>
model.transformer_module.decoder.layers.3.self_attn_layer_norm name mod LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
model.transformer_module.decoder.layers.3.cross_attn name mod MultiheadAttention(<br>
(out_proj): NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
)<br>
model.transformer_module.decoder.layers.3.cross_attn.out_proj name mod NonDynamicallyQuantizableLinear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.3.cross_attn_layer_norm name mod LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
model.transformer_module.decoder.layers.3.fc1 name mod Linear(in_features=256, out_features=2048, bias=True)<br>
model.transformer_module.decoder.layers.3.fc2 name mod Linear(in_features=2048, out_features=256, bias=True)<br>
model.transformer_module.decoder.layers.3.final_layer_norm name mod LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
model.transformer_module.decoder.layernorm name mod LayerNorm((256,), eps=1e-05, elementwise_affine=True)<br>
model.transformer_module.decoder.mask_predictor name mod Mask2FormerMaskPredictor(<br>
(mask_embedder): Mask2FormerMLPPredictionHead(<br>
(0): Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): ReLU()<br>
)<br>
(1): Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): ReLU()<br>
)<br>
(2): Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): Identity()<br>
)<br>
)<br>
)<br>
model.transformer_module.decoder.mask_predictor.mask_embedder name mod Mask2FormerMLPPredictionHead(<br>
(0): Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): ReLU()<br>
)<br>
(1): Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): ReLU()<br>
)<br>
(2): Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): Identity()<br>
)<br>
)<br>
model.transformer_module.decoder.mask_predictor.mask_embedder.0 name mod Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): ReLU()<br>
)<br>
model.transformer_module.decoder.mask_predictor.mask_embedder.0.0 name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.mask_predictor.mask_embedder.0.1 name mod ReLU()<br>
model.transformer_module.decoder.mask_predictor.mask_embedder.1 name mod Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): ReLU()<br>
)<br>
model.transformer_module.decoder.mask_predictor.mask_embedder.1.0 name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.mask_predictor.mask_embedder.1.1 name mod ReLU()<br>
model.transformer_module.decoder.mask_predictor.mask_embedder.2 name mod Mask2FormerPredictionBlock(<br>
(0): Linear(in_features=256, out_features=256, bias=True)<br>
(1): Identity()<br>
)<br>
model.transformer_module.decoder.mask_predictor.mask_embedder.2.0 name mod Linear(in_features=256, out_features=256, bias=True)<br>
model.transformer_module.decoder.mask_predictor.mask_embedder.2.1 name mod Identity()<br>
model.transformer_module.level_embed name mod Embedding(3, 256)<br>
class_predictor name mod Linear(in_features=256, out_features=29, bias=True)<br>
criterion name mod Mask2FormerLoss(<br>
(matcher): Mask2FormerHungarianMatcher()<br>
)<br>
criterion.matcher name mod Mask2FormerHungarianMatcher()</p>

HDCharles: <p>get the fqn of your embedding module and then just set module.qconfig = different_qconfig</p>
<p>e.g. model.transformer_module.position_embedder.qconfig = different_qconfig</p>
<p>do that for each embedding layer</p>