Conversion of quantized pytorch transformer model to pyscript throws error Runtime Error: method cannot be used as a value

Vikas_Kumar_Ojha: <p>Below is the code for converting the parseq model based on transformer for OCR. If I comment the quantization part the model successfully gets converted to torchscript but when using quantization its throwing error:</p>
<p>RuntimeError:<br>
method cannot be used as a value:<br>
File “/usr/local/lib/python3.10/dist-packages/torch/nn/modules/activation.py”, line 1100<br>
self.in_proj_weight,<br>
self.in_proj_bias,<br>
self.out_proj.weight,<br>
~~~~~~~~~~~~~~~~~~~~ &lt;— HERE<br>
self.out_proj.bias,<br>
)</p>
<p>import torch</p>
<p>from PIL import Image</p>
<p>from strhub.data.module import SceneTextDataModule</p>
<p>from strhub.models.utils import load_from_checkpoint, parse_model_args</p>
<p>from torch.quantization import QuantStub, DeQuantStub, float_qparams_weight_only_qconfig, default_qconfig</p>
<p>from torch.utils.mobile_optimizer import optimize_for_mobile</p>
<h1><a name="load-model-and-image-transforms-1" class="anchor" href="#load-model-and-image-transforms-1"></a>Load model and image transforms</h1>
<p>parseq = torch.hub.load(‘baudm/parseq’, ‘parseq’, pretrained=True).eval()</p>
<p>img_transform = SceneTextDataModule.get_transform(parseq.hparams.img_size)</p>
<p>Below is the forked repo I am using:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://github.com/VikasOjha666/parseq">
  <header class="source">
      <img src="https://github.githubassets.com/favicons/favicon.svg" class="site-icon" width="32" height="32">

      <a href="https://github.com/VikasOjha666/parseq" target="_blank" rel="noopener nofollow ugc">GitHub</a>
  </header>

  <article class="onebox-body">
    <div class="aspect-image" style="--aspect-ratio:690/345;"><img src="https://opengraph.githubassets.com/f67ec2fc9b4c5179c1903f9961175215a6d42c25fdcd414e095b23f822e14694/VikasOjha666/parseq" class="thumbnail" width="690" height="345"></div>

<h3><a href="https://github.com/VikasOjha666/parseq" target="_blank" rel="noopener nofollow ugc">GitHub - VikasOjha666/parseq: Scene Text Recognition with Permuted...</a></h3>

  <p>Scene Text Recognition with Permuted Autoregressive Sequence Models (ECCV 2022) - GitHub - VikasOjha666/parseq: Scene Text Recognition with Permuted Autoregressive Sequence Models (ECCV 2022)</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>

<p>Torch version: 1.12.1</p>
<p>Below is the model architecture:<br>
PARSeq(<br>
(encoder): Encoder(<br>
(patch_embed): PatchEmbed(<br>
(proj): QuantizedConv2d(3, 192, kernel_size=(4, 8), stride=(4, 8), scale=1.0, zero_point=0)<br>
(norm): Identity()<br>
)<br>
(pos_drop): QuantizedDropout(p=0.0, inplace=False)<br>
(norm_pre): Identity()<br>
(blocks): Sequential(<br>
(0): Block(<br>
(norm1): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(attn): Attention(<br>
(qkv): QuantizedLinear(in_features=192, out_features=576, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(attn_drop): QuantizedDropout(p=0.0, inplace=False)<br>
(proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(proj_drop): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls1): Identity()<br>
(drop_path1): Identity()<br>
(norm2): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(mlp): Mlp(<br>
(fc1): QuantizedLinear(in_features=192, out_features=768, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(act): GELU(approximate=none)<br>
(drop1): QuantizedDropout(p=0.0, inplace=False)<br>
(fc2): QuantizedLinear(in_features=768, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(drop2): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls2): Identity()<br>
(drop_path2): Identity()<br>
)<br>
(1): Block(<br>
(norm1): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(attn): Attention(<br>
(qkv): QuantizedLinear(in_features=192, out_features=576, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(attn_drop): QuantizedDropout(p=0.0, inplace=False)<br>
(proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(proj_drop): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls1): Identity()<br>
(drop_path1): Identity()<br>
(norm2): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(mlp): Mlp(<br>
(fc1): QuantizedLinear(in_features=192, out_features=768, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(act): GELU(approximate=none)<br>
(drop1): QuantizedDropout(p=0.0, inplace=False)<br>
(fc2): QuantizedLinear(in_features=768, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(drop2): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls2): Identity()<br>
(drop_path2): Identity()<br>
)<br>
(2): Block(<br>
(norm1): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(attn): Attention(<br>
(qkv): QuantizedLinear(in_features=192, out_features=576, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(attn_drop): QuantizedDropout(p=0.0, inplace=False)<br>
(proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(proj_drop): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls1): Identity()<br>
(drop_path1): Identity()<br>
(norm2): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(mlp): Mlp(<br>
(fc1): QuantizedLinear(in_features=192, out_features=768, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(act): GELU(approximate=none)<br>
(drop1): QuantizedDropout(p=0.0, inplace=False)<br>
(fc2): QuantizedLinear(in_features=768, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(drop2): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls2): Identity()<br>
(drop_path2): Identity()<br>
)<br>
(3): Block(<br>
(norm1): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(attn): Attention(<br>
(qkv): QuantizedLinear(in_features=192, out_features=576, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(attn_drop): QuantizedDropout(p=0.0, inplace=False)<br>
(proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(proj_drop): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls1): Identity()<br>
(drop_path1): Identity()<br>
(norm2): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(mlp): Mlp(<br>
(fc1): QuantizedLinear(in_features=192, out_features=768, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(act): GELU(approximate=none)<br>
(drop1): QuantizedDropout(p=0.0, inplace=False)<br>
(fc2): QuantizedLinear(in_features=768, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(drop2): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls2): Identity()<br>
(drop_path2): Identity()<br>
)<br>
(4): Block(<br>
(norm1): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(attn): Attention(<br>
(qkv): QuantizedLinear(in_features=192, out_features=576, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(attn_drop): QuantizedDropout(p=0.0, inplace=False)<br>
(proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(proj_drop): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls1): Identity()<br>
(drop_path1): Identity()<br>
(norm2): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(mlp): Mlp(<br>
(fc1): QuantizedLinear(in_features=192, out_features=768, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(act): GELU(approximate=none)<br>
(drop1): QuantizedDropout(p=0.0, inplace=False)<br>
(fc2): QuantizedLinear(in_features=768, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(drop2): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls2): Identity()<br>
(drop_path2): Identity()<br>
)<br>
(5): Block(<br>
(norm1): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(attn): Attention(<br>
(qkv): QuantizedLinear(in_features=192, out_features=576, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(attn_drop): QuantizedDropout(p=0.0, inplace=False)<br>
(proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(proj_drop): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls1): Identity()<br>
(drop_path1): Identity()<br>
(norm2): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(mlp): Mlp(<br>
(fc1): QuantizedLinear(in_features=192, out_features=768, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(act): GELU(approximate=none)<br>
(drop1): QuantizedDropout(p=0.0, inplace=False)<br>
(fc2): QuantizedLinear(in_features=768, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(drop2): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls2): Identity()<br>
(drop_path2): Identity()<br>
)<br>
(6): Block(<br>
(norm1): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(attn): Attention(<br>
(qkv): QuantizedLinear(in_features=192, out_features=576, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(attn_drop): QuantizedDropout(p=0.0, inplace=False)<br>
(proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(proj_drop): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls1): Identity()<br>
(drop_path1): Identity()<br>
(norm2): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(mlp): Mlp(<br>
(fc1): QuantizedLinear(in_features=192, out_features=768, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(act): GELU(approximate=none)<br>
(drop1): QuantizedDropout(p=0.0, inplace=False)<br>
(fc2): QuantizedLinear(in_features=768, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(drop2): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls2): Identity()<br>
(drop_path2): Identity()<br>
)<br>
(7): Block(<br>
(norm1): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(attn): Attention(<br>
(qkv): QuantizedLinear(in_features=192, out_features=576, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(attn_drop): QuantizedDropout(p=0.0, inplace=False)<br>
(proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(proj_drop): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls1): Identity()<br>
(drop_path1): Identity()<br>
(norm2): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(mlp): Mlp(<br>
(fc1): QuantizedLinear(in_features=192, out_features=768, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(act): GELU(approximate=none)<br>
(drop1): QuantizedDropout(p=0.0, inplace=False)<br>
(fc2): QuantizedLinear(in_features=768, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(drop2): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls2): Identity()<br>
(drop_path2): Identity()<br>
)<br>
(8): Block(<br>
(norm1): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(attn): Attention(<br>
(qkv): QuantizedLinear(in_features=192, out_features=576, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(attn_drop): QuantizedDropout(p=0.0, inplace=False)<br>
(proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(proj_drop): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls1): Identity()<br>
(drop_path1): Identity()<br>
(norm2): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(mlp): Mlp(<br>
(fc1): QuantizedLinear(in_features=192, out_features=768, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(act): GELU(approximate=none)<br>
(drop1): QuantizedDropout(p=0.0, inplace=False)<br>
(fc2): QuantizedLinear(in_features=768, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(drop2): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls2): Identity()<br>
(drop_path2): Identity()<br>
)<br>
(9): Block(<br>
(norm1): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(attn): Attention(<br>
(qkv): QuantizedLinear(in_features=192, out_features=576, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(attn_drop): QuantizedDropout(p=0.0, inplace=False)<br>
(proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(proj_drop): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls1): Identity()<br>
(drop_path1): Identity()<br>
(norm2): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(mlp): Mlp(<br>
(fc1): QuantizedLinear(in_features=192, out_features=768, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(act): GELU(approximate=none)<br>
(drop1): QuantizedDropout(p=0.0, inplace=False)<br>
(fc2): QuantizedLinear(in_features=768, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(drop2): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls2): Identity()<br>
(drop_path2): Identity()<br>
)<br>
(10): Block(<br>
(norm1): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(attn): Attention(<br>
(qkv): QuantizedLinear(in_features=192, out_features=576, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(attn_drop): QuantizedDropout(p=0.0, inplace=False)<br>
(proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(proj_drop): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls1): Identity()<br>
(drop_path1): Identity()<br>
(norm2): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(mlp): Mlp(<br>
(fc1): QuantizedLinear(in_features=192, out_features=768, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(act): GELU(approximate=none)<br>
(drop1): QuantizedDropout(p=0.0, inplace=False)<br>
(fc2): QuantizedLinear(in_features=768, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(drop2): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls2): Identity()<br>
(drop_path2): Identity()<br>
)<br>
(11): Block(<br>
(norm1): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(attn): Attention(<br>
(qkv): QuantizedLinear(in_features=192, out_features=576, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(attn_drop): QuantizedDropout(p=0.0, inplace=False)<br>
(proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(proj_drop): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls1): Identity()<br>
(drop_path1): Identity()<br>
(norm2): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(mlp): Mlp(<br>
(fc1): QuantizedLinear(in_features=192, out_features=768, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(act): GELU(approximate=none)<br>
(drop1): QuantizedDropout(p=0.0, inplace=False)<br>
(fc2): QuantizedLinear(in_features=768, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(drop2): QuantizedDropout(p=0.0, inplace=False)<br>
)<br>
(ls2): Identity()<br>
(drop_path2): Identity()<br>
)<br>
)<br>
(norm): QuantizedLayerNorm((192,), eps=1e-06, elementwise_affine=True)<br>
(fc_norm): Identity()<br>
(head): Identity()<br>
)<br>
(decoder): Decoder(<br>
(layers): ModuleList(<br>
(0): DecoderLayer(<br>
(self_attn): MultiheadAttention(<br>
(out_proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
)<br>
(cross_attn): MultiheadAttention(<br>
(out_proj): QuantizedLinear(in_features=192, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
)<br>
(linear1): QuantizedLinear(in_features=192, out_features=768, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(dropout): QuantizedDropout(p=0.1, inplace=False)<br>
(linear2): QuantizedLinear(in_features=768, out_features=192, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(norm1): QuantizedLayerNorm((192,), eps=1e-05, elementwise_affine=True)<br>
(norm2): QuantizedLayerNorm((192,), eps=1e-05, elementwise_affine=True)<br>
(norm_q): QuantizedLayerNorm((192,), eps=1e-05, elementwise_affine=True)<br>
(norm_c): QuantizedLayerNorm((192,), eps=1e-05, elementwise_affine=True)<br>
(dropout1): QuantizedDropout(p=0.1, inplace=False)<br>
(dropout2): QuantizedDropout(p=0.1, inplace=False)<br>
(dropout3): QuantizedDropout(p=0.1, inplace=False)<br>
)<br>
)<br>
(norm): QuantizedLayerNorm((192,), eps=1e-05, elementwise_affine=True)<br>
)<br>
(head): QuantizedLinear(in_features=192, out_features=95, scale=1.0, zero_point=0, qscheme=torch.per_tensor_affine)<br>
(text_embed): TokenEmbedding(<br>
(embedding): QuantizedEmbedding(num_embeddings=97, embedding_dim=192, dtype=torch.quint8, qscheme=torch.per_channel_affine_float_qparams)<br>
)<br>
(dropout): QuantizedDropout(p=0.1, inplace=False)<br>
)</p>

Vikas_Kumar_Ojha: <p>Was able to convert with pytorch 2.0</p>