Error when use quantize and torch.jit.script for transformerencoder

username: chenxie95
cooked: <p>Hi,<br>
I tried to use the Pytorch build-in transformerencoder for my model training. In inference, I tried to quantize it and export it for libtorch in C++. I found there was some strange error when I use both quantize and torch.jit.script. However, if I only use quantize or torch.jit.script, it works well.<br>
I am using Pytorch 1.5.1 and cuda 10.2</p>
<p>The error is as follow when it calls torch.jit.script(module)</p>
<p>RuntimeError:                                                                                                                                                                                                      method cannot be used as a value:                                                                                                                                                                                    File “/usr/local/lib/python3.6/dist-packages/torch/nn/modules/activation.py”, line 831                                                                                                                                           self.in_proj_weight, self.in_proj_bias,                                                                                                                                                                            self.bias_k, self.bias_v, self.add_zero_attn,                                                                                                                                                                      self.dropout, self.out_proj.weight, self.out_proj.bias,                                                                                                                                                                          ~~~~~~~~~~~~~~~~~~~~ &lt;— HERE                                                                                                                                                                       training=self.training,                                                                                                                                                                                            key_padding_mask=key_padding_mask, need_weights=need_weights,</p>
<p>You could reproduce this error easily by running the following sample code.</p>
<p>import torch                                                                                                                                                                                                                                                                                                                                                                                                                          class MyModule(torch.nn.Module):<br>
def <strong>init</strong>(self, hidden_size, nhead):<br>
super(MyModule, self).<strong>init</strong>()<br>
encoder_layer = torch.nn.TransformerEncoderLayer(hidden_size, nhead=nhead,<br>
dim_feedforward=hidden_size, dropout=0.1, activation=‘gelu’)<br>
encoder_norm = torch.nn.LayerNorm(hidden_size)<br>
self.encoder = torch.nn.TransformerEncoder(encoder_layer, 20, encoder_norm)<br>
def forward(self, x):<br>
out = self.encoder(out)<br>
return out</p>
<p>my_module = MyModule(1024, 8)<br>
my_module = torch.quantization.quantize_dynamic(my_module, dtype=torch.qint8)<br>
sm = torch.jit.script(my_module)<br>
torch.jit.save(sm, ‘test.jit.model’)</p>