Try to use torchscript on torch.nn.transformer

username: huoge
cooked: <p>Hello everyone. I attempt to use torch.jit.script on the torch.nn.transformer, but it doesn’t work。<br>
Has anyone ever done any related work?</p>
<p>I build the pytorch from source and the torch version is 1.4.0a0+2e7dd54</p>
<p>I’d appreciate if anybody can help me! Or if there is a workable implementation, please let me know! Thanks in advance!</p>
<p>here is the code:</p>
<blockquote>
<p>import torch<br>
import torch.nn as nn<br>
torch.manual_seed(2)<br>
transformer_model = nn.Transformer(nhead=16, num_encoder_layers=12)<br>
trans_model = torch.jit.script(transformer_model)<br>
trans_model.save(‘test.pt’)</p>
</blockquote>
<p>and here is the log:</p>
<blockquote>
<p>Traceback (most recent call last):<br>
File “transformer_demo.py”, line 13, in <br>
trans_model = torch.jit.script(transformer_model)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/<strong>init</strong>.py”, line 1239, in script<br>
return torch.jit.torch.jit._recursive.recursive_script(obj)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 508, in recursive_script<br>
return create_script_module(nn_module, infer_methods_to_compile(nn_module))<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 305, in create_script_module<br>
concrete_type = concrete_type_store.get_or_create_concrete_type(nn_module)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 243, in get_or_create_concrete_type<br>
raw_concrete_type = infer_raw_concrete_type(nn_module)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 91, in infer_raw_concrete_type<br>
sub_concrete_type = concrete_type_store.get_or_create_concrete_type(item)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 243, in get_or_create_concrete_type<br>
raw_concrete_type = infer_raw_concrete_type(nn_module)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 91, in infer_raw_concrete_type<br>
sub_concrete_type = concrete_type_store.get_or_create_concrete_type(item)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 240, in get_or_create_concrete_type<br>
scripted = create_constant_iterable_module(nn_module)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 539, in create_constant_iterable_module<br>
modules[key] = recursive_script(submodule)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 508, in recursive_script<br>
return create_script_module(nn_module, infer_methods_to_compile(nn_module))<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 308, in create_script_module<br>
return create_script_module_impl(nn_module, concrete_type, cpp_module, stubs)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 358, in create_script_module_impl<br>
script_module = torch.jit.RecursiveScriptModule._construct(cpp_module, init_fn)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/<strong>init</strong>.py”, line 1612, in _construct<br>
init_fn(script_module)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 340, in init_fn<br>
scripted = recursive_script(orig_value)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 508, in recursive_script<br>
return create_script_module(nn_module, infer_methods_to_compile(nn_module))<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 308, in create_script_module<br>
return create_script_module_impl(nn_module, concrete_type, cpp_module, stubs)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 362, in create_script_module_impl<br>
create_methods_from_stubs(concrete_type, stubs)<br>
File “/home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/jit/_recursive.py”, line 268, in create_methods_from_stubs<br>
concrete_type._create_methods(defs, rcbs, defaults)<br>
RuntimeError:<br>
Module ‘MultiheadAttention’ has no attribute ‘q_proj_weight’ :<br>
at /home/anaconda3/envs/py37/lib/python3.7/site-packages/torch/nn/modules/activation.py:771:30<br>
key_padding_mask=key_padding_mask, need_weights=need_weights,<br>
attn_mask=attn_mask, use_separate_proj_weight=True,<br>
q_proj_weight=self.q_proj_weight, k_proj_weight=self.k_proj_weight,<br>
~~~~~~~~~~~~~~~~~~ &lt;— HERE<br>
v_proj_weight=self.v_proj_weight)<br>
else:</p>
</blockquote>

username: driazati
cooked: <p>This was brought up <a href="https://github.com/pytorch/pytorch/issues/24173" rel="nofollow noopener">here</a> but the fix <a href="https://github.com/pytorch/pytorch/pull/24204" rel="nofollow noopener">here</a> never landed due to backwards-compatibility issues, I can try to get it through soon.</p>

username: huoge
cooked: <p>ok, thanks! I tried the statement  ‘self.register_parameter(‘q_proj_weight’, None)’，but it seems doesn’t work。</p>

username: driazati
cooked: <p>It turns out there are some other fixes needed, see these two PRs for details (<a href="https://github.com/pytorch/pytorch/pull/28555" rel="nofollow noopener">#28555</a> and <a href="https://github.com/pytorch/pytorch/pull/28561" rel="nofollow noopener">#28561</a>). If you need it now you can check out <span class="hashtag">#28561</span> and <a href="https://github.com/pytorch/pytorch#from-source" rel="nofollow noopener">build from source</a>, otherwise it should be in the nightly package in a few days.</p>

username: huoge
cooked: <p>Thx a lot. it’s really helpful to me.</p>