Jit trace failure (First diverging operator Node diff)

username: hongxue.org
cooked: <p>I have a problem with torch.jit.trace execution failure.</p>
<p>First diverging operator:<br>
Node diff:<br>
- %transformer : <strong>torch</strong>.torch.nn.modules.transformer.Transformer = prim::GetAttr<a>name=“transformer”</a><br>
+ %transformer : <strong>torch</strong>.torch.nn.modules.transformer.___torch_mangle_297.Transformer = prim::GetAttr<a>name=“transformer”</a><br>
?                                                       ++++++++++++++++++++</p>
<blockquote>
<h1>encoding:utf-8</h1>
<p>import math</p>
<p>import torch<br>
import torch.nn as nn<br>
from tokenizers import Tokenizer<br>
from pathlib import Path<br>
from torch.utils.mobile_optimizer import optimize_for_mobile</p>
<p>device = torch.device(‘cpu’)</p>
<p>max_length = 72</p>
<p>print(torch.<strong>version</strong>)</p>
<p>class PositionalEncoding(nn.Module):</p>
<pre><code>def __init__(self, d_model, dropout, max_len=5000):
    super(PositionalEncoding, self).__init__()
    self.dropout = nn.Dropout(p=dropout)

    pe = torch.zeros(max_len, d_model).to(device)
    position = torch.arange(0, max_len).unsqueeze(1)
    div_term = torch.exp(torch.arange(0, d_model, 2) * -(math.log(10000.0) / d_model))
    pe[:, 0::2] = torch.sin(position * div_term)
    pe[:, 1::2] = torch.cos(position * div_term)
    pe = pe.unsqueeze(0)
    self.register_buffer("pe", pe)

def forward(self, x):
    x = x + self.pe[:, : x.size(1)].requires_grad_(False)
    return self.dropout(x)
</code></pre>
<p>class TranslationModel(nn.Module):</p>
<pre><code>def __init__(self, d_model, src_vocab, tgt_vocab, dropout=0.1):
    super(TranslationModel, self).__init__()

    self.src_embedding = nn.Embedding(len(src_vocab), d_model, padding_idx=2)
    self.tgt_embedding = nn.Embedding(len(tgt_vocab), d_model, padding_idx=2)
    self.positional_encoding = PositionalEncoding(d_model, dropout, max_len=max_length)
    self.transformer = nn.Transformer(d_model, dropout=dropout, batch_first=True)
    self.predictor = nn.Linear(d_model, len(tgt_vocab))

def forward(self, src, tgt):
    tgt_mask = nn.Transformer.generate_square_subsequent_mask(tgt.size()[-1])
    src_key_padding_mask = TranslationModel.get_key_padding_mask(src)
    tgt_key_padding_mask = TranslationModel.get_key_padding_mask(tgt)
    src = self.src_embedding(src)
    tgt = self.tgt_embedding(tgt)
    src = self.positional_encoding(src)
    tgt = self.positional_encoding(tgt)

    out = self.transformer(src, tgt, tgt_mask=tgt_mask, src_key_padding_mask=src_key_padding_mask,
                           tgt_key_padding_mask=tgt_key_padding_mask)

    return out

@staticmethod
def get_key_padding_mask(tokens):
    return tokens == 2
</code></pre>
<h1>加载基础的分词器模型，使用的是基础的bert模型。<code>uncased</code>意思是不区分大小写</h1>
<p>tokenizer = Tokenizer.from_pretrained(“bert-base-uncased”)</p>
<p>def en_tokenizer(line):<br>
return tokenizer.encode(line, add_special_tokens=False).tokens</p>
<p>work_dir = Path(“./dataset”)<br>
en_vocab = None<br>
zh_vocab = None<br>
en_vocab_file = work_dir / “vocab_en.pt”<br>
zh_vocab_file = work_dir / “vocab_zh.pt”<br>
en_vocab = torch.load(en_vocab_file, map_location=“cpu”)<br>
zh_vocab = torch.load(zh_vocab_file, map_location=“cpu”)</p>
<p>model_dir = Path(“./drive/MyDrive/model/transformer_checkpoints”)<br>
model_checkpoint = “model_475000.pt”</p>
<p>model = torch.load(model_dir / model_checkpoint, map_location=torch.device(‘cpu’))<br>
model = model.to(device)</p>
<p>model = model.eval()</p>
<p>src = torch.tensor([0, 163, 1]).unsqueeze(0).to(device)<br>
tgt = torch.tensor([[0]]).to(device)</p>
<p>quantized_model = torch.quantization.convert(model)</p>
<p>scripted_module = torch.jit.trace(quantized_model, (tgt, src))<br>
opt_module = optimize_for_mobile(scripted_module)<br>
opt_module._save_for_lite_interpreter(“model_quantized.ptl”)</p>
</blockquote>

username: tzuching916
cooked: <p>Hi Hongxue,<br>
Did u find the solution to solve the graph_diff_errors?<br>
I’m having the same problem as well while trying to tune my model through optuna.<br>
Thx</p>