mxmax/Chinese_Chat_T5_Base 模型怎么用torch.jit.trace追踪
Author: ling976
Pull request: no

我使用下面的代码进行模型转换

    tokenizer = AutoTokenizer.from_pretrained('./outputs/model_files/')
    model = AutoModelForSeq2SeqLM.from_pretrained('./outputs/model_files/')
    device = torch.device("cpu")
    model.to(device)
    model.eval()

    tokenized_dict = tokenizer(
        ["please answer the following question: what is the boiling point of nitrogen",], ["-320.4F",], 
        return_tensors="pt"
    )

    input_tuple = (tokenized_dict['input_ids'], tokenized_dict['attention_mask'], torch.Tensor([[2]]).long())
    traced_model = torch.jit.trace(model, input_tuple)
    traced_model.save("./model.pt")

但是得到这样一堆错误信息

       D:\Program Files\Python310\lib\site-packages\transformers\modeling_utils.py:701: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
       if causal_mask.shape[1] < attention_mask.shape[1]:
       Traceback (most recent call last):
       File "E:\Python\project\Chinese_Chat_T5_Base-main\convertModel.py", line 25, in <module>
    traced_model = torch.jit.trace(model, input_tuple)
      File "D:\Program Files\Python310\lib\site-packages\torch\jit\_trace.py", line 759, in trace
    return trace_module(
      File "D:\Program Files\Python310\lib\site-packages\torch\jit\_trace.py", line 976, in trace_module
    module._c._create_method_from_trace(
    RuntimeError: Tracer cannot infer type of Seq2SeqLMOutput(loss=None, logits=tensor([[[-10.4197,   6.3242,   8.7392,  ..., -10.0839,  -7.8809,  -8.4109]]],
       grad_fn=<UnsafeViewBackward0>), past_key_values=((tensor([[[[-9.3662e-02, -2.6494e-01,  2.7725e-01,  3.5019e-01,  5.3944e-01,
           -2.6313e-01, -5.9071e-01,  5.1579e-01, -5.2901e-01, -5.9420e-01,
           -9.2730e-02,  1.2436e-03, -8.6124e-01, -1.4801e-01, -6.9207e-01,
       ......

    [ 2.7600e-02, -2.4005e-02, -7.1618e-02,  ...,  1.9455e-01,
           1.0591e-02, -8.1877e-02],
         [ 5.6630e-02, -2.8372e-03,  3.5540e-02,  ...,  1.0443e-01,
           3.7175e-02, -5.7037e-02],
         [-5.6965e-04,  1.0548e-04,  9.4504e-04,  ..., -1.7588e-04,
           8.6722e-04, -8.3949e-04]]], grad_fn=<MulBackward0>), encoder_hidden_states=None, encoder_attentions=None)
    :Dictionary inputs to traced functions must have consistent type. Found Tensor and Tuple[Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor]]
  
   

Comments:
Hi @ling976, thanks for raising this issue! 

Unfortunately, I don't speak Chinese :/ , is it possible to share the issue description in english? 

Could you also follow the issue template and share information such that this can be reproduced, including: 
* The running environment: run `transformers-cli env` in the terminal and copy-paste the output
* The line in the code the error is triggered on - is it the model save? 
* The checkpoint or architecture being run? 

transformers 版本是4.22.1
我现在需要将.bin格式的模型文件转换成.pt格式.
现在得到一个错误信息

    File "D:\Program Files\Python310\lib\site-packages\torch\jit\_trace.py", line 976, in trace_module
    module._c._create_method_from_trace(
    RuntimeError: Tracer cannot infer type of Seq2SeqLMOutput(loss=None, logits=tensor([[[-8.0331, -0.6127,  1.7029,  ..., -6.0205, -4.9355, -7.5521]]],

@ling976 Please try passing `torchscript=True` as an argument when loading the model i.e `model = AutoModelForSeq2SeqLM.from_pretrained('./outputs/model_files/', torchscript=True)`



我加上torchscript=True后报了一个新的错误信息
   
    D:\Program Files\Python310\lib\site-packages\transformers\modeling_utils.py:701: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
      if causal_mask.shape[1] < attention_mask.shape[1]:

@ling976 [As mentioned above](https://github.com/huggingface/transformers/issues/22925#issuecomment-1518062046), could you please follow the issue template and the necessary information such that we can replicate the issue? 

这个已经可以了,前面的试过警告而不是错误,是我看错了

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.