Repository: pytorch
Issue: Tracer cannot infer type of Seq2SeqLMOutput
Labels: oncall: jit

### Tracer cannot infer type of Seq2SeqLMOutput

 使用torch.jit.trace对模型进行追踪的时候发生了一个错误
 
错误信息为:
    
    RuntimeError: Tracer cannot infer type of Seq2SeqLMOutput

下面是我这边的代码:
   
     tokenizer = AutoTokenizer.from_pretrained('./outputs/model_files/')
     model = AutoModelForSeq2SeqLM.from_pretrained('./outputs/model_files/')

     device = torch.device("cpu")
     model.to(device)
     model.eval()

     sample_sentence = "generate some numbers"
     encoding = tokenizer(sample_sentence, 
                        padding="max_length",
                        max_length=5,
                        return_tensors="pt",
                        return_attention_mask=True,
                        truncation=True)
     input_ids = encoding.input_ids
     attention_mask = encoding.attention_mask
     decoder_input_ids = torch.ones(1,1, dtype=torch.int32) * model.config.decoder_start_token_id

     traced_model = torch.jit.trace(model, (input_ids,attention_mask,decoder_input_ids),strict=False)
     traced_model.save("./model.pt")

具体的错误为信息:
  
     D:\Program Files\Python310\lib\site-packages\transformers\modeling_utils.py:701: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
     if causal_mask.shape[1] < attention_mask.shape[1]:
     Traceback (most recent call last):
      File "E:\Python\project\Chinese_Chat_T5_Base-main\convertModel.py", line 37, in <module>
    traced_model = torch.jit.trace(model, (input_ids,attention_mask,decoder_input_ids),strict=False)
      File "D:\Program Files\Python310\lib\site-packages\torch\jit\_trace.py", line 759, in trace
    return trace_module(
      File "D:\Program Files\Python310\lib\site-packages\torch\jit\_trace.py", line 976, in trace_module
    module._c._create_method_from_trace(
    RuntimeError: Tracer cannot infer type of Seq2SeqLMOutput(loss=None, logits=tensor([[[-8.0331, -0.6127,  1.7029,  ..., -6.0205, -4.9355, -7.5521]]],
       grad_fn=<UnsafeViewBackward0>), past_key_values=((tensor([[[[-4.1845e-01, -3.1748e+00,  3.5584e-01,  1.3317e-01, -4.8382e-01,
            4.9041e-01,  1.2883e+00,  5.5251e-01,  2.3777e+00,  3.6629e-01,
           -2.3793e-01,  1.6337e+00,  9.4133e-01, -1.0904e+00, -2.8644e+00,
           -5.2565e-02,  2.9996e-01, -4.1858e-01, -7.8744e-01, -1.7734e+00,
           -1.0728e+00,  5.5014e-01, -1.5405e+00,  2.7343e+00,  3.5340e+00,
           -1.5999e-02, -7.7990e-01,  4.5489e-01, -2.4964e-01, -2.9343e-01,
            7.0564e-01,  9.1929e-01,  3.4561e+00, -6.6381e-01,  8.5702e-01,
            6.3156e-01, -7.5711e-01,  1.6548e+00, -8.5602e-01, -9.3094e-01,
            9.1188e-02, -8.6472e-01,  6.4054e-01,  4.7034e-01,  3.4763e+00,
           -1.0079e+00,  1.2279e-01,  1.5227e+00,  1.6583e-01,  9.4017e-01,
            1.5735e+00,  3.4655e-01, -8.0972e-01,  9.2279e-01,  3.1652e-01,
           -2.3178e+00,  5.2484e-02,  4.8382e-01, -1.7146e-01,  2.4539e+00,

    .......

         [-2.7458e-03, -4.8062e-02, -5.2608e-02,  ..., -4.8220e-03,
            5.0419e-02,  2.8005e-03]]]], grad_fn=<TransposeBackward0>))), decoder_hidden_states=None, decoder_attentions=None, cross_attentions=None, encoder_last_hidden_state=tensor([[[-0.0070,  0.1318, -0.0300,  ...,  0.0244, -0.0696,  0.0580],
         [-0.0274,  0.0240, -0.0552,  ..., -0.0846, -0.0992,  0.0408],
         [-0.0647,  0.0068, -0.0779,  ...,  0.0064,  0.0316,  0.0111],
         [-0.0445, -0.0067, -0.0273,  ...,  0.0320,  0.0382,  0.0814],
         [-0.0006,  0.0002,  0.0010,  ..., -0.0002,  0.0009, -0.0009]]],
       grad_fn=<MulBackward0>), encoder_hidden_states=None, encoder_attentions=None)
      :Dictionary inputs to traced functions must have consistent type. Found Tensor and Tuple[Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor], Tuple[Tensor, Tensor, Tensor, Tensor]]

原模型地址为:https://huggingface.co/mxmax/Chinese_Chat_T5_Base

### Versions

torch2.0

cc @EikanWang @jgong5 @wenzhe-nrv @sanchitintel