T5 AttributeError: 'T5Encoder' object has no attribute 'main_input_name'
Author: weilixiang
Pull request: no

I trained a T5 model and when I ran the inference it gave me error as mentioned in title. I installed required package as shown below. It was working yesterday but now it returned me this attribute error.

 ```
!pip install sentencepiece==0.1.95
!pip install torch
!pip install transformers
!pip install pytorch_lightning==0.7.5
```
```py
import torch
from transformers import T5ForConditionalGeneration,T5Tokenizer

def get_model_output(sentence,model):
    c_output = []
    text =  sentence + ' 
'

    max_len = 256

    encoding = tokenizer.encode_plus(text,pad_to_max_length=True, return_tensors="pt")
    input_ids, attention_masks = encoding["input_ids"].to("cpu"), encoding["attention_mask"].to("cpu")


    # set top_k = 50 and set top_p = 0.95 and num_return_sequences = 3
    beam_outputs= model.generate(
          input_ids=input_ids, attention_mask=attention_masks,
          max_length=64,
          num_beams=10, 
          no_repeat_ngram_size=2, 
          do_sample = False,
          num_return_sequences=5, 
          early_stopping=True
      )
    final_outputs =[]
    for beam_output in beam_outputs:
        sent = tokenizer.decode(beam_output, skip_special_tokens=True,clean_up_tokenization_spaces=True)
        if sent.lower() != sentence.lower() and sent not in final_outputs:
            final_outputs.append(sent)

    for final_output in final_outputs:
        c_output.append(final_output)
    return `c_output[0]`
```

Here is the error message: 
```
[/usr/local/lib/python3.7/dist-packages/transformers/generation_utils.py](https://localhost:8080/#) in generate(self, inputs, max_length, min_length, do_sample, early_stopping, num_beams, temperature, top_k, top_p, repetition_penalty, bad_words_ids, bos_token_id, pad_token_id, eos_token_id, length_penalty, no_repeat_ngram_size, encoder_no_repeat_ngram_size, num_return_sequences, max_time, max_new_tokens, decoder_start_token_id, use_cache, num_beam_groups, diversity_penalty, prefix_allowed_tokens_fn, logits_processor, stopping_criteria, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, forced_bos_token_id, forced_eos_token_id, remove_invalid_values, synced_gpus, **model_kwargs)
   1067         # otherwise model_input_name is None
   1068         # all model-specific keyword inputs are removed from `model_kwargs`
-> 1069         inputs_tensor, model_input_name, model_kwargs = self._prepare_model_inputs(inputs, bos_token_id, model_kwargs)
   1070         batch_size = inputs_tensor.shape[0]
   1071 

[/usr/local/lib/python3.7/dist-packages/transformers/generation_utils.py](https://localhost:8080/#) in _prepare_model_inputs(self, inputs, bos_token_id, model_kwargs)
    392             self.config.is_encoder_decoder
    393             and hasattr(self, "encoder")
--> 394             and self.encoder.main_input_name != self.main_input_name
    395         ):
    396             input_name = self.encoder.main_input_name

[/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py](https://localhost:8080/#) in __getattr__(self, name)
   1176                 return modules[name]
   1177         raise AttributeError("'{}' object has no attribute '{}'".format(
-> 1178             type(self).__name__, name))
   1179 
   1180     def __setattr__(self, name: str, value: Union[Tensor, 'Module']) -> None:

AttributeError: 'T5Encoder' object has no attribute 'main_input_name'
```
Can anyone suggest? Thanks in advance!

Comments:
try to Reduce the version of transformers

Hey, there is some information lacking. Could you please provide:
- A fully working code example: there is no call to `get_model_output` so we're unaware of what is the model and the sentence
- The result of `transformers-cli env`
- What result did you expect

Thank you.

I got the same issue,the version of my transformer is 4.16.2.
Solved the issue by reducing the version to 4.10.0.


> I got the same issue,the version of my transformer is 4.16.2.
> Solved the issue by reducing the version to 4.10.0.

Thanks a lot.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi, I meet the same problem for transformers 4.17.0. However, the rest of the project depends on newer transformers version. How should I change the code accordingly to avoid this error?

Reduce version of transformers


发自我的iPhone

------------------ Original ------------------
From: Yuqing Xie ***@***.***&gt;
Date: 周四,8月 25,2022 08:13
To: huggingface/transformers ***@***.***&gt;
Cc: yuanhuachao ***@***.***&gt;, Comment ***@***.***&gt;
Subject: Re: [huggingface/transformers] T5 AttributeError: 'T5Encoder' object has no attribute 'main_input_name' (Issue #15610)





 
Hi, I meet the same problem for transformers 4.17.0. However, the rest of the project depends on newer transformers version. How should I change the code accordingly to avoid this error?
 
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: ***@***.***&gt;

> However, the rest of the project depends on newer transformers version. How should I change the code accordingly to avoid this error?

I met the same issue. Have you solved it? 😭 

I have had a lot of trouble trying to get `fastT5` to work. Eventually, this issue kept coming up. It took a while to figure out exactly what I needed to just run the conversion from T5 to ONNX. Like @magichz86 noted, downgrading transformers works. I took these steps:
```
conda create -n fastT5
```
```
conda activate fastT5
```
```
conda install python=3.9 
```
```
pip install transformers==4.10.0
```
```
pip install fastT5
```
I had to use a python environment just for these two libraries, as there were conflicts with others. Also, python 3.9 or lower is needed.

I meet this problem when i study FiD project. But I can't reduce the version of transformers, beacuse i need transformers>=4.30. So i solve this problem by add the following code:
` self.main_input_name = encoder.main_input_name  # input_ids`
the all code is as follow:
`class EncoderWrapper(torch.nn.Module):

    def __init__(self, encoder, use_checkpoint=False):
        super().__init__()

       # key code
        self.main_input_name = encoder.main_input_name

        self.encoder = encoder
        apply_checkpoint_wrapper(self.encoder, use_checkpoint)

    def forward(self, input_ids=None, attention_mask=None, **kwargs, ):
        # total_length = n_passages * passage_length
        bsz, total_length = input_ids.shape
        passage_length = total_length // self.n_passages
        input_ids = input_ids.view(bsz * self.n_passages, passage_length)
        attention_mask = attention_mask.view(bsz * self.n_passages, passage_length)
        outputs = self.encoder(input_ids, attention_mask, **kwargs)
        outputs = (outputs[0].view(bsz, self.n_passages * passage_length, -1),) + outputs[1:]
        return outputs

`