AttributeError: 'MarianMTModel' object has no attribute 'generation_config'
Author: kigenchesire
Pull request: no

tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-swc")
def translate(text):
    translated = model22.generate(**tokenizer(text, return_tensors="pt", padding=True).to("cpu"))
    return [tokenizer.decode(t, skip_special_tokens=True) for t in translated][0]

Error when trying to deploy model on streamlit

Comments:
Hi @kigenchesire, thanks for raising an issue! 

So that we can best help you, can you make sure to follow the issue template and share: 
* A full traceback of the error
* A code example that we can run to reproduce the error
* The running environment: run `transformers-cli env` in the terminal and copy-paste the output

I am finetuning a translation model using pytorch 
after fine-tuning and saving it using torch.save(model , 'model24.pt')
when i try to deploy the model on streamlit  using this code 
tokenizer = AutoTokenizer.from_pretrained("Helsinki-NLP/opus-mt-en-swc")
def translate(text):
translated = model22.generate(**tokenizer(text, return_tensors="pt", padding=True).to("cpu"))
return [tokenizer.decode(t, skip_special_tokens=True) for t in translated][0
I run into [MarianMTModel' object has no attribute 'generation_config']
The model does well on colab notebook

@kigenchesire When saving a transformers model, it's recommended to use `model.save_pretrained(checkpoint_name)`. This ensures everything, including any necessary files such as the model config are saved alongside the weights. 


TL;DR; `pip install transformers<4.28`


---

With different project, had same error message.


(Maybe unexpectedly?) It seems to break backward compatibility, so that makes issue on several project that was built on old `TrainingArguments`.

In my case, with https://github.com/alexa/massive, error log was like

```
Traceback (most recent call last):
  File "massive/scripts/train.py", line 102, in <module>
    main()
  File "massive/scripts/train.py", line 89, in main
    trainer = trainer_cls(
  File "massive/src/massive/utils/trainer.py", line 264, in __init__
    super().__init__(*args, **kwargs)
  File "transformers/trainer_seq2seq.py", line 72, in __init__
    if self.args.generation_config is not None:
AttributeError: 'MASSIVETrainingArguments' object has no attribute 'generation_config'
```

I found `generation_config` concept(?) created with https://github.com/huggingface/transformers/commit/5506d0496957cde19318eee3d34ee682b654abe8, which is I think the cause of this issue.

So one who suffers from with this issue, use `transformers<4.28`, which means before the first version applied above commit.

Also I suggest `transformers` project to check whether `generation_config` attribute even exists before checking it is `None` or not, if maintainers think it is required change to keep backward compatibility or to be a bit more safer code :)

https://github.com/huggingface/transformers/blob/46d2468695a85dfcc2be0524caa912edefcf2391/src/transformers/trainer_seq2seq.py#L72

Of course user may use `Seq2SeqTrainingArguments` to use with `Seq2SeqTrainer`, not `TrainingArguments`.


@cgbahk It don't believe it's necessary to downgrade the transformers version. If there's a model which was created before generation configs were introduced, then you can load and resave and the generation config will be created e.g. for MarianMT 

```python
from transformers import MarianMTModel

model = MarianMTModel.from_pretrained(my_checkpoint)
model.save_pretrained(my_checkpoint)
```

@gante Is this correct? Are there any other considerations for the generation config? 

Oh, I didn't know `MarianMTModel` is of transformer builtin. In my case, `MASSIVETrainingArguments` is custom built class which don't know about `generation_config`.

I don't fully understand, transformers internal, but https://github.com/huggingface/transformers/issues/22939#issuecomment-1550749034 resolved my case. Hopefully that resonates with any other who encounters same error :)

@cgbahk No worries - it's a big library! And commenting on what resolves issues is useful for everyone :) 

To your first comment, yes, we certainly want to make sure things are backwards compatible. In this case, it seems that the docs aren't clear. It is recommended to use `Seq2SeqTrainingArguments` for `Seq2SeqTrainer`, however the `args` input type is listed as `TrainingArguments` [here](https://huggingface.co/docs/transformers/v4.29.1/en/main_classes/trainer#transformers.Seq2SeqTrainer). For `MASSIVETrainingArguments` - subclassing from `Seq2SeqTrainingArguments` should be enough to resolve the issue if doing seq2seq training. 

Came here to give pretty much the same reply as @amyeroberts just did :) 

Also, we can't ensure retrocompatibility when classes get overwritten, it's impossible to anticipate all changes in advance 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.