AttributeError: 'TrainingArguments' object has no attribute 'generation_max_length'
Author: CaffreyR
Pull request: no

### System Info

```shell
transformers :4.20.1
platform: Colab
python : 3.7
```


### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

"ade_corpus_v2" in Huggingface RAFT
### Reproduction

```
def compute_metrics(p):
    pred, labels = p
    pred = np.argmax(pred, axis=1)

    accuracy = accuracy_score(y_true=labels, y_pred=pred)
    recall = recall_score(y_true=labels, y_pred=pred)
    precision = precision_score(y_true=labels, y_pred=pred)
    f1 = f1_score(y_true=labels, y_pred=pred)

    return {"accuracy": accuracy, "precision": precision, "recall": recall, "f1": f1}

# Define Trainer
args = TrainingArguments(
    output_dir="output",
    evaluation_strategy="steps",
    eval_steps=500,
    per_device_train_batch_size=8,
    per_device_eval_batch_size=8,
    num_train_epochs=100,
    seed=0,
    load_best_model_at_end=True,
)
from transformers import Seq2SeqTrainer
trainer = Seq2SeqTrainer(
#     model=delta_model3,
    model=model,
    args=args,
    train_dataset=train_dataset,
    eval_dataset=val_dataset,
    compute_metrics=compute_metrics,
    callbacks=[EarlyStoppingCallback(early_stopping_patience=3)],
)

# Train pre-trained model
trainer.train()
```

### Expected behavior

```shell
It went wrong.
>
/usr/local/lib/python3.7/dist-packages/transformers/optimization.py:310: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning
  FutureWarning,
***** Running training *****
  Num examples = 40
  Num Epochs = 100
  Instantaneous batch size per device = 8
  Total train batch size (w. parallel, distributed & accumulation) = 8
  Gradient Accumulation steps = 1
  Total optimization steps = 500
 [500/500 06:18, Epoch 100/100]
Step	Training Loss	Validation Loss
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-11-52043b3bb24a> in <module>()
     33 
     34 # Train pre-trained model
---> 35 trainer.train()

3 frames
/usr/local/lib/python3.7/dist-packages/transformers/trainer.py in train(self, resume_from_checkpoint, trial, ignore_keys_for_eval, **kwargs)
   1411             resume_from_checkpoint=resume_from_checkpoint,
   1412             trial=trial,
-> 1413             ignore_keys_for_eval=ignore_keys_for_eval,
   1414         )
   1415 

/usr/local/lib/python3.7/dist-packages/transformers/trainer.py in _inner_training_loop(self, batch_size, args, resume_from_checkpoint, trial, ignore_keys_for_eval)
   1726                     self.control = self.callback_handler.on_step_end(args, self.state, self.control)
   1727 
-> 1728                     self._maybe_log_save_evaluate(tr_loss, model, trial, epoch, ignore_keys_for_eval)
   1729                 else:
   1730                     self.control = self.callback_handler.on_substep_end(args, self.state, self.control)

/usr/local/lib/python3.7/dist-packages/transformers/trainer.py in _maybe_log_save_evaluate(self, tr_loss, model, trial, epoch, ignore_keys_for_eval)
   1910         metrics = None
   1911         if self.control.should_evaluate:
-> 1912             metrics = self.evaluate(ignore_keys=ignore_keys_for_eval)
   1913             self._report_to_hp_search(trial, epoch, metrics)
   1914 

/usr/local/lib/python3.7/dist-packages/transformers/trainer_seq2seq.py in evaluate(self, eval_dataset, ignore_keys, metric_key_prefix, max_length, num_beams)
     66             dictionary also contains the epoch number which comes from the training state.
     67         """
---> 68         self._max_length = max_length if max_length is not None else self.args.generation_max_length
     69         self._num_beams = num_beams if num_beams is not None else self.args.generation_num_beams
     70         return super().evaluate(eval_dataset, ignore_keys=ignore_keys, metric_key_prefix=metric_key_prefix)

AttributeError: 'TrainingArguments' object has no attribute 'generation_max_length'
```
@sgugger 

### Checklist

- [X] I have read the migration guide in the readme. ([pytorch-transformers](https://github.com/huggingface/transformers#migrating-from-pytorch-transformers-to-transformers); [pytorch-pretrained-bert](https://github.com/huggingface/transformers#migrating-from-pytorch-pretrained-bert-to-transformers))
- [X] I checked if a related official extension example runs on my machine.

Comments:
You need to use `Seq2SeqTrainingArguments` to go with `Seq2SeqTrainer`.

> You need to use `Seq2SeqTrainingArguments` to go with `Seq2SeqTrainer`.
Well the thing is when I try to use `Seq2SeqTrainingArguments`, and the dataset value is tensor, it can train. However when I tried to use `TrainingArguments`, and the dataset value is not tensor, it can train.

But whatever way I code, the training loss is missed until the last epoch. 

What is the tricky here. Thanks! @sgugger 

```
### dataset change
import torch
class Dataset(torch.utils.data.Dataset):
    def __init__(self, encodings, labels=None):
        self.encodings = encodings
        self.labels = labels

    def __getitem__(self, idx):
        item = {key: torch.tensor(val[idx]) for key, val in self.encodings.items()}
        if self.labels:
            item["labels"] = torch.tensor(self.labels[idx]-1)
        return item

    def __len__(self):
        return len(self.encodings["input_ids"])

import torch
class Dataset(torch.utils.data.Dataset):
    def __init__(self, encodings, labels=None):
        self.encodings = encodings
        self.labels = labels

    def __getitem__(self, idx):
        item = {key: val[idx] for key, val in self.encodings.items()}
        if self.labels:
            item["labels"] = self.labels[idx]-1
        return item

    def __len__(self):
        return len(self.encodings["input_ids"])

train_dataset = Dataset(X_train_tokenized, y_train)
val_dataset = Dataset(X_val_tokenized, y_val)


import numpy as np
from sklearn.metrics import accuracy_score, recall_score, precision_score, f1_score
def compute_metrics(pred):
    print(pred)
    predict_res = torch.Tensor(pred.predictions[0]) # size：[验证集样本量, label的token长度, vocab大小]

    pred_ids = predict_res.argmax(dim=2)
 
    ## 2.处理 pred.label_ids
    labels_actual = torch.LongTensor(pred.label_ids)
    
    ## 3.计算accuracy
    total_num = labels_actual.shape[0]
    acc = torch.sum(torch.all(torch.eq(pred_ids, labels_actual), dim=1))/total_num
    return {'accuracy': acc}


# Define Trainer
args = Seq2SeqTrainingArguments(
    output_dir="output",
    evaluation_strategy="steps",
    eval_steps=25,
    per_device_train_batch_size=8,
    per_device_eval_batch_size=8,
    num_train_epochs=100,
    seed=0,
    load_best_model_at_end=True,
)
trainer = Trainer(
#     model=delta_model3,
    model=model,
    args=args,
    train_dataset=train_dataset,
    eval_dataset=val_dataset,
    compute_metrics=compute_metrics,
    callbacks=[EarlyStoppingCallback(early_stopping_patience=3)],
)

# Train pre-trained model
trainer.train()
```

> The trainning log is as bellow
<img width="450" alt="image" src="https://user-images.githubusercontent.com/84232793/178263827-c967b616-6279-484a-a265-5353b7241687.png">


Please use the [forums](https://discuss.huggingface.co/) to debug your code as we keep issues for bugs and feature requests only. You didn't indicate you want to log the training loss every 25 steps in your training arguments, so it uses the default of 500.

> Please use the [forums](https://discuss.huggingface.co/) to debug your code as we keep issues for bugs and feature requests only. You didn't indicate you want to log the training loss every 25 steps in your training arguments, so it uses the default of 500.

Sorry about that! I will use the forums in the future! Thanks

i have the same problem here how did you resolve this issue