Trainer.__init__() got an unexpected keyword argument 'model_flops'
Author: andysingal
Pull request: no

### System Info

colab notebook

### Who can help?

@ArthurZucker @pacman100 @you

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

During Andrew Ng new course on fine tuning LLM
Dataset: Andyrasika/llama_dataset
they used:
```
training_config = {
    "model": {
        "pretrained_name": model_name,
        "max_length" : 2048
    },
    "datasets": {
        "use_hf": use_hf,
        "path": dataset
    },
    "verbose": True
}

```
```
model_flops = (
  base_model.floating_point_ops(
    {
       "input_ids": torch.zeros(
           (1, training_config["model"]["max_length"])
      )
    }
  )
  * training_args.gradient_accumulation_steps
)

print(base_model)
print("Memory footprint", base_model.get_memory_footprint() / 1e9, "GB")
print("Flops", model_flops / 1e9, "GFLOPs")
```
and further
```
trainer = Trainer(
    model=base_model,
    model_flops=model_flops,
    total_steps=max_steps,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=test_dataset,
)
```
i get the following error:
```
trainer = Trainer(
    model=base_model,
    model_flops=model_flops,
    total_steps=max_steps,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=test_dataset,
)

```
i have seen model_flops but since these courses are in collabration with huggingface, wanted to check if this something new?
```
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
[<ipython-input-69-0c6be6067a75>](https://localhost:8080/#) in <cell line: 1>()
----> 1 trainer = Trainer(
      2     model=base_model,
      3     model_flops=model_flops,
      4     total_steps=max_steps,
      5     args=training_args,

TypeError: Trainer.__init__() got an unexpected keyword argument 'model_flops'
```

### Expected behavior

run normally

Comments:
I can't see any `model_flops` argument in the codebase. Is this `Trainer` from `transformers` library?

> I can't see any `model_flops` argument in the codebase. Is this `Trainer` from `transformers` library?

this is part of Andrew Ng course released yesterday: https://learn.deeplearning.ai/finetuning-large-language-models/lesson/6/training-process. 
<img width="1847" alt="Screenshot 2023-08-24 at 7 58 43 PM" src="https://github.com/huggingface/transformers/assets/20493493/0f57c3d4-5acf-4580-badb-fe48462cbfee">
@ydshieh @pacman100 @ArthurZucker 

You should post the question on that course forum. There is no `model_flops` argument in `transformers` codebase.

They probably want to put `model_flops` in `training_args` and not pass directly to the Trainer's `__init__`. Only they know better their usage.

Thanks for your reply, I already did. I wanted run it by your team to see
if this is something in your pipeline. Advantages of doing this is it
quantizes during training . Anyways, if you find something that’s good
otherwise you can close the ticket.

On Thu, Aug 24, 2023 at 20:57 Yih-Dar ***@***.***> wrote:

> You should post the question on that course forum. There is no model_flops
> argument in transformers codebase.
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/25724#issuecomment-1691897501>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AE4LJNIYEB6PY5DAWYUQAC3XW5XETANCNFSM6AAAAAA342OOII>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>


How to solve this problem?

where is code after the correction???


I got one possible solution from here: https://medium.com/@balci.pelin/llm-finetuning-410e8a2738ef

```python
trainer = Trainer(
    model= model,
    # model_flops=model_flops,
    # total_steps=max_steps,
    args=training_args,
    train_dataset=tokenized_dataset,
    eval_dataset=tokenized_dataset,
)
```