GLUs misleadingly named MLP
Author: emergenz
Pull request: no

### System Info

- `transformers` version: 4.40.0
- Platform: Linux-4.18.0-513.11.1.el8_9.x86_64-x86_64-with-glibc2.28
- Python version: 3.11.9
- Huggingface_hub version: 0.22.2
- Safetensors version: 0.4.3
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.2.2+cu121 (True)

### Who can help?

@ArthurZucker @younesbelkada 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Throughout the `transformers` repository, GLU classes are incorrectly and misleadingly named as `ModelMLP`. Examples for this are:
https://github.com/huggingface/transformers/blob/84c4b72ee99e8e65a8a5754a5f9d6265b45cf67e/src/transformers/models/llama/modeling_llama.py#L188
https://github.com/huggingface/transformers/blob/84c4b72ee99e8e65a8a5754a5f9d6265b45cf67e/src/transformers/models/qwen2/modeling_qwen2.py#L173
https://github.com/huggingface/transformers/blob/84c4b72ee99e8e65a8a5754a5f9d6265b45cf67e/src/transformers/models/mistral/modeling_mistral.py#L160

### Expected behavior

GLUs are not MLPs. Therefore, I either expect these classes to be named `ModelGLU`, or more generally `ModelFeedForward`.

What are the maintainer's opinions on this? I am happy to open a PR.

Comments:
Hi @emergenz 
Technically you are right, those classes correspond to Gated Linear Units. We could rename them but we need to be careful with public classes as these standalone classes are used by many codebases, e.g.: [here](https://github.com/search?q=from+transformers.models.llama.modeling_llama+import+LlamaMLP&type=code) to be renamed in order to not affect these - this might introduce some complexity in the modeling codebase so I will let the core maintainers decide on this @amyeroberts @ArthurZucker 


I don't find it misleading at all TBH

> I don't find it misleading at all TBH

@ArthurZucker Can you elaborate? GLUs are not MLPs. In fact, I was looking at the `modeling_qwen2.py` code when I saw `hidden_states = self.mlp(hidden_states)` in the `DecoderLayer` forward pass. I was extremely confused as to why the Qwen models are using MLPs instead of GLUs and therefore checked their technical report, where I realized that they indeed do not use MLPs. Only then did I realize that the `Qwen2MLP` class indeed represents a GLU.

I would argue that this is highly misleading naming.

@emergenz @younesbelkada @ArthurZucker Outside of the debate of whether it's confusing or not, we're somewhat limited here on renaming. It's possible to rename the class itself (it's not importable from the `__init__` and this _shouldn't_ affect existing copies, other than creating a divergence). However, we can't rename the layer attribute in the other layers i.e. `self.mlp`, as this would change the state dict. As this would then create a mismatch i.e. `self.mlp = LlamaGLU(...)` I think we should leave as is (possibly with a clarifying comment somewhere in the code).  

Agreed with the statement above !

Oh, unfortunate but I understand. Thanks for clarification!