[v5, Mistral models broken] Embeddings are automatically tied
Author: patrickvonplaten
Pull request: no
Labels: name: bug

### System Info

...

### Who can help?

```py
import torch
from transformers import Mistral3ForConditionalGeneration, AutoTokenizer

model_id = "mistralai/Mistral-Small-3.2-24B-Instruct-2506"

# Load tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Load model
model = Mistral3ForConditionalGeneration.from_pretrained(
    model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
).eval()

# Chat template input
messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "text": "Hello",
            }
        ],
    }
]

# Tokenize
tokenized = tokenizer.apply_chat_template(messages, return_dict=True)
input_ids = torch.tensor(tokenized["input_ids"], device="cuda").unsqueeze(0)
attention_mask = torch.tensor(tokenized["attention_mask"], device="cuda").unsqueeze(0)

# Generate
with torch.inference_mode():
    output = model.generate(
        input_ids=input_ids,
        attention_mask=attention_mask,
        max_new_tokens=128,
    )[0]

# Decode
decoded_output = tokenizer.decode(output, skip_special_tokens=True)
print(decoded_output)
```

is broken an current main

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

execute the code

### Expected behavior

To give non gibberish

**Note**: Only happens on v5

Comments:
Can be fixed by setting `tie_word_embeddings=False` for the `config.json`

Acknowledged - current bug is on `main` branch (4.57 dev), to fix before v5 release. 

Yeah, we need to change the default to `False` in individual config. It was always `True` but never enforced. Sorry about that

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

This should be solved thanks to https://github.com/huggingface/transformers/pull/42507. Tested it, just FYI this also is needed to make it work https://github.com/huggingface/transformers/pull/42997 (there's a bug in the automapping). 

On our radar in any case for the full v5.0 release!

Edit: now this is fixed. From repro code posted, now the output yields a warning relative to config correctness, and generates correctly. 

```bash
Fetching 10 files: 100%|███████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 9993.58it/s]
Download complete: : 0.00B [00:00, ?B/s]                                                      | 0/10 [00:00<?, ?it/s]
Loading weights: 100%|█| 585/585 [00:05<00:00, 113.46it/s, Materializing param=model.vision_tower.transformer.layers.
The tied weights mapping and config for this model specifies to tie model.language_model.embed_tokens.weight to lm_head.weight, but both are present in the checkpoints, so we will NOT tie them. You should update the config with `tie_word_embeddings=False` to silence this warning
The following generation flags are not valid and may be ignored: ['temperature']. Set `TRANSFORMERS_VERBOSITY=info` for more details.
Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.


Hello
Hello! 😊 How can I assist you today?


```