noamaanMulla-03/transformers-issues huggingface dataset:
[i18n-ar] Translated file : `docs/source/ar/notebooks.md` into Arabic
Author: AhmedAlmaghz
Pull request: yes


## What does this PR do?
Translated the `docs/source/ar/notebooks.md` file of the documentation to `Arabic العربية`. I use transformers for my side-project and wanted to contribute. As a beginner, this seemed appropriate.

### Some questions:

- I can't access the pr documentation endpoint for `Arabic العربية`. Is there a missing step?
- I left all the other files alone after copying the English docs. Is this acceptable for future updates or should I add contents to the `_toctree.yml` only when translation is complete? Should I delete all other files for a cleaner review?
- In `Arabic`, people use both `Transformer` and `المحولات` to describe the model. However, as this is more of a product of HuggingFace I opted not to translate it. May you please let me know which you would prefer?

Thank you in advance for your review.

Part of #32435

### Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/) ?Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

 ### Who can review?
 @stevhliu , @MKhalusova , @abodacs, may you please review this PR?
 

Comments:
Thank you @abodacs 
Done update notebooks.md
Please, Review it again

cc @stevhliu 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33049). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Attentions Behavior are different
Author: efsotr
Pull request: no
Labels: name: Core: Modeling

name: bug

name: Flash Attention

### System Info

x

### Who can help?

@ArthurZucker 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

https://github.com/huggingface/transformers/blob/v4.44.2/src/transformers/models/llama/modeling_llama.py#L658C9-L667C10

The Attention.is_causal of models like Llama, Gemma2, and Mistral cannot be used in eager, sdpa, but can be used in flash attention 2. This does not align with the expectation of changing a causal model to a bidirectional model after setting Attention.is_causal to False.

### Expected behavior

x

Comments:
Thanks @efsotr, we are indeed aiming to reduce these differences and align everything; it'll likely be much simpler as we adopt `FlexAttention` from PyTorch. Stay tuned!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

#34282 for more details!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add token cost + runtime monitoring to Agent and HfEngine children
Author: aymeric-roucher
Pull request: yes
Labels: name: Agents

# What does this PR do?

Add callbacks + monitoring options to agents, with the end goal of being usable in a monitoring platform such as Arize AI's [Phoenix](https://docs.arize.com/phoenix).

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34548). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Fix `BertGeneration`
Author: ydshieh
Pull request: yes

# What does this PR do?

`BertGeneration` doesn't prepare the attention mask correctly when it is used as a decoer + `use_cache=True`.

Running the script above:

The hidden states difference (the first item printed) looks

- `[0.0, 0.004747, 0.0062393]` with `main`
- `[0.0, 0.0, 0.0]` with this PR

This is also the reason why 

> tests/models/bert_generation/test_modeling_bert_generation.py::BertGenerationEncoderTest::test_assisted_decoding_matches_greedy_search_1_same

is flaky. With this PR, running 10000 times and all pass.

### Script

```python
import torch
from tests.models.bert_generation.test_modeling_bert_generation import BertGenerationEncoderTest, BertGenerationEncoderTester

torch_device = "cpu"
self = BertGenerationEncoderTest()
self.setUp()
model_class = self.all_generative_model_classes[0]
config, inputs_dict = self.prepare_config_and_inputs_for_generate(batch_size=1)
config.is_decoder = True

inputs_dict['attention_mask'][0, :3] = 0
inputs_dict['attention_mask'][0, 3:] = 1

inputs_dict['use_cache'] = True
inputs_dict['output_hidden_states'] = True
inputs_dict['output_attentions'] = True

model = model_class(config).to(torch_device).eval()

outputs1 = model(**inputs_dict)

shorter_inputs_dict = {k: v for k, v in inputs_dict.items()}
shorter_inputs_dict['input_ids'] = inputs_dict['input_ids'][:, :-1]
shorter_inputs_dict['attention_mask'] = inputs_dict['attention_mask'][:, :-1]

outputs2 = model(**shorter_inputs_dict)
diffs = [float(torch.amax(torch.abs(outputs1.hidden_states[idx][:, -2] - outputs2.hidden_states[idx][:, -1]))) for idx in range(len(outputs1.hidden_states))]
print(diffs)

print(inputs_dict['attention_mask'])
# The attention of the last token in `outputs2` should be the same as of the token at position -2 in `outputs1`
print(outputs1['attentions'][0][0, 0][-2, :-1].detach().tolist())
print(outputs2['attentions'][0][0, 0][-1, :].detach().tolist())
```


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35043). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
SequenceClassification for all Model types should have the option to add weights in Cross Entropy loss
Author: SimonStanley1
Pull request: no
Labels: name: Feature request

### Feature request

Qwen2ForSequenceClassification and all other SequenceClassification wrappers for transformer models don't have the possibility to add class weights. This leads to not using the implementation. 

### Motivation

We need it, because we have highly imbalanced data

### Your contribution

I could submit a PR but this is probably a bigger architectural topic

Comments:
fix:#34611 by expanding class_pos_embed
Author: Swastik-Swarup-Dash
Pull request: yes

 Expands class_pos_embed to match batch size #34611 

Comments:
@Swastik-Swarup-Dash sorry for following up late on #34611. I think the following changes are needed as well
`class_pos_embed_expanded = jnp.repeat(class_pos_embed[jnp.newaxis, :], hidden_states.shape[0], axis=0)`
`patch_pos_embed_expanded = jnp.repeat(patch_pos_embed, hidden_states.shape[0], axis=0)`
`return jnp.concatenate([class_pos_embed_expanded, patch_pos_embed_expanded], axis=1)`
If the patch position embeds are not repeated for the batch_size, the concatenation afterwards can't happen. You can test this with a sample example script. The same thing could also be done using jnp.tile I guess. 

Could you also please add the following in the example docstring section
` >>> model = FlaxDinov2Model.from_pretrained("facebook/dinov2-base")`
to `>>> model = FlaxDinov2Model.from_pretrained("facebook/dinov2-base", from_pt=True)`
and
` >>> model = FlaxDinov2ForImageClassification.from_pretrained("facebook/dinov2-base-imagenet1k-1-layer")`
to ` >>> model = FlaxDinov2ForImageClassification.from_pretrained("facebook/dinov2-base-imagenet1k-1-layer", from_pt = True)`

Seems like I missed this important detail while implementing the conversion. Thanks!
Not Receiving Identity Verification Link via Email
Author: fawzighu
Pull request: no
Labels: name: bug

### System Info

with the identity verification process on Hugging Face. I have submitted my request for the identity verification link but have not received any email confirmation or the link itself.

I have checked my spam/junk folders and ensured that I entered the correct email address during the registration process. I also attempted to resend the verification link, but I still have not received anything.

This issue is preventing me from accessing certain features on the platform, and I would appreciate any assistance in resolving it.

Thank you!


### Who can help?

@meg-huggingface 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. Navigate to the Hugging Face website.


2. Register or log in to your account.


3. Go to the identity verification section.


4. Submit a request for the identity verification link.


5. Wait for the confirmation email to arrive.


6. Check the inbox and the spam/junk folder.



Result: I did not receive any link or confirmation via email.


### Expected behavior

Upon submitting a request for the identity verification link, I should receive an email confirmation with the verification link within a reasonable timeframe (typically within a few minutes). This would allow me to complete the identity verification process and access the necessary features on the Hugging Face platform.


Comments:
Hi @fawzighu Can you please email us at website@huggingface.co and provide your HF username as well? We're happy to investigate this for you.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
The CPU memory does not free in the jupyter notebook
Author: gxcuit
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.45.1
- Platform: Linux-5.15.154+-x86_64-with-glibc2.35
- Python version: 3.10.14
- Huggingface_hub version: 0.25.0
- Safetensors version: 0.4.5
- Accelerate version: 0.34.2
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.4.0 (True)
- Tensorflow version (GPU?): 2.16.1 (True)
- Flax version (CPU?/GPU?/TPU?): 0.8.4 (gpu)
- Jax version: 0.4.26
- JaxLib version: 0.4.26.dev20240620
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: Tesla T4

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Hi, I'm using the pipeline to perform the `text-generation `  tasks.    

I was trying the [Llama official example](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct) in the jupyter notebook(kaggle) with GPU support. 

When I first load the model and inference with the prompt, everything just works fine. See the following picture:

![image](https://github.com/user-attachments/assets/bc94eb17-491d-4d35-8e98-dadecc6c615f)

When I restart the kernel (NOTE: does not restart the session), the CPU memory is free.

![image](https://github.com/user-attachments/assets/438b453a-6be2-4174-b499-ec069bfb9f6e)

However, when I load the model AGAIN, it will take up a lot of CPU memory. And even if I restart the kernel, the memory does not free up. Only the GPU memory is free.
![image](https://github.com/user-attachments/assets/35847c19-ee21-45a8-a5b9-ed21503399b9)

![image](https://github.com/user-attachments/assets/2150805e-2af5-434d-b8fe-39e162e7e6c4)

I'm wondering is it that normal?



### Expected behavior

The CPU memory is also free.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @gxcuit, restarting the kernel should free everything held by Python, so I can't imagine any `transformers` bug that would leak memory in that scenario.

My guess is that the CPU memory usage here is probably more like memory mapping or cache or something like that? Do you actually get OOM errors or disk swapping as a result of this bug?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Added explicit error handling to a few modules
Author: pwjvr
Pull request: yes

# What does this PR do?

Added explicit error catching and more detailed error logging in the following modules:
src/transformers/feature_extraction_utils.py
src/transformers/models/bert/tokenization_bert_tf.py
src/transformers/models/deprecated/jukebox/tokenization_jukebox.py
src/transformers/models/fuyu/image_processing_fuyu.py
src/transformers/utils/import_utils.py

## Before submitting

- [X ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? [ValueError: Unable to create tensor, you should probably activate truncation and/or padding with 'padding=True' 'truncation=True' to have batched tensors with the same length](https://github.com/huggingface/transformers/issues/33999)
- [X ] Did you write any new necessary tests? No new tests (as its just improved error handling, no new functionality), ran existing tests to ensure no breaking changes introduced


## Who can review?

@qubvel 

Comments:
Support for easier multimodal use of modular
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

This improves the modular converter when adding a new multi-modal model (with some `Vision` and `Text` part added to the model name) + allow correct renaming for all uppercase model names (e.g. `CLIP`, see [example](https://github.com/huggingface/transformers/compare/multimodal-modular?expand=1#diff-0363c40d762885e8a1376b54964b426b0640606f9497cb298137d8cce47ead93)).

A multi-modal model with one part (e.g. the `Text` part) being a copy/paste version of an all text model (e.g. `Llama`), can directly use the new prefix `NewModelText` instead of `NewModel`, and everything will be renamed accordingly in `Llama` (see [example](https://github.com/huggingface/transformers/compare/multimodal-modular?expand=1#diff-ecfccbecf7f2bf28a10ed66f42c28e4f9c80540fa41cb3be0f8b7c61eec065e3)). However, due to previous inconsistencies in naming, if inheriting from another multi-modal model which did not use the same patterns with the `Text` and `Vision` part, the renaming will not be possible, in which case default renaming (`OldModel` -> `NewModel`) will be used, and intermediate classes without the prefix will need to be manually added with the new prefix (see [example](https://github.com/huggingface/transformers/compare/multimodal-modular?expand=1#diff-85f4d9cb9e1a70fc20e54bbf023f477c8635a7d6fa23e5a425ca24b892dd924f)).  
In all cases, a warning will be raised to tell which prefix is used in case of potential ambiguous situation, and explain that intermediate classes may need to be manually added to the modular.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35056). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
[Critical Bug in LM forward labels] For llama language modeling training, the original code of right-shifting label by 1 is gone?
Author: ShengYun-Peng
Pull request: no
Labels: name: bug

### System Info

transformers==4.46.1

### Who can help?

@ArthurZucker

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

What are the expected labels in the CausalLM.forward? Should user right-shift the label manually or not?

Mistral model right shifts in the forward: https://github.com/huggingface/transformers/blob/329f5dbf97a5cb2473914c88c05aa3dcb242e19a/src/transformers/models/mistral/modeling_mistral.py#L1088-L1100
GPT2 model right shifts in the forward: https://github.com/huggingface/transformers/blob/329f5dbf97a5cb2473914c88c05aa3dcb242e19a/src/transformers/models/gpt2/modeling_gpt2.py#L1490-L1495
The early llama model also right shifts in the forward: https://github.com/huggingface/transformers/blob/f1d822ba337499d429f832855622b97d90ac1406/src/transformers/models/llama/modeling_llama.py#L1205-L1215

BUT that right-shift logic is gone in the latest version: https://github.com/huggingface/transformers/blob/329f5dbf97a5cb2473914c88c05aa3dcb242e19a/src/transformers/models/llama/modeling_llama.py#L1181-L1183

Could you check whether this logic was incorrectly deleted or the entire code was moved to somewhere else?


### Expected behavior

See above

Comments:
Maybe the code is migrated to here? https://github.com/huggingface/transformers/blob/329f5dbf97a5cb2473914c88c05aa3dcb242e19a/src/transformers/loss/loss_utils.py#L32-L47

It is indeed! 🤗 hope your fears are calmed, we refactored this to make it easier for the community to hack the loss! 

Thanks!
[docs] Increase visibility of torch_dtype="auto"
Author: stevhliu
Pull request: yes

Fixes #34743.

Increases visibility of `torch_dtype="auto"` throughout the docs to ensure users are aware that `"auto"` loads weights in the dtype specified in `config.json`. Temporarily resolves the issue, refer to #34919 to track the more permanent solution.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35067). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Add dinov2 with registers diff
Author: BernardZach
Pull request: yes
Labels: name: New model

name: Vision

# What does this PR do?
This PR is a follow up and rebase-ing of #32905 and it seems to be working well from the work done by @NielsRogge. 

## Relevent reviewers:
@ArthurZucker 


Comments:
I think you need to change the diff_dinov2_with_registers.py to modular_dinov2_with_registers.py and then use the linter as described here: https://huggingface.co/docs/transformers/modular_transformers.

So you can run:
```bash
python PATH/utils/modular_model_converter.py --files_to_parse PATH/modular_dinov2_with_registers.py
```
cc @Cyrilvallez 


Indeed!

I believe that I have changed the file name correctly and ran the command provided by @NielsRogge, some tests failing however.

Great, let me know if you need any help fixing the tests, once they're green I can assign someone to review your PR

@NielsRogge having difficulty addressing test issues if you have a moment to take a look. Apoligies, first time contributing. 

Something went wrong with the git diff, showing nearly 3k files changed. Could you reverse that?

I have messed something up bad, closing this PR and attempting to start over from #32905.

Do not use
When using `save_pretrained` to save a model loaded with `from_pretrained`, its size becomes twice as large.
Author: Polarisamoon
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.46.1
- Platform: Linux-5.15.0-25-generic-x86_64-with-glibc2.35
- Python version: 3.10.15
- Huggingface_hub version: 0.26.2
- Safetensors version: 0.4.5
- Accelerate version: 1.0.1
- Accelerate config:    - compute_environment: LOCAL_MACHINE
        - distributed_type: DEEPSPEED
        - mixed_precision: fp16
        - use_cpu: False
        - debug: False
        - num_processes: 2
        - machine_rank: 0
        - num_machines: 1
        - rdzv_backend: static
        - same_network: True
        - main_training_function: main
        - enable_cpu_affinity: False
        - deepspeed_config: {'gradient_accumulation_steps': 1, 'gradient_clipping': 1.0, 'offload_optimizer_device': 'none', 'offload_param_device': 'none', 'zero3_init_flag': False, 'zero_stage': 2}
        - downcast_bf16: no
        - tpu_use_cluster: False
        - tpu_use_sudo: False
        - tpu_env: []
- PyTorch version (GPU?): 2.5.1+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA GeForce RTX 3090

### Who can help?

@ArthurZucker @Rocketknight1  @muellerzr

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
import transformers
from transformers import AutoTokenizer

model = transformers.OPTForCausalLM.from_pretrained("facebook/opt-6.7b")
tokenizer = AutoTokenizer.from_pretrained("facebook/opt-6.7b")

model.save_pretrained("/data/llm/opt-6.7b-testSave")
tokenizer.save_pretrained("/data/llm/opt-6.7b-testSave")
```

### Expected behavior

The weights of `facebook/opt-6.7b` downloaded from Hugging Face are 12.4GB. After using `save_pretrained`, the saved weights became 24.8GB, which is twice as large as the original size. This leads to an "OUT OF MEMORY" error when I load the model for inference. I want the model saved with `save_pretrained` to be the same size as the downloaded one.

Comments:
Hey! This is expected: if you check the dtype of the model it is in `float32` because you did not specify `torch_dtype=torch.float16` or `torch_dtype="auto"` will be fixed by #34919 

Got it, I understand now. Thank you.

> Hey! This is expected: if you check the dtype of the model it is in `float32` because you did not specify `torch_dtype=torch.float16` or `torch_dtype="auto"` will be fixed by #34919
[`tokenizers`] bump to 0.21
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Bump tokenizers version as python3.7 and 3.8 support was dropped

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34972). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Update Mistral conversion script
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Update/refactor Mistral conversion script to work with latest models.

cc @ArthurZucker 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34829). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

IMO it makes more sense now (the keyword is more descriptive, and this case is very unlikely to reappear in future conversions), but I can revert if you think it can break some backward stuff! 🤗