noamaanMulla-03/transformers-issues huggingface dataset:
[i18n-ar] Translated file: `docs/source/ar/tasks/language_modeling.md` into Arabic
Author: AhmedAlmaghz
Pull request: yes


## What does this PR do?
Translated the `docs/source/ar/tasks/language_modeling.md` file of the documentation to `Arabic العربية`. I use transformers for my side-project and wanted to contribute. As a beginner, this seemed appropriate.

### Some questions:

- I can't access the pr documentation endpoint for `Arabic العربية`. Is there a missing step?
- I left all the other files alone after copying the English docs. Is this acceptable for future updates or should I add contents to the `_toctree.yml` only when translation is complete? Should I delete all other files for a cleaner review?
- In `Arabic`, people use both `Transformer` and `المحولات` to describe the model. However, as this is more of a product of HuggingFace I opted not to translate it. May you please let me know which you would prefer?

Thank you in advance for your review.

Part of #32435

### Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/) ?Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

### Who can review?
@stevhliu , @MKhalusova , @abodacs, may you please review this PR?



Comments:
How to convert my Mask2Former model (ResNet-50 backbone) to Hugging Face transformer
Author: yujunwei04
Pull request: no

### System Info

```shell
- `transformers` version: 4.34.0
- Platform: Linux-6.8.0-31-generic-x86_64-with-glibc2.17
- Python version: 3.8.20
- Huggingface_hub version: 0.17.3
- Safetensors version: 0.4.5
- Accelerate version: 0.23.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.0.1+cu117 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No
```


### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I found the following script, but it only supports conversion for Mask2Former model (swin backbone) https://github.com/huggingface/transformers/blob/main/src/transformers/models/mask2former/convert_mask2former_original_pytorch_checkpoint_to_pytorch.py

May I ask for some guidance on how to adjust the script so that it can support ResNet-50 architecture?

### Expected behavior

```shell
Convert my Mask2Former model (ResNet-50 backbone) to Hugging Face transformer
```


### Checklist

- [X] I have read the migration guide in the readme. ([pytorch-transformers](https://github.com/huggingface/transformers#migrating-from-pytorch-transformers-to-transformers); [pytorch-pretrained-bert](https://github.com/huggingface/transformers#migrating-from-pytorch-pretrained-bert-to-transformers))
- [X] I checked if a related official extension example runs on my machine.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Is non-determinism in outputs generated by LlamaForCausalLM, the expected behavior?
Author: rachithaiyappa
Pull request: no

https://github.com/huggingface/transformers/blob/e42587f596181396e1c4b63660abf0c736b10dae/src/transformers/models/llama/modeling_llama.py#L727

I've noticed that this class does not lead to deterministic outputs. 
See discussion on the hugging face hub [here](https://discuss.huggingface.co/t/making-llama-text-generation-deterministic/50437).

**Actual issue:** 

Following the text generation code template [here](https://huggingface.co/docs/transformers/main/model_doc/llama2), I’ve been trying to generate some outputs from llama2 but running into stochastic generations.
For instance, running the same prompt through the model.generate() twice results in two different outputs as shown in the example below.

I’ve used model.generate() with other LLMs (e.g., flant5) with the other parameters remaining the same and have obtained deterministic outputs.
Also tried AutoModelForCausalLM instead of LLamaForCausalLM but still got different outputs each time for the same prompt.

How do I make sure I get the same text generated each time?

Code to reproduce:

```
from transformers import AutoTokenizer, LlamaForCausalLM

model_name = "meta-llama/Llama-2-13b-chat-hf"
tokenizer = AutoTokenizer.from_pretrained(model_name, cache_dir="/data2/racball/llms")
model = LlamaForCausalLM.from_pretrained(
    model_name,
    cache_dir="/data2/racball/llms",
    device_map = "sequential",
)

prompt = "What is up?"
inputs = tokenizer(prompt, return_tensors="pt")
# Generate
generate_ids = model.generate(inputs.input_ids, max_length=30)

tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]

# Run1: 'What is up?\n\nI have a problem with my `docker-compose.yml` file. I have a service that should run a'
# Run2: "What is up?\n\nIt's been a while since I've posted, but I've been pretty busy with work and other"
```

Comments:
cc @gante
I think the model has random sampling enabled by default in its [generation config](https://huggingface.co/meta-llama/Llama-2-13b-chat-hf/blob/main/generation_config.json). For reproducible results, you will need to set `do_sample` to `False`.

Thank you!

Shouldn't it be possible to produce deterministic output even when do_sample=True by setting a random seed?

@shuangwu This is exactly my problem as well. I need the sampling behavior for multinomial sampling, but setting a fixed seed should make it reproducible. However, my observations are that sometimes it isn't. However, in my case, I can not rule out that due to the fact that they may run on different hardware, which brings the problem.
Fix Whisper CI
Author: ydshieh
Pull request: yes

Finally CI is green

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34617). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> tests/models/rt_detr/test_image_processing_rt_detr.py::RtDetrImageProcessingTest::test_fast_is_faster_than_slow

it's flaky

Should this be merged?

> Should this be merged?

Yes, but I am waiting for an approval. Currently, `whisper` has ~170 failed tests without this PR.

Hey guys 👋 @ydshieh @ArthurZucker 

The generate changes in this PR are incorrect (changing `max_length`) :) I'm reverting them as part of https://github.com/huggingface/transformers/pull/34807, but I'd recommend rolling back this PR.

If Whisper's tests are failing, then it is probably something wrong with either whisper or whisper's tests




Hi @gante This PR is only adding

> if generation_config.max_length == GenerationConfig().max_length

to avoid the failure caused in #34377 (which was an attempt fix for #34026)

> The generate changes in this PR are incorrect

Is the code changes in the previous PR #34377  (and/or #34026) are already wrong, or it's wrong because of the `if` condition added in this PR?

Yeah I think it's from the older pipeline PR 😭 (`generation_config.max_length = generation_config.max_length + input_ids_seq_length` shouldn't exist)

But the check for `max_position_embeddings` should exist either -- some models can generate beyond this value (e.g. llama)

So if we want to rollback, it's more involved (i.e. not revert this PR but some previous changes too).
In any case, ping me before merging a new change on this PR so I can see if the Whisper CI is still OK 🙏 .
(or if you want to check that part on your side 😄 )

This PR messes up generation tests. I noticed this when working on GPT2Model PR #35761.
looking forward to merging of https://github.com/huggingface/transformers/pull/34807 or some other solution
Update torchao.md: use auto-compilation
Author: martin0258
Pull request: yes

# What does this PR do?

Fixes #34310 with the new auto compilation introduced in #34923

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@SunMarc 


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35490). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
run_mlm_flax on tpu v5-pods
Author: peregilk
Pull request: no
Labels: name: Flax

name: bug

### System Info

Latest update of both transformers and jax

### Who can help?

 @ArthurZucker I am trying to use the `run_mlm_flax.py` to train a Roberta model on a v5-256 pod. However, while a single v3-8 is capable of running with `per_device_batch_size=128`, the v5-256 are only able to run with` per_device_batch_size=2`. Any ideas?

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Using default code.

### Expected behavior

I would expect a v5-256 to run a lot faster here.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
fix gradient accumulation steps without using deepspeed
Author: baoyf4244
Pull request: yes

# What does this PR do?

when training without deepspeed,  before loss backward, the loss is not scaling with  gradient_accumulation_steps,  this is caused by the parameter gradient_accumulation_steps not passed to Accelerator when creating this object.


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @ylacombe, @eustlb
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
Ok, I'm sorry I have not find that issue a few days ago.
Fix Aria CI and testing
Author: molbap
Pull request: yes

# What does this PR do?

Tests Aria on the hub revision that corresponds to the transformers version of this model. 

The current hub version supports remote code rather than current impl of transformers, so we ought to support FA2, see #35670 . 
This should at least make the tests run on the nightly + keep the revision reference around. 

cc @aymeric-roucher (FYI, no action needed)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35674). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@molbap I've updated the Aria hub repo to make the main branch support the transformer impl, so we no longer need to specify the reversion.
is possible convert transforms tokenizers in sentence piece .model? 
Author: Caio-lima-santos
Pull request: no


i want use text-generators in mobile plataforms with onnxruntime in c++, but i can't convert tokenizer in a way for use it in c++

![image](https://github.com/user-attachments/assets/857e3d08-ac49-4bb9-9ebd-9d6dd5f6677e)

is possible convert these files in .model for sentence piece or another tokenizer that can load transforms tokenizers in c++?

(note:these files are intalled from transform python api and save in the local machine)

Comments:
I don't believe we support this! This checkpoint is a "Fast" transformers tokenizer, with most of the tokenizer data stored in `tokenizer.json`. We have methods to convert original "Slow" tokenizers, which are often sentencepiece `.model` files, into fast `tokenizer.json` files, but we don't have any methods to do the inverse conversion.
Aria processor does not work with images
Author: DarkLight1337
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.48.0
- Platform: Linux-5.4.0-174-generic-x86_64-with-glibc2.31
- Python version: 3.9.20
- Huggingface_hub version: 0.26.2
- Safetensors version: 0.4.5
- Accelerate version: 1.0.1
- Accelerate config:    not found
- PyTorch version (GPU?): 2.5.1+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA A10

### Who can help?

@zucchini-nlp @aymeric-roucher 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

```py
>>> import numpy as np
>>> from transformers import AutoProcessor
>>> processor = AutoProcessor.from_pretrained("rhymes-ai/Aria")
>>> processor(text="
", images=[np.zeros((3, 224, 224))])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/cyrus/miniconda3/envs/vllm/lib/python3.9/site-packages/transformers/models/aria/processing_aria.py", line 129, in __call__
    sample = sample.replace(self.tokenizer.image_token, self.tokenizer.image_token * num_crops)
  File "/home/cyrus/miniconda3/envs/vllm/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 1108, in __getattr__
    raise AttributeError(f"{self.__class__.__name__} has no attribute {key}")
AttributeError: LlamaTokenizerFast has no attribute image_token
```

The processor for Aria model doesn't work because of missing `image_token` attribute.

### Expected behavior

Similar to other models, we should add `image_token` directly to the processor and use that instead of referring to `tokenizer.image_token`.

Comments:
I'll take a look if the configs on hub side were updated, since the conversion script adds an `image_token` 

Oke, I see this commit (https://huggingface.co/rhymes-ai/Aria/commit/3e774c984a6e0699a566d9c83b9d562df329a42e) updates the hub files to align with transformers. Feel free to close if the issue is resolved

Can confirm it is fixed now. Is it preferred to have `image_token` in the processor or tokenizer instance?

@DarkLight1337 More preferred in tokenizer as we've been trying to add all multimodal special tokens there. Especially useful for models with bunch of special tokens to wrap each image grid by row and column

But I agree that currently not all models save it in tokenizer, the feature was introduced a couple releases ago

Thanks for the info! I'll close this issue now.
Fix CI for VLMs
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

As per title, main changes are:
- Removed some tests, not needed anymore as they tested old merging behavior in VLMs
- Removed logit level check as those break a lot because of precision errors when we update torch for ex. Test on token-level generation should be enough
- Other tests actually needed to update their `expected_text` so updated them
- Since it was a tiny change, propagated Emu3 checkpoint name update to docs (can move to a new PR to make it cleaner)
- Mllama tests are failing because of gated repo (meta-llama/Llama-3.2-11B-Vision) cc @ydshieh 
- Emu3 also needs a bit more GPU memory to run (around 16GB), that is the minimum possible value if we get a tiny image as input. So I don't know if we should just remove batched slow tests cc @ydshieh 

Note: Emu3/Aria flex-attn test will be failing and needs a different PR to fix the way attention is dispatched on some VLMs. Apparently it never worked for Emu3-like models

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35690). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

> Mllama tests are failing because of gated repo (meta-llama/Llama-3.2-11B-Vision) 

Asked the infrao. It can't be downloaded in EU region ...

> Emu3 also needs a bit more GPU memory to run (around 16GB), that is the minimum possible value if we get a tiny image as input. So I don't know if we should just remove batched slow tests

which (full) test name? @Cyrilvallez talked to me we might need to introduce @need_a10_gpu decorator if we don't have other way

> Asked the infrao. It can't be downloaded in EU region ...

😢 


> which (full) test name? @Cyrilvallez talked to me we might need to introduce @need_a10_gpu decorator if we don't have other way

Cool, the decorator wouuld definitely help a lot! The tests are `test_model_generation_batched` and `test_model_generation_multi_image`

See the decorator `@require_torch_large_gpu` if needed! Just merged it
Security fix for `self-comment-ci.yml`
Author: ydshieh
Pull request: yes


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35548). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

a failing run looks like [this](https://github.com/ydshieh/check_comment/actions/runs/12655846330/job/35267101466)
[ViTPose] Convert more checkpoints
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR converts more ViTPose checkpoints, in particular the ViTPose+ checkpoints.

Notebook I'm using to run inference on the original implementation (ViTPose+ checkpoints): https://colab.research.google.com/drive/1QDX_2POTpl6JaZAV2WIFjuiqDsDwiqMZ?usp=sharing. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_35638). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Implement SuperPoint / SuperGlue
Author: sbucaille
Pull request: no
Labels: name: New model

### Model description

The SuperGlue network is a Graph Neural Network combined with an Optimal Matching layer that is trained to perform matching on two sets of sparse image features. 
SuperGlue is built on top of SuperPoint model which consists of detecting the most interesting keypoints in an image. With the keypoints of two different images, SuperGlue proceeds to the matching.

I noticed there was no image matching models implemented in transformers library so I propose this first one. I extensively used it in other activities and am new to transformers git contributions, so I am willing to implement it myself as a first contribution.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

SuperPoint and SuperGlue code and weights are available at https://github.com/magicleap/SuperGluePretrainedNetwork
The original paper of SuperPoint : https://arxiv.org/abs/1712.07629
The original paper of SuperGlue : https://arxiv.org/abs/1911.11763

Comments:
I could help here, I have worked with both of these models as well as LoFTR which is the current SOTA in image matching. I am new to Transformers contributions as well, but would love to contribute if any of the repo maintainers agree to the inclusion of these models

Hi @arkouda , I was about to use this model as an opportunity to contribute to the project, so I'll be working on the SuperGlue model implementation myself but feel free to take care of the LoFTR :smile: