noamaanMulla-03/transformers-issues huggingface dataset:
[i18n-ar] Translated file : `docs/source/ar/pipeline_webserver.md` into Arabic
Author: AhmedAlmaghz
Pull request: yes


## What does this PR do?
Translated the `docs/source/ar/pipeline_webserver.md` file of the documentation to `Arabic العربية`. I use transformers for my side-project and wanted to contribute. As a beginner, this seemed appropriate.

### Some questions:

- I can't access the pr documentation endpoint for `Arabic العربية`. Is there a missing step?
- I left all the other files alone after copying the English docs. Is this acceptable for future updates or should I add contents to the `_toctree.yml` only when translation is complete? Should I delete all other files for a cleaner review?
- In `Arabic`, people use both `Transformer` and `المحولات` to describe the model. However, as this is more of a product of HuggingFace I opted not to translate it. May you please let me know which you would prefer?

Thank you in advance for your review.

Part of #32435

### Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/) ?Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

 ### Who can review?
 @stevhliu , @MKhalusova , @abodacs, may you please review this PR?
 

Comments:
Thank you @abodacs
[i18n-ar] Translated file : `docs/source/ar/model_memory_anatomy.md` into Arabic
Author: AhmedAlmaghz
Pull request: yes


## What does this PR do?
Translated the `docs/source/ar/model_memory_anatomy.md` file of the documentation to `Arabic العربية`. I use transformers for my side-project and wanted to contribute. As a beginner, this seemed appropriate.

### Some questions:

- I can't access the pr documentation endpoint for `Arabic العربية`. Is there a missing step?
- I left all the other files alone after copying the English docs. Is this acceptable for future updates or should I add contents to the `_toctree.yml` only when translation is complete? Should I delete all other files for a cleaner review?
- In `Arabic`, people use both `Transformer` and `المحولات` to describe the model. However, as this is more of a product of HuggingFace I opted not to translate it. May you please let me know which you would prefer?

Thank you in advance for your review.

Part of #32435

### Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/) ?Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

 ### Who can review?
 @stevhliu , @MKhalusova , @abodacs, may you please review this PR?
 

Comments:
Thank you @abodacs 

اشكرك اخي عبدالله @abodacs  على جهودك الرائعة.

عمل رائع وعظيم 

التالي (الأخير في القائمة الحالية) : 🥇 

# CONCEPTUAL GUIDES

- [x]  Philosophy #33064
- [x]  Glossary #33038
- [x]  What 🤗 Transformers can do #33073
- [x]  How 🤗 Transformers solve tasks #33074
- [x]  The Transformer model family #33047
- [x]  Summary of the tokenizers #33078
- [x]  Attention #33021
- [x]  Padding and truncation #33050
- [x]  BERTology #33024
- [x]  Perplexity of fixed-length models #33063
- [x]  Pipelines for webserver inference #33066
- [x]  Model training anatomy #33045
- [ ] Getting the most out of LLMs #33043
[i18n-ar] Translated file : `docs/source/ar/llm_tutorial_optimization.md` into Arabic
Author: AhmedAlmaghz
Pull request: yes


## What does this PR do?
Translated the `docs/source/ar/llm_tutorial_optimization.md` file of the documentation to `Arabic العربية`. I use transformers for my side-project and wanted to contribute. As a beginner, this seemed appropriate.

### Some questions:

- I can't access the pr documentation endpoint for `Arabic العربية`. Is there a missing step?
- I left all the other files alone after copying the English docs. Is this acceptable for future updates or should I add contents to the `_toctree.yml` only when translation is complete? Should I delete all other files for a cleaner review?
- In `Arabic`, people use both `Transformer` and `المحولات` to describe the model. However, as this is more of a product of HuggingFace I opted not to translate it. May you please let me know which you would prefer?

Thank you in advance for your review.

Part of #32435

### Before submitting

- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/) ?Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

 ### Who can review?
 @stevhliu , @MKhalusova , @abodacs, may you please review this PR?
 

Comments:
مرحبا أخي عبدالله @abodacs 
اشكرك على المراجعة وأعذرني على الإنقطاع اليومين السابقين لقد  كنت مريضاً
سوف أراجع الملف و اتحقق من الناقص

يمكنك البدء بالقسم التالي :

# DEVELOPER GUIDES

- [ ]  Use fast tokenizers from 🤗 Tokenizers #33034
- [ ]  Run inference with multilingual models #33048
- [ ]  Use model-specific APIs #33030
- [ ]  Share a custom model #33031
- [ ]  Templates for chat models #33026
- [ ]  Trainer #33080
- [ ]  Run training on Amazon SageMaker #33071
- [ ]  Export to ONNX #33072
- [ ]  Export to TFLite #33077
- [ ]  Export to TorchScript #33079
- [ ]  Benchmarks #33023
- [ ]  Notebooks with examples #33049
- [ ]  Community resources #33027
- [ ]  Troubleshoot #33017
- [ ]  Interoperability with GGUF files #33037

@AhmedAlmaghz 
شفاك الله وعافاك أخى

سأحاول أراجع ﻷن السبوع القادم عندى امتحانات

حفظك الله ورعاك أخي عبدالله @abodacs 
بالتوفيق إن شاء الله في إمتحاناتك
اعمل approving  لكل PRs الجاهزة حتى يتم دمجها
لقد راجعت هذا الملف وأكملت الناقص أشكرك  يمكنك مراجعته

تحياتي وتقديري الكبيرين لشخصكم الكريم
Bug in Llama3.1 8B Instruct Tokenizer
Author: tginart
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.44.2
- Platform: Linux-5.15.0-1056-aws-x86_64-with-glibc2.31
- Python version: 3.10.14
- Huggingface_hub version: 0.23.3
- Safetensors version: 0.4.3
- Accelerate version: 0.33.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.2.0+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA H100 80GB HBM3

### Who can help?

@ArthurZucker 



### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

1. import llama tokenizer
2. get token id for 'Ð¾'
3. test decoding
`tokenizer = AutoTokenizer.from_pretrained('meta-llama/Meta-Llama-3.1-8B-Instruct')`
`tokenizer.get_vocab()['Ð¾']`
- above should give you 1482
`tokenizer.decode(1482)`
- above should give you 'о' (Cyrillic)

### Expected behavior

Seems like a bug or inconsistency between the tokenizer's stated vocabulary and the actual encoding function. This causes confusion and errors in encoding/decoding. 

Comments:
cc @itazap 😉 

Hey @tginart ! This is expected as it is related to how llama trains using `tokenizers.byte_level` and converts characters into bytes that can be mapped to different UTF-8 Unicode characters. For example `о` maps to `Ð¾` which has the encoding `U+043E` both map to  [(utf-8 map table)](https://www.utf8-chartable.de/unicode-utf8-table.pl?start=1024&names=-&utf8=char#:~:text=U%2B043E,%C3%90%C2%BE)

This is discussed in https://github.com/huggingface/tokenizers/issues/254 , https://github.com/huggingface/tokenizers/issues/203, https://github.com/huggingface/tokenizers/issues/268 which might provide more in-depth discussions 🤗 Let me know I can answer more questions about this!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
audio pipeline utility ffmpeg_microphone_live only works with PulseAudio in Linux (KDE Manjaro)
Author: Dakai
Pull request: no
Labels: name: bug

name: Audio

### System Info

- `transformers` version: 4.44.0
- Platform: Linux-6.10.3-1-cachyos-bore-x86_64-with-glibc2.40
- Python version: 3.12.4
- Huggingface_hub version: 0.24.5
- Safetensors version: 0.4.4
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.4.0+cu121 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@Narsil

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

follow https://huggingface.co/learn/audio-course/chapter7/voice-assistant


### Expected behavior

The course's code running in my local Linux machine does not record anything, I tried to use the latest ffmpeg to record the sound with command `ffpmeg -f alsa -i default -t 5 test.wav` and got an empty wav file, then `ffmpeg -f pulse -i default -t 5 test.wav` worked perfectly.

So I edited the file, `venv/lib/python3.12/site-packages/transformers/pipelines/audio_utils.py` at line 30, changed `fomat_ = "alsa"` into `format_ = "pulse"` , then the code works as expected and successfully recognised the wake word.

Can you please look into this issue with Linux audio system and make it compatible with both ALSA and PulseAudio?


Comments:
@ylacombe 

Hey @Dakai, thanks for opening this issue! Looks like it might be an issue with your KDE, have you been able to use Alsa in your terminal?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Whisper Inference in BF16 precision.
Author: Aditya-Scalers
Pull request: no
Labels: name: Core: Pipeline

name: Audio

### System Info

A google colab instance.

### Who can help?

@sanchit-gandhi 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

My sample code:
```py
import torch
from transformers import pipeline

device = "cpu"

pipe = pipeline(
  "automatic-speech-recognition",
  model = "openai/whisper-base",
  device=device,
  torch_dtype = torch.bfloat16,
)

sample = "/content/sample.mp3"

prediction = pipe(sample)["text"]
print(prediction)
```

### Expected behavior

I used the sample pipeline code from the huggingface whisper samples and in the dtype argument for the pipeline i wanted the inference to run in BF16 precision. Below is the error i encountered. 

>RuntimeError: Input type (torch.FloatTensor) and weight type (CPUBFloat16Type) should be the same or input should be a MKLDNN tensor and weight is a dense tensor

Comments:
Gentle ping @sanchit-gandhi 

another ping @sanchit-gandhi 

cc @kamilakesbi 

cc @ylacombe 

I can't find the right PR, but it seems to have be solved when I look at the [pipeline code](https://github.com/huggingface/transformers/blame/97c0f45b9c4274ee2b1cfe821ba52b8a70029e76/src/transformers/pipelines/base.py#L1066-L1082).  There's been a fix tentative in #29486 but again, seems to have been fixed looking directly at the code!

Besides, when running your code snippet, everything seems to work fine! Can you try again on the latest transformers version and confirm it works as expected!


Also cc @eustlb for visibilty


This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Tokenizer for MyT5 Model
Author: tomlimi
Pull request: no
Labels: name: New model

### Model description

MyT5 is a sister model of ByT5 trained on Morphologically-dervied byte sequences (MYTES).
The model itself shares implementation with T5ForConditionalGeneration, so only addition of custom MyT5Tokenizer is needed to run model from hugging-face.


### Open source status

- [x] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

The tokenizer implementation is available at: https://github.com/tomlimi/MYTE/tree/main/src/myt5
MYTEs and MyT5 training are described in a research paper: https://arxiv.org/pdf/2403.10691
Model cards: https://huggingface.co/Tomlim/myt5-large

Comments:
cc @ArthurZucker 

FWY @itazap
Reduces redundancy and optimizes the test setup and execution.
Author: SpEXterXD
Pull request: yes

 -> Removed redundant context reset:
`context._context = None` and `ops.enable_eager_execution_internal()` can be removed if it's not necessary for the tests setup

-> Optimize device config : 
Check if logical device configuration is necessary and only configure it if needed.

-> Combined Similar Operations to reduce redundancy


Comments:
Mistral with flash attention cannot return `attention_weights`
Author: Junyoungpark
Pull request: no

Hi all,

I’ve discovered that Mistral models with flash attention cannot return attention_weights due to a reference error. I anticipate that we can address this issue by passing `return_attn_probs=True` to the flash attention API, but there’s still some uncertainty. It appears that `flash_attn_func` can return the `attn_weights`, although it’s worth noting that the output weights may not be entirely correct, according to the official API doc.

You can find the relevant code snippet in the Mistral modeling file here.
https://github.com/huggingface/transformers/blob/1c31b7aa3bb4e7ef24c77596d2a76f45a770159f/src/transformers/models/mistral/modeling_mistral.py#L468 

Comments:
Hi @Junyoungpark 
Due to the fact the returned attention might not be correct (e.g.: https://github.com/Dao-AILab/flash-attention/blob/61a777247900f6c2a37376f3ffd7134385fdc95c/flash_attn/flash_attn_interface.py#L668) unfortunately we had to force-disable `output_attention` to `False` for all FA-2 models. We can consider to enable it once we have gurantees on FA-2 side that the returned output attentions are correct

Hi @younesbelkada.

Thanks for the detailed reply. I understand the current situation. Thanks 👍  

Thank you @Junyoungpark !

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @younesbelkada

I am facing the same issue: I need to return the attention maps using Flash Attention 2.
So, it would be appreciated if you can clarify these points:
- The only solution for now is to turn off flash attention to be able to return the attention maps.
- Can you give us more details on why the current attention maps using Flash attention are not trusted or reliable? 

Thanks in advance!
Add DBRX GGUF Support Error
Author: cjfghk5697
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.46.0.dev0
- Platform: Linux-6.1.85+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.24.7
- Safetensors version: 0.4.5
- Accelerate version: 0.34.2
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.4.1+cu121 (True)
- Tensorflow version (GPU?): 2.17.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.8.5 (gpu)
- Jax version: 0.4.33
- JaxLib version: 0.4.33
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: Tesla T4

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

 [Check it in Colab](https://colab.research.google.com/drive/1-bwbck3IjOf0-uGw4amheQEUWk7XJVBb?usp=sharing)
 Draft Code #33977 

### Expected behavior

Hi, I'm working on dbrx to support GGUF architecture(#33260). `ggml.py` has been modified for this purpose, but I'm encountering the following error: "TypeError: expected str, bytes or os.PathLike object, not NoneType".

It seems the error is caused by `gguf_checkpoint_path` being defined as `None` in the `load_gguf_checkpoint` function in `modeling_gguf_pytorch_utils.py`. I'm currently tracking down why it is being set to `NoneType`.

Upon investigation, I found that the value is being passed as `NoneType` at line 2441 in `tokenization_utils_base.py`. I need to understand why this value, which previously worked correctly, is now being set to `NoneType` and address this issue.
```python
try:
    tokenizer = cls(*init_inputs, **init_kwargs)
```
It seems that the initial value for the GGUF file is being set to `NoneType`.

Here's the code for reference: [Check it in Colab](https://colab.research.google.com/drive/1-bwbck3IjOf0-uGw4amheQEUWk7XJVBb?usp=sharing)

Any help would be greatly appreciated.

Comments:
cc @SunMarc 

Hey @cjfghk5697, @SunMarc ! If I am not mistaken, It is not a bug, It happens because tokenization_utils_fast.py L122 - `gguf_param = load_gguf_checkpoint(kwargs.get("vocab_file"))` tries to get vocab_file from tokenizer which is None by default. If you don't explicitly assign `vocab_file=vocab_file` and `merges_file=merges_file` in your case I suppose in tokenization_gpt2_fast.py L100-101, then, even if you provide those in tokenizer object creation, kwargs.get("vocab_file") still will not get vocab file and returns None. So, you could try to fix `vocab_file=vocab_file` and `merges_file=merges_file` in tokenization_gpt2_fast.py and check the behaviour. 

@VladOS95-cyber, thank you so much!! 😭😭😭
I really appreciate your help. I checked that my tokenizer works well, haha. Thanks a lot!
Flash-attn performance: remove cuda sync during inference
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

https://github.com/huggingface/transformers/pull/31629 & https://github.com/huggingface/transformers/pull/32241 introduced a functionality in FA2 intended for training efficiency. However, it adds unnecessary cuda synchronization at inference time in every forward pass due to always checking `(torch.diff(position_ids, dim=-1) >= 0).all()`  in the `elif` condition. This PR fixes the performance issue by simply switching the order of the different checks in the `elif` condition, to make good use of Python's default short-circuit evaluation. Indeed, at inference time, `query_length` will always be 1 except during prefill, thus we will short-circuit torch synchronization all the time.

Performance degradation was not so significant, but this PR allows to win back around 5-10% speed at inference time from the quick tests I ran.


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33570). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

cc @ArthurZucker, forgot to ping you
[Docs] Improve VLM docs
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR improve the docs of various VLMs:

* llava: batched inference
* llava-onevision: minor nit on model naming
* qwen2-vl: add to appropriate multimodal section in toctree, add figure
* pixtral: add `apply_chat_template`, add figure

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_33393). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Error happens when using SiglipVisionModel.forward with interpolate_pos_encoding=True
Author: kaitolucifer
Pull request: no
Labels: name: bug

### System Info

`transformers==4.46.0.dev0`

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

If the size of `pixel_values` is 384 x 384, below will throw an error.

```python
vision_tower =  SiglipVisionModel.from_pretrained("google/siglip-so400m-patch14-384")
image_outputs = vision_tower(pixel_values, interpolate_pos_encoding=True)
```

Error:
```
embeddings = embeddings + self.interpolate_pos_encoding(embeddings, height, width)
                         ~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TypeError: unsupported operand type(s) for +: 'Tensor' and 'Embedding'
```

### Expected behavior

Shouldn't raise an error even though pos_encoding doesn't need to be interpolated when `interpolate_pos_encoding=True`

Comments:
Hey! I think this is also related to #33965

Thanks for opening a PR seems right indeed! 🤗