noamaanMulla-03/transformers-issues huggingface dataset:
[i18n-fr] Translate installation to French
Author: NoB0
Pull request: yes

# What does this PR do?

Translated the quicktour.mdx file of the documentation to French.

Part of #21456

Thank you in advance for your review.


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

French speaking contributors. 
Documentation: @stevhliu and @MKhalusova



Comments:
Heads up, a few typos I was able to spot:

- line 31: missing 's' ->  environnements
- line 57: en seule ligne -> en une seule ligne
- line 74 : suivant -> suivants
- line 110: utilie -> utile

> Heads up, a few typos I was able to spot:
> 
> * line 31: missing 's' ->  environnements
> * line 57: en seule ligne -> en une seule ligne
> * line 74 : suivant -> suivants
> * line 110: utilie -> utile

Thanks for pointing this out! I corrected them and checked for others, I believe it is now ok.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_27657). All of your documentation changes will be reflected on that endpoint.

Thank you for working on this! Please make sure the CI checks are green. You can find more information here https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md
Fix unsupported setting of self._n_gpu in training_args on XPU devices
Author: Liangliang-Ma
Pull request: yes

In current training_args, self._n_gpu is set to device_count on XPU device, which will cause crash on XPU devices.
In Trainer, if self.args.n_gpu greater than one, it will utilize torch.nn.DataParallel to wrap the model. But Ipex(intel_extension_for_pytorch) don't support DataParallel, while it suggests using DDP instead.
So to make huggingface Trainer work on intel devices, this fix should be applied.



Comments:
> Hey! This seems to have been introduce in #25714, and thus I am not convinced that the fix is as simple as that! Would you mind also sharing a reproducer of the issue? Might be related to specific hard / soft versions

 Hi, @ArthurZucker! I have discussed with @abhilash1910 about the issue before this PR and we thought it can be fixed like this. This could be reproduced in a multi intel GPUs env, for the `distributedType `should be set to `MULTI_XPU` [in accelerator](https://github.com/huggingface/accelerate/blob/main/src/accelerate/state.py#L230). I think most example scripts using Trainer and more than one gpu should be reproducer.

@ArthurZucker Seems CI got block by Internet issue. Could you please check with that? Thanks

Seems like I can't would you mind merging with main to trigger it?! 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_27716). All of your documentation changes will be reflected on that endpoint.

Thanks both 🤗
Fix precision errors from casting rotary parameters to FP16 with AMP
Author: kevinhu
Pull request: yes

# What does this PR do?
When training with AMP, using `einsum` to multiply `t` and `self.inv_freq` will introduce precision errors because it casts the result to FP16. This can be avoided by using `torch.outer` instead, as originally mentioned here: https://github.com/Dao-AILab/flash-attention/blob/2c3baba4a63c4007c8a132c5380edc9430f88a22/flash_attn/layers/rotary.py#L396C1-L398C45

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @pacman100

Integrations:

- deepspeed: HF Trainer/Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc and @younesbelkada

Documentation: @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_27700). All of your documentation changes will be reflected on that endpoint.

FYI @gante and @Rocketknight1 if we see anything failing. I ran slow tests locally and it was all good
Add persistent_workers parameter to Dataloader
Author: Sorrow321
Pull request: no

### Feature request

**persistent_workers** is a parameter that can be passed to Dataloader. It's set to **False** by default. What it does ([quote](https://discuss.pytorch.org/t/what-are-the-dis-advantages-of-persistent-workers/102110/2)):

> With this option to false, every time your code hits a line line for sample in dataloader:, it will create a brand new set of workers to do this loading and will kill them on exit.
> Meaning that if you have multiple dataloaders, the workers will be killed when you are done with one instantly.
> 
> If you make them persist, these workers will stay around (with their state) waiting for another call into that dataloader.
> 
> Setting this to True will improve performances when you call into the dataloader multiple times in a row (as creating the workers is expensive). But it also means that the dataloader will have some persistent state even when it is not used (which can use some RAM depending on your dataset).

Currently there is no option to pass **persistent_workers=True** to internal Dataloader without inheriting from **Trainer** class. Here is the code snippet from **trainer.py**:
  
```
def get_train_dataloader(self) -> DataLoader:
    """
    Returns the training [`~torch.utils.data.DataLoader`].

    Will use no sampler if `train_dataset` does not implement `__len__`, a random sampler (adapted to distributed
    training if necessary) otherwise.

    Subclass and override this method if you want to inject some custom behavior.
    """
    # ...
    # (some code)
    # ...
    return DataLoader(
        train_dataset,
        batch_size=self._train_batch_size,
        sampler=train_sampler,
        collate_fn=data_collator,
        drop_last=self.args.dataloader_drop_last,
        num_workers=self.args.dataloader_num_workers,
        pin_memory=self.args.dataloader_pin_memory,
        worker_init_fn=seed_worker,
    )
```

I suggest to add this parameter to **TrainingArguments** class. Set it to **False** by default to not break anyone's pipeline. 

### Motivation

Motivation is very simple: setting this parameter to **True** can speed up training process in some cases (but will require more RAM). 

### Your contribution

I can make the PR

Comments:
cc @muellerzr 

A PR for this would be great @Sorrow321! :) 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
pip install transformers==4.34.1 not usable on SDXL automatic1111 - conflict with tokenizer and huggingface_hub
Author: DennisBalkan
Pull request: no

### System Info

pip install transformers==4.34.1 not usable on SDXL automatic1111 - conflict with tokenizer and huggingface_hub. it s very long test. try it yourself to update and you will see all conflict steps. 

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

updating without conflicts

### Expected behavior

updating without conflicts

Comments:
Hello, sorry I have no idea how to reproduce your issue or use this library. A snippet or a traceback would bel helpful! 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Rating on NVIDIA A30 GPU is lower than RTX2080 and equal to GTX1070
Author: ThangPetros
Pull request: no

### Model description

Currently I'm evaluating a model on HuggingFace, it's called model_name = 'nguyenvulebinh/wav2vec2-base-vi'

When running on GTX1070, I evaluated the average result between audio track time and text conversion time and the result was 63x
With RTX2080 it is 130x
As for the A30, it is 68x

Is there any conflict and why the A30 should have achieved the highest results but it did not meet expectations!

I'm a newbie so I really hope for help from everyone

Thank!
```python
#!pip install transformers==4.20.0
#!pip install https://github.com/kpu/kenlm/archive/master.zip
#!pip install pyctcdecode==0.4.0
#!pip install huggingface_hub==0.10.0

from transformers.file_utils import cached_path, hf_bucket_url
from importlib.machinery import SourceFileLoader
from transformers import Wav2Vec2ProcessorWithLM
from IPython.lib.display import Audio
import torchaudio
import torch

# Load model & processor
model_name = "nguyenvulebinh/wav2vec2-large-vi-vlsp2020"
model = SourceFileLoader("model", cached_path(hf_bucket_url(model_name,filename="model_handling.py"))).load_module().Wav2Vec2ForCTC.from_pretrained(model_name)
processor = Wav2Vec2ProcessorWithLM.from_pretrained(model_name)

# Load an example audio (16k)
audio, sample_rate = torchaudio.load(cached_path(hf_bucket_url(model_name, filename="t2_0000006682.wav")))
input_data = processor.feature_extractor(audio[0], sampling_rate=16000, return_tensors='pt')

# Infer
output = model(**input_data)

# Output transcript without LM
print(processor.tokenizer.decode(output.logits.argmax(dim=-1)[0].detach().cpu().numpy()))

# Output transcript with LM
print(processor.decode(output.logits.cpu().detach().numpy()[0], beam_width=100).text)`
```
### Open source status

- [X] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
Hey, the model you are using seems to be using `SourceFileLoader` , and custom code.
First I would recommend you to use `trust_remote_code = True` and properly register your model to avoid using `model = SourceFileLoader("model", cached_path(hf_bucket_url(model_name,filename="model_handling.py"))).load_module().Wav2Vec2ForCTC.from_pretrained(model_name)
p`. 
Now you have custom code and I can't debug it for you. If you find that the models we natively supports are indeed slower on better hardwares we could dig in but otherwise would recommend you to ask on the [forum](https://discuss.huggingface.co/) instead as we reserve github for issues or bugs.  

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add `persistent_workers` parameter to `TrainingArguments`
Author: Sorrow321
Pull request: yes

# What does this PR do?

This PR gives the users the ability to pass `persistent_workers` parameters as `TrainingArguments` parameter to `Trainer`. Without this PR, you have to inherit from `Trainer` class to alter the behavior of `get_train_dataloader` function.

This parameters does the [following thing](https://pytorch.org/docs/stable/data.html): 

> If True, the data loader will not shut down the worker processes after a dataset has been consumed once. This allows to maintain the workers Dataset instances alive.

[Explanation](https://discuss.pytorch.org/t/what-are-the-dis-advantages-of-persistent-workers/102110/2?u=ifedorov):

> With this option to false, every time your code hits a line line for sample in dataloader:, it will create a brand new set of workers to do this loading and will kill them on exit.
> Meaning that if you have multiple dataloaders, the workers will be killed when you are done with one instantly.
> 
> If you make them persist, these workers will stay around (with their state) waiting for another call into that dataloader.
> 
> Setting this to True will improve performances when you call into the dataloader multiple times in a row (as creating the workers is expensive). But it also means that the dataloader will have some persistent state even when it is not used (which can use some RAM depending on your dataset).



Fixes [#27058](https://github.com/huggingface/transformers/issues/27058)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
It adds description of the parameter to the docs as well.
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
Yes
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
https://github.com/huggingface/transformers/issues/27058
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
Yes
- [ ] Did you write any new necessary tests?
No

## Who can review?

- trainer: @muellerzr @pacman100


Comments:
Thanks @Sorrow321 🤗
[`ModelOnTheFlyConversionTester`] Mark as slow for now
Author: ArthurZucker
Pull request: yes

# What does this PR do?
The ModelOnTheFlyConversionTester were run on each individual PRs, should not be the case IMO this should only be a slow / only be run on the main branch and not fetched all the time. 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_27823). All of your documentation changes will be reflected on that endpoint.
Could we Add Linear Projection Layer in pre-trained model?
Author: DopeorNope-Lee
Pull request: no
Labels: name: New model

### Model description

As you know, if our model is too high positional_embedding size, then input vectors become too sparse data, so it leads to a performance decrease. 


However, I have an idea to overcome it. 

Before positional embedding, we add a new trainable linear projection layer to reduce its dimension. 

For example, If our model’s original positional embedding size is 4096, then we enlarge our input size as 8192.

Then trainable linear projection layer projects 8192 to 4096 and then we perform positional embedding.


let LP is a trainable Linear Projection Layer, PE(X) is positional Embedding.

Then our calculation formula is as follows.

"Original Structure"

X(R^4096)-> PE(X, where X is in R^4096)-> model(PE(X))


"Proposed Structure"

X(R^8192)-> LP(X, where X^8192) -> X'(R^4096) -> PE(X') -> model(PE(X'))


Can we run this with the current library? If not, can you add?

### Open source status

- [ ] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

_No response_

Comments:
[OWLv2] Add method
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR adds a new `image_guided_detection_v2` method, which unlike the original method, leverages the objectness head to get the top predicted object in the query image.

It also fixes the documentation example which has bad threshold values.

Fixes #26920

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_27003). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Inaccurate Inference Results When Using OWLv2's `image_guided_detection`
Author: Itto1992
Pull request: no

### System Info

- `transformers` version: 4.35.2
- Platform: Linux-5.15.120+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.19.4
- Safetensors version: 0.4.1
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.1.0+cu118 (True)
- Tensorflow version (GPU?): 2.14.0 (True)
- Flax version (CPU?/GPU?/TPU?): 0.7.5 (gpu)
- Jax version: 0.4.20
- JaxLib version: 0.4.20
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker
@younesbelkada 
@amyeroberts

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Run the sample code of OWLv2: https://huggingface.co/docs/transformers/model_doc/owlv2#transformers.Owlv2ForObjectDetection.image_guided_detection.example

### Expected behavior

- The bounding boxes intended to encircle two cats in the sample code appear in inappropriate positions.
- In the sample code (https://huggingface.co/docs/transformers/model_doc/owlv2#transformers.Owlv2ForObjectDetection.image_guided_detection.example), 13 boxes appear in locations unrelated to cats. I ran the exact same code in Google Colab and was able to replicate the same inference results.
- The previous model (OWLViT) was able to detect without any issues.

I am unsure if this behavior is expected, so I would appreciate some advice. If you are aware of any methods to improve the inference performance, please let me know.

Comments:
Hi,

This was addressed in #27698. It's already fixed on main: https://huggingface.co/docs/transformers/main/en/model_doc/owlv2#transformers.Owlv2ForObjectDetection

Thank you for your quick response!
I will try the main branch😁
[tests][models]An error about Owlv2ModelIntegrationTest.test_inference_object_detection
Author: Tyx-main
Pull request: no

### System Info

torch: 2.1.0
transformers: 4.35.2
accelerate: 0.24.1

### Who can help?

@ArthurZucker @younesbelkada 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

In the line 844 of tests/model/owlv2
/test_modeling_owlv2.py:
```
self.assertTrue(torch.allclose(outputs.logits[0, :3, :3], expected_slice_logits, atol=1e-4))
```

The function 'torch.allclose' expects that all the params should be on the same device, but the result seems that the 'outputs.logits[0, :3, :3]' is on the cuda with the 'expected_slice_logits' on the cpu, it makes the following error:
```
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
```

Is it a problem we can fix it?


### Expected behavior

Expected the function 'test_inference_object_detection' of  'Owlv2ModelIntegrationTest' can be passed on the cuda.

Comments:
Hi,

This was fixed in https://github.com/huggingface/transformers/pull/27793

> Hi,
> 
> This was fixed in #27793

Thanks！