noamaanMulla-03/transformers-issues huggingface dataset:
Fix type annotations on neftune_noise_alpha and fsdp_config TrainingArguments parameters
Author: peblair
Pull request: yes

# What does this PR do?

This PR updates the type annotations on the `neftune_noise_alpha` and `fsdp_config` parameters in `TrainingArguments` to reflect their full range of values. Currently, if one attempts to round-trip a `TrainingArguments` object with a data validation library such as `pydantic`, re-serialization fails due to these missing annotations.


## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@pacman100 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28942). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
pass kwargs in stopping criteria list
Author: zucchini-nlp
Pull request: yes

# What does this PR do?

This PR passes kwargs to each criteria when calling `StoppingCriteriaList`.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.
@gante 


Comments:
@zucchini-nlp Thanks for opening this PR! Can you provide some more context on why this is necessary? Passing around kwargs is a behaviour we try to avoid if possible. 

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28927). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@amyeroberts the original API, designed before we joined, expects `**kwargs` to be passed ([e.g.](https://github.com/huggingface/transformers/blob/0b693e90e0748e16427a2764d516e9f5ba801bcc/src/transformers/generation/stopping_criteria.py#L65)). In other words, if individual stopping criteria classes support it, so should the class that wraps them as a list :)

@michaelbenayoun pointed it out today, I'm assuming he's planning to use it :p
transformers.onnx support mistral 
Author: xiaoyaolangzhi
Pull request: no

### Feature request

mistral is not supported yet

### Motivation

*

### Your contribution

*

Comments:
Hey! I think you should ask this on the Optimum repo where the onnx conversio should already be supported no @fxmarty 

Thank you @xiaoyaolangzhi. As indicated in the documentation (https://huggingface.co/docs/transformers/v4.34.0/en/serialization#export-to-onnx), the ONNX export through transformers.onnx has been deprecated and moved to the Optimum library. The support for Mistral ONNX export has been added here: https://github.com/huggingface/optimum/pull/1425, but is not yet available in a release of Optimum, so you would need to install th lib from source.

Then you can use:
```
optimum-cli export onnx --help
optimum-cli export onnx --model mistralai/Mistral-7B-v0.1 mistral_onnx
```

@fxmarty 

I ran the as stated by you in Colab. But execution halts 

pip install optimum[exporters-tf]
!optimum-cli export onnx --model mistralai/Mistral-7B-v0.1 mistral_onnx

<img width="659" alt="image" src="https://github.com/huggingface/transformers/assets/141845438/8ac1f077-0d34-43c8-af2d-85cb5d7b7963">
[i18n-de] Translate README.md to German
Author: khipp
Pull request: yes

# What does this PR do?

This PR adds a German translation for the README.md.
The text is written in accordance with the [translation guidelines](https://github.com/huggingface/course/blob/main/chapters/de/TRANSLATING.txt) of the German Hugging Face course, and translations for technical terms are taken from the corresponding [glossary](https://huggingface.co/learn/nlp-course/de/glossary/1).

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

Documentation: @stevhliu

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28933). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

I don't know anyone in particular, I'm afraid.

Maybe @flozi00 can help me since he contributed to the existing German translations? :grin:

I remember for an discussion about the form if we should "du" or "sie"
Last time the hugging face team decided for the more formulary "sie"

The rest LGTM

Thanks, I will update the translation accordingly.

It seems that the German translation for the Hugging Face Hub also uses the formal 'Sie', while the Hugging Face NLP Course is the outlier here.

@flozi00 I made the appropriate changes and did some minor edits. Ready for another review.
Unclear Tokenizer Algorithm Documentation
Author: kitkhai
Pull request: no

**Bug description.**
In the docs, for example for [NLLB ](https://huggingface.co/docs/transformers/model_doc/nllb), the "slow"  & "fast" tokenizers are documented to be based on SentencePiece & BPE respectively. 
I do think that is a little confusing as:
- Saying that the "slow" tokenizer is based on SentencePiece, it is unclear whether is it implementing the BPE or unigram model?
- Saying that the "fast" tokenizer is based on BPE, is also unclear whether it refers to SentencePiece's implementation of BPE? Or just the standalone BPE algorithm? 

**Describe the expected behaviour**
Can be more explicit when describing what the tokenizer is based on, such as:

- Unigram(spm)
- BPE (spm)
- Unigram
- BPE

Comments:
Hey! Thanks for raising this issue. Do you want to open a PR for a fix? 🤗 

Hey! Thanks for the swift response.

I'm still rather new and am actually confused about what the "slow" & "fast" Tokenizers are based on... (I don't really understand why one is based on "sentencepiece" and the other is based on "BPE"? Aren't they supposed to be trained the same way just implemented in different programming languages?)

Hence, I don't think I am in a position to open a PR for a fix? Sorry about that 😬

Your intuition is correct, the slow version uses the `sentencepiece` backend, so the BPE implementation of `sentencepiece` library. While the fast uses the `tokenizers` backend, with the `BPE` implementation of `tokenizers` that is based on `sentencepiece`! 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Simple Bug in modeling_attn_mask_utils.py
Author: Adam1679
Pull request: no

### System Info

(torch) (base) anxiang.zhang@n214-176-142:~/DeepSeek-Coder$ transformers-cli env
WARNING:tensorflow:From /data02/home/anxiang.zhang/miniconda3/envs/torch/lib/python3.10/site-packages/transformers/commands/env.py:100: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.config.list_physical_devices('GPU')` instead.
An NVIDIA GPU may be present on this machine, but a CUDA-enabled jaxlib is not installed. Falling back to cpu.

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

- `transformers` version: 4.36.0
- Platform: Linux-5.4.56.bsk.9-amd64-x86_64-with-glibc2.28
- Python version: 3.10.13
- Huggingface_hub version: 0.20.1
- Safetensors version: 0.4.1
- Accelerate version: 0.25.0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.1.2 (True)
- Tensorflow version (GPU?): 2.9.3 (True)
- Flax version (CPU?/GPU?/TPU?): 0.7.4 (cpu)
- Jax version: 0.4.18
- JaxLib version: 0.4.18
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

At transformers.modeling_attn_mask_utils.py:238.

The code is 
```
tmp = torch.arange(attention_mask.shape[1], 0, -1)
indices = torch.argmax(attention_mask.cpu() * tmp, 1, keepdim=True)
````

The attention_mask.cpu() is clear an error when the global default tensor type is not a cpu dtype. For example, if you set torch.set_default_tensor_type(torch.cuda.HalfTensor). then the  torch.arange(attention_mask.shape[1], 0, -1) would return a tensor on cuda instead of CPU, which will lead to error by multiplying a CPU tensor with CUDA tensor. 

A simple fix would be replace attention_mask.cpu() as attention_mask.to(tmp.device)

### Expected behavior

A simple fix would be replace attention_mask.cpu() as attention_mask.to(tmp.device)

Comments:
Makes sense, would you like to open a PR for the fix? 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
TabR: Retrieval-Augmented Tabular Deep Learning
Author: Samoed
Pull request: no
Labels: name: New model

### Model description

TabR is a retrieval-augmented transformer model specifically designed for tabular data tasks.


I would like to add this model, but I'm not sure if it would fit in Transformers or if it should be a separate library.

### Open source status

- [X] The model implementation is available
- [ ] The model weights are available

### Provide useful links for the implementation

[Paper](https://arxiv.org/abs/2307.14338)
[Author's implementation](https://github.com/yandex-research/tabular-dl-tabr)

Comments:
Does m2m_100 support multiple forced_bos_token_id?
Author: sfc-gh-zhwang
Pull request: no

```
generated_tokens = model.generate(input_ids = ids_tensor, attention_mask = attention_padded, forced_bos_token_id=[tokenizer.get_lang_id("es"),tokenizer.get_lang_id("fr")])
```
Seems above code will output the same language output.

Comments:
You should use `forced_decoder_ids` https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.GenerationConfig.forced_decoder_ids 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
[`Don't merg`] Tokenizer-release
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Test that everything works fine

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_28957). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.
Switch Transformers Jitter Noise in Inference
Author: drunkcoding
Pull request: no

### System Info

- `transformers` version: 4.36.2
- Platform: Linux-5.15.0-1033-gkeop-x86_64-with-glibc2.17
- Python version: 3.8.18
- Huggingface_hub version: 0.20.1
- Safetensors version: 0.3.1
- Accelerate version: not installed
- Accelerate config: not found
- PyTorch version (GPU?): 2.1.2+cu121 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: False

### Who can help?

@ArthurZucker 

Switch Transformers add jitter noise 
```python
if self.jitter_noise > 0:
```

According to the paper, jitter noise can only be added during training 

![image](https://github.com/huggingface/transformers/assets/14305648/1f1ac27a-375a-42d0-8bcc-4c5402221715)


### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
model = AutoModel.from_pretrained("google/switch-base-128")
tokenizer = AutoTokenizer.from_pretrained("google/switch-base-128")
input_text = "Hello, my dog is cute"
input_ids = tokenizer(input_text, return_tensors="pt").input_ids
attention_mask = torch.ones_like(input_ids)

set_seed(42)

outputs = model.generate(
    input_ids=input_ids,
    attention_mask=attention_mask,
    max_length=20,
    decoder_start_token_id=0,
    do_sample=False,
    return_dict_in_generate=True,
    output_attentions=False,
    output_hidden_states=False,
)
```

### Expected behavior

Output is different every time running generate

Comments:
Hey! Before I can help you, could provide an actual reproducer? 
This one fails with `TypeError: The current model class (SwitchTransformersModel) is not compatible with `.generate()`, as it doesn't have a language model head. Please use one of the following classes instead: {'SwitchTransformersForConditionalGeneration'}`.


Moreover when I use the correct class, disabeling `return_dict_in_generate` for convenience, I always get the same output. 
https://github.com/huggingface/transformers/blob/90224dd59e92e11d99b5b09be84d3fe7794636b9/src/transformers/models/switch_transformers/modeling_switch_transformers.py#L171 

adds jitter only if training

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Insertion Constraint
Author: massabaali7
Pull request: yes

My contribution lies mainly in the constraints class, which is represented in the constrained beam search. It enables conditional token injection in the constrained beam search. The new constraint allows the insertion of one or more tokens from a list of words into the output. Also, you have the ability to not insert anything.  
Example: 
insertfromListOfWords = ["uh","um","exactly", "yes"] 
possible_outputs == [
   "The woman went exactly to school.",
   "um the woman went to um uh school",
 "The woman went to school",
 ]

Comments:
cc @gante 

@amyeroberts @gante did you get the chance to review my code?
Thanks in advance 

Hi @massabaali7 👋 

It is my understanding that constrained beam search is not very used by the community. As such, I'm not adding more code related to it at the moment, as we have limited bandwidth to empower the community 🤗 

I am doing a research and publishing a paper about this. Its a very important idea proposed for backchannel insertion. 

@massabaali7 and we'd be delighted to add it, if the published results show significant improvements. However, we do not add features while they are in a research phase -- that's why we made the whole codebase open to everyone, so you can freely experiment locally or in a forked repository 🤗 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Memory leak (not released) when calling Seq2SeqTrainer for fine-tuning
Author: xyx361100238
Pull request: no

### System Info

- `transformers` version: 4.34.0
- Platform: Linux-5.4.0-167-generic-x86_64-with-glibc2.31
- Python version: 3.9.18
- Huggingface_hub version: 0.17.3
- Safetensors version: 0.4.0
- Accelerate version: 0.23.0
- Accelerate config:    not found
- PyTorch version (GPU?): 1.13.1+cu116 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@sanchit-gandhi  @muellerzr @pacman100

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I was use project finetune.py from [Whisper-Finetune](https://github.com/yeyupiaoling/Whisper-Finetune)  to finetune whisper large on my own datasets，which code use transformers  library. If I use the validation process during fine-tuning, it will lead to an increase in system memory, and there is a chance that it will not be released after validation. Over time, this can lead to Out Of Memory (OOM) and cause a crash. Is this a bug in the tool, or do I need to make some settings?

### Expected behavior

After each verification is completed, normalize the memory to maintain it at a stable level.

Comments:
Hey @xyx361100238! Do you have a script I can use to reproduce the error and identify the memory increase? It's maybe worth trying adding the flag for full fp16 eval as well to avoid upcasting the weights to fp32 for inference:

```
--fp16_full_eval
```

_c.f._ https://huggingface.co/docs/transformers/v4.36.1/en/main_classes/trainer#transformers.TrainingArguments.fp16_full_eval

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Adding flash attention to GPT2
Author: canberk17
Pull request: yes

# What does this PR do?
Adding Flash Attention2 to GPT2, here are my tests:

![image](https://github.com/huggingface/transformers/assets/33362633/ebece185-023b-4816-81b5-928a4c10e753)



<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

Contributing to : [#26350](https://github.com/huggingface/transformers/issues/26350)


## Who can review?
Hey guys @younesbelkada @ArthurZucker, could you please review it when you get a chance.

I was trying to debug why I was getting these test failures,  some of them point to falcon model ( even though I haven not touched that file ). 

Also I ran the flash attention test on another model that has been merged, and these are the test results I am getting: 
![image](https://github.com/huggingface/transformers/assets/33362633/7e0191c6-8d02-43cb-8ead-c7baeeff3a7e)

am I on the right path here?  I couldn’t    address why some of these test failures are happening. 


Comments:
@younesbelkada,  thanks for the tips ! Now most of the tests  are passing. However, I'm facing a challenge with to address the issues in the following test:

`tests_torch failure` referring to `Speech2TextModelTest` which is puzzling since I haven't made changes to this part of the code.

For `tests_torch`  I don't know how the tests on google colab passed. I used the following statement to run these test both for  the files in this branch:
`!RUN_SLOW=1 pytest -sv --disable-warnings -k flash_attn_2 tests/models/gpt2/test_modeling_gpt2.py
`
Similarly, I used the same command for the recently merged GPT variant:
`!RUN_SLOW=1 pytest -sv --disable-warnings -k flash_attn_2 tests/models/gpt_bigcode/test_modeling_gpt_bigcode.py
`
and I got the results mentioned in my comment. I am wondering if I am executing these tests on Colab properly, as I don't see any error messages when I run these tests.  When you get a change could you give me some insights on my approach  please  

cc @younesbelkada 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.