noamaanMulla-03/transformers-issues huggingface dataset:
Fixed RAG script which was failing on dummy example
Author: kaustubhdhole
Pull request: yes

# What does this PR do?
Fixed RAG script which was failing on test_epoch_end in dummy example
The dummy example fails when test_epoch_end is called. The prefix="test" should be dynamic in the log metrics too.
Also, test_finetune.sh was failing when test file was not present.

@shamanez would be ideal to review.

Let me know if more information is needed.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi there. This is an unmaintained research project, so we normally don't accept PRs on it. You can try pinging the original authors and see if they accept your suggestion :-)

@sgugger  looks all good to me.

Thanks for having a look @shamanez and thanks for your contribution @kaustubhdhole !
ONNX support for encoder/decoder separately
Author: yanksyoon
Pull request: no

### Feature request

Hello, thanks for your work!

I am trying to leverage NVIDIA's TRITON inference server to speed up generation for Blenderbot model for production.

If I export the model in ONNX format, it exports the entire model whereas generative models go through a separate encoder->decoder cycles(if I understand correctly).

Would you be able to point me towards a direction on how I would be able to manage this?

I've tried:
1. writing an ONNX config and generating the model.onnx files separately for each encoder & decoder. Got stuck at converting encoder outputs to decoder inputs and such, mainly due to mismatch in ONNX model outputs and generative model outputs(such as hidden layers)
2. using the whole model exported as onnx and loading it into TRITON inference server. Also stuck at generation parameters & handling input/outputs, reason same as above.

### Motivation

To use generative model for blenderbot 3 on TRITON inference server

### Your contribution

Would love to make it work and share the results, either through PR for separate encoder/decoder onnx configs or sharing the results.

Comments:
Might be of interest to @lewtun 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hi @yanksyoon we have a new ONNX exporter in `optimum` [docs](https://huggingface.co/docs/optimum/exporters/onnx/usage_guides/export_a_model) that splits the encoder and decoder separately for fast inference with T5 models. I recommend opening your feature request there

cc @fxmarty 

Hi @yanksyoon , indeed Triton Inference Server should be quite good! For reference, the latest release should help to export separately encoder/decoder, see the section "Extended ONNX export for encoder-decoder and decoder models": https://github.com/huggingface/optimum/releases/tag/v1.6.0

We are very open to contributions to make the use of Triton Inference Server for decoder models smoother in Optimum. Feel free to open an issue there so that we can track and help!

@fxmarty @lewtun  Thanks for the guidance! Will have a look this weekend :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Generate: TF can now accept custom logits processors
Author: gante
Pull request: yes

# What does this PR do?

TF generation test addition PR 1 (out of ???).

In an effort to move generation integration tests to be framework-agnostic, I'll be adding low-hanging fruit to TF. This PR brings custom logits processors to TF `.generate()`. The code added is almost copy-paste from PT.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[examples] improve block_size warning message
Author: stas00
Pull request: yes

there is an odd warning inside the examples wrt model_max_length value e.g. in `run_clm.py`
```
01/28/2023 16:03:50 - WARNING - __main__ - The tokenizer picked seems to have a very large `model_max_length` 
(1000000000000000019884624838656). Picking 1024 instead. You can change that default value by passing --block_size xxx.
```

As the models now can work with much longer sequence lengths (bloom, opt, others) should it try to truncate it to 1024?

But actually for me what stood out is `1000000000000000019884624838656` - when I see such huge numbers it usually means a bug, so it is worrying and suggests that either I am doing something wrong or there is a bug somewhere.

so this PR is proposing to reword the message to be as informative, but not as scary:

```
01/28/2023 16:03:50 - WARNING - __main__ - The chosen tokenizer supports a `model_max_length` that is longer than the 
default `block_size` value of 1024. If you would like to use a longer `block_size` up to `tokenizer.model_max_length` you 
can override this default with `--block_size xxx`.
```

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[i18n-fr] Translate index page to French
Author: NoB0
Pull request: yes

# What does this PR do?

Translated the `index.mdx` file of the documentation to French. 

Part of #21456 

Thank you in advance for your review.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger, could you review this PR?

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts and @NielsRogge
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
OPT: BLIP2-ready `prepare_inputs_for_generation`
Author: gante
Pull request: yes

# What does this PR do?

This PR makes 2 changes to OPT's `prepare_inputs_for_generation`:
1. Adds the possibility of passing `inputs_embeds`
2. Removes the case when `attantion_mask is None`: a) it is redundant, the base case of inferring an attention mask with all ones is also in the model itself [here](https://github.com/huggingface/transformers/blob/baf4bacb1f10ecb63f0efc98d07463ae8799c7e3/src/transformers/models/opt/modeling_opt.py#L636) b) the shape isn't right when `inputs_embeds` is passed

✅ Slow OPT tests were run locally.

Comments:
cc @NielsRogge 

_The documentation is not available anymore as the PR was closed or merged._
Update quality tooling for formatting
Author: sgugger
Pull request: yes

# What does this PR do?

This updates the quality tools for 2023. Mainly:
- use black 23
- replace isort and flake8 by ruff, which is faster and merges imports contrarily to isort
- change a few rules in the isorting

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Incorrect type for TrainerArgs#report_to
Author: davidgilbertson
Pull request: no

### System Info

A minor thing...
The type of the `TrainerArgs` `report_to` argument is defined here: https://github.com/huggingface/transformers/blob/main/src/transformers/training_args.py#L901 as `Optional[List[str]]`, but the [docstring](https://github.com/huggingface/transformers/blob/main/src/transformers/training_args.py#L430) describes `Optional[str | List[str]]`. The default is wrong too.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

NA

### Expected behavior

NA

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

I would like to politely question the logic of auto-closing issues. Do you not want GitHub issues to be a register of things that should be fixed? Auto-closing just means this bug will live on to annoy another user at some point in the future, and maybe they'll report it too.
Deprecate parallelize API
Author: sgugger
Pull request: yes

# What does this PR do?

This PR deprecates the parallelize API now that the big model API has been tested for a bit. Using `device_map="balanced"` in the call to `from_pretrained` will do the same thing as the API, and it's still possible to pass along a custom `device_map` (although they are not in the same format).

Comments:
_The documentation is not available anymore as the PR was closed or merged._
[CI ] Remove `past` in favor of `pat_key_values`
Author: ArthurZucker
Pull request: yes

# What does this PR do?
Related to #20944, The `past` arg was removed.

Comments:
Other tests might also pass! 

_The documentation is not available anymore as the PR was closed or merged._

Git overides the `use_cache` arguments (only if labels are provided) to `False` see [here](https://github.com/ArthurZucker/transformers/blob/c2f9aacee9326b9db886036497a4d157666cb040/src/transformers/models/git/modeling_git.py#L1475-L1476). When generating, `use_cache` is set to false, but when we run `model.group_beam_search`, the ` self.prepare_inputs_for_generation(input_ids, **model_kwargs)` method forces `use_cache` to True see [here ](https://github.com/ArthurZucker/transformers/blob/c2f9aacee9326b9db886036497a4d157666cb040/src/transformers/models/git/modeling_git.py#L1516-L1534). 
EDIT: just update this tests will pass now. 

The failing tests are unrelated, will merge
Parallelize LongT5
Author: JamesDeAntonis
Pull request: no

### Feature request

Similar to regular T5, it'd be nice if LongT5 had parallelization support

### Motivation

This allows for training larger models / input sizes by using several gpus

### Your contribution

I have a pull request ready for this

Comments:
Hi,

We've just deprecated the parallelize API as it can now be done using the `from_pretrained` method.

See https://github.com/huggingface/transformers/pull/21448
ForcedBOSTokenLogitsProcessor take input_ids.shape[-1] as number of generated tokens
Author: Dounm
Pull request: no

### System Info

`ForcedBOSTokenLogitsProcessor` enforces the specified token as the first generated token.

In the code below, it takes `input_ids.shape[-1]` as the length of the generated tokens, but as I know, `input_ids.shape[-1]` equals to `prompt_length + generated_length`.
https://github.com/huggingface/transformers/blob/0db5d911fc94604f9568b4b212e005ec4600d157/src/transformers/generation/logits_process.py#L769

So, is this a bug, or there is something I've missing?

### Who can help?

@gante 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

None

### Expected behavior

None

Comments:
Hey @Dounm 👋 

`ForcedBOSTokenLogitsProcessor` is normally used with encoder-decoder models. For those models, when the generation loop begins, it only contains one token per batch/beam by default -- `model.generation_config.decoder_start_token_id ` or `model.generation_config.bos_token_ids`. As such, `ForcedBOSTokenLogitsProcessor` forces an additional token at the beginning of the sequence in those cases.

I hope this makes it clearer 🤗 

Much thanks for your rely!
Generate: TF can now generate from embeddings in encoder-decoder models
Author: gante
Pull request: yes

# What does this PR do?

TF generation test addition PR 2 (out of ???).

In an effort to move generation integration tests to be framework-agnostic, I'll be adding low-hanging fruit to TF. This PR brings the ability to generate from input embeddings, with encoder-decoder models (or, more specifically, from non-input_ids inputs). The code added is almost copy-paste from PT.

Since this PR made a few changes in the main code path for generate, the following slow tests were run to ensure XLA compatibility:
- [x] GPT2
- [x] T5

Comments:
_The documentation is not available anymore as the PR was closed or merged._

(merging -- the failing test, `test_from_pretrained_dynamic_model_distant` is a known flaky)
Add support of backward_prefetch and forward_prefetch
Author: raghavanone
Pull request: yes

#21156 

Adds support for backward_prefetch and forward_prefetch in trainer.

@sgugger @pacman100 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger Done, But not sure why this test is failing. Any pointers on how to make this build green would help.

@sgugger @pacman100  Need pointer on why this test is failing.

> The test is a flaky one, don't worry about it. Thanks for iterating, I just have one last comment on the deprecation warning for `fsdp_min_num_params` and we can merge this!

Done


@sgugger @pacman100  Can we merge
 this PR ? 

?Hello @raghavanone , could you please resolve the comments above that I have unresolved as they are yet to be addressed ?

> ?Hello @raghavanone , could you please resolve the comments above that I have unresolved as they are yet to be addressed ?

Done

Thank you @raghavanone for iterating and addressing the comments and for the overall contribution! 🚀
Add variant to transformers
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

This PR adds a `"variant"` keyword argument to PyTorch's `from_pretrained` and `save_pretrained` so that multiple weight variants can be saved in the model repo. 

You can try it out by running:

```python
from transformers import CLIPTextModel

path = "huggingface/the-no-branch-repo"  # or ./text_encoder if local

print("This should work!:")
model = CLIPTextModel.from_pretrained(path, subfolder="text_encoder", variant="no_ema")
print("This should work!:")
model = CLIPTextModel.from_pretrained(path, subfolder="text_encoder", variant="fp16")
print("This should work!:")
model = CLIPTextModel.from_pretrained(path, subfolder="text_encoder")
print("This should NOT work!:")
model = CLIPTextModel.from_pretrained(path, subfolder="text_encoder", variant="other")
```

From this repo:  https://huggingface.co/huggingface/the-no-branch-repo/tree/main/text_encoder . The repo is a dummy stable diffusion model and folder structure looks as follows:

```
├── feature_extractor
│   └── preprocessor_config.json
├── load.py
├── model_index.json
├── safety_checker
│   ├── config.json
│   └── pytorch_model.bin
├── save.py
├── scheduler
│   └── scheduler_config.json
├── text_encoder
│   ├── config.json
│   ├── pytorch_model.bin
│   ├── pytorch_model.fp16.bin
│   └── pytorch_model.no_ema.bin
├── tokenizer
│   ├── merges.txt
│   ├── special_tokens_map.json
│   ├── tokenizer_config.json
│   └── vocab.json
├── unet
│   ├── config.json
│   └── diffusion_pytorch_model.bin
└── vae
    ├── config.json
    └── diffusion_pytorch_model.bin
```

cc @pcuenca @patil-suraj @sgugger @LysandreJik @julien-c @osanseviero 


**[Update] This PR should be ready for merge**

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> pytorch_model.{variant}.bin sounds better to me, to keep the file-extension (not so important for .bin, but more important for .h5, .safetensors or any other format)

Even for `.bin` files, I'd say it's good to keep the file extension as it does not break the LFS property for existing `.gitattributes` files (see [huggingface/the-no-branch-repo](https://huggingface.co/huggingface/the-no-branch-repo/tree/main/text_encoder) where bin files are uploaded as regular).

Failing test is unrelated. Think this PR is good for merge. 

@wauplin @julien-c good for you? 

The resulting folder structure now looks as described in the PR statement: https://github.com/huggingface/transformers/pull/21332#issue-1559421203

Thanks for the reviews! Merging

cc @sgugger would it be possible to add this feature to `push_to_hub` as well?

I'd like to use it for BLIP-2. For the moment it seems the only way to do this is calling `save_pretrained("...", variant="fp16")` and then manually upload the PyTorch checkpoint to the model repo

Happy to review a PR.
BertTokenizer cannot properly tokenize words with dashes
Author: poteminr
Pull request: no

### System Info

- `transformers` version: 4.26.0
- Platform: Linux-5.10.147+-x86_64-with-glibc2.29
- Python version: 3.8.10
- Huggingface_hub version: 0.12.0
- PyTorch version (GPU?): 1.13.1+cu116 (False)
- Tensorflow version (GPU?): 2.9.2 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

### Who can help?

@ArthurZucker

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

The `BertTokenizer` doesn't tokenize words with dashes correctly. I tried to use tokenizer for **Italian** and **English** languages and got unexpected results. This issue similar to https://github.com/huggingface/transformers/issues/5136. 

```python3
from transformers import AutoTokenizer, BertTokenizer
model1 = 'cointegrated/rubert-tiny2'
model2 = 'Babelscape/wikineural-multilingual-ner' 
tokenizer = BertTokenizer.from_pretrained(model1, model_max_length=50)

>>> tokenizer.tokenize('so-called')
['so', '-', 'called']

>> tokenizer.tokenize('era il figlio di ghazi-ud-din haidar.')
['era', 'il', 'fi', '##gli', '##o', 'di', 'gh', '##azi', '-', 'ud', '-', 'din', 'hai', '##dar', '.']
```



### Expected behavior

Expected something like that:
```python3
>>> tokenizer.tokenize('so-called')
['so', '##-', '##called']

>> tokenizer.tokenize('era il figlio di ghazi-ud-din haidar.')
['era', 'il', 'fi', '##gli', '##o', 'di', 'gh', '##azi', '##-', '##ud', '##-', '##din', 'hai', '##dar', '.']
```

Comments:
I am not sure what you would like us to do about that. If we change the tokenizer, the model will get inputs different from its training and thus won't perform as well. You should use a different model with a tokenizer that suits your needs :-)

This is not an issue on our side, it's just the way the WordPiece algorithm works, given the corpus that the BERT authors trained on.

Check out our course for more info on tokenization algorithms: https://huggingface.co/course/chapter6/1?fw=pt

Closing this issue, feel free to reopen.
[docs] improve issue template for i18n
Author: wonhyeongseo
Pull request: yes

# What does this PR do?

The initial issue template (https://github.com/huggingface/transformers/pull/20199) includes minor typos and a closed PR link.
It makes it seem that the index page of a new language is already translated when it isn't.
Also some comment regarding what `langCode` or `langName` is would be helpful.

[x] Fixed typos.
[x] Removed irrelevant PR link.
[x] Explained what `langCode` or `langName` is more easily. (Replaced it rather)

<!-- Remove if not applicable -->

Fixes https://github.com/huggingface/transformers/issues/20955


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
May you please review this PR, @sgugger?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20956). All of your documentation changes will be reflected on that endpoint.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Typos and minor changes needed for i18n issue template
Author: wonhyeongseo
Pull request: no

The initial issue template (https://github.com/huggingface/transformers/pull/20199) includes minor typos and a closed PR link.
It makes it seem that the index page of a new language is already translated when it isn't.
Also some comment regarding what `langCode` or `langName` is would be helpful.

[ ] Fix typos.
[ ] Remove irrelevant PR link.
[ ] Explain what `langCode` or `langName` is more easily.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.