noamaanMulla-03/transformers-issues huggingface dataset:
[i18n-<languageCode>] Translating docs to <languageName>
Author: Alainterieur85
Pull request: no
Labels: name: WIP

<!--
Note: Please search to see if an issue already exists for the language you are trying to translate.
-->

Hi!

Let's bring the documentation to all the <languageName>-speaking community 🌐 (currently 0 out of 267 complete)

Who would want to translate? Please follow the 🤗 [TRANSLATING guide](https://github.com/huggingface/transformers/blob/main/docs/TRANSLATING.md). Here is a list of the files ready for translation. Let us know in this issue if you'd like to translate any, and we'll add your name to the list.

Some notes:

* Please translate using an informal tone (imagine you are talking with a friend about transformers 🤗).
* Please translate in a gender-neutral way.
* Add your translations to the folder called `<languageCode>` inside the [source folder](https://github.com/huggingface/transformers/tree/main/docs/source).
* Register your translation in `<languageCode>/_toctree.yml`; please follow the order of the [English version](https://github.com/huggingface/transformers/blob/main/docs/source/en/_toctree.yml).
* Once you're finished, open a pull request and tag this issue by including #issue-number in the description, where issue-number is the number of this issue. Please ping @ArthurZucker, @sgugger for review.
* 🙋 If you'd like others to help you with the translation, you can also post in the 🤗 [forums](https://discuss.huggingface.co/).

## Get Started section

- [x] [index.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/index.mdx) https://github.com/huggingface/transformers/pull/20180
- [x] [quicktour.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/quicktour.mdx) (waiting for initial PR to go through)
- [x] [installation.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/installation.mdx).

## Tutorial section
- [x] [pipeline_tutorial.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/pipeline_tutorial.mdx)
- [ ]  [autoclass_tutorial.mdx](https://github.com/huggingface/transformers/blob/master/docs/source/autoclass_tutorial.mdx)
- [ ]  [preprocessing.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/preprocessing.mdx)
- [ ]  [training.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/training.mdx)
- [ ]  [accelerate.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/accelerate.mdx)
- [ ]  [model_sharing.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/model_sharing.mdx)
- [ ]  [multilingual.mdx](https://github.com/huggingface/transformers/blob/main/docs/source/en/multilingual.mdx)

<!--
Keep on adding more as you go 🔥
-->

Comments:
MBart: Fix docs and doctests
Author: gante
Pull request: yes

# What does this PR do?

Related issue: #22397 

Most fixes are on the TF side. Elaborated a sentence in the main docstring -- MBart can be used for summarization, but none of the pre-trained models were trained for summarization.

Alongside this PR, TF checkpoints for MBart are being pushed to the Hub 🤗 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Default Datatype issue with model on OPT-13B
Author: lanking520
Pull request: no

### System Info

Any CPU Machine with Transformer 4.26.0

### Who can help?

_No response_

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [x] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained("facebook/opt-13b")
print(model.dtype)
```
torch.float32 is printed out

### Expected behavior

Expect to be float16. The model saved in the HuggingFace repo is under float16 format. Convert to Float32 may mess up the behavior.

Comments:
That is incorrect. The dtype of a model in PyTorch is always float32, regardless of the dtype of the checkpoint you saved. If you load a float16 checkpoint in a model you create (which is in float32 by default), the dtype that is kept at the end is the dtype of the model, not the dtype of the checkpoint. This is because many hardwares do not actually support other dtypes than float32 (for instance you won't be able to generate on the CPU if your model is in float16).

To load a model in float16, you have to ask explicitly with `torch_dtype=torch.float16` in your `from_pretrained` call. To load the model in the precision saved, you have to use `torch_dtype="auto"`.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
ValueError: Tokenizer class `NllbTokenizer` does not exist or is not currently imported when using NLLB (On Paperspace)
Author: svngoku
Pull request: no

### System Info

Hello,

When i use the code below
```py
...
 tokenizer = AutoTokenizer.from_pretrained("facebook/nllb-200-distilled-600M")
...
```

Models : `nllb-distilled-600M`

I got an error in my notebook instance (on `paperspace`) and I thought the problem was with the version of huggingface `(4.26.0.dev0)` even if I was on the right one it still doesn't work.

🤗
 

### Who can help?

_No response_

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```py
...
def load_models():
    # build model and tokenizer
    model_name_dict = {'nllb-distilled-600M': 'facebook/nllb-200-distilled-600M',
                  #'nllb-1.3B': 'facebook/nllb-200-1.3B',
                  #'nllb-distilled-1.3B': 'facebook/nllb-200-distilled-1.3B',
                  #'nllb-3.3B': 'facebook/nllb-200-3.3B',
                  }

    model_dict = {}

    for call_name, real_name in model_name_dict.items():
        print('\tLoading model: %s' % call_name)
        model = AutoModelForSeq2SeqLM.from_pretrained(real_name)
        tokenizer = AutoTokenizer.from_pretrained(real_name)
        model_dict[call_name+'_model'] = model
        model_dict[call_name+'_tokenizer'] = tokenizer

    return model_dict
...
```

### Expected behavior

See my model working as except on the Gradio space

Comments:
Bug resolved ! 
I relaunch my instance many times and run this command `!pip3 install git+https://github.com/huggingface/transformers.git`

I have the same problem. help!!!
Why ViltProcessor couldn't resize image to 384*384?
Author: Yam0214
Pull request: no

### System Info


- `transformers` version: 4.27.1
- Platform: Linux-5.15.0-52-generic-x86_64-with-glibc2.35
- Python version: 3.10.8
- Huggingface_hub version: 0.11.1
- PyTorch version (GPU?): 1.13.1+cu117 (True)
- Tensorflow version (GPU?): 2.11.0 (False)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@amyeroberts

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

```python
import numpy as np
from transformers import ViltProcessor

# create two image
shape_list = [(283, 400), (550, 469)]
fake_imgs = [np.random.randint(0, 255, (3,) + shape) for shape in shape_list]

# preprocess
processor = ViltProcessor.from_pretrained("dandelin/vilt-b32-mlm")

inputs = processor(
    fake_imgs, 
    ["hello word"] * len(shape_list),
    return_tensors="pt"
    )
# check shape
print(inputs.pixel_values.shape)
# torch.Size([2, 3, 448, 512])

```


### Expected behavior

It seems that `ViltProcessor` is supposed to resize image to 384 * 384 according to `preprocessor_config` (https://huggingface.co/dandelin/vilt-b32-mlm/blob/main/preprocessor_config.json). 
However, the script below gets 448 * 512.

Comments:
Hi @Yam0214, 

This is because of the padding step that occurs when processing the image - cf [this line](https://github.com/huggingface/transformers/blob/b29fd6971d9cd6ba2a824628effe243f543b8f61/src/transformers/models/vilt/image_processing_vilt.py#L475). This pads all the images in the batch to the largest height and width dimensions in the batch. 

The largest height and width dimensions in the batch are `(448, 512)`. This is because the first image is resized to `(3, 384, 512)` and the second to `(3, 448, 384)`. The reason these images are not resized to `(3, 384, 384)` is because this models preprocessing follows torchvision [resizing logic](https://pytorch.org/vision/main/generated/torchvision.transforms.Resize.html) where: 
* `size` is 384
* `max_size` is `int(1333 / 800 * 384)`

@amyeroberts Thank you for getting back to me, it's very helpful.
whisper model's default task should be "transcribe"
Author: chenhaitao2026
Pull request: no

### System Info

- `transformers` version: 4.27.2
- Platform: Linux-3.10.0-1062.9.1.el7.x86_64-x86_64-with-glibc2.17
- Python version: 3.9.16
- Huggingface_hub version: 0.13.2
- PyTorch version (GPU?): 1.12.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: <fill in>
- Using distributed or parallel set-up in script?: <fill in>

### Who can help?

@sanchit-gandhi  @ArthurZucker

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

In transformers v4.26.1, the following script will output right language, some chinese text. It's the right task "transcribe".
However, in version 4.27.2, it will output translated english text, which is another task "translate".

[<script src="https://gist.github.com/chenht2010/174f2480641b6780cbccd588431176b8.js"></script>](https://gist.github.com/chenht2010/174f2480641b6780cbccd588431176b8)





### Expected behavior

Do ASR, and Output chinese

Comments:
cc @ArthurZucker and @sanchit-gandhi 🙏 

Hey! As you can see [here](https://github.com/ArthurZucker/transformers/blob/d2854e753bcfd62dce2f968d6088232d0fc41f8c/src/transformers/models/whisper/modeling_whisper.py#L1586) the default (if the generation_config does not have a `task` set is still `transcribe`. What changed is the `configuration.json` see this [commit](https://huggingface.co/openai/whisper-large-v2/commit/e823955b7861a1d66fef509b8601ada6d7762c03) where the default went from `transcribe` ( 50358) to `translate` (50359 in the forced_decoder_ids). The update in transformers just makes sure to properly use this, while the previous version did not take it into account. 

This is more a fix than a breaking change IMO

Thank you for your explanation.
Add clean_up_tokenization_spaces to config
Author: ArthurZucker
Pull request: yes

# What does this PR do?
DRAFT

Comments:
_The documentation is not available anymore as the PR was closed or merged._

cc @Narsil 

Also linked to #20846. A follow up PR can now be made to add a simple warning if the default value is set to `True` / put the default value to True

It seems like no model defaulted to use `cleanup_tokenization_spaces = False` so this should be seamless. Otherwise the tokenizer's `__init__` should be updated
[DeBerta V2] The vocab size of DeBerta V2 is incorrect
Author: hjptriplebee
Pull request: no

### Who can help

@LysandreJik

## Information

I am using Deberta V2. The document and vocab of the tokenizer shows that its vocab size is 128K. But the vocab size in its [configuration](https://huggingface.co/microsoft/deberta-v2-xlarge/resolve/main/config.json) is 128100 which is 100 larger.
![image](https://user-images.githubusercontent.com/22477665/123915975-ae34c780-d9b3-11eb-916a-1159e04922e6.png)
It will make embedding not work as expected.

## Expected behavior

May be we should modify the vocab size in configuration?


Comments:
Maybe @BigBird01 can chime in :)

@LysandreJik I emailed to the author. He leave some space to add customized tokens for downstream tasks and this is by design.

v2/v3: 128000 + 100 (special tokens, unused tokens, e.g. reservation for PostOCR customized tokens) ---> 128100 in total

https://huggingface.co/transformers/v4.9.2/model_doc/deberta_v2.html
Hyperparameter search reporting to W&B
Author: NoB0
Pull request: yes

# What does this PR do?

Fixes #22429


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker and @younesbelkada
- vision models: @amyeroberts
- speech models: @sanchit-gandhi
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @gante
- pipelines: @Narsil
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @sgugger

Integrations:

- deepspeed: HF Trainer: @stas00, Accelerate: @pacman100
- ray/raytune: @richardliaw, @amogkam

Documentation: @sgugger, @stevhliu and @MKhalusova

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: @sgugger
- TensorFlow: @Rocketknight1

 -->


Comments:
_The documentation is not available anymore as the PR was closed or merged._
[`bnb`] fix bnb failing test
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes https://github.com/huggingface/transformers/actions/runs/4538560416/jobs/7997676956

The PR https://github.com/huggingface/transformers/pull/22377 introduced the correct way to compute the device_map for int8 models to avoid issues in some corner cases. 

The `max_memory` argument was causing some issues and led to failing test due to the presence of a non-empty `special_dtypes`. Script to reproduce:

```python
import torch
from accelerate import init_empty_weights, infer_auto_device_map
from transformers import BloomForCausalLM, AutoConfig

config = AutoConfig.from_pretrained("bigscience/bloom-560m")

max_memory = {0: 1000000000, 1: 1000000000}

with init_empty_weights():
    model = BloomForCausalLM(config)

torch_dtype = torch.float16
modules_not_to_convert = ['transformer.word_embeddings', 'lm_head']
special_dtypes = {
    name: torch_dtype
    for name, _ in model.named_parameters()
    if any(m in name for m in modules_not_to_convert)
}

device_map = infer_auto_device_map(model, max_memory=max_memory, no_split_module_classes=['BloomBlock'], dtype=torch.int8, special_dtypes=special_dtypes)

print(set(device_map.values()))
```
I assume that the previous test was too much a corner case (with hardcoded `max_memory`), the correct fix should be just to use `balanced` in the `device_map` argument.

cc @sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Beam search not always better Greedy search
Author: vermouthmjl
Pull request: no

@patrickvonplaten

I was recently tinkering around the generation strategies for a decoder, and came across this [blog post](https://huggingface.co/blog/how-to-generate) on decoding methods. It was quite useful to clarify how generation works for transformers. Thank you for this excellent work.
I notice that in the section on Beam Search, it is stated that:

> **Beam search will always find an output sequence with higher probability than greedy search**, but is not guaranteed to find the most likely output.

It seems to me that the highlighted part is not true, which can illustrated by the following, extended from the example in your blog post.
 
![image](https://user-images.githubusercontent.com/3142085/228549175-d37bb8af-ed62-44bf-93f8-38d22bf6aba3.png)

Basically if better beams at a certain time are followed by more tokens in which the preference of tokens is not that strong (distribution being flatter), then the greedy result could have a higher probability, and being missed out by the beam search.
I got curious about this claim, because I tried beam search and greedy search in a vision encoder decoder model (Donut), and got lower probability generations quite often with beam search.

What do you think? Is there something else that I over-looked in this example?

Thank you and have a nice day.

Comments:
Actually, copied to the issues section of the blog.
Don't hard error when cache version can't be converted to int
Author: sgugger
Pull request: yes

# What does this PR do?

As reported in #22412 if the string stored in the cache version file is not an int, all import from the Transformers library will fail. This PR adds more safeguards.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Is `ValueError` the only kind of error that can be returned when this fails?

Yup
Revert "Error (also in original) model, scaling only q matrix not qk.T dot product (qk.T/sqrt(dim_per_head))"
Author: sgugger
Pull request: yes

Reverts huggingface/transformers#21627

This PR changed the modeling code and as a result broke training as reported in #22426 
Moreover, since the "error" was in the original code from the authors, it's not an error anymore but a feature.

This will be put in the patch v4.27.4

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_22444). All of your documentation changes will be reflected on that endpoint.
Problem of training gpt-neox from transformer 
Author: pure-rgb
Pull request: no

### System Info

```
PyTorch
Transformer

2x A100 (40 GB)
```




### Who can help?

_No response_

### Information

- [X] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

I couldn't fine tune the gpt-neox 20b from transformer library. Also, there is no official documentation how to train or fine tune the LLM with ease with transformer library. 

https://huggingface.co/EleutherAI/gpt-neox-20b

### Expected behavior

It should run.

Comments:
Maybe you should try getting help on the [forums](https://discuss.huggingface.co/)? We only use GitHub issues for bugs and feature requests only, and I don't think this is relevant here.

@sgugger Unfortunately this forum is not much helpful, you can search with 'gpt-neox' word, and you'd find many questions without response. Not blaming, but I like to know if transformer support training with this large LM model or just the inference? The documentaiton is missing for this model, or I've missed! 



Transformers supports training and inference of all its models, but you will probably need to use other libraries like DeepSpeed or PyTorch FSDP to do such training depending on your hardware. Since this is a model that will 80GB in float32 by itself, it's very unlikely you will be able to train it on the hardware your provided at the top. Training those models from scratch require a lot more GPU compute (which is also why there are fewer tutorials available as it doesn't touch as many people as smaller models).

You might have more luck fine-tuning the exisiting model with the [peft library](https://github.com/huggingface/peft). Otherwise this [doc page](https://huggingface.co/docs/transformers/perf_train_gpu_many) will list resources you can use.
[`Pix2Struct`] Fix slow test
Author: younesbelkada
Pull request: yes

# What does this PR do?

Fixes: https://github.com/huggingface/transformers/actions/runs/4538560416/jobs/7997667941 

This test was currently failing due to the fact that I forgot to add `.to(torch_device)` on a slow test

cc @sgugger 


Comments:
Revert "Fix --bf16 option support for Neuron after PR #22300"
Author: jeffhataws
Pull request: yes

This reverts commit fd81746dbec5f17c8285a0fdc72ca4b4c025cc33.

# What does this PR do?

This reverts https://github.com/huggingface/transformers/pull/22307 as CPU AMP doesn't cause torch/xla to emit the correctly autocasted XLA HLO, while GPU AMP path correctly emits autocasted XLA HLO. We are left with "RuntimeError: No CUDA GPUs are available" as noted in the previous PR which can be workaround with "torch.cuda.is_bf16_supported = lambda: True".

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@sgugger


Comments:
Avoid sgugger become THE bot
Author: ydshieh
Pull request: yes

# What does this PR do?

With more and more such 
<img width="380" alt="Screenshot 2023-03-29 192000" src="https://user-images.githubusercontent.com/2521628/228618109-0f4f33d7-db0b-43a5-b425-7dac596448d3.png">
`sgugger` will become a bot one day soon (or at least Google will think `sgugger` is a bot).

Let's save them from becoming a bot.

------
Serious part: This is discussed offline

> Would be good to create HF-testing Bot user and use that token as people really think it's me updating those models :sweat_smile:

**I already set this secret in the repository.**

Comments:
> 🤖 🤖 🤖 LGTM! 🤖 🤖 🤖

I am not sure if sgugger will say LGTM 😆 but I am going to merge!

_The documentation is not available anymore as the PR was closed or merged._