noamaanMulla-03/transformers-issues huggingface dataset:
Translate index.mdx (to ES) and add Spanish models to quicktour.mdx examples
Author: omarespejel
Pull request: yes

# What does this PR do?
Related to issue #15947 
1. Translate `index.mdx` to Spanish
2. Replace English models and a dataset in `quicktour.mdx` with Spanish versions.

## Relevant
- Translation of `index.mdx` included the list of compatible models (not the papers´ and models´ names). Since this should not be updated manually, I can come back to the original text if required.
- Spanish models selected for quicktour are:
  - clasificador = pipeline('sentiment-analysis', model="pysentimiento/robertuito-sentiment-analysis")
  - reconocedor_de_voz = pipeline("automatic-speech-recognition", model="jonatasgrosman/wav2vec2-large-xlsr-53-spanish", device=0)
- Spanish ASR dataset selected is:
  - dataset = datasets.load_dataset("PolyAI/minds14", name="es-ES", split="train") 
 



## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger, I ran `make style` and made some manual debug but cannot get `check_code_quality` to pass. Any idea why could it be? 

I think the error is in[ these lines](https://github.com/huggingface/transformers/pull/16685/files#diff-517e793a1e18859abaf368b0c3f4d344231747ebc1ceb5154ca94159a7207bf1R110-R116):

`
A continuación, carga el dataset (ve 🤗 Datasets [Quick Start](https://huggingface.co/docs/datasets/quickstart.html) para más detalles) sobre el que quisieras iterar. Por ejemplo, vamos a cargar el dataset [MInDS-14](https://huggingface.co/datasets/PolyAI/minds14):

```py
>>> from datasets import load_dataset, Audio

>>> dataset = load_dataset("PolyAI/minds14", name="es-ES", split="train")  # doctest: +IGNORE_RESULT
```
`

Make sure you have the latest version of `hf-doc-builder` installed (`pip install hf-doc-builder -U`) then run `make style`.

EDIT: Fixed by @osanseviero in PR #17197.

Sorry @sgugger, even with updating `hf-doc-builder` to the latest version, `0.4.0`,  and running `make style` the `check_code_quality` error continues appearing. I am not able to find the error and the [feedback in CircleCI](https://app.circleci.com/pipelines/github/huggingface/transformers/39893/workflows/35203812-3c00-42fa-b7e6-2d1789c21b12/jobs/449693) is not revealing (`ValueError: 1 files should be restyled!`).

When running `make style` I get this error (I tried to debug in other ways but the error continues):
![Screen Shot 2022-05-11 at 23 45 35](https://user-images.githubusercontent.com/4755430/167993881-683ab4e8-7662-4697-9d1e-8e5e929ddd21.png)

In the meantime, I merged the PR with the error so we can get the index and quicktour before the release tomorrow. However, please let me know if we should proceed in another way. Sorry for this cumbersome merge. They are looking fine in the docs:
- [Index](https://huggingface.co/docs/transformers/main/es/index);
- [Quicktour](https://huggingface.co/docs/transformers/main/es/quicktour).
[RFC] Laying down building stone for more flexible ONNX export capabilities
Author: mfuntowicz
Pull request: yes

This PR aims at reworking the way the ONNX export tool work by introducing a static, checked description format to provide ONNX exporters (pt almost done, TF will follow) all the required knobs.

More specifically this PR introduces the following concepts:
- `OnnxConfig` dataclass which enforces a model to be supported to describe all the properties to generate proper export
- `OnnxVariable` namedtuple which describe a variables w.r.t  the name of the variable, shape and potentially how many time it's "repeated" => Useful for `past_keys` 

Test case was done initially for BART model, without `use_cache=True` supports. 

For the sake of completeness, dropping support for `use_cache=True` is currently needed because we have a double nested tuple at the core of the `past_keys` output structure which would require multiple level of dynamic axis, not currently supported by ONNX.

This might be something we can work on in the future, potentially introducing a ONNX compatible output structure getting rid of the nested tuples layout and activable from a config property (_to be discussed further later on_).

**Update 1:** 

- I managed to enable exporting with nested structures such as `past_key_values` for GPT2.
- Need to work on enabling the same for using such values as inputs to the model

Supported models: 
- [x] ALBERT
- [x] BART (with & without past)
- [x] BERT 
- [x] DistilBERT
- [ ] Longformer => I've support for this, but the exporting fails because of missing ops ... need investigations.
- [x] GPT2 (with & without past)
- [x] Roberta
- [x] T5
- [x] XLM-Roberta

Comments:
Example of potential command line to export `bert-base-cased` => 

`python3 -m transformers.onnx  -f pytorch --model=bert-base-cased --features=default --optimize --optimization-level=all onnx/bert-base-cased/`

See the contributed docs here https://235542-155220641-gh.circle-artifacts.com/0/docs/_build/html/serialization.html

Idea: Rename the `convert_pytorch` to `export` so we have the exact same hierarchy than PyTorch: 
- PyTorch: `torch.onnx.export`
- Transformers: `transformers.onnx.export`

wdyt? 

That's a great idea!

@Narsil we moved forward on your suggestion, can you have a look _(one more time 😄)_ 🙏🏻 

Hello, when we can use the transformers.onnx?

You already can when installing from source:
```
pip install git+https://github.com/huggingface/transformers
```

We'll do a release this week (probably Thursday or Friday) and it will be in a pypi release then.

hi, this thread is super important. 
Is there support for bart text2text_generation export to onnx (more specifically for summarization tasks) ?
LayoutLMV3 
Author: logan-markewich
Pull request: no
Labels: name: New model

### Model description

LayoutLMv3 is a pre-trained multimodal Transformer for Document AI with unified text and image masking. The simple unified architecture and training objectives make LayoutLMv3 a general-purpose pre-trained model. For example, LayoutLMv3 can be fine-tuned for both text-centric tasks, including form understanding, receipt understanding, and document visual question answering, and image-centric tasks such as document image classification and document layout analysis.

LayoutLMv3 greatly simplifies training and reduces the number of parameters compared to v3, making it an important milestone in document understanding.

[LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://arxiv.org/abs/2204.08387) Yupan Huang, Tengchao Lv, Lei Cui, Yutong Lu, Furu Wei, Preprint 2022.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

[Huggingface Pretrained Download](https://huggingface.co/microsoft/layoutlmv3-base) 

Comments:
Duplicate of #16914 

@[NielsRogge](https://github.com/NielsRogge)
I have one question for layoutlmv3, does layoutlmv3 can support RE and SER task for xfund dataset now?
Update data2vec.mdx to include a Colab Notebook link (that shows fine-tuning)
Author: sayakpaul
Pull request: yes

# What does this PR do?

This PR includes a link to a Colab Notebook that shows how to fine-tune the Data2Vec vision model on the task of image classification. 

@sgugger @Rocketknight1 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Yes, sure. Doing it in a while. 

@sgugger done.
ByT5 parallelization
Author: patrickvonplaten
Pull request: no

## ByT5 parallelization

Regarding ByT5, we observed the following and it looks like a potential bug:
ByT5 models are handled by HF through the same set of classes that handle T5 models, for instance T5ForConditionalGeneration [link](https://github.com/huggingface/transformers/blob/ad0cba08ea295cd7450484468dd34b53816c85fb/src/transformers/models/t5/modeling_t5.py#L1456)
One issue this creates is the way parallelization is performed by calling the parallelize method [link](https://github.com/huggingface/transformers/blob/ad0cba08ea295cd7450484468dd34b53816c85fb/src/transformers/models/t5/modeling_t5.py#L1494)
The method allows using a single device map to distribute attention blocks across devices. The device map is either inferred from the number of layers of the encoder or can be passed as an argument.
The bug stems from the fact that while T5 model have the same number of layers for encoder and decoder, this is not the case for ByT5 models which have a 3:1 structure (see for instance [here](https://github.com/google-research/byt5/blob/master/byt5/gin/models/byt5.xxl.gin#L4-L5) or Table 1 in the [arXiv paper](https://arxiv.org/pdf/2105.13626.pdf)).
We worked around this by reimplementing parallelize to have different encoder and decoder configurations:

```
encoder_device_map = (
    get_device_map(len(model.encoder.block), range(torch.cuda.device_count()))
    if encoder_device_map is None
    else encoder_device_map
    )
decoder_device_map = (
    get_device_map(len(model.decoder.block), range(torch.cuda.device_count()))
    if decoder_device_map is None
    else decoder_device_map
    )
```

Gently pinging @stas00 here. Do we still use this method to parallelize models such as T5? How do we currently parallelize models like T5?

Comments:
Until we implement a proper PP (pipeline parallelism) (if we ever do) we only have this naive approach for PP and yes, it assumes a balanced number of layers between encoder and decoder. 

There is absolutely no reason to use the naive PP (pipeline parallelism) as Deepspeed ZeRO solves this issue far more efficiently. but we left these in T5 and GPT2 models for now in case someone still uses those.

The next best alternative should someone not want to use Deepspeed ZeRO is TP (tensor parallelism), which soon will come via OSLO and Deepspeed-Inference projects.

For now you can probably just assert if someone tries to use the naive PP with other than T5, since it was designed for T5 in mind. Or you can fix it to handle unbalanced number of layers but it might be a waste of time if nobody uses it.

Thanks a lot @stas00 !

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fill-Mask Pipeline BERT on a sentence ending with/without a dot
Author: fangkuoyu
Pull request: no

## Environment info
- `transformers` version: 4.8.1
- Platform: Windows 10
- Python version: 3.7.10
- PyTorch version: 1.9.0

## Problem
Why do the following sentences give different predicted words?
Sentence 1: [MASK] is the capital of France
Sentence 2: [MASK] is the capital of France.
where the second sentence is with a dot as the ending.

## Program
from transformers import pipeline

fillmask_bert = pipeline('fill-mask', model="bert-base-uncased")
pred_1 = fillmask_bert("[MASK] is the capital of France")
for item in pred_1:
    print(item)
print()

pred_2 = fillmask_bert("[MASK] is the capital of France.")
for item in pred_2:
    print(item)
print()

## Fill-Mask Results
"[MASK] is the capital of France"
{'sequence': 'it is the capital of france', 'score': 0.33803579211235046, 'token': 2009, 'token_str': 'it'}
{'sequence': 'paris is the capital of france', 'score': 0.30488064885139465, 'token': 3000, 'token_str': 'paris'}
{'sequence': 'toulouse is the capital of france', 'score': 0.04690197855234146, 'token': 17209, 'token_str': 'toulouse'}
{'sequence': 'lyon is the capital of france', 'score': 0.02363533526659012, 'token': 10241, 'token_str': 'lyon'}
{'sequence': 'marseille is the capital of france', 'score': 0.022472301498055458, 'token': 16766, 'token_str': 'marseille'}

"[MASK] is the capital of France."
{'sequence': 'paris is the capital of france.', 'score': 0.6210566163063049, 'token': 3000, 'token_str': 'paris'}
{'sequence': 'it is the capital of france.', 'score': 0.07383635640144348, 'token': 2009, 'token_str': 'it'}
{'sequence': 'toulouse is the capital of france.', 'score': 0.036227378994226456, 'token': 17209, 'token_str': 'toulouse'}
{'sequence': 'marseille is the capital of france.', 'score': 0.031809303909540176, 'token': 16766, 'token_str': 'marseille'}
{'sequence': 'lyon is the capital of france.', 'score': 0.03100072219967842, 'token': 10241, 'token_str': 'lyon'}




Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Fine-tuning longformer for Question Answering
Author: SumeetSandhu
Pull request: no

I wish to fine-tune longformer for Question Answering. I've tried both pretrained triviaQA and squad huggingface longformer models:
- https://huggingface.co/allenai/longformer-large-4096-finetuned-triviaqa 
- https://huggingface.co/valhalla/longformer-base-4096-finetuned-squadv1

with kaggle data on tensorflow in this public notebook https://www.kaggle.com/code/sumeetsandhu/longformer-qa-train.

I get this error when training:
`ValueError: Shape mismatch: The shape of labels (received (1,)) should equal the shape of logits except for the last dimension (received (1024, 1)).`

According to documentation for both models:
- https://huggingface.co/docs/transformers/model_doc/longformer#transformers.TFLongformerForQuestionAnswering
- https://huggingface.co/docs/transformers/main/en/model_doc/longformer#transformers.LongformerForQuestionAnswering

the start_positions and end_positions input labels should be index numbers (not one-hot vectors).

What am I missing?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
doctest for BART and IBERT
Author: abdouaziz
Pull request: yes

# What does this PR do?

Adds BART and IBERT to the doc tests.

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16459). All of your documentation changes will be reflected on that endpoint.

Hi @ydshieh  , I would need your help to move forward in my case 

Hi, @abdouaziz 

Sorry for the late response. The Bart model is in fact already done by our internal developer.
Could you revert your change on the Bart model, please? 

Regarding iBERT, I will review later. Don't hesitate to drop a comment if you get any problem.

Apologies for this situation 🙏

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Add test to ensure models can take int64 inputs
Author: Rocketknight1
Pull request: yes

Like it says on the tin, this is a test to ensure that our models can take `tf.int64` inputs. I expect this will cause some models to break, in which case this PR will also include patches to ensure that they can take `tf.int64`.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I added small patches to all the models that were failing this test. In all cases, the patch shouldn't break `int32` inputs, since I just cast the dtype of the other operand (which was previously invisibly hardcoded as `tf.int32`) to the dtype of the input tensor.
Different logits for single/batch inputs on T5ForConditionalGeneration
Author: rafikg
Pull request: no
Labels: name: bug

### System Info

```shell
transformers                 4.18.0 
python 3.8.10
ubuntu
pytorch
T5ForConditionalGeneration
```


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
tokenizer = AutoTokenizer.from_pretrained("t5-small")
model = AutoModelForSeq2SeqLM.from_pretrained("t5-small")
model.resize_token_embeddings(len(tokenizer))
model.to("cuda")
model.eval()



# sequences
seq1 = "summarize: Calling the model (which means the forward method) uses the labels for teacher forcing. This means inputs to the decoder are the labels shifted by one"
output1 = "calling the model uses the labels for teacher forcing. inputs to the decoder"

seq2 = "summarize: When you call the generate method, the model is used in the autoregressive fashion"
output2 = "the model is used in the autoaggressive fashion."

seq3 = "summarize: However, selecting the token is a hard decision, and the gradient cannot be propagated through this decision"
output3 = "the token is a hard decision, and the gradient cannot be propagated through this decision"

input_sequences = [seq1, seq2, seq3]
output_seq = [output1, output2, output3]

# encoding input and attention mask
encoding = tokenizer.batch_encode_plus(
    input_sequences,
    padding="longest",
    truncation=True,
    return_tensors="pt",
)

input_ids, attention_mask = encoding.input_ids.to("cuda"), encoding.attention_mask.to("cuda")

# labels
target_encoding = tokenizer.batch_encode_plus(
    output_seq, padding="longest",  truncation=True, return_tensors="pt"
)
labels = target_encoding.input_ids.to("cuda")
labels[labels == tokenizer.pad_token_id] = -100

# Call the models
logits = model(input_ids=input_ids, labels=labels).logits

# Apply softamx() and batch_decode()

X = logits
X = F.softmax(X, dim=-1)
ids = X.argmax(dim=-1)
y = tokenizer.batch_decode(sequences=ids, skip_special_tokens=True)

print(y)
# results: batch_size=3

# [
#   'call the model uses the labels for teacher forcing  inputs to the decoder are',
#   'model is used in the constructegressgressive fashion ',
#   'token can a token decision, and the gradient cannot be propagated through this decision '

# ]

# results: batch_size =1 i.e. consider 1 seq each time

# ['call the model uses the labels for teacher forcing  inputs to the decoder are']

# ['the model is used in the auto-gressgressive fashion ']

# ['the token is a hard decision, and the gradient cannot be propagated through this decision ']
```

### Expected behavior

```shell
Having the same output sequences.
```


Comments:
@patrickvonplaten can you please have a look ?
Black preview
Author: sgugger
Pull request: yes

# What does this PR do?

This PR switches `make style`, `make fixup` and `make quality` to use `black --preview`, which reformats the docstrings (also done by `hf-doc-styler` so nothing new here) as well as all error/logger/warning strings to respect the char limit.

This will avoid the annoying comments from the nefarious sgugger on PRs.

Note: for the preview feature there are differences between minor versions, so needed to update the setup a bit.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix typo in bug report template
Author: fxmarty
Pull request: yes

# What does this PR do?

Fix a typo in issue templates.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add support for Opt in tf and flax
Author: ArthurZucker
Pull request: yes

# What does this PR do?

Adds support for OPT in Flax and TF. 
Also clean Pytorch code a bit. 

## Who can review?
 @LysandreJik, @patrickvonplaten, @patil-suraj, @sgugger


Comments:
Closing for a new pull request where history is fixed
Bert that receives text triplet as an input
Author: MiriamFarber
Pull request: no
Labels: name: wontfix

I would like to train bert on triplets of texts as inputs (for example, something like (context, question, answer)). encode_plus (https://huggingface.co/transformers/internal/tokenization_utils.html#transformers.tokenization_utils_base.PreTrainedTokenizerBase.encode_plus) receives either a single text, or a text_pair. Is there a way to use it with triplets?

Comments:
Unfortunately we do not have such a method, as it would imply having an opinionated approach on how to do it. The `encode_plus` method follows what was done during each model's training, since our aim is to replicate as closely as possible the original approach.

I would recommend encoding your sequences by placing your own special tokens, and by specifying `add_special_tokens=False` so that the encoding method does not add these tokens automatically. Let me know if you want a code sample showing this.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


@LysandreJik If you can share a code sample that would be great!


This issue has been automatically marked as stale and been closed because it has not had recent activity. Thank you for your contributions.

If you think this still needs to be addressed please comment on this thread.

I wonder why `text_pair` gets a special treatment. I guess datasets which have two strings as "input" (like glue's mnli with premise and hypothesis) are more common, but aren't there datasets with three or more input strings?
Added translation of installation.mdx to Portuguese Issue #16824
Author: rzimmerdev
Pull request: yes

# What does this PR do?

Creates folder pt in docs/source for translating documentation to Portuguese
Currently, only the installation.mdx file was translated as of this PR.

Fixes issue #16824

## Before submitting
- [X] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [X] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [X] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [X] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [X] Did you write any new necessary tests?



Comments:
_The documentation is not available anymore as the PR was closed or merged._

I'm still working on translating the remaining files, although there's already some work done on three files.

Maybe ~@gante~ @omarespejel ? :smile:  

Obrigado @rzimmerdev! @sgugger, LGTM. Ready to merge and start the Portuguese docs 🤗

I removed the preprocessing doc from this PR because it was not ready yet.