noamaanMulla-03/transformers-issues huggingface dataset:
[New Model] Donut: Document Understanding Transformer
Author: WaterKnight1998
Pull request: no
Labels: name: New model

### Model description

Donut doughnut, Document understanding transformer, is a new method of document understanding that utilizes an OCR-free end-to-end Transformer model. Donut does not require off-the-shelf OCR engines/APIs, yet it shows state-of-the-art performances on various visual document understanding tasks, such as visual document classification or information extraction (a.k.a. document parsing). In addition, we present SynthDoG dog, Synthetic Document Generator, that helps the model pre-training to be flexible on vairous languages and domains.

### Open source status

- [X] The model implementation is available
- [X] The model weights are available

### Provide useful links for the implementation

Code @clovaai : https://github.com/clovaai/donut

Weights: 
- https://huggingface.co/naver-clova-ix/donut-base
- https://huggingface.co/naver-clova-ix/donut-base-finetuned-cord-v1-2560
- https://huggingface.co/naver-clova-ix/donut-base-finetuned-zhtrainticket
- https://huggingface.co/naver-clova-ix/donut-base-finetuned-rvlcdip
- https://huggingface.co/naver-clova-ix/donut-base-finetuned-cord-v1
- https://huggingface.co/naver-clova-ix/donut-base-finetuned-docvqa
- https://huggingface.co/naver-clova-ix/donut-base-finetuned-cord-v2

Comments:
See #18488 

Cool to see you working there, thank you very much =D
[VideoMAE] Add model to doc tests
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR fixes the fact that VideoMAE supports the doc tests, but wasn't actually run.

cc @ydshieh, could you point me where I need to add pip install decord to the setup of the machine that runs the doc tests?

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi @NielsRogge , it is this file to add

```
docker/transformers-all-latest-gpu/Dockerfile
```

(The image will only be built tonight)

If you want to build it now + even run the doctest to make  sure it works, let me now
Update perf_train_gpu_one.mdx
Author: mishig25
Pull request: yes

Fixes doc newlines (which is causing markdown parser errors)

preview rendering correctly:

<img width="500" alt="Screenshot 2022-08-08 at 18 35 08" src="https://user-images.githubusercontent.com/11827707/183468189-0be58ab5-b1fa-4a98-a4f4-ae4751960933.png">



Comments:
_The documentation is not available anymore as the PR was closed or merged._
Spanish translation of summarization.mdx (#15947)
Author: AguilaCudicio
Pull request: yes

Spanish translation of summarization.mdx (#15947)

<!-- Remove if not applicable -->

Fixes #15947 (issue)

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).


## Who can review?
@omarespejel @osanseviero @sgugger

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Muchas gracias @AguilaCudicio por la traducción! Agregué algunos comentarios en el review.

Gracias @AguilaCudicio for the translation! 🚀

@sgugger LGTM :)
Let's not cast them all
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR is an alternative solution (and cleaner) to https://github.com/huggingface/transformers/pull/18467
An issue has been found when running this script:
```
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("Salesforce/codegen-2B-mono")
model = AutoModelForCausalLM.from_pretrained("Salesforce/codegen-2B-mono", device_map="auto", torch_dtype=torch.float16)

text = "def quicksort(l):"

encoded_input = tokenizer(text, return_tensors='pt')
output_sequences = model.generate(input_ids=encoded_input['input_ids'], attention_mask=encoded_input['attention_mask'])
print(tokenizer.decode(output_sequences[0], skip_special_tokens=True))
```
Since `torch_dtype=torch.float16` will cast all parameters of the models including the buffers, this will include also causal masks for some models. 

In some niche cases those buffers are in `uint` or `bool` instead of `int`. This PR should address this issue and check if the parameter is either an `uint`, `int` or `bool` before casting it.

cc @sgugger 

Ran codegen slow tests and the tests are passing, let me know if we need more checks! 



Comments:
Thanks also for giving me the right pointer to the rootcause!

_The documentation is not available anymore as the PR was closed or merged._
TF Camembert not improving over epochs
Author: bourrel
Pull request: no
Labels: name: wontfix

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):
`jplu/tf-camembert-base`

Language I am using the model on (English, Chinese ...):
French

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1.  Get a custom multi-class dataset with imbalanced data
2. Train TFCamembertForSequenceClassification on this dataset
3. Try with and without `class_weight` or under-sample biggest classes (accuracy and loss change but still don't improve over epochs)

```python
import tensorflow as tf
from transformers import TFCamembertForSequenceClassification, CamembertTokenizer

model = TFCamembertForSequenceClassification.from_pretrained("jplu/tf-camembert-base", num_labels=len(labels))
tokenizer = CamembertTokenizer.from_pretrained("jplu/tf-camembert-base")

model.compile(
    optimizer=tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0),
    loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True),
    metrics=["accuracy"],
)

model.fit(
    custom_generator(), # generator yield encoded sample (by tokenizer) and encoded label (by OneHotEncoder)
    epochs=10,
    max_queue_size=2,
    steps_per_epoch=25,
    #class_weight=class_weights,
    validation_data=custom_generator(),
    validation_steps=4
)
```

## Expected behavior

The classifier should improve over each epochs. In this case it stay at the same accuracy and loss, it just vary with more or less 5% accuracy.
To compare, I tried to run the same code but with `TFFlaubertForSequenceClassification.from_pretrained("jplu/tf-flaubert-base-cased")` and it worked as expected.

## Environment info
     
- `transformers` version: 2.5.1
- Platform: Linux-4.9.0-12-amd64-x86_64-with-debian-9.12 (Google AI Platform)
- Python version: 3.7.6
- PyTorch version (GPU?): 1.4.0 (True)
- Tensorflow version (GPU?): 2.1.0 (True)
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

For information, I already posted this problem on [Stack Overflow](https://stackoverflow.com/questions/60761761/hugging-face-transformer-classifier-fail-on-imbalance-dataset) which lead me here.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


@bourrel 
Why do you use `jplu/tf-flaubert-base-cased`? (https://huggingface.co/jplu/tf-flaubert-base-cased)
Any particular reason not to use `flaubert/flaubert_base_cased`? (https://huggingface.co/flaubert/flaubert_base_cased)

It was 2 years ago, I don't remember sorry 😅
https://github.com/huggingface/transformers/blob/f0d496828d3da3bf1e3c8fbed394d7847e839fa6/src/transformers/models/funnel/modeling_funnel.py#L1004
Author: Tharolzakariya
Pull request: no

https://github.com/huggingface/transformers/blob/f0d496828d3da3bf1e3c8fbed394d7847e839fa6/src/transformers/models/funnel/modeling_funnel.py#L1004

Comments:
This does not look like an issue :) Closing 

Feel free to reopen with the actual question.
[Test] Fix redirected links issue
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR tries to address the issue of loading a model when the original link is redirected. This happened for BLOOM models where the repo ids has been changed but the code does not take into account redirected links. 
I am not sure how to properly test if this does not break anything so I am putting this PR as a test PR, so feel free to ignore it.
Now loading BLOOM models with old naming works

```
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "bigscience/bloom-350m"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
```

Also this can be done probably on the `huggingface_hub` level but I am not sure

Related to #18531


Comments:
_The documentation is not available anymore as the PR was closed or merged._

This should be fixed upstream (there's an open issue IIRC)

Okay I see! I think that you are referring to this issue: https://github.com/huggingface/transformers/issues/17582 posting it here for visibility!
I can't see any related PR to this issue for now, maybe it is hidden in another PR? 

EDIT: it will be fixed once `transformers` will use `huggingface_hub` behind the scenes for loading the models

It should be fixed on the Hugging Face Hub side at this stage (the issue reported incorrectly that it works for `huggingfCe_hub` tools but it does not), there is nothing left to do in Transformers.

note that in the meantime you can always opt to re-rename your repos if it's a big issue
Use new huggingface_hub tools for download models
Author: sgugger
Pull request: yes

# What does this PR do?

This PR migrates Transformers to fully rely on `huggingface_hub` for the internal download and cache of all objects in Transformers (models, configs, tokenizers, feature extractors etc). To achieve this, a new function `cached_file` is introduced to replace the old `cached_path`, which relies on `hf_hub_download`. The whole refining of exceptions is left in this function, which allows for a lot of refactor of duplicate code across files, as well as removing some ugly try/except chains when we try several files in a row.

The cache is in the new format of huggingface_hub after this PR. To avoid breaking changes, a script will automatically converts the cache of users from the old format to the new format at the first transformers import. A warning is raised for offline users, and if the move fails for any reason or is interrupted, the user can still try later on with `transformers.util.move_cache()` (did not make a CLI command of it, but it's doable if we want).

**Note:** To avoid this PR being too heavy, all uses of `cached_path` and other old hub utils are not changed, only the main ones in the `from_pretrained` methods. A follow-up PR will hunt all remaining instances and remove those utils from the lib.

In the CI we trust! As you can see from the tests, this all comes at zero breaking changes (detected by the CI). The only modifications to the tests are small adaptations needed for some mock tests simulating no connection. Anticipated small breaking changes are:
- some error messages have slightly changed.
- if a user relied on offline mode and does not update their cache while updating Transformers, it will break. They need to be online to convert their cache to the new format.

For full backward compatibility with what `cached_path` used to do though, a few hacks were necessary. Some of those can be removed in the future if changes are made to `huggingface_hub`:
1. having methods to allow for enabling/disabling progress bars
2. throwing a `FileNotFoundError` instead of a `ValueError` when in offline mode and the file is not in the cache
3. having `hf_hub_download` look for files in the cache in case of connection errors (so if a user has the filed cached and hf.co is down, they still get their last updated version).

For 1, I had to do a contextmanager that patches huggingface_hub.
For 2, I match the exact error message for the exception I want to catch, but if no change is made in hf hub, we'll at least need a comment in bold telling the maintainers there to never update the message.
For 3, a new function `try_to_load_from_cache` is created, which can definitely leave in Transformers forever if it's not deemed suitable for `huggingface_hub`.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Yay! TYSM!!!
BART - Fix attention mask device issue on copied models
Author: younesbelkada
Pull request: yes

# What does this PR do?

This PR fixes a small issue when combining `device_map=auto` and OPT. When running the script below (tested it on my VM + Google Colab) (`pip install accelerate && pip install transformers`)

```
from transformers import AutoModelForCausalLM, AutoTokenizer

MAX_NEW_TOKENS = 128
model_name = "facebook/opt-2.7b"
text = "Hello my name is"

tokenizer = AutoTokenizer.from_pretrained(model_name)
input_ids = tokenizer(text, return_tensors="pt").input_ids

model = AutoModelForCausalLM.from_pretrained(model_name, device_map='auto')
generated_ids = model.generate(input_ids, max_length=MAX_NEW_TOKENS)
print(model.hf_device_map)
print(tokenizer.decode(generated_ids[0], skip_special_tokens=True))
```

We are getting: 
```
8 frames
[/usr/local/lib/python3.7/dist-packages/transformers/models/opt/modeling_opt.py](https://localhost:8080/#) in _prepare_decoder_attention_mask(self, attention_mask, input_shape, inputs_embeds, past_key_values_length)
    533             expanded_attn_mask = _expand_mask(attention_mask, inputs_embeds.dtype, tgt_len=input_shape[-1])
    534             combined_attention_mask = (
--> 535                 expanded_attn_mask if combined_attention_mask is None else expanded_attn_mask + combined_attention_mask
    536             )
    537 

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
```

This is because `_expand_mask` creates the mask on the cpu whereas `combined_attention_mask` is always created on the same device as `inputs_embeds`. This PR fixes this issue

Thanks @ArthurZucker !
cc @sgugger 

All OPT slow tests are passing with this fix!

Comments:
_The documentation is not available anymore as the PR was closed or merged._

I guess the reason we did not see it yet for other models using the same attention mask pre-processing function is that those models does not support `device_map=auto` yet (tried it with PegasusForCausalLM only) 

BART slow tests are passing! Merging now
DisjunctiveConstraint fails in corner case
Author: boy2000-007man
Pull request: no
Labels: name: bug

### System Info

```shell
- `transformers` version: 4.20.1
- Platform: Windows-10-10.0.19044-SP0
- Python version: 3.8.12
- Huggingface_hub version: 0.4.0
- PyTorch version (GPU?): 1.11.0 (True)
- Tensorflow version (GPU?): 2.8.0 (True)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: no
- Using distributed or parallel set-up in script?: no
```


### Who can help?

@patrickvonplaten

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

The current **trie-only** implementation failed to handle the second corner case introduced in the  `Figure 1b` of [Improved Lexically Constrained Decoding for Translation and Monolingual Rewriting](https://aclanthology.org/N19-1090/),
where the prefix of one constraint is not prefix but subsequence of another.
The minimal code snippet to reproduce is following:
```python3
>>> import transformers
>>> c = transformers.ConstraintListState([transformers.DisjunctiveConstraint(
>>>     [[1, 2, 3, 4],
>>>         [2, 3, 5],
>>>            [3, 6],
>>>               [7]]
>>> )])
>>> c.reset([1, 2, 3, 5])
>>> print(c.completed)
False
>>> c.reset([1, 2, 3, 6])
>>> print(c.completed)
False
>>> c.reset([1, 2, 3, 7])
>>> print(c.completed)
False
>>> c.reset([1, 2, 3, 4])
>>> print(c.completed)
True
```

### Expected behavior

```shell
all print statements should output True instead.
The [`AC automaton`](https://en.wikipedia.org/wiki/Aho%E2%80%93Corasick_algorithm) is the desired algorithm to supersede Trie here.
I can prepare a PR to do the upgrade and fix if necessary.
```


Comments:
Interesting edge case! @cwkeam has you encountered this before?

Hey @boy2000-007man,

Sorry to reply so late here. I'm a bit hesitant to add so much new code. Could you maybe show a case with input strings and generate and how the current implementation fails? E.g. above I see with some abstract numbers how it fails, but could you maybe also show how the current `generate(...)` method fails for an edge case?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Restore _init_weights value in no_init_weights
Author: YouJiacheng
Pull request: yes

Fend for potential nested use, and match the intuitive expectation for context managers().
In addition, users might modify private no_init_weights as well.

@patrickvonplaten @stas00 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@sgugger has also been contributing extensively to that part of the code and might have something to say!

Thanks a lot for iterating on this!
Save huggingface checkpoint as artifact in mlflow callback
Author: swethmandava
Pull request: yes

# What does this PR do?

1. Store model checkpoints including tokenizers that are needed to reload the model from mlflow as artifacts
2. Allow model to be register-able. (they are not if log_artifacts is used to log the model)

Fixes # (issue)
https://github.com/huggingface/transformers/issues/15495
https://github.com/huggingface/transformers/issues/10881
https://github.com/huggingface/transformers/issues/7698

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks again!

Hi there! @swethmandava Thanks for adding this functionality. Quick question: because the artifact logging was removed, wouldn't the intermediate checkpoints not be tracked? Only the latest checkpoint would be logged as a model, right?

> Hi there! @swethmandava Thanks for adding this functionality. Quick question: because the artifact logging was removed, wouldn't the intermediate checkpoints not be tracked? Only the latest checkpoint would be logged as a model, right?

It should now save all the checkpoints. every time on_save is called
Bugfix for the bloom model. The tensor is not moved to the right gpu causing error.
Author: prajdabre
Pull request: yes

# What does this PR do?

In the implementation of the BLOOM model, on line 307, a tensor is made but not moved to a device. By default this is cpu but if someone wants to use a gpu then this will cause the code to throw an error. @patrickvonplaten, @LysandreJik

## Before submitting
- [N] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [Y] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [N] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [N] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [N] Did you write any new necessary tests?


## Who can review? 


Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- albert, bert, xlm: @LysandreJik
- blenderbot, bart, marian, pegasus, encoderdecoder,  t5: @patrickvonplaten, @patil-suraj
- longformer, reformer, transfoxl, xlnet: @patrickvonplaten
- fsmt: @stas00
- funnel: @sgugger
- gpt2: @patrickvonplaten, @LysandreJik
- rag: @patrickvonplaten, @lhoestq
- tensorflow: @LysandreJik

Library:

- benchmarks: @patrickvonplaten
- deepspeed: @stas00
- ray/raytune: @richardliaw, @amogkam
- text generation: @patrickvonplaten
- tokenizers: @n1t0, @LysandreJik
- trainer: @sgugger
- pipelines: @LysandreJik

Documentation: @sgugger

HF projects:

- datasets: [different repo](https://github.com/huggingface/datasets)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Examples:

- maintained examples (not research project or legacy): @sgugger, @patil-suraj
- research_projects/bert-loses-patience: @JetRunner
- research_projects/distillation: @VictorSanh

 -->


Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_18449). All of your documentation changes will be reflected on that endpoint.