noamaanMulla-03/transformers-issues huggingface dataset:
Add missing lang tokens in M2M100Tokenizer.get_vocab
Author: guillaumekln
Pull request: yes

# What does this PR do?

The lang tokens were missing from `M2M100Tokenizer.get_vocab`. The `get_vocab` method is updated to match other multilingual tokenizers such as `NllbTokenizer` and `MBart50Tokenizer`.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [x] Did you write any new necessary tests?


## Who can review?

@n1t0, @LysandreJik, @SaulLu 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

A friendly re-ping to @patil-suraj :hugs: 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Maybe of interest to @ArthurZucker :)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Re-ping of @ArthurZucker
Generate: contrastive search cosmetic tweaks
Author: gante
Pull request: yes

# What does this PR do?

Makes some cosmetic tweaks to contrastive search in advance of the TF PR:
- fixes some documentation strings out of place;
- correct type hints;
- limits comments to 120 chars;
- removes redundant variables.

Changes validated against the slow tests for contrastive search.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Fix somehow incorrect model - tokenizer mapping in tokenization testing
Author: ydshieh
Pull request: yes

# What does this PR do?

In this method
https://github.com/huggingface/transformers/blob/371337a95b5d82cc9376c2595ed2022a5eb2ee6e/tests/test_tokenization_common.py#L107

we update the mapping whenever we find a tokenizer/model found for a configuration
```
tokenizer[_fast]: (configuration, model)
```

However, **multiple models could share the same tokenizer class**, and, for example, we get `LiLT` (recently added) model for the tokenizer class `LayoutLMv3Tokenizer`. Some tests like the following fails
```bash
tests/models/layoutlmv3/test_tokenization_layoutlmv3.py::LayoutLMv3TokenizationTest::test_torch_encode_plus_sent_to_model
(line 1130)  TypeError: forward() got an unexpected keyword argument 'pixel_values'
```
as the model used for `LayoutLMv3TokenizationTest` is `LiLT` model instead of `LayoutLMv3Model`.

1. This is somehow undesirable, and we would prefer to test the original/canonical mode/tokenizer pair.
2. This PR adds some condition to ensure the desired property in 1.
3. We can probably extend the test to test each possible pair of `(model_1, tokenizer)`, `(model_2, tokenizer)`, ...etc. But I would leave this in another PR.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
`--with_tracking`  doesn't seem to work
Author: Ericmututu
Pull request: no
Labels: name: bug

### System Info

Hi,
When I enable --with_tracking within run_glue_no_trainer.py, nothing seems to happen. After the training is over, I don't find any log files in output_dir. How do I save these training results (e.g., accuracy, training_loss, ...) as shown in the following figure?
![image](https://user-images.githubusercontent.com/49343975/190840432-b9a2559c-17b3-4653-838d-97c1d3cb56d0.png)
Thanks in advance!

### Who can help?

@sgugger @muellerzr 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

`Execute the script`

python run_glue_no_trainer.py \
    --model_name_or_path bert-large-uncased \
    --task_name mrpc \
    --output_dir ./output_dir \
    --per_device_train_batch_size 32 \
    --per_device_eval_batch_size 32 \
    --learning_rate 2e-5 \
    --num_train_epochs 100 \
    --seed 42 \
    --with_tracking


### Expected behavior

I want to save the results of the training process in a log file by enabling `--with_tracking`.

Comments:
@muellerzr will correct me if I'm wrong, but this API is for external trackers (TensorBoard, WandB etc.). To save results on disk, just use regular python code like `json.dump`.

@sgugger exactly. @Ericmututu do you have any tracking libraries installed on your system?

(I can probably raise an error in the scripts if it's tried and none are available)

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
ConvNextModel doesn't work well on M1 mac for batches containing more than 2 images
Author: hiroalchem
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.21.3
- Platform: macOS-12.3.1-arm64-arm-64bit
- Python version: 3.9.13
- Huggingface_hub version: 0.9.1
- PyTorch version (GPU?): 1.12.1 (False)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Using mps
- Using distributed or parallel set-up in script?: no

### Who can help?

@LysandreJik @NielsRogge @sgugger

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```
from transformers import ConvNextFeatureExtractor, ConvNextModel

device = "mps"
#device = "cpu"
feature_extractor = ConvNextFeatureExtractor.from_pretrained("facebook/convnext-large-224-22k-1k")
model = ConvNextModel.from_pretrained("facebook/convnext-large-224-22k-1k").to(device)

import numpy as np
arr1 = np.zeros((224, 224, 3))
inputs = feature_extractor(arr1, return_tensors="pt").to(device)
print(inputs['pixel_values'].shape)
output = model(**inputs).pooler_output.detach().cpu().numpy().copy()
print(output)
arr2 = [np.zeros((224, 224, 3), np.uint8) for x in range(2)]
inputs = feature_extractor(arr2, return_tensors="pt").to(device)
print(inputs['pixel_values'].shape)
output = model(**inputs).pooler_output.detach().cpu().numpy().copy()
print(output)
```

arr1
```
torch.Size([1, 3, 224, 224])
[[-0.07712477 -0.21273601  0.08057457 ...  0.40773717  0.17893904
   0.25740874]]
```
arr2
```
torch.Size([2, 3, 224, 224])
RuntimeError: view size is not compatible with input tensor's size and stride (at least one dimension spans across two contiguous subspaces). Use .reshape(...) instead.
```


### Expected behavior

Should be able to make inferences even for arr2.

Comments:
Hello, I have an M1 Mac and would love to try to tackle this issue

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
`$from transformers import ALBERT_PRETRAINED_CONFIG_ARCHIVE_MAP`
Author: sanhok99
Pull request: no

        `$from transformers import ALBERT_PRETRAINED_CONFIG_ARCHIVE_MAP`
this solved my issue. Thank you everyone for support. you are great.
https://hjlabs.in

_Originally posted by @hemangjoshi37a in https://github.com/huggingface/transformers/issues/5848#issuecomment-1242951220_
      

Comments:
Can you please mention the directory and the file name where you are changing this MAP to LIST.
Thank You.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.
Update doc for revision and token
Author: sgugger
Pull request: yes

# What does this PR do?

This PR updates the docstrings of all `from_pretrained` methods to:
- adapt the documentation for `use_auth_token`
- remove the Tip about `use_auth_token=True` for private models
- add a tip about checking out PRs using the revision argument

(For now just did the changes on the config but will copy paste everywhere once I have had opinions :-) )

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Removing BertConfig inheritance from configuration_roberta.py
Author: soma2000-lang
Pull request: yes

Related to https://github.com/huggingface/transformers/issues/19303
Pinging @sgugger for review

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19856). All of your documentation changes will be reflected on that endpoint.
[DOCTEST] `configuration_layoutlm.py` , `configuration_layoutlmv2.py` ,` configuration_layoutlmv3.py`
Author: Revanth2002
Pull request: yes

Based on #19487 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19806). All of your documentation changes will be reflected on that endpoint.

cc @ydshieh
Spanish translation of multiple_choice.mdx, question_answering.mdx.
Author: alceballosa
Pull request: yes

# What does this PR do?

Translates `multiple_choice.mdx` and `question_answering.mdx` into Spanish. Also updates the `_toctree.yml` file accordingly. 

Fixes #15947

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?

@osanseviero @sgugger 


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks @osanseviero, I just updated all files to reflect your suggestions :) 

Thanks a lot! :fire: 

@sgugger thanks for the merge! However, shouldn't the original issue (#15947) addressing the translations remain open? Think it was closed automatically bc I referenced it here.

Yes, that's because you used the word "Fixes". You should have said something like "Related to ..." :-)

Oops, sorry! Lesson learned :')
[DOCTEST] Add `configuration_mbart.py` , `configuration_mctc.py` , `configuration_layoutlm.py` , `configuration_layoutlmv2.py` ,` configuration_layoutlmv3.py`
Author: Revanth2002
Pull request: yes

Based on #19487 . 
Resolved #19806  and #19805 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Finetuning transformers for long document summarisation
Author: ra-MANUJ-an
Pull request: no

I'm wondering if there are any sample codes or blogs which can help me understand finetuning of transformer models for long document summarisation?

Comments:
@patil-suraj 
@patrickvonplaten 

Please use the [forums](https://discuss.huggingface.co/) to ask such questions as we keep the issues for bugs and feature requests only.

Sure, and apologies. I'll ask on the forum now. Thanks for the information.
Generate: validate arguments
Author: gante
Pull request: yes

# What does this PR do?

NOTE: this PR is very experimental, feel free to trash it in the review process :)

A common cause for issues in `generate` is around it not behaving as expected, as arguments can be silently ignored as part of the selected generation submethod (greedy_search, sample, ...). Typos also often fly under the radar, as the method accepts `**model_kwargs`, which in turn are passed to models that also accept `**kwargs`.

This PR adds argument validation to `generate` in two separate steps:
1. `model_kwargs` are verified as soon as the method is called. Only arguments that the model actually uses in `prepare_inputs_for_generation` or in its forward pass are accepted. This means that typos are caught immediately. The exception enumerates all arguments that triggered this failed check, so the user can correct them.
2. Before calling the appropriate generate submethod, which is picked from the arguments, checks that all passed arguments will actually be used. If the user passes an argument that is not used in that particular submethod, throws an exception indicating the submethod that was triggered and the unaccepted arguments, so the user can fix either problem (correct the submethod or correct the arguments).

Although I think the checks are super useful, the code around it is not the prettiest. The first check has some logic for edge cases, and the second case requires passing the list of methods that will be called before the submethod in question. The PR is heavily commented in GH, feel free to cast your judgment!

P.S.: (seemingly) unrelated accelerate tests are failing in `run_examples_torch`

### Related issues
- https://github.com/huggingface/transformers/issues/18130
- https://github.com/huggingface/transformers/pull/17196
- (many other issues where users were confused because they were trying to use certain arguments that had no effect on the picked submethod)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

> For 2, the way you chose feels very very magical with lots of ad-hoc code that is going to be hard to maintain.

Yeah, I agree, that was the number 1 reason why I left so many comments and caveats. It works but would be annoying to maintain.

(@sgugger) If I got it right, the suggestion was to pop used arguments from `generation_inputs` as we call functions, correct? Something like `consume_arguments(generation_inputs, <function that was just called>)` after most calls, with a small validation function at the end of generate?

Meanwhile, I'm going to do as suggested, and move the model kwargs validation to its own PR :)

> Something like consume_arguments(generation_inputs, <function that was just called>) after most calls, with a small validation function at the end of generate

No, something more like `result, generation_inputs = <function to call>(generation_inputs)`

Closing in place of two PRs:
- https://github.com/huggingface/transformers/pull/18261 for the model_kwargs validation
- TBD for the validation of other arguments, as per comments above
Generate: `TFNoRepeatNGramLogitsProcessor` is now XLA-compatible
Author: gante
Pull request: yes

# What does this PR do?

This PR attempts to rewrite `TFNoRepeatNGramLogitsProcessor` so that it is also compatible with XLA. 

It does pass the tests, but has a huuuuuuge memory footprint and it is very slow (even with XLA). It also doesn't seem to work in practice. Not putting the PR through the review process as the cost is not worth the benefits.

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Pin maximum TF version
Author: gante
Pull request: yes

# What does this PR do?

We now also depend on `tensorflow-text`, whose minor versions are typically released a few days after new `tensorflow` releases. From tests against the `tensorflow` release candidate, `tensorflow-text`-based functions fail when these two libraries do not have the same version.

This PR pins the maximum TF version so that our CI doesn't break with the upcoming `tensorflow` release. When the corresponding `tensorflow-text` library gets released we should be able to unpin it again.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@ydshieh the CI fail seems unrelated 🤔 do you know potential causes?

Unfortunately no. Let me find some time to take a look, but I think you are good to merge :-)

Yes I think that's an unrelated error linked to a new cache being made as the `setup.py` is updated. It's unrelated to the PR, but we need to have a look at what's going on.

Let's merge this PR!

Thanks for your contribution :)
Generation: fix TopPLogitsWarper edge case
Author: gante
Pull request: yes

# What does this PR do?

As raised by @ekagra-ranjan in #18976, the PT implementation for `TopPLogitsWarper` is failing in the case where the sum of the top tokens is exactly `top_p` (the current implementation adds an additional token). TF and FLAX's implementation does not suffer from this, so PT's implementation is changed to match it.

It also adds a test for the edge case, to ensure we don't regress.

Fixes #18976 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Superseded by #18984
Generate: contrastive search uses existing abstractions and conventions
Author: gante
Pull request: yes

# What does this PR do?

This PR updates contrastive search to follow existing abstractions conventions in other generation functions. It consists of several tiny changes, with the reasoning for each change in the PR comments below.

This is part of the effort to make converting to TF easier. All slow tests pass (`RUN_SLOW=1 py.test tests/generation/test_generation_utils.py -k contrastive -vv`)

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_19895). All of your documentation changes will be reflected on that endpoint.
Correct README image text
Author: KayleeDavisGitHub
Pull request: yes

# What does this PR do?

Fixes README typo involving the location of a cat and remote predictions. It reverses the "left" and "right" references so it is correct when looking at the images. 

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


## Who can review?

Documentation: @sgugger



Comments:
_The documentation is not available anymore as the PR was closed or merged._
No conv bn folding in ipex to avoid warning
Author: sanderland
Pull request: yes

# What does this PR do?

Removes attempting convolution-batchnorm folding from ipex optimization, as it always fails and throws a warning.
See https://github.com/intel/intel-extension-for-pytorch/issues/250

Most models won't even benefit from attempting, but note that even a model like ResNet fails due to checks like:
```
if num_channels != self.num_channels:
```

## Who can review?

- trainer: @sgugger


Comments:
_The documentation is not available anymore as the PR was closed or merged._

> 

Thanks for pointing this out.
Actually,  IPEX is doing optimizations as much as it could, and convolution-batchnorm folding does have a chance to bring benefit to vision-related models, e.g, beit/resnet/data2vec_vision, etc. 
Hence if we set it as false, there could be losing some potential optimization chances. 
Back for the warning message, it could be improved on the IPEX side. 


> Thanks for pointing this out. Actually, IPEX is doing optimizations as much as it could, and convolution-batchnorm folding does have a chance to bring benefit to vision-related models, e.g, beit/resnet/data2vec_vision, etc. Hence if we set it as false, there could be losing some potential optimization chances. 

The philosophy in the huggingface models seems to be to do a lot of input checks, which is incompatible with the tracing used in ipex. I could not find a single model which doesn't fail.
Discussion on whether or not this tracing could be improved in ipex does not seem to have much traction, see the linked issue. *edit* it actually goes all the way up to pytorch internals, will see if there is traction there.

> Back for the warning message, it could be improved on the IPEX side.

This is certainly true as well. 

> So let's leave it as `False` for now and revisit once IPEX has better support?

Yes, will record this enhancement as TODO for IPEX.
Documentation related to token_type_ids for RoBERTa needs to be updated
Author: fssh72
Pull request: no

### System Info

This is about online documentation at https://huggingface.co/docs/transformers/v4.23.1/en/model_doc/roberta#transformers.RobertaModel.forward.token_type_ids


### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

RoBERTa has no token type embeddings by default, while the documentation, indicates:

```
token_type_ids (torch.LongTensor of shape (batch_size, sequence_length), optional) — Segment token indices to indicate first and second portions of the inputs. Indices are selected in [0, 1]:

* 0 corresponds to a sentence A token,
* 1 corresponds to a sentence B token.
```

### Expected behavior

The documentation should reflect that the default RoBERTa model has no token type embeddings.




Comments:
The argument is accepted and treated by the model, though.

Hi @sgugger, although the argument is accepted, when providing a `token_type_ids` with 0 and 1 values, as described in the documentation, an index out of range exception occurs. This is because by default RoBERTa does not have two token_type_id embeddings. I checked, it only has one which contains all zeros (since by default it doesn't use token_type_ids).

In my opinion, the documentation should state that this parameter can only be used when token types have been added.
`RobertaEmbeddings` uses `config.type_vocab_size` for this.




Would you like to open a PR with your suggested changes?

Yes, sure. Not able to give a timeline though.