noamaanMulla-03/transformers-issues huggingface dataset:
Quick fix for `test_torch_fx` under torch 1.13
Author: ydshieh
Pull request: yes

# What does this PR do?

Quick fix for `test_torch_fx` under torch 1.13.

Since `torch 1.13`, using `meta` device gives some noisy tensors, which breaks `test_torch_fx`.
@michaelbenayoun is more qualified than me to explain this more clearly.

This PR changes the device to "cpu" if `fx.py` is used in the testing. 

running all `test_torch_fx_*` tests:
  -  with "**meta**": 1m33s
  - with "**cpu**": 1m45s

So the speed-up of using `meta` is tiny , and we can just use `cpu`.

However,

**Questions**:
  - It looks like `fx.py` is only used in the testing (at least in `transformers`)?
  - Is this module in `optimum`?
    - If so:
      -  is it used only in testing?
      -  is it used for large real models?

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_20222). All of your documentation changes will be reflected on that endpoint.

Apart from the exceptional case where we don't want to use the prod server for testing, I am very much against using env variable/constants that change for testing. Why would the test fail for the meta device but in real life the code would work? It doesn't make any sense.

Well, using `meta` with `torch 1.13` gives random values, which fail some **input checks** in some modeling files, like the one at the end.

If this is the intended behavior of `meta` (i.e. if the behavior in PT < 1.13 was wrong - but we took advantage of it), then moving away from `meta` seems the way to me.

If you still think we should stick with `meta`, I will leave @michaelbenayoun to take care of the related test though :-)

#### More details

(notice that here we don't provide any input tensor)
```ptyon
traced_model = symbolic_trace(model, input_names)
```
During this call, in `fx.py`, there are tensor preparation, which use `meta` and it gives random values which fail the check in some modeling files

(bart)
```python
        eos_mask = input_ids.eq(self.config.eos_token_id)

        if len(torch.unique_consecutive(eos_mask.sum(1))) > 1:
            raise ValueError("All examples must have the same number of <eos> tokens.")
```

close for now, unless one day we need this.
Use `config.layer_norm_eps` in some `nn.LayerNorm`.
Author: ydshieh
Pull request: yes

# What does this PR do?

⚠️⚠️⚠️ This changes the `eps` of those LayerNorm layers from (the default) `1e-5` to `1e-12`, and the outputs will have slightly differences before/after this PR. ⚠️⚠️⚠️

----

Similar to #20554, but this time instead of removing the attribute from config, we use `config.layer_norm_eps` in some `nn.LayerNorm`.




Comments:
_The documentation is not available anymore as the PR was closed or merged._

Just to confirm:

> - these changes don't have an impact on the integration tests, right? This only results in different outputs when one would train these models from scratch?

It has impact even for integration tests: the change is the **constant** `eps` being changed, which affects the forward call both in inference as well as in training.


- technically we should check the `layer_norm_eps` for each paper to match the original code

I agree - but I am not sure, for recent models, if all these attributes are set according to the papers, or people just used add new model like templates ...

This is too breaking I think. We need to be more careful on new models added that this attribute is consistently used but I don't think we should touch old models like this as it will change the results of the forward.

OK! I will keep this list of models to skip in the WIP PR where we add a test for checking unused config attributes.

Close as it is too breaking!
KeyError: 'src_texts' in train_distil_marian_enro.sh
Author: Bachstelze
Pull request: no
Labels: name: bug

### System Info

```shell
0% 0/6 [00:00<?, ?it/s][INFO|trainer_utils.py:686] 2022-06-20 14:53:01,661 >> The following columns in the training set don't have a corresponding argument in `MarianMTModel.forward` and have been ignored: id, src_texts, tgt_texts. If id, src_texts, tgt_texts are not expected by `MarianMTModel.forward`,  you can safely ignore this message.
Traceback (most recent call last):
  File "transformers/examples/legacy/seq2seq/finetune_trainer.py", line 375, in <module>
    main()
  File "transformers/examples/legacy/seq2seq/finetune_trainer.py", line 313, in main
    model_path=model_args.model_name_or_path if os.path.isdir(model_args.model_name_or_path) else None
  File "/usr/local/lib/python3.7/dist-packages/transformers/trainer.py", line 1413, in train
    ignore_keys_for_eval=ignore_keys_for_eval,
  File "/usr/local/lib/python3.7/dist-packages/transformers/trainer.py", line 1625, in _inner_training_loop
    for step, inputs in enumerate(epoch_iterator):
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 530, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 570, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 52, in fetch
    return self.collate_fn(data)
  File "/usr/local/lib/python3.7/dist-packages/transformers/trainer_utils.py", line 696, in __call__
    return self.data_collator(features)
  File "/content/transformers/examples/legacy/seq2seq/utils.py", line 298, in __call__
    batch = self._encode(batch)
  File "/content/transformers/examples/legacy/seq2seq/utils.py", line 334, in _encode
    [x["src_texts"] for x in batch],
  File "/content/transformers/examples/legacy/seq2seq/utils.py", line 334, in <listcomp>
    [x["src_texts"] for x in batch],
KeyError: 'src_texts'
  0% 0/6 [00:00<?, ?it/s]
```
```


### Who can help?

Model Marian: @patil-suraj

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [X] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

./examples/legacy/seq2seq/train_distil_marian_enro.sh

### Expected behavior

```shell
fine-tune the marian model
```


Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Same issue

pip install transformers==4.15.0
Fixes path for Graphormer checkpoint
Author: clefourrier
Pull request: yes

# What does this PR do?

@ydshieh - should fix the graphormer checkpoint path problem.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).

## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@clefourrier We now get some other issues, see the [failed job run](https://github.com/huggingface/transformers/actions/runs/4060331156/jobs/6989383705)

Could you take a look 🙏? Don't hesitate if you need some help. 

Error message provided here
```bash
FAILED tests/models/graphormer/test_modeling_graphormer.py::GraphormerModelTest::test_model_from_pretrained - RuntimeError: Error(s) in loading state_dict for GraphormerForGraphClassification:
	size mismatch for classifier.classifier.weight: copying a param with shape torch.Size([1, 768]) from checkpoint, the shape in current model is torch.Size([2, 768]).
	You may consider adding `ignore_mismatched_sizes=True` in the model `from_pretrained` method.
FAILED tests/models/graphormer/test_modeling_graphormer.py::GraphormerModelIntegrationTest::test_inference_graph_classification - RuntimeError: Error(s) in loading state_dict for GraphormerForGraphClassification:
	size mismatch for classifier.classifier.weight: copying a param with shape torch.Size([1, 768]) from checkpoint, the shape in current model is torch.Size([2, 768]).
	You may consider adding `ignore_mismatched_sizes=True` in the model `from_pretrained` method.
```



@ydshieh Opened PR #21419 to fix this!
UnboundLocalError: local variable 'image_processor_class' referenced before assignment
Author: shikhartuli
Pull request: no

### System Info

- `transformers` version: 4.26.0.dev0
- Platform: Linux-4.18.0-425.3.1.el8.x86_64-x86_64-with-glibc2.28
- Python version: 3.9.16
- Huggingface_hub version: 0.12.0
- PyTorch version (GPU?): 1.13.1 (False)
- Tensorflow version (GPU?): 2.11.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.5.3 (cpu)
- Jax version: 0.3.6
- JaxLib version: 0.3.5
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: False

### Who can help?

@sgugger 

### Information

- [X] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

When I try to add a new model as per the tutorial [here](https://huggingface.co/docs/transformers/add_new_model), I get the following error with the given set of inputs:

```
$ transformers-cli add-new-model-like
What is the model you would like to duplicate? Please provide the lowercase `model_type` (e.g. roberta): roberta
What is the name (with no special casing) for your new model in the paper (e.g. RoBERTa)? NewTransformer
What identifier would you like to use for the `model_type` of this model?  [newtransformer] 
What lowercase name would you like to use for the module (folder) of this model?  [newtransformer] 
What prefix (camel-cased) would you like to use for the model classes of this model (e.g. Roberta)?  [NewTransformer] 
What prefix (upper-cased) would you like to use for the constants relative to this model?  [NEWTRANSFORMER] 
What will be the name of the config class for this model?  [NewTransformerConfig] 
Please give a checkpoint identifier (on the model Hub) for this new model (e.g. facebook/roberta-base): 
Will your new model use the same processing class as roberta (RobertaTokenizer) (yes/no)? no
What will be the name of the tokenizer class for this model?  [NewTransformerTokenizer] 
Traceback (most recent call last):
  File "/home/stuli/.conda/envs/bin/transformers-cli", line 8, in <module>
    sys.exit(main())
  File "/scratch/gpfs/stuli/transformers/src/transformers/commands/transformers_cli.py", line 54, in main
    service = args.func(args)
  File "/scratch/gpfs/stuli/transformers/src/transformers/commands/add_new_model_like.py", line 1351, in add_new_model_like_command_factory
    return AddNewModelLikeCommand(config_file=args.config_file, path_to_repo=args.path_to_repo)
  File "/scratch/gpfs/stuli/transformers/src/transformers/commands/add_new_model_like.py", line 1382, in __init__
    ) = get_user_input()
  File "/scratch/gpfs/stuli/transformers/src/transformers/commands/add_new_model_like.py", line 1583, in get_user_input
    image_processor_class=image_processor_class,
UnboundLocalError: local variable 'image_processor_class' referenced before assignment
```

### Expected behavior

There should be no error with the given sequence of inputs when creating a new model. 

Comments:
Fix image_processor_class bug
Author: shikhartuli
Pull request: yes

# What does this PR do?

This PR fixes the image_process_class bug.

Fixes #21411

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [x] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [x] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

@sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
(maybe) redundant code in transformers/src/transformers/models/bert/
Author: Joqsan
Pull request: no

Hi,

I was looking at the BERT code and noticed that `BertOutput` and `BertSelfOutput` are almost the same, except for the `in_features` argument to `nn.Linear`?

https://github.com/huggingface/transformers/blob/92ce53aab859012f7714dae6d6fce7a7d701e75f/src/transformers/models/bert/modeling_bert.py#L376-L387

https://github.com/huggingface/transformers/blob/92ce53aab859012f7714dae6d6fce7a7d701e75f/src/transformers/models/bert/modeling_bert.py#L454-L465

I was wondering if there is a reason to have it in this way, instead of just one `Output` class accepting an `in_hidden_size` for when it needs to change.

Something like this:

```
class BertNewOutput(nn.Module):
    def __init__(self, config, in_hidden_size):
        super().__init__()
        self.dense = nn.Linear(in_hidden_size, config.hidden_size)
        ...
```

The pattern with `BertOutput` and `BertSelfOutput` is repeated in similar parts of the code such as in `src/transformers/models/bert/modeling_tf_bert.py` where 

https://github.com/huggingface/transformers/blob/92ce53aab859012f7714dae6d6fce7a7d701e75f/src/transformers/models/bert/modeling_tf_bert.py#L349

is the same code as in 

https://github.com/huggingface/transformers/blob/92ce53aab859012f7714dae6d6fce7a7d701e75f/src/transformers/models/bert/modeling_tf_bert.py#L427

Comments:
Please use the [forums](https://discuss.huggingface.co/) for such questions as we keep issues for bugs and feature requests only.
Allow to add more information in `is_flaky`
Author: ydshieh
Pull request: yes

# What does this PR do?

As mentioned once offline, I think it's better for us to make a bit more effort to describe the situation for tests decorated by `is_flaky`. We don't always know the exact reasons (and for known cases, we don't always have good way to fix - at least not in a few months sometimes). A `description` is always good IMO.

For future PRs, if new tests being decorated with `is_flaky`, let's keep 👀 on if `description` is provided - despite it's an optional parameter 🙏 .

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add distinct section names for PyTorch and TF
Author: Rocketknight1
Pull request: yes

Super-small fix here - the section names for PyTorch and TF were identical in the notebooks doc, which meant that when you clicked on one of the TF categories in the TOC you got sent to the PyTorch one instead (because it came first).

This PR adds separate section names so this doesn't happen! (Thanks @mishig25 for telling me how to do that!)

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thank you for fixing this!
Fixes bug in the creation of ExponentialDecayLengthPenalty
Author: jorgemcgomes
Pull request: yes

input_ids_seq_length doesn't exist in the GenerationConfig, it exists as local variable in the function.

Setting exponential_decay_length_penalty therefore results in an error: `AttributeError: 'GenerationConfig' object has no attribute 'input_ids_seq_length'`

This simple change fixes this issue, and the exponential_decay_length_penalty works as expected.

@gante 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

merging as the failing test (`tests/models/cvt/test_modeling_cvt.py::CvtModelTest::test_save_load_fast_init_to_base`) is a known flaky test
Update task summary 
Author: stevhliu
Pull request: yes

This is the second part of updating the task summary to be more conceptual. After a brief introduction and background to the tasks Transformers can solve in [part 1](https://github.com/huggingface/transformers/pull/21014), this PR is a bit more advanced and digs deeper into explaining how Transformer solves these tasks.

### To-do:
- [x] Add computer vision section
- [x] Add NLP section

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Ok, I'm finally finished with the first draft (took a bit longer to learn some models I wasn't familiar with)! I'd appreciate a general review of the scope of this page to make sure we're aligned (ie, are some sections too in-depth, are some not explained well enough?). Thanks in advance @sgugger @MKhalusova ! 🥹

Afterward, I'll ping one of our audio and computer vision experts for a more in-depth review of those sections 🙂 

Thanks for the feedback, I added some images to go along with the text!

@NielsRogge, would you mind reviewing the computer vision section? This guide is a high-level overview, and the goal is to help users understand how a certain task is solved by a model. Please feel free to let me know if it's too detailed, not detailed enough, or if I got something wrong! Also, if you know of a good beginner's resource for computer vision we can link to, that'd be great as well to set expectations for the reader. Thanks! 👍

@sanchit-gandhi, if you could do the same with the audio section, that'd be awesome. Thank you! 👍
Add VQGAN-CLIP research project
Author: ErwannMillon
Pull request: yes

# What does this PR do?
Implements VQGAN-CLIP using huggingface CLIP models

Related to #21064 

This Research Project allows users to generate or edit images with a single line of code. It wraps the huggingface CLIPProcessor class, allowing images to be processed as torch tensors in order to preserve gradient flow through the transformations. 

Features:
- Positive and negative prompts
- Multiple prompts
- Prompt Weights
- Creating GIF animations of the transformations
- Wandb logging

Tagging @amyeroberts for review :)


Comments:
_The documentation is not available anymore as the PR was closed or merged._

Hi there,
Thanks for the feedback, I fixed the style issues and removed the face image. 
Also refactored the code to have more accurate function names, used just the tokenizer, changed the assertions to exceptions, and removed some extraneous code (eg double crop, freeze_module)
Have a great day :)
Erwann

@ErwannMillon - thanks for the updates. It's looking good 😎 ! 

Just two last things before I think we're ready to merge: 
* Removing the `face.jpg` file and instead pointing to a place where it can be downloaded
* Could you also remove the notebook? Like the image, we want to avoid adding large files as much as possible. Happy for you to link to e.g. a colab which shows this demo in the README.
Add the GeLU activation from pytorch with the tanh approximation
Author: jlamypoirier
Pull request: yes

Fixes #21344. See that issue for more details.

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Thanks for working on this! Does the new implementation in Pytorch produce the exact same results as `gelu_fast`? If that is the case, I would prefer we just replace the current `gelu_fast` with this when PyTorch is 1.12 or above.

> Thanks for working on this! Does the new implementation in Pytorch produce the exact same results as `gelu_fast`? If that is the case, I would prefer we just replace the current `gelu_fast` with this when PyTorch is 1.12 or above.

The results are similar but there are still rounding errors, see my analysis in the related issue #21344. I would also be in favor of replacing the existing implementation / using it as default, but I would introduce small numerical differences in some models, is that a problem?

Ah yes, the difference is quite significant sadly, so this will probably introduce a difference that is too big :-/
So let's go with a new activation. Maybe `gelu_pytorch` is a better name?

> Ah yes, the difference is quite significant sadly, so this will probably introduce a difference that is too big :-/ So let's go with a new activation. Maybe `gelu_pytorch` is a better name?

Wouldn't it cause confusion with the default pytorch implementation? That one is currently named "gelu". (And the one named "gelu_python").

Also should I add an explicit pytorch version check?


Ok for the name then. For the version check, you will need to create a function that returns the instance of GELU and issues an import error if the  PyTorch version is too low, then put that function in the mappinh.

> Ok for the name then. For the version check, you will need to create a function that returns the instance of GELU and issues an import error if the PyTorch version is too low, then put that function in the mappinh.

Made a class to match the other activations, and raising a `NotImplementedError` (I don't think an `ImportError` is the best here since the function exists in earlier versions.) Also added to `test_get_activation`.

Failure is unrelated so merging. Thanks again for your contribution!
BertTokenizer cannot properly tokenize words with dashes
Author: poteminr
Pull request: no

### System Info

The `BertTokenizer` doesn't tokenize words with dashes correctly. I tried to use tokenizer for **Italian** and **English** languages and got unexpected results. This issue similar to https://github.com/huggingface/transformers/issues/5136. 

### Who can help?

@ArthurZucker

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python3
from transformers import AutoTokenizer, BertTokenizer
model1 = 'cointegrated/rubert-tiny2'
model2 = 'Babelscape/wikineural-multilingual-ner' 
tokenizer = BertTokenizer.from_pretrained(model1, model_max_length=50)

>>> tokenizer.tokenize('so-called')
['so', '-', 'called']

>> tokenizer.tokenize('era il figlio di ghazi-ud-din haidar.')
['era', 'il', 'fi', '##gli', '##o', 'di', 'gh', '##azi', '-', 'ud', '-', 'din', 'hai', '##dar', '.']
```



### Expected behavior

Expected something like that:
```python3
>>> tokenizer.tokenize('so-called')
['so', '##-', '##called']

>> tokenizer.tokenize('era il figlio di ghazi-ud-din haidar.')
['era', 'il', 'fi', '##gli', '##o', 'di', 'gh', '##azi', '##-', '##ud', '##-', '##din', 'hai', '##dar', '.']
```

Comments:
layoutlmv3-base-chinese convert onnx
Author: githublsk
Pull request: no

### Feature request

When use layoutlmv3-base-chinese converting to onnx, the comand line is as follows: python3.7 -m transformers.onnx --model=model path --feature token-classification, we find it lacks of vocab_file and merges_file, can you support us above two files, thank you very much, if not, can you give us some method to solve above problem?

### Motivation

No.

### Your contribution

No.

Comments:
ONNX will only convert the model, not the tokenizer.

ONNX conversion is now moved to the Optimum package, no need to pass --feature anymore as it will infer that automatically based on the checkpoint. Docs here: https://huggingface.co/docs/optimum/exporters/onnx/usage_guides/export_a_model.

Closing this as the issue seems resolved.