noamaanMulla-03/transformers-issues huggingface dataset:
RuntimeError opening google/pegasus-xsum
Author: bogolese
Pull request: no

I get the error:

```
Traceback (most recent call last):
File “/usr/local/lib/python3.6/site-packages/IPython/core/interactiveshell.py”, line 3343, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File “”, line 3, in
model=“google/pegasus-large”)
File “/usr/local/lib/python3.6/site-packages/transformers/pipelines.py”, line 2742, in pipeline
model = model_class.from_pretrained(model, config=config, **model_kwargs)
File “/usr/local/lib/python3.6/site-packages/transformers/modeling_auto.py”, line 1079, in from_pretrained
pretrained_model_name_or_path, *model_args, config=config, **kwargs
File “/usr/local/lib/python3.6/site-packages/transformers/modeling_utils.py”, line 923, in from_pretrained
model = cls(config, *model_args, **model_kwargs)
File “/usr/local/lib/python3.6/site-packages/transformers/modeling_bart.py”, line 978, in init
base_model = BartModel(config)
File “/usr/local/lib/python3.6/site-packages/transformers/modeling_bart.py”, line 857, in init
self.encoder = BartEncoder(config, self.shared)
File “/usr/local/lib/python3.6/site-packages/transformers/modeling_bart.py”, line 298, in init
config.max_position_embeddings, embed_dim, self.padding_idx
File “/usr/local/lib/python3.6/site-packages/transformers/modeling_bart.py”, line 1344, in init
self.weight = self._init_weight(self.weight)
File “/usr/local/lib/python3.6/site-packages/transformers/modeling_bart.py”, line 1355, in _init_weight
out[:, 0 : dim // 2] = torch.FloatTensor(np.sin(position_enc[:, 0::2])) # This line breaks for odd n_pos
RuntimeError: a view of a leaf Variable that requires grad is being used in an in-place operation.
```

when I run:

```
from transformers import pipeline
summarizer = pipeline(“summarization”,
model=“google/pegasus-large”)
```

I get the same error trying to load google/pegasus-large.

I also tried loading the models locally. The config and tokenizer load fine. The error occurs when trying to load the model:

```
model = BartForConditionalGeneration.from_pretrained(model_dir,
                                                                                          config=model_config)
```

FYI, from site-packages/torch/version.py:

```
version = ‘1.8.1+cu102’
debug = False
cuda = ‘10.2’
git_version =
```

Comments:
This appears to be a version issue.  I upgraded transformers to 4.1.0 to address another model issue and this now loads.l
Fix Seq2SeqTrainingArguments docs
Author: gchhablani
Pull request: yes

# What does this PR do?
This PR fixes the Seq2SeqTrainingArguments docs. The current docs have a problem in rendering as shown in the image below:

![image](https://user-images.githubusercontent.com/29076344/159262299-a854b114-a67e-4ef6-a90a-91b6eb476cb6.png).

Specifically, the parameters added in the sub-class are now showing up in the parameter list but as a messy paragraph at the end.

I believe the issue is because of the indentation in the docs for Seq2SeqTrainingArguments. If there is any other way this should be addressed, please let me know.

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).


## Who can review?
Documentation: @sgugger

## EDIT 1:
This is after fixing the indentation:
![image](https://user-images.githubusercontent.com/29076344/159266247-c27b3e97-e357-4a99-a42c-be59e800add0.png)


Comments:
_The documentation is not available anymore as the PR was closed or merged._

The style checks are failing because `style_doc.py` check/fixing will undo the changes I made.

I may be totally off here, but:

1. Does this point to a potential issue in the `style_doc.py`? The `style_docstring` method only takes the current file into consider, and not the fact that some additional docstring is added in the beginning or end?

2. Or, should this be handled in the `add_start_docstrings` decorator? In that case, should we indent things manually always?

I believe the second option sounds better since we almost always add only parameters when we use `add_start_docstrings`?

Would love to get your opinion on this. If needed, I can quickly create a PR which does this?

Wdyt @sgugger?
CC: @patil-suraj 

I'd have to take a closer look, but I think it's actually black undoing your changes, not `style_doc`. This may need some more work on our internal tooling.
[xtreme-s] Update Minds14 results
Author: anton-l
Pull request: yes

# What does this PR do?

Updated results for the XTREME-S experiments with the new Minds14 splits

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Let's maybe put the new eval code (to do per lang eval) directly into this here as well?

@patrickvonplaten The per-language results are now reflected in the model card too: https://huggingface.co/anton-l/xtreme_s_xlsr_300m_minds14 (will update the MLS results after running one more full experiment).
The names look a bit janky due to how metrics are parsed by the trainer, but overall the PR is ready to merge :slightly_smiling_face: 

Perfect! Thanks for fixing this :-)
Update Makefile Phonies
Author: gchhablani
Pull request: yes

# What does this PR do?
This PR fixes vestiges of previously existing targets in the Makefile, and potentially adds missing entries as well.

Please let me know if any other recipes are to be added? My understanding is that we want to force the invokation of the recipe even if the file with same name exists, based on [this](https://www.gnu.org/software/make/manual/html_node/Phony-Targets.html#:~:text=A%20phony%20target%20is%20one,name%2C%20and%20to%20improve%20performance). Please correct me if I'm wrong.

## Who can review?
@sgugger 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Add GLPN
Author: NielsRogge
Pull request: yes

# What does this PR do?

This PR implements [GLPN](https://arxiv.org/abs/2201.07436), Global Local Path Networks, for state-of-the-art depth estimation (number 2 on paperswithcode for KITTI and NYUv2 benchmarks).

To do:

- [x] transfer weights to author's username
- [x] add tests for GLPNFeatureExtractor
- [x] rename `in_index` to something better
- [x] define `DepthEstimatorOutput`

Comments:
_The documentation is not available anymore as the PR was closed or merged._

Ok I've addressed all comments. Waiting for the author to respond such that I can transfer the weights to his name.
Add type hints for Pegasus model (PyTorch)
Author: Tegzes
Pull request: yes

Adding type hints for forward methods in user-facing class for Pegasus model (PyTorch) as mentioned in #16059

@Rocketknight1 

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16164). All of your documentation changes will be reflected on that endpoint.

it seems like BART model was not updated yet, hence the inconsistency

@Tegzes Unfortunately this will have to wait for the BART PR to avoid breaking repository consistency, but I'll take a look at that point!

@Rocketknight1  I saw that BART model was updated, along with multiple models depending on it, including this one. Should I close the PR?

Hi @Tegzes, there might still be portions of `modeling_pegasus.py` that need type hints - if you could check that and either submit a Pegasus PR or close this one that would be great. And sorry for the confusion caused by the copying dependencies!
is this a bug in TFTrainer.get_train_tfdataset?
Author: TheHonestBob
Pull request: no

my environment below:
transformers==4.0.0
tensorflow-gpu==2.3.0
python==3.7
when I use TFTrainer(model, train_dataset, val_dataset), wether my dataset is tf.data.Dataset.from_generator or tf.data.TFRecordDataset, their dataset.cardinality() always -2, so in TFTrainer.get_train_tfdataset() func, 
self.num_train_examples = self.train_dataset.cardinality().numpy()
if self.num_train_examples < 0:
      raise ValueError("The training dataset must have an asserted cardinality")
I coludn't use TFTrainer, I think tf.data.Dataset.from_generator or tf.data.TFRecordDataset, should be supported.

Comments:
Hi @TheHonestBob! The `TFTrainer` class was deprecated a while ago, so it is likely that it fails under a few circumstances (and we don't have plans to maintain it).

Have you tried using the native Keras API? Here are our docs for fine-tuning with Keras: https://huggingface.co/docs/transformers/master/en/training#finetune-with-keras

Let me know if the Keras API solves your issues :)

thanks for your reply, I also use tf.keras, so I solve it with your docs.

I'm closing this issue, but feel free to reopen it if the problem persists.
TF generate refactor - past without encoder outputs
Author: gante
Pull request: yes

# What does this PR do?

As discussed in the original TF generate refactor plan (https://github.com/huggingface/transformers/pull/15562), removes the `encoder_outputs` from `past`. In practice, these changes consist mostly in:
1. Delete the lines flagged by Patrick;
2. Adapt `prepare_inputs_for_generation` and `_reorder_cache` from PT to TF, for each class.

Three important notes:
1. Beam search was still in the old format, and a few changes there were needed to enable the changes above. They were mostly about how `past` or `encoder_outputs` were handled;
2. Some models have `cross_attn_head_mask` in `prepare_inputs_for_generation`, in their PT implementation, but raised errors in TF -> I've deleted it from the function output;
3. I've run `RUN_SLOW=1 pytest -vv tests/model_name/test_modeling_tf_model_name.py` for all affected models.

Comments:
Let's merge this? cc @Rocketknight1 ?
Added type hints for Pytorch Marian calls
Author: clefourrier
Pull request: yes

As the title suggests, this PR adds type hints to the PyTorch Marian calls. (#16059 )

@Rocketknight1 

Comments:
_The documentation is not available anymore as the PR was closed or merged._

@patil-suraj Hm, do I need to change the BART too for consistency?  
```
src/transformers/models/marian/modeling_marian.py: copy does not match models.bart.modeling_bart.BartAttention at line 144
src/transformers/models/marian/modeling_marian.py: copy does not match models.bart.modeling_bart.BartEncoderLayer at line 289
src/transformers/models/marian/modeling_marian.py: copy does not match models.bart.modeling_bart.BartDecoderLayer at line 356
src/transformers/models/marian/modeling_marian.py: copy does not match models.bart.modeling_bart.BartForCausalLM at line 1529
src/transformers/models/pegasus/modeling_pegasus.py: copy does not match models.marian.modeling_marian.MarianSinusoidalPositionalEmbedding at line 109
src/transformers/models/roformer/modeling_roformer.py: copy does not match models.marian.modeling_marian.MarianSinusoidalPositionalEmbedding at line 73
Run `make fix-copies` or `python utils/check_copies.py --fix_and_overwrite` to fix them.
```

Running `make fix-copies` basically reverts all my changes. 


Hi @clefourrier, sorry for the delay! We will indeed need to make changes to BART to make the tests work - but we already have an existing BART PR here: https://github.com/huggingface/transformers/pull/16270. Probably the easiest thing to do is:

1. Wait until we merge that PR
2. If any changes in Marian are still needed, rebase your current PR onto it (make sure to force push afterwards!)
3. Check tests again afterwards

Hi @Rocketknight1 , perfect! Can you ping me once the BART PR is merged then?

@clefourrier It was just merged about half an hour ago! Take a look - if any changes are needed, you'll probably need to pull changes from upstream to the master branch of your fork, and then rebase (with force push!) onto that.

@Rocketknight1 Well, it seems the last failures comes from children archs (pegasus and roformer) - do they have their own PR?
Wav2vec2 pretraining
Author: aniket7joshi
Pull request: no

How can I pretrain the base wav2vec2 model using the *transformers.Wav2Vec2ForPreTraining* class on my own data?
I saw the example given on the official website -https://huggingface.co/transformers/model_doc/wav2vec2.html#wav2vec2forpretraining
But it wasn't of much help. Is there any notebook or script which I can follow? 

Comments:
https://github.com/huggingface/transformers/blob/master/examples/research_projects/wav2vec2/run_pretrain.py

Does this script helps you ?

cc @patrickvonplaten 

Hey @aniket7joshi,

I'm currently working on it. Hope to have a good blog/notebook in 2,3 weeks!

I'll let you know :-)

Hey @patrickvonplaten , Any update regarding this blog really looking forward to it. 

Wav2vec2 Pretraining at the moment only works for PyTorch. I want to make it work for Flax to be able to showcase it nicely in a google colab with 8 TPUs. Still working on it! 

Hey @patrickvonplaten Can you help me as to how I use the script here: https://github.com/huggingface/transformers/blob/master/examples/research_projects/wav2vec2/run_pretrain.py

As input, I have a folder of audio files based on which I want to pretrain my model. I don't want to use any datasets available online or on huggingface. 
TIA :) 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

Planning to look into this again in 2 weeks

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.
rembert/splinter (torch) type annotations
Author: jacobdineen
Pull request: yes

#16059

Type annotations for Rembert & Splinter (Torch) and copies

Comments:
@Rocketknight1 this passed tests - Think I was missing an Optional typing import on Splinter.
No build on PR though

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_16207). All of your documentation changes will be reflected on that endpoint.

@Rocketknight1 Rembert and Splinter done, as well as copies from Bert.
Tests failing on flax, which wasn't modified. 

Hi @jacobdineen there's a lot of changes in unrelated files here, mostly just reformatting. Did you do that manually, or did it happen when you ran something like `make style` or `make fixup`?

@Rocketknight1 The auxiliary files that were updated were results of running make fix-copies, and then make fixup. I tried several different things, but the build always failed in a single place that I wasn't able to trackdown. Something about an Optional call not defined in the model template file, which was not something that was modified.

Hi @jacobdineen, that's weird. Can you make a new PR (make sure to fork/branch from the most recent version of the library!) and just make the type hint changes to Rembert + Splinter, without any formatting changes or things like `make fixup` or `make fix-copies`? Tests will fail, but I'll review it and run the fixup myself here, and I should be able to resolve any problems at my end.

Thanks for your patience here - hopefully we can figure this one out!

@Rocketknight1 - Will do! Closing this PR now.
Add type annotations for Rembert/Splinter and copies
Author: jacobdineen
Pull request: yes

Splinter and Rembert (Torch) type annotations.
Dependencies on Bert (Torch).
Raw files w/o running make fixup or make fix-copies

#16059
@Rocketknight1 

Comments:
By the way, GitHub is having some issues right now and tests may not be running. Feel free to continue with the PR, but we'll need to wait for them to come back before we can merge!

_The documentation is not available anymore as the PR was closed or merged._

Tests are back, going to try fixing copies etc. now, one sec!

Is this a private file? This was the main error I was receiving after run fix-copies (from the build strack trace)

```
self = <module 'transformers.models.template_bi' from '/home/runner/work/transformers/transformers/src/transformers/models/template_bi/__init__.py'>
module_name = 'modeling_template_bi'

    def _get_module(self, module_name: str):
        try:
            return importlib.import_module("." + module_name, self.__name__)
        except Exception as e:
            raise RuntimeError(
                f"Failed to import {self.__name__}.{module_name} because of the following error (look up to see its traceback):\n{e}"
>           ) from e
E           RuntimeError: Failed to import transformers.models.template_bi.modeling_template_bi because of the following error (look up to see its traceback):
E           name 'Optional' is not defined```

Ah, I see the problem now! This error is in our template files, and yes, it can be hard to spot, because it only appears when one of our tests tries to generate a new class from the template. Give me a second and I'll see if I can fix it!

I see. Definitely scratched my head at these for a while - Thanks so much for your help!
Fix code repetition in serialization guide
Author: osanseviero
Pull request: yes

# What does this PR do?


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?
@sgugger @stevhliu 

Comments:
_The documentation is not available anymore as the PR was closed or merged._
TFLongformer: Add missing type hints and unpack inputs decorator
Author: johnnygreco
Pull request: yes

# What does this PR do?

1. Adds missing type hints to the TF Longformer model classes.
    
    - Issue: #16059
    - Note: I also added `Numpy array` as a possible argument type to parameters listed in `LONGFORMER_INPUTS_DOCSTRING`.
    
2. Unpacks TF Longformer model inputs using the `unpack_inputs` decorator. 
    
    - Issue:  #16051

### Tests / Checks
- `RUN_SLOW=1 py.test -vv tests/longformer/test_modeling_tf_longforme.py` ran without any failures.
- I ran `make fixup` before my final commit. 
    
_This PR is part of the Great Code Cleanup 2022!_

@gante @Rocketknight1

Comments:
Lol just realized I accidentally committed my changes using my work email address 🤦‍♂️

One thing I note in the docstring changes is that you wrote `Numpy array` rather than `np.ndarray`, which might be causing our doc builder some confusion! Can you try changing that and seeing if we get a green light on the doc building tests afterwards?

_The documentation is not available anymore as the PR was closed or merged._

...I take it back, the doc builder is fine. Can we make that change anyway, though?

Done! 

LGTM!
Add missing type hints for PyTorch Longformer models
Author: johnnygreco
Pull request: yes

# What does this PR do?

- Adds missing type hints to the PyTorch Longformer model classes.
    
    - Issue: #16059
    
## Tests / Checks
- I ran `make fixup` before my final commit. 
- `RUN_SLOW=1 py.test -vv tests/longformer/test_modeling_longformer.py` ran without any failures.

_This PR is part of the Great Code Cleanup 2022!_

@Rocketknight1

Comments:
_The documentation is not available anymore as the PR was closed or merged._
Got unexpected result when using BertTokenizer in Chinese
Author: hxxxxh
Pull request: no

## Environment info
- `transformers` version: 4.7.0
- Platform: Linux-5.4.0-74-generic-x86_64-with-debian-buster-sid
- Python version: 3.7.10
- PyTorch version (GPU?): 1.8.1 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: False
- Using distributed or parallel set-up in script?: False
- `tokenizers` version: 0.10.3


### Who can help
@LysandreJik 
@sgugger 


## Information

Model I am using (Bert, XLNet ...): bert-base-chinese

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
When I use BertTokenizer, I got unexpected results in some "strange" chars.
```python
text = "这是一个���中文句子"
tokenizer = BertTokenizer.from_pretrained("bert-base-chinese")
print(tokenizer.tokenize(text))
```
Output: ```['这', '是', '一', '个', '中', '文', '句', '子']```  
And I got the same result using BertTokenizerFast too.

## Expected behavior
I think it may cause an error or affect performance when doing sequence labeling tasks, like NER or Word Segment. 
And I noticed that I will get a more reasonable result when I downgrade the transformers to version 3.3.0 (with tokenizers==0.8.1rc2):  
```['这', '是', '一', '个', '[UNK]', '中', '文', '[UNK]', '句', '子']```

why this happens？Is there any way to get the correct result in the new version of transformers？


Comments:
Pinging @JetRunner 

This behavior seems to be related to `BertTokenizerFast`. I'm not very sure whether I know how to fix it but I can give it a look at `tokenizers`.

I wouldn't say the current tokenization is "wrong", but I would prefer `BertTokenizerFast` to be consistent with Google's tokenizer. I'll look into this and let's see what's the behavior of Google BERT.

![image](https://user-images.githubusercontent.com/22514219/122767175-f594c700-d2d4-11eb-9168-954fa29297e7.png)
Here's what you get with the original Google BERT tokenizer. Kinda confusing..

@hxxxxh Given the chaotic output of different implementations, rather than try to "fix" one to be consistent with another, I would recommend you to clean the text before using it as an input.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.