noamaanMulla-03/transformers-issues huggingface dataset:
[Seq2Seq Templates] Fix check_repo.py templates file
Author: patrickvonplaten
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments:
How to use gpt-2-xl with run_generation.py
Author: avianion
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
I am currently running the run_generation.py and putting the model as gpt-2, however I am not satisfied with the results frm the small model.

How can I insert the big model instead?

Comments:
You're probably using the command:
```
python run_generation.py --model_type=gpt2 --model_name_or_path=gpt2
```

to use GPT2-XL for generation you would change the last argument to `gpt2-xl`:
```
python run_generation.py --model_type=gpt2 --model_name_or_path=gpt2-xl
```

ah. its because the latest version isnt available on pypi yet. and i pip installed it. 2.1.1 is the version I've got installed.

Is there any way to force pip to install the latest version that was commited 21 hours ago.

Ah yes we're yet to release a pypi version with GPT2-XL. To install from source you would have to do:

```
pip install git+https://github.com/huggingface/transformers
```

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


 pip install transformers==3.4.0
this version of transforrmers solved my problem: OSError: Model name 'ckiplab/gpt2-base-chinese' was not found in model name list (gpt2, gpt2-medium, gpt2-large, gpt2-xl, distilgpt2). We assumed 'ckiplab/gpt2-base-chinese' was a path or url to a configuration file named config.json or a directory containing such a file but couldn't find any such file at this path or url.
how can I change the AlbertModel's vocab
Author: aLowMagic
Pull request: no

# 🚀 Feature request

<!-- A clear and concise description of the feature proposal.
     Please provide a link to the paper and code in case they exist. -->
how can I change the AlbertModel's vocab
Thanks

## Motivation

I noticed that I can change the Bert's vocab by change the vocab.txt. But when I use the Albert's API, the document suggests:
tokenizer = AlbertTokenizer.from_pretrained('albert-base-v2'). So how can I change that.
Thanks

## Your contribution



Comments:
This should help: https://github.com/huggingface/transformers/issues/1413#issuecomment-538083512

Tanks a lot.
tapas utils
Author: shashankMadan-designEsthetics
Pull request: yes

# What does this PR do?

<!--
Congratulations! You've made it this far! You're not quite done yet though.

Once merged, your PR is going to appear in the release notes with the title you set, so make sure it's a great title that fully reflects the extent of your awesome contribution.

Then, please replace this with a description of the change and which issue is fixed (if applicable). Please also include relevant motivation and context. List any dependencies (if any) that are required for this change.

Once you're done, someone will review your PR shortly (see the section "Who can review?" below to tag some potential reviewers). They may suggest changes to make the code even better. If no one reviewed your PR after a week has passed, don't hesitate to post a new comment @-mentioning the same persons---sometimes notifications get lost.
-->

<!-- Remove if not applicable -->

Fixes # (issue)


## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments:
Minor documentation revisions from copyediting
Author: connorbrinton
Pull request: yes

# What does this PR do?

Minor changes to the documentation to correct typos and improve readability. I noticed these typos while reading through the docs to familiarize myself with the library for a project, and thought it would be nice to make a PR for them 😊

I've already tested building the docs from these changes, and all changes seem to have taken effect properly 👍 

## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests), Pull Request section?

## Who can review?

@sgugger would you mind reviewing this PR? I'm happy to make any changes (or remove any changes) you want 🙂

Comments:
Looks like you need to run `make style` on your branch to fix the formatting of the doc files. Let me know if you run into any trouble doing that.

Thanks @sgugger 😄 I was able to run `make style` successfully (and update `preprocessing.rst` with the changes), but it looks like the `check_code_quality` check ran out of memory this time 😅 

![Screen Shot 2020-12-23 at 9 33 15 AM](https://user-images.githubusercontent.com/1848731/103007806-0e791700-4502-11eb-9f2c-b1946dc64dbf.png)

I tried rerunning it, but it seems like I don't have permission. Could you try rerunning it?

I'm also happy to bump the `resource_class` for the check from `medium` to `medium+` if that would be helpful 🙂 

Yes it was just a spurious failure. Thanks!
Deberta Tokenizatiion
Author: yaysummeriscoming
Pull request: no

## Environment info

- `transformers` version: 4.0.0
- Platform: Linux
- Python version: 3.8
- PyTorch version (GPU?): 1.7

### Who can help
@BigBird01 @LysandreJik 

## Information

I'd like to use the new deberta model, but it seems that the tokens aren't mapped correctly?

```
from transformers import AutoTokenizer

test_string = 'hello, I am a dog'

tokenizer = AutoTokenizer.from_pretrained('roberta-base')
print('Roberta output is: ', tokenizer.tokenize(test_string))

tokenizer = AutoTokenizer.from_pretrained('microsoft/deberta-base')
print('Deberta output is: ', tokenizer.tokenize(test_string))
```

Roberta output is:  ['hello', ',', 'ĠI', 'Ġam', 'Ġa', 'Ġdog']
Deberta output is:  ['31373', '11', '314', '716', '257', '3290']

I'd expect deberta to give an output similar to roberta, rather than numbers.

Comments:
@LysandreJik any update on this?

@yaysummeriscoming  To get sub words instead of numbers, you can call `tokenizer.gpt2_tokenizer.decode(tokens)`. Please take a look at [our code](https://github.com/huggingface/transformers/blob/52c9e842854a701a7d1b608600a614278b4407d3/src/transformers/tokenization_deberta.py#L396) for reference.

That did the trick, thanks!
[hf args] shouldn't match partial arg names
Author: stas00
Pull request: no

For `--label_smoothing_factor` I can pass `--label_smoothing` and it still works - which is a bug, as it should do a full match and not a substring. This is with master.

context: finetune_trainer just switched from `--label_smoothing` to `--label_smoothing_factor` (different functionality) and we were puzzling over why  `--label_smoothing`  still worked.

this is definitely not urgent

@sgugger 





Comments:
This looks like it's actually an intended behavior of `ArgumentParser`: see [here](https://docs.python.org/3/library/argparse.html#argparse.ArgumentParser.parse_known_args) and in general of `argparse` (see [here](https://docs.python.org/3/library/argparse.html#prefix-matching)). So I don't think it categorizes as a bug, even if it should be documented.

Oh, fantastic - thank you for finding that out, @sgugger 

This feature surely bit us yesterday.
Add caching mechanism to BERT, RoBERTa
Author: patil-suraj
Pull request: yes

# What does this PR do?
- This PR adds past key/values caching mechanism to `BertLMHeadModel`, `BertGenerationDecoder`, and `RobertaForCausalLM` to speed up the generation of `EncoderDecoder` models

-  delete the `CausalLMOutputWithPastAndCrossAttentions` class and add `past_key_values`  to `CausalLMOutputWithCrossAttentions` and `BaseModelOutputWithPoolingAndCrossAttentions`  . All `ModelOutputs` that have a `cross-attention` should also have a `past_key_values`

- by default caching is enabled for `BertLMHeadModel`, `BertGenerationDecoder`, and `RobertaForCausalLM` during inference (not just generation, also for the forward pass) and now they also output `past_key_values` by default during inference, which is a small breaking change.
  specifically when `config.output_attentions=True` in a `EncoderDecoderModel` model then the 2nd index of the output will be `past_key_values` instead of `attentions`

```python3
model = EncoderDecoderModel.from_pretrained(...)
outputs = model(input_ids)
attentions = outputs[1]  # 2nd index will be past_key_values, instead of attentions
```

- this will only affect the output during generation for `EncoderDecoder` models. Caching will be disabled when the models are used as standalone encoders, so the default output, in that case, is unchanged.

Fixes #9052



Comments:
`python utils/check_copies.py --fix_and_overwrite` should be run to make the `check_code_quality` test pass

> `python utils/check_copies.py --fix_and_overwrite` should be run to make the `check_code_quality` test pass

Yeah, did that, for some reason, it's not working for `RobertaEmbeddings`, the code is the same as that of `BertEmbeddings`

> > `python utils/check_copies.py --fix_and_overwrite` should be run to make the `check_code_quality` test pass
> 
> Yeah, did that, for some reason, it's not working for `RobertaEmbeddings`, the code is the same as that of `BertEmbeddings`

If Roberta has to be different feel free to remove the copy statement

I just merged a PR that made `cache` related tests a bit more aggressive: https://github.com/huggingface/transformers/pull/9256. It would be awesome if you could run your PR on the `EncoderDecoderModel` slow tests to make sure the cache doesn't change the results.

Merging !
Disable progress bar for Trainer
Author: Nickil21
Pull request: no

I am referencing a similar code compared to [run_glue.py]
(https://github.com/huggingface/transformers/blob/master/examples/text-classification/run_glue.py).

I am running a preprocessing function that does the tokenization of text as well as `trainer.predict` on a pandas dataframe. How do I disable the progress bar from showing the progress made on each row of the dataframe? I thought it would work by disabling logging as well as `tqdm`, but it is not the case here. #3050 

Comments:
If you set `disable_tqdm=False` in your `TrainingArguments`, you shouldn't have any progress bar from the library.

Well, I think you meant `disable_tqdm=True`. By the way, the following worked:

`args = TrainingArguments(disable_tqdm=True, output_dir="tmp_trainer")`

I am still getting progress bars for the `dataset.map()` though. Is there something like `verbose=False`?

`dataset.map` comes from the Datasets library, not Transformers. So you should open an issue there for this part :-)
Using FP16 on BartModel
Author: AOZMH
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): BART

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: (give details below)

The tasks I am working on is:
* [x] an official GLUE/SQUaD task: CNN/DM
* [ ] my own task or dataset: (give details below)

## To reproduce

I've installed the master branch of transformers but I still encountered the same issue as #3117 when using FP16 BartModel. I just initialized the model without loading the pretarined weights, but I guess the model should still be able to correctly forward the input LongTensor(batch, seq_length). The code is shown below, simply initialize a model and forward an input:

```
model = BartModel(BartConfig())
model = model.cuda().half()
cur_inputs = torch.zeros(4,16,dtype=torch.long).cuda()
cur_res = model(cur_inputs)
```

The error is:

>~\Anaconda3\envs\pytorch\lib\site-packages\transformers\modeling_bart.py in forward(self, query, key, value, key_padding_mask, layer_state, need_weights, static_kv, attn_mask)
    assert v is not None
-->  attn_output = torch.bmm(attn_probs, v)
    assert attn_output.size() == (bsz * self.num_heads, tgt_len, self.head_dim)
    attn_output = attn_output.transpose(0, 1).contiguous().view(tgt_len, bsz, embed_dim)
 RuntimeError: Expected object of scalar type Float but got scalar type Half for argument #2 'mat2' in call to _th_bmm

@sshleifer The model is quite novel to me, so am I using it incorrectly or there's still a bug in BertModel class? Thanks in advance for the help!

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->


## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: master branch
- Platform: Windows
- Python version: 3.7.0
- PyTorch version (GPU?): 1.4.0
- Tensorflow version (GPU?): /
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Comments:
@sshleifer May I ask could you reproduce the error in your machine? I ran the same code on a Linux machine with master-branch of transformers, but still got the same error. I'm planning to use BartModel these days so please notify me at your earliest convenience if there're any updates. Many thanks!

Yes, will try to fix it today! Thanks for reporting!

> Yes, will try to fix it today! Thanks for reporting!

Thanks Sam,

The code works well this time! Thanks again for the contribution.
BatchEncoding.to accepted types too restrictive
Author: jethrokuan
Pull request: no

## Environment info
     
- `transformers` version: 4.1.1
- Platform: Linux-4.14.81.bm.15-amd64-x86_64-with-debian-9.11
- Python version: 3.7.9
- PyTorch version (GPU?): 1.7.0 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: yes

### Who can help

## Information

In `BatchEncoding.to`, the only accepted class types are `str` and `torch.device`. I think some libraries like pytorch-lightning call `.to` with the integer value for the GPU number, and HF complains about this when it is perfectly valid:

```
>>> x = torch.zeros(1)
>>> x.to(0)
tensor([0.], device='cuda:0')
```

## Expected behavior

Also allow int values in BatchEncoding.to

Comments:
Fix typo in file_utils.py
Author: jungwhank
Pull request: yes

# What does this PR do?

Fix typo of `add_code_sample_docstrings` in `file_utils.py` 


## Before submitting
- [x] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [x] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/master/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/master/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors which may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

 albert, bert, XLM: @LysandreJik
 GPT2: @LysandreJik, @patrickvonplaten
 tokenizers: @mfuntowicz
 Trainer: @sgugger
 Benchmarks: @patrickvonplaten
 Model Cards: @julien-c
 examples/distillation: @VictorSanh
 nlp datasets: [different repo](https://github.com/huggingface/nlp)
 rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)
 Text Generation: @patrickvonplaten, @TevenLeScao
 Blenderbot, Bart, Marian, Pegasus: @patrickvonplaten
 T5: @patrickvonplaten
 Rag: @patrickvonplaten, @lhoestq
 EncoderDecoder: @patrickvonplaten
 Longformer, Reformer: @patrickvonplaten
 TransfoXL, XLNet: @TevenLeScao, @patrickvonplaten
 examples/seq2seq: @patil-suraj
 examples/bert-loses-patience: @JetRunner
 tensorflow: @jplu
 examples/token-classification: @stefan-it
 documentation: @sgugger
 FSMT: @stas00
 -->


Comments:
Can not convert the the custom trained BERT model to pytorch model for further use which should give me .bin file
Author: anidiatm41
Pull request: no
Labels: name: wontfix

name: Migration

# 📚 Migration

## Information

I pre trained a bert model from scratch with custom corpus and now want to use that model for further purpose like Qand A, masked word prediction etc. After pre- training I got below files:

bert_config.json bert_model.ckpt.data-00000-of-00001 bert_model.ckpt.index vocab.txt bert_model.ckpt.meta

Now , I need to convert the the model to pytorch model for further use which will give me .bin file . I am running below :+1: 



%cd /content/drive/My Drive/Anirban_test_pytorch/
!python convert_bart_original_pytorch_checkpoint_to_pytorch.py  "/content/drive/My Drive/Anirban_test_pytorch/model.ckpt.index" "/content/sample_data"\


But getting below error:

  File "convert_bart_original_pytorch_checkpoint_to_pytorch.py", line 75, in load_xsum_checkpoint
    sd = torch.load(checkpoint_path, map_location="cpu")
  File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 585, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/usr/local/lib/python3.6/dist-packages/torch/serialization.py", line 755, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '\x00'.

Please help me to fix the same.





Comments:
Hello! It seems you're running the BART conversion script for a BERT model?

Yeah, but I got to fix this.
-Anirban

On Thu, 15 Oct, 2020, 2:52 PM Lysandre Debut, <notifications@github.com>
wrote:

> Hello! It seems you're running the BART conversion script for a BERT model?
>
> —
> You are receiving this because you authored the thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/7801#issuecomment-709032258>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AMTD2W4GWGTXTL5DFR3DHPDSK25O3ANCNFSM4SRMD3AQ>
> .
>


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.