noamaanMulla-03/transformers-issues huggingface dataset:
issues while modifying modeling_roberta.py file
Author: nrjvarshney
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
I need the CLS representation from RobertaForMultipleChoice

For this, I changed 
 outputs = (reshaped_logits,) + outputs[2:] 
To 
 outputs = (reshaped_logits,) + outputs[1:] 
in file https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_roberta.py
since outputs[1] gives the CLS representation. 

Now to incorporate this change 
I need to import RobertaForMultipleChoice from this file  and replace "from transformers import ..." in https://github.com/huggingface/transformers/blob/master/examples/run_multiple_choice.py

to "from modeling_roberta.py import ..."

I am getting import issues 
![image](https://user-images.githubusercontent.com/19836137/76137311-8d937200-5ff8-11ea-98fc-4d361eee0b32.png)

Can somebody help in resolving this?

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

Comments:
That's more of a general Python question related to imports than an issue with the library, you would have more luck in trying with stack overflow.

I believe the easiest way to modify files is to clone the repository and install it in your environment as an editable:

```
git clone https://github.com/huggingface/transformers
cd transformers
pip install -e .
```

Every file modification will be directly reflected in your Python runtime (if you're on Jupyter you would need to restart your kernel for it to take effect).

Thanks a lot. 
Actually the problem was that I was appending the path (i.e src/transformers) at the end in PYTHON_PATH because of which it was loading modules from the transformers library. I added the source path at index 0 and now it is working the way I want it to work.

and thanks  @LysandreJik for the restart kernel trick.
Where is Model2Model PreTrainedEncoderDecoder in run_summerization_finetune
Author: yeliu918
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->


Comments:
@yeliu918 was there any ```run_summerization_finetune``` script before? Since I cannot find it now.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How to finetune PreTrainedEncoderDecoder
Author: fengzhangyin
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
PreTrainedEncoderDecoder is great. 
Now I have the following questions :
(1) How to use my data to finetune the PreTrainedEncoderDecoder?
(2) If I want to use the pretrained RoBERTa as encoder and deocder, what should I do ?

Comments:
Hi,

Thanks for the nice work. I have the same question.
Would appreciate your reply. 

Yes , example would be nice

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
links of the model's pretrained weights
Author: hzg0601
Pull request: no

where can I find all the links of the model's pretrained weights,please?to download them by IDE is too slow.

Comments:
We have a `HfApi.model_list()` method on this PR https://github.com/huggingface/transformers/pull/3132 that might be of interest to you.

Do let us know if it solves your use case or not.

I use the following script in an interactive enviroment:

```python
>> from transformers.hf_api import HfApi

>> HfApi.model_list()
```
But I got an error:` AttributeError: type object 'HfApi' has no attribute 'model_list'`

And my `transformers.__version__` is 2.5.1

Did I miss something ?

> We have a `HfApi.model_list()` method on this PR #3132 that might be of interest to you.
> 
> Do let us know if it solves your use case or not.

same problem with @sevenights ,remain unsolved,Did we miss something?


#3132 was just merged on master, you should be able to try on master now.

I get it, thank you :)
adding special tokens after truncating in run_lm_finetuning.py 
Author: lucy3
Pull request: no
Labels: name: wontfix

## 🐛 Bug

<!-- Important information -->

Model I am using (Bert, XLNet....): Bert

Language I am using the model on (English, Chinese....): English

The problem arise when using:
* [x] the official example scripts: (give details)

In `run_lm_finetuning.py`, we have the following
```
for i in range(0, len(tokenized_text) - block_size + 1, block_size):  # Truncate in block of block_size
    self.examples.append(tokenizer.build_inputs_with_special_tokens(tokenized_text[i : i + block_size]))
```

If we add special tokens after truncating to `block_size`, the example is no longer of length `block_size`, but longer. 

In the help information for `block_size` as an input argument, it says "Optional input sequence length after tokenization. The training dataset will be truncated in block of this size for training. Default to the model max input length for single sentence inputs (**take into account special tokens**)." This may be confusing, because the `block_size` written as the default input is 512, but if you use BERT-base as the model you're pretraining from, the `block_size` input in that function is actually 510. 

The [original BERT code](https://github.com/google-research/bert/blob/master/extract_features.py) makes sure all examples are `block_size` after adding special tokens: 
```
if tokens_b:
      # Modifies `tokens_a` and `tokens_b` in place so that the total
      # length is less than the specified length.
      # Account for [CLS], [SEP], [SEP] with "- 3"
      _truncate_seq_pair(tokens_a, tokens_b, seq_length - 3)
else:
      # Account for [CLS] and [SEP] with "- 2"
      if len(tokens_a) > seq_length - 2:
        tokens_a = tokens_a[0:(seq_length - 2)]
```

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
how to use distilledgpt2
Author: jackfeinmann5
Pull request: no
Labels: name: wontfix

Hi
I want to use distilledgpt2, I cannot see the config file and modeling files, could you please assist me how to use it
thanks

Comments:
Hi, you can use it as such:

```py
from transformers import GPT2Model, GPT2Tokenizer

model = GPT2Model.from_pretrained("distilgpt2")
tokenizer = GPT2Tokenizer.from_pretrained("distilgpt2")
```

You can see the list of available models in the [pretrained section of our documentation](https://huggingface.co/transformers/pretrained_models.html).

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
About the examples document of bert with SQuAD 2.0
Author: senp98
Pull request: no

I'm wondering if there is a mistake.
In the README document of /examples, the training parameters setting for training on SQuAD dataset, the first code block:
```
export SQUAD_DIR=/path/to/SQUAD

python run_squad.py \
  --model_type bert \
  --model_name_or_path bert-base-cased \
  --do_train \
  --do_eval \
  --do_lower_case \
  --train_file $SQUAD_DIR/train-v1.1.json \
  --predict_file $SQUAD_DIR/dev-v1.1.json \
  --per_gpu_train_batch_size 12 \
  --learning_rate 3e-5 \
  --num_train_epochs 2.0 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --output_dir /tmp/debug_squad/
```
Because this model is bert-base-cased, which means it's cased, I think the ``--do_lower_case`` shouldn't be here. Is that a mistake?

Comments:
Indeed, I think this is a mistake, I think it should be `bert-base-uncased` instead. I'm updating it.
fix repetition penalty mask in tf
Author: patrickvonplaten
Pull request: yes

Fixed bug with TF 2.0 `repetition_penalty` when doing generation add make `early_stopping` an argument to the function `generate()`.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3190?src=pr&el=h1) Report
> Merging [#3190](https://codecov.io/gh/huggingface/transformers/pull/3190?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/b29fed790bdaa4be38b6d2c5de88e307474ea38d?src=pr&el=desc) will **increase** coverage by `0.1%`.
> The diff coverage is `42.85%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3190/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/3190?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##           master    #3190     +/-   ##
=========================================
+ Coverage   77.98%   78.09%   +0.1%     
=========================================
  Files          98       98             
  Lines       16641    16645      +4     
=========================================
+ Hits        12978    12999     +21     
+ Misses       3663     3646     -17
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3190?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/configuration\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3190/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb25maWd1cmF0aW9uX3V0aWxzLnB5) | `96.74% <100%> (+0.02%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3190/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `91.38% <33.33%> (+3.59%)` | :arrow_up: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3190/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `94.56% <0%> (-0.16%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3190?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3190?src=pr&el=footer). Last update [b29fed7...847d370](https://codecov.io/gh/huggingface/transformers/pull/3190?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Good to merge for me.

> Small typo bug otherwise it's good to go

Thanks for spotting!
Where is the default download address for pre-trained weight
Author: 649459021
Pull request: no

# ❓ Questions & Help

```
from transformers import DistilBertTokenizer, DistilBertModel

tokenizer = DistilBertTokenizer.from_pretrained('distilbert-base-uncased')
model = DistilBertModel.from_pretrained('distilbert-base-uncased')
```
I can't find the downloaded file.
Thanks for your help

Comments:
It's in your torch home:

```py
>>> from torch.hub import _get_torch_home
>>> _get_torch_home()
'/home/<USER>/.cache/torch'
```
Model card for albert-base-v2-squad2
Author: traviemcg
Pull request: yes
Labels: name: model card

Just creating model card for new community model!

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3199?src=pr&el=h1) Report
> Merging [#3199](https://codecov.io/gh/huggingface/transformers/pull/3199?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/49debe62fdc96e161f866dd8914d5915477bb742?src=pr&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3199/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/3199?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##           master   #3199      +/-   ##
=========================================
+ Coverage   77.98%     78%   +0.01%     
=========================================
  Files          98      98              
  Lines       16645   16645              
=========================================
+ Hits        12981   12984       +3     
+ Misses       3664    3661       -3
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3199?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3199/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `68% <0%> (-0.41%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3199/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `94.4% <0%> (-0.16%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3199/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.64% <0%> (+0.97%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3199?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3199?src=pr&el=footer). Last update [49debe6...f53348d](https://codecov.io/gh/huggingface/transformers/pull/3199?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thanks for sharing. [Model page](https://huggingface.co/twmkn9/albert-base-v2-squad2)
What dataset was used for the NER results reported in the docs for bert/roberta-large-cased and distilbert-base-uncased models?
Author: ohmeow
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

Regarding [this section in the docs](https://huggingface.co/transformers/examples.html#comparing-bert-large-cased-roberta-large-cased-and-distilbert-base-uncased) and the NER results using bert-large-cased, roberta-large-cased, and distillbert-base-uncased ...

**What dataset was used?**

When I try them with the GermanEval2014 dataset, the results are inferior to that of the multi-lingual models (which makes sense) ... so my question, upon what dataset(s) were these models trained on that go the most excellent F scores reported in the docs?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Add integration tests lm generate torch tf
Author: patrickvonplaten
Pull request: yes

Add integration tests for all LM models that are able to generate language.

- All integration tests use `do_sample=False` (greedy) generation and verify that TF 2.0 and PT yield the same results. 
- Fixed a small bug with TFXLMModelWithLMHead

Comments:
Approx. how long do these tests take on a single V100?

> Approx. how long do these tests take on a single V100?

Don't know how long they take on a V100. On a cpu, all tests combined (7 model tests for PT and 6 model tests for TF) take less than 10min (whereas `test_modeling_tf_xlnet.py`, `test_modeling_xlnet.py` and `test_modeling_transfo_xl.py` combined take ca. 8min)

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3191?src=pr&el=h1) Report
> Merging [#3191](https://codecov.io/gh/huggingface/transformers/pull/3191?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e03129ad447ad7670fcc6206e5eb27a5435d4d86?src=pr&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3191/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/3191?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3191      +/-   ##
==========================================
+ Coverage   78.15%   78.16%   +0.01%     
==========================================
  Files          98       98              
  Lines       16641    16641              
==========================================
+ Hits        13006    13008       +2     
+ Misses       3635     3633       -2
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3191?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/3191/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl94bG0ucHk=) | `90.4% <100%> (ø)` | :arrow_up: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3191/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `94.56% <0%> (+0.15%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3191/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `93.89% <0%> (+0.19%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3191?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3191?src=pr&el=footer). Last update [e03129a...9050ffe](https://codecov.io/gh/huggingface/transformers/pull/3191?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Good to merge for me
Error when runnig run_tf_ner.py
Author: mrm8488
Pull request: no

# 🐛 Bug

## Information
```python
!python3 /content/transformers/examples/ner/run_tf_ner.py --data_dir /content/ner_dataset \
--model_type bert \
--labels /content/labels.txt \
--model_name_or_path dccuchile/bert-base-spanish-wwm-cased \
--output_dir model_output \
--max_seq_length 256 \
--num_train_epochs 5\
--per_gpu_train_batch_size 42 \
--save_steps 2000\
--do_train \
--do_eval

Traceback (most recent call last):
  File "/content/transformers/examples/ner/run_tf_ner.py", line 14, in <module>
    from transformers import (
ImportError: cannot import name 'GradientAccumulator'
```

## Transformer version ```transformers==2.5.1```

Comments:
You should not have any issues with this as `GradientAccumulator` is correctly imported in the __init__.py file:

https://github.com/huggingface/transformers/blob/298bed16a841fae3608d334441ccae4d9043611f/src/transformers/__init__.py#L426-L427

Is `transformers` correctly installed in your pip environment, or did you simply clone the repository?

Ok. Thank you. I cannot check it now because Colab PRO accounts are having problems. I will let you know ASAP.

I met analogous problem，but it was solved after rebooting computer. And another issue is it can not find the installed module，this issue can be solved by reinstall the module that can not be found.

In my case I was not using tf 2.x

El mar., 10 mar. 2020 9:50, over-shine <notifications@github.com> escribió:

> I met analogous problem，but it was solved after rebooting computer. And
> another issue is it can not find the installed module，this issue can be
> solved by reinstall the module that can not be found.
>
> —
> You are receiving this because you modified the open/close state.
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/3087?email_source=notifications&email_token=AA34BHNOLTNBPN37WKT4UELRGX5MHA5CNFSM4K73DTO2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOKQYOY#issuecomment-596970555>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AA34BHIJAGHRKSZE5TNILB3RGX5MHANCNFSM4K73DTOQ>
> .
>
Update README.md
Author: mrm8488
Pull request: yes
Labels: name: model card

- Clarify that the model is not trained on the evaluation dataset

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3202?src=pr&el=h1) Report
> Merging [#3202](https://codecov.io/gh/huggingface/transformers/pull/3202?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/5ca356a464e98e065488205f3fcf9247f56c3832?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3202/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/3202?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3202      +/-   ##
==========================================
+ Coverage   77.96%   77.97%   +<.01%     
==========================================
  Files          98       98              
  Lines       16668    16668              
==========================================
+ Hits        12996    12997       +1     
+ Misses       3672     3671       -1
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3202?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3202/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `94.72% <0%> (+0.15%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3202?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3202?src=pr&el=footer). Last update [5ca356a...bce6ca3](https://codecov.io/gh/huggingface/transformers/pull/3202?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Roberta tokenizer fails on certain unicode characters
Author: 0xEdgar
Pull request: no

## 🐛 Bug

<!-- Important information -->

Model I am using: `Roberta`:

Language I am using the model on (English, Chinese....): English

The problem arise when using:
- The `roberta-base` tokenizer and tokenizing unicode accents
```
from pytorch_transformers import *

tokenizer = RobertaTokenizer.from_pretrained('roberta-base')
phrase = "I visited the Côte d'Azur"
for word in phrase.split():
    print(tokenizer.tokenize(word))

```
this outputs: 

```
['I']
['vis', 'ited']
['the']
['C', 'Ã´', 'te']
['d', "'", 'Az', 'ur']
```

## Expected behavior
```
['I']
['vis', 'ited']
['the']
['C', 'ô', 'te']
['d', "'", 'Az', 'ur']
```
## Environment

* OS: MacOS 10.14.6
* Python version: 3.6
* PyTorch version: 1.1.0.post2
* PyTorch Transformers version (or branch): 1.1.0
* Using GPU ? no
* Distributed of parallel setup ? no
* Any other relevant information: xlnet and bert do not face this same tokenization issue. It appears that the issue comes from the gpt2-tokenizer 

## Additional context

<!-- Add any other context about the problem here. -->

Comments:
Does `fairseq` exhibit the same behavior?

If it does, I would ask upstream. But in any case, I'm not sure it's a bug (it's just the internal encoding used by the neural net)

I think you need to tokenize by XLM-Roberta-Tokenizer.
I tokenized for Korean.