noamaanMulla-03/transformers-issues huggingface dataset:
Create README.md
Author: DukeEnglish
Pull request: yes
Labels: name: model card

roberta_chinese_large card

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3340?src=pr&el=h1) Report
> Merging [#3340](https://codecov.io/gh/huggingface/transformers/pull/3340?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/20139b7c8d88f380f1e4e0ae2baf0b0ac9351039&el=desc) will **not change** coverage by `%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3340/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3340?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3340   +/-   ##
=======================================
  Coverage   77.63%   77.63%           
=======================================
  Files         100      100           
  Lines       16943    16943           
=======================================
  Hits        13154    13154           
  Misses       3789     3789           
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3340?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3340?src=pr&el=footer). Last update [20139b7...b81687c](https://codecov.io/gh/huggingface/transformers/pull/3340?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


I think the filepath for this one is incorrect.

Also could you add
```
---
language: chinese
---
```
at the top of the file? Thanks!

Merged in 73d6a2f9019960c327f19689c1d9a6c0fba31d86
Create README.md
Author: DukeEnglish
Pull request: yes
Labels: name: model card

xlnet_chinese_large card

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3339?src=pr&el=h1) Report
> Merging [#3339](https://codecov.io/gh/huggingface/transformers/pull/3339?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/20139b7c8d88f380f1e4e0ae2baf0b0ac9351039&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3339/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3339?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3339      +/-   ##
==========================================
- Coverage   77.63%   77.63%   -0.01%     
==========================================
  Files         100      100              
  Lines       16943    16943              
==========================================
- Hits        13154    13153       -1     
- Misses       3789     3790       +1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3339?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3339/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `88.19% <0.00%> (-0.18%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3339?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3339?src=pr&el=footer). Last update [20139b7...a6ee180](https://codecov.io/gh/huggingface/transformers/pull/3339?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


same issue as #3340 

Merged in 73d6a2f9019960c327f19689c1d9a6c0fba31d86
Add model tag
Author: mrm8488
Pull request: yes
Labels: name: model card

Add model tag to be correctly indexed while working on card description

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3709?src=pr&el=h1) Report
> Merging [#3709](https://codecov.io/gh/huggingface/transformers/pull/3709?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/6435b9f908e7361330db89e263a65b0a58060d11&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3709/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3709?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #3709      +/-   ##
==========================================
- Coverage   78.13%   78.12%   -0.01%     
==========================================
  Files         104      104              
  Lines       17723    17723              
==========================================
- Hits        13847    13846       -1     
- Misses       3876     3877       +1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3709?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3709/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.79% <0.00%> (-0.17%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3709?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3709?src=pr&el=footer). Last update [6435b9f...be477cf](https://codecov.io/gh/huggingface/transformers/pull/3709?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
How to input hidden state vectors from GPT2Model directly into mc_head of the GPT2DoubleHeads Model?
Author: h56cho
Pull request: no

Hello,

I want to perform the following task:

**1.** _Feed in a text sequence to a GPT2Model (the GPT-2 model without output head);_

**2.** _Extract the hidden state vectors of the text sequence that are generated at each layer of the GPT2Model;_

**3.** _Input the hidden state vectors from each layer of the GPT2Model directly into the mc_head of the GPT2DoubleHeadsModel, and calculate the **mc_loss** that results from inputting the hidden state vectors._


I know that the code below can be used to input hidden state vectors into each individual layer of a HuggingFace GPT-2 model, but I am not sure if similar task can be done on the mc_head of the GPT2DoubleHeadsModel:

```python
model.transformer.h[i](embed)
```

My questions is:
 Is there anyway that I can input the hidden state vectors from each layer of the GPT2Model directly into the multiple choice head of the GPT2DoubleHeadsModel, and extract the **mc_loss** that results from it (i.e. is there any way that I can perform the task **3**)?

Thank you,

Comments:
[FIX] #2658 Inconsistent values returned by batch_encode_plus and enc…
Author: WoodyFleurant
Pull request: yes
Labels: name: wontfix

As ticket describe, when using batch_encode_plus, instead of encode_plus, tokens type and mask are different. They should be the same using batch processing or not. Proposed fix here solve the issue

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2659?src=pr&el=h1) Report
> Merging [#2659](https://codecov.io/gh/huggingface/transformers/pull/2659?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e0849a66accda8aa435a3db164c373175115a5b0?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2659/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2659?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #2659      +/-   ##
==========================================
+ Coverage   74.51%   74.51%   +<.01%     
==========================================
  Files          87       87              
  Lines       14920    14920              
==========================================
+ Hits        11117    11118       +1     
+ Misses       3803     3802       -1
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2659?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2659/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `85.85% <ø> (+0.16%)` | :arrow_up: |
| [src/transformers/commands/serving.py](https://codecov.io/gh/huggingface/transformers/pull/2659/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9jb21tYW5kcy9zZXJ2aW5nLnB5) | `0% <ø> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2659?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2659?src=pr&el=footer). Last update [e0849a6...6efcbec](https://codecov.io/gh/huggingface/transformers/pull/2659?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
unable to load model 'bert', tensor 'input_ids': the model expects 1 dimensions but the model configuration specified 2 dimensions
Author: laohur
Pull request: no
Labels: name: Usage

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->
[https://github.com/NVIDIA/triton-inference-server/issues/1338](url)
how to support batchsize for bert onnx in triton-inference-server?

    use bert from https://github.com/huggingface/transformers,

    fellow https://github.com/microsoft/onnxruntime/tree/master/onnxruntime/python/tools/bert#export-a-bert-model-from-pytorch.

    gotten "model.onnx" with batchsize as first dim (netron).

i set dynamic_batching { preferred_batch_size: [ 4, 8, 32 ] max_queue_delay_microseconds: 100 } in "config.pbtxt"

wrong
max_batch_size : 1024 input [ { name: "input_ids" data_type: TYPE_INT64 dims: [-1, 128] } ]

wrong max_batch_size : 1024 input [ { name: "input_ids" data_type: TYPE_INT64 dims: [1, 128] reshape: { shape: [-1,128 ] } } ]

ok max_batch_size : 1024 input [ { name: "input_ids" data_type: TYPE_INT64 dims: [128] } ]

all latest two version.
## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

Comments:
I'm assuming you read https://blog.einstein.ai/benchmarking-tensorrt-inference-server/ ?

> 
> 
> I'm assuming you read https://blog.einstein.ai/benchmarking-tensorrt-inference-server/ ?
ok: torch.jit.trace(bert)
Failed :torch.jit.script
Failed:torch.jit.trace(bert+after)
Failed: to onnx: 
  File "/usr/local/lib/python3.6/dist-packages/transformers/modeling_bert.py", line 686, in forward
    extended_attention_mask = extended_attention_mask.to(dtype=next(self.parameters()).dtype)  # fp16 compatibility
UnboundLocalError: local variable 'extended_attention_mask' referenced before assignment   
（pip transformers==2.2.0， fixed latested source）

solved by remove all none default option argments. minize uncertainty
When using the Bert model
Author: bigzhouj
Pull request: no

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
pregenerate_training_data.py and  finetune_on_pregenerated.py Is it in the project.If so, where?

Comments:
These two files are **not** in the Transformers library **now**.

> ##  Questions & Help
> pregenerate_training_data.py and finetune_on_pregenerated.py Is it in the project.If so, where?

As @TheEdoardo93 says, these files were community maintained and have been removed a few months ago. 

so what is the procedure to fine tune BERT on my data?
Add CALBERT (Catalan ALBERT) base-uncased model card
Author: txus
Pull request: yes
Labels: name: model card

Hi there!

I just uploaded a new ALBERT model pretrained on 4.3 GB of Catalan text. Thank you for these fantastic libraries and platform to make this a joy!

Comments:
Looks good! Model page for [CALBERT](https://huggingface.co/codegram/calbert-base-uncased)

One additional tweak: you should add a 

```
---
language: catalan
---
```

metadata block at the top of the model page. Thanks!
[WIP] Add a trainer tool class to make the TF2 model training easier
Author: jplu
Pull request: yes

Hello,

I decided to open a cleaner PR because the other was a bit too messy in my opinion. Here the features that this Trainer class will be able to handle:

- [x] Single / Multiple GPU training. Distributed is across GPUs in the same host. Distribution across multiple machines will be for a future version.
- [ ] The training can be configured with a JSON file.
- [x] Handle multiple data processor to be able to train a model over different datasets.
- [x] Select and configure a specific loss/optimizer for a training
- [x] Create multiple checkpoints during the training in order to make it fault-tolerant.
- [x] Create the logs to be able to visualize the training in Tensorboard
- [x] The final model is saved in Hugging face transformer format and in TF saved model
- [x] Able to give a data directory where to find the datasets
- [x] Automatically handle dataset/model caching
- [ ] Run an evaluation over a test dataset with proper printed results such as the one proposed by the `seqeval` package

Currently the trainer class can be used over glue and xnli datasets with the available examples `examples/run_tf_xnli_with_trainer.py` and `examples/run_tf_glue_with_trainer.py`. I will add new examples for differents tasks and datasets.

The list of features above will be checked as things progress.

Ping @julien-c @LysandreJik @thomwolf : Do not hesitate to make proposals if you have new ideas of features or advices on a better implementation of this trainer.

Comments:
Question regarding glue examples
Author: Mahmedturk
Pull request: no

hi,

I have used the run_glue.py for QQP dataset and the tested the saved model for a small test set. I get really high F1-score on dev set but get lower F1-score on the test set. My question is that, is the dev set used in glue examples is used to set parameters or is it a held out set?


Comments:
I don't understand the question. The labels for the test sets for GLUE are privately-held, if that's what you're asking.

(Closing this as it's not really specific to this repo)
run_ner.py load checkpoint issue
Author: giuliorav
Pull request: no
Labels: name: wontfix

Hi,
I just launched the **transformers/examples/run_ner.py** script with my custom model:
`python3 transformers/examples/run_ner.py --data_dir $INPUT_DATA_DIR \
--tokenizer_name $TOKENIZER_FILE_PATH  --output_dir $OUTPUT_DIR --model_type camembert --labels $LABELS_DIR --model_name_or_path $BERT_MODEL  --max_seq_length  $MAX_LENGTH  --num_train_epochs $NUM_EPOCHS --gradient_accumulation_steps $ACCUMULATION_STEPS --per_gpu_train_batch_size $BATCH_SIZE --save_steps $SAVE_STEPS --do_lower_case --do_train --do_eval --do_predict`

Once the data for the train has been loaded, an error appear:
`Traceback (most recent call last):
  File "transformers/examples/run_ner.py", line 567, in <module>
    main()
  File "transformers/examples/run_ner.py", line 496, in main
    global_step, tr_loss = train(args, train_dataset, model, tokenizer, labels, pad_token_label_id)
  File "transformers/examples/run_ner.py", line 132, in train
    global_step = int(args.model_name_or_path.split('-')[-1].split('/')[0])
ValueError: invalid literal for int() with base 10: 'pytorch_dump_folder'`

Launching the same script a few hours ago the error did not appear, is it something related to the last updates #2134 ?

Thanks in advance.

Comments:
Can confirm this issue -> a temporary workaround would be to change the line to:

```python
if os.path.exists(args.model_name_or_path) and "checkpoint" in args.model_name_or_path:
```

See also related (recent) fix on master: https://github.com/huggingface/transformers/commit/4d36472b96d144887cbe95b083f0d2091fd5ff03

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Generation with gpt-2
Author: simonefrancia
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

Hi @julien-c and all,
we are developing a training with GPT-2 language model based on our custom dataset.
Firstly, we train a custom Tokenizer with tokenizers ( as described [here](https://github.com/huggingface/tokenizers/issues/166) ) .
Now I have a dummy GPT-2 model trained on 1 epoch but my problem is on inference.
The result is always the same when I feed the model with the starting text.

Code (is the same at link ) 
```
from transformers import GPT2LMHeadModel, GPT2Tokenizer
import torch

tokenizer = GPT2Tokenizer.from_pretrained("my-gpt2-tokenizer")
model = GPT2LMHeadModel.from_pretrained('my-gpt2-model')

generated = tokenizer.encode("The Manhattan bridge")
context = torch.tensor([generated])
past = None

for i in range(100):
    print(i)
    output, past = model(context, past=past)
    token = torch.argmax(output[..., -1, :])

    generated += [token.tolist()]
    context = token.unsqueeze(0)

sequence = tokenizer.decode(generated)
print(sequence)
```
Using this code, the output is repeated:

```
The Manhattan bridge<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit<nl> I'm a little bit
```
I think that problems are two basically:
1. the first is that for every inference we take only the argument that maximizes probability, and I don't use any type of sampler based on some type of distribution.
2. the second is that the content fed after the first step is always the next word predicted , so it goes in loop.

I don't have any other ideas. Is there an example of inference with huggingface lib?
Thanks



Comments:
There is an example: https://github.com/huggingface/transformers/blob/master/examples/run_generation.py

Thanks!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Training dataset is not available
Author: wboleksii
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

As stated [here](https://github.com/huggingface/transformers/blob/1ab8dc44b3d84ed1894f5b6a6fab58fb39298fc7/examples/distillation/README.md), the model was trained using Toronto Book Corpus and English Wikipedia. Neither this repository or BERT repository provides links to obtain this data. Upon further investigation the Toronto Book Corpus is no longer public. Please advise on how to get this data.

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


paging @LysandreJik 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Fine-tuning the model using classification tasks
Author: ankush20m
Pull request: no
Labels: name: wontfix

name: Ex: Sequence Classification

name: Usage

Hello All,

Could anyone tell how can I fine-tune the language model using classification tasks, however not using any GLUE data, as I have my own custom dataset?
Is there any solution and/or method to do the classification using a custom dataset?

Comments:
Hi, the `run_glue` example script was designed to showcase how to fine-tune any model to a classification task. It showcases many things you maybe don't need, such as data-parallel, checkpointing, half-precision, etc. You can adapt this script or study the training loop to create your own.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
PreTrainedTokenizer returns potentially incorrect attention mask
Author: ab-10
Pull request: no
Labels: name: Discussion

name: wontfix

name: Core: Tokenization

# 🐛 Bug

## Information

When deriving the `attention_mask` `PreTrainedTokenizer` makes an assumption in `prepare_for_model` that the input hasn't been padded prior, this assumption can be false. For example, in the case where one precomputes padded token ids for sentences separately and then uses `BertTokenizer.encode_plus` to join them.

I'm submitting this issue, in order to find out whether this assumption has been made on purpose and if it hasn't I can easily submit a PR fixing it.

In the `PreTrainedTokenizer`, the  `attention_mask` is obtained in two places:

- line `1175`: `encoded_inputs["attention_mask"] = [0] * difference + [1] * len(encoded_inputs["attention_mask"] = [0] * difference + [1] * len(encoded_inputs["input_ids"]) `

- line `1188`: `encoded_inputs["attention_mask"] = [1] * len(encoded_inputs["input_ids"])`.

I suggest that instead of making the assumption attention mask is derived as:

`encoded_inputs["attention_mask"] = encoded_inputs["input_ids"] != 0`

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.