noamaanMulla-03/transformers-issues huggingface dataset:
Summarization pipeline fails to initialize
Author: singulart
Pull request: no

# 🐛 Bug

## Information

Model I am using (T5):

Language I am using the model on (Englis):

The problem arises when using:
* [+] the official example scripts: (give details below)
https://huggingface.co/transformers/main_classes/pipelines.html#transformers.SummarizationPipeline
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:
```
from transformers import pipeline
summarizer = pipeline("summarization", model="t5-base", tokenizer="t5-base")
```
Produces the error:

```
Downloading: 100%|██████████| 230/230 [00:00<00:00, 231kB/s]
Couldn't reach server at 'https://s3.amazonaws.com/models.huggingface.co/bert/t5-base-modelcard.json' to download model card file.
Creating an empty model card.
Traceback (most recent call last):
  File "D:\anaconda3\envs\gpt2\lib\site-packages\transformers\configuration_utils.py", line 243, in get_config_dict
    raise EnvironmentError
OSError
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "D:\anaconda3\envs\gpt2\lib\site-packages\IPython\core\interactiveshell.py", line 3331, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-42-1a545b8d35ed>", line 1, in <module>
    summarizer = pipeline("summarization", model="t5-base", tokenizer="t5-base")
  File "D:\anaconda3\envs\gpt2\lib\site-packages\transformers\pipelines.py", line 1423, in pipeline
    model = model_class.from_pretrained(model, config=config, **model_kwargs)
  File "D:\anaconda3\envs\gpt2\lib\site-packages\transformers\modeling_utils.py", line 434, in from_pretrained
    **kwargs,
  File "D:\anaconda3\envs\gpt2\lib\site-packages\transformers\configuration_utils.py", line 192, in from_pretrained
    config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
  File "D:\anaconda3\envs\gpt2\lib\site-packages\transformers\configuration_utils.py", line 262, in get_config_dict
    raise EnvironmentError(msg)
OSError: Can't load 't5-base'. Make sure that:
- 't5-base' is a correct model identifier listed on 'https://huggingface.co/models'
- or 't5-base' is the correct path to a directory containing a 'config.json' file
```
## Expected behavior
No exception

## Environment info

- `transformers` version: 2.6.0
- Platform: Windows
- Python version: 3,7
- PyTorch version (GPU?): 1.4.0
- Tensorflow version (GPU?): 2.1.0
- Using GPU in script?: No
- Using distributed or parallel set-up in script?:


Comments:
You're using `transformers` version **2.6.0**, but T5 model was released in version **2.7.0**.

Update your library with :

`pip install --upgrade transformers`

Thanks, error is not reproduced in 2.8.0
[Generation, EncoderDecoder] Apply Encoder Decoder 1.5GB memory savings to TF as well
Author: patrickvonplaten
Pull request: yes

As was done by @sshleifer for torch, improved the memory usage for TF Encoder Decoder models. 
Straight-forward translation of PR: https://github.com/huggingface/transformers/pull/3370.

Comments:
Tested on `RUN SLOW=1 pytest tests/test_modeling_tf_t5.py` and all tests pass.
ImportError: cannot import name 'HfArgumentParser' from 'transformers'
Author: songproducer
Pull request: no

Hi! When running:

```
python ./examples/run_glue.py \
    --model_type bert \
    --model_name_or_path bert-base-uncased \
    --task_name $TASK_NAME \
    --do_train \
    --do_eval \
    --data_dir $GLUE_DIR/$TASK_NAME \
    --max_seq_length 128 \
    --per_gpu_eval_batch_size=8   \
    --per_gpu_train_batch_size=8   \
    --learning_rate 2e-5 \
    --num_train_epochs 3.0 \
    --output_dir /tmp/$TASK_NAME/

```

I get

```
Traceback (most recent call last):
  File "./examples/run_glue.py", line 34, in <module>
    from transformers import (
ImportError: cannot import name 'HfArgumentParser' from 'transformers' (/Users/leotreasure/opt/anaconda3/lib/python3.7/site-packages/transformers/__init__.py)

```

This is my python path
/Users/leotreasure/opt/anaconda3/bin/python

I ran the GLUE download script in the same folder
python download_glue_data.py
and exported: 
export GLUE_DIR=/Users/leotreasure/transformers
export TASK_NAME=MRPC

Comments:
You need to install from source, as explained [here](https://github.com/huggingface/transformers#run-the-examples).

Thanks!
Fine-tuning TF models on Colab TPU
Author: NaxAlpha
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

<!-- A clear and concise description of the question. -->

Hi, 

I am trying to fine-tune TF BERT on Imdb dataset on Colab TPU. Here is the full notebook:

https://colab.research.google.com/drive/16ZaJaXXd2R1gRHrmdWDkFh6U_EB0ln0z

Can anyone help me what I am doing wrong?
Thanks



Comments:
I've read your code, and I don't see anything strange in it (I hope). It seems to be an error due to training a (whatever) model on TPUs rather than Transformers.

Do you see [this](https://github.com/tensorflow/tensorflow/issues/29896) issue reported in TensorFlow's GitHub? It seems to be the same error, and someone gives indications about how to resolve.

> ##  Questions & Help
> Hi,
> 
> I am trying to fine-tune TF BERT on Imdb dataset on Colab TPU. Here is the full notebook:
> 
> https://colab.research.google.com/drive/16ZaJaXXd2R1gRHrmdWDkFh6U_EB0ln0z
> 
> Can anyone help me what I am doing wrong?
> Thanks

Thanks for the help. Actually my code was inspired from [this colab notebook. This](https://colab.research.google.com/github/CyberZHG/keras-bert/blob/master/demo/tune/keras_bert_classification_tpu.ipynb) notebook works perfectly but there is one major difference might be causing the problem:

- I am force installing tensorflow 2.x for transformer notebook because transformer works only for TF>=2.0 but colab uses TF 1.15 on colab for TPUs

So I went to GCP and used a TPU for TF 2.x. The error changed to this:
![image](https://user-images.githubusercontent.com/11090613/71145292-e138bf80-2242-11ea-9ce8-31a53a0f9a1d.png)

But yes, you are right this issue might be related to TF in general.

In the official docs of GCP [here](https://cloud.google.com/tpu/docs/supported-versions), they show the current list of supported TensorFlow and Cloud TPU versions; _only_ **TensorFlow 1.13, 1.14 and 1.15 are supported** (they don't mention TensorFlow 2.0).
I think it's why you have the problem you've highlighted.

> Thanks for the help. Actually my code was inspired from [this colab notebook. This](https://colab.research.google.com/github/CyberZHG/keras-bert/blob/master/demo/tune/keras_bert_classification_tpu.ipynb) notebook works perfectly but there is one major difference might be causing the problem:
> 
> * I am force installing tensorflow 2.x for transformer notebook because transformer works only for TF>=2.0 but colab uses TF 1.15 on colab for TPUs
> 
> So I went to GCP and used a TPU for TF 2.x. The error changed to this:
> ![image](https://user-images.githubusercontent.com/11090613/71145292-e138bf80-2242-11ea-9ce8-31a53a0f9a1d.png)
> 
> But yes, you are right this issue might be related to TF in general.

tpu supports . tf2.1

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
unconditional generation with run_generation.py
Author: ehsan-soe
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help

Is it possible to have unconditional generation with ``` run_generation.py``` ? I realized the previous script ```run_gpt2.py``` had this option.
Can we use the same ```start_token``` trick?

Comments:
Hi, by unconditional generation do you mean generating a sequence from no context? If so, if using GPT-2, you can set your initial context to be: `
`. This will generate sequences with no other initial context.

You could do so like this:

```
from pytorch_transformers import GPT2Tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
context = [tokenizer.encoder["
"]]
```

@LysandreJik Yeah I did that but It is not performing good, it generated the start like this:
```
"
ingly anticipation passations, out he didn't realize that any or products in order to stand on. As Eric's disappointment, he threw into those o-bag's fanware vugeless chainsas. Finally, Chris went on a hob, and grabbed the crurne cartocos juice!"
```
I wonder if I can add a starting special token (sth like [CLS]) to my inputs and finetune gpt2 with this added vocabulary?
I have asked this [here](https://github.com/huggingface/pytorch-transformers/issues/1145) though

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


> @LysandreJik Yeah I did that but It is not performing good, it generated the start like this:
> 
> ```
> "
ingly anticipation passations, out he didn't realize that any or products in order to stand on. As Eric's disappointment, he threw into those o-bag's fanware vugeless chainsas. Finally, Chris went on a hob, and grabbed the crurne cartocos juice!"
> ```
> 
> I wonder if I can add a starting special token (sth like [CLS]) to my inputs and finetune gpt2 with this added vocabulary?
> I have asked this [here](https://github.com/huggingface/pytorch-transformers/issues/1145) though

Hi Have you figured out this issue?
Fix force_download of files on Windows
Author: calpt
Pull request: yes

On Windows, `os.rename` gives the error
```
FileExistsError: [WinError 183] Cannot create a file when that file already exists:
```
when trying to re-download a model that already exists in cache using `force_download=True`.

Comments:
Can't test this right now as I don't have access to a Windows machine - does this still work if the model doesn't already exist?

> 
> 
> Can't test this right now as I don't have access to a Windows machine - does this still work if the model doesn't already exist?

Yes, I tested that 😄. There shouldn't be any further differences between `rename` and `replace`. Here's [what the documentation says](https://docs.python.org/3/library/os.html#os.replace).
convert_tokens_to_ids(self, tokens)中的ids.append(self.vocab[token])，KeyError
Author: jiangjiaqi6
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

## Details
<!-- Description of your issue -->
KeyError                                  Traceback (most recent call last)
<ipython-input-20-dfb0a32a8e67> in <module>()
     19 for mask_pos in mask_positions:
     20     candidates = options[num]
---> 21     candidates_ids = tokenizer.convert_tokens_to_ids(candidates)
     22     token_ids = tokenizer.convert_tokens_to_ids(tokenized_text)
     23     tokens_tensor = torch.tensor([token_ids])

~/anaconda3/lib/python3.6/site-packages/pytorch_pretrained_bert/tokenization.py in convert_tokens_to_ids(self, tokens)
    119         ids = []
    120         for token in tokens:
--> 121             ids.append(self.vocab[token])
    122         if len(ids) > self.max_len:
    123             logger.warning(

KeyError: 'persuading'

How to solve KeyError problems?Thank you.

Comments:
The token that you are trying to convert to ids doesn't exist. 'persuading' is a long word, so it's likely that it is not as such present in the vocab. Instead you'll have to tokenize it first into subword units.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
In BertForSequenceClassification, why is loss initialised in every forward?
Author: BramVanroy
Pull request: no
Labels: name: wontfix

Looking at [the source](https://github.com/huggingface/pytorch-transformers/blob/master/pytorch_transformers/modeling_bert.py#L902-L910) I can see that the correct loss function is initialized in each call to forward.

https://github.com/huggingface/pytorch-transformers/blob/master/pytorch_transformers/modeling_bert.py#L902-L910

Can you explain why? Why isn't the loss function set up as part of `init()`? Is there any advantage of always re-initialising it on each forward? 

Edit: I see that you do this in other parts as well, e.g. the ReLU layer in distilbert: https://github.com/huggingface/pytorch-transformers/blob/master/pytorch_transformers/modeling_distilbert.py#L598

Comments:
Also it will be nice if the user gets to use the loss_func itself, Like currently i am using that class with slight modifications to match the pipeline with different losses rather than only CrossEntropy loss. (plus add class_weights etc as well to it)

Though this is what i did actually to use a different loss function, just grab the logits from the model and apply your own..

You can always subclass the class, to make it your own.

Some extra information for this issue: in an issue over at pytorch, it came to light that loss functions are actually meant to be imported as functions (from nn.functional) rather than modules (from nn). 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


Unstale. 

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
export to onnx issue
Author: jian16
Pull request: no
Labels: name: wontfix

name: External

Hi experts,

I got an error when running the onnx model after conversion. Can anyone please help to take a look? 

code:
`torch.onnx.export(model, 
    (input_ids, attention_mask, token_type_ids), 
    "bert.onnx", 
    input_names=['input_ids', 'attention_mask', 'token_type_ids'], 
    export_params=True, verbose=True)`

`sess = rt.InferenceSession("bert.onnx")
inputs = {'input_ids': input_ids, 'attention_mask': attention_mask, 'token_type_ids': token_type_ids}
outputs = sess.run(None, inputs)
`

error:

Traceback (most recent call last):
  File "test.py", line 29, in <module>
    outputs = sess.run(None, inputs)
  File "/usr/local/lib/python3.6/dist-packages/onnxruntime/capi/session.py", line 142, in run
    return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Non-zero status code returned while running Gather node. Name:'' Status Message: indices element out of data bounds, idx=1 must be within the inclusive range [-1,0]


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
why take the first hidden state for sequence classification (DistilBertForSequenceClassification)
Author: junhuang-ifast
Pull request: no
Labels: name: wontfix

name: Core: Modeling

name: Usage

In the last few layers of sequence classification [here][1], the first hidden state of the sequence length of the transformer output to be used for classification. 

    hidden_state = distilbert_output[0]  # (bs, seq_len, dim) <-- transformer output
    pooled_output = hidden_state[:, 0]  # (bs, dim)           <-- first hidden state
    pooled_output = self.pre_classifier(pooled_output)  # (bs, dim)
    pooled_output = nn.ReLU()(pooled_output)  # (bs, dim)
    pooled_output = self.dropout(pooled_output)  # (bs, dim)
    logits = self.classifier(pooled_output)  # (bs, dim)

Is there any benefit to taking the first hidden state over the last, average, or even the use of a Flatten layer instead?

I've also asked this question on [Stack Overflow](https://stackoverflow.com/questions/60087613/why-take-the-first-hidden-state-for-sequence-classification-distilbertforsequen)


  [1]: https://github.com/huggingface/transformers/blob/33d3072e1c54bcd235447b98c6dea1b4cb71234c/src/transformers/modeling_distilbert.py#L634

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Multi-text files support for run_lm_finetuning
Author: agemagician
Pull request: no
Labels: name: wontfix

name: Ex: LM (Finetuning)

# 🚀 Feature request

Support multi-text files for run_lm_finetuning.

## Motivation

Currently, you support training from scratch but it only supports a single file. Usually, when we train from scratch we train a model using multi-text files, not a single text file.
It will be great to support multi-text file and maybe separate finetuning from training from scratch files.



Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
I can not import transformers
Author: over-shine
Pull request: no

# 🐛 Bug

## Information
when I execute "from transformers import TFBertModel, BertModel" in ipython, the error, "ImportError: cannot import name 'BartConfig' from 'transformers.configuration_auto'" was raised.
This error occured after the tensorflow update from version 2.0 to version 2.1 and the python update from version 3.6 to 3.7.

In addition, after I update the python from version 3.6 to 3.7, I installed torch1.4 and tensorflow2.1 in a same env, the "import" is always failed but when I "import transformers" in other env which only include torch1.4 and python3.7, it succeed
I want to know how to make it. Thank you.

- `transformers` version:2.5.1
- Platform: windows10
- Python version: 3.7.6
- PyTorch version (GPU?):GPU1.4
- Tensorflow version (GPU?):GPU2.1

Comments:
I have solved it by rebooting my computer and reinstall some modules that transformers can not found but have been installed

This issue still exists. I have Tensorflow 2.1 with GPU and transformers version 2.8.0

I tried with rebooting and creating new conda environments multiple times but still no success :(

I can import it and my python version is 3.7.6 , 2.8.0 for transformers and 2.1 for tf. Maybe you could try to upgrade your python if its' version is under 3.7.

> This issue still exists. I have Tensorflow 2.1 with GPU and transformers version 2.8.0
> 
> I tried with rebooting and creating new conda environments multiple times but still no success :(

I can import it and my python version is 3.7.6 , 2.8.0 for transformers and 2.1 for tf. Maybe you could try to upgrade your python if its' version is under 3.7.

Hmm, the next day I tried it again with the same environment and it works xD Probably because of rebooting or something. But I also rebooted the first day, idk...
Why force tokens in Bart decoding
Author: chqiwang
Pull request: no

https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bart.py#L955

What's the meaning of this line? Why decode \<S\>  when cur_len = 1? Why not when cur_len = 0?

Comments:
see https://github.com/huggingface/transformers/issues/3668

> see #3668

thanks a lot!
ImportError: cannot import name 'TF_MODEL_FOR_TOKEN_CLASSIFICATION_MAPPING'
Author: pascalhuszar
Pull request: no

Wanted to use the ner demo, but after running:

 `!python3 ./ner/run_tf_ner.py --data_dir /data --model_type bert --labels .data/labels.txt --model_name_or_path bert-base-multilingual-cased --output_dir germeval-model --max_seq_length 128 --num_train_epochs 3 --per_device_train_batch_size 32 --save_steps 750 --seed 1 --do_train --do_eval --do_predict` 

i encounter the ImportError

Any suggestions? 

Comments:
Importing `TF_MODEL_FOR_TOKEN_CLASSIFICATION_MAPPING` from `transformers` works for me. Can you update your transformers library and see whether you still get the import error.

Hi,

I have the same issue here. I have installed transformers from source with the latest update.
 But still this **ImportError: cannot import name 'TF_MODEL_FOR_TOKEN_CLASSIFICATION_MAPPING'**

Somehow, on a second try `pip install transformers` worked for me. But cant tell why it didnt worked at the beginning.


Thanks for the quick response. I will try again by installing it not from source.
...
Nope, still not working.


Make sure you really update and install from source. If in doubt, recreate your virtual env.

`pip uninstall transformers ` and `pip install transformers ` make sure your transformers version is update.

Thanks, I got around the problem by converting it to pytorch model.  But now I am facing other issues !  Anyway thank you again for your helps
distilberttokenizer.encode_plus() token_type_ids are non-default
Author: cronoik
Pull request: yes

DistilBert doesn't use token_type_ids. Therefore the encode_plus() method of the DistilBertTokenizer should generate them per default. This fix sets the default value of return_token_type_ids to False.

Closes #2702

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2870?src=pr&el=h1) Report
> Merging [#2870](https://codecov.io/gh/huggingface/transformers/pull/2870?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/73028c5df0c28ca179fbe565482a9c2143787f61?src=pr&el=desc) will **increase** coverage by `<.01%`.
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2870/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2870?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #2870      +/-   ##
==========================================
+ Coverage   75.06%   75.06%   +<.01%     
==========================================
  Files          94       94              
  Lines       15288    15290       +2     
==========================================
+ Hits        11476    11478       +2     
  Misses       3812     3812
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/2870?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/2870/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `86.1% <ø> (ø)` | :arrow_up: |
| [src/transformers/tokenization\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/2870/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fZGlzdGlsYmVydC5weQ==) | `100% <100%> (ø)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2870?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2870?src=pr&el=footer). Last update [73028c5...db509eb](https://codecov.io/gh/huggingface/transformers/pull/2870?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


obsolet since 2.6 release