noamaanMulla-03/transformers-issues huggingface dataset:
GPT-2 models are unpickable
Author: Lawiss
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): GPT-2

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [ ] the official example scripts: (give details below)
* [x] my own modified scripts: 

Hi,

I'm trying to train a GPT-2 Double Heads Model (based on your transfer-learning-conv-ai guide) using Pytorch Lightning. However I have a problem when trying to train the model on ddp distributed backend : the `GPT2DoubleHeadsModel` class seems to be unpickable and my training script fails with the following error : 
`TypeError: can't pickle torch._C.ScriptFunction objects`



## To reproduce

Run :
```
from transformers import GPT2DoubleHeadsModel
import pickle
model = GPT2DoubleHeadsModel.from_pretrained("gpt2-medium")
pickle.dump(model,open("test.bin","wb"))
````
The problem does not occur when using `bert-base-uncased` for example. I tried to search which part of GPT-2 class contains ` torch._C.ScriptFunction objects` without success. Do you have an idea to avoid this error ?

Thanks in advance.

- `transformers` version: 2.8
- Platform: Ubuntu 18.04
- Python version: 3.7
- PyTorch version (GPU?): 1.5 Cuda 10.2
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: Yes, ddp


Comments:
Hi! I think this may come from the [activation functions](https://github.com/huggingface/transformers/blob/master/src/transformers/activations.py), some of which are jitted. This allows better performance.

In what situation would you use `pickle` rather than our API `save_pretrained` ?

```py
from transformers import GPT2DoubleHeadsModel
import pickle
model = GPT2DoubleHeadsModel.from_pretrained("gpt2-medium")
model.save_pretrained("test")
```

Hi Lysandre,

Thanks for your answer !
I use [Pytorch Lightning](https://github.com/PyTorchLightning/pytorch-lightning) which requires the model to be picklable (https://pytorch-lightning.readthedocs.io/en/stable/multi_gpu.html#make-model-picklable). I see from your link that if the PyTorch version is <1.4, the activation function is not jitted : 
https://github.com/huggingface/transformers/blob/fa49b9afeab5545f14b3661b35195b829fcf8ef5/src/transformers/activations.py#L32-L44

I think i'll test again downgrading my PyTorch version to 1.3.1.

I understand, this is indeed an issue. We've had other several issues due to this, I think it would be better to revert to picklable option. Will open a PR with this objective in a bit.

Thank you for your responsiveness !

It should be fixed on master now. Could you let me know if running the code from the master branch solves your issue?

Yes, it works, thank you 👌

I am getting the same error trying to save AlbertForTokenClassification() model using `torch.save(model, 'temp.pt')`
But `torch.save(model.state_dict(), 'temp.pt')` works.

@paramansh  Do you mind opening an issue with your specific problem, your software versions and code sample so that we may debug on our side? Thanks.
Getting the output of the from the forward function of the GPT-2
Author: h56cho
Pull request: no
Labels: name: Help wanted

name: wontfix

name: Core: Modeling

Hello,

Is there any way that I can extract the output of the ```self.merge_heads(a)``` and the ```self.c_proj(a)``` from the forward function for the Hugging Face GPT-2, which are found[ here](https://github.com/huggingface/transformers/blob/73028c5df0c28ca179fbe565482a9c2143787f61/src/transformers/modeling_gpt2.py#L192) and [here](https://github.com/huggingface/transformers/blob/73028c5df0c28ca179fbe565482a9c2143787f61/src/transformers/modeling_gpt2.py#L193)?

Thank you,

Comments:
There isn't a way to retrieve that from the current API, but feel free to clone the repository and modify it to fit your needs.

Hello,

Thank you for your reply. Can I make a request to the Hugging Face so that the code can be modified for users to extract ```self.merge_heads(a)``` and ```self.c_proj(a)```? If yes, how can I make the request?

Thank you,

> Hello,
> 
> Thank you for your reply. Can I make a request to the Hugging Face so that the code can be modified for users to extract `self.merge_heads(a)` and `self.c_proj(a)`? If yes, how can I make the request?
> 
> Thank you,

You can suggest feature requests here, as a feature (so here in this topic), but chances are very slim that it will be picked up. It is not high in priority, I would think. @LysandreJik suggests that you clone the repository and implement it yourself. If you successfully do so, you can do a pull request so that your code becomes part of this repository!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
unk_token not set when loading TransformerXLTokenizer.from_pretrained() from a save_pretrained() 
Author: mfuntowicz
Pull request: no
Labels: name: wontfix

name: Core: Tokenization

Using `TransfomerXLTokenizer.from_pretrained()` when loading from `.save_pretrained()` generated files.

Python complains about missing <unknown> token: 

```
line 175, in _build_from_file
    raise ValueError("No <unkown> token in vocabulary")
ValueError: No <unkown> token in vocabulary
```

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Support for torch-lightning in NER examples
Author: srush
Pull request: yes

Update of https://github.com/huggingface/transformers/pull/2816

This PR creates a new example coding style for the pytorch code.

* Uses pytorch-lightning for the underlying training.
* Separates out the base transformer loading from the individual training.
* Moves each individual example to its own directory.
* Move the code in the readme to bash scripts.
The only two new files are run_pl_ner.py and transformers_base.py.

The goal is to keep the same format as the original command-line. Most of the argument names are preserved. I have verified that for NER the results of the same on GPU.

There are several nice benefits of lightning -> somewhat nicer logging and library integration (e.g. wandb), auto-checkpointing. Mostly the goal though is code readability with identical functionality.

Tests I ran:

* make sure that the test results are identical.
* print test results after training.
* test multi-gpu and apex (multigpu gives a nice speedup)

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2890?src=pr&el=h1) Report
> Merging [#2890](https://codecov.io/gh/huggingface/transformers/pull/2890?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/0dbddba6d2c5b2c6fc08866358c1994a00d6a1ff?src=pr&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/2890/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/2890?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #2890   +/-   ##
=======================================
  Coverage   75.06%   75.06%           
=======================================
  Files          94       94           
  Lines       15288    15288           
=======================================
  Hits        11476    11476           
  Misses       3812     3812
```



------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/2890?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/2890?src=pr&el=footer). Last update [0dbddba...8f8137f](https://codecov.io/gh/huggingface/transformers/pull/2890?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


@LysandreJik 
This implementation does not work with `pytorch_lightning` > `0.7.1`.

It throws the exception `'Trainer' object has no attribute 'avg_loss'` because since version `0.7.2` they removed the `avg_loss` field from the `Trainer` class.

See https://github.com/huggingface/transformers/pull/2890/files#diff-d68a6ecfacd8231c59af0ea67d77bb9cR120

@simonepri can you file an issue? I guess we should just remove that key/value pair. 

@simonepri did you try 0.7.5?
NER: parse args from .args file or JSON
Author: stefan-it
Pull request: yes

Hi,

thanks to @julien-c , args parsing from `.args` or json-based configuration files were introduced in #3934 into the internal argparser class.

This PR adds support for it in the `run_ner.py` script.

It also extends the NER documentation and shows how to use a json-based configuration with `run_ner.py`.

Comments:
Albert large QA model pretrained from baidu webqa and baidu dureader datasets.
Author: wptoux
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4112?src=pr&el=h1) Report
> Merging [#4112](https://codecov.io/gh/huggingface/transformers/pull/4112?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/d713cfc5ebfb1ed83de1fce55dd7279f9db30672&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4112/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4112?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4112      +/-   ##
==========================================
- Coverage   78.84%   78.84%   -0.01%     
==========================================
  Files         114      114              
  Lines       18688    18688              
==========================================
- Hits        14735    14734       -1     
- Misses       3953     3954       +1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4112?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4112/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.14% <0.00%> (-0.42%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4112?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4112?src=pr&el=footer). Last update [d713cfc...a442791](https://codecov.io/gh/huggingface/transformers/pull/4112?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Really cool. [Model page](https://huggingface.co/wptoux/albert-chinese-large-qa)
model card for surajp/albert-base-sanskrit
Author: parmarsuraj99
Pull request: yes
Labels: name: model card

Added Model card for `surajp/albert-base-sanskrit`

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4114?src=pr&el=h1) Report
> Merging [#4114](https://codecov.io/gh/huggingface/transformers/pull/4114?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/abb1fa3f374811ea09d0bc3440d820c50735008d&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4114/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4114?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4114   +/-   ##
=======================================
  Coverage   78.85%   78.85%           
=======================================
  Files         114      114           
  Lines       18688    18688           
=======================================
  Hits        14736    14736           
  Misses       3952     3952           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4114?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4114/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.14% <0.00%> (-0.42%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4114/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.04% <0.00%> (+0.12%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4114?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4114?src=pr&el=footer). Last update [abb1fa3...e50935f](https://codecov.io/gh/huggingface/transformers/pull/4114?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Awesome! And our first model for Sanskrit. 

[model page](https://huggingface.co/surajp/albert-base-sanskrit)
pipeline("sentiment-analysis")() can't handle more than 2 sentences
Author: xxbidiao
Pull request: no
Labels: name: Core: Pipeline

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...): pipeline("sentiment-analysis")

Language I am using the model on (English, Chinese ...): English

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [x] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

```
>>> from transformers import pipeline
>>> analyzer = pipeline('sentiment-analysis')
Downloading: 100%|██████████████████████████████| 230/230 [00:00<00:00, 146kB/s]
>>> analyzer(["OK"]*10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../lib/python3.6/site-packages/transformers/pipelines.py", line 490, in __call__
    scores = np.exp(outputs) / np.exp(outputs).sum(-1)
ValueError: operands could not be broadcast together with shapes (10,2) (10,) 
>>> 
```


`

<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->
Getting 10 results

## Environment info
<!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version: 2.5.0
- Platform: ubuntu 19.04
- Python version: 3.6
- PyTorch version (GPU?): 1.4.0 GPU
- Tensorflow version (GPU?): 1.14.0 GPU
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No


Comments:
`scores = np.exp(outputs) / np.exp(outputs).sum(-1).reshape(-1,1)` works for me, but I'm not sure whether it breaks other things.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


This still happens on the latest version. I still have to apply
`scores = np.exp(outputs) / np.exp(outputs).sum(-1).reshape(-1,1)`
For the code to work.

Yes, this is in the process of being solved by @mfuntowicz
Fix #2941
Author: xxbidiao
Pull request: yes

Reshaped score array to avoid `numpy` ValueError. This should allow the sentiment analyzer pipeline to run.

Comments:
Uh, I guess it should be `reshape(-1, 1)` instead of `reshape(-1,1)` regarding code quality issues, but I'm not sure whether it's the correct fix.

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4109?src=pr&el=h1) Report
> Merging [#4109](https://codecov.io/gh/huggingface/transformers/pull/4109?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/d713cfc5ebfb1ed83de1fce55dd7279f9db30672&el=desc) will **not change** coverage.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4109/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4109?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4109   +/-   ##
=======================================
  Coverage   78.84%   78.84%           
=======================================
  Files         114      114           
  Lines       18688    18688           
=======================================
  Hits        14735    14735           
  Misses       3953     3953           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4109?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/pipelines.py](https://codecov.io/gh/huggingface/transformers/pull/4109/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9waXBlbGluZXMucHk=) | `74.94% <100.00%> (ø)` | |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4109/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.14% <0.00%> (-0.42%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4109/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `92.77% <0.00%> (+0.16%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4109?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4109?src=pr&el=footer). Last update [d713cfc...e12b3d5](https://codecov.io/gh/huggingface/transformers/pull/4109?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
adding support for commonsense qa for multiple choice question answering
Author: nrjvarshney
Pull request: yes
Labels: name: wontfix


Comments:
Can somebody help me in fixing the failed test?

Hi, thanks for your addition, that's really cool! Did you check the [contributions guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md#start-contributing-pull-requests)?

You should make sure you have the correct versions of `black`, `isort` and `flake8` installed and then run `make style`/`make quality` to identify the code quality issues.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Make format consistent with that of PreTrainedTokenizer
Author: ranamihir
Pull request: yes
Labels: name: wontfix


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3022?src=pr&el=h1) Report
> :exclamation: No coverage uploaded for pull request base (`master@c913eb9`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit).
> The diff coverage is `100%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3022/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/3022?src=pr&el=tree)

```diff
@@            Coverage Diff            @@
##             master    #3022   +/-   ##
=========================================
  Coverage          ?   77.26%           
=========================================
  Files             ?       98           
  Lines             ?    16040           
  Branches          ?        0           
=========================================
  Hits              ?    12393           
  Misses            ?     3647           
  Partials          ?        0
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3022?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3022/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdXRpbHMucHk=) | `91.57% <100%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3022?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3022?src=pr&el=footer). Last update [c913eb9...34be93e](https://codecov.io/gh/huggingface/transformers/pull/3022?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Use my own pretrained BERT model
Author: rabbitwayne
Pull request: no
Labels: name: wontfix

name: Usage

How can I use my own pretrained BERT model in SQUAD finetuing? How to do this in the code? Can anyone provide any instructions?

Comments:
Hi, where did you obtain this model? Is it using the official google-research implementation or using huggingface/transformers?

@LysandreJik Thank you for your reply! I have pretrained BERT base using my own pytorch code and generated checkpoint for BERT base. Then I want to do squad finetuning using my own pretrained BERT base model but in huggingface/transformers framework. I saw that huggingface/transformers by default download a pretrained BERT base model from amazon aws. How can I change this to use my own checkpoint? Thanks a lot!

Are you able to load your model using `model = BertModel.from_pretrained("model_dir")`? The architecture of the model would need to be the same, and have the same layer names so that the torch state dict may be loaded onto one of our architectures.

Thank you! Let me try that.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
No decoder specific error message for T5Stack.forward
Author: enzoampil
Pull request: no

When I run a forward pass with `T5Model` with a missing value for `decoder_input_ids`, it still gives me the same error message compared to when there is a missing value for `input_ids` (below).

`ValueError: You have to specify either input_ids or inputs_embeds`

I've confirmed with @patrickvonplaten that it makes sense to have a specific error message for the case when `self.is_decoder == True` in issue #3626.

Comments: