noamaanMulla-03/transformers-issues huggingface dataset:
Fix deprecation warnings due to invalid escape sequences.
Author: tirkarthi
Pull request: yes

Fixes #3754 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4924?src=pr&el=h1) Report
> Merging [#4924](https://codecov.io/gh/huggingface/transformers/pull/4924?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e80d6c689bd62f805a5c8d77ec0cc3b09f240d14&el=desc) will **increase** coverage by `0.07%`.
> The diff coverage is `100.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4924/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4924?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4924      +/-   ##
==========================================
+ Coverage   77.10%   77.17%   +0.07%     
==========================================
  Files         128      128              
  Lines       21723    21723              
==========================================
+ Hits        16749    16765      +16     
+ Misses       4974     4958      -16     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4924?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/4924/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fdHJhbnNmb194bC5weQ==) | `40.67% <100.00%> (ø)` | |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4924/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.79% <0.00%> (ø)` | |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4924/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.42% <0.00%> (+0.15%)` | :arrow_up: |
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/4924/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `39.48% <0.00%> (+0.23%)` | :arrow_up: |
| [src/transformers/modeling\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/4924/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ194bG5ldC5weQ==) | `78.65% <0.00%> (+2.29%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4924?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4924?src=pr&el=footer). Last update [e80d6c6...51acaab](https://codecov.io/gh/huggingface/transformers/pull/4924?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Deprecation warning due to invalid escape sequences in Python 3.7
Author: tirkarthi
Pull request: no

# 🐛 Bug

## To reproduce

Deprecation warnings are raised due to invalid escape sequences. This can be fixed by using raw strings or escaping the literals.

Steps to reproduce the behavior:

```
 find . -iname '*.py' | grep -v example | xargs -P 4 -I{} python3.8 -Wall -m py_compile {} 
./src/transformers/tokenization_transfo_xl.py:123: DeprecationWarning: invalid escape sequence \:
  self.punctuation_symbols = '!"#$%&()*+,-./\:;<=>?@[\\]^_`{|}~'  # noqa: W605
./src/transformers/tokenization_transfo_xl.py:150: DeprecationWarning: invalid escape sequence \s
  look_ahead_to_match_all_except_space = "(?=[^\s])"  # noqa: W605
```

## Expected behavior

No warnings

## Environment info
     
- `transformers` version: master branch
- Platform: linux
- Python version: 3.7
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


The bug is still valid and I have raised https://github.com/huggingface/transformers/pull/4924
[ElectraForQuestionAnswering] fix qa example in doc
Author: patil-suraj
Pull request: yes

This PR fixes the QA example in the doc for `ElectraForQuestionAnswering`. In the last example `token_type_ids` were not used, this PR fixes that

@sgugger 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4929?src=pr&el=h1) Report
> Merging [#4929](https://codecov.io/gh/huggingface/transformers/pull/4929?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e80d6c689bd62f805a5c8d77ec0cc3b09f240d14&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4929/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4929?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4929   +/-   ##
=======================================
  Coverage   77.10%   77.11%           
=======================================
  Files         128      128           
  Lines       21723    21723           
=======================================
+ Hits        16749    16751    +2     
+ Misses       4974     4972    -2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4929?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_electra.py](https://codecov.io/gh/huggingface/transformers/pull/4929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19lbGVjdHJhLnB5) | `78.16% <ø> (ø)` | |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.38% <0.00%> (-0.41%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.61% <0.00%> (+0.11%)` | :arrow_up: |
| [src/transformers/modeling\_transfo\_xl.py](https://codecov.io/gh/huggingface/transformers/pull/4929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190cmFuc2ZvX3hsLnB5) | `78.81% <0.00%> (+0.19%)` | :arrow_up: |
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/4929/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `39.48% <0.00%> (+0.23%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4929?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4929?src=pr&el=footer). Last update [e80d6c6...4a895d8](https://codecov.io/gh/huggingface/transformers/pull/4929?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
name 'ElectraForSequenceClassification' is not defined
Author: Gemini77
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Comments:
Hi @Gemini77 , what is your transformers version ? 

`ElectraForSequenceClassification` is available  in transformers >=2.10.0 version
Issue with an inline code comment
Author: manishiitg
Pull request: no

# 🐛 Bug

## Information

Model I am using (Bert, XLNet ...):

Language I am using the model on (English, Chinese ...):

The problem arises when using:
* [x ] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ x] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

https://github.com/huggingface/transformers/blob/d4c2cb402d6674211726fd5f4803d1090664e438/examples/question-answering/run_squad.py#L322
This specific comment written is wrong.

I wasted lot of time due to this comment. 
i and feature_index are not the same no

:)


<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

<!-- A clear and concise description of what you would expect to happen. -->

## Environment info
<!-- You can run the command `transformers-cli env` and copy-and-paste its output below.
     Don't forget to fill out the missing fields in that output! -->
     
- `transformers` version:
- Platform:
- Python version:
- PyTorch version (GPU?):
- Tensorflow version (GPU?):
- Using GPU in script?:
- Using distributed or parallel set-up in script?:


Comments:
Fixing TPU training by disabling wandb.watch gradients logging
Author: misrasaurabh1
Pull request: yes

Fixes issue https://github.com/huggingface/transformers/issues/4814
PyTorch TPU trainer.py had a bug where the training would freeze up during the logging step. On investigation, the culprit was found to be a wandb.watch call which was trying to log gradients. This operation is suspected to be unsupported by Wandb for TPUs. Waiting for a confirmation of TPU gradient logging support by the Wandb team.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4926?src=pr&el=h1) Report
> Merging [#4926](https://codecov.io/gh/huggingface/transformers/pull/4926?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/e80d6c689bd62f805a5c8d77ec0cc3b09f240d14&el=desc) will **increase** coverage by `0.07%`.
> The diff coverage is `0.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4926/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4926?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4926      +/-   ##
==========================================
+ Coverage   77.10%   77.17%   +0.07%     
==========================================
  Files         128      128              
  Lines       21723    21723              
==========================================
+ Hits        16749    16765      +16     
+ Misses       4974     4958      -16     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4926?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/4926/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `39.48% <0.00%> (+0.23%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4926/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.79% <0.00%> (ø)` | |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4926/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.42% <0.00%> (+0.15%)` | :arrow_up: |
| [src/transformers/modeling\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/4926/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ194bG5ldC5weQ==) | `78.65% <0.00%> (+2.29%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4926?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4926?src=pr&el=footer). Last update [e80d6c6...d857f37](https://codecov.io/gh/huggingface/transformers/pull/4926?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Remove deprecation warning TF2.2
Author: astariul
Pull request: yes

`strategy.experimental_run_v2()` is deprecated in favor of `strategy.run()`
https://github.com/tensorflow/tensorflow/blob/2b96f3662bd776e277f86997659e61046b56c315/tensorflow/python/distribute/distribute_lib.py#L953-L957

Fix #4992 

Comments:
The only CI message is :

```
would reformat /home/circleci/transformers/src/transformers/trainer_tf.py
Oh no! 💥 💔 💥
1 file would be reformatted, 299 files would be left unchanged.
```

How can I see what to change to fit the code style ?

Thanks for the PR, but merging this, will remove the trainer from being compliant with TF <= 2.1. Which is not what we want yet. But we keep it here when we will be ready to do so ;)
Fix TF WarmUp class
Author: astariul
Pull request: yes

This commit fixes the TF WarmUp learning rate scheduler.

The LR shape was wrong due to warmup steps. See linked issue for more details.

Fix #5098

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5099?src=pr&el=h1) Report
> Merging [#5099](https://codecov.io/gh/huggingface/transformers/pull/5099?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/efeb75b8054cc299698cf8bc09f395ada2660745&el=desc) will **increase** coverage by `0.04%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5099/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5099?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #5099      +/-   ##
==========================================
+ Coverage   77.24%   77.29%   +0.04%     
==========================================
  Files         133      133              
  Lines       22134    22134              
==========================================
+ Hits        17097    17108      +11     
+ Misses       5037     5026      -11     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5099?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/optimization\_tf.py](https://codecov.io/gh/huggingface/transformers/pull/5099/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9vcHRpbWl6YXRpb25fdGYucHk=) | `57.27% <ø> (ø)` | |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5099/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `91.06% <0.00%> (+0.12%)` | :arrow_up: |
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/5099/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `39.62% <0.00%> (+0.23%)` | :arrow_up: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5099/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `75.40% <0.00%> (+0.40%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5099/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.26% <0.00%> (+1.24%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5099?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5099?src=pr&el=footer). Last update [efeb75b...d99033f](https://codecov.io/gh/huggingface/transformers/pull/5099?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Thanks @Colanim! Good catch!! Unfortunately it is a duplicate of #4940 :smile: and should be merged soon :)

Haha didn't see x)
[docs] fix T5 training doc
Author: patil-suraj
Pull request: yes

This PR fixes T5 training doc. In a recent commit `lm_labels` is changed to `labels`. Made the doc changes accordingly. Regarding issue #5079

@patrickvonplaten 

Comments:
Great thanks @patil-suraj 

# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5080?src=pr&el=h1) Report
> Merging [#5080](https://codecov.io/gh/huggingface/transformers/pull/5080?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/ebab096e864a619717a497089d864d10e21bc536&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/5080/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/5080?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #5080   +/-   ##
=======================================
  Coverage   77.26%   77.27%           
=======================================
  Files         128      128           
  Lines       21854    21854           
=======================================
+ Hits        16886    16887    +1     
+ Misses       4968     4967    -1     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/5080?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5080/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.38% <0.00%> (-0.41%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5080/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.61% <0.00%> (+0.11%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/5080/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.42% <0.00%> (+0.15%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/5080?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/5080?src=pr&el=footer). Last update [ebab096...2f8a93c](https://codecov.io/gh/huggingface/transformers/pull/5080?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
What is the different options for pooler_type in Bert config ?
Author: ClementViricel
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I want to change the pooling type at the top of the output hidden states of Bert.
I search in the documentation and find nothing. Can anyone help me ? I just want the different option of pooling (max, average etc.). Here's a piece of code to see the option i am talking about.
`import transformers
encoder = transformers.TFBertModel.from_pretrained("bert-base-uncased")
encoder.config`
<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**:


Comments:
Hi! The pooler is actually a linear layer, that is used on top of the BERT encoder (the hidden layers). It's not doing a pooling operation like average or max pooling.


Ok i read that somewhere. But my is on what component ? If you use a pooler on the last layer of bert encoder you will get an output of size [batch, step, dense_unit], but i am sure that the pooler output is a squeeze_first output. Why is there a information about pooler_type in the config, if we cannot change the pooler_type ?
OpenAIGPTDoubleHeadsModel Don't have the "labels" attributes as it is described in the documentation
Author: ghsama
Pull request: no

# 🐛 Bug

## Information

Model I am using OpenAIGPTDoubleHeadsModel:

Language I am using the model on English:

The problem arises when using:
* The official example scripts: 
I'm using transformer version : '2.11.0'

In the documentation i found that i can pass a "labels" attribute to the OpenAIGPTDoubleHeadsModel Model to be used in the loss:
[Documentation](https://huggingface.co/transformers/model_doc/gpt.html#openaigptdoubleheadsmodel)
But when i pass it he didn't recognize it : 

``model(input_ids=input_ids,` token_type_ids=token_type_ids, position_ids=None, head_mask=None, mc_token_ids = mc_token_ids, labels=lm_labels, mc_labels = mc_labels)``

``TypeError                                 Traceback (most recent call last)
<ipython-input-67-cc8f7864d8a7> in <module>()
----> 1 model(input_ids=input_ids, token_type_ids=token_type_ids, position_ids=None, head_mask=None, mc_token_ids = mc_token_ids, labels=lm_labels, mc_labels = mc_labels)
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    548             result = self._slow_forward(*input, **kwargs)
    549         else:
--> 550             result = self.forward(*input, **kwargs)
    551         for hook in self._forward_hooks.values():
    552             hook_result = hook(self, input, result)
TypeError: forward() got an unexpected keyword argument 'labels' ``

## Question : 

Is passing the labels thorough GPT implimentation is depricated ? if yes is there a way to pass it ?
Thank you 

Comments:
Hi @ghsama `lm_labels` is changed to `labels` after 2.11.0.  So if you are using version <=2.11.0 use `lm_labels`  and if you're using master branch then use `labels`

Yes ! i just figure it out . Thank you @patil-suraj :D
Etract all last hidden states of the input dequences for Question and answering Bert
Author: LincLabUCCS
Pull request: no
Labels: name: wontfix

Hello everyone, 

I am using run_squad.py to finetune Bert for question answering. I would like to save and later extract all last hidden states for all sequences for further use. For example, if I have a dataset of 100 sequences, 64 length, and 768 features. I will eventually have a tensor of (100,64,768).
Is there any possible to do that using the run_squad.py script..

Thank you all

Comments:
Modify the script :)

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.