noamaanMulla-03/transformers-issues huggingface dataset:
MBartTokenizer:add language codes
Author: sshleifer
Pull request: yes

The mbart tokenizer is meant to 
- not add bos token at the beginning
- end `input_ids` with [eos, src_lang_code]
- end `decoder_input_ids` with [eos, tgt_lang_code]

I have posted a fairseq issue to confirm this, but all that will change is the ordering of special tokens.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3776?src=pr&el=h1) Report
> Merging [#3776](https://codecov.io/gh/huggingface/transformers/pull/3776?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/6e603cb7892b49a2cbbc10ba859759f92c3fb7a6&el=desc) will **increase** coverage by `0.00%`.
> The diff coverage is `90.90%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/3776/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/3776?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #3776   +/-   ##
=======================================
  Coverage   77.00%   77.00%           
=======================================
  Files         128      128           
  Lines       21602    21624   +22     
=======================================
+ Hits        16634    16652   +18     
- Misses       4968     4972    +4     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/3776?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/tokenization\_bart.py](https://codecov.io/gh/huggingface/transformers/pull/3776/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90b2tlbml6YXRpb25fYmFydC5weQ==) | `94.73% <90.90%> (-5.27%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3776/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.26% <0.00%> (-0.16%)` | :arrow_down: |
| [src/transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/3776/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ191dGlscy5weQ==) | `90.49% <0.00%> (-0.12%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/3776?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/3776?src=pr&el=footer). Last update [6e603cb...acbdaf3](https://codecov.io/gh/huggingface/transformers/pull/3776?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[Model card] model card for electra-base QA model
Author: patil-suraj
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4936?src=pr&el=h1) Report
> Merging [#4936](https://codecov.io/gh/huggingface/transformers/pull/4936?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/699541c4b34479451c91b3c6c204d904f62bed83&el=desc) will **increase** coverage by `0.40%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4936/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4936?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4936      +/-   ##
==========================================
+ Coverage   76.77%   77.17%   +0.40%     
==========================================
  Files         128      128              
  Lines       21723    21723              
==========================================
+ Hits        16677    16764      +87     
+ Misses       5046     4959      -87     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4936?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4936/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.79% <0.00%> (+0.40%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4936/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.26% <0.00%> (+0.77%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_xlm.py](https://codecov.io/gh/huggingface/transformers/pull/4936/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl94bG0ucHk=) | `75.85% <0.00%> (+19.75%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4936?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4936?src=pr&el=footer). Last update [699541c...4301a6e](https://codecov.io/gh/huggingface/transformers/pull/4936?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Is it feasible to set num_workers>=1 in DataLoader to quickly load data?
Author: lixinsu
Pull request: no


Comments:
Are you asking if it possible or do you want this change included to the code?

I don't see why this change would cause a problem, if we choose to implement it we should add a command line argument to specify this value.

Yes, feel free to submit a PR if you have a working implementation.

@thomwolf @rodgzilla Is it still the case with the new Trainer?
[Benchmark] fix indentation error
Author: patrickvonplaten
Pull request: yes

Because of a wrong indentation, the memory usage was calculated incorrectly

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4941?src=pr&el=h1) Report
> Merging [#4941](https://codecov.io/gh/huggingface/transformers/pull/4941?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/699541c4b34479451c91b3c6c204d904f62bed83&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `0.00%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4941/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4941?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4941      +/-   ##
==========================================
- Coverage   77.17%   77.17%   -0.01%     
==========================================
  Files         128      128              
  Lines       21723    21722       -1     
==========================================
- Hits        16764    16763       -1     
  Misses       4959     4959              
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4941?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/benchmark/benchmark.py](https://codecov.io/gh/huggingface/transformers/pull/4941/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9iZW5jaG1hcmsvYmVuY2htYXJrLnB5) | `69.42% <0.00%> (+0.56%)` | :arrow_up: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4941/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.26% <0.00%> (-0.16%)` | :arrow_down: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4941?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4941?src=pr&el=footer). Last update [699541c...7f4d433](https://codecov.io/gh/huggingface/transformers/pull/4941?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
[mbart] Fix fp16 testing logic
Author: sshleifer
Pull request: yes

the expected logits must be in the same dtype as the resulting logits.

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4949?src=pr&el=h1) Report
> Merging [#4949](https://codecov.io/gh/huggingface/transformers/pull/4949?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/473808da0d476792070f0e7dfebcf1121a12a34f&el=desc) will **not change** coverage.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4949/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4949?src=pr&el=tree)

```diff
@@           Coverage Diff           @@
##           master    #4949   +/-   ##
=======================================
  Coverage   77.14%   77.14%           
=======================================
  Files         128      128           
  Lines       21745    21745           
=======================================
  Hits        16775    16775           
  Misses       4970     4970           
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4949?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4949/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.26% <0.00%> (-0.16%)` | :arrow_down: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4949/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `73.79% <0.00%> (+0.40%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4949?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4949?src=pr&el=footer). Last update [473808d...a9ebdd5](https://codecov.io/gh/huggingface/transformers/pull/4949?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Custom collate function that pads only to the longest sequence?
Author: ZhaofengWu
Pull request: no
Labels: name: wontfix

Currenty all input is padded to `max_seq_length` but in most cases the longest sequence in a batch is shorter than that, sometimes by a significant amount. If there is a custom collate function that pads only to the longest sequence in a batch, that will probably save quite some memory and time. Is that feasible?

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Reinitializing layers in BERT
Author: paul-you
Pull request: no
Labels: name: wontfix

Hello, 

I have a question regarding re-initialising the encoder layers in BERT. What happens if I call the __init__() method of a BERT layer, is the the layer re-initialised using the pre-trained BERT weights or does it get completely new weights ? 

`model.bert.encoder.layer[0].__init__(config)`

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
How do I fine-tune hyperparameters for a model from Huggingface library
Author: thak123
Pull request: no

# ❓ Questions & Help


## Details
Hi, I am new to Hugging-face library and want to fine tune hyper-parameters of the mBert model. I have a simple classification head on top the cls token. I am getting around 76% accuracy. 
Is there any read-me or notebook available for doing the same.



Comments:
You can take a look at this blog post https://mccormickml.com/2019/07/22/BERT-fine-tuning/

Thank You
Create README.md
Author: ypapanik
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4872?src=pr&el=h1) Report
> Merging [#4872](https://codecov.io/gh/huggingface/transformers/pull/4872?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/9f5d5a531d769d07403f59661884e254f8420afe&el=desc) will **increase** coverage by `0.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4872/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4872?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4872      +/-   ##
==========================================
+ Coverage   76.55%   76.56%   +0.01%     
==========================================
  Files         128      128              
  Lines       21502    21502              
==========================================
+ Hits        16461    16464       +3     
+ Misses       5041     5038       -3     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4872?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4872/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.28% <0.00%> (-0.32%)` | :arrow_down: |
| [src/transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4872/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9maWxlX3V0aWxzLnB5) | `74.27% <0.00%> (+0.41%)` | :arrow_up: |
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/4872/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `39.62% <0.00%> (+0.94%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4872?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4872?src=pr&el=footer). Last update [9f5d5a5...aad9cb1](https://codecov.io/gh/huggingface/transformers/pull/4872?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
Create README.md
Author: mrm8488
Pull request: yes
Labels: name: model card


Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4865?src=pr&el=h1) Report
> Merging [#4865](https://codecov.io/gh/huggingface/transformers/pull/4865?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/9f5d5a531d769d07403f59661884e254f8420afe&el=desc) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/4865/graphs/tree.svg?width=650&height=150&src=pr&token=9qOlN6Hb1c)](https://codecov.io/gh/huggingface/transformers/pull/4865?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #4865      +/-   ##
==========================================
- Coverage   76.55%   76.54%   -0.01%     
==========================================
  Files         128      128              
  Lines       21502    21502              
==========================================
- Hits        16461    16459       -2     
- Misses       5041     5043       +2     
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/4865?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [src/transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/4865/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ19vcGVuYWkucHk=) | `79.58% <0.00%> (-1.39%)` | :arrow_down: |
| [src/transformers/modeling\_tf\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/4865/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy9tb2RlbGluZ190Zl91dGlscy5weQ==) | `87.28% <0.00%> (-0.32%)` | :arrow_down: |
| [src/transformers/trainer.py](https://codecov.io/gh/huggingface/transformers/pull/4865/diff?src=pr&el=tree#diff-c3JjL3RyYW5zZm9ybWVycy90cmFpbmVyLnB5) | `39.62% <0.00%> (+0.94%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/4865?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/4865?src=pr&el=footer). Last update [9f5d5a5...c1b9024](https://codecov.io/gh/huggingface/transformers/pull/4865?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
How can I judge whether is in the dictionary?
Author: smelly-dog
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
Will different word get the same ids in tokenizer?
Cause I just meet this situation.
And it looks like this
candidate is ['charge', 'greet', 'treat', 'reward']
 and candidate_ids is [10813, 1, 13581, 1]
<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: 

Comments:
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Allow token regression in ForTokenClassification models
Author: gsarti
Pull request: no
Labels: name: wontfix

# 🚀 Feature request

Current `ForTokenClassification` class implementation for all models does not support regression. I propose to adapt the current implementation for all the `ForTokenClassification` models in order to enable out-of-the-box token regression when `num_labels == 1`, similarly to what is currently available in the `ForSentenceClassification` models.

Concretely, this would mean converting this (taken from `AlbertForTokenRegression` as an example, line 873 in `modeling_albert.py`):

```python
if labels is not None:
    loss_fct = CrossEntropyLoss()
    # Only keep active parts of the loss
    if attention_mask is not None:
        active_loss = attention_mask.view(-1) == 1
        active_logits = logits.view(-1, self.num_labels)[active_loss]
        active_labels = labels.view(-1)[active_loss]
        loss = loss_fct(active_logits, active_labels)
    else:
        loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
    outputs = (loss,) + outputs
return outputs  # (loss), logits, (hidden_states), (attentions)
```

into something like this:

```python
if labels is not None:
    if self.num_labels == 1:
        # We are doing regression
        loss_fct = MSELoss()
        logits_view = logits.view(-1)
    else:
        # We are doing classification
        loss_fct = CrossEntropyLoss()
        logits_view = logits.view(-1, self.num_labels)
    # Only keep active parts of the loss
    if attention_mask is not None:
        active_loss = attention_mask.view(-1) == 1
        active_logits = logits_view[active_loss]
	active_labels = labels.view(-1)[active_loss]
	loss = loss_fct(active_logits, active_labels)
    else:
	loss = loss_fct(logits_view, labels.view(-1))
    outputs = (loss,) + outputs

return outputs  # (loss), logits, (hidden_states), (attentions)
```

## Motivation

I am currently working with token-level regression using multiple transformer models to predict eye-tracking metrics that are commonly considered as proxies for cognitive processing in psycholinguistics (e.g. word reading times, fixation counts, etc.). Given that most of those are continuous metrics, the ability to use `transformers` for token regression would make my work much faster. Moreover, I believe that this functionality can benefit other researchers working with token-level continuous metrics.

## Your contribution

If this feature is regarded as interesting by maintainers, I can submit a PR with the suggested changes applied to all currently supported models.


Comments:
I think that's a reasonable feature – Thoughts @LysandreJik @thomwolf?

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
Data used for training MarianMT models
Author: yashkant
Pull request: no

# ❓ Questions & Help
Are there specifications available on the amount and type of training data used for the released MarianMT [models](https://huggingface.co/transformers/model_doc/marian.html)?

## Details
<!-- Description of your issue -->
I want to use these models for back translation in order to augment my current training data (the [VQA dataset](http://visualqa.org/) which is in English). I understand that for this, I could use any number of models in conjunction with source and target languages as `en`. 

However, I am concerned about the domain mismatch of the VQA dataset vs training data of the MT model. For example, using `en-ROMANCE` and `ROMANCE-en` models, I find that back-translation of the input sentence: 

`['>>lmo<< What do you think are the children playing with?']` is 

`['What do you think of the youth of the jubilee?']` 

Also, do you have any suggestions/guidelines on which/how many models to use particularly for this use case? I did not find this question suitable for SO hence it posted here directly

Thank you for releasing the models! 


Comments:
Hi, @yashkant, you might be able to find these details here https://github.com/Helsinki-NLP/Opus-MT
How to feeding hidden state vectors from one transformer directly into a layer of different transformer
Author: h56cho
Pull request: no

Hello,

I want to perform the following task:

**1.** _Feed in a text sequence to a GPT2Model (the GPT-2 model without output head);_

**2.** _Extract the hidden state vectors of the text sequence that are generated at each layer of the GPT2Model;_

**3.** _Feed the hidden state vectors from each layer of the GPT2Model as an input to the GPT2DoubleHeadsModel (assuming that the n_embd of the GPT2Model and the GPT2DoubleHeadsModel are equal), and calculate the **mc_loss** that results from inputting the hidden state vectors. My GPT2DoubleHeadsModel would only be consisted of (1 layer) + (the output heads)._


I know that the code below can be used to feed in hidden state vectors as inputs to each individual layer of a HuggingFace GPT-2 model:

```python
model.transformer.h[i](embed)
```
But the code above will only give me the output of an individual layer, so it wouldn't give me the mc_loss that I am looking for (since to calculating the mc_loss from the hidden state vectors, I would first need to feed in the hidden state vectors into the 1st layer of the GPT2DoubleHeadsModel, and somehow feed in the output of the layer as an input to the mc_head (multiple choice output head) of the GPT2DoubleHeadsModel) 

My questions are:
1. Is there any way to perform the task 3 with the Hugging Face GPT-2 models?
2. Is there anyway that I can access the GPT2DoubleHeadsModel's mc_head (multiple choice head), similar to the code that I wrote above?

Thank you,

Comments:
Did you ever find the answer?