Need multiple capabilities
Author: SreeramV181
Pull request: yes

Need both Generative Finetuning and Distilling Capabilities

Comments:
# [Codecov](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139?src=pr&el=h1) Report
> Merging [#1139](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139?src=pr&el=desc) into [generative-finetuning](https://codecov.io/gh/huggingface/pytorch-transformers/commit/529a16dec6cc9bfcf8954a1b16546960f2fab6fa?src=pr&el=desc) will **increase** coverage by `0.87%`.
> The diff coverage is `96.42%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139?src=pr&el=tree)

```diff
@@                    Coverage Diff                    @@
##           generative-finetuning    #1139      +/-   ##
=========================================================
+ Coverage                  79.61%   80.48%   +0.87%     
=========================================================
  Files                         42       46       +4     
  Lines                       6918     7411     +493     
=========================================================
+ Hits                        5508     5965     +457     
- Misses                      1410     1446      +36
```


| [Impacted Files](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pytorch\_transformers/modeling\_transfo\_xl.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvbW9kZWxpbmdfdHJhbnNmb194bC5weQ==) | `57.12% <ø> (-0.42%)` | :arrow_down: |
| [pytorch\_transformers/tokenization\_roberta.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvdG9rZW5pemF0aW9uX3JvYmVydGEucHk=) | `95.37% <ø> (-0.93%)` | :arrow_down: |
| [pytorch\_transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvbW9kZWxpbmdfcm9iZXJ0YS5weQ==) | `75.89% <ø> (ø)` | :arrow_up: |
| [pytorch\_transformers/modeling\_utils.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvbW9kZWxpbmdfdXRpbHMucHk=) | `82.14% <ø> (-1.28%)` | :arrow_down: |
| [pytorch\_transformers/modeling\_gpt2.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvbW9kZWxpbmdfZ3B0Mi5weQ==) | `75.84% <ø> (ø)` | :arrow_up: |
| [pytorch\_transformers/modeling\_xlm.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvbW9kZWxpbmdfeGxtLnB5) | `86.66% <ø> (ø)` | :arrow_up: |
| [pytorch\_transformers/tokenization\_distilbert.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvdG9rZW5pemF0aW9uX2Rpc3RpbGJlcnQucHk=) | `100% <100%> (ø)` | |
| [pytorch\_transformers/modeling\_xlnet.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvbW9kZWxpbmdfeGxuZXQucHk=) | `79.01% <100%> (ø)` | :arrow_up: |
| [pytorch\_transformers/modeling\_bert.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvbW9kZWxpbmdfYmVydC5weQ==) | `87.98% <100%> (ø)` | :arrow_up: |
| [...torch\_transformers/tests/tokenization\_bert\_test.py](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139/diff?src=pr&el=tree#diff-cHl0b3JjaF90cmFuc2Zvcm1lcnMvdGVzdHMvdG9rZW5pemF0aW9uX2JlcnRfdGVzdC5weQ==) | `98.66% <100%> (ø)` | :arrow_up: |
| ... and [20 more](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139?src=pr&el=footer). Last update [529a16d...e0caab0](https://codecov.io/gh/huggingface/pytorch-transformers/pull/1139?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).