German DistilBERT
Author: stefan-it
Pull request: yes

Hi,

this PR adds the German DistilBERT to the library 🤗

Thanks to the Hugging Face team (incl. hardware support) the German DistilBERT was trained on 1/2 of the data that was used for training the [German DBMDZ BERT](https://github.com/dbmdz/german-bert) model for ~4 days.

Evaluation on NER tasks (German CoNLL and GermEval) shows a performance difference of 1.3% on average compared to the German BERT model.

---

Remaining tasks:

* [x] Model, configuration and vocab is already uploaded to S3, only file permissions need to be adjusted

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1873?src=pr&el=h1) Report
> Merging [#1873](https://codecov.io/gh/huggingface/transformers/pull/1873?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/f3386d938348628c91457fc7d8650c223317a053?src=pr&el=desc) will **increase** coverage by `1.35%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/1873/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/1873?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #1873      +/-   ##
==========================================
+ Coverage   82.72%   84.08%   +1.35%     
==========================================
  Files          97       97              
  Lines       14316    14316              
==========================================
+ Hits        11843    12037     +194     
+ Misses       2473     2279     -194
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/1873?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [transformers/tokenization\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/1873/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rva2VuaXphdGlvbl9kaXN0aWxiZXJ0LnB5) | `100% <ø> (ø)` | :arrow_up: |
| [transformers/configuration\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/1873/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL2NvbmZpZ3VyYXRpb25fZGlzdGlsYmVydC5weQ==) | `89.74% <ø> (ø)` | :arrow_up: |
| [transformers/modeling\_distilbert.py](https://codecov.io/gh/huggingface/transformers/pull/1873/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX2Rpc3RpbGJlcnQucHk=) | `95.87% <ø> (ø)` | :arrow_up: |
| [transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/1873/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX29wZW5haS5weQ==) | `82% <0%> (+1.33%)` | :arrow_up: |
| [transformers/modeling\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/1873/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX2N0cmwucHk=) | `96.46% <0%> (+2.21%)` | :arrow_up: |
| [transformers/modeling\_xlnet.py](https://codecov.io/gh/huggingface/transformers/pull/1873/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX3hsbmV0LnB5) | `73.61% <0%> (+2.43%)` | :arrow_up: |
| [transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/1873/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX3JvYmVydGEucHk=) | `71.76% <0%> (+12.35%)` | :arrow_up: |
| [transformers/tests/modeling\_tf\_common\_test.py](https://codecov.io/gh/huggingface/transformers/pull/1873/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX3RmX2NvbW1vbl90ZXN0LnB5) | `97.08% <0%> (+15.53%)` | :arrow_up: |
| [transformers/modeling\_tf\_pytorch\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/1873/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX3RmX3B5dG9yY2hfdXRpbHMucHk=) | `92.95% <0%> (+83.09%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/1873?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1873?src=pr&el=footer). Last update [f3386d9...da06afa](https://codecov.io/gh/huggingface/transformers/pull/1873?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


fyi I just adjusted the permissions on the s3 objects.

Perfect!