Adding CTRL
Author: keskarnitish
Pull request: yes

EDIT 10/04 
Almost complete (tests pass / generation makes sense).
Please comment with issues if you find them. 

**Incomplete - Adding to facilitate collaboration**
This PR would add functionality to perform inference on CTRL (https://github.com/salesforce/ctrl) in the `🤗/transformers` repo. 
Commits will be squashed later before merging. 

Comments:
# [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1383?src=pr&el=h1) Report
> Merging [#1383](https://codecov.io/gh/huggingface/transformers/pull/1383?src=pr&el=desc) into [master](https://codecov.io/gh/huggingface/transformers/commit/1c5079952f5f10eeac4cb6801b4fd1f36b0eff73?src=pr&el=desc) will **increase** coverage by `1.63%`.
> The diff coverage is `92.38%`.

[![Impacted file tree graph](https://codecov.io/gh/huggingface/transformers/pull/1383/graphs/tree.svg?width=650&token=9qOlN6Hb1c&height=150&src=pr)](https://codecov.io/gh/huggingface/transformers/pull/1383?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master    #1383      +/-   ##
==========================================
+ Coverage   83.79%   85.42%   +1.63%     
==========================================
  Files          84       91       +7     
  Lines       12587    13464     +877     
==========================================
+ Hits        10547    11502     +955     
+ Misses       2040     1962      -78
```


| [Impacted Files](https://codecov.io/gh/huggingface/transformers/pull/1383?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [transformers/modeling\_roberta.py](https://codecov.io/gh/huggingface/transformers/pull/1383/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX3JvYmVydGEucHk=) | `80.57% <ø> (+15.1%)` | :arrow_up: |
| [transformers/tests/modeling\_tf\_gpt2\_test.py](https://codecov.io/gh/huggingface/transformers/pull/1383/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX3RmX2dwdDJfdGVzdC5weQ==) | `94.73% <0%> (ø)` | :arrow_up: |
| [transformers/tests/modeling\_tf\_common\_test.py](https://codecov.io/gh/huggingface/transformers/pull/1383/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rlc3RzL21vZGVsaW5nX3RmX2NvbW1vbl90ZXN0LnB5) | `95.38% <100%> (+7.88%)` | :arrow_up: |
| [transformers/modeling\_openai.py](https://codecov.io/gh/huggingface/transformers/pull/1383/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX29wZW5haS5weQ==) | `81.75% <100%> (+1.35%)` | :arrow_up: |
| [transformers/file\_utils.py](https://codecov.io/gh/huggingface/transformers/pull/1383/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL2ZpbGVfdXRpbHMucHk=) | `74.17% <100%> (ø)` | :arrow_up: |
| [transformers/modeling\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/1383/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL21vZGVsaW5nX2F1dG8ucHk=) | `51.85% <20%> (-2.1%)` | :arrow_down: |
| [transformers/configuration\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/1383/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL2NvbmZpZ3VyYXRpb25fYXV0by5weQ==) | `58.82% <33.33%> (-2.47%)` | :arrow_down: |
| [transformers/tokenization\_auto.py](https://codecov.io/gh/huggingface/transformers/pull/1383/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rva2VuaXphdGlvbl9hdXRvLnB5) | `67.64% <33.33%> (-3.33%)` | :arrow_down: |
| [transformers/tokenization\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/1383/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL3Rva2VuaXphdGlvbl9jdHJsLnB5) | `83.6% <83.6%> (ø)` | |
| [transformers/configuration\_ctrl.py](https://codecov.io/gh/huggingface/transformers/pull/1383/diff?src=pr&el=tree#diff-dHJhbnNmb3JtZXJzL2NvbmZpZ3VyYXRpb25fY3RybC5weQ==) | `88.88% <88.88%> (ø)` | |
| ... and [18 more](https://codecov.io/gh/huggingface/transformers/pull/1383/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/huggingface/transformers/pull/1383?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/huggingface/transformers/pull/1383?src=pr&el=footer). Last update [1c50799...d9e60f4](https://codecov.io/gh/huggingface/transformers/pull/1383?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).


Ok for merge

Thanks for adding this! I'm currently doing some experiments with the CTRL model, and I've a question about the tokenization:

```bash
tokenizer.tokenize("Munich and Berlin are nice cities.")               
Out[6]: ['m@@', 'unic@@', 'h', 'and', 'ber@@', 'lin', 'are', 'nice', 'cities', '.']
```

Do you have any idea, why the output returns lowercased tokens only - `Berlin` and `Munich` do both appear in the vocab file (cased, and the splitting of `Munich` looks really weird 😅).

Yes, we are aware of the issue.

We are fixing this problem in #1480.