AttributeError: 'CTRLTokenizer' object has no attribute 'control_codes'
Author: vessenes
Pull request: no

## 🐛 Bug

I can't seem to get ctrl generation working. 

This is with a pull of the repo from master, and pip3 install as recommended during installation:

The problem arise when using:
* [ X ] the official example scripts: 

```bash
$ uname -a
Linux ctrl 4.9.0-11-amd64 #1 SMP Debian 4.9.189-3+deb9u1 (2019-09-20) x86_64 GNU/Linux
```


<!-- If you have a code sample, error messages, stack traces, please provide it here as well. -->
```bash
python3 ./examples/run_generation.py     --model_type=ctrl     --length=20     --model_name_or_path=ctrl     --temperature=0     --repetition_penalty=1.2 
/home/vessenes/.local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/vessenes/.local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/vessenes/.local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/vessenes/.local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/vessenes/.local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/vessenes/.local/lib/python3.5/site-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
/home/vessenes/.local/lib/python3.5/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:541: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/vessenes/.local/lib/python3.5/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:542: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/vessenes/.local/lib/python3.5/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:543: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/vessenes/.local/lib/python3.5/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:544: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/vessenes/.local/lib/python3.5/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:545: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  _np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/vessenes/.local/lib/python3.5/site-packages/tensorboard/compat/tensorflow_stub/dtypes.py:550: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
  np_resource = np.dtype([("resource", np.ubyte, 1)])
10/25/2019 22:11:28 - INFO - transformers.tokenization_utils -   loading file https://raw.githubusercontent.com/salesforce/ctrl/master/ctrl-vocab.json from cache at /home/vessenes/.cache/torch/transformers/a858ad854d3847b02da3aac63555142de6a05f2a26d928bb49e881970514e186.285c96a541cf6719677cfb634929022b56b76a0c9a540186ba3d8bbdf02bca42
10/25/2019 22:11:28 - INFO - transformers.tokenization_utils -   loading file https://raw.githubusercontent.com/salesforce/ctrl/master/ctrl-merges.txt from cache at /home/vessenes/.cache/torch/transformers/aa2c569e6648690484ade28535a8157aa415f15202e84a62e82cc36ea0c20fa9.26153bf569b71aaf15ae54be4c1b9254dbeff58ca6fc3e29468c4eed078ac142
10/25/2019 22:11:29 - INFO - transformers.configuration_utils -   loading configuration file https://storage.googleapis.com/sf-ctrl/pytorch/ctrl-config.json from cache at /home/vessenes/.cache/torch/transformers/d6492ca334c2a4e079f43df30956acf935134081b2b3844dc97457be69b623d0.1ebc47eb44e70492e0c20494a084f108332d20fea7fe5ad408ef5e7a8f2baef4
10/25/2019 22:11:29 - INFO - transformers.configuration_utils -   Model config {
  "attn_pdrop": 0.1,
  "dff": 8192,
  "embd_pdrop": 0.1,
  "finetuning_task": null,
  "from_tf": false,
  "initializer_range": 0.02,
  "layer_norm_epsilon": 1e-06,
  "n_ctx": 512,
  "n_embd": 1280,
  "n_head": 16,
  "n_layer": 48,
  "n_positions": 50000,
  "num_labels": 1,
  "output_attentions": false,
  "output_hidden_states": false,
  "output_past": true,
  "pruned_heads": {},
  "resid_pdrop": 0.1,
  "summary_activation": null,
  "summary_first_dropout": 0.1,
  "summary_proj_to_labels": true,
  "summary_type": "cls_index",
  "summary_use_proj": true,
  "torchscript": false,
  "use_bfloat16": false,
  "vocab_size": 246534
}
10/25/2019 22:00:19 - INFO - transformers.modeling_utils -   loading weights file https://storage.googleapis.com/sf
-ctrl/pytorch/seqlen256_v1.bin from cache at /home/vessenes/.cache/torch/transformers/c146cc96724f27295a0c3ada1fbb3
632074adf87e9aef8269e44c9208787f8c8.b986347cbab65fa276683efbb9c2f7ee22552277bcf6e1f1166557ed0852fdf0
10/25/2019 22:01:17 - INFO - __main__ -   Namespace(device=device(type='cpu'), length=20, model_name_or_path='ctrl'
, model_type='ctrl', n_gpu=0, no_cuda=False, padding_text='', prompt='', repetition_penalty=1.2, seed=42, stop_toke
n=None, temperature=0.0, top_k=0, top_p=0.9, xlm_lang='')
Model prompt >>> Link Thid is a test article
Traceback (most recent call last):
  File "./examples/run_generation.py", line 256, in <module>
    main()
  File "./examples/run_generation.py", line 228, in main
    if not any(context_tokens[0] == x for x in tokenizer.control_codes.values()):
AttributeError: 'CTRLTokenizer' object has no attribute 'control_codes'
```

## Expected behavior

I expect to be able to type in a prompt and see text generated.


## Environment

* OS: Debian 4.9
* Python version: 3.5.3
* PyTorch version: 1.3.0 
* PyTorch Transformers version (or branch): not sure how to find this info
* Using GPU ?  Yes - V100 on GCP
* Distributed of parallel setup ? No
* Any other relevant information:

## Additional context

<!-- Add any other context about the problem here. -->


Comments:
I had the same issue. As a temporary workaround you can simply comment out the following lines as long as you remember to use a control token at the beginning of every prompt that you supply to ctrl:
if args.model_type == "ctrl":
            if not any(context_tokens[0] == x for x in tokenizer.control_codes.values()):
                logger.info("WARNING! You are not starting your generation from a control code so you won't get good results")

Hmm, not sure what happens there. Have you tried doing:
```python
from transformers.tokenization_ctrl import CTRLTokenizer

tokenizer = CTRLTokenizer.from_pretrained('ctrl')
print(tokenizer.control_codes)
```
?

Your version of Python is 3.5.x?
Is there a `control_codes = CONTROL_CODES` attributed defined inside your `CTRLTokenizer` class?

can you also paste your `pip list`?


 @julien-c 

I've encountered the same bug too! I don't know how to resolve this problem!
Keep reading my description below because it could be very interesting what I wrote!

### WHEN THE BUG HAS BEEN FOUND
First of all, I've created a virtual environment dedicated to trying out Transformers library. After that, I've installed _tensorflow-gpu 2.0_ and _PyTorch 1.3.0_. Finally, I've installed transformers today with the following command:
`pip install transformers`

I'm trying to use the CTRL by SalesForce model for text generation purposes. I've gone to the **examples** directory and after that I've executed the script called _run_generation.py_ with the following statement:
`python run_generation.py --model_type ctrl --model_name_or_path ctrl --temperature 0.5 --repetition_penalty 1.2 --no_cuda`.

### EXPECTED BEHAVIOUR
I expect to be able to type in a prompt and insert a control code I like and see the text generated by CTRL model.

### A BIT OF REVERSE ENGINEERING
After I've found this error, I've opened a command line launching **python** (**version 3.6.9**) and I've written the following code lines:

```
from transformers.tokenization_ctrl import CTRLTokenizer
tokenizer = CTRLTokenizer.from_pretrained('ctrl')
tokenizer.control_codes
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'CTRLTokenizer' object has no attribute 'control_codes'
```

After found the same error, I've used the **inspect** module of Python in order to view internally what **CTRLTokenizer** class contains. The result opens a doubts:

`'class CTRLTokenizer(PreTrainedTokenizer):\n    """\n    CTRL BPE tokenizer. Peculiarities:\n        - Byte-level Byte-Pair-Encoding\n        - Requires a space to start the input string => the encoding methods should be called with the\n          ``add_prefix_space`` flag set to ``True``.\n          Otherwise, this tokenizer ``encode`` and ``decode`` method will not conserve\n          the absence of a space at the beginning of a string: `tokenizer.decode(tokenizer.encode("Hello")) = " Hello"`\n    """\n    vocab_files_names = VOCAB_FILES_NAMES\n    pretrained_vocab_files_map = PRETRAINED_VOCAB_FILES_MAP\n    max_model_input_sizes = PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES\n\n    def __init__(self, vocab_file, merges_file, unk_token="<unk>", **kwargs):\n        super(CTRLTokenizer, self).__init__(unk_token=unk_token, **kwargs)\n        self.max_len_single_sentence = self.max_len # no default special tokens - you can update this value if you add special tokens\n        self.max_len_sentences_pair = self.max_len # no default special tokens - you can update this value if you add special tokens\n\n        self.encoder = json.load(open(vocab_file, encoding="utf-8"))\n        self.decoder = {v:k for k,v in self.encoder.items()}\n        merges = open(merges_file, encoding=\'utf-8\').read().split(\'\\n\')[1:-1]\n        merges = [tuple(merge.split()) for merge in merges]\n        self.bpe_ranks = dict(zip(merges, range(len(merges))))\n        self.cache = {}\n\n    @property\n    def vocab_size(self):\n        return len(self.encoder)\n\n    def bpe(self, token):\n        if token in self.cache:\n            return self.cache[token]\n        word = tuple(token)\n        word = tuple(list(word[:-1]) + [word[-1]+\'</w>\'])\n        pairs = get_pairs(word)\n\n        if not pairs:\n            return token\n\n        while True:\n            bigram = min(pairs, key = lambda pair: self.bpe_ranks.get(pair, float(\'inf\')))\n            if bigram not in self.bpe_ranks:\n                break\n            first, second = bigram\n            new_word = []\n            i = 0\n            while i < len(word):\n                try:\n                    j = word.index(first, i)\n                    new_word.extend(word[i:j])\n                    i = j\n                except:\n                    new_word.extend(word[i:])\n                    break\n\n                if word[i] == first and i < len(word)-1 and word[i+1] == second:\n                    new_word.append(first+second)\n                    i += 2\n                else:\n                    new_word.append(word[i])\n                    i += 1\n            new_word = tuple(new_word)\n            word = new_word\n            if len(word) == 1:\n                break\n            else:\n                pairs = get_pairs(word)\n        word = \'@@ \'.join(word)\n        word = word[:-4]\n        self.cache[token] = word\n        return word\n\n    def _tokenize(self, text):\n        """ Tokenize a string.\n        """\n        split_tokens = []\n\n        text = text.split(\' \')\n\n        for token in text:\n            split_tokens.extend([t for t in self.bpe(token).split(\' \')])\n        return split_tokens\n\n    def _convert_token_to_id(self, token):\n        """ Converts a token (str/unicode) in an id using the vocab. """\n        return self.encoder.get(token, self.encoder.get(self.unk_token))\n\n    def _convert_id_to_token(self, index):\n        """Converts an index (integer) in a token (string/unicode) using the vocab."""\n        return self.decoder.get(index, self.unk_token)\n\n    def convert_tokens_to_string(self, tokens):\n        """ Converts a sequence of tokens (string) in a single string. """\n        out_string = \' \'.join(tokens).replace(\'@@ \', \'\').strip()\n        return out_string\n\n    def save_vocabulary(self, save_directory):\n        """Save the tokenizer vocabulary and merge files to a directory."""\n        if not os.path.isdir(save_directory):\n            logger.error("Vocabulary path ({}) should be a directory".format(save_directory))\n            return\n        vocab_file = os.path.join(save_directory, VOCAB_FILES_NAMES[\'vocab_file\'])\n        merge_file = os.path.join(save_directory, VOCAB_FILES_NAMES[\'merges_file\'])\n\n        with open(vocab_file, \'w\', encoding=\'utf-8\') as f:\n            f.write(json.dumps(self.encoder, ensure_ascii=False))\n\n        index = 0\n        with open(merge_file, "w", encoding="utf-8") as writer:\n            writer.write(u\'#version: 0.2\\n\')\n            for bpe_tokens, token_index in sorted(self.bpe_ranks.items(), key=lambda kv: kv[1]):\n                if index != token_index:\n                    logger.warning("Saving vocabulary to {}: BPE merge indices are not consecutive."\n                                   " Please check that the tokenizer is not corrupted!".format(merge_file))\n                    index = token_index\n                writer.write(\' \'.join(bpe_tokens) + u\'\\n\')\n                index += 1\n\n        return vocab_file, merge_file\n'
`
It is strange because it is **different from the source code reported in GitHub of the CTRLTokenizer class** [https://github.com/huggingface/transformers/blob/master/transformers/tokenization_ctrl.py](url). Maybe the code is an old version of this Python script?

Moreover, by using the **inspect** module another time, I've found that the _tokenization_ctrl.py_ Python script contains the following source code (no "CONTROL_CODES" is into this script). It seems to be a bug problem of not using the correct Python class (i.e. not the same script in GitHub):

`'# coding=utf-8\n# Copyright 2018 Salesforce and The HuggingFace Inc. team.\n#\n# Licensed under the Apache License, Version 2.0 (the "License");\n# you may not use this file except in compliance with the License.\n# You may obtain a copy of the License at\n#\n#     http://www.apache.org/licenses/LICENSE-2.0\n#\n# Unless required by applicable law or agreed to in writing, software\n# distributed under the License is distributed on an "AS IS" BASIS,\n# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n# See the License for the specific language governing permissions and\n# limitations under the License.\n"""Tokenization classes for Salesforce CTRL."""\nfrom __future__ import (absolute_import, division, print_function,\n                        unicode_literals)\n\nimport json\nimport logging\nimport os\nimport regex as re\nfrom io import open\n\nfrom .tokenization_utils import PreTrainedTokenizer\n\nlogger = logging.getLogger(__name__)\n\nVOCAB_FILES_NAMES = {\n    \'vocab_file\': \'vocab.json\',\n    \'merges_file\': \'merges.txt\',\n}\n\nPRETRAINED_VOCAB_FILES_MAP = {\n    \'vocab_file\':\n    {\n        \'ctrl\': "https://raw.githubusercontent.com/salesforce/ctrl/master/ctrl-vocab.json",\n    },\n    \'merges_file\':\n    {\n        \'ctrl\': "https://raw.githubusercontent.com/salesforce/ctrl/master/ctrl-merges.txt",\n    },\n}\n\nPRETRAINED_POSITIONAL_EMBEDDINGS_SIZES = {\n    \'ctrl\': 256,\n}\n\ndef get_pairs(word):\n    """Return set of symbol pairs in a word.\n\n    Word is represented as tuple of symbols (symbols being variable-length strings).\n    """\n    pairs = set()\n    prev_char = word[0]\n    for char in word[1:]:\n        pairs.add((prev_char, char))\n        prev_char = char\n\n    pairs = set(pairs)\n    return pairs\n\nclass CTRLTokenizer(PreTrainedTokenizer):\n    """\n    CTRL BPE tokenizer. Peculiarities:\n        - Byte-level Byte-Pair-Encoding\n        - Requires a space to start the input string => the encoding methods should be called with the\n          ``add_prefix_space`` flag set to ``True``.\n          Otherwise, this tokenizer ``encode`` and ``decode`` method will not conserve\n          the absence of a space at the beginning of a string: `tokenizer.decode(tokenizer.encode("Hello")) = " Hello"`\n    """\n    vocab_files_names = VOCAB_FILES_NAMES\n    pretrained_vocab_files_map = PRETRAINED_VOCAB_FILES_MAP\n    max_model_input_sizes = PRETRAINED_POSITIONAL_EMBEDDINGS_SIZES\n\n    def __init__(self, vocab_file, merges_file, unk_token="<unk>", **kwargs):\n        super(CTRLTokenizer, self).__init__(unk_token=unk_token, **kwargs)\n        self.max_len_single_sentence = self.max_len # no default special tokens - you can update this value if you add special tokens\n        self.max_len_sentences_pair = self.max_len # no default special tokens - you can update this value if you add special tokens\n\n        self.encoder = json.load(open(vocab_file, encoding="utf-8"))\n        self.decoder = {v:k for k,v in self.encoder.items()}\n        merges = open(merges_file, encoding=\'utf-8\').read().split(\'\\n\')[1:-1]\n        merges = [tuple(merge.split()) for merge in merges]\n        self.bpe_ranks = dict(zip(merges, range(len(merges))))\n        self.cache = {}\n\n    @property\n    def vocab_size(self):\n        return len(self.encoder)\n\n    def bpe(self, token):\n        if token in self.cache:\n            return self.cache[token]\n        word = tuple(token)\n        word = tuple(list(word[:-1]) + [word[-1]+\'</w>\'])\n        pairs = get_pairs(word)\n\n        if not pairs:\n            return token\n\n        while True:\n            bigram = min(pairs, key = lambda pair: self.bpe_ranks.get(pair, float(\'inf\')))\n            if bigram not in self.bpe_ranks:\n                break\n            first, second = bigram\n            new_word = []\n            i = 0\n            while i < len(word):\n                try:\n                    j = word.index(first, i)\n                    new_word.extend(word[i:j])\n                    i = j\n                except:\n                    new_word.extend(word[i:])\n                    break\n\n                if word[i] == first and i < len(word)-1 and word[i+1] == second:\n                    new_word.append(first+second)\n                    i += 2\n                else:\n                    new_word.append(word[i])\n                    i += 1\n            new_word = tuple(new_word)\n            word = new_word\n            if len(word) == 1:\n                break\n            else:\n                pairs = get_pairs(word)\n        word = \'@@ \'.join(word)\n        word = word[:-4]\n        self.cache[token] = word\n        return word\n\n    def _tokenize(self, text):\n        """ Tokenize a string.\n        """\n        split_tokens = []\n\n        text = text.split(\' \')\n\n        for token in text:\n            split_tokens.extend([t for t in self.bpe(token).split(\' \')])\n        return split_tokens\n\n    def _convert_token_to_id(self, token):\n        """ Converts a token (str/unicode) in an id using the vocab. """\n        return self.encoder.get(token, self.encoder.get(self.unk_token))\n\n    def _convert_id_to_token(self, index):\n        """Converts an index (integer) in a token (string/unicode) using the vocab."""\n        return self.decoder.get(index, self.unk_token)\n\n    def convert_tokens_to_string(self, tokens):\n        """ Converts a sequence of tokens (string) in a single string. """\n        out_string = \' \'.join(tokens).replace(\'@@ \', \'\').strip()\n        return out_string\n\n    def save_vocabulary(self, save_directory):\n        """Save the tokenizer vocabulary and merge files to a directory."""\n        if not os.path.isdir(save_directory):\n            logger.error("Vocabulary path ({}) should be a directory".format(save_directory))\n            return\n        vocab_file = os.path.join(save_directory, VOCAB_FILES_NAMES[\'vocab_file\'])\n        merge_file = os.path.join(save_directory, VOCAB_FILES_NAMES[\'merges_file\'])\n\n        with open(vocab_file, \'w\', encoding=\'utf-8\') as f:\n            f.write(json.dumps(self.encoder, ensure_ascii=False))\n\n        index = 0\n        with open(merge_file, "w", encoding="utf-8") as writer:\n            writer.write(u\'#version: 0.2\\n\')\n            for bpe_tokens, token_index in sorted(self.bpe_ranks.items(), key=lambda kv: kv[1]):\n                if index != token_index:\n                    logger.warning("Saving vocabulary to {}: BPE merge indices are not consecutive."\n                                   " Please check that the tokenizer is not corrupted!".format(merge_file))\n                    index = token_index\n                writer.write(\' \'.join(bpe_tokens) + u\'\\n\')\n                index += 1\n\n        return vocab_file, merge_file\n\n    # def decode(self, token_ids, skip_special_tokens=False, clean_up_tokenization_spaces=True):\n    #     filtered_tokens = \' \'.join(self.convert_ids_to_tokens(token_ids, skip_special_tokens=skip_special_tokens))\n    #     tokens_generated_so_far = re.sub(\'(@@ )\', \'\', string=filtered_tokens)\n    #     tokens_generated_so_far = re.sub(\'(@@ ?$)\', \'\', string=tokens_generated_so_far)\n    #     return \'\'.join(tokens_generated_so_far)\n'
` 

### STACK TRACE

```
2019-10-31 15:02:03.443162: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2019-10-31 15:02:03.455996: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-10-31 15:02:03.456755: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1618] Found device 0 with properties: 
name: GeForce GTX 980 Ti major: 5 minor: 2 memoryClockRate(GHz): 1.076
pciBusID: 0000:01:00.0
2019-10-31 15:02:03.456943: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.0
2019-10-31 15:02:03.457919: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10.0
2019-10-31 15:02:03.458684: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10.0
2019-10-31 15:02:03.458868: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10.0
2019-10-31 15:02:03.460032: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10.0
2019-10-31 15:02:03.460829: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10.0
2019-10-31 15:02:03.460921: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcudnn.so.7'; dlerror: libcudnn.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64
2019-10-31 15:02:03.460930: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1641] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.
Skipping registering GPU devices...
2019-10-31 15:02:03.461171: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-10-31 15:02:03.485286: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3408000000 Hz
2019-10-31 15:02:03.485895: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x559a4bb637a0 executing computations on platform Host. Devices:
2019-10-31 15:02:03.485911: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): Host, Default Version
2019-10-31 15:02:03.525426: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1006] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2019-10-31 15:02:03.525984: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x559a4bb3de90 executing computations on platform CUDA. Devices:
2019-10-31 15:02:03.525999: I tensorflow/compiler/xla/service/service.cc:175]   StreamExecutor device (0): GeForce GTX 980 Ti, Compute Capability 5.2
2019-10-31 15:02:03.526083: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1159] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-10-31 15:02:03.526090: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1165]      
10/31/2019 15:02:05 - INFO - transformers.tokenization_utils -   loading file https://raw.githubusercontent.com/salesforce/ctrl/master/ctrl-vocab.json from cache at /home/vidiemme/.cache/torch/transformers/a858ad854d3847b02da3aac63555142de6a05f2a26d928bb49e881970514e186.285c96a541cf6719677cfb634929022b56b76a0c9a540186ba3d8bbdf02bca42
10/31/2019 15:02:05 - INFO - transformers.tokenization_utils -   loading file https://raw.githubusercontent.com/salesforce/ctrl/master/ctrl-merges.txt from cache at /home/vidiemme/.cache/torch/transformers/aa2c569e6648690484ade28535a8157aa415f15202e84a62e82cc36ea0c20fa9.26153bf569b71aaf15ae54be4c1b9254dbeff58ca6fc3e29468c4eed078ac142
10/31/2019 15:02:05 - INFO - transformers.configuration_utils -   loading configuration file https://storage.googleapis.com/sf-ctrl/pytorch/ctrl-config.json from cache at /home/vidiemme/.cache/torch/transformers/d6492ca334c2a4e079f43df30956acf935134081b2b3844dc97457be69b623d0.1ebc47eb44e70492e0c20494a084f108332d20fea7fe5ad408ef5e7a8f2baef4
10/31/2019 15:02:05 - INFO - transformers.configuration_utils -   Model config {
  "attn_pdrop": 0.1,
  "dff": 8192,
  "embd_pdrop": 0.1,
  "finetuning_task": null,
  "from_tf": false,
  "initializer_range": 0.02,
  "layer_norm_epsilon": 1e-06,
  "n_ctx": 512,
  "n_embd": 1280,
  "n_head": 16,
  "n_layer": 48,
  "n_positions": 50000,
  "num_labels": 1,
  "output_attentions": false,
  "output_hidden_states": false,
  "output_past": true,
  "pruned_heads": {},
  "resid_pdrop": 0.1,
  "summary_activation": null,
  "summary_first_dropout": 0.1,
  "summary_proj_to_labels": true,
  "summary_type": "cls_index",
  "summary_use_proj": true,
  "torchscript": false,
  "use_bfloat16": false,
  "vocab_size": 246534
}

10/31/2019 15:02:05 - INFO - transformers.modeling_utils -   loading weights file https://storage.googleapis.com/sf-ctrl/pytorch/seqlen256_v1.bin from cache at /home/vidiemme/.cache/torch/transformers/c146cc96724f27295a0c3ada1fbb3632074adf87e9aef8269e44c9208787f8c8.b986347cbab65fa276683efbb9c2f7ee22552277bcf6e1f1166557ed0852fdf0
10/31/2019 15:02:37 - INFO - __main__ -   Namespace(device=device(type='cpu'), length=20, model_name_or_path='ctrl', model_type='ctrl', n_gpu=1, no_cuda=True, padding_text='', prompt='', repetition_penalty=1.2, seed=42, stop_token=None, temperature=0.5, top_k=0, top_p=0.9, xlm_lang='')
Model prompt >>> Hi, my name is Edward and i'm 26 years old
Traceback (most recent call last):
  File "run_generation.py", line 256, in <module>
    main()
  File "run_generation.py", line 228, in main
    if not any(context_tokens[0] == x for x in tokenizer.control_codes.values()):
AttributeError: 'CTRLTokenizer' object has no attribute 'control_codes'
```

### REQUIREMENTS.TXT OF MY VIRTUAL ENVIRONMENT
```
Package              Version  
-------------------- ---------
absl-py              0.8.1    
astor                0.8.0    
boto3                1.10.6   
botocore             1.13.6   
cachetools           3.1.1    
certifi              2019.9.11
chardet              3.0.4    
Click                7.0      
docutils             0.15.2   
gast                 0.2.2    
google-auth          1.6.3    
google-auth-oauthlib 0.4.1    
google-pasta         0.1.7    
grpcio               1.24.3   
h5py                 2.10.0   
idna                 2.8      
jmespath             0.9.4    
joblib               0.14.0   
Keras-Applications   1.0.8    
Keras-Preprocessing  1.1.0    
Markdown             3.1.1    
numpy                1.17.3   
oauthlib             3.1.0    
opt-einsum           3.1.0    
pandas               0.25.2   
Pillow               6.2.1    
pip                  19.3.1   
protobuf             3.10.0   
pyasn1               0.4.7    
pyasn1-modules       0.2.7    
python-dateutil      2.8.0    
pytz                 2019.3   
PyYAML               5.1.2    
regex                2019.8.19
requests             2.22.0   
requests-oauthlib    1.2.0    
rsa                  4.0      
s3transfer           0.2.1    
sacremoses           0.0.35   
scikit-learn         0.21.3   
scipy                1.3.1    
sentencepiece        0.1.83   
setuptools           41.4.0   
six                  1.12.0   
tensorboard          2.0.1    
tensorflow-estimator 2.0.1    
tensorflow-gpu       2.0.0    
termcolor            1.1.0    
torch                1.3.0    
torchtext            0.4.0    
torchvision          0.4.1    
tqdm                 4.36.1   
transformers         2.1.1    
urllib3              1.25.6   
Werkzeug             0.16.0   
wheel                0.33.6   
wrapt                1.11.2  
```

### ENVIRONMENT

```
>>> import platform; print("Platform", platform.platform())
Platform Linux-4.15.0-66-generic-x86_64-with-debian-buster-sid
>>> import sys; print("Python", sys.version)
Python 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 19:07:31) 
[GCC 7.3.0]
>>> import torch; print("PyTorch", torch.__version__)
PyTorch 1.3.0
>>> import tensorflow; print("Tensorflow", tensorflow.__version__)
Tensorflow 2.0.0
```

Oh ok I think I know what happens to you guys. This repo contains both a **lib** (pushed to Pypi) and a set of **example scripts**. To reliably run the versions of the scripts that are on master, you also need to install the lib from master (i.e. not the last pypi release).

Can you run `pip install -e .` from master? This will ensure the lib's code and the scripts are in sync.
cc @thomwolf @LysandreJik 

Closing this as I don't think it's a bug per se.

As suggested correctly by @julien-c, in order to solve the problem pointed out in #1636, you have to:

1. download the entire GitHub repository with `git clone https://github.com/huggingface/transformers.git` command
2. enter to the directory you have just downloaded with `cd transformers` command
3. install the repo by running `pip install -e .` command
4. go to "examples" directory
5. now you can run `run_generation.py` script

Hoping it is helpful for developers that want to trying out CTRL model by HuggingFace.

> As suggested correctly by @julien-c, in order to solve the problem pointed out in #1636, you have to:
> 
> 1. download the entire GitHub repository with `git clone https://github.com/huggingface/transformers.git` command
> 2. enter to the directory you have just downloaded with `cd transformers` command
> 3. install the repo by running `pip install -e .` command
> 4. go to "examples" directory
> 5. now you can run `run_generation.py` script
> 
> Hoping it is helpful for developers that want to trying out CTRL model by HuggingFace.

I using anaconda. When `pip install -e`, it ran but only installed certain package.

> > As suggested correctly by @julien-c, in order to solve the problem pointed out in #1636, you have to:
> > 
> > 1. download the entire GitHub repository with `git clone https://github.com/huggingface/transformers.git` command
> > 2. enter to the directory you have just downloaded with `cd transformers` command
> > 3. install the repo by running `pip install -e .` command
> > 4. go to "examples" directory
> > 5. now you can run `run_generation.py` script
> > 
> > Hoping it is helpful for developers that want to trying out CTRL model by HuggingFace.
> 
> I using anaconda. When `pip install -e`, it ran but only installed certain package.

@mzjuin please be more detailed about your problem