Can not find vocabulary file for Chinese model
Author: zlinao
Pull request: no

After I convert the TF model to pytorch model, I run a classification task on a new Chinese dataset, but get this:

CUDA_VISIBLE_DEVICES=3 python run_classifier.py   --task_name weibo --do_eval --do_train --bert_model chinese_L-12_H-768_A-12 --max_seq_length 128   --train_batch_size 32   --learning_rate 2e-5   --num_train_epochs 3.0   --output_dir bert_result


11/18/2018 21:56:59 - INFO - __main__ -   device cuda n_gpu 1 distributed training False
11/18/2018 21:56:59 - INFO - pytorch_pretrained_bert.tokenization -   loading vocabulary file chinese_L-12_H-768_A-12
Traceback (most recent call last):
  File "run_classifier.py", line 661, in <module>
    main()
  File "run_classifier.py", line 508, in main
    tokenizer = BertTokenizer.from_pretrained(args.bert_model)
  File "/home/lin/jpmorgan/pytorch-pretrained-BERT/pytorch_pretrained_bert/tokenization.py", line 141, in from_pretrained
    tokenizer = cls(resolved_vocab_file, do_lower_case)
  File "/home/lin/jpmorgan/pytorch-pretrained-BERT/pytorch_pretrained_bert/tokenization.py", line 94, in __init__
    "model use `tokenizer = BertTokenizer.from_pretrained(PRETRAINED_MODEL_NAME)`".format(vocab_file))
ValueError: Can't find a vocabulary file at path 'chinese_L-12_H-768_A-12'. To load the vocabulary from a Google pretrained model use `tokenizer = BertTokenizer.from_pretrained(PRETRAINED_MODEL_NAME)`

Comments:
need to specify the path of vocab.txt for:
tokenizer = BertTokenizer.from_pretrained(args.bert_model)

@zlinao ，i try to load the vocab using the following code：
tokenizer = BertTokenizer.from_pretrained("bert-base-chinese//vocab.txt"

however，get errors
11/19/2018 15:33:13 - INFO - pytorch_pretrained_bert.tokenization -   loading vocabulary file bert-base-chinese//vocab.txt
Traceback (most recent call last):
  File "E:/PythonWorkSpace/PytorchBert/BertTest/torchTest.py", line 6, in <module>
    tokenizer = BertTokenizer.from_pretrained("bert-base-chinese//vocab.txt")
  File "C:\anaconda\lib\site-packages\pytorch_pretrained_bert-0.1.2-py3.6.egg\pytorch_pretrained_bert\tokenization.py", line 141, in from_pretrained
  File "C:\anaconda\lib\site-packages\pytorch_pretrained_bert-0.1.2-py3.6.egg\pytorch_pretrained_bert\tokenization.py", line 95, in __init__
  File "C:\anaconda\lib\site-packages\pytorch_pretrained_bert-0.1.2-py3.6.egg\pytorch_pretrained_bert\tokenization.py", line 70, in load_vocab
UnicodeDecodeError: 'gbk' codec can't decode byte 0x81 in position 1564: illegal multibyte sequenc

do you have the same problem？

Hi,
Why don't you guys just do `tokenizer = BertTokenizer.from_pretrained('bert-base-chinese')` as [indicated in the readme](https://github.com/huggingface/pytorch-pretrained-BERT#loading-google-ais-pre-trained-weigths-and-pytorch-dump) and the `run_classifier.py` example?

> Hi,
> Why don't you guys just do `tokenizer = BertTokenizer.from_pretrained('bert-base-chinese')` as [indicated in the readme](https://github.com/huggingface/pytorch-pretrained-BERT#loading-google-ais-pre-trained-weigths-and-pytorch-dump) and the `run_classifier.py` example?

Yes, it is easier to use shortcut name. Thanks for your great work.

> @zlinao ，i try to load the vocab using the following code：
> tokenizer = BertTokenizer.from_pretrained("bert-base-chinese//vocab.txt"
> 
> however，get errors
> 11/19/2018 15:33:13 - INFO - pytorch_pretrained_bert.tokenization - loading vocabulary file bert-base-chinese//vocab.txt
> Traceback (most recent call last):
> File "E:/PythonWorkSpace/PytorchBert/BertTest/torchTest.py", line 6, in 
> tokenizer = BertTokenizer.from_pretrained("bert-base-chinese//vocab.txt")
> File "C:\anaconda\lib\site-packages\pytorch_pretrained_bert-0.1.2-py3.6.egg\pytorch_pretrained_bert\tokenization.py", line 141, in from_pretrained
> File "C:\anaconda\lib\site-packages\pytorch_pretrained_bert-0.1.2-py3.6.egg\pytorch_pretrained_bert\tokenization.py", line 95, in **init**
> File "C:\anaconda\lib\site-packages\pytorch_pretrained_bert-0.1.2-py3.6.egg\pytorch_pretrained_bert\tokenization.py", line 70, in load_vocab
> UnicodeDecodeError: 'gbk' codec can't decode byte 0x81 in position 1564: illegal multibyte sequenc
> 
> do you have the same problem？

you can change you encoding to 'utf-8' when you load the vocab.txt