error when trying to use multilingual model for fine tuning
Author: KavyaGujjala
Pull request: no
Labels: name: wontfix

I wanted to use fine tuning for hindi language data. For that I tried to give bert-base-mutlilingual model but I am getting the following error

> python pregenerate_training_data.py --train_corpus=./hindi_pytorch_bert_data_1.txt --bert_model=bert-base-multilingual --output_dir=./hindi_train_data_1_3epochs/ --epochs_to_generate=3

```
Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex.
Model name 'bert-base-multilingual' was not found in model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese). We assumed 'bert-base-multilingual' was a path or url but couldn't find any file associated to this path or url.
Traceback (most recent call last):
  File "pregenerate_training_data.py", line 292, in <module>
    main()
  File "pregenerate_training_data.py", line 255, in main
    vocab_list = list(tokenizer.vocab.keys())
AttributeError: 'NoneType' object has no attribute 'vocab'
```

I tried giving bert-base-multilingual-cased as well then I ran into this error

> python pregenerate_training_data.py --train_corpus=./hindi_pytorch_bert_data_1.txt --bert_model=bert-base-multilingual-cased --output_dir=./hindi_train_data_1_3epochs/ --epochs_to_generate=3

```
Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex.
usage: pregenerate_training_data.py [-h] --train_corpus TRAIN_CORPUS
                                    --output_dir OUTPUT_DIR --bert_model
                                    {bert-base-uncased,bert-large-uncased,bert-base-cased,bert-base-multilingual,bert-base-chinese}
                                    [--do_lower_case] [--reduce_memory]
                                    [--epochs_to_generate EPOCHS_TO_GENERATE]
                                    [--max_seq_len MAX_SEQ_LEN]
                                    [--short_seq_prob SHORT_SEQ_PROB]
                                    [--masked_lm_prob MASKED_LM_PROB]
                                    [--max_predictions_per_seq MAX_PREDICTIONS_PER_SEQ]
pregenerate_training_data.py: error: argument --bert_model: invalid choice: 'bert-base-multilingual-cased' (choose from 'bert-base-uncased', 'bert-large-uncased', 'bert-base-cased', 'bert-base-multilingual', 'bert-base-chinese')
```

How to resolve this issue?

Comments:
I made changes in the code pregenerate_training_data.py
from 
```
parser.add_argument("--bert_model", type=str, required=True,
                        choices=["bert-base-uncased", "bert-large-uncased", "bert-base-cased",
                                 "bert-base-multilingual", "bert-base-chinese"])
```
to
```
parser.add_argument("--bert_model", type=str, required=True,
                        choices=["bert-base-uncased", "bert-large-uncased", "bert-base-cased",
                                 "bert-base-multilingual-cased", "bert-base-multilingual-uncased", "bert-base-chinese"])
```


and it worked.

It occured to me maybe because I forgot to install pytorch. I installed pytorch then it's solved.

Hi, 
I followed your code, and got this error:

Traceback (most recent call last):      | 6796/185072 [00:00<00:18, 9787.42it/s]
  File "pregenerate_training_data.py", line 308, in <module>
    main()
  File "pregenerate_training_data.py", line 293, in main
    vocab_list=vocab_list)
  File "pregenerate_training_data.py", line 208, in create_instances_from_document
    assert len(tokens_b) >= 1
AssertionError

Can you please share your code?

What computer specification to train your corpus? How big it is and how long you need to training your corpus?

I wanna too train my corpus using fine tuning, maybe your answers give me an insight about how relevant me to training the corpus, thanks

> Hi,
> I followed your code, and got this error:
> 
> Traceback (most recent call last): | 6796/185072 [00:00<00:18, 9787.42it/s]
> File "pregenerate_training_data.py", line 308, in 
> main()
> File "pregenerate_training_data.py", line 293, in main
> vocab_list=vocab_list)
> File "pregenerate_training_data.py", line 208, in create_instances_from_document
> assert len(tokens_b) >= 1
> AssertionError
> 
> Can you please share your code?

I run into the same problem. Wondering if you have solved your problem. Thanks!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.