How can I continue finetuning from checkpoint using the NER script?
Author: ChessMateK
Pull request: no
Labels: name: wontfix

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->
I'm trying to execute [this script](https://github.com/huggingface/transformers/tree/master/examples/ner) using `run_ner.py` but everything I tried to continue fine tuning from checkpoint failed. Any ideas?

I run it using Google Colab. Hereafter the cell content I run:

    %cd "/content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27"
    %pip install .
    %pip install --upgrade .
    %pip install seqeval
    from fastai import * 
    from transformers import *
    %cd "/content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner"

    !python "/content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/run_ner.py" --data_dir ./ \
                                                                                                              --model_type bert \
                                                                                                              --labels ./labels.txt \
                                                                                                              --model_name_or_path "/content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/bert-base-256/checkpoint-10000" \
                                                                                                              --output_dir "/content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/bert-base-256/check" \
                                                                                                              --max_seq_length "256" \
                                                                                                              --num_train_epochs "5" \
                                                                                                              --per_gpu_train_batch_size "4" \
                                                                                                              --save_steps "10000" \
                                                                                                              --seed "1" \
                                                                                                              --do_train --do_eval --do_predict

As you can see, I already tried to substitute model_name_or_path parameter value (that was "bert-base-cased") with checkpoint directory but several errors occurred, asking for the right model name and missing files.

    04/28/2020 15:16:36 - INFO - transformers.tokenization_utils -   Model name '/content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/bert-base-256/checkpoint-10000' not found in model shortcut name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc, bert-base-german-dbmdz-cased, bert-base-german-dbmdz-uncased, bert-base-finnish-cased-v1, bert-base-finnish-uncased-v1, bert-base-dutch-cased). Assuming '/content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/bert-base-256/checkpoint-10000' is a path, a model identifier, or url to a directory containing tokenizer files.
    04/28/2020 15:16:36 - INFO - transformers.tokenization_utils -   Didn't find file /content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/bert-base-256/checkpoint-10000/vocab.txt. We won't load it.
    04/28/2020 15:16:36 - INFO - transformers.tokenization_utils -   Didn't find file /content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/bert-base-256/checkpoint-10000/added_tokens.json. We won't load it.
    04/28/2020 15:16:36 - INFO - transformers.tokenization_utils -   Didn't find file /content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/bert-base-256/checkpoint-10000/special_tokens_map.json. We won't load it.
    04/28/2020 15:16:36 - INFO - transformers.tokenization_utils -   Didn't find file /content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/bert-base-256/checkpoint-10000/tokenizer_config.json. We won't load it.
    Traceback (most recent call last):
    File "/content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/run_ner.py", line 290, in <module>
    main()
    File "/content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/run_ner.py", line 149, in main
    use_fast=model_args.use_fast,
    File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_auto.py", line 197, in from_pretrained
    return tokenizer_class_py.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
    File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils.py", line 868, in from_pretrained
    return cls._from_pretrained(*inputs, **kwargs)
    File "/usr/local/lib/python3.6/dist-packages/transformers/tokenization_utils.py", line 971, in _from_pretrained
    list(cls.vocab_files_names.values()),
    OSError: Model name '/content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/bert-base-256/checkpoint-10000' was not found in tokenizers model name list (bert-base-uncased, bert-large-uncased, bert-base-cased, bert-large-cased, bert-base-multilingual-uncased, bert-base-multilingual-cased, bert-base-chinese, bert-base-german-cased, bert-large-uncased-whole-word-masking, bert-large-cased-whole-word-masking, bert-large-uncased-whole-word-masking-finetuned-squad, bert-large-cased-whole-word-masking-finetuned-squad, bert-base-cased-finetuned-mrpc, bert-base-german-dbmdz-cased, bert-base-german-dbmdz-uncased, bert-base-finnish-cased-v1, bert-base-finnish-uncased-v1, bert-base-dutch-cased). We assumed '/content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/bert-base-256/checkpoint-10000' was a path, a model identifier, or url to a directory containing vocabulary files named ['vocab.txt'] but couldn't find such vocabulary files at this path or url.

Thank you in advance.

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->
**A link to original question on Stack Overflow**: https://stackoverflow.com/questions/61482518/how-can-i-continue-finetuning-from-checkpoint-using-the-ner-script


Comments:
Hello! You're pointing the script to a file or folder `bert-base-256/checkpoint-10000`. Is that a folder containing model file, or is that a file in itself?

> Hello! You're pointing the script to a file or folder `bert-base-256/checkpoint-10000`. Is that a folder containing model file, or is that a file in itself?

It is the folder created by the script containing these files:
- optimizer.pt and scheduler.pt
- pytorch_model.bin and training_args.bin
- config.json



Ah, I think I know where the issue stems from. We've changed the paradigm for our examples, which now rely on a `Trainer` abstraction. Until c811526 five days ago, the tokenizer was unfortunately not saved, which is fixed now.

I would recommend you use the lastest script so that it doesn't happen anymore. 

To fix this error, you could manually save your tokenizer in that folder as so:

```py
from transformers import BertTokenizer

tokenizer = BertTokenizer.from_pretrained("bert-base-cased")
tokenizer.save_pretrained("/content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/bert-base-256/checkpoint-10000")
```

This will save the tokenizer file in the appropriate folder. Please keep in mind that this is to reload the original  `bert-base-cased` tokenizer. If you have modified your tokenizer in any way, you should save that tokenizer in the aforementioned folder. Please let me know if I can be of further help!

> Ah, I think I know where the issue stems from. We've changed the paradigm for our examples, which now rely on a `Trainer` abstraction. Until [c811526](https://github.com/huggingface/transformers/commit/c81152600452ad1bec4ab705356788d29a3573ee) five days ago, the tokenizer was unfortunately not saved, which is fixed now.
> 
> I would recommend you use the lastest script so that it doesn't happen anymore.
> 
> To fix this error, you could manually save your tokenizer in that folder as so:
> 
> ```python
> from transformers import BertTokenizer
> 
> tokenizer = BertTokenizer.from_pretrained("bert-base-cased")
> tokenizer.save_pretrained("/content/drive/My Drive/Colab Notebooks/NER/Batteria/transformers-master_2020_04_27/examples/ner/bert-base-256/checkpoint-10000")
> ```
> 
> This will save the tokenizer file in the appropriate folder. Please keep in mind that this is to reload the original `bert-base-cased` tokenizer. If you have modified your tokenizer in any way, you should save that tokenizer in the aforementioned folder. Please let me know if I can be of further help!

I downloaded transformers library on 27 April, two days ago, after [c811526](https://github.com/huggingface/transformers/commit/c81152600452ad1bec4ab705356788d29a3573ee). So, does this problem still persist?

@ChessMateK 
This might sound a bit silly but did you check if the drive was mounted in colab when you ran that command ?

@patil-suraj Yes, I checked. Furthermore, if I insert as output path the folder where all the checkpoints are saved, even if they are not loaded, the "startup" time of the script increases considerably.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.