How to load trained model of distilbert
Author: ANSHUMAN87
Pull request: no
Labels: name: wontfix

## ❓ Questions & Help
Hi,
 I have trained distilbert using the steps mentioned in example/distillation.
saved the checkpoints into one directory.

But i cant use run_glue.py using the checkpoint path i saved for distilbert.
It throws error for tokenizer missing.

Would you please help me, how to achieve that. If i am doing any mistake in my step.
TIA!!!
<!-- A clear and concise description of the question. -->


Comments:
Hello @ANSHUMAN87,

Could you share the command you're using (and the error you get)?
You should have at least these arguments: `--model_type distilbert --model_name_or_path <your_model_path>`.

Victor

I have mentioned below the steps i followed.

Step 1: python3 train.py --student_type distilbert --student_config training_configs/distilbert-base-uncased.json --teacher_type bert --teacher_name bert-base-uncased --alpha_ce 5.0 --alpha_mlm 2.0 --alpha_cos 1.0 --mlm --freeze_pos_embs --dump_path serialization_dir/my_first_training --data_file binarized_text.bert-base-uncased.pickle  --token_counts token_counts.bert-base-uncased.pickle --force

Result: Successful

Step 2: python3 run_glue.py --model_type distilbert --model_name_or_path distillation/serialization_dir/my_first_training/ --task_name CoLA --do_eval --do_lower_case --data_dir /home/anshuman/3/GLUE-Dataset/glue_data/CoLA/ --max_seq_length 128  --output_dir  distillation/serialization_dir/my_first_training/

Error: 
![image](https://user-images.githubusercontent.com/32511895/67922533-fee99100-fbd0-11e9-878b-8193c54fef75.png)




Thank you, I understand what's happening now.

It happens because you're not launching any training (`--do_train`) before evaluating. What happens in `run_glue.py` is that when you do the evaluation, the tokenizer is loaded from the one saved in `output_dir` (see [here](https://github.com/huggingface/transformers/blob/master/examples/run_glue.py#L524)). The latter has been saved a few lines before ([here](https://github.com/huggingface/transformers/blob/master/examples/run_glue.py#L510)) in `do_train`... So basically, you're trying to load something that doesn't exist yet...

One way to quickly bypass this is: a/ adding `--do_train --num_train_epochs 0.0`, b/ set the return to `global_step, tr_loss / 1` (see [here](https://github.com/huggingface/transformers/blob/master/examples/run_glue.py#L207)) to avoid division by 0.

Of course, by doing that, you're evaluating on a GLUE task a model that hasn't been finetuned for the GLUE task in question (i.e. you're doing zero-shot).

Also, I recommend to use a different `output_dir` in the `run_glue.py` command: run_glue will overwrite your pre-training (step 1) when saving the model under the name `pytorch_model.bin`.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


> Thank you, I understand what's happening now.
> 
> It happens because you're not launching any training (`--do_train`) before evaluating. What happens in `run_glue.py` is that when you do the evaluation, the tokenizer is loaded from the one saved in `output_dir` (see [here](https://github.com/huggingface/transformers/blob/master/examples/run_glue.py#L524)). The latter has been saved a few lines before ([here](https://github.com/huggingface/transformers/blob/master/examples/run_glue.py#L510)) in `do_train`... So basically, you're trying to load something that doesn't exist yet...
> 
> One way to quickly bypass this is: a/ adding `--do_train --num_train_epochs 0.0`, b/ set the return to `global_step, tr_loss / 1` (see [here](https://github.com/huggingface/transformers/blob/master/examples/run_glue.py#L207)) to avoid division by 0.
> 
> Of course, by doing that, you're evaluating on a GLUE task a model that hasn't been finetuned for the GLUE task in question (i.e. you're doing zero-shot).
> 
> Also, I recommend to use a different `output_dir` in the `run_glue.py` command: run_glue will overwrite your pre-training (step 1) when saving the model under the name `pytorch_model.bin`.

Hello,@VictorSanh,
I have completed model train in pytorch. But how can I use the trained model to do some new test on a new test.tvs? my run.sh is:
export TASK_NAME=mytask

python src/run_glue.py \
    --model_name_or_path ch/ \
    --task_name $TASK_NAME \
    --do_predict \
    --data_dir data/ \
    --max_seq_length 128 \
    --output_dir saved_test_moels/ \
    --overwrite_cache
it doesn't work. What should I change?  Thank you.