How to load a fine-tuned model pytorch_model.bin produced by run_bert_swag.py
Author: xiami2019
Pull request: no

Hi, guys! I have a little question about how to  load a fine-tuned model 'pytorch_model.bin' produced by run_bert_swag.py.
When I load a fine-tuned model pytorch_model.bin with .from_pretrained methods, runtime error occurs as follow:
RuntimeError: storage has wrong size: expected 4357671300540823961 got 589824.
I fine-tuned a bert-base-uncased model.
 

Comments:
That's a strange error, what are the exact process you are using and full error log?

> That's a strange error, what are the exact process you are using and full error log?

Hi, thanks for the reply.
I used distributed training in one node with 2GPUs and my command is:
export SWAG_DIR=SWAG; export export CUDA_VISIBLE_DEVICES=2,3; python -m torch.distributed.launch --nproc_per_node=2 run_bert_swag.py --bert_model bert-base-uncased --do_train --do_lower_case --do_eval --data_dir $SWAG_DIR/data --train_batch_size 16 --learning_rate 2e-5 --num_train_epochs 1.0 --max_seq_length 80 --output_dir /home/disk1/chengqinyuan/pt_transformer_examples/swag_output --gradient_accumulation_steps 1

I modified run_swag.py with follow lines:
    if args.do_train:
        # Save a trained model, configuration and tokenizer
        model_to_save = model.module if hasattr(model, 'module') else model  # Only save the model it-self

        # If we save using the predefined names, we can load using `from_pretrained`
        output_model_file = os.path.join(args.output_dir, WEIGHTS_NAME)
        output_config_file = os.path.join(args.output_dir, CONFIG_NAME)

        # torch.save(model.state_dict(), output_model_file)
        model_to_save.save_pretrained(args.output_dir)
        model_to_save.config.to_json_file(output_config_file)
        tokenizer.save_vocabulary(args.output_dir)

        # Load a trained model and vocabulary that you have fine-tuned
        model = BertForMultipleChoice.from_pretrained(args.output_dir, num_choices=4)
        tokenizer = BertTokenizer.from_pretrained(args.output_dir, do_lower_case=args.do_lower_case)
    else:
        model = BertForMultipleChoice.from_pretrained(args.output_dir, num_choices=4)
        tokenizer = BertTokenizer.from_pretrained(args.output_dir, do_lower_case=args.do_lower_case)
    model.to(device)

And the error log is:
Traceback (most recent call last):
  File "run_bert_swag.py", line 571, in <module>
    main()
  File "run_bert_swag.py", line 505, in main
    model = BertForMultipleChoice.from_pretrained(args.output_dir)
  File "/home/chengqinyuan/anaconda3/envs/py3/lib/python3.6/site-packages/pytorch_transformers-1.0.0-py3.6.egg/pytorch_transformers/modeling_utils.py", line 406, in from_pretrained
  File "/home/chengqinyuan/anaconda3/envs/py3/lib/python3.6/site-packages/torch/serialization.py", line 387, in load
    return _load(f, map_location, pickle_module, **pickle_load_args)
  File "/home/chengqinyuan/anaconda3/envs/py3/lib/python3.6/site-packages/torch/serialization.py", line 581, in _load
    deserialized_objects[key]._set_from_file(f, offset, f_should_read_directly)
RuntimeError: storage has wrong size: expected -4807048246308659860 got 589824

By the way, when I used the script to save a pre_trained model without fine-tune(I skipped the training process and then saved the model), it can normally load the saved model and do evaluation. But the error would occur when I loaded a fine-tuned model.

Or could you please give me a brief guideline to execute run_swag.py with pytorch_transformers. I followed the guideline at https://huggingface.co/pytorch-transformers/examples.html and encounter many bugs. Thank you very much!

I solved this error by use multi-GPU training instead of distributed training, it seems like something wrong in distributed training setting, thanks for your reply : )

> I solved this error by use multi-GPU training instead of distributed training, it seems like something wrong in distributed training setting, thanks for your reply : )

我在用单机多卡训练后保存模型也遇到了这个问题，请问你是怎么解决的，用DataParallel还是DistributedDataParallel？