Question & Answering Model for 'Save Your Minutes' from Dobby-AI Distilbert_Base fine-tuned on SQuAD2.0 and custom QA dataset
This model is [twmkn9/distilbert-base-uncased-squad2] trained on additional custom dataset as:
!python3 run_squad.py --model_type distilbert \
--model_name_or_path /content/distilbert_base_384 \
--do_lower_case \
--output_dir /content/model/\
--do_train \
--train_file $data_dir/additional_qa.json\
--version_2_with_negative \
--do_lower_case \
--num_train_epochs 3 \
--weight_decay 0.01 \
--learning_rate 3e-5 \
--max_grad_norm 0.5 \
--adam_epsilon 1e-6 \
--max_seq_length 512 \
--doc_stride 128 \
--threads 12 \
--logging_steps 50 \
--save_steps 1000 \
--overwrite_output_dir \
--per_gpu_train_batch_size 4
We used Google Colab for training the model,