Edit model card
YAML Metadata Warning: empty or missing yaml metadata in repo card (https://huggingface.co/docs/hub/model-cards#model-card-metadata)

Model

monologg/biobert_v1.1_pubmed fine-tuned on SQuAD V2 using run_squad.py

This model is cased.

Training Parameters

Trained on 4 NVIDIA GeForce RTX 2080 Ti 11Gb

BASE_MODEL=monologg/biobert_v1.1_pubmed
python run_squad.py \
  --version_2_with_negative \
  --model_type albert \
  --model_name_or_path $BASE_MODEL \
  --output_dir $OUTPUT_MODEL \
  --do_eval \
  --do_lower_case \
  --train_file $SQUAD_DIR/train-v2.0.json \
  --predict_file $SQUAD_DIR/dev-v2.0.json \
  --per_gpu_train_batch_size 18 \
  --per_gpu_eval_batch_size 64 \
  --learning_rate 3e-5 \
  --num_train_epochs 3.0 \
  --max_seq_length 384 \
  --doc_stride 128 \
  --save_steps 2000 \
  --threads 24 \
  --warmup_steps 550 \
  --gradient_accumulation_steps 1 \
  --fp16 \
  --logging_steps 50 \
  --do_train

Evaluation

Evaluation on the dev set. I did not sweep for best threshold.

val
exact 75.97068980038743
f1 79.37043950121722
total 11873.0
HasAns_exact 74.13967611336032
HasAns_f1 80.94892513460755
HasAns_total 5928.0
NoAns_exact 77.79646761984861
NoAns_f1 77.79646761984861
NoAns_total 5945.0
best_exact 75.97068980038743
best_exact_thresh 0.0
best_f1 79.37043950121729
best_f1_thresh 0.0

Usage

See huggingface documentation. Training on SQuAD V2 allows the model to score if a paragraph contains an answer:

start_scores, end_scores = model(input_ids) 
span_scores = start_scores.softmax(dim=1).log()[:,:,None] + end_scores.softmax(dim=1).log()[:,None,:]
ignore_score = span_scores[:,0,0] #no answer scores
    
Downloads last month
239
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.