YAML Metadata
Warning:
empty or missing yaml metadata in repo card
(https://huggingface.co/docs/hub/model-cards#model-card-metadata)
Multiple Prediction Heads
- ExtractiveQA Head
- Three Class Classification Head, classes => (yes, no, extra_qa) to answer binary questions or direct to ExtractiveQA Head
BoolQ Validation dataset Evaluation:
support => 3270
accuracy => 0.73
macro f1 => 0.71
SQuAD Validation dataset Evaluation:
eval_HasAns_exact = 78.0196
eval_HasAns_f1 = 84.0327
eval_HasAns_total = 5928
eval_NoAns_exact = 81.8167
eval_NoAns_f1 = 81.8167
eval_NoAns_total = 5945
eval_best_exact = 79.9208
eval_best_f1 = 82.9231
eval_exact = 79.9208
eval_f1 = 82.9231
eval_samples = 12165
eval_total = 11873
Uasge in transformers
Import the script from here
from multitask_model import RobertaForMultitaskQA
from transformers import RobertaTokenizerFast
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = RobertaForMultitaskQA.from_pretrained(
"shahrukhx01/roberta-base-squad2-boolq-baseline",
task_labels_map={"squad_v2": 2, "boolq": 3},
).to(device)
tokenizer = RobertaTokenizerFast.from_pretrained("shahrukhx01/roberta-base-squad2-boolq-baseline")
- Downloads last month
- 4
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.