Back to all models
question-answering mask_token: [MASK]
Context
Query this model
馃敟 This model is currently loaded and running on the Inference API. 鈿狅笍 This model could not be loaded by the inference API. 鈿狅笍 This model can be loaded on the Inference API on-demand.
JSON Output
API endpoint  

鈿★笍 Upgrade your account to access the Inference API

Share Copied link to clipboard

Monthly model downloads

mrm8488/bert-tiny-finetuned-squadv2 mrm8488/bert-tiny-finetuned-squadv2
2,531 downloads
last 30 days

pytorch

tf

Contributed by

mrm8488 Manuel Romero
156 models

How to use this model directly from the 馃/transformers library:

			
Copy to clipboard
from transformers import AutoTokenizer, AutoModelForQuestionAnswering tokenizer = AutoTokenizer.from_pretrained("mrm8488/bert-tiny-finetuned-squadv2") model = AutoModelForQuestionAnswering.from_pretrained("mrm8488/bert-tiny-finetuned-squadv2")

BERT-Tiny fine-tuned on SQuAD v2

BERT-Tiny created by Google Research and fine-tuned on SQuAD 2.0 for Q&A downstream task.

Mode size (after training): 16.74 MB

Details of BERT-Tiny and its 'family' (from their documentation)

Released on March 11th, 2020

This is model is a part of 24 smaller BERT models (English only, uncased, trained with WordPiece masking) referenced in Well-Read Students Learn Better: On the Importance of Pre-training Compact Models.

The smaller BERT models are intended for environments with restricted computational resources. They can be fine-tuned in the same manner as the original BERT models. However, they are most effective in the context of knowledge distillation, where the fine-tuning labels are produced by a larger and more accurate teacher.

Details of the downstream task (Q&A) - Dataset

SQuAD2.0 combines the 100,000 questions in SQuAD1.1 with over 50,000 unanswerable questions written adversarially by crowdworkers to look similar to answerable ones. To do well on SQuAD2.0, systems must not only answer questions when possible, but also determine when no answer is supported by the paragraph and abstain from answering.

Dataset Split # samples
SQuAD2.0 train 130k
SQuAD2.0 eval 12.3k

Model training

The model was trained on a Tesla P100 GPU and 25GB of RAM. The script for fine tuning can be found here

Results:

Metric # Value
EM 48.60
F1 49.73
Model EM F1 score SIZE (MB)
bert-tiny-finetuned-squadv2 48.60 49.73 16.74
bert-tiny-5-finetuned-squadv2 57.12 60.86 24.34

Model in action

Fast usage with pipelines:

from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="mrm8488/bert-tiny-finetuned-squadv2",
    tokenizer="mrm8488/bert-tiny-finetuned-squadv2"
)

qa_pipeline({
    'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately",
    'question': "Who has been working hard for hugginface/transformers lately?"

})

# Output:
{
  "answer": "Manuel Romero",
  "end": 13,
  "score": 0.05684709993458714,
  "start": 0
}

Yes! That was easy 馃帀 Let's try with another example

qa_pipeline({
    'context': "Manuel Romero has been working hardly in the repository hugginface/transformers lately",
    'question': "For which company has worked Manuel Romero?"
})

# Output:
{
  "answer": "hugginface/transformers",
  "end": 79,
  "score": 0.11613431826808274,
  "start": 56
}

It works!! 馃帀 馃帀 馃帀

Created by Manuel Romero/@mrm8488 | LinkedIn

Made with in Spain