Model Description: The DistilBERT model was proposed in the blog post Smaller, faster, cheaper, lighter: Introducing DistilBERT, a distilled version of BERT, and the paper DistilBERT, a distilled version of BERT: smaller, faster, cheaper, and lighter. DistilBERT is a small, fast, cheap, and light Transformer model trained by distilling BERT base. It has 40% fewer parameters than bert-base-uncased, runs 60% faster while preserving over 95% of BERT's performance as measured on the GLUE language understanding benchmark.
This model is a fine-tune checkpoint of distilbert-base-cased-distilled-squad, fine-tuned the task layer using QNLI dataset. Our primary objective was to leverage the model's acquired knowledge during question answering to identify entailment relations between context sentences and questions to explain the model reasoning.
Model Type: Transformer-based language model
Language(s): English
License: cc
Related Models: distilbert/distilbert-base-cased-distilled-squad
How to Get Started with the Model: >
from transformers import pipeline
classifier = pipeline("zero-shot-classification",
model="HeZhang1019/distilbert-base-cased-distilled-squad-qnli-v1")
sentences = [
"Hamlet is a play written by William Shakespeare in the early 17th century.",
"It is one of his most well-known works, featuring the famous quote 'To be, or not to be.'"
]
question = 'Who wrote "Hamlet"?'
results = list()
for sentence in sentences:
result = classifier(question, sentence)
results.append((sentence, result['scores'][0]))
print(results)
[('Hamlet is a play written by William Shakespeare in the early 17th century.', 0.8046764731407166), ("It is one of his most well-known works, featuring the famous quote 'To be, or not to be.'", 0.39214783906936646)]
Uses: This model can be used for entailment classification.
Model Training
- Script:
- Evaluation:
- Epoch: 13
- Training Loss (0.310600)
- Validation Loss (0.298734)
- Accuracy (0.886509)
- Precision (0.886524)
- Recall (0.886509)
- F1 Score (0.886502)
- Downloads last month
- 6