BERT-Base Uncased Fine-Tuned on SQuAD

Overview

This repository contains a BERT-Base Uncased model fine-tuned on the SQuAD (Stanford Question Answering Dataset) for Question Answering (QA) tasks. The model has been fine-tuned for 2 epochs, making it suitable for extracting answers from given contexts by predicting start and end token positions.

The Model predicts 2 probabilities among all the tokens in the vocab , One indicating the start token and the other indicating the end token, Then the answer between both these tokens are extracted.

Model Details

Model Type: BERT-Base Uncased
Fine-Tuning Dataset: SQuAD (Stanford Question Answering Dataset)
Number of Epochs: 2
Task: Question Answering
Base Model: BERT-Base Uncased

Usage

How to Load the Model

You can load the model using the transformers library from Hugging Face:

from transformers import BertForQuestionAnswering, BertTokenizer

# Load the tokenizer and model
tokenizer = BertTokenizer.from_pretrained("Abdo36/Bert-SquAD-QA")
model = BertForQuestionAnswering.from_pretrained("Abdo36/Bert-SquAD-QA")

context = "BERT is a method of pre-training language representations."
question = "What is BERT?"

inputs = tokenizer.encode_plus(question, context, return_tensors="pt")

# Perform inference
outputs = model(**inputs)
start_scores = outputs.start_logits
end_scores = outputs.end_logits

# Extract answer
start_index = start_scores.argmax()
end_index = end_scores.argmax()
answer = tokenizer.decode(inputs["input_ids"][0][start_index:end_index + 1])

print("Answer:", answer)

Citation

If you use this model in your research, please cite the original BERT paper:

@article{devlin2018bert,
  title={BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding},
  author={Devlin, Jacob and Chang, Ming-Wei and Lee, Kenton and Toutanova, Kristina},
  journal={arXiv preprint arXiv:1810.04805},
  year={2018}
}

Abdo36
/

Bert-SquAD-QA