MhoOmm/HotPotQA_DEBERT

DeBERTa-v3-base for Open-Domain Multi-Hop Question Answering

A fine-tuned DeBERTa-v3-base model for extractive question answering on the HotpotQA dataset. The model is designed to work as the reader component of a Retrieval-Augmented Question Answering (QA) pipeline, where a dense retriever first retrieves relevant documents and the reader extracts the answer span from the retrieved context.

---

Model Overview

• Base Model: microsoft/deberta-v3-base
• Task: Extractive Question Answering
• Framework: Hugging Face Transformers
• Dataset: HotpotQA (Distractor Setting)
• Language: English

Unlike generative language models, this model predicts the exact answer span from a retrieved context instead of generating free-form text.

---

Pipeline Architecture

Question
      │
      ▼
Dense Retriever (BGE-base-en-v1.5 + FAISS)
      │
Top-10 Retrieved Documents
      │
      ▼
DeBERTa-v3-base Reader
      │
Predicted Answer Span
      │
      ▼
(Optional) LLM Explanation Layer

The retriever identifies relevant Wikipedia articles, while DeBERTa extracts the answer span from the retrieved passages.

---

Dataset

The model was fine-tuned on the HotpotQA dataset.

Data Preparation

For each question:

1. Retrieve the top-10 Wikipedia articles using a dense retriever.
2. Assemble the retrieved passages into a context.
3. If retrieval misses supporting documents, use the gold supporting passages from HotpotQA.
4. Remove:
   • Yes/No questions
   • Samples where the answer cannot be aligned within the context

Final Dataset Size

Split	Examples
Training	84,959
Validation	4,687

---

Retriever

The reader depends on an external dense retriever.

Embedding Model

• BAAI/bge-base-en-v1.5

Vector Database

• FAISS Flat Index

Corpus

• 507,494 Wikipedia articles

Embedding Dimension

• 768

Retrieval Performance

Metric	Score
Recall@3	0.7100
Recall@5	0.7650
Recall@10	0.8150
Recall@20	0.8200

---

Training Configuration

Parameter	Value
Base Model	microsoft/deberta-v3-base
Optimizer	AdamW
Learning Rate	2e-5
Weight Decay	0.01
Epochs	3
Batch Size	8
Max Sequence Length	512
Document Stride	128

---

Tokenization

The model uses the official DeBERTa tokenizer.

Configuration:

• truncation="only_second"
• padding="max_length"
• max_length=512
• stride=128
• return_overflowing_tokens=True
• return_offsets_mapping=True

Answer spans are aligned using character-level offsets.

Windows that do not contain the answer are assigned start/end position 0.

---

Evaluation

Performance on the HotpotQA validation set:

Metric	Score
Exact Match (EM)	60.53
F1 Score	74.21

Comparison with other transformer readers:

Model	EM	F1
DeBERTa-v3-base	60.53	74.21
RoBERTa-base	60.17	73.96
ALBERT-base-v2	56.73	70.31

---

Example Usage

from transformers import pipeline

qa_pipeline = pipeline(
    "question-answering",
    model="YOUR_USERNAME/deberta-hotpotqa",
    tokenizer="YOUR_USERNAME/deberta-hotpotqa"
)

result = qa_pipeline(
    question="Which magazine was started first, Arthur's Magazine or First for Women?",
    context=context
)

print(result["answer"])

---

Intended Use

This model is suitable for:

• Open-domain Question Answering
• Multi-hop Question Answering
• Educational Assistants
• Search Systems
• Retrieval-Augmented QA Pipelines
• Research in Extractive Question Answering

---

Limitations

This model:

• Requires an external retrieval system.
• Cannot retrieve documents by itself.
• Produces extractive answer spans only.
• Does not generate explanations.
• Performance depends heavily on retrieval quality.
• Was trained only on English Wikipedia.
• Does not support Yes/No questions since they were removed during preprocessing.

---

Future Improvements

Potential improvements include:

• Hybrid BM25 + Dense Retrieval
• Cross-Encoder Re-ranking
• Paragraph-level indexing
• Iterative Multi-hop Retrieval
• Confidence estimation
• Citation generation
• Integration with an instruction-tuned LLM for grounded answer explanations

---

---

License

This project is intended for research and educational purposes. Please ensure compliance with the licenses of the original HotpotQA dataset and the DeBERTa-v3-base model.