metadata

title: RAFT-QA
sdk: gradio
emoji: 💻
colorFrom: purple
colorTo: gray
pinned: true
thumbnail: >-
  https://cdn-uploads.huggingface.co/production/uploads/67c714e90b99a2332e310979/TrWTI_vofjfGgx4PILFY3.jpeg
short_description: Retrieval-Augmented Fine-Tuning for Question Answering
sdk_version: 5.25.2
language:
  - en
tags:
  - retrieval-augmented-learning
  - question-answering
  - fine-tuning
  - transformers
  - llm
license: mit
datasets:
  - pubmedqa
  - hotpotqa
  - gorilla
library_name: transformers
model-index:
  - name: RAFT-QA
    results:
      - task:
          type: question-answering
          name: Open-Domain Question Answering
        dataset:
          name: PubMedQA
          type: question-answering
        metrics:
          - name: Exact Match (EM)
            type: exact_match
            value: 79.3
          - name: F1 Score
            type: f1
            value: 87.1

RAFT-QA: Retrieval-Augmented Fine-Tuning for Question Answering

Model Overview

RAFT-QA is a sophisticated retrieval-augmented question-answering model designed to significantly enhance answer accuracy through the integration of retrieved documents during the fine-tuning process. By utilizing retrieval-enhanced training, it advances traditional fine-tuning techniques.

Model Details

Base Model Options: mistral-7b, falcon-40b-instruct, or other leading large language models (LLMs)
Fine-Tuning Technique: RAFT (Retrieval-Augmented Fine-Tuning)
Retrieval Strategy: FAISS-based document embedding retrieval
Training Datasets Included: PubMedQA, HotpotQA, Gorilla

How It Works

Retrieve Relevant Documents: FAISS efficiently retrieves the most pertinent documents in response to a query.
Augment Input with Retrieved Context: Incorporates the retrieved documents into the input data.
Fine-Tune the Model: The model learns to effectively weigh the retrieved context to produce improved answers.

Performance Comparison

Model	Exact Match (EM)	F1 Score
GPT-3.5 (baseline)	74.8	84.5
Standard Fine-Tuning	76.2	85.6
RAFT-QA (ours)	79.3	87.1

Usage

To load the model using the transformers library:

from transformers import AutoModelForQuestionAnswering, AutoTokenizer

model_name = "your-hf-username/raft-qa"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForQuestionAnswering.from_pretrained(model_name)

Limitations

The model's performance is contingent on the quality of the retrieved documents.
For optimal results, domain-specific tuning may be necessary.

Citation

If you utilize this model in your work, please cite it as follows:

@article{raft2025,
  title={Retrieval-Augmented Fine-Tuning (RAFT) for Enhanced Question Answering},
  author={Your Name et al.},
  journal={ArXiv},
  year={2025}
}

License

This model is released under the Apache 2.0 License.

This version provides clarity and conciseness, ensuring all sections are clear and correctly formatted according to the Hugging Face repository standards. Make sure the dataset type (question-answering) matches your intended use case.