Gilper: Fine-Tuned BERT Model for Question Answering

Gilper is a fine-tuned BERT-based model, designed specifically for question-answering tasks. It excels at answering questions given a relevant context, making it ideal for applications like customer support, knowledge base queries, and more.

Features

Model Architecture: Based on BERT (bert-large-uncased-whole-word-masking-finetuned-squad).
Fine-Tuned Tasks: Specialized in question answering with input in the form of question + context.
Performance: Optimized for accuracy and relevance in providing answers.
Use Cases: Customer support bots, knowledge retrieval systems, educational tools, and more.

How to Use

Installation

To use Gilper, you need the Hugging Face Transformers library. Install it using:

pip install transformers

Quick Start

Here’s an example of how to use Gilper with the Transformers library:

from transformers import BertTokenizerFast, BertForQuestionAnswering, pipeline
import torch

# Load the tokenizer and model
tokenizer = BertTokenizerFast.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')
model = BertForQuestionAnswering.from_pretrained('bert-large-uncased-whole-word-masking-finetuned-squad')

# Define question-answering pipeline
question_answerer = pipeline(
    "question-answering",
    model="fawez9/gilper",
    tokenizer="fawez9/gilper",
    device=0 if torch.cuda.is_available() else -1
)

# Example input
question = "How many parameters does BLOOM have?"
context = "BLOOM has 176 billion parameters and can generate text in 46 natural languages and 13 programming languages."

# Get response
response = question_answerer(question=question, context=context)
print(response)

Input Format

Question: A string representing the query.
Context: A string containing the information from which the model will extract the answer.

Output Format

The model returns the most relevant span of text from the context as the answer.

Training Details

Base Model: bert-large-uncased-whole-word-masking-finetuned-squad.
Dataset: Fine-tuned on SQuAD.
Optimization: Trained using the AdamW optimizer with a learning rate of 2e-6.
Epochs: 3.
Batch Size: 4.
Training Metrics:
- Global Steps: 750
- Training Loss: 0.2977
- Training Runtime: 3884.56 seconds
- Samples per Second: 3.089
- Steps per Second: 0.193
- Total FLOPs: 8358362929152000.0

Model Card

Model Name: Gilper
Architecture: BERT-based
Fine-Tuned Task: Question Answering
Languages: English

Limitations

While Gilper is highly effective at question answering, it has the following limitations:

It relies heavily on the relevance and quality of the provided context.
It may not perform as well on questions requiring external knowledge not present in the context.
Biases from the training dataset may influence its responses.

Citation

If you use Gilper in your research or applications, please cite it as:

@misc{gilper2024,
  title={Gilper: Fine-Tuned BERT Model for Question Answering},
  author={Fawez},
  year={2024},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/fawez9/gilper}}
}

Feedback and Contributions

We welcome feedback and contributions! If you encounter issues or have suggestions for improvements, please open an issue or submit a pull request.

Acknowledgments

Hugging Face for their excellent Transformers library.
The creators of the base BERT model.
The SQuAD dataset used for fine-tuning.

fawez9
/

gilper