DatTran0509/Finetune_XLM_R_base_QA

🤖 Vietnamese Question Answering Model

Welcome to my Question Answering model! This model is designed to answer questions based on a given passage (context).

📝 Problem Introduction

The Question Answering (QA) task requires a model to automatically answer questions based on a given text. The model takes a passage (context) and a question, and returns the answer that can be found within the passage. This task is crucial in natural language processing (NLP) and has wide applications in information retrieval, virtual assistants, and other automated services.

🔧 Model

This model is based on XLM-R (XLM-RoBERTa), a powerful multilingual model developed by Facebook AI. XLM-R is highly effective in natural language tasks and can handle multiple languages.

Training Hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 128
• optimizer: Use adamw_torch with betas=(0.9, 0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10

The model is fine-tuned on a QA dataset with Vietnamese questions and passages.

🚀 How to Use

1. Run the Model Online

You can use this model online without needing to install anything. To try the model, follow these steps:

1. Visit my Hugging Face Spaces demo:

   Hugging Face Demo

2. Enter a question and context:

   • In the "Enter your question" box, type the question you want to ask.
   • In the "Enter the context you want to ask about" box, type the passage you want the model to find the answer from.
   • Example test: context

      Ông Nguyễn Ngọc Hà cũng nêu một số điểm mới trong công tác tổ chức thi. Theo đó, một trong những vấn đề mà các sở lo lắng là quá trình sắp xếp phòng thi. Kỳ thi chỉ tổ chức 3 buổi thi, đặc biệt buổi thứ 3 chỉ có 1 buổi nhưng có 2 môn thi. Ví dụ môn vật lý có thể thi trong 2 khung giờ trong buổi thi đó. Việc này được xử lý ra sao?
      Để tiện lợi cho học sinh, các em được sắp xếp ở một phòng thi duy nhất, không phải di chuyển trong các buổi thi. Cho dù một môn thi ở hai khung giờ khác nhau, để đảm bảo yếu tố an ninh thay vì chỉ có 24 mã đề thì nâng lên thành 48 mã đề, 24 mã/khung giờ.
     

     question

      Sau khi thay đổi có tổng cộng bao nhiêu đề thi?
      Thi tổng cộng bao nhiêu buổi?
      Tổng cộng có bao nhiêu mã đề?
     

3. Click the "RUN" button:

   • The model will return the answer if it exists, or display "Please ask another question, no answer found in the context" if no answer is found in the text.

4. Click the "Clear" button to reset the inputs and start again.

2. Running the Model on Colab, Kaggle or your device

If you want to run the model on your local machine, you can follow these steps:

Install the required libraries:

First, install the necessary libraries:

pip install transformers torch gradio

Run the Python Code

Create a Python file (e.g., app.py) and copy the following code into your file:

import gradio as gr
from transformers import AutoModelForQuestionAnswering, AutoTokenizer
import torch

# Initialize model and tokenizer
model_id = "DatTran0509/Finetune_XLM_R_base_QA"
model = AutoModelForQuestionAnswering.from_pretrained(model_id)
tokenizer = AutoTokenizer.from_pretrained(model_id)

def answer_question(question, context):
    inputs = tokenizer(
        question,
        context,
        return_tensors="pt",
        truncation=True,
        max_length=384,
        return_offsets_mapping=True
    )

    offset_mapping = inputs.pop("offset_mapping")

    with torch.no_grad():
        outputs = model(**inputs)
        start_logits = outputs.start_logits
        end_logits = outputs.end_logits

    start_index = torch.argmax(start_logits)
    end_index = torch.argmax(end_logits) + 1

    if end_index <= start_index:
        return "Please ask another question, no answer found in the context."

    offsets = offset_mapping[0]
    start_char = offsets[start_index][0].item()
    end_char = offsets[end_index - 1][1].item()
    answer = context[start_char:end_char]
    
    return answer

iface = gr.Interface(
    fn=answer_question,
    inputs=[ 
        gr.Textbox(label="🌟 Enter your question", placeholder="Type your question here...", lines=2, max_length=500),
        gr.Textbox(label="📜 Enter the context you want to ask about", placeholder="Type your context here...", lines=4, max_length=1000)
    ],
    outputs="text",
    title="🤖 Question Answering Model",
    description="This is a Question Answering model that allows you to input a question and context to get an answer from the text.",
    allow_flagging="never",
    live=False,
    submit_btn="RUN",
    clear_btn="Clear"
)

iface.launch()

Then, run the Python file:

python app.py

3. Requirements and Dependencies

• Python: 3.x
• Required Libraries: transformers, torch, gradio

📚 References:

• Transformers Documentation
• Gradio Documentation

---

Thank you for using my model! Feel free to share and experiment with it. If you encounter any issues, don't hesitate to open an issue or ask questions!