metadata

library_name: peft
base_model: google/gemma-7b-it
datasets:
  - nmarafo/truthful_qa_TrueFalse-Feedback
language:
  - en
  - es
license: other
license_name: gemma-terms-of-use
license_link: https://ai.google.dev/gemma/terms

Model Card for Model ID

This is an adapter prepared to return True or False depending on whether the student's answer ("student_answer") is correct based on the question ("question") and comparing it with a given answer ("best_answer"). The prompt has the following structure:

<start_of_turn>user\nAnalyze the question, the expected answer, and the student's response.
Determine if the student's answer is correct or not. It only returns True if the student's answer is correct with respect to the expected answer or False otherwise.
Add a brief comment explaining why the answer is correct or incorrect.\n\n
Question: {question}\n
Expected Answer: {best_answer}\n
Student Answer: {student_answer}<end_of_turn><start_of_turn>model"

How to Get Started with the Model

In Google Colab:


!pip install -q -U bitsandbytes
!pip install -q -U git+https://github.com/huggingface/transformers.git
!pip install -q -U git+https://github.com/huggingface/peft.git
!pip install -q -U git+https://github.com/huggingface/accelerate.git
!pip install -q -U gradio

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig, GemmaTokenizer
from peft import AutoPeftModelForCausalLM
import torch

# Carga el modelo y el tokenizer
model_id = "google/gemma-7b-it"
adapter = "nmarafo/Gemma-7B-it-4bit-TrueFalse-Feedback"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16
)

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = AutoPeftModelForCausalLM.from_pretrained(adapter, quantization_config=bnb_config, device_map={"":0})

def predict(question, best_answer, student_answer, language):
    if language == "English":
        system_message = "Analyze the question, the expected answer, and the student's response. Determine if the student's answer is conceptually correct in relation to the expected answer, regardless of the exact wording. Return True if the student's answer is correct or False otherwise. Add a brief comment explaining the rationale behind the answer being correct or incorrect."
    else:  # Asumimos que cualquier otra opción será Español
        system_message = "Analiza la pregunta, la respuesta esperada y la respuesta del estudiante. Determina si la respuesta del estudiante es conceptualmente correcta en relación con la respuesta esperada, independientemente de la redacción exacta. Devuelve Verdadero si la respuesta del estudiante es correcta o Falso en caso contrario. Añade un breve comentario explicando el razonamiento detrás de la corrección o incorrección de la respuesta."

    prompt = f"{system_message}\n\nQuestion: {question}\nExpected Answer: {best_answer}\nStudent Answer: {student_answer}"
    prompt_template=f"<start_of_turn>user{prompt}<end_of_turn><start_of_turn>model"

    # Ajusta aquí para incluir attention_mask
    encoding = tokenizer(prompt_template, return_tensors='pt', padding=True, truncation=True, max_length=256)
    input_ids = encoding['input_ids'].cuda()
    attention_mask = encoding['attention_mask'].cuda()

    output = model.generate(input_ids, attention_mask=attention_mask,
                            temperature=0.7, do_sample=True, top_p=0.95,
                            top_k=40, max_new_tokens=256, pad_token_id=tokenizer.eos_token_id)
    response = tokenizer.decode(output[0], skip_special_tokens=True)

    return response

import gradio as gr

iface = gr.Interface(
    fn=predict,
    inputs=[
        gr.Textbox(lines=2, placeholder="Pregunta"),
        gr.Textbox(lines=2, placeholder="Mejor Respuesta"),
        gr.Textbox(lines=2, placeholder="Respuesta del Estudiante"),
        gr.Radio(choices=["English", "Español"], label="Idioma")
    ],
    outputs=gr.Textbox(label="Respuesta del Modelo")
)
iface.launch(share=True,debug=True)

Framework versions

PEFT 0.8.2