Edit model card

Commonsense-QA-Mistral-7B

This is a finetuned model of mistralai/Mistral-7B-Instruct-v0.1 with tau/commonsense_qa dataset.

The model is loaded in 4-bit and fine-tuned with LoRA.

Usage

Loading of model:

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "rvv-karma/Commonsense-QA-Mistral-7B",
    low_cpu_mem_usage=True,
    return_dict=True,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

tokenizer = AutoTokenizer.from_pretrained("rvv-karma/Commonsense-QA-Mistral-7B", trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "left"

Sample:

pipe = pipeline(
    task="text-generation",
    model=model,
    tokenizer=tokenizer,
    return_full_text=False,
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=13,
    max_new_tokens=8
)

prompt = """<s>
QUESTION:
The sensor would just the distance then set off an alarm, the installation expert explained it was called a what kind of sensor?

OPTIONS:
["near", "closeness", "here", "proximity", "this"]

ANSWER:
"""
result = pipe(prompt)
generated = result[0]['generated_text']
print(generated)

# Output: proximity

Fine-tuning script

Kaggle Notebook

Downloads last month
29
Safetensors
Model size
7.24B params
Tensor type
BF16
·
Inference Examples
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Dataset used to train rvv-karma/Commonsense-QA-Mistral-7B