File size: 1,604 Bytes
7c13b3b 0b7d8f4 7c13b3b 0b7d8f4 7c13b3b 61f5241 7c13b3b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 |
---
language:
- en
license: apache-2.0
tags:
- text-generation
- finetuned
datasets:
- tau/commonsense_qa
pipeline_tag: text-generation
---
# Commonsense-QA-Mistral-7B
This is a finetuned model of [mistralai/Mistral-7B-Instruct-v0.1](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1)
with [tau/commonsense_qa](https://huggingface.co/datasets/tau/commonsense_qa) dataset.
The model is loaded in 4-bit and fine-tuned with LoRA.
## Usage
### Loading of model:
```python
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"rvv-karma/Commonsense-QA-Mistral-7B",
low_cpu_mem_usage=True,
return_dict=True,
torch_dtype=torch.bfloat16,
device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("rvv-karma/Commonsense-QA-Mistral-7B", trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "left"
```
### Sample:
```python
pipe = pipeline(
task="text-generation",
model=model,
tokenizer=tokenizer,
return_full_text=False,
pad_token_id=tokenizer.pad_token_id,
eos_token_id=13,
max_new_tokens=8
)
prompt = """<s>
QUESTION:
The sensor would just the distance then set off an alarm, the installation expert explained it was called a what kind of sensor?
OPTIONS:
["near", "closeness", "here", "proximity", "this"]
ANSWER:
"""
result = pipe(prompt)
generated = result[0]['generated_text']
print(generated)
# Output: proximity
```
## Fine-tuning script
[Kaggle Notebook](https://www.kaggle.com/rvkarma/commonsense-qa-mistral-7b) |