File size: 1,936 Bytes
7b045c0 1f52f44 372ef1f 7b045c0 1f4d4bf 7b045c0 464ea09 7b045c0 372ef1f 7b045c0 372ef1f 7b045c0 372ef1f 7b045c0 372ef1f 7b045c0 372ef1f 7b045c0 372ef1f 7b045c0 372ef1f 7b045c0 372ef1f 464ea09 372ef1f 7b045c0 372ef1f 7b045c0 372ef1f 7b045c0 464ea09 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 |
---
language:
- en
license: llama3
library_name: transformers
tags:
- biology
- medical
datasets:
- thesven/SyntheticMedicalQA-4336
---
# Llama3-8B-SFT-SyntheticMedical-bnb-4bit
<!-- Provide a quick summary of what the model is/does. -->
## Model Details
### Model Description
Llama3-8B-SFT-SSyntheticMedical-bnb-4bit is trained using the SFT method via QLoRA on 4336 rows of medical data to enhance it's abilities in the realm of scientific anatomy.
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
### Using the model with transformers
```python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
model_name_or_path = "thesven/Llama3-8B-SFT-SyntheticMedical-bnb-4bit"
# BitsAndBytesConfig for loading the model in 4-bit precision
bnb_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype="float16",
)
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)
model = AutoModelForCausalLM.from_pretrained(
model_name_or_path,
device_map="auto",
trust_remote_code=False,
revision="main",
quantization_config=bnb_config
)
model.pad_token = model.config.eos_token_id
prompt_template = '''
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
You are an expert in the field of anatomy, help explain its topics to me.<|eot_id|><|start_header_id|>user<|end_header_id|>
What is the function of the hamstring?<|eot_id|><|start_header_id|>assistant<|end_header_id|>
'''
input_ids = tokenizer(prompt_template, return_tensors='pt').input_ids.cuda()
output = model.generate(inputs=input_ids, temperature=0.1, do_sample=True, top_p=0.95, top_k=40, max_new_tokens=512)
print(generated_text)
``` |