File size: 2,063 Bytes
539d32e 6bd5066 81cac91 539d32e 6bd5066 c28f3cc 81cac91 c7d1097 81cac91 c28f3cc ad8b95a 6bd5066 81cac91 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 |
---
library_name: peft
license: apache-2.0
datasets:
- b-mc2/sql-create-context
language:
- en
pipeline_tag: text-generation
---
## Training procedure
The following `bitsandbytes` quantization config was used during training:
- quant_method: bitsandbytes
- load_in_8bit: False
- load_in_4bit: True
- llm_int8_threshold: 6.0
- llm_int8_skip_modules: None
- llm_int8_enable_fp32_cpu_offload: False
- llm_int8_has_fp16_weight: False
- bnb_4bit_quant_type: nf4
- bnb_4bit_use_double_quant: True
- bnb_4bit_compute_dtype: bfloat16
## Model Description
This is an SFT(Supervised Fine-Tuned) Model meant for SQl-based text generation tasks. We have used the LoRa(Low-Ranking Adaptors) method for Fine-Tuning.
## Model Summary
train/loss : 0.4354
train/learning_rate: 0.00017567567567567568
train/epoch : 5.0
train/global_step : 10
## Inference Code
After doing necessary imports
device_map = {"": 0}
model_id = "mistralai/Mistral-7B-v0.1"
new_model = "Akil15/mistral_SQL_v.0.1"
# Reload model in FP16 and merge it with LoRA weights
base_model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.float16,
device_map=device_map,
)
model = PeftModel.from_pretrained(base_model, new_model)
model = model.merge_and_unload()
# Reload tokenizer to save it
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
tokenizer.pad_token = tokenizer.eos_token
tokenizer.padding_side = "right"
sample text(example):
text = """Question: How many vegetable farms with over 100 acres of cultivated land utilize organic farming methods, and what is the average yield per acre for these farms?
Context:CREATE TABLE vegetable_farm (Acres INTEGER,Organic BOOLEAN,Yield_Per_Acre DECIMAL);"""
text = input()
inputs = tokenizer(text, return_tensors="pt").to(device)
outputs = model.generate(**inputs, max_new_tokens=20)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Note: Change the max_new_tokens length based on the question-context text input or just define it to 100
### Framework versions
- PEFT 0.4.0 |