mkly
/

PEFT
English
mkly commited on
Commit
ae2afc1
1 Parent(s): 8d429ed

Update README with usage

Browse files
Files changed (1) hide show
  1. README.md +22 -1
README.md CHANGED
@@ -45,4 +45,25 @@ TRON is a blockchain-based operating system that eliminates the middleman, reduc
45
  who founded the roanoke settlement?
46
 
47
  ### ANSWER
48
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
45
  who founded the roanoke settlement?
46
 
47
  ### ANSWER
48
+ ```
49
+
50
+ ## Usage
51
+
52
+ ```python
53
+ base_model_name = "meta-llama/Llama-2-7b-chat-hf"
54
+
55
+ bnb_config = BitsAndBytesConfig(
56
+ load_in_4bit=True,
57
+ bnb_4bit_quant_type="nf4",
58
+ bnb_4bit_compute_dtype=torch.bfloat16,
59
+ )
60
+
61
+ base_model = AutoModelForCausalLM.from_pretrained(
62
+ base_model_name,
63
+ quantization_config=bnb_config,
64
+ device_map="auto",
65
+ trust_remote_code=True,
66
+ )
67
+
68
+ model = PeftModel.from_pretrained(base_model, "mkly/crypto-sales")
69
+ ```