emre570 commited on
Commit
ca2b2f4
1 Parent(s): 11822f6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +42 -1
README.md CHANGED
@@ -25,6 +25,47 @@ You can access the fine-tuning code below.
25
  ## Example Usage
26
  You can use the adapter model with PEFT.
27
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
28
  ### **Important Notes**
29
- - You should use an Alpaca Prompt Template or another template, otherwise you can see generations with no meanings or repeating the same sentence constantly.
30
  - Use the model with a CUDA supported GPU.
 
25
  ## Example Usage
26
  You can use the adapter model with PEFT.
27
 
28
+ `from peft import PeftModel, PeftConfig
29
+ from transformers import AutoModelForCausalLM, AutoTokenizer
30
+
31
+ base_model = AutoModelForCausalLM.from_pretrained("unsloth/llama-3-8b-bnb-4bit")
32
+ model = PeftModel.from_pretrained(base_model, "myzens/llama3-8b-tr-finetuned")
33
+ tokenizer = AutoTokenizer.from_pretrained("myzens/llama3-8b-tr-finetuned")
34
+
35
+ alpaca_prompt = """
36
+ Instruction:
37
+ {}
38
+
39
+ Input:
40
+ {}
41
+
42
+ Response:
43
+ {}"""
44
+
45
+ inputs = tokenizer([
46
+ alpaca_prompt.format(
47
+ "",
48
+ "Ankara'da gezilebilecek 3 yeri söyle ve ne olduklarını kısaca açıkla.",
49
+ "",
50
+ )], return_tensors = "pt").to("cuda")
51
+
52
+
53
+ outputs = model.generate(**inputs, max_new_tokens=256)
54
+ print(tokenizer.decode(outputs[0], skip_special_tokens=True))`
55
+
56
+ Output:
57
+
58
+ `Instruction:
59
+
60
+
61
+ Input:
62
+ Ankara'da gezilebilecek 3 yeri söyle ve ne olduklarını kısaca açıkla.
63
+
64
+ Response:
65
+ 1. Anıtkabir - Mustafa Kemal Atatürk'ün mezarı
66
+ 2. Gençlik ve Spor Sarayı - spor etkinliklerinin yapıldığı yer
67
+ 3. Kızılay Meydanı - Ankara'nın merkezinde bulunan bir meydan`
68
+
69
  ### **Important Notes**
70
+ - We recommend you to use an Alpaca Prompt Template or another template, otherwise you can see generations with no meanings or repeating the same sentence constantly.
71
  - Use the model with a CUDA supported GPU.