MaziyarPanahi commited on
Commit
cfc4a71
1 Parent(s): 07a942a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +112 -3
README.md CHANGED
@@ -1,3 +1,112 @@
1
- ---
2
- license: llama3
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: other
5
+ library_name: transformers
6
+ tags:
7
+ - axolotl
8
+ - finetune
9
+ - facebook
10
+ - meta
11
+ - pytorch
12
+ - llama
13
+ - llama-3
14
+ base_model: MaziyarPanahi/Llama-3-8B-Instruct-v0.9
15
+ model_name: Llama-3-8B-Instruct-v0.10
16
+ pipeline_tag: text-generation
17
+ license_name: llama3
18
+ license_link: LICENSE
19
+ inference: false
20
+ model_creator: MaziyarPanahi
21
+ ---
22
+
23
+ <img src="./llama-3-merges.webp" alt="Llama-3 DPO Logo" width="500" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
24
+
25
+
26
+ # Llama-3-8B-Instruct-v0.10
27
+
28
+ This model was developed based on `MaziyarPanahi/Llama-3-8B-Instruct-v0.9` model.
29
+
30
+ # ⚡ Quantized GGUF
31
+
32
+ All GGUF models are available here: [MaziyarPanahi/Llama-3-8B-Instruct-v0.10-GGUF](https://huggingface.co/MaziyarPanahi/Llama-3-8B-Instruct-v0.10-GGUF)
33
+
34
+ # 🏆 [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard)
35
+
36
+ coming soon!
37
+
38
+ # Prompt Template
39
+
40
+ This model uses `ChatML` prompt template:
41
+
42
+ ```
43
+ <|begin_of_text|><|start_header_id|>system<|end_header_id|>
44
+
45
+ {system_prompt}<|eot_id|><|start_header_id|>user<|end_header_id|>
46
+
47
+ {prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
48
+ ````
49
+
50
+ # How to use
51
+
52
+ You can use this model by using `MaziyarPanahi/Llama-3-8B-Instruct-v0.10` as the model name in Hugging Face's
53
+ transformers library.
54
+
55
+ ```python
56
+ from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
57
+ from transformers import pipeline
58
+ import torch
59
+
60
+ model_id = "MaziyarPanahi/Llama-3-8B-Instruct-v0.10"
61
+
62
+ model = AutoModelForCausalLM.from_pretrained(
63
+ model_id,
64
+ torch_dtype=torch.bfloat16,
65
+ device_map="auto",
66
+ trust_remote_code=True,
67
+ # attn_implementation="flash_attention_2"
68
+ )
69
+
70
+ tokenizer = AutoTokenizer.from_pretrained(
71
+ model_id,
72
+ trust_remote_code=True
73
+ )
74
+
75
+ streamer = TextStreamer(tokenizer)
76
+
77
+ pipeline = pipeline(
78
+ "text-generation",
79
+ model=model,
80
+ tokenizer=tokenizer,
81
+ model_kwargs={"torch_dtype": torch.bfloat16},
82
+ streamer=streamer
83
+ )
84
+
85
+ # Then you can use the pipeline to generate text.
86
+
87
+ messages = [
88
+ {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
89
+ {"role": "user", "content": "Who are you?"},
90
+ ]
91
+
92
+ prompt = tokenizer.apply_chat_template(
93
+ messages,
94
+ tokenize=False,
95
+ add_generation_prompt=True
96
+ )
97
+
98
+ terminators = [
99
+ tokenizer.eos_token_id,
100
+ tokenizer.convert_tokens_to_ids("<|eot_id|>")
101
+ ]
102
+
103
+ outputs = pipeline(
104
+ prompt,
105
+ max_new_tokens=512,
106
+ eos_token_id=terminators,
107
+ do_sample=True,
108
+ temperature=0.6,
109
+ top_p=0.105,
110
+ )
111
+ print(outputs[0]["generated_text"][len(prompt):])
112
+ ```