MaziyarPanahi commited on
Commit
a2be865
1 Parent(s): 0e20fa4

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +137 -0
README.md ADDED
@@ -0,0 +1,137 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ language:
4
+ - en
5
+ - fr
6
+ - es
7
+ - de
8
+ - it
9
+ ---
10
+
11
+
12
+
13
+ Original README
14
+ ---
15
+
16
+ # Model Card for Mixtral-8x22B-Instruct-v0.1
17
+ The Mixtral-8x22B-Instruct-v0.1 Large Language Model (LLM) is an instruct fine-tuned version of the [Mixtral-8x22B-v0.1](https://huggingface.co/mistralai/Mixtral-8x22B-v0.1).
18
+
19
+ ## Run the model
20
+ ```python
21
+ from transformers import AutoModelForCausalLM
22
+ from mistral_common.protocol.instruct.messages import (
23
+ AssistantMessage,
24
+ UserMessage,
25
+ )
26
+ from mistral_common.protocol.instruct.tool_calls import (
27
+ Tool,
28
+ Function,
29
+ )
30
+ from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
31
+ from mistral_common.tokens.instruct.normalize import ChatCompletionRequest
32
+
33
+ device = "cuda" # the device to load the model onto
34
+
35
+ tokenizer_v3 = MistralTokenizer.v3()
36
+
37
+ mistral_query = ChatCompletionRequest(
38
+ tools=[
39
+ Tool(
40
+ function=Function(
41
+ name="get_current_weather",
42
+ description="Get the current weather",
43
+ parameters={
44
+ "type": "object",
45
+ "properties": {
46
+ "location": {
47
+ "type": "string",
48
+ "description": "The city and state, e.g. San Francisco, CA",
49
+ },
50
+ "format": {
51
+ "type": "string",
52
+ "enum": ["celsius", "fahrenheit"],
53
+ "description": "The temperature unit to use. Infer this from the users location.",
54
+ },
55
+ },
56
+ "required": ["location", "format"],
57
+ },
58
+ )
59
+ )
60
+ ],
61
+ messages=[
62
+ UserMessage(content="What's the weather like today in Paris"),
63
+ ],
64
+ model="test",
65
+ )
66
+
67
+ encodeds = tokenizer_v3.encode_chat_completion(mistral_query).tokens
68
+ model = AutoModelForCausalLM.from_pretrained("mistralai/Mixtral-8x22B-Instruct-v0.1")
69
+ model_inputs = encodeds.to(device)
70
+ model.to(device)
71
+
72
+ generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
73
+ sp_tokenizer = tokenizer_v3.instruct_tokenizer.tokenizer
74
+ decoded = sp_tokenizer.decode(generated_ids[0])
75
+ print(decoded)
76
+ ```
77
+
78
+ # Instruct tokenizer
79
+ The HuggingFace tokenizer included in this release should match our own. To compare:
80
+ `pip install mistral-common`
81
+
82
+ ```py
83
+ from mistral_common.protocol.instruct.messages import (
84
+ AssistantMessage,
85
+ UserMessage,
86
+ )
87
+ from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
88
+ from mistral_common.tokens.instruct.normalize import ChatCompletionRequest
89
+
90
+ from transformers import AutoTokenizer
91
+
92
+ tokenizer_v3 = MistralTokenizer.v3()
93
+
94
+ mistral_query = ChatCompletionRequest(
95
+ messages=[
96
+ UserMessage(content="How many experts ?"),
97
+ AssistantMessage(content="8"),
98
+ UserMessage(content="How big ?"),
99
+ AssistantMessage(content="22B"),
100
+ UserMessage(content="Noice 🎉 !"),
101
+ ],
102
+ model="test",
103
+ )
104
+ hf_messages = mistral_query.model_dump()['messages']
105
+
106
+ tokenized_mistral = tokenizer_v3.encode_chat_completion(mistral_query).tokens
107
+
108
+ tokenizer_hf = AutoTokenizer.from_pretrained('mistralai/Mixtral-8x22B-Instruct-v0.1')
109
+ tokenized_hf = tokenizer_hf.apply_chat_template(hf_messages, tokenize=True)
110
+
111
+ assert tokenized_hf == tokenized_mistral
112
+ ```
113
+
114
+ # Function calling and special tokens
115
+ This tokenizer includes more special tokens, related to function calling :
116
+ - [TOOL_CALLS]
117
+ - [AVAILABLE_TOOLS]
118
+ - [/AVAILABLE_TOOLS]
119
+ - [TOOL_RESULT]
120
+ - [/TOOL_RESULTS]
121
+
122
+ If you want to use this model with function calling, please be sure to apply it similarly to what is done in our [SentencePieceTokenizerV3](https://github.com/mistralai/mistral-common/blob/main/src/mistral_common/tokens/tokenizers/sentencepiece.py#L299).
123
+
124
+ # The Mistral AI Team
125
+ Albert Jiang, Alexandre Sablayrolles, Alexis Tacnet, Antoine Roux,
126
+ Arthur Mensch, Audrey Herblin-Stoop, Baptiste Bout, Baudouin de Monicault,
127
+ Blanche Savary, Bam4d, Caroline Feldman, Devendra Singh Chaplot,
128
+ Diego de las Casas, Eleonore Arcelin, Emma Bou Hanna, Etienne Metzger,
129
+ Gianna Lengyel, Guillaume Bour, Guillaume Lample, Harizo Rajaona,
130
+ Jean-Malo Delignon, Jia Li, Justus Murke, Louis Martin, Louis Ternon,
131
+ Lucile Saulnier, Lélio Renard Lavaud, Margaret Jennings, Marie Pellat,
132
+ Marie Torelli, Marie-Anne Lachaux, Nicolas Schuhl, Patrick von Platen,
133
+ Pierre Stock, Sandeep Subramanian, Sophia Yang, Szymon Antoniak, Teven Le Scao,
134
+ Thibaut Lavril, Timothée Lacroix, Théophile Gervet, Thomas Wang,
135
+ Valera Nemychnikova, William El Sayed, William Marshall
136
+
137
+ ---