akjindal53244
commited on
Commit
•
44f9bb9
1
Parent(s):
0ff6652
Update README.md
Browse files
README.md
CHANGED
@@ -25,6 +25,8 @@ Authors: [Ashvini Kumar Jindal](https://www.linkedin.com/in/ashvini-jindal-26653
|
|
25 |
|
26 |
**🤗 Hugging Face Announcement Blog**: https://huggingface.co/blog/akjindal53244/llama31-storm8b
|
27 |
|
|
|
|
|
28 |
<br>
|
29 |
|
30 |
# Llama-3.1-Storm-8B-GGUF
|
@@ -95,6 +97,7 @@ Llama-3.1-Storm-8B is a powerful generalist model useful for diverse application
|
|
95 |
1. `BF16`: [Llama-3.1-Storm-8B](https://huggingface.co/akjindal53244/Llama-3.1-Storm-8B)
|
96 |
2. ⚡ `FP8`: [Llama-3.1-Storm-8B-FP8-Dynamic](https://huggingface.co/akjindal53244/Llama-3.1-Storm-8B-FP8-Dynamic)
|
97 |
3. ⚡ `GGUF`: [Llama-3.1-Storm-8B-GGUF](https://huggingface.co/akjindal53244/Llama-3.1-Storm-8B-GGUF)
|
|
|
98 |
|
99 |
## 💻 How to Use GGUF Model
|
100 |
|
@@ -131,6 +134,55 @@ res = llm(prompt, **generation_kwargs)
|
|
131 |
print(res["choices"][0]["text"])
|
132 |
```
|
133 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
134 |
|
135 |
## Alignment Note
|
136 |
While **Llama-3.1-Storm-8B** did not undergo an explicit model alignment process, it may still retain some alignment properties inherited from the Meta-Llama-3.1-8B-Instruct model.
|
|
|
25 |
|
26 |
**🤗 Hugging Face Announcement Blog**: https://huggingface.co/blog/akjindal53244/llama31-storm8b
|
27 |
|
28 |
+
**🚀Ollama:** `ollama run ajindal/llama3.1-storm:8b`
|
29 |
+
|
30 |
<br>
|
31 |
|
32 |
# Llama-3.1-Storm-8B-GGUF
|
|
|
97 |
1. `BF16`: [Llama-3.1-Storm-8B](https://huggingface.co/akjindal53244/Llama-3.1-Storm-8B)
|
98 |
2. ⚡ `FP8`: [Llama-3.1-Storm-8B-FP8-Dynamic](https://huggingface.co/akjindal53244/Llama-3.1-Storm-8B-FP8-Dynamic)
|
99 |
3. ⚡ `GGUF`: [Llama-3.1-Storm-8B-GGUF](https://huggingface.co/akjindal53244/Llama-3.1-Storm-8B-GGUF)
|
100 |
+
4. 🚀 Ollama: `ollama run ajindal/llama3.1-storm:8b`
|
101 |
|
102 |
## 💻 How to Use GGUF Model
|
103 |
|
|
|
134 |
print(res["choices"][0]["text"])
|
135 |
```
|
136 |
|
137 |
+
### Function Calling Example with [Ollama](https://ollama.com/)
|
138 |
+
```
|
139 |
+
import ollama
|
140 |
+
tools = [{
|
141 |
+
'type': 'function',
|
142 |
+
'function': {
|
143 |
+
'name': 'get_current_weather',
|
144 |
+
'description': 'Get the current weather for a city',
|
145 |
+
'parameters': {
|
146 |
+
'type': 'object',
|
147 |
+
'properties': {
|
148 |
+
'city': {
|
149 |
+
'type': 'string',
|
150 |
+
'description': 'The name of the city',
|
151 |
+
},
|
152 |
+
},
|
153 |
+
'required': ['city'],
|
154 |
+
},
|
155 |
+
},
|
156 |
+
},
|
157 |
+
{
|
158 |
+
'type': 'function',
|
159 |
+
'function': {
|
160 |
+
'name': 'get_places_to_vist',
|
161 |
+
'description': 'Get places to visit in a city',
|
162 |
+
'parameters': {
|
163 |
+
'type': 'object',
|
164 |
+
'properties': {
|
165 |
+
'city': {
|
166 |
+
'type': 'string',
|
167 |
+
'description': 'The name of the city',
|
168 |
+
},
|
169 |
+
},
|
170 |
+
'required': ['city'],
|
171 |
+
},
|
172 |
+
},
|
173 |
+
},
|
174 |
+
]
|
175 |
+
response = ollama.chat(
|
176 |
+
model='ajindal/llama3.1-storm:8b',
|
177 |
+
messages=[
|
178 |
+
{'role': 'system', 'content': 'Do not answer to nay vulgar questions.'},
|
179 |
+
{'role': 'user', 'content': 'What is the weather in Toronto and San Francisco?'}
|
180 |
+
],
|
181 |
+
tools=tools
|
182 |
+
)
|
183 |
+
print(response['message']) # Expected Response: {'role': 'assistant', 'content': "<tool_call>{'tool_name': 'get_current_weather', 'tool_arguments': {'city': 'Toronto'}}</tool_call>"}
|
184 |
+
```
|
185 |
+
|
186 |
|
187 |
## Alignment Note
|
188 |
While **Llama-3.1-Storm-8B** did not undergo an explicit model alignment process, it may still retain some alignment properties inherited from the Meta-Llama-3.1-8B-Instruct model.
|