pgarbacki commited on
Commit
cab4c81
1 Parent(s): fe1446e

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +96 -0
README.md ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ tags:
4
+ - function-calling
5
+ ---
6
+
7
+ # Fireworks Function Calling (FireFunction) Model V1
8
+
9
+ FireFunction is a state-of-the-art function calling model with a commercially viable license.
10
+
11
+ 💡 The model is hosted on the [Fireworks](https://fireworks.ai/models/fireworks/firefunction-v1) platform, offering blazing fast inference and API compatible with [OpenAI function calling](https://platform.openai.com/docs/guides/function-calling).
12
+ ```sh
13
+ OPENAI_API_BASE=https://api.fireworks.ai/inference/v1
14
+ OPENAI_API_KEY=<YOUR_FIREWORKS_API_KEY>
15
+ MODEL=accounts/fireworks/models/firefunction-v1
16
+ ```
17
+
18
+ ## Intended Use and Limitations
19
+
20
+ ### Primary Use
21
+ Although the model was trained on a variety of tasks, it performs best on:
22
+ * single-turn request routing to a function picked from a pool of up to 20 function specs.
23
+ * structured information extraction.
24
+
25
+ ### Out-of-Scope Use
26
+ The model was not optimized for the following use cases:
27
+ * general multi-turn chat,
28
+ * parallel and nested function calls in a single response. These can be broken into multiple messages.
29
+
30
+ ## How to use the model
31
+
32
+ ```python
33
+ from transformers import AutoModelForCausalLM, AutoTokenizer
34
+ import json
35
+
36
+ device = "cuda" # the device to load the model onto
37
+
38
+ model = AutoModelForCausalLM.from_pretrained("fireworks-ai/firefunction-v1")
39
+ tokenizer = AutoTokenizer.from_pretrained("fireworks-ai/firefunction-v1")
40
+
41
+ function_spec = [
42
+ {
43
+ "name": "get_stock_price",
44
+ "description": "Get the current stock price",
45
+ "parameters": {
46
+ "type": "object",
47
+ "properties": {
48
+ "symbol": {
49
+ "type": "string",
50
+ "description": "The stock symbol, e.g. AAPL, GOOG"
51
+ }
52
+ },
53
+ "required": [
54
+ "symbol"
55
+ ]
56
+ }
57
+ },
58
+ {
59
+ "name": "check_word_anagram",
60
+ "description": "Check if two words are anagrams of each other",
61
+ "parameters": {
62
+ "type": "object",
63
+ "properties": {
64
+ "word1": {
65
+ "type": "string",
66
+ "description": "The first word"
67
+ },
68
+ "word2": {
69
+ "type": "string",
70
+ "description": "The second word"
71
+ }
72
+ },
73
+ "required": [
74
+ "word1",
75
+ "word2"
76
+ ]
77
+ }
78
+ }
79
+ ]
80
+ functions = json.dumps(functions, indent=4)
81
+
82
+ messages = [
83
+ {'role': 'functions', 'content': functions},
84
+ {'role': 'system', 'content': 'You are a helpful assistant with access to functions. Use them if required.'},
85
+ {'role': 'user', 'content': 'Hi, can you tell me the current stock price of AAPL?'}
86
+ ]
87
+
88
+ encoded = tokenizer.apply_chat_template(messages, return_tensors="pt")
89
+
90
+ model_inputs = encodeds.to(device)
91
+ model.to(device)
92
+
93
+ generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
94
+ decoded = tokenizer.batch_decode(generated_ids)
95
+ print(decoded[0])
96
+ ```