pgarbacki commited on
Commit
b0702d8
1 Parent(s): 2082258

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -11
README.md CHANGED
@@ -18,14 +18,7 @@ FireFunction is a state-of-the-art function calling model with a commercially vi
18
 
19
  💡 Hosted on the [Fireworks](https://fireworks.ai/models/fireworks/firefunction-v2) platform
20
 
21
- ## Resources
22
- * [Fireworks discord with function calling channel](https://discord.gg/mMqQxvFD9A)
23
- * [Documentation](https://readme.fireworks.ai/docs/function-calling)
24
- * [UI Demo app](https://functional-chat.vercel.app/)
25
- * [Try in Fireworks prompt playground UI](https://fireworks.ai/models/fireworks/firefunction-v2)
26
-
27
-
28
- # Intended Use and Limitations
29
 
30
  ### Supported usecases
31
  The model was tuned to perfom well on a range of usecases including:
@@ -40,6 +33,18 @@ The model was not optimized for the following use cases:
40
  * 100+ function specs
41
  * nested function calling
42
 
 
 
 
 
 
 
 
 
 
 
 
 
43
  ## Example Usage
44
 
45
  See [documentation](https://readme.fireworks.ai/docs/function-calling) for more detail.
@@ -110,6 +115,8 @@ decoded = tokenizer.batch_decode(generated_ids)
110
  print(decoded[0])
111
  ```
112
 
113
- ## Demo App
114
-
115
- Check our easy-to-extend [demo chat app](https://github.com/fw-ai/forge/tree/main/apps/functional_chat) with function calling capabilities built on Firefunction model.
 
 
 
18
 
19
  💡 Hosted on the [Fireworks](https://fireworks.ai/models/fireworks/firefunction-v2) platform
20
 
21
+ ## Intended Use and Limitations
 
 
 
 
 
 
 
22
 
23
  ### Supported usecases
24
  The model was tuned to perfom well on a range of usecases including:
 
33
  * 100+ function specs
34
  * nested function calling
35
 
36
+ ## Metrics
37
+
38
+ | | Firefunction v1 | Firefunction v2 | Llama 3 70b Instruct | Gpt-4o |
39
+ |------------------------------------|-----------------|-----------------|----------------------|--------|
40
+ | Gorilla simple | 0.91 | 0.94 | 0.925 | 0.88 |
41
+ | Gorilla multiple_function | 0.92 | 0.91 | 0.86 | 0.91 |
42
+ | Gorilla parallel_function | 0 | 0.9 | 0.86 | 0.89 |
43
+ | Gorilla parallel_multiple_function | 0 | 0.8 | 0.615 | 0.72 |
44
+ | Nexus parallel | 0.38 | 0.53 | 0.3 | 0.47 |
45
+ | Mtbench | 0.73 | 0.84 | 0.89 | 0.93 |
46
+ | Average | 0.49 | 0.82 | 0.74 | 0.8 |
47
+
48
  ## Example Usage
49
 
50
  See [documentation](https://readme.fireworks.ai/docs/function-calling) for more detail.
 
115
  print(decoded[0])
116
  ```
117
 
118
+ ## Resources
119
+ * [Fireworks discord with function calling channel](https://discord.gg/mMqQxvFD9A)
120
+ * [Documentation](https://readme.fireworks.ai/docs/function-calling)
121
+ * [Demo app](https://functional-chat.vercel.app/)
122
+ * [Try in Fireworks prompt playground UI](https://fireworks.ai/models/fireworks/firefunction-v2)