Update README.md
Browse files
README.md
CHANGED
@@ -24,6 +24,18 @@ In extensive testing and benchmarks, Omnia has proven to be an exceptionally str
|
|
24 |
## Benchmark Results
|
25 |
|
26 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
27 |
**Benchmark Metrics:**
|
28 |
|
29 |
### Complexity
|
@@ -49,4 +61,6 @@ Similarity: 0.969
|
|
49 |
|
50 |
```
|
51 |
|
52 |
-
*While the benchmark provides some insights, it is important to consider that the specific undisclosed details of the benchmark may introduce biases. Therefore, it is recommended to take this result with a grain of salt for now.*
|
|
|
|
|
|
24 |
## Benchmark Results
|
25 |
|
26 |
|
27 |
+
| Model | average | complexity | contextual maintenance | similarity to ground truth |
|
28 |
+
| -------------------------------------- | -------- | ---------- | ---------------------- | -------------------------- |
|
29 |
+
| **Omnia-2x7B** | **82.2** | **55.5** | **95.9** | **95.3** |
|
30 |
+
| Omnia-7B | 81.8 | 55.0 | 95.7 | 94.8 |
|
31 |
+
| Antler-RP-ja-westlake-chatvector | 81.6 | 54.2 | 95.6 | 95.0 |
|
32 |
+
| LightChatAssistant-TypeB-2x7B | 81.5 | 55.1 | 95.0 | 94.5 |
|
33 |
+
| Antler-7B | 81.5 | 53.7 | 95.8 | 95.1 |
|
34 |
+
| chatntq-ja-7b-v1.0-westlake-chatvector | 79.7 | 49.8 | 95.0 | 94.3 |
|
35 |
+
| chatntq-ja-7b-v1.0 | 79.7 | 52.0 | 93.9 | 93.2 |
|
36 |
+
|
37 |
+
|
38 |
+
|
39 |
**Benchmark Metrics:**
|
40 |
|
41 |
### Complexity
|
|
|
61 |
|
62 |
```
|
63 |
|
64 |
+
*While the benchmark provides some insights, it is important to consider that the specific undisclosed details of the benchmark may introduce biases. Therefore, it is recommended to take this result with a grain of salt for now.*
|
65 |
+
|
66 |
+
If you want to know the full generation results of the benchmark, please contact me at [mazikat@na-cat.com](mailto:mazikat@na-cat.com)
|