Update README.md
Browse files
README.md
CHANGED
@@ -21,15 +21,15 @@ This model is intended for various NLP tasks that require understanding and gene
|
|
21 |
Bahasa-4b was trained on a 10 billion subset data of Indonesian dataset from a collected pool of 100 billion.
|
22 |
|
23 |
## Benchmarks
|
24 |
-
The following table shows the performance of Bahasa-4b compared to
|
25 |
-
|
26 |
-
| Dataset | Version | Metric | Mode | Sailor_4b | Bahasa-4b-hf |
|
27 |
-
|
28 |
-
| tydiqa-id | 0e9309 | EM | gen | 53.98 | 55.04 |
|
29 |
-
| tydiqa-id | 0e9309 | F1 | gen | 73.48 | 75.39 |
|
30 |
-
| xcopa-id | 36c11c | EM | ppl | 69.2 | 73.2 |
|
31 |
-
| xcopa-id | 36c11c | F1 | ppl | 69.2 | 73.2 |
|
32 |
-
| m3exam-id-ppl | ede415 | EM | ppl | 31.27 | 44.47 |
|
33 |
-
| belebele-id-ppl| 7fe030 | EM | ppl | 41.33 | 42.33 |
|
34 |
-
|
35 |
-
This data demonstrates that Bahasa-4b consistently outperforms the
|
|
|
21 |
Bahasa-4b was trained on a 10 billion subset data of Indonesian dataset from a collected pool of 100 billion.
|
22 |
|
23 |
## Benchmarks
|
24 |
+
The following table shows the performance of Bahasa-4b compared to the models Sailor_4b and Mistral-7B-v0.1 across several benchmarks:
|
25 |
+
|
26 |
+
| Dataset | Version | Metric | Mode | Sailor_4b | Bahasa-4b-hf | Mistral-7B-v0.1 |
|
27 |
+
|----------------|---------|--------|------|-----------|--------------|-----------------|
|
28 |
+
| tydiqa-id | 0e9309 | EM | gen | 53.98 | 55.04 | 63.54 |
|
29 |
+
| tydiqa-id | 0e9309 | F1 | gen | 73.48 | 75.39 | 78.73 |
|
30 |
+
| xcopa-id | 36c11c | EM | ppl | 69.2 | 73.2 | 62.40 |
|
31 |
+
| xcopa-id | 36c11c | F1 | ppl | 69.2 | 73.2 | - |
|
32 |
+
| m3exam-id-ppl | ede415 | EM | ppl | 31.27 | 44.47 | 26.68 |
|
33 |
+
| belebele-id-ppl| 7fe030 | EM | ppl | 41.33 | 42.33 | 41.33 |
|
34 |
+
|
35 |
+
This data demonstrates that Bahasa-4b consistently outperforms the Sailor_4b model in various Indonesian language tasks, showing improvements in both EM (Exact Match) and F1 scores across different datasets, and is competitive with the Mistral-7B-v0.1 model.
|