acul3 commited on
Commit
24ce9a5
1 Parent(s): 6471e72

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -12
README.md CHANGED
@@ -21,15 +21,15 @@ This model is intended for various NLP tasks that require understanding and gene
21
  Bahasa-4b was trained on a 10 billion subset data of Indonesian dataset from a collected pool of 100 billion.
22
 
23
  ## Benchmarks
24
- The following table shows the performance of Bahasa-4b compared to another 4b model, sailor_4b, across several benchmarks:
25
-
26
- | Dataset | Version | Metric | Mode | Sailor_4b | Bahasa-4b-hf |
27
- |----------------|---------|--------|------|-----------|--------------|
28
- | tydiqa-id | 0e9309 | EM | gen | 53.98 | 55.04 |
29
- | tydiqa-id | 0e9309 | F1 | gen | 73.48 | 75.39 |
30
- | xcopa-id | 36c11c | EM | ppl | 69.2 | 73.2 |
31
- | xcopa-id | 36c11c | F1 | ppl | 69.2 | 73.2 |
32
- | m3exam-id-ppl | ede415 | EM | ppl | 31.27 | 44.47 |
33
- | belebele-id-ppl| 7fe030 | EM | ppl | 41.33 | 42.33 |
34
-
35
- This data demonstrates that Bahasa-4b consistently outperforms the sailor_4b model in various Indonesian language tasks, showing improvements in both EM (Exact Match) and F1 scores across different datasets.
 
21
  Bahasa-4b was trained on a 10 billion subset data of Indonesian dataset from a collected pool of 100 billion.
22
 
23
  ## Benchmarks
24
+ The following table shows the performance of Bahasa-4b compared to the models Sailor_4b and Mistral-7B-v0.1 across several benchmarks:
25
+
26
+ | Dataset | Version | Metric | Mode | Sailor_4b | Bahasa-4b-hf | Mistral-7B-v0.1 |
27
+ |----------------|---------|--------|------|-----------|--------------|-----------------|
28
+ | tydiqa-id | 0e9309 | EM | gen | 53.98 | 55.04 | 63.54 |
29
+ | tydiqa-id | 0e9309 | F1 | gen | 73.48 | 75.39 | 78.73 |
30
+ | xcopa-id | 36c11c | EM | ppl | 69.2 | 73.2 | 62.40 |
31
+ | xcopa-id | 36c11c | F1 | ppl | 69.2 | 73.2 | - |
32
+ | m3exam-id-ppl | ede415 | EM | ppl | 31.27 | 44.47 | 26.68 |
33
+ | belebele-id-ppl| 7fe030 | EM | ppl | 41.33 | 42.33 | 41.33 |
34
+
35
+ This data demonstrates that Bahasa-4b consistently outperforms the Sailor_4b model in various Indonesian language tasks, showing improvements in both EM (Exact Match) and F1 scores across different datasets, and is competitive with the Mistral-7B-v0.1 model.