airesearch
/

WangchanLion7B

Text Generation

text-generation-inference

Model card Files Files and versions Community

Rasu23 commited on Feb 27

Commit

f0a4dc0

•

1 Parent(s): 729a5f4

Update README.md

Files changed (1) hide show

README.md +11 -11

README.md CHANGED Viewed

@@ -92,19 +92,19 @@ Finetuning datasets are sourced from [LAION OIG chip2 and infill_dbpedia (Apache
 # Evaluation
 We performed human and machine evaluations on XQuAD zero-shot and one-shot settings:
 ## XQuAD
-|      Model     | Exact Match (Zero-shot) | F1 (Zero-shot) | Exact Match (One-shot) | F1 (One-shot) |
-|:--------------:|:-----------------------:|:--------------:|:----------------------:|:-------------:|
-| openthaigpt7B  |         18.57           |     28.40      |         30.42          |    39.76      |
-| SeaLLM7B       |           6.30          |    21.67       |      34.45         |  47.80    |
-| Typhoon-7b     |         -               |     34.46      |            -           |    **54.03**      |
-| WangchanLion7B |        **37.56**        |   **49.84**    |         **39.24**          |    51.07      |
 ## iAPP Wiki QA
-|      Model     | Exact Match (Zero-shot) | F1 (Zero-shot) | Exact Match (One-shot) | F1 (One-shot) |
-|:--------------:|:-----------------------:|:--------------:|:----------------------:|:-------------:|
-| openthaigpt7B  |         22.06         |     40.07    |         31.39        |    47.98    |
-| SeaLLM7B       |          8.25         |     34.40    |         40.05        |    58.27    |
-| WangchanLion7B |       **55.48**       |   **67.93**  |       **56.43**      |  **68.85**  |
 # What WangchanLion offers:
 - Transparent pretrained model: The development of SEA-LION is community-driven, with different ASEAN collaborators contributing pretraining datasets. The SEA-LION developers ensure that all datasets are safe and can be utilized without commercial restrictions. This transparency extends to the provision of pretraining code, ensuring anyone can replicate SEA-LION using the provided datasets.

 # Evaluation
 We performed human and machine evaluations on XQuAD zero-shot and one-shot settings:
 ## XQuAD
+|      Model     | F1 (Zero-shot) | F1 (One-shot) |
+|:--------------:|:--------------:|:-------------:|
+| openthaigpt7B  |     27.3487      |    34.3104      |
+| SeaLLM7B V2      |    16.1104       |  25.7399    |
+| Typhoon-7b     |     34.46      |    **54.03**      |
+| WangchanLion7B |   **45.8763**    |    49.9145      |
 ## iAPP Wiki QA
+|      Model     | F1 (Zero-shot) |  F1 (One-shot) |
+|:--------------:|:--------------:|:-------------:|
+| openthaigpt7B  |     40.0614    |    46.6883    |
+| SeaLLM7B V2      |     23.6425    |    28.9934    |
+| WangchanLion7B |   **58.9051**  |  **62.9776**  |
 # What WangchanLion offers:
 - Transparent pretrained model: The development of SEA-LION is community-driven, with different ASEAN collaborators contributing pretraining datasets. The SEA-LION developers ensure that all datasets are safe and can be utilized without commercial restrictions. This transparency extends to the provision of pretraining code, ensuring anyone can replicate SEA-LION using the provided datasets.