Afrizal Hasbi Azizy commited on
Commit
073bba8
โ€ข
1 Parent(s): 5776ba4

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -18
README.md CHANGED
@@ -19,25 +19,9 @@ language:
19
 
20
  Selamat datang!
21
 
22
- If you're like me, you love models that are:
23
 
24
- ๐Ÿค Small, but capable!
25
-
26
- ๐Ÿ”“ Open, free-to-use
27
-
28
- ๐Ÿ‡ฎ๐Ÿ‡ฉ Fluent in Indonesian
29
-
30
- That's why I'm proud to announce... the ๐ŸฆŒ Kancil! It's a fine-tuned version of Llama 3 8B with the TumpengQA, an instruction dataset of 28 million words. Both the model and dataset is openly available in Huggingface.
31
-
32
- What makes this model so cool? ๐Ÿคจ
33
-
34
- ๐Ÿ“š The dataset is synthetically generated from Llama 3 70B. A big problem with existing Indonesian instruction dataset is they're really badly translated versions of English datasets. Llama 3 70B can generate fluent Indonesian! (with minor caveats ๐Ÿ˜”)
35
-
36
- ๐Ÿ”จ Llama 3 8B can already respond in Indonesian... but it's highly inconsistent ๐Ÿ˜ญ and needs lots of tedious prompt engineering. This model is highly consistent in responding in Indonesian!
37
-
38
- How did I go about it?
39
-
40
- โœˆ Scaling up synthetic data generation! Companies like Microsoft and Meta realized it is absolutely essential for developing LMs. From this and previous experience in creating Jawa Krama dataset, this is surprisingly useful for low-medium resource languages.
41
 
42
  ๐Ÿฆš This was highly inspired by last year's efforts from Merak-7B, a collection of open, fine-tuned Indonesian models. However, Kancil leveraged synthetic data in a very creative way, which makes it unique from Merak!
43
 
 
19
 
20
  Selamat datang!
21
 
22
+ I'm super stoked to announce... the ๐ŸฆŒ Kancil! It's a fine-tuned version of Llama 3 8B with the TumpengQA, an instruction dataset of 28 million words. Both the model and dataset is openly available in Huggingface.
23
 
24
+ ๐Ÿ“š The dataset was synthetically generated from Llama 3 70B. A big problem with existing Indonesian instruction dataset is they're really badly translated versions of English datasets. Llama 3 70B can generate fluent Indonesian! (with minor caveats ๐Ÿ˜”)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
 
26
  ๐Ÿฆš This was highly inspired by last year's efforts from Merak-7B, a collection of open, fine-tuned Indonesian models. However, Kancil leveraged synthetic data in a very creative way, which makes it unique from Merak!
27