KobayashiKanna01
commited on
Commit
•
78b2c29
1
Parent(s):
bddbe90
Update README.md
Browse files
README.md
CHANGED
@@ -5,14 +5,14 @@ datasets:
|
|
5 |
- uonlp/CulturaX
|
6 |
language:
|
7 |
- en
|
8 |
-
-
|
9 |
base_model:
|
10 |
- meta-llama/Llama-2-7b-hf
|
11 |
---
|
12 |
|
13 |
A continually pre-trained model based on Llama-2-7b-hf.
|
14 |
|
15 |
-
We use the **
|
16 |
|
17 |
#### Hyper-parameters:
|
18 |
* lr: 3e-5
|
|
|
5 |
- uonlp/CulturaX
|
6 |
language:
|
7 |
- en
|
8 |
+
- ta
|
9 |
base_model:
|
10 |
- meta-llama/Llama-2-7b-hf
|
11 |
---
|
12 |
|
13 |
A continually pre-trained model based on Llama-2-7b-hf.
|
14 |
|
15 |
+
We use the **Tamil texts** in CulturaX and **English texts** in RedPajama with a proportion of **4:1** for training.
|
16 |
|
17 |
#### Hyper-parameters:
|
18 |
* lr: 3e-5
|