OrionZheng
commited on
Commit
•
edf2730
1
Parent(s):
63dd209
Upload README.md
Browse files
README.md
CHANGED
@@ -19,9 +19,9 @@ As a small student team, instead of pursuing the best model with better data, co
|
|
19 |
|
20 |
|
21 |
## Model Weights
|
22 |
-
Currently, three models are released in total: OpenMoE-base, OpenMoE-8B
|
23 |
|
24 |
-
The table below lists the 8B model that has completed training
|
25 |
|
26 |
| Model Name | Description | #Param |Huggingface |
|
27 |
|----------------|-------------------------------------------------|----------|-------------|
|
@@ -29,7 +29,7 @@ The table below lists the 8B model that has completed training, along with its c
|
|
29 |
| **OpenMoE-8B-Chat (1.1T+SFT)** | OpenMoE-8B-1.1T supervised finetuned on the [WildChat GPT-4 Subset](https://huggingface.co/datasets/allenai/WildChat-nontoxic) |8B |[Link](https://huggingface.co/OrionZheng/openmoe-8b-chat) |
|
30 |
|
31 |
|
32 |
-
Besides, we also provide our intermediate checkpoints(base, 8B, 34B) for research purposes.
|
33 |
|
34 |
| Model Name | Description | #Param |Huggingface |
|
35 |
|----------------|-------------------------------------------------|----------|-------------|
|
|
|
19 |
|
20 |
|
21 |
## Model Weights
|
22 |
+
Currently, three models are released in total: OpenMoE-base, OpenMoE-8B/8B-Chat, and OpenMoE-34B(at 200B tokens).
|
23 |
|
24 |
+
The table below lists the 8B/8B-Chat model that has completed training on 1.1T tokens.
|
25 |
|
26 |
| Model Name | Description | #Param |Huggingface |
|
27 |
|----------------|-------------------------------------------------|----------|-------------|
|
|
|
29 |
| **OpenMoE-8B-Chat (1.1T+SFT)** | OpenMoE-8B-1.1T supervised finetuned on the [WildChat GPT-4 Subset](https://huggingface.co/datasets/allenai/WildChat-nontoxic) |8B |[Link](https://huggingface.co/OrionZheng/openmoe-8b-chat) |
|
30 |
|
31 |
|
32 |
+
Besides, we also provide all our intermediate checkpoints(base, 8B, 34B) for research purposes.
|
33 |
|
34 |
| Model Name | Description | #Param |Huggingface |
|
35 |
|----------------|-------------------------------------------------|----------|-------------|
|