DopeorNope
commited on
Commit
•
ba3a216
1
Parent(s):
af0633b
Update README.md
Browse files
README.md
CHANGED
@@ -24,9 +24,9 @@ license: cc-by-nc-sa-4.0
|
|
24 |
**Output** Models generate text only.
|
25 |
|
26 |
**Model Architecture**
|
27 |
-
COKAL-
|
28 |
|
29 |
-
**Base Model** [DopeorNope/
|
30 |
|
31 |
DopeorNope/COKAL_pre_DPO_Test_v2-13b is the SFT model to train with DPO methodology.
|
32 |
|
@@ -40,7 +40,7 @@ This dataset was constructed by directly collecting and reorganizing data by Dop
|
|
40 |
This dataset is based on ["kyujinpy/OpenOrca-KO"](https://huggingface.co/datasets/kyujinpy/OpenOrca-KO) and has been processed using the Near Dedup algorithm to remove items with a Jaccard Similarity threshold of 0.8 or higher. In addition, inconsistent inputs have been cleaned and modified.
|
41 |
|
42 |
**Training**
|
43 |
-
The difference between "DopeorNope/COKAL-DPO_test-v2" and this model is that this model has different
|
44 |
|
45 |
I developed the model in an environment with four RTX 3090 GPUs running Ubuntu 18.04.
|
46 |
|
|
|
24 |
**Output** Models generate text only.
|
25 |
|
26 |
**Model Architecture**
|
27 |
+
COKAL-DPO_13b-v2 is an auto-regressive 13B language model based on the LLaMA2 transformer architecture.
|
28 |
|
29 |
+
**Base Model** [DopeorNope/COKAL_pre_DPO_Test_v2-13b](https://huggingface.co/DopeorNope/COKAL_pre_DPO_Test_v2-13b)
|
30 |
|
31 |
DopeorNope/COKAL_pre_DPO_Test_v2-13b is the SFT model to train with DPO methodology.
|
32 |
|
|
|
40 |
This dataset is based on ["kyujinpy/OpenOrca-KO"](https://huggingface.co/datasets/kyujinpy/OpenOrca-KO) and has been processed using the Near Dedup algorithm to remove items with a Jaccard Similarity threshold of 0.8 or higher. In addition, inconsistent inputs have been cleaned and modified.
|
41 |
|
42 |
**Training**
|
43 |
+
The difference between "DopeorNope/COKAL-DPO_test-v2" and this model is that this model has different hyper-parameters from the one in that setting regarding the final version.
|
44 |
|
45 |
I developed the model in an environment with four RTX 3090 GPUs running Ubuntu 18.04.
|
46 |
|