Update README.md
Browse files
README.md
CHANGED
@@ -16,7 +16,7 @@ datasets:
|
|
16 |
|
17 |
# OpenOrca x OpenChat - Preview2 - 13B
|
18 |
|
19 |
-
We have used our own [OpenOrca dataset](https://huggingface.co/datasets/Open-Orca/OpenOrca) to fine-tune Llama2-13B using [OpenChat](https://huggingface.co/openchat) packing
|
20 |
This dataset is our attempt to reproduce the dataset generated for Microsoft Research's [Orca Paper](https://arxiv.org/abs/2306.02707).
|
21 |
|
22 |
This second preview release is trained on a curated filtered subset of most of our GPT-4 augmented data.
|
@@ -28,7 +28,7 @@ As well, this is done with <1/10th the compute requirement and using <20% of the
|
|
28 |
We have run extensive evaluations internally and expect this model to **place number 1** on both the HuggingFaceH4 Open LLM Leaderboard and the GPT4ALL Leaderboard for 13B models.
|
29 |
|
30 |
"One" of [OpenChat](https://huggingface.co/openchat) has joined our team, and we'd like to provide special thanks for their training of this model!
|
31 |
-
We have utilized OpenChat
|
32 |
This has significantly reduced training time, with efficiency improvement of 3-10X over traditional methods.
|
33 |
|
34 |
|
|
|
16 |
|
17 |
# OpenOrca x OpenChat - Preview2 - 13B
|
18 |
|
19 |
+
We have used our own [OpenOrca dataset](https://huggingface.co/datasets/Open-Orca/OpenOrca) to fine-tune Llama2-13B using [OpenChat](https://huggingface.co/openchat) packing.
|
20 |
This dataset is our attempt to reproduce the dataset generated for Microsoft Research's [Orca Paper](https://arxiv.org/abs/2306.02707).
|
21 |
|
22 |
This second preview release is trained on a curated filtered subset of most of our GPT-4 augmented data.
|
|
|
28 |
We have run extensive evaluations internally and expect this model to **place number 1** on both the HuggingFaceH4 Open LLM Leaderboard and the GPT4ALL Leaderboard for 13B models.
|
29 |
|
30 |
"One" of [OpenChat](https://huggingface.co/openchat) has joined our team, and we'd like to provide special thanks for their training of this model!
|
31 |
+
We have utilized OpenChat [MultiPack algorithm](https://github.com/imoneoi/multipack_sampler) which achieves 99.85% bin-packing efficiency on our dataset.
|
32 |
This has significantly reduced training time, with efficiency improvement of 3-10X over traditional methods.
|
33 |
|
34 |
|