TheBloke commited on
Commit
3296895
1 Parent(s): c1926a5

Initial GPTQ model commit

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -230,7 +230,7 @@ Thank you to all my generous patrons and donaters!
230
 
231
  # OpenOrca x OpenChat - Preview2 - 13B
232
 
233
- We have used our own [OpenOrca dataset](https://huggingface.co/datasets/Open-Orca/OpenOrca) to fine-tune Llama2-13B using [OpenChat](https://huggingface.co/openchat) packing and conditional behavior cloning.
234
  This dataset is our attempt to reproduce the dataset generated for Microsoft Research's [Orca Paper](https://arxiv.org/abs/2306.02707).
235
 
236
  This second preview release is trained on a curated filtered subset of most of our GPT-4 augmented data.
@@ -242,7 +242,7 @@ As well, this is done with <1/10th the compute requirement and using <20% of the
242
  We have run extensive evaluations internally and expect this model to **place number 1** on both the HuggingFaceH4 Open LLM Leaderboard and the GPT4ALL Leaderboard for 13B models.
243
 
244
  "One" of [OpenChat](https://huggingface.co/openchat) has joined our team, and we'd like to provide special thanks for their training of this model!
245
- We have utilized OpenChat conditional behavior cloning and [MultiPack algorithm](https://github.com/imoneoi/multipack_sampler) which achieves 99.85% bin-packing efficiency on our dataset.
246
  This has significantly reduced training time, with efficiency improvement of 3-10X over traditional methods.
247
 
248
 
 
230
 
231
  # OpenOrca x OpenChat - Preview2 - 13B
232
 
233
+ We have used our own [OpenOrca dataset](https://huggingface.co/datasets/Open-Orca/OpenOrca) to fine-tune Llama2-13B using [OpenChat](https://huggingface.co/openchat) packing.
234
  This dataset is our attempt to reproduce the dataset generated for Microsoft Research's [Orca Paper](https://arxiv.org/abs/2306.02707).
235
 
236
  This second preview release is trained on a curated filtered subset of most of our GPT-4 augmented data.
 
242
  We have run extensive evaluations internally and expect this model to **place number 1** on both the HuggingFaceH4 Open LLM Leaderboard and the GPT4ALL Leaderboard for 13B models.
243
 
244
  "One" of [OpenChat](https://huggingface.co/openchat) has joined our team, and we'd like to provide special thanks for their training of this model!
245
+ We have utilized OpenChat [MultiPack algorithm](https://github.com/imoneoi/multipack_sampler) which achieves 99.85% bin-packing efficiency on our dataset.
246
  This has significantly reduced training time, with efficiency improvement of 3-10X over traditional methods.
247
 
248