davanstrien's picture
davanstrien HF staff
Add merge tag
b9205ba
|
raw
history blame
690 Bytes
metadata
license: mit
datasets:
  - Open-Orca/SlimOrca
  - beaugogh/openorca-multiplechoice-10k
language:
  - en
metrics:
  - accuracy
tags:
  - merge

This model is based on the LLama 7b model as a backbone, and datasets from various Orcas have been fine-tuned and merged.

The three models were combined, and the model with the best ARC and MMLU performance was given the highest weight.

First: fine-tuning beaugogh/openorca-multiplechoice-10k on llama2 7b, but using the NEFTune method.

Second: model fine-tuned with the SlimOrca dataset on llama2 7b.

Third : Model with beaugogh/openorca-multiplechoice-10k fine-tuned on llama2 7b.

We'll add the results once we have the official results