It seems the original model repo was pulled
It gives a 404 at the time of this post
Noticed that as well. Curious as to why it was pulled.
My gut instinct tells me it was related to a political reason
That had crossed my mind. But it could be just as valid for any other reason as well. I wanted to see the original model to do some research on what the "Ensemble" part meant.
Very true
That had crossed my mind. But it could be just as valid for any other reason as well. I wanted to see the original model to do some research on what the "Ensemble" part meant.
Judging from the author's other works, "Ensemble" should refer to fine-tuning with mixture of different dataset and their methods like QLoRA and instruction-tuning and so forth.
So models like Orca or Guannco are mainly made public in order to demonstrate their paper-purposed SoTA or innovative method of fine-tuning. This ensemble will be like pipelining these SoTA methods one after another