Orca 2 upgrade planned?
#6
by
Thireus
- opened
Just curious to know if you plan to release an upgrade of dolphin using Orca 2 (and not sure if possible).
Thank you.
Working on grokking the paper
Brainstorming.
This is based on llama 2 rather than Mistral or Yi. I'm very confident those are much stronger bases than llama 2 and so it's not gonna be worth doing a dolphin finetune.
Getting it into a 34b form to match Yi might take some very creative work (@winglian has done this kind of thing with Phi)
I'm not sure it's possible. But I'm definitely looking into it.