is this a merged model?
#1
by
leejunhyeok
- opened
Hi.
in the model card, this model is created by merging method, like below
<I have built a model using the merge method, utilizing each of these models as the base.>
does this mean that you have used merging methods like slerp, lerp, etc?
thanks
yes. a merge of previous MoE's into one.
proving that u can consolidate this further. Now the next test
@DopeorNope
i think is to get this one into a moe and repeat the operation, see what happens maybe? have u tried?
@fblgit Thanks for replying instead of me..! I also have a plan to generate new ones as you explained above. You can see my result after this week..!
I will share!