Bruh this is crazy
Holy moly dude, this thing is crazy.
Huge uplift from rose 1, incredibly impressive stuff
Great stuff!
(ps congrats on #1 <100m)
Also the smallest model to ever crack 20 on the int index, by like 25m parameters, insane.
CRAZY
Thank You mates !!
I was myself surprised by its performance...
FInally Something that i m very proud of.
And Base Bench?
It was 1087 ig
ouhhh look what i found
Rose-1.5-Mini!!!!!!
yeah actually i was planned to to make Mini 100M param & medium 135M param Along with 150M for pro , but later i decided , to not confuse with it , SO i name it "Medium" , this is duplicated from the original repo
As i m not gonna train a 50M param model
oh bruhhh... thought found ROSe 1.5 50M lol
which datasets?
a combination of dataset , mostly dclm - finweb edu - smoll-corpus
crazy work indeed
dclm is pretty good!