Onwards and upwards!
#2
by Datdanboi25 - opened
Cool model man! Great to see so much improvement over Ant-5m.
Would be curious to know what data it was trained on
Thanks !!
Could you please add this model in LeaderBoard ,
Well About Dataset It is trained on samedataset last time I trained Ant-5M on 1.5B tokens like subset of this dataset ,
This time I trained on full 3B tokens
It is curated dataset Minx of 60% - Fine HQ , 20% Cosmopedia , 10 % -Finemath rest 10 % Misc like my Arithmetic Dataset & Some pyhton codes.
Its already up!