SupraLabs

/

Supra2-100M-Base

like 21

Follow

SupraLabs 352

114 440

lr?

I don't remember right now.

Please don't ask me questions right now. This isn't finished and i will release it once it's done. Stay tuned 🚀

100K steps ONLY? for a 100M, dang...
Either way, good luck! very excited to see a new model from SupraLabs since like... a lot of time haha

100K steps ONLY? for a 100M, dang...
Either way, good luck! very excited to see a new model from SupraLabs since like... a lot of time haha

Thank you 🤗

100K steps ONLY? for a 100M, dang...
Either way, good luck! very excited to see a new model from SupraLabs since like... a lot of time haha

It's not the total steps i think, it is the current step.

Nope, it's total steps

lr?

0.001, but there's warming (warmup_ratio=0.0175)

That's not even correct 😭😭😭

lr?

0.001, but there's warming (warmup_ratio=0.0175)

That's not even correct 😭😭😭

ik

Benchmarking access?

Benchmarking access?

As I said, the model releases next monday.
But here are some early benchmarks after 18B tokens of training:

bro... that's barely any progress. The 100M model should be able to beat our 50M on all benchmarks.

Why did my comment get marked as low quality if someone just asking for benchmarks was retained?

Why did my comment get marked as low quality if someone just asking for benchmarks was retained?

Maybe the HF system flagged it as the R word. Lol sometimes AI just go wild, but it could be someone on SupraLabs too lol

Why did my comment get marked as low quality if someone just asking for benchmarks was retained?

Maybe the HF system flagged it as the R word. Lol sometimes AI just go wild, but it could be someone on SupraLabs too lol

We didn't hided nothing.

Dang.... 1b model soon?

Dang.... 1b model soon?

dont think so not alot compute

I mean... maybe like a B200 for like 10 hours, that's like 70 bucks, but if yall want, its alays an option

I mean... maybe like a B200 for like 10 hours, that's like 70 bucks, but if yall want, its alays an option

yea but for chinchilla its 10 hours but for overtrain to actually good more than 500$

I mean... maybe like a B200 for like 10 hours, that's like 70 bucks, but if yall want, its alays an option

but a 200M that i would like

also where did Qyrou 1 go? its gone from thhe org

I mean... maybe like a B200 for like 10 hours, that's like 70 bucks, but if yall want, its alays an option

yea but for chinchilla its 10 hours but for overtrain to actually good more than 500$

nvm not that expensive,

For 1B with 100B tokens
Compute: 6 × 1e9 × 100e9 = 6e20 FLOPs

Time on a single B300:

BF16, 40% MFU (6 PFLOPS effective): 6e20 / 6e15 ≈ 28 hours (1.15 days)
FP8, 35% MFU (10 PFLOPS effective): 6e20 / 10e15 ≈ ~17 hours

also where did Qyrou 1 go? its gone from thhe org

need to do a bit more testing

also @Banaxi-Tech , u should prop join the SFZ discord server. thats the place we all hangout lol, its only u who's not there