Awesome work and thanks, but why are the Bobs significantly smaller?
#1
by eatmemark - opened
I think I speak for everyone when I say we greatly appreciate you doing the conversion.
Had a question though: Why are the Bob variants so much smaller than the FP8s? Int8 ConvRot shouldn't be over 5GB smaller... if anything, they should be slightly larger. It must be stripping something out, and I think that's probably why you're seeing it be the fastest. I'm just not sure what we're losing. Have any idea?