NousResearch

/

Yarn-Mistral-7b-128k

like 566

Hi there,

A recent fix improved the perpelexity of models like mistral at long context lengths. Here is a figure showing the before-and-after.

I'm wondering if this would impact the figure on your model card. This only happens in fp16, afaik.