What does exl2-4bpw-rpcal in the model name mean?

#1
by BigDeeper - opened

I don't see anything in the card to explain the difference with the source model.

Owner

This is the original model quantized in exllamav2 format up to 4-bit, using the calibration of the rp dataset. and yes, the extended context is achieved there by increasing the rope_theta parameter, I get acceptable results in long rp chats.

Sign up or log in to comment