imatrix question

#1
by eleius - opened

I don't understand the imatrix thing too well but I noticed bartowski uses this for calibration. I was wondering if you use different data for your quants and, in that case, if the results may differ?

Our imatrix dataset is based on the one from bartowski but with a lot of confidential proprietary high-quality data added so we can offer the highest possible quality of general purpose imatrix quants. Our imatrix quants should be of slightly higher quality than the ones offered by bartowski due to a larger dataset and so more resources being used to computing the imatrix. They should especially excel in story writing and roleplay workloads where bartowski's imatrix dataset falls short. This topic was recently discussed under https://huggingface.co/mradermacher/Meta-Llama-3.1-70B-i1-GGUF/discussions/1 so maybe give it a read.

Thanks a lot for the detailed info!

eleius changed discussion status to closed
Your need to confirm your account before you can post a new comment.

Sign up or log in to comment