Experimental running Quants of "Dark Sapling V2 7B Ultra Quality" (32k context) in Imatrix Plus 2 and Imatrix X protocols.
Imatrix Plus 2 is an advanced Imatrix quanting method using multiple datasets.
Imatrix X takes Imatrix Plus 2 a step further and introduces a "pass-through merge" method between Imatrix Plus 2 and regular Quants.
This leverages the full power of both Imatrix Plus 2 and regular quant (which is based on Ultra Quality 32 bit precision).
This marries (to varying degrees) the best of THREE worlds: The Ultra Quality model, the Imatrix Plus 2 (and enhancements from the datasets) , and the raw power of the unfiltered/unaltered Ultra Quality model itself.
These methods target the weak point(s) and enhance the strong point(s) at the same time.
Imatrix Plus 2 and Imatrix X are the next steps in Ultra Quality experience.
The 11 quants hosted here are various versions of "Imatrix X" (with Imatrix Plus 2), with the exception of "DarkSapling-V2-Ultra-Quality-7B-Q4_k_m-imat1.gguf" which is stand alone Imatrix Plus 2.
All quants operate, however there is more work in progress and these quants are subject to revision without notice.
For testing purposes (if you download more than one) set "TEMP:0" and test your prompt(s).
You should use prompts with "no right answer" to fully evalute the quant (and vs other quants here).
Not all of these quants are suitable for all purposes and protocol(s) are still under developement.
Dark Sapling V2 7B is a fiction, roleplay, fantasy and story telling model.
Although all quants at this repo are all "Q4_K_M Imatrix 2" quants they differ greatly due to Imatrix X protocols.
Some will be far more verbose than others, answer differently, make stuff up, lie to you or worse... it might flirt with you.
Others will be more "sciency" in their replies, whereas others will be more "fictional".
Some Imat X versions even bring the core model to new levels of performance, above and beyond current 32 bit Ultra Quality levels.
Caution: Can generate NSFW content.
For more information on Dark Sapling V2 Ultra Quality please see:
[ https://huggingface.co/DavidAU/DarkSapling-V2-Ultra-Quality-7B-GGUF ]
- Downloads last month
- 306