license: apache-2.0
language:
- en
tags:
- creative
- story
- writing
- fiction
- float32
- roleplaying
- rp
- enhanced
- space whale
- 32 bit upscale
Ultra High Quality + NEO , NEO "X" Quants Remaster of the incredible: Psyonic-Cetacean-20B.
This is a Floating Point 32 upscale, where all components and merges were remastered to floating point 32. This includes all the merges (recreated with master files), and where possible subbing full FP32 models.
This repo contains 3 quants of the model all at IQ4XS (links to full quants Ultra, and Ultra Imatrix below).
The goal: Carry forward maximum precision right up to the point where it is "GUFFed".
This includes F32 master file for GGUF too... at a whopping 78 GBs. (compare at 38 GBs average for 20B models)
WHY?
Because the difference between F32 vs BF16 is... over 8 DECIMAL places.
And as each merge / model is modified there are "losses" along the way.
These losses are carried forward and in turn lead to more losses.
And decimal points are critical to model performance.
SMALL?
Yes... but multipled by each merge(s), and compression(s): 20 billion times.
The result of Ultra Quality:
At Q2K an impressive drop of 533 points in perplexity. (lower is better) (VS: Q2K original base model: PPL = 9.8077 +/- 0.06821 )
At Q4KM a whopping drop of 976 points in perplexity. (VS: Q4km original base model -> PPL = 8.7858 +/- 0.06074)
At Q6 an awesome drop of 234 points in perplexity. (VS: Q6 original base model -> PPL = 8.6070 +/- 0.05907 )
To put this in perspective "Q6" now operates ABOVE the orginal full precision version of "Psyonic-Cetacean-20b" and Q4KM operates at close to Q6 level quality.
This because at "Q6" the quant / compressed model is considered to be accurate within "+0.0008 ppl" of the full, uncompressed / unquanted model and it exceeds this threshold by over 200 points.
But... what about Q8?
The mountain moved:
150 points better: PPL = 8.5850 +/- 0.05881 VS: BASE/ORGINAL: PPL = 8.6012 +/- 0.05900
NEO Imatrix and NEO X Quants:
Quant 1 Neo V1 Imatrix:
This is a NEO Imatrix upgrade of the model, with the base model being the "Ultra Quality 20B." This upgrade is above Ultra Quality, Imatrix Plus and Imatrix Plus2.
Quant 2 Neo V1 Imatrix X Quant "Alpha":
This is a NEO Imatrix upgrade of the model, with the base model being the "Ultra Quality 20B.", with specialized "Alpha" X quant settings that are specific to Psyonic-Cetacean-20B model structure only.
Quant 3 Neo V1 Imatrix X Quant "Beta":
This is a NEO Imatrix upgrade of the model, with the base model being the "Ultra Quality 20B.", with specialized "Beta" X quant settings that are specific to Psyonic-Cetacean-20B model structure only.
Compare the output of each of these new versions below in the "examples" area.
THE RESULTS ARE IN:
AS per Jeb Carter, original creator of the model:
- instruction following has improved dramatically.
- new abilities have emerged.
- he had to REDUCE the instructions sets used because the model no longer needed as specific instructions.
- prose, nuance and depth have all improved.
- known issues with the original model have disappeared.
This is not "something for nothing" ; it is method of ensuring maximum precision at every step just before "ggufing" the model.
The methods employed only ensure precision loss is minimized or eliminated.
It is mathematical and theory sound.
The bottom line here is this:
Higher quality instruction following and output.
Likewise you can use a smaller compression, with higher token per second and still get great quality.
Same great model... turbo charged.
This is the first group of remasters.
The FOUR Horsemen:
This repo will be followed by a "reg quant plus" repo, which added additional components into the GGUF (all levels) at floating point 32 precision to further increase the sheer creativity and raw AI horsepower.
This process shaves at extra 50-100 points off perplexity... again.
Following this group will be a full float 32 precision Imatrix (including reg quants "imatrixed").
Test results VS org and "ultra" regular quants will be posted when they come in.
Ultra Quality Non-Imatrix Quants:
[ https://huggingface.co/DavidAU/Psyonic-Cetacean-Ultra-Quality-20b-GGUF ]
Imatrix Repo: [ https://huggingface.co/DavidAU/Psyonic-Cetacean-Ultra-Quality-20b-GGUF-imatrix ]
Imatrix Plus 2 Repo: [ https://huggingface.co/DavidAU/Psyonic-Cetacean-Ultra-Quality-20b-GGUF-imat-plus2 ]
Details of all methods (and pitfalls to avoid) employed to make this high precision remasters will be posted shortly along with comparision of original model and new ultra remaster.
Thanks again to Jeb Carter, the original creator of "Psyonic-Cetacean 20B"
[ https://huggingface.co/jebcarter/psyonic-cetacean-20B ]