cpral/Ornith-1.0-397B-3bpw-EXL3

3.04bpw EXL3 quant of Ornith 1.0 397B. You'll need ~170+ GiB of VRAM to run it. Probably at least 7 x 3090 or 2x 6000 Pro. Use with TabbyAPI.

 -- A perplexity:  3.20393975
 -- B perplexity:  3.26597043
 -- A label in top-K:
      K = 1: 0.7186
      K = 2: 0.8140
      K = 3: 0.8549
      K = 4: 0.8786
      K = 5: 0.8937
 -- B label in top-K:
      K = 1: 0.7141
      K = 2: 0.8116
      K = 3: 0.8527
      K = 4: 0.8767
      K = 5: 0.8923
 -- Top-K agreement, A vs B:
      K = 1: 0.9422
      K = 2: 0.7667
      K = 3: 0.5596
      K = 4: 0.3816
      K = 5: 0.2486
 -- KL divergence (A, B):  0.04249165
 -- KL divergence (B, A):  0.03919449