unsloth
/

gemma-2-9b-bnb-4bit

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

danielhanchen commited on 8 days ago

Commit

77cf94e

•

1 Parent(s): 971b773

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -37,7 +37,7 @@
     "load_in_8bit": false,
     "quant_method": "bitsandbytes"
   },
-  "query_pre_attn_scalar": 224,
   "rms_norm_eps": 1e-06,
   "rope_theta": 10000.0,
   "sliding_window": 4096,

     "load_in_8bit": false,
     "quant_method": "bitsandbytes"
   },
+  "query_pre_attn_scalar": 256,
   "rms_norm_eps": 1e-06,
   "rope_theta": 10000.0,
   "sliding_window": 4096,