LnL-AI
/

TinyLlama-1.1B-intermediate-step-1341k-3T-autoround-lm_head-symFalse

Text Generation

text-generation-inference

Inference Endpoints

4-bit precision

Model card Files Files and versions Community

Qubitium commited on 24 days ago

Commit

810c185

•

1 Parent(s): 6e749ee

Update config.json

Files changed (1) hide show

config.json +1 -1

config.json CHANGED Viewed

@@ -29,7 +29,7 @@
       "iters": 400,
       "lr": 0.0025,
       "minmax_lr": 0.0025,
-      "packer": "autogptq:0.8.0.dev1",
       "quantizer": "intel/auto-round:0.1",
       "scale_dtype": "torch.float16",
       "use_quant_input": false

       "iters": 400,
       "lr": 0.0025,
       "minmax_lr": 0.0025,
+      "packer": "gptqmodel:0.9.0",
       "quantizer": "intel/auto-round:0.1",
       "scale_dtype": "torch.float16",
       "use_quant_input": false