OPEA
/

Llama-3.3-70B-Instruct-int2-sym-inc-preview

intel/auto-round

Model card Files Files and versions Community

cicdatopea commited on 27 days ago

Commit

49da73c

·

verified ·

1 Parent(s): 6616a74

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -129,9 +129,10 @@ auto-round  \
 --device 0 \
 --group_size 32 \
 --nsamples 1024 \
---bits 4 \
 --iter 2000 \
 --disable_eval \
 --enable_norm_bias_tuning \
 --low_gpu_mem_usage \
 --format 'auto_gptq,auto_round' \

 --device 0 \
 --group_size 32 \
 --nsamples 1024 \
+--bits 2 \
 --iter 2000 \
 --disable_eval \
+--model_dtype "fp16" \
 --enable_norm_bias_tuning \
 --low_gpu_mem_usage \
 --format 'auto_gptq,auto_round' \