Hugging Face's logo

soiji
/
Llama-3.2-1B-Instruct_Mobilequant

PyTorch
llama
Model card Files
xet
 Community

Configs & command(run.sh)

MODEL=Llama-3.2-1B-Instruct_Converted
CKPT=/models/meta-llama/${MODEL}
BATCH_SIZE=1

LET_LR=1e-3
LET_MIN_LR=1e-4
LWC_LR=1e-2
LWC_MIN_LR=1e-3
LRL_LR=1e-6
LRL_MIN_LR=1e-7

EPOCHS=60
NSAMPLES=1024
OUTPUT_DIR=/models/meta-llama/${MODEL}-mobilequant-s${NSAMPLES}-e${EPOCHS}

python ptq/mobilequant.py --lwc --let --lrl \
    --act_bitwidth 16 --epochs ${EPOCHS} --nsamples ${NSAMPLES} \
    --deactive_amp --hf_path ${CKPT} --output_dir ${OUTPUT_DIR} \
    --let_lr ${LET_LR} --let_min_lr ${LET_MIN_LR} \
    --lwc_lr ${LWC_LR} --lwc_min_lr ${LWC_MIN_LR} \
    --lrl_lr ${LRL_LR} --lrl_min_lr ${LRL_MIN_LR} \
    --batch_size ${BATCH_SIZE} --mode e2e --weight_bitwidth 16
Downloads last month
11
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for soiji/Llama-3.2-1B-Instruct_Mobilequant

Base model

meta-llama/Llama-3.2-1B-Instruct
Finetuned
(1834)
this model

Dataset used to train soiji/Llama-3.2-1B-Instruct_Mobilequant

Collection including soiji/Llama-3.2-1B-Instruct_Mobilequant