Quantization
Collection
A collection of quantized models. All the models can be fine-tuned by adding a LoRA Adapter.
•
83 items
•
Updated
•
3
The Quantized LLaMA 3.1 70B Instruct Model
Original Base Model: meta-llama/Meta-Llama-3.1-70B-Instruct.
Link: https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct
Quantization Configurations
"quantization_config": {
"bits": 4,
"checkpoint_format": "gptq",
"damp_percent": 0.01,
"desc_act": true,
"group_size": 128,
"model_file_base_name": null,
"model_name_or_path": null,
"quant_method": "gptq",
"static_groups": false,
"sym": true,
"true_sequential": true
},
Source Codes
Source Codes: https://github.com/vkola-lab/medpodgpt/tree/main/quantization.
- Downloads last month
- 121
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social
visibility and check back later, or deploy to Inference Endpoints (dedicated)
instead.