RichardErkhov/Na0s_-_Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT - GGUF

• Model creator: https://huggingface.co/Na0s/
• Original model: https://huggingface.co/Na0s/Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT/

Name	Quant method	Size
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q2_K.gguf	Q2_K	2.59GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.IQ3_XS.gguf	IQ3_XS	2.86GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.IQ3_S.gguf	IQ3_S	2.99GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q3_K_S.gguf	Q3_K_S	2.98GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.IQ3_M.gguf	IQ3_M	3.07GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q3_K.gguf	Q3_K	3.25GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q3_K_M.gguf	Q3_K_M	3.25GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q3_K_L.gguf	Q3_K_L	3.49GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.IQ4_XS.gguf	IQ4_XS	3.63GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q4_0.gguf	Q4_0	3.77GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.IQ4_NL.gguf	IQ4_NL	3.8GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q4_K_S.gguf	Q4_K_S	3.79GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q4_K.gguf	Q4_K	3.97GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q4_K_M.gguf	Q4_K_M	3.97GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q4_1.gguf	Q4_1	4.14GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q5_0.gguf	Q5_0	4.52GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q5_K_S.gguf	Q5_K_S	4.52GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q5_K.gguf	Q5_K	4.62GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q5_K_M.gguf	Q5_K_M	4.62GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q5_1.gguf	Q5_1	4.89GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q6_K.gguf	Q6_K	5.31GB
Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT.Q8_0.gguf	Q8_0	6.87GB

Original model description:

library_name: transformers license: apache-2.0 datasets: - Open-Orca/SlimOrca pipeline_tag: text-generation base_model: Na0s/Llama-3.1-8b-Pruned-4-Layers

Model Card for Na0s/Llama-3.1-8B-Pruned-4-Layers_LoRA-PEFT

Model Details

Model Description

• Finetuned from model:[Na0s/Llama-3.1-8b-Pruned-4-Layers]

Training Details

    LoRA BF16, 
    batch_size=2, 
    steps=10000, gradient_accumulation_steps = 4,
    warmup_steps = 5,
    max_steps = 10000
    learning_rate = 2e-4,
    fp16 = not is_bfloat16_supported(),
    bf16 = is_bfloat16_supported(),
    logging_steps = 1,
    optim = "adamw_8bit",
    weight_decay = 0.01,
    lr_scheduler_type = "linear",
    seed = 3407

Training Data

[Open-Orca/SlimOrca]

Evaluation

MMLU Pro 0-shot: 0.2937

Evaluation Data

[TIGER-AI-Lab/MMLU-Pro]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).