RichardErkhov/MadMarx37_-_deepseek-coder-1.3b-python-peft-gguf

Quantization made by Richard Erkhov.

Github

Discord

Request more models

deepseek-coder-1.3b-python-peft - GGUF

• Model creator: https://huggingface.co/MadMarx37/
• Original model: https://huggingface.co/MadMarx37/deepseek-coder-1.3b-python-peft/

Name	Quant method	Size
deepseek-coder-1.3b-python-peft.Q2_K.gguf	Q2_K	0.52GB
deepseek-coder-1.3b-python-peft.IQ3_XS.gguf	IQ3_XS	0.57GB
deepseek-coder-1.3b-python-peft.IQ3_S.gguf	IQ3_S	0.6GB
deepseek-coder-1.3b-python-peft.Q3_K_S.gguf	Q3_K_S	0.6GB
deepseek-coder-1.3b-python-peft.IQ3_M.gguf	IQ3_M	0.63GB
deepseek-coder-1.3b-python-peft.Q3_K.gguf	Q3_K	0.66GB
deepseek-coder-1.3b-python-peft.Q3_K_M.gguf	Q3_K_M	0.66GB
deepseek-coder-1.3b-python-peft.Q3_K_L.gguf	Q3_K_L	0.69GB
deepseek-coder-1.3b-python-peft.IQ4_XS.gguf	IQ4_XS	0.7GB
deepseek-coder-1.3b-python-peft.Q4_0.gguf	Q4_0	0.72GB
deepseek-coder-1.3b-python-peft.IQ4_NL.gguf	IQ4_NL	0.73GB
deepseek-coder-1.3b-python-peft.Q4_K_S.gguf	Q4_K_S	0.76GB
deepseek-coder-1.3b-python-peft.Q4_K.gguf	Q4_K	0.81GB
deepseek-coder-1.3b-python-peft.Q4_K_M.gguf	Q4_K_M	0.81GB
deepseek-coder-1.3b-python-peft.Q4_1.gguf	Q4_1	0.8GB
deepseek-coder-1.3b-python-peft.Q5_0.gguf	Q5_0	0.87GB
deepseek-coder-1.3b-python-peft.Q5_K_S.gguf	Q5_K_S	0.89GB
deepseek-coder-1.3b-python-peft.Q5_K.gguf	Q5_K	0.93GB
deepseek-coder-1.3b-python-peft.Q5_K_M.gguf	Q5_K_M	0.93GB
deepseek-coder-1.3b-python-peft.Q5_1.gguf	Q5_1	0.95GB
deepseek-coder-1.3b-python-peft.Q6_K.gguf	Q6_K	1.09GB
deepseek-coder-1.3b-python-peft.Q8_0.gguf	Q8_0	1.33GB

Original model description:

library_name: transformers tags: - code license: mit datasets: - ArtifactAI/arxiv_python_research_code language: - en pipeline_tag: text-generation

Model Card for Model ID

A parameter-efficient finetune (using LoRA) of DeepSeek Coder 1.3B finetuned on Python code.

Model Details

Model Description

A finetune of DeepSeek Coder 1.3B finetuned on 1000 examples of Python code from the ArtifactAI/arxiv_python_research_code dataset.

• Model type: Text Generation
• Language(s) (NLP): English, Python
• Finetuned from model: deepseek-ai/deepseek-coder-1.3b-base

Model Sources [optional]

• Repository: https://github.com/kevin-v96/python-codecomplete-lm

Uses

To generate Python code

How to Get Started with the Model

Use the code below to get started with the model.

from transformers import pipeline

model_name = "MadMarx37/deepseek-coder-1.3b-python-peft"

def generate_output(input):
    # Run text generation pipeline with our next model
    pipe = pipeline(task="text-generation", model=model_name, tokenizer=model_name, max_length=max_length)
    result = pipe(input)
    print(result[0]['generated_text'])

Training Details

Training Hyperparameters

• Training regime: fp16 mixed-precision with original model loaded in 4bits with bitsandbytes
• learning_rate = 2e-3
• lr_scheduler_type = 'cosine_with_restarts'
• max_grad_norm = 0.001
• weight_decay = 0.001
• num_train_epochs = 15
• eval_strategy = "steps"
• eval_steps = 25

Speeds, Sizes, Times [optional]

1.3B parameters. Training time of ~2 hours on an RTX3080.

Evaluation

Testing Data, Factors & Metrics

Testing Data

https://huggingface.co/datasets/ArtifactAI/arxiv_python_research_code

Metrics

Standard training and eval loss from the HF SFTTrainer.

Results

Training Loss: 0.074100 Validation Loss: 0.022271

Summary

The training had some unstability in the gradient norms, but the overall trend in both training and validation loss were downward, and validation loss has almost plateaud, which is ideally where we want our model. The code generation on the same prompts that we tested the original model on also seem better with the finetuned model. A good way to make the model better, if we wanted to increase the finetuning data, would be to also increase the epochs.

The training run metrics can be seen here: https://wandb.ai/kevinv3796/python-autocomplete-deepseek/reports/Supervised-Finetuning-run-for-DeepSeek-Coder-1-3B-on-Python-Code--Vmlldzo3NzQ4NjY0?accessToken=bo0rlzp0yj9vxf1xe3fybfv6rbgl97w5kkab478t8f5unbwltdczy63ba9o9kwjp