rombodawg/DeepMagic-Coder-7b

DeepMagic-Coder-7b

(Note: From short testing, the Alt version generated much better code)

Alternate version:

• https://huggingface.co/rombodawg/DeepMagic-Coder-7b-Alt

This is an extremely successful merge of the deepseek-coder-6.7b-instruct and Magicoder-S-DS-6.7B models, bringing an uplift in overall coding performance without any compromise to the models integrity (at least with limited testing).

This is the first of my models to use the merge-kits task_arithmetic merging method. The method is detailed bellow, and its clearly very usefull for merging ai models that were fine-tuned from a common base:

Task Arithmetic:

Computes "task vectors" for each model by subtracting a base model. 
Merges the task vectors linearly and adds back the base. 
Works great for models that were fine tuned from a common ancestor. 
Also a super useful mental framework for several of the more involved 
merge methods.

The original models used in this merge can be found here:

• https://huggingface.co/ise-uiuc/Magicoder-S-DS-6.7B

• https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-instruct

The Merge was created using Mergekit and the paremeters can be found bellow:

models:
  - model: deepseek-ai_deepseek-coder-6.7b-instruct
    parameters:
      weight: 1
  - model: ise-uiuc_Magicoder-S-DS-6.7B
    parameters:
      weight: 1
merge_method: task_arithmetic
base_model: ise-uiuc_Magicoder-S-DS-6.7B
parameters:
  normalize: true
  int8_mask: true
dtype: float16