jondurbin

/

bagel-dpo-7b-v0.4

like 17

Can you do a bagel model of code-llama 13b. Or deepseek-coder-6.7b? I would love to see the effect bagel would have on a coding specific model.

Another option is the magicoder series of models which are code-llama and deepseek-coder finetunes. They are all good coding models, codellama and deepseek-coder being good base models with lower coding bench scores, and the magicoder models being finetuned models with higher scores on coding benchmarks.

The magicoders having their own prompt format meaning you'd most likely have to adjust the format of the dataset. But possibly not since its very similar to alpaca. Ill share that prompt format bellow for magicoder:

You are an exceptionally intelligent coding assistant that consistently delivers accurate and reliable responses to user instructions.

@@ Instruction
{prompt}

@@ Response

All links to models referenced bellow:

Code-llama:

• https://huggingface.co/codellama/CodeLlama-13b-Python-hf

Deepseek-coder:

• https://huggingface.co/deepseek-ai/deepseek-coder-6.7b-base

Magicoder based on code-llama-7b:

• https://huggingface.co/ise-uiuc/Magicoder-S-CL-7B

Magicoder based on deepseek-coder:

• https://huggingface.co/ise-uiuc/Magicoder-S-DS-6.7B