File size: 3,137 Bytes
0982a20 bc5ba67 28a4471 e5bc2df 0982a20 9755981 0982a20 28a4471 0982a20 5af760d 0982a20 faac55e 0982a20 faac55e 0982a20 faac55e 0982a20 faac55e 0982a20 9755981 0982a20 28a4471 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 |
---
license: llama3
datasets:
- ajibawa-2023/Code-290k-ShareGPT
- m-a-p/CodeFeedback-Filtered-Instruction
- m-a-p/Code-Feedback
- microsoft/orca-math-word-problems-200k
language:
- en
tags:
- code
- Python
- Cpp
- PHP
- JS
- Java
- Rust
- Ruby
- SQL
- MySql
- R
- Julia
---
**Code-Llama-3-8B**
This Model is trained on refined version of my dataset [Code-290k-ShareGPT](https://huggingface.co/datasets/ajibawa-2023/Code-290k-ShareGPT).
Besides this it is trained on following datasets:
[Code-Feedback](https://huggingface.co/datasets/m-a-p/Code-Feedback)
[orca-math-word-problems-200k](https://huggingface.co/datasets/microsoft/orca-math-word-problems-200k)
[CodeFeedback-Filtered-Instruction](https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction)
The idea was to check how this Model will perform with both Code & Maths datasets. This model is very good with Coding.
Maths outputs are also very good. You can test out this model.
It is very very good in Code generation in various languages such as **Python, Java, JavaScript, GO, C++, Rust, Ruby, Sql, MySql, R, Julia, Haskell**, etc..
This model will also generate detailed explanation/logic behind each code.
This Model is trained on massive datasets so the results are very good. You can check the Examples given below.
I have used ChatML prompt format.
This is Fully Finetuned Model.
**GGUF & Exllama**
GGUF: [Link](https://huggingface.co/bartowski/Code-Llama-3-8B-GGUF)
Exllama v2: [Link](https://huggingface.co/bartowski/Code-Llama-3-8B-exl2)
Special Thanks to [Bartowski](https://huggingface.co/bartowski) for quantizing this model.
**Training:**
Entire dataset was trained on 4 x A100 80GB. For 2 epoch, training took more than 160 Hours. Axolotl & Deepspeed codebase was used for training purpose.
Entire data is trained on Llama-3-8B by Meta.
**Example Prompt:**
This model uses **ChatML** prompt format.
```
<|im_start|>system
You are a helpful Coding assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant
```
You can modify above Prompt as per your requirement.
One example will be:
```
This is a conversation with your helpful Coding assistant. Assistant can generate Code in various Programming Languages along with necessary explanation.
```
I want to say special Thanks to the Open Source community for helping & guiding me to better understand the AI/Model development.
Thank you for your love & support.
**Example Output**
Example 1
Example 2
Example 3
Example 4
Example 5
 |