Add README.md
Browse files
README.md
CHANGED
|
@@ -1,21 +1,39 @@
|
|
| 1 |
---
|
| 2 |
-
base_model: unsloth/qwen2.5-coder-3b-instruct-bnb-4bit
|
| 3 |
-
tags:
|
| 4 |
-
- text-generation-inference
|
| 5 |
-
- transformers
|
| 6 |
-
- unsloth
|
| 7 |
-
- qwen2
|
| 8 |
-
license: apache-2.0
|
| 9 |
language:
|
| 10 |
- en
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
---
|
| 12 |
|
| 13 |
-
#
|
|
|
|
|
|
|
| 14 |
|
| 15 |
-
- **
|
| 16 |
-
- **
|
| 17 |
-
- **
|
| 18 |
|
| 19 |
-
|
| 20 |
|
| 21 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
language:
|
| 3 |
- en
|
| 4 |
+
license: apache-2.0
|
| 5 |
+
tags:
|
| 6 |
+
- duoneural
|
| 7 |
+
- sft
|
| 8 |
+
- qwen
|
| 9 |
+
- qwen2.5-coder
|
| 10 |
+
base_model: Qwen/Qwen2.5-Coder-3B-Instruct
|
| 11 |
+
datasets:
|
| 12 |
+
- DuoNeural/Gemma4-E2B-SFT-WebCode
|
| 13 |
---
|
| 14 |
|
| 15 |
+
# Qwen2.5-Coder-3B-SFT-WebCode
|
| 16 |
+
|
| 17 |
+
**📊 Recorded** — SFT fine-tune by [DuoNeural](https://huggingface.co/DuoNeural).
|
| 18 |
|
| 19 |
+
- **Base model:** [Qwen/Qwen2.5-Coder-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct)
|
| 20 |
+
- **Dataset:** [DuoNeural/Gemma4-E2B-SFT-WebCode](https://huggingface.co/datasets/DuoNeural/Gemma4-E2B-SFT-WebCode)
|
| 21 |
+
- **Training:** LoRA rank=16 α=32, 3 epochs, lr=2e-4, effective batch=16
|
| 22 |
|
| 23 |
+
- **Eval:** GSM8K + ARC-Challenge via lm_eval 0.4.x
|
| 24 |
|
| 25 |
+
## Benchmark Results
|
| 26 |
+
|
| 27 |
+
| Model | GSM8K flex | ARC-norm | ARC-acc |
|
| 28 |
+
|---|---|---|---|
|
| 29 |
+
| Baseline | 0.5807 | 0.4957 | 0.4590 |
|
| 30 |
+
| **Qwen2.5-Coder-3B-SFT-WebCode** | **0.3207** | **0.4957** | **0.4590** |
|
| 31 |
+
| Δ | -0.2600 | +0.0000 | +0.0000 |
|
| 32 |
+
|
| 33 |
+
## About DuoNeural
|
| 34 |
+
|
| 35 |
+
Post-training research lab exploring emergent behaviors in small language models.
|
| 36 |
+
We publish datasets, models, and [research papers](https://zenodo.org/communities/duoneural).
|
| 37 |
+
|
| 38 |
+
---
|
| 39 |
+
*Generated by Archon — DuoNeural lab AI*
|