DuoNeural commited on
Commit
8b991d5
·
verified ·
1 Parent(s): 5827b15

Add README.md

Browse files
Files changed (1) hide show
  1. README.md +31 -13
README.md CHANGED
@@ -1,21 +1,39 @@
1
  ---
2
- base_model: unsloth/qwen2.5-coder-3b-instruct-bnb-4bit
3
- tags:
4
- - text-generation-inference
5
- - transformers
6
- - unsloth
7
- - qwen2
8
- license: apache-2.0
9
  language:
10
  - en
 
 
 
 
 
 
 
 
 
11
  ---
12
 
13
- # Uploaded finetuned model
 
 
14
 
15
- - **Developed by:** DuoNeural
16
- - **License:** apache-2.0
17
- - **Finetuned from model :** unsloth/qwen2.5-coder-3b-instruct-bnb-4bit
18
 
19
- This qwen2 model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.
20
 
21
- [<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
 
 
 
 
 
 
 
2
  language:
3
  - en
4
+ license: apache-2.0
5
+ tags:
6
+ - duoneural
7
+ - sft
8
+ - qwen
9
+ - qwen2.5-coder
10
+ base_model: Qwen/Qwen2.5-Coder-3B-Instruct
11
+ datasets:
12
+ - DuoNeural/Gemma4-E2B-SFT-WebCode
13
  ---
14
 
15
+ # Qwen2.5-Coder-3B-SFT-WebCode
16
+
17
+ **📊 Recorded** — SFT fine-tune by [DuoNeural](https://huggingface.co/DuoNeural).
18
 
19
+ - **Base model:** [Qwen/Qwen2.5-Coder-3B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-3B-Instruct)
20
+ - **Dataset:** [DuoNeural/Gemma4-E2B-SFT-WebCode](https://huggingface.co/datasets/DuoNeural/Gemma4-E2B-SFT-WebCode)
21
+ - **Training:** LoRA rank=16 α=32, 3 epochs, lr=2e-4, effective batch=16
22
 
23
+ - **Eval:** GSM8K + ARC-Challenge via lm_eval 0.4.x
24
 
25
+ ## Benchmark Results
26
+
27
+ | Model | GSM8K flex | ARC-norm | ARC-acc |
28
+ |---|---|---|---|
29
+ | Baseline | 0.5807 | 0.4957 | 0.4590 |
30
+ | **Qwen2.5-Coder-3B-SFT-WebCode** | **0.3207** | **0.4957** | **0.4590** |
31
+ | Δ | -0.2600 | +0.0000 | +0.0000 |
32
+
33
+ ## About DuoNeural
34
+
35
+ Post-training research lab exploring emergent behaviors in small language models.
36
+ We publish datasets, models, and [research papers](https://zenodo.org/communities/duoneural).
37
+
38
+ ---
39
+ *Generated by Archon — DuoNeural lab AI*