Update README.md
Browse files
README.md
CHANGED
|
@@ -13,7 +13,9 @@ language:
|
|
| 13 |
# Mini-Coder
|
| 14 |
|
| 15 |
Mini-Coder is build on top of Qwen3.5-9B model with Continual Pretraining (CPT), we feed ~500k high-quality curated luau samples to improves the luau coding tasks capability.
|
|
|
|
| 16 |
We also inject over 14k samples from open-source of claude 4.6 distillations with a fews additional samples for Supervised-Finetuning (SFT) to improves the model reasnoning, We also see the average consumed tokens has drastically reduced.
|
|
|
|
| 17 |
It's fine-tuned efficiently using LoRA (16-bit) and rsLoRA with Rank (r) set to 64 and Alpha (α) set to 128, ensuring strong adaptation and retention of new complex logic, it were trained specifically to handle up to 32,768 (32k) tokens of maximum output (recommended).
|
| 18 |
|
| 19 |
# Uploaded finetuned model
|
|
|
|
| 13 |
# Mini-Coder
|
| 14 |
|
| 15 |
Mini-Coder is build on top of Qwen3.5-9B model with Continual Pretraining (CPT), we feed ~500k high-quality curated luau samples to improves the luau coding tasks capability.
|
| 16 |
+
|
| 17 |
We also inject over 14k samples from open-source of claude 4.6 distillations with a fews additional samples for Supervised-Finetuning (SFT) to improves the model reasnoning, We also see the average consumed tokens has drastically reduced.
|
| 18 |
+
|
| 19 |
It's fine-tuned efficiently using LoRA (16-bit) and rsLoRA with Rank (r) set to 64 and Alpha (α) set to 128, ensuring strong adaptation and retention of new complex logic, it were trained specifically to handle up to 32,768 (32k) tokens of maximum output (recommended).
|
| 20 |
|
| 21 |
# Uploaded finetuned model
|