Agent Dense
Collection
6 items • Updated
🤖 Agents-A1-4B-kimi-Preview
4B Coding Agent Model — SFT Fine-tuned · Kimi K3 Traces · Tool-Calling
Preview release — Fine-tuned from InternScience/Agents-A1-4B on Kimi K3 coding and debugging traces. Specializes in coding agent behavior, debugging, and step-by-step tool-use.
✨ Key Features
| Capability | Description |
|---|---|
| 💻 Coding Agent | Trained on real Kimi K3 coding traces — planning, debugging, building |
| 🧠 Step-by-step Reasoning | Autonomous agent-style reasoning |
| 🔧 Tool Calling | Hermes/Qwen function-calling format |
| 💬 Multi-turn | 61% multi-turn conversations |
| 🌏 Thai + English | Native bilingual support |
| 🐍 Multi-language | Python, C, C++, Go, Java, Rust, Bash, and more |
| ⚡ Fast Inference | 4B params — runs on consumer GPUs (6-8 GB VRAM) |
🚀 Usage
Python (Transformers)
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained(
"hotdogs/Agents-A1-4B-kimi-Preview",
torch_dtype="auto",
device_map="auto",
trust_remote_code=True
)
tokenizer = AutoTokenizer.from_pretrained("hotdogs/Agents-A1-4B-kimi-Preview")
messages = [{"role": "user", "content": "Write a Python function to sort a list"}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256, temperature=0.6)
print(tokenizer.decode(outputs[0]))
GGUF (llama.cpp)
GGUF quantized version coming soon.
📦 Downloads
| File | Size | Description |
|---|---|---|
model.safetensors |
8.41 GB | Full BF16 precision (Transformers) |
🧬 Architecture
| Parameter | Value |
|---|---|
| Base | InternScience/Agents-A1-4B |
| Parameters | ~4.29B |
| Hidden Size | 2,560 |
| Attention | Linear + Standard hybrid (8 full-attn layers) |
| Context | 4,096 tokens (trained) |
| Precision | BF16 |
| Format | ChatML (Jinja2 template) |
| Vocab | 248,320 tokens |
🏋️ Training Details
Base Model
InternScience/Agents-A1-4B — A 4B VLM based on Qwen3.5 architecture with hybrid linear/full attention. Fine-tuned on the text decoder only.
Dataset
greghavens/kimi-k3-coding-and-debugging-traces — 4,928 high-quality coding agent traces from Kimi K3, featuring:
- Real coding and debugging trajectories
- Multi-step tool-use patterns
- 156 task categories (build-game, dependency-planning, debug-concurrency, etc.)
- Languages: Python, C, C++, Go, Java, Rust, Bash, etc.
- Auto-generated coding agent reasoning
Preprocessed: 3,389 clean rows after deduplication and filtering (3,219 train / 170 val).
Training Pipeline
┌─────────────────────────────────────────────────────────────┐
│ Training Pipeline │
├─────────────────────────────────────────────────────────────┤
│ │
│ Base Model LoRA Adapter │
│ (Agents-A1-4B) (r=64, α=128) │
│ │ │ │
│ └───────────┬──────────────────┘ │
│ │ │
│ Unsloth FastLanguageModel │
│ device_map="auto" · 4× RTX 3060 12GB │
│ │ │
│ ▼ │
│ ┌──────────────────────────┐ │
│ │ Custom SFTDataset │ │
│ │ • apply_chat_template() │ │
│ │ • Tokenize (4096 max) │ │
│ │ • Mask prompt tokens │ │
│ └──────────┬───────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────┐ │
│ │ HF Trainer (no SFTTr.) │ │
│ │ • BF16 · LR=1e-4 │ │
│ │ • max_grad_norm=1.0 │ │
│ │ • 3 epochs · batch=1 │ │
│ │ • ~8.5 hours on 4×3060 │ │
│ └──────────┬───────────────┘ │
│ │ │
│ ▼ │
│ Final Model (scale=0.4) │
│ (LoRA merged into text decoder) │
│ │
└─────────────────────────────────────────────────────────────┘
Merge Quality
The LoRA was tested at multiple merge scales to find the optimal balance between preserving base knowledge and applying the fine-tuned behavior:
| Merge Scale | Max Weight Diff | Safety |
|---|---|---|
| 0.3 | 0.007812 | Very safe |
| 0.6 | 0.015625 | ❌ Looping |
| 0.4 | 0.010498 | Optimal ✅ |
Max weight diff measures the largest change in any single weight tensor compared to the base model. Values below 0.02 are considered safe — the model retains base knowledge while incorporating new behaviors. The final merge uses scale=0.4 for a good balance.
Hardware
| Component | Spec |
|---|---|
| GPUs | 4× NVIDIA RTX 3060 12GB |
| Total VRAM | 48 GB |
| Training Time | ~8 hours 32 min |
| Framework | Unsloth + HuggingFace Trainer |
Hyperparameters
| Parameter | Value |
|---|---|
| LoRA Rank | 64 |
| LoRA Alpha | 128 |
| Learning Rate | 1e-4 |
| Scheduler | Cosine (warmup 3%) |
| Optimizer | AdamW 8-bit |
| Max Grad Norm | 1.0 |
| Sequence Length | 4,096 |
| Batch Size | 1 per GPU (eff. 4) |
| Epochs | 3 |
| Gradient Checkpointing | ✅ |
| Precision | BF16 |
| Merge Scale | 0.4 |
✅ What This Model Excels At
- Coding agent tasks — Planning, debugging, building real projects
- Step-by-step reasoning — Autonomous agent trace style
- Tool calling — File operations, shell commands, testing
- Multi-language — Python, C, C++, Go, Java, Rust, Bash, etc.
- Multi-turn — Full trajectory conversations
- Thai + English — Native bilingual capability
📊 Dataset Statistics
| Metric | Value |
|---|---|
| Original rows | 4,928 |
| Cleaned rows | 3,389 |
| Train / Val | 3,219 / 170 |
| Multi-turn rate | 61.3% |
| Mean tokens | 1,055 |
| Max tokens | 3,579 |
| Categories | 156 |
⚠️ Limitations
- 4B model — Smaller than 27B/72B models
- Preview — Early release
- Text-only — Text decoder only (no vision encoder)
- Coding-focused — Excels at coding/debugging, general knowledge may vary
💖 Support / โปรดสนับสนุน
If you find this model useful, please consider supporting my work!
หากคุณคิดว่าโมเดลนี้มีประโยชน์ กรุณาสนับสนุนผลงานของฉันด้วยนะคะ! 🙏
₿ Bitcoin — BTC:
bc1qf27cyk3vmugcdyv9xdtuv5jwz37863crpj5c9v
Thank you for your support! 🙏✨
ขอบคุณมากๆ สำหรับการสนับสนุนค่า! 💖🤗
🙏 Acknowledgements / ขอบคุณ
- InternScience — For the Agents-A1-4B base model
- greghavens — For the Kimi K3 coding traces dataset
- Qwen Team (Alibaba) — For the Qwen3.5 architecture
- Unsloth AI — For the training optimizations
- All dataset contributors and the open-source AI community ❤️
Built with ❤️ by UKA — 18-year-old coder & cybersecurity expert
- Downloads last month
- -
Model tree for hotdogs/Agents-A1-4B-kimi-Preview
Base model
InternScience/Agents-A1-4B