readme.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,57 @@
|
|
| 1 |
-
---
|
| 2 |
-
|
| 3 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Atlas-1B: Lightweight Fine-tuned LLM for Edge and Low-Memory Devices
|
| 2 |
+
|
| 3 |
+
🚀 **Atlas-1B** is a 1.2-billion parameter model fine-tuned from **BaseLLM-1B** to deliver improved accuracy, reasoning, and efficiency on low-power inference devices (e.g., Jetson, Ryzen APU, and mobile-based LLM frameworks).
|
| 4 |
+
This version introduces **quantization-aware finetuning**, **dataset specialization**, and **token efficiency optimization**, making it a solid drop-in model for on-device AI use cases.
|
| 5 |
+
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
## 🧠 Model Overview
|
| 9 |
+
|
| 10 |
+
- **Base model:** BaseLLM-1B v1.3 (transformer-based autoregressive)
|
| 11 |
+
- **Architecture:** Decoder-only transformer
|
| 12 |
+
- **Parameters:** 1.2B
|
| 13 |
+
- **Precision support:** FP16 / INT8 / INT4
|
| 14 |
+
- **Context length:** 16K tokens
|
| 15 |
+
- **Tokenizer:** SentencePiece (32K vocab)
|
| 16 |
+
- **Frameworks supported:** PyTorch, vLLM, and sglang
|
| 17 |
+
|
| 18 |
+
This model was optimized specifically for **edge inference** and **multi-request throughput**, providing ~30% lower memory bandwidth usage at batch=4 compared to the base model.
|
| 19 |
+
|
| 20 |
+
---
|
| 21 |
+
|
| 22 |
+
## 🧩 Use Cases
|
| 23 |
+
|
| 24 |
+
- On-device chat assistants
|
| 25 |
+
- Smart IoT response systems
|
| 26 |
+
- Embedded analytics (offline summarization, intent detection, etc.)
|
| 27 |
+
- Lightweight reasoning for robotics
|
| 28 |
+
|
| 29 |
+
---
|
| 30 |
+
|
| 31 |
+
## 🔧 Fine-tuning Details
|
| 32 |
+
|
| 33 |
+
| Attribute | Description |
|
| 34 |
+
|------------|-------------|
|
| 35 |
+
| **Dataset** | Blend of 50M tokens curated for code, chat, and reasoning |
|
| 36 |
+
| **Training framework** | PyTorch + DeepSpeed ZeRO-2 |
|
| 37 |
+
| **Optimizer** | AdamW |
|
| 38 |
+
| **Learning rate** | 2e-5 (cosine decay) |
|
| 39 |
+
| **Batch size** | 512 tokens per GPU |
|
| 40 |
+
| **Epochs** | 3 |
|
| 41 |
+
| **Loss function** | Cross-entropy (token-level) |
|
| 42 |
+
| **Special techniques** | LoRA adapters (rank=8), QLoRA-aware finetuning, FlashAttention-2 integration |
|
| 43 |
+
|
| 44 |
+
---
|
| 45 |
+
|
| 46 |
+
## 🧪 Performance Benchmarks
|
| 47 |
+
|
| 48 |
+
| Metric | BaseLLM-1B | Atlas-1B |
|
| 49 |
+
|--------|-------------|----------|
|
| 50 |
+
| **MMLU (Subset)** | 30.2 | 38.7 |
|
| 51 |
+
| **CodeEval (Python)** | 22.4 | 29.1 |
|
| 52 |
+
| **Average latency (Jetson Orin, INT4)** | 213ms | 158ms |
|
| 53 |
+
| **Memory usage (FP16)** | 7.9GB | 5.4GB |
|
| 54 |
+
|
| 55 |
+
> Benchmarks measured with vLLM 0.4.2 and sglang backend on an RTX 3060 (12GB) and Jetson Orin AGX.
|
| 56 |
+
|
| 57 |
+
|