🚀 Snapgate Surge V3
Official AI Assistant of Snapgate AI
Model Surge V3 is the latest version of Snapgate Surge, fine-tuned on a custom conversation dataset to shape a consistent persona and response style for Snapgate AI.
⚡ Model Specifications
| Specification | Detail |
|---|---|
| Model Name | Snapgate Surge V3 |
| Parameters | ~7B |
| Fine-tuning Method | LoRA (r=16, alpha=32), 4-bit quantization |
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Framework | Unsloth + TRL SFTTrainer |
| Languages | Indonesian & English |
✨ Key Capabilities
| Feature | Description |
|---|---|
| 💻 Coding Assistant | PHP, JavaScript, Python, HTML/CSS, and other languages |
| 🌐 Web Development | Building, debugging, and optimizing websites |
| 💬 Bilingual Chat | Indonesian and English |
| 🧠 General Knowledge | Answers general questions about technology & AI |
| 🏷️ Consistent Persona | Always identifies itself as Surge from Snapgate AI |
🚀 Usage
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "kadalicious22/snapgate-surge-v3-moe",
max_seq_length = 2048,
load_in_4bit = True,
)
FastLanguageModel.for_inference(model)
messages = [
{"role": "system", "content": "You are Surge, an AI assistant from Snapgate AI."},
{"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
).to("cuda")
outputs = model.generate(input_ids=inputs, max_new_tokens=256, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
📊 Training Details
- Dataset: custom Snapgate AI conversations (identity, coding, general Q&A)
- Fine-tuning method: LoRA on top of a 4-bit quantized base model
- Checkpoints saved periodically to support resumable training
Training Progress
| Step | Training Loss |
|---|---|
| 10 | 1.788681 |
| 20 | 0.343310 |
| 30 | 0.151723 |
| 50 | 0.107738 |
| 60 | 0.086470 |
| 70 | 0.037520 |
| 80 | 0.026020 |
| 90 | 0.017798 |
| 100 | 0.015079 |
| 110 | 0.013477 |
| 120 | 0.012742 |
| 130 | 0.011483 |
| 140 | 0.010780 |
| 150 | 0.010536 |
| 160 | 0.010176 |
| 170 | 0.008932 |
| 180 | 0.009087 |
| 190 | 0.008987 |
| 200 | 0.009272 |
| 210 | 0.008632 |
| 220 | 0.008642 |
| 230 | 0.008501 |
| 240 | 0.009492 |
| 250 | 0.009299 |
| 260 | 0.009250 |
| 270 | 0.009292 |
| 280 | 0.009150 |
| 290 | 0.009015 |
| 300 | 0.008798 |
Training loss dropped sharply within the first ~100 steps and stabilized in the 0.008–0.01 range from step ~150 onward, indicating the model converged well on the training persona/style without significant further improvement past that point.
🔗 Links
- Website: snapgate.tech