bitai-hub/distilled_v3

BITAI Distilled v3 — Stock & Crypto Trading Agent

AI Forging AI — Distilled from multi-model cross-discussions on 4,000 topics, hardened through real live trading.

---

Model Description

distilled_v3 is a lightweight (1.5B) trading and financial analysis model distilled from the BITAI Collective multi-model cross-discussion pipeline. Teachers (Gemma-4-26B + Ornith-1.0-9B) debated 4,000+ topics covering cryptocurrency trading, stock analysis, financial risk assessment, and legal compliance. The resulting 3,764 high-quality samples (quality score ≥ 0.5) were used for multi-answer knowledge distillation with label smoothing, preventing the student from memorizing a single answer while learning the distribution of expert reasoning.

The model has been validated through live paper-trading simulations with 25x leverage on BTC/USDT perpetual contracts, making autonomous open/close decisions based on real market data. It includes a built-in loss self-reflection mechanism — after each losing trade, the model analyzes the mistake and appends a structured reflection to its memory file, enabling continuous learning through experience.

This is not a theoretical model — it has been battle-tested in simulated real-time trading environments on the BITAI platform.

---

模型描述

distilled_v3 是基于 BITAI Collective 多模型交叉推理讨论蒸馏的轻量级 (1.5B) 交易与金融分析模型。教师模型（Gemma-4-26B + Ornith-1.0-9B）就 4,000+ 个话题进行了对抗性辩论，涵盖加密货币交易、股票分析、金融风险评估和法律合规等领域。经质量门控（得分 ≥ 0.5）筛选出 3,764 条高质量样本，采用多答案蒸馏 + 标签平滑训练，防止学生死记硬背单一答案，学会专家推理的分布。

该模型已通过 25 倍杠杆 BTC/USDT 永续合约的实盘模拟交易验证，基于真实市场数据自主决策开平仓。内置亏损自我反省机制——每次亏损交易后，模型自动分析错误原因并将结构化反思追加到记忆文件中，实现从经验中持续学习。

这不是理论模型——它已在 BITAI 平台的实时模拟交易环境中经受实战考验。

---

Training Details / 训练细节

Parameter	Value
Base model	Qwen/Qwen2.5-1.5B-Instruct
Training method	Multi-answer Knowledge Distillation + LoRA
LoRA rank	32
LoRA alpha	64
LoRA dropout	0.2
Target modules	q_proj, v_proj, k_proj, o_proj
Label smoothing	0.1
Optimizer	AdamW (weight_decay=0.01)
LR scheduler	CosineAnnealing (T_max=5)
Learning rate	3e-4
Batch size	2
Epochs	2
Discussion topics	4,000+ (crypto, stock, legal, finance)
Training samples	3,764 (quality ≥ 0.5 gate)
Hardware	NVIDIA H100 (80GB × 2)

Teacher Models / 教师模型

• **Gemma-4-26B — Large model, establishes the analytical framework
• Ornith-1.0-9B — Critical examiner, identifies weaknesses and gaps
• The two engage in adversarial debate: the large model builds a framework, the small model critically examines it, and they converge on a consensus answer

Data Pipeline / 数据流水线

1. Submit question via management panel
2. Cross-discussion: Gemma + Ornith debate in parallel
3. Quality gate: score ≥ 0.5 (min_length_100 check + semantic evaluation)
4. Multi-answer extraction: consensus + participant responses + multi-round replies
5. Distillation training: label smoothing + LoRA

---

Usage / 使用方法

Python (HuggingFace Transformers)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model = "Qwen/Qwen2.5-1.5B-Instruct"
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, "bitai-hub/distilled-v3")

tokenizer = AutoTokenizer.from_pretrained(base_model)

prompt = "Analyze BTC/USDT technical indicators and give a trading signal."
messages = [
    {"role": "system", "content": "You are a professional crypto trading analyst."},
    {"role": "user", "content": prompt},
]
text = tokenizer.apply_chat_template(messages, tokenize=False)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7, top_p=0.9)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

llama.cpp / GGUF

If GGUF version available:

./llama-cli -m distilled-v3-Q4_K_M.gguf \
  -p "Analyze BTC/USDT technical indicators and give a trading signal." \
  -n 256 -t 8 --temp 0.7

---

Trading Features / 交易功能

Live Trading Simulation / 实盘模拟交易

The model is designed to work with the BITAI trading engine (crypto_trading_sim.py):

python crypto_trading_sim.py \
  --api-url http://localhost:1236/v1 \
  --model distilled-v3 \
  --runs 20 \
  --capital 1000

• Leverage: 25x on BTC/USDT perpetual contracts
• Direction: Long/Short both supported
• Decision cycle: Every 30 seconds (real-time market data)
• Risk control: 30% margin per position, auto-stop logic

Loss Self-Reflection / 亏损自我反省

After every losing trade, the model automatically:

1. Logs the trade details (entry/exit price, PnL, direction)
2. Analyzes the mistake (wrong direction? bad timing? market anomaly?)
3. Saves structured reflection to ./trade_data/memory_*.json
4. References past reflections in future decisions to avoid repeating errors

---

Quantized Versions / 量化版本

Format	Size	Description
F16 (full)	~3.0 GB	Full precision, best quality
Q8_0	~1.6 GB	High quality, negligible loss
Q4_K_M	~0.9 GB	Recommended balance of size/speed
Q4_K_S	~0.8 GB	Smallest, slightly lower quality

---

Evaluation Results / 评测结果

Live 25x leverage trading simulation (BTC/USDT):

Metric	Value
Win rate	Reported after live run
Profit factor	Reported after live run
Max drawdown	Reported after live run
Total PnL	Reported after live run

Contact admin@bitai.one for latest evaluation reports.

---

Links / 相关链接

• 🌐 Website: https://bitai.one
• 📄 Paper: Multi-Model Adversarial Cross-Discussion
• 🖥️ Management Panel: https://bitai.one/demo
• 🔗 Blockchain Explorer: https://bitai.one/explorer
• 💬 Community Forum: https://bitai.one/forum
• 📧 Contact: admin@bitai.one

---

Citation / 引用

@misc{bitai-distilled-v3,
  author = {BITAI Chain},
  title = {BITAI Distilled v3: Stock and Crypto Trading Agent from Multi-Model Co-Distillation},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/bitai-hub/distilled-v3}},
}

---

License

This model is released under Apache 2.0. The base model Qwen2.5-1.5B-Instruct is governed by its own license (Qwen License).