dustin2050/finrag-trading-expert-qwen3-32b

FinRAG Trading Expert — Qwen3-32B (Distilled)

Overview

A domain-specific fine-tuned model distillation of Qwen3-32B for quantitative trading strategy analysis, backtest auditing, and risk management. The model was trained on 6,305 high-quality examples generated by GLM-5.2 as teacher, covering forex, crypto, stocks, and commodities markets.

Training Details

Parameter	Value
Base Model	Qwen/Qwen3-32B (4-bit quantized)
Method	QLoRA (4-bit NF4 + LoRA r=64)
Teacher	GLM-5.2 via OpenRouter
Framework	Unsloth 2026.7.5 + TRL 0.24.0
Hardware	NVIDIA RTX PRO 6000 Blackwell (96GB VRAM)
Training Time	11h 20min
Total Steps	1,146 (3 epochs)
Batch Size	16 (8 × 2 gradient accumulation)
Learning Rate	2e-4
Max Sequence Length	2048
LoRA Rank	64
LoRA Alpha	64
LoRA Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Trainable Parameters	536,870,912 (1.61% of 33.3B)
Optimizer	adamw_8bit
Packing	Enabled (Unsloth padding-free)

Training Data

The training dataset contains 6,305 examples from 7 sources:

Source	Examples	Description
arXiv q-fin Abstracts	2,334	Papers on trading, momentum, arbitrage, risk management (2023-2026)
Self-Instruct v1+v2	1.892	Strategy design, cost analysis, regime detection, risk management scenarios
Trading-Fable5	319	Real failure patterns from 38 trading strategy tracks (look-ahead bias, OOS collapse)
Forex_Renko Phase-Docs	232	Phase 0-40 audit results, Codex reviews, backtest validation
Workspace Projects	827	Bot_KI_Agents, Reverse_Engineering_XAUUSD, stock3, ML-DT-Trading, xauusd-cbot
arXiv Fulltext Papers	200	200 papers as full text (15k chars each) with deep audit prompts
FinGPT	200	Financial sentiment and headline classification

Data Quality Pipeline

1. Paper Collection: 6,089 unique papers from arXiv (q-fin.*) and OpenAlex
2. Fulltext Download: 200 top papers downloaded as PDF, extracted to text (~15k chars each)
3. Teacher Generation: GLM-5.2 generates detailed answers with chain-of-thought reasoning
4. Post-Processing:
   • <think> tags for reasoning separation (99% tagged)
   • Deduplication (max 30 per template for Self-Instruct, 50 for papers)
   • Truncation to max 14k chars (seq_length compatible)
   • Removal of generic finance Q&A (FinAlpaca removed — not trading-relevant)

Evaluation Results

Evaluated on 40 trading-specific questions comparing Base Qwen3-32B vs. Distilled:

Metric	Base Qwen3-32B	Distilled (LoRA)
Avg Response Length	8,450 chars	6,854 chars
Avg Time per Answer	116.7s	152.6s
Questions Answered	40/40	40/40

Qualitative Improvements

The distilled model shows:

1. Direct Skepticism: Immediately flags suspicious CAGR (>100%) as red flag, unlike base model which explains basic concepts first
2. Structured Output: Uses numbered lists with concrete bias types (look-ahead, survivorship, data-snooping)
3. Domain Precision: Mentions specific concepts (Kelly criterion, McLean & Pontiff alpha decay, walk-forward validation, OOS degradation)
4. Compact Expertise: Less introductory text, more actionable analysis
5. Real Failure Patterns: Draws on training data from actual failed trading strategies (Trading-Fable5 tracks, Forex_Renko phases)

Category Coverage

Category	Questions	Topics
Overfitting Detection	4	CAGR red flags, Sharpe degradation, compound CAGR bias
Validation	4	Walk-forward, train/test split, param vs cost sensitivity
Risk Management	6	Drawdown filters, Kelly criterion, kill-switch, vol-targeting
Bias Detection	4	Look-ahead, survivorship, data snooping vs overfitting
Strategy Design	6	BTC momentum, crypto pairs, execution plans, funding arbitrage
Strategy Comparison	4	Carry vs momentum, position sizing methods, portfolio construction
Cost Analysis	4	Transaction costs, breakeven, Calmar ratio
Debugging	2	Backtest vs live gaps, suspicious Sharpe
Paper Critique	3	Factor replication, ML accuracy claims, alpha decay
Regime Detection	3	Regime change detection, Hurst exponent, grid trading

Usage

With Unsloth

from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="dustin2050/finrag-trading-expert-qwen3-32b",
    max_seq_length=2048,
    dtype=None,
    load_in_4bit=True,
)
FastLanguageModel.for_inference(model)

messages = [{"role": "user", "content": "Design a momentum strategy for EUR/USD with walk-forward validation."}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
outputs = model.generate(input_ids=inputs, max_new_tokens=2048, temperature=0.6, top_p=0.95)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

With PEFT (transformers)

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-32B", torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(base_model, "dustin2050/finrag-trading-expert-qwen3-32b")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-32B")

messages = [{"role": "user", "content": "Audit this backtest: 567% CAGR, 30% DD. What are the red flags?"}]
inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
outputs = model.generate(input_ids=inputs, max_new_tokens=2048, temperature=0.6, top_p=0.95)
print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))

Recommended Inference Settings

Parameter	Value
Temperature	0.6 (thinking mode)
Top_P	0.95
Top_K	20
Max New Tokens	2048

Limitations

1. Not Financial Advice: This model generates analysis for research purposes only. Do not use for real trading decisions without independent validation.
2. No Live Trading: The model cannot execute trades or access real-time market data.
3. Knowledge Cutoff: Training data covers papers up to July 2026.
4. Reasoning Depth: While the model shows structured trading analysis, it cannot replace full backtesting or professional risk management.
5. GGUF Export: Currently only LoRA adapter is available. GGUF export pending.

Training Infrastructure

• Cloud Provider: Vast.ai (RTX PRO 6000 Blackwell, 96GB VRAM)
• Training Framework: Unsloth 2026.7.5 (2x faster, 70% less VRAM)
• Teacher API: OpenRouter (GLM-5.2 / z-ai/glm-5.2)
• Data Pipeline: Custom Python pipeline (arXiv API + OpenAlex API + PyMuPDF)

Acknowledgments

• Unsloth for the optimized training framework
• Qwen Team for the base model
• OpenRouter for API access to GLM-5.2
• arXiv and OpenAlex for paper access

License

Apache 2.0 (inherited from Qwen3-32B base model)