dustin2050/finrag-gpt-oss-120b-v2

FinRAG Trading Expert — gpt-oss-120b (v2)

A LoRA adapter that turns openai/gpt-oss-120b into a quantitative trading analyst: it audits backtests for bias, extracts implementable strategy specifications from research papers, reasons about market microstructure and derivatives, and designs risk management for live deployment.

The adapter is distilled from GLM-5.2 teacher responses over 500 full-text quantitative-finance arXiv papers, real trading-project material, and a curated slice of public finance reasoning data.

⚠️ Not investment advice. See Limitations & Risks. This model produces analysis and hypotheses, not trading signals. Nothing it outputs should be traded without independent validation.

---

What it is good at

Capability	Example task
Backtest auditing	"567 % CAGR with 30 % max DD — list the red flags before trusting this."
Bias detection	look-ahead, survivorship, data-snooping, in-sample overfitting, cost underestimation
Paper → strategy	extract entry/exit rules, position sizing, parameters, and pseudocode from a full paper
Replication planning	what data, what walk-forward split, what would falsify the paper's claim
Strategy design	complete specs for equities / futures / FX / crypto incl. risk limits
Quant concepts	convexity & DV01, vol-surface skew, CDS-bond basis, VWAP/implementation shortfall, MEV & AMM mechanics
Realism	skepticism toward extreme CAGR claims, explicit transaction-cost and capacity reasoning

The training data is written in an "elite quant strategist and backtest auditor" voice that thinks step by step, flags biases, and distinguishes in-sample from out-of-sample results.

---

Model Details

• Developed by: dustin2050
• Model type: LoRA adapter (PEFT) for a Mixture-of-Experts causal LM
• Base model: openai/gpt-oss-120b — 117 B total / ~5.1 B active parameters (trained through the 4-bit build unsloth/gpt-oss-120b-unsloth-bnb-4bit)
• Language: English
• License: Apache-2.0 (adapter). The base model's own license applies to the base weights.
• Finetuned from teacher: GLM-5.2 (z-ai/glm-5.2, via OpenRouter) — responses distilled into the student

LoRA configuration

Parameter	Value
Rank r	16
lora_alpha	32
lora_dropout	0.05
Bias	none
Target modules	q_proj, k_proj, v_proj, o_proj, gate_up_proj, down_proj (attention and MoE expert projections)
Trainable params	~1.07 B of 117.9 B (0.91 %)

---

Training Data — distill_v3 (14,452 examples)

Bucket	Examples	Description
Full-text papers	2,324	GLM-5.2 analyses of 500 unique q-fin arXiv papers (full PDF text, not abstracts) across 4 task types: deep audit, critique, strategy extraction, implementation plan
Paper abstracts	3,230	arXiv + OpenAlex q-fin abstracts → replication plans, signal combination, critique
Real trading projects	1,357	audits/lessons from actual bot codebases and backtests (XAUUSD, Forex/Renko, ML-DT, stock screening)
Self-instruct	1,542	synthetic quant scenarios (cost analysis, regime shifts, capacity, black-swan)
External quant Q&A	6,000	filtered from ianncity/GLM-5.2-Finance-80000x (Apache-2.0)

Filtering of the external set: from 79,112 rows, corporate-finance/accounting/PE/insurance topics were removed (11,993), and the remainder was balanced across 9 quant themes (~690 each): microstructure & execution · derivatives & options · fixed income & rates · crypto/DeFi · risk & portfolio · strategy & alpha · liquidity & credit · macro & FX · behavioral markets. It was chosen because it shares this project's teacher (GLM-5.2), its <think> reasoning format, and has 0 % truncation.

Quality properties:

• 99 % of examples carry explicit <think> reasoning before the answer
• Median ~2,850 tokens per example; 99.9 % of content fits within the 12,288-token training window
• Mid-generation truncated answers removed; teacher regenerated with max_tokens=12000
• Deduplicated on the full question (a prefix-based key would have silently deleted nearly all full-text paper examples, since they share an identical instruction prefix)
• Composition deliberately keeps 58 % domain-specific trading distillation vs 41 % general quant knowledge so the specialised auditing behaviour stays dominant

---

Training Procedure

Setting	Value
Method	QLoRA (4-bit NF4 base) with Unsloth Faster-MoE kernels
Sequence length	12,288
Batch	3 per device × 4 grad-accum (effective 12)
Epochs	1 (1,169 optimizer steps over 14,018 training examples)
Optimizer / LR	adamw_8bit, 2e-4, 30 warmup steps, linear decay
Precision	bf16 compute, 4-bit base weights
Batching	group_by_length (gpt-oss does not support sample packing in Unsloth)
Hardware	1× NVIDIA RTX PRO 6000 Blackwell (96 GB), AMD Ryzen 9 9950X
Wall-clock	9 h 59 min
Train loss	5.01 → 0.82

Why sequence length 12,288 matters

An earlier iteration of this project trained at max_seq_length=2048, which captured only ~54 % of the training content — 93 % of examples were cut mid-answer and full-text papers were truncated to ~38 % of their length. At 12,288 tokens, 99.9 % of the content reaches the model, and full papers fit whole.

---

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

BASE = "unsloth/gpt-oss-120b-unsloth-bnb-4bit"   # 4-bit, ~65 GB VRAM
ADAPTER = "dustin2050/finrag-gpt-oss-120b-v2"

tok = AutoTokenizer.from_pretrained(ADAPTER)
model = AutoModelForCausalLM.from_pretrained(BASE, torch_dtype=torch.bfloat16, device_map="auto")
model = PeftModel.from_pretrained(model, ADAPTER)

messages = [{"role": "user", "content":
    "A momentum strategy on S&P 500 futures shows 340% CAGR over 4 years with 18% max drawdown "
    "and 40 trades/month. Audit it: what would you check before believing it?"}]

inputs = tok.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True).to(model.device)
out = model.generate(inputs, max_new_tokens=2048, temperature=0.7, top_p=0.9, do_sample=True)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

With Unsloth (faster load):

from unsloth import FastLanguageModel
model, tok = FastLanguageModel.from_pretrained(
    "unsloth/gpt-oss-120b-unsloth-bnb-4bit", max_seq_length=12288, load_in_4bit=True)
from peft import PeftModel
model = PeftModel.from_pretrained(model, "dustin2050/finrag-gpt-oss-120b-v2")
FastLanguageModel.for_inference(model)

Hardware requirements

Setup	Requirement
4-bit inference	~65 GB VRAM (e.g. RTX PRO 6000 96 GB, H100 80 GB, A100 80 GB)
Apple Silicon	96 GB+ unified memory (int4); ~25–35 tok/s on a 128 GB M-series machine
Not sufficient	RTX 4090 / 5090 (24–32 GB) without heavy CPU offloading

The prompt style the model was trained on: a direct analytical question, optionally with a paper excerpt or project code pasted in. It responds with <think> reasoning followed by a structured markdown answer.

---

Intended Use

In scope

• Research assistance for quantitative traders and strategy developers
• Critical review of backtests, papers, and strategy write-ups
• Educational exploration of quant finance concepts
• Generating hypotheses and implementation plans to be validated independently

Out of scope

• Generating live trading signals or portfolio allocations for real capital
• Any use as a substitute for licensed financial advice
• Fully autonomous trade execution
• Non-English use (training data is English-only)

---

Limitations & Risks

• Not investment advice. Outputs are analytical text, not recommendations. Trading involves risk of substantial loss.
• Teacher-inherited errors. The model imitates GLM-5.2; factual mistakes, invented citations, or plausible-but-wrong numerical reasoning can be reproduced confidently. Verify every number and citation.
• No verified numeric accuracy. The training data was not numerically ground-truthed; arithmetic in cost/Sharpe calculations may be wrong.
• Knowledge cut-off & drift. Papers are ~2024–2026 q-fin arXiv; market regimes and published alphas decay. The model may present decayed edges as viable.
• Single-epoch, single-run. No held-out evaluation was used during training (the gpt-oss eval path is broken in this Unsloth build), so quality is evidenced by training loss and qualitative review only. A systematic evaluation is in progress.
• Style bias. Long, structured, markdown-heavy answers with a skeptical tone — this is intended, but can read as overconfident.
• Distribution skew. 41 % of training data is general quant Q&A in a uniform "How does X impact Y when Z" template, which may bias the model toward that answer shape.

---

Acknowledgements

• Base model: OpenAI gpt-oss-120b
• Teacher: Z.ai GLM-5.2 (via OpenRouter)
• Training stack: Unsloth (Faster-MoE kernels), TRL, PEFT, Transformers
• External data: ianncity/GLM-5.2-Finance-80000x (Apache-2.0)
• Source literature: open-access q-fin preprints from arXiv and OpenAlex

Citation

@misc{finrag_gptoss120b_v2_2026,
  title  = {FinRAG Trading Expert: a GLM-5.2-distilled LoRA for gpt-oss-120b},
  author = {dustin2050},
  year   = {2026},
  note   = {LoRA adapter, distilled from 500 full-text q-fin papers and curated quant reasoning data},
  url    = {https://huggingface.co/dustin2050/finrag-gpt-oss-120b-v2}
}