Super-Fin-4B (Run 3 — current adapter)

Super-Fin is a financial research agent language model. This repo holds the Super-Fin LoRA adapter (QLoRA, r64) for the base model Qwen/Qwen3.5-4B (Qwen Team, Alibaba), released under the Apache License 2.0. This adapter is likewise released under Apache-2.0; attribution to the base model is made here as required. The adapter currently at HEAD is Run 3; the Run 1 and Run 2 sections are preserved below for the record.

Research / education only. Super-Fin produces analysis for research and educational purposes. Nothing it outputs is personal financial advice, and it must not be presented as such. Order execution is disabled by design — Super-Fin has no capability to place trades, and none will be added.

Run 1 summary

• Method: QLoRA SFT — 4-bit NF4 base, LoRA r64 / alpha 128 / dropout 0.05, all-linear targets, lr 1e-4 cosine, effective batch 16, max seq 2048, 1,032 optimizer steps (2 epochs over the SFT set, delivered as eight chained 129-step segments due to Colab runtime instability). Final segment train loss 0.288 (first segment 0.424).
• Venue: Google Colab A100-SXM4-40GB via the Colab CLI.

Training data (8,423 examples; train 8,255 / val 168)

Source	Examples	Notes
Market literacy (computed daily/weekly summaries from collected OHLCV)	6,834	254 records excluded under the bench held-out rule (see below)
Verified tool-use SFT (8 finance tools, real tool outputs)	480	tool calls in <tool_call> format
SEC EDGAR fundamentals Q&A/summaries (49 US companies)	1,109	every number from SEC company facts; growth computed in code
FRED macro	0	series unavailable at build time (endpoint timeouts); documented gap

Held-out enforcement: Finance Bench v1 (symbol, date) pairs were excluded from training with the Phase 1 split rule — 131 daily records by exact pair + 123 weekly records whose window contains a bench date; programmatic assertion verified 0 bench (symbol, date) pairs in training. Known limitation: bench and tool-SFT share the same 12 symbols, so v1 measures date-held-out, not symbol-held-out, generalisation (Bench v2 item).

Finance Bench v1 (440 items, deterministic, program-computed answers)

Category	n	BASE (Qwen3.5-4B)	TUNED (Super-Fin Run 1)
candle_reading	120	0.9833	0.4000
indicator_interpretation	100	0.7800	0.7800
return_volatility_calc	80	0.1625	0.2875
risk_identification	60	0.9833	0.9667
tool_selection	80	0.1354	0.6979
Overall	440	0.6337	0.5973

Reading the result honestly: Run 1 is the proof run. Tool selection improved +0.56 and calculation +0.13, but candle_reading regressed sharply: the tuned model over-triggers tool calls (it tries to fetch a summary that is already in the prompt) and answers verbosely where the bench demands a terse value. Both failure modes are data-mixture issues identified from the prediction logs and are the top fix items for Run 2 (in-context-answer examples, terse-format variants, negative tool-use examples). Sample generations confirm the trained skills work in-format: correct tool name/arguments for indicator requests and correct fundamentals answer structure (exact figure recall is still imperfect — by design, numbers are meant to come from tools at inference time).

Run 2 summary

• Method: QLoRA SFT — identical configuration to Run 1 (the only lever changed is the data): 4-bit NF4 base, LoRA r64 / alpha 128 / dropout 0.05, all-linear targets, lr 1e-4 cosine, effective batch 16, max seq 2048. 1,420 optimizer steps = exactly 2 epochs over the 11,322-example training set, delivered as ten chained 142-step chunks with the adapter downloaded locally after every chunk (Colab runtime-reclamation insurance). Chunk train losses declined monotonically: 0.421 → 0.330 → 0.322 → 0.316 → 0.312 → 0.308 → 0.301 → 0.303 → 0.298 → 0.295.
• Venue: Google Colab A100-SXM4-40GB via the Colab CLI. One incident: chunk 5 crashed with a CUDA OOM at step 16 (a long-sequence batch spike) and was relaunched with PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True, which was kept for all later chunks; no other chunk was affected and no runtime was reclaimed this run.

Training data v2 (11,553 examples; train 11,322 / val 231)

Source	Examples	Notes
Market literacy (computed daily/weekly summaries, 95 symbols)	7,779	scaled up from 8 headline symbols (Run 1) via the Phase 1 cleaning/indicators pipeline; all dated content ≤ 2024-12-31
Verified tool-use SFT (carried over)	453	of the Phase 1 480; 27 dropped by the Bench v2 date rule
Tool chains (NEW)	240	verified 2-step chains (e.g. get_price_history → get_indicators); every final-answer number produced by actually calling the tools
Fix data (NEW)	1,220	targets Run 1's failure mode: in-context extractive/interpretive/calculation questions answered directly and tersely + negative tool-use examples (data already present → no tool call)
SEC EDGAR fundamentals (44 companies)	1,501	5 bench-holdout tickers excluded
FRED macro (NEW)	360	from the 8 FRED series available (GDP/CPI/UNRATE still unreachable from this network — documented gap)

Held-out enforcement: Bench v2 applies both holdout rules — a fixed 2025-01-01 date cutoff and symbol-level holdout (an 18-symbol bench set disjoint from all training symbols). Programmatic verification: symbol overlap 0, (symbol, date) overlap 0, and a token-level audit found 0 holdout-symbol mentions anywhere in the training files.

Finance Bench v2 (440 items, deterministic, program-computed answers)

Bench v2 is a new, stricter bench — different items from v1, unseen symbols, post-2025 dates. Its scores are not directly comparable to the v1 numbers above; the valid comparison is BASE vs TUNED within v2.

Category	n	BASE (Qwen3.5-4B)	TUNED (Super-Fin Run 2)
candle_reading	120	1.0000	1.0000
indicator_interpretation	100	0.8500	0.9200
return_volatility_calc	80	0.0500	0.0000 (see note)
risk_identification	60	0.9500	1.0000
tool_selection	80	0.1344	0.3979
Overall	440	0.6290	0.6905

Reading the result honestly: Run 2's headline fix landed — candle_reading stayed at 1.0000 with zero spurious tool calls in the tuned predictions (Run 1's tuned model collapsed to 0.40 on v1 largely from over-triggering tools), and the qualitative in-context sample answers tersely and correctly. Indicator interpretation and risk identification improved. Two scores need context:

• return_volatility_calc = 0.0 is an extraction mismatch, not zero capability. The tuned model writes worked calculations in the trained "show calculations" style — e.g. (48.55 / 51.66 - 1) x 100 = -6.02% — ending in the correct result, but the frozen bench harness extracts the first number in the output (an operand) and scores it wrong. Re-scoring the identical predictions with last-number extraction gives 0.588 (BASE: 0.050 under either extraction). The official 0.0 stands as measured; a final-line answer format is a Run 3 fix item.
• tool_selection (+0.26) understates the gain the same way: of the 44 zero-scored items, 43 chose the right tool with the right core arguments but kept generating after the answer (a second call in <tool_call> format or narration), which breaks the harness's JSON-span extraction; only 1 zero is a pure argument mismatch. Remaining real gaps: some get_price_history calls use limit where the bench expects an explicit start date (partial credit 0.83), and generation does not always stop after the tool call.

Fundamentals recall remains near-miss from memory (e.g. MSFT FY2025 revenue answered $245.12B vs SEC actual $281.72B — the figure given matches FY2024), reinforcing the design rule: exact numbers must come from tools at inference time, not from weights.

Run 3 summary

• Method: QLoRA SFT — same recipe as Runs 1–2 (4-bit NF4 + double quant, bf16, LoRA r64 / alpha 128 / dropout 0.05, all-linear targets, lr 1e-4 cosine with warmup, effective batch 16, grad checkpointing, language-model-only). The one necessary adaptation is sequence length: dataset v3 contains a long-context pack up to 30,168 tokens, so training used length-bucketed batching (bulk examples at ≤4,096 tokens; long examples in their own micro-batches of 1 with max seq 32,768, accumulation adjusted to keep the effective batch ≈16) with chunked cross-entropy over backbone hidden states. ~1,930 optimizer steps ≈ 2 epochs over 15,586 training examples, delivered as 20 chained data chunks with per-chunk adapter checkpoints (Colab A100 runtimes were reclaimed ~60 times during this run; every chunk boundary was captured locally and, where possible, backed up under runs/run3/chunks/).
• Engineering note: long-sequence OOM at ~30K tokens was root-caused to prepare_model_for_kbit_training casting input embeddings to fp32 (SDPA falls back to a quadratic fp32 math path). Recasting the frozen embeddings back to bf16 after adapter setup fixed it (a 29,954-token step peaks at ~21 GiB on an A100-40GB). The chunked GatedDeltaNet kernels from flash-linear-attention are required for this base model's linear-attention layers at long sequence lengths.
• Venue: Google Colab A100-SXM4-40GB via the Colab CLI. Total Run 3 cost (prep + training + evals): ~228 compute units.

Training data v3 (15,904 examples; train 15,586 / val 318)

Source	Examples	Notes
Dataset v2 carried over (all six v2 sources, holdout-clean)	11,553	unchanged from Run 2
Fix pack — calculation format (NEW)	697	answers end with a single Final answer: <value> line
Fix pack — tool JSON, clean stop (NEW)	291	one JSON object, generation stops
Fix pack — <tool_call> format, clean stop (NEW)	228
Fix pack — fresh verified 2-step chains (NEW)	200
Long-context pack — SEC filing excerpts (NEW)	65	10-K/10-Q MD&A excerpts, ~9–17K tokens
Long-context pack — multi-document bundles (NEW)	235	sized in two bands (~8–14K, ~16–30K tokens)
Teacher distillation (NEW)	2,635	Qwen/Qwen3.5-9B generations (Apache-2.0), verifier-kept 59.9% of 4,400: every number traced to its grounding pack, holdout/date/advice scans, claim cross-checks

Token lengths (full renders): mean 699, p95 1,353, max 30,168. Held-out enforcement: same rules as v2 (2025-01-01 cutoff + symbol holdout) re-verified for v3 — token-level holdout-symbol scan 0 hits, (symbol, date) overlap 0, post-cutoff dates 0 in new text (filing excerpts exempt by design, as in Runs 1–2 fundamentals SFT; still covered by the symbol/pair scans).

Finance Bench v2 (440 items — same frozen bench, BASE re-measured for Run 3)

Category	n	BASE (Qwen3.5-4B)	TUNED (Super-Fin Run 2)	TUNED (Super-Fin Run 3)
candle_reading	120	1.0000	1.0000	0.9833
indicator_interpretation	100	0.8500	0.9200	0.9800
return_volatility_calc	80	0.0250	0.0000 (see Run 2 note)	0.2250 (see note)
risk_identification	60	0.9667	1.0000	1.0000
tool_selection	80	0.1302	0.3979	0.8500
Overall	440	0.6259	0.6905	0.8227

Long-context bench (72 items, NEW in Run 3 — same scorer)

Category	n	BASE	TUNED (Run 3)
longctx_exact	20	0.4500	0.5500
longctx_filing	10	0.9000	0.8000
longctx_numeric	30	0.5667	0.7667
longctx_tool	12	0.1910	1.0000
Overall	72	0.5179	0.7500

Verdict (loop rule): KEEP — Run 3 is the new current best. Bench v2 overall 0.8227 beats Run 2's 0.6905 with no material category collapse, so the Run 3 adapter is promoted to HEAD.

Reading the result honestly:

• tool_selection 0.13 → 0.85 is the headline: the clean-stop fix packs worked. Zero of the 360 non-tool predictions contain a spurious tool call.
• return_volatility_calc official 0.225 still understates capability: the frozen harness extracts the first number in the output, and only 20/80 tuned predictions end with the trained Final answer: line. Re-scoring the identical predictions with last-number extraction gives 0.750 (Run 2's equivalent rescore: 0.588). The remaining gap is format discipline under worked-calculation style, not arithmetic.
• candle_reading 0.9833 is two items below perfect (Run 2: 1.0000) — a small regression against a large overall gain; noted for the next run's data mix.
• longctx_filing 0.90 → 0.80 (10 items; one additional miss vs base) while the other long-context categories improved sharply, tool use at long context most of all (0.19 → 1.00).
• Sample generations verified against source data: a long-context NVIDIA filing question returns the exact SEC figures ($215,938M FY2026 vs $130,497M FY2025, +65.5%); the calc sample ends Final answer: 28.28 exactly; the tool sample emits the expected portfolio_risk_check JSON byte-identically and stops.

Files

LoRA adapter — Run 3 (adapter_model.safetensors, ~520 MB) + adapter_config.json + tokenizer files for Qwen/Qwen3.5-4B. Per-chunk training checkpoints for Run 3 are preserved under runs/run3/chunks/. Load with PEFT: PeftModel.from_pretrained(base, "Itsmesujan/Super-Fin-4B").

Lineage

Base: Qwen/Qwen3.5-4B (Apache-2.0). Data: self-collected market data (Stooq/Yahoo, Binance public archives, DexScreener/GeckoTerminal snapshots), SEC EDGAR company facts, all stored in the private dataset repo Itsmesujan/finance-agent-data.