FiledFact-Qwen3-8B-LoRA

A research-preview specialist model for SEC/XBRL financial fact extraction. FiledFact-Qwen3-8B-LoRA is a LoRA adapter for Qwen/Qwen3-8B that reads a passage of SEC filing text and emits the XBRL-aligned numeric facts it contains as structured JSON — concept, full-unit value, unit, period, and segment dimensions. It was fine-tuned on the FiledFact corpus, the parent of the StockAlloy/filedfact-100k dataset.

It is a specialist, not a general model: it is not a question-answering system, not an advisor, and it is not production-ready. Its outputs require human review before use.

On a held-out-by-company benchmark of 1,045 SEC filing passages and 10,200 gold facts, fine-tuning improved Fact-F1 from 0.295 to 0.787 and paired concept accuracy from 0.048 to 0.514.

🚀 Try it live: FiledFact demo Space — paste a filing passage, get the facts back as JSON. No setup.

Intended use

• Research on XBRL grounding, financial information extraction, and structured generation.
• A fact-layer generator inside a pipeline: extractor (this model) → reasoner (a general model) → verifier (checks values against the source document).
• Error analysis and benchmarking alongside the FiledFact-100K dataset.

Not intended for:

• Investment advice or investment decisions of any kind.
• Chat or general question answering.
• Arithmetic or multi-hop reasoning over filings.
• Fully autonomous filing-processing systems.

Quick start

Transformers + PEFT

The adapter's config records its training base as unsloth/qwen3-8b-unsloth-bnb-4bit (a 4-bit mirror of Qwen/Qwen3-8B); it loads fine onto Qwen/Qwen3-8B.

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "Qwen/Qwen3-8B"
tokenizer = AutoTokenizer.from_pretrained(base_id)
model = AutoModelForCausalLM.from_pretrained(base_id, torch_dtype="auto", device_map="auto")
model = PeftModel.from_pretrained(model, "StockAlloy/filedfact-qwen3-8b-lora")

# PROMPT_TEMPLATE is the training prompt from the "Prompt format" section below
prompt = PROMPT_TEMPLATE.format(form_type="10-K", filed_at="2024-11-01",
                                pre_context="(none)", heading="Results of Operations",
                                text=passage)
text = tokenizer.apply_chat_template(
    [{"role": "user", "content": prompt}],
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,  # REQUIRED — without this, Qwen3 thinking blocks pollute the JSON
)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=4096, do_sample=False)  # greedy; 4096 avoids truncation on fact-dense passages
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Ollama (local)

Convert the adapter with llama.cpp and layer it on the stock qwen3:8b:

python llama.cpp/convert_lora_to_gguf.py <adapter-dir> --base-model-id Qwen/Qwen3-8B --outtype f16

# Modelfile
FROM qwen3:8b
ADAPTER ./filedfact-qwen3-8b-adapter.gguf
PARAMETER temperature 0
PARAMETER num_ctx 8192

ollama create filedfact-qwen3:8b -f Modelfile

Append /no_think to the end of the prompt when running through Ollama (this disables Qwen3 thinking, matching how the model was trained and evaluated). The Ollama build applies the adapter over a Q4_K_M-quantized base; expect somewhat lower accuracy than the benchmarked configuration (see Evaluation setup).

Prompt format

The model was trained and evaluated with exactly this prompt. Use it verbatim; other phrasings are out of distribution:

You are a financial data extraction system. Below is a passage from an SEC filing ({form_type}, filed {filed_at}). Extract EVERY numeric financial fact that appears in the passage.

Return ONLY valid JSON in this exact shape:
{"facts": [{"concept": "...", "value": 0, "unit": "...", "period": "...", "dimensions": [{"axis": "...", "member": "..."}]}]}

Rules:
- concept: the US-GAAP/IFRS XBRL concept name if you know it (e.g. "RevenueFromContractWithCustomerExcludingAssessedTax"); otherwise a precise CamelCase label.
- value: the numeric value in FULL units. Expand the table scale: if the table is "in thousands" and a cell shows 1,634 the value is 1634000. Use negative numbers for values shown in parentheses.
- unit: ISO currency code (USD, EUR, ...) for monetary values; "shares" for share counts; "pure" for ratios.
- period: "instant:YYYY-MM-DD" for point-in-time balances, or "duration:YYYY-MM-DD..YYYY-MM-DD" for flows over a period.
- dimensions: the slice the value belongs to (business segment, geography, product, plan type, share class, ...) as axis/member pairs. Use [] for company-wide totals.

CONTEXT IMMEDIATELY BEFORE THE PASSAGE (may contain the table caption and scale, e.g. "in thousands"):
{pre_context}

SECTION HEADING: {heading}

PASSAGE:
{text}

Fill {form_type}, {filed_at}, {pre_context} (use (none) if unavailable), {heading}, and {text}.

Note on the prompt's wording: although the prompt says "EVERY numeric financial fact," the model is trained to emit XBRL-aligned facts — in practice it ignores dates, page numbers, and narrative figures that were not tagged in the filing's iXBRL. The parentheses rule reflects display convention; for some concepts, XBRL sign conventions mean the reported value's polarity differs from the displayed sign (see Limitations). The model also sometimes emits an extra member_label field inside dimension objects (a human-readable member name) beyond the schema in the prompt — parsers should tolerate it.

Evaluation

Held-out-by-company benchmark (eval-v3, 2026-07): 1,045 passages / 141 companies / 10,200 scored gold facts, none of the companies seen in training. Scoring is value-anchored fact pairing with per-field accuracy; "full-fact" means concept, value, unit, period, and dimensions all correct at once, with no partial credit.

Evaluation setup

Model	Runtime / access	Configuration	n
Qwen3-8B base and + FiledFact (this adapter)	unsloth, 4-bit (bnb) base, 1× H100	greedy (do_sample=False), max_new_tokens=4096, thinking disabled	200 and 1,045
Gemma-2-9B base and + FiledFact	transformers + PEFT, bf16 base, attn_implementation="eager"	greedy	200
gpt-5.5	OpenAI Batch API	reasoning_effort="medium", response_format={"type": "json_object"}	200
gpt-4o-mini	OpenAI API	temperature 0	200

All models were run with the same prompt and scored with the same scorer. The model comparison uses one fixed 200-passage subset of eval-v3; the API baselines were cost-bounded to those 200 passages, and all confidence intervals are computed on this subset via passage-block bootstrap. The base-vs-adapter comparison is additionally reported on the full 1,045 passages. No number in any table compares models evaluated on different passage sets. Baseline configurations are reasonable defaults rather than per-model optimizations; a higher reasoning-effort setting or model-specific prompt tuning could move the baseline numbers.

Base model vs. adapter (full benchmark, 1,045 passages)

Metric	Base Qwen3-8B	+ FiledFact (this adapter)	Lift
Valid-JSON rate	0.901	0.966*	1.07×
Fact-F1	0.295	0.787	2.7×
Precision	0.332	0.789	2.4×
Recall	0.266	0.784	2.9×
Concept accuracy (paired)	0.048	0.514	10.7×
Unit accuracy (paired)	0.882	0.973	1.1×
Period accuracy (paired)	0.449	0.855	1.9×
Dimensions-F1	0.121	0.467	3.9×
Full-fact exact	0.003	0.263	94×

* At max_new_tokens=4096; 36 of 1,045 passages contain more facts than even 4096 tokens hold — see Limitations.

Model comparison (200-passage subset; best value per row in bold)

Metric	Qwen3-8B base	Qwen3-8B + FiledFact	Gemma-2-9B + FiledFact	gpt-4o-mini	gpt-5.5
Fact-F1	0.302	0.781	0.634	0.425	0.712
Recall	0.267	0.756	0.558	0.484	0.892
Precision	0.347	0.808	0.733	0.379	0.592
Concept accuracy	0.040	0.427	0.334	0.058	0.196
Full-fact	0.002	0.185	0.110	0.004	0.065

The tuned model's deltas versus gpt-5.5 on Fact-F1, concept accuracy, precision, dimensions-F1, and full-fact are statistically significant (passage-block bootstrap; the F1 margin is narrow, 95% CI [+0.01, +0.13] — full intervals in Additional diagnostics), while gpt-5.5 retained higher recall (0.892 vs 0.756).

Cross-architecture check (Gemma-2-9B, same 200 passages)

The identical recipe applied to a second 9B architecture produces the same pattern, indicating the improvement is data-driven rather than architecture-specific:

Metric	Gemma-2-9B base	+ FiledFact
Fact-F1	0.438	0.634
Recall	0.438	0.558
Precision	0.438	0.733
Concept accuracy	0.061	0.334
Full-fact	0.003	0.110

The Gemma adapter weights are not released because Gemma derivatives carry the Gemma Terms of Use.

Example

Input passage (10-K, filed 2024-11-01, heading "Results of Operations", no pre-context):

Total net sales for fiscal 2024 were $391,035 million, an increase of 2% compared to $383,285 million in fiscal 2023. Products revenue was $294,866 million and Services revenue reached a record $96,169 million for the year ended September 28, 2024.

Verbatim output from the Ollama build (temperature 0):

{"facts": [
  {"concept": "RevenueFromContractWithCustomerExcludingAssessedTax", "value": 391035000000, "unit": "USD", "period": "duration:2023-09-30..2024-09-28", "dimensions": []},
  {"concept": "RevenueFromContractWithCustomerExcludingAssessedTax", "value": 383285000000, "unit": "USD", "period": "duration:2022-10-01..2023-09-29", "dimensions": []},
  {"concept": "RevenueFromContractWithCustomerExcludingAssessedTax", "value": 294866000000, "unit": "USD", "period": "duration:2023-09-30..2024-09-28", "dimensions": [{"axis": "ProductOrServiceAxis", "member": "ProductMember", "member_label": "Product [Member]"}]},
  {"concept": "RevenueFromContractWithCustomerExcludingAssessedTax", "value": 96169000000, "unit": "USD", "period": "duration:2023-09-30..2024-09-28", "dimensions": [{"axis": "ProductOrServiceAxis", "member": "ServiceMember", "member_label": "Service [Member]"}]}
]}

All four values are expanded to full units, the product/service split carries the correct dimension, and the prior-year comparative receives its own period. One limitation is visible in this output: the passage states only "September 28, 2024," so the fiscal-year start dates and the FY2023 boundaries were inferred by the model and are approximate. Note also the extra member_label field.

Limitations

• Extracts XBRL-aligned facts only. It does not return every number visible in a passage; dates, page numbers, and untagged narrative figures are excluded by design.
• Concept errors remain, especially on long-tail and company-specific concepts (paired concept accuracy is 0.514).
• Dimension attribution is not production-grade (dimensions-F1 0.467); verify segment/axis assignments before use.
• Period boundaries may be inferred when the passage does not state them explicitly, as in the example above.
• Very fact-dense passages need a higher max_new_tokens or passage splitting — 36 of the 1,045 evaluation passages exceeded even a 4096-token output budget.
• Not designed for arithmetic, multi-hop reasoning, or question answering (see Additional diagnostics).
• Outputs require human review.

Training data

The adapter was trained with LoRA SFT on ~82K private passage-complete records from the parent FiledFact corpus (71.7M span-grounded, value-verified XBRL fact/text alignments across 6,100+ SEC filers) — each training example is one filing passage paired with all of its XBRL facts. Prompt tokens were masked from the loss; max sequence length 4096. The 150 evaluation companies were excluded from training.

The model was not trained on the public FiledFact-100K rows. That sample is fact-sampled (one fact per row) and is not passage-complete. Do not group its rows by chunk_id and treat the result as passage-complete extraction labels; a passage-complete public sample is planned.

Additional diagnostics

A TAT-QA transfer probe (n=200, simplified EM scoring) confirms the adapter should not be used for general QA: overall exact-match fell from 0.390 (base) to 0.265 (tuned), with arithmetic and multi-span answers degrading most while single-span extraction improved slightly. This is reported to clarify scope; it is not part of the headline benchmark. Full bootstrap intervals for the 200-passage comparison versus gpt-5.5 (20-block passage bootstrap, 4,000 resamples): Fact-F1 [+0.01, +0.13], concept accuracy [+0.19, +0.30], precision [+0.16, +0.27], dimensions-F1 [+0.09, +0.26], full-fact [+0.09, +0.16].

License

• These adapter weights: Apache-2.0.
• Base model Qwen/Qwen3-8B: Apache-2.0.
• The FiledFact-100K public sample: CC BY-NC 4.0 (research use).
• The full FiledFact corpus (71.7M pairs), quarterly updates, custom slices, and the point-in-time restatement history are available under commercial license: data@stockalloy.com · stockalloy.com/datasets.

Citation

@misc{stockalloy_filedfact_qwen3_2026,
  title  = {FiledFact-Qwen3-8B-LoRA: A Specialist Model for SEC/XBRL Financial Fact Extraction},
  author = {StockAlloy Research},
  year   = {2026},
  url    = {https://huggingface.co/StockAlloy/filedfact-qwen3-8b-lora}
}

@dataset{stockalloy_filedfact_2026,
  title   = {FiledFact-100K: Span-Grounded, Value-Verified SEC Financial Facts},
  author  = {StockAlloy Research},
  year    = {2026},
  url     = {https://huggingface.co/datasets/StockAlloy/filedfact-100k}
}