FinChart-VL-4B (FinChart-OCR-Qwen3VL-4B, LoRA)

A LoRA adapter for Qwen3-VL-4B-Instruct specialised for financial chart understanding: question answering over charts, and chart to markdown-table extraction. Called FinChart-VL-4B in the ICAIF '26 paper From Curriculum to Rehearsal Learning: Towards Small Financial Chart VLMs.

At 4B parameters it outperforms every open-source model reported in the FinChart-Bench paper's Table 2, including models 6x its size.

Results

FinChart-Bench (7,019 questions, official protocol)

Scored with the benchmark's own protocol: Result = [[ label ]] bracket format with strict equality.

model	params	TF	MC	QA	AVG
Qwen3-VL-4B (base)	4B	.931	.747	.314	.664
this model	4B	.954	.894	.548	.798
(same recipe, mean of 3 seeds)	4B	.950	.894	.535	.793 +/- .005
Mistral-3.1-24B *	24B	.930	.842	.449	.740
Qwen2.5-VL-7B *	7B	.951	.828	.373	.717
Claude Sonnet 4 *	-	.969	.917	.636	.841

AVG is the unweighted mean of TF, MC and QA in every row (the FinChart-Bench paper prints question-weighted averages, which differ by up to .005).

* published numbers from the FinChart-Bench paper, not re-run under this harness -- prompts and image budgets may differ. The rows for this model and its base were run under an identical harness.

The released checkpoint is the best of three training seeds (.798). The honest summary statistic for the recipe is the mean, .793 +/- .005; individual seeds scored .789 / .798 / .791.

Chart to markdown extraction (50 held-out real financial charts)

model	value F1	cell F1	header F1
Qwen3-VL-4B (base)	.745	.401	.671
this model	.785	.483	.683

Intended use

• Question answering over financial charts (values, comparisons, trends, simple arithmetic over chart data)
• Transcribing financial charts into markdown tables
• A drafting aid for chart digitisation with a human in the loop

Out of scope / limitations

Read this before deploying.

• Not an unattended table extractor. Cell-level F1 is about .48, i.e. roughly half of cells land wrong or under a mismatched header. Suitable for drafting, not for autonomous ingestion.

• Degraded on general financial document OCR. Chart specialisation costs general document fidelity. On FinCriticalED (848 pages, markdown-OCR prompt), measured on the seed-4242 sibling checkpoint:

  model	fact recall	numeric precision
  Qwen3-VL-4B (base)	.966	.949
  this recipe	.916	.720
  Qwen3-VL-8B	.953	.992

  Numeric precision of .720 means roughly 28 percent of the numbers it emits on dense document pages do not appear on the page. For document (non-chart) OCR, use the base model or a larger one -- Qwen3-VL-8B is strictly better at that task.

• Trained on English-language charts.

• No safety tuning beyond the base model's.

Training

base	Qwen3-VL-4B-Instruct
method	LoRA r=32, alpha=64, dropout .05, on q/k/v/o/gate/up/down_proj
data	19,039 samples: 5,359 TF / 2,212 MC / 8,967 QA / 2,501 chart-to-table (the question generator is seeded, so counts vary slightly by seed; the paper reports the seed-4242 mixture, 19,024)
schedule	1 epoch (1,190 steps), bf16, lr 1e-4 cosine, warmup 15
batch	4 x 4 grad accum, gradient checkpointing
seed	1337
hardware	A100-80GB, about 139 min
versions	transformers 5.14.1, peft 0.20.0, torch 2.11.0+cu128

Training data derives from varadsrivastava/financial-chart-ocr: real financial charts whose markdown tables were drafted by Claude Opus 4.6 and verified against the image by human annotators, plus Matplotlib-rendered synthetic charts whose tables are the generating data. QA/TF/MC supervision is generated programmatically from the gold tables, so every derived answer is exact with respect to its table.

Contamination control. The corpus was screened against FinChart-Bench by document-and-crop identifier before training, and the 50-chart held-out extraction benchmark (the benchmark split of the dataset) was excluded from training, enforced by an assertion inside the training loop.

What the extraction data actually does

An ablation removing only the 2,501 chart-to-table samples, holding QA/TF/MC identical, over 3 seeds:

• chart reasoning is unaffected: +.003 +/- .004 on FinChart-Bench (n.s.)
• chart extraction collapses without it: -.182 +/- .056 value F1 (p about .03), falling below the untrained base model

So the extraction objective acts as rehearsal against catastrophic forgetting rather than as a source of new capability. The reasoning gains come from the derived computational supervision, not from extraction training.

Usage

from transformers import AutoModelForImageTextToText, AutoProcessor
from peft import PeftModel

BASE = "Qwen/Qwen3-VL-4B-Instruct"
model = AutoModelForImageTextToText.from_pretrained(
    BASE, dtype="bfloat16", device_map="cuda")
model = PeftModel.from_pretrained(model, "varadsrivastava/FinChart-VL-4B")
processor = AutoProcessor.from_pretrained("varadsrivastava/FinChart-VL-4B")

messages = [{"role": "user", "content": [
    {"type": "image", "image": "chart.png"},
    {"type": "text", "text": "Convert this financial chart into a markdown "
                             "table. Include the chart title and units."}]}]
text = processor.apply_chat_template(messages, tokenize=False,
                                     add_generation_prompt=True)

For benchmark-style QA, append the official instruction so the answer is parseable: Output correct answer only in this format: 'Result = [[ answer ]]'

Reproducibility note

max_pixels passed to AutoProcessor has no effect when images are prepared via qwen_vl_utils.process_vision_info, which resizes before the processor sees them. To control the vision-token budget, pass max_pixels inside the message image dict, or pre-resize the image.

Citation

@inproceedings{srivastava2026finchartvl,
  title     = {From Curriculum to Rehearsal Learning: Towards Small Financial Chart {VLMs}},
  author    = {Srivastava, Varad and Shukla, Abhay and Casas Hernandez, Victor},
  booktitle = {Proceedings of the ACM International Conference on AI in Finance (ICAIF '26)},
  year      = {2026}
}

Disclaimer

This work was carried out at AI Lab, Barclays. The views expressed here are those of the authors and do not represent the views of Barclays.