Privoid-ai/Privoid-Parse-FinTables

Privoid Parse FinTables

Reads images of financial tables into structured LaTeX, more accurately than the model it was built on.

Privoid Parse FinTables is a LoRA adapter for NVIDIA Nemotron 3.5 Parse, built by Privoid for private, locally deployable document AI. It keeps the base model's native output format, so existing Nemotron tools still work.

Cell F1	Number F1	Character errors	Word errors
0.701 to 0.762	0.782 to 0.819	0.147 to 0.108	0.363 to 0.205

Measured on 1,600 test images of 800 tables from 45 companies that were never used in training or model selection. The improvement in cell F1 is statistically clear: its 95% range is +0.053 to +0.070.

Experimental release. This adapter is more accurate on average, but it is slower and fails in a specific new way. See Known problems before using it.

Results

Measure	Unchanged	Privoid Parse	Change	95% range
Cell F1	0.701	0.762	+0.061	+0.053 to +0.070
Number F1	0.782	0.819	+0.037	+0.026 to +0.050
Exact row and column count	0.612	0.682	+0.071	+0.042 to +0.101
Valid table output	0.955	0.976	+0.021	+0.001 to +0.038
Character error rate (lower is better)	0.147	0.108	-0.040	-0.054 to -0.026
Word error rate (lower is better)	0.363	0.205	-0.157	-0.181 to -0.135
Hit the token limit (lower is better)	0.003	0.022	+0.019	+0.007 to +0.031

How the scores work. Cell F1 is this project's main score. Rows and columns are aligned, then a cell counts only if its text is exactly right. Number F1 checks that the right numbers appear. Cell F1 here is a custom measure, not TEDS or GriTS, so it can't be compared directly with published scores. The 95% ranges come from resampling whole companies, because tables from one company are related.

Where the gain comes from

Group	Images	Unchanged	Privoid Parse	Change	95% range
All test images	1,600	0.701	0.762	+0.061	+0.053 to +0.070
Clean images	800	0.770	0.827	+0.057	+0.047 to +0.068
Simulated scan damage	800	0.633	0.698	+0.066	+0.055 to +0.076
Small tables	1,036	0.682	0.756	+0.073	+0.063 to +0.084
Medium tables	518	0.736	0.781	+0.046	+0.033 to +0.060
Large tables	46	0.738	0.703	-0.035	-0.133 to +0.022
Tables with merged cells	758	0.736	0.775	+0.039	+0.024 to +0.053

The gain holds on clean images, damaged images, small and medium tables, and tables with merged cells. Large tables are the exception: on 46 images the change was -0.035, and its range of -0.133 to +0.022 includes both a loss and a gain, so this model is not shown to help them.

Speed and memory

	Unchanged	Privoid Parse
Median time per table, one at a time	2.8 s	6.1 s

Measured on a sample of 20 tables with an NVIDIA L4, through Transformers with the adapter applied. Privoid Parse was about twice as slow in this sample. Treat this as a rough figure. Peak allocated GPU memory was about 2 GB in both evaluations, while Colab's monitor showed up to 18.7 GB in use during batched evaluation, so plan for more than 2 GB.

Examples

These are real test images. The first two are among the largest wins and the third is among the largest losses, so they show the extremes, not typical results.

Example 1: a table of contents, clean image

Cell F1: unchanged 0.000, Privoid Parse 1.000. Source: TTWO.2006.page_74.pdf_140971.png, condition: clean.

Show both outputs

Unchanged model:

<x_0.0><y_0.0>Page
Reports of Independent Registered Public Accounting Firms 73
Consolidated Balance Sheets—At October 31, 2006 and 2005 (restated) 76
Consolidated Statements of Operations—For the years ended October 31, 2006, 2005 (restated) and 2004 (restated) 77
Consolidated Statements of Cash Flows—For the years ended October 31, 2006, 2005 (restated) and 2004 (restated) 78
Consolidated Statements of Stockholders' Equity—For the years ended October 31, 2004 (restated), 2005 (restated) and 2006 80
Notes to the Consolidated Financial Statements 81<x_0.9971><y_0.9938><class_Text></s>

Privoid Parse:

\begin{tabular}{cc}
\thead{\mbox{}} & \thead{Page}\\
Reports of Independent Registered Public Accounting Firms & 73\\
Consolidated Balance Sheets—At October 31, 2006 and 2005 (restated) & 76\\
Consolidated Statements of Operations—For the years ended October 31, 2006, 2005 (restated) and 2004 (restated) & 77\\
Consolidated Statements of Cash Flows—For the years ended October 31, 2006, 2005 (restated) and 2004 (restated) & 78\\
Consolidated Statements of Stockholders’ Equity—For the years ended October 31, 2004 (restated), 2005 (restated) and 2006 & 80\\
Notes to the Consolidated Financial Statements & 81\\
\end{tabular}

Example 2: a tax table with simulated scan damage

Cell F1: unchanged 0.000, Privoid Parse 0.971. Source: MMC.2013.page_79.pdf_86666.png, condition: scan damaged.

Show both outputs

Unchanged model:

<x_0.0><y_0.0>(a) Net of valuation allowances of $12 million in 2013 and $7 million in 2012.<x_0.999><y_0.9992><class_Text></s>

Privoid Parse:

\begin{tabular}{ccc}
\thead{Unrealized investment holding gains} & \thead{\$2} & \thead{\$2}\\
Differences related to non-U.S. operations & 112 & 107\\
Depreciation and amortization & 273 & 245\\
Accrued retirement \& postretirement benefits - non-U.S. operations & 89 & —\\
Other & 3 & 4\\
\mbox{} & \$479 & \$358\\
(a) Net of valuation allowances of \$12 million in 2013 and \$7 million in 2012. & \mbox{} & \mbox{}\\
\end{tabular}

Example 3: a failure

Cell F1: unchanged 0.772, Privoid Parse 0.000. Source: A.2010.page_24.pdf_134472.png, condition: scan damaged.

Show both outputs

Unchanged model:

\begin{tabular}{ccccc}
\multirow{2}{*}{\thead{\mbox{}}} & \multicolumn{4}{c}{\thead{October 31,}}\\
\mbox{} & \thead{2010} & \thead{2009} & \thead{2008} & \thead{2007} & \thead{2006}\\
\mbox{} & \mbox{} & \mbox{} & \mbox{} & \mbox{} & \mbox{}\\
Consolidated Balance Sheet Data: & \mbox{} & \mbox{} & \mbox{} & \mbox{} & (1)\\
Cash and cash equivalents and & \mbox{} & \mbox{} & \mbox{} & \mbox{} & \mbox{}\\
Non-term investments & \$2,649 & \$2,493 & \$1,429 & \$1,826 & \$2,392\\
Working capital & ... & 3,086 & \$2,838 & \$1,852 & \$2,008\\
Long-term restricted cash and cash equivalents & \$ 6 & \$1,566 & \$1,582 & \$1,615 & \$1,606\\
...

Privoid Parse:

\begin{tabular}{ccccc}
\thead{\mbox{}} & \multicolumn{2}{c}{\thead{October 31,}} & \multicolumn{2}{c}{\thead{\mbox{}}\\
\mbox{} & 2010 & 2009 & 2008 & 2007\\
\mbox{} & \multicolumn{4}{c}{(In millions)}\\
Consolidated Balance Sheet Data: & \mbox{} & \mbox{} & \mbox{} & (1)\\
Cash and cash equivalents and & \mbox{} & \mbox{} & \mbox{} & \mbox{}\\
Short-term investments & \$2,649 & \$2,493 & \$1,429 & \$1,826 & \$2,392\\
Working capital & \$3,086 & \$2,838 & \$1,852 & \$2,008 & \$2,420\\
Long-term restricted cash and cash equivalents & \$6 & \$1,566 & \$1,582 & \$1,615 & \$1,606\\
...

Known problems

• Repetition loops. 35 of 1,600 outputs (2.2%) hit the token limit, against 4 for the unchanged model. In the case inspected, the model kept repeating rows of empty \mbox{} cells. These outputs are slow and can be wrong. Add a guard for repeated rows if you use this in a pipeline.
• Slower generation. See Speed and memory.
• Large tables. There is no proven gain, and the sample is small.
• Long tables were excluded. Tables over 1,200 target tokens were left out, about 5.7% of test candidates. Results don't cover them.
• Accuracy is not perfect. A cell F1 of 0.762 means many tables still contain errors, and these are financial numbers. Have a person review anything important.

Limits of this evaluation

• Not real scans. The tables are digital renderings. Scan damage was simulated, so real scanned documents may behave differently.
• The base model may already know this data. NVIDIA does not name its Parse training datasets. If FinTabNet was included, the unchanged model's scores may be higher here than on new documents.
• Crops only. The model was trained and tested on images already cropped to one table. Whole-page parsing was not evaluated.
• Not tested with vLLM. Results apply to Transformers. Serving through vLLM, and its MTP speed-up, were not tested with this adapter.

How to use it

This is an adapter, so it needs the base model at the pinned revision. It is downloaded automatically on first use. After that, it can run without internet access. Linux and a GPU are required.

pip install "transformers==5.14.1" "timm==1.0.22" "einops==0.8.1" peft pillow
hf download Privoid-ai/Privoid-Parse-FinTables --local-dir privoid-parse
python privoid-parse/run_privoid_parse.py table.png

Use an image cropped to one table. The script upscales it 1x, as in the evaluation, and prints the model's raw output, for example:

<x_0.0><y_0.0>\begin{tabular}{ccc}
\thead{\mbox{}} & \thead{2018} & \thead{2017}\\
Net sales & \$ 1,098 & \$ 985\\
\end{tabular}<x_0.999><y_0.999><class_Table></s>

That example shows the format only. Use NVIDIA's postprocessing.py, included here, to convert it to HTML or Markdown. This release reads tables only. Pulling out fields such as a total is a separate step.

Training

Base model	nvidia/Nemotron-3.5-Parse, revision 4035ffb1d8f8
Method	LoRA, r=32, on all 10 decoder layers
Frozen	Vision encoder, adapter layers and MTP head
Trainable parameters	5.57M of 636.4M
Data	docling-project/FinTabNet_OTSL, split by company
Tables	Train 24,000 (338 companies), validation 96 (35), test 800 (45)
Augmentation	Simulated scan damage on half of training images
Image scale	1x upscaling, chosen on validation data only
Learning rate	0.0001 with warm-up and cosine decay
Batch	2 x 4 accumulation = 8 tables per step
Stopped	Early stopping at step 2,100. Selected step 1,650.
Hardware	One NVIDIA L4 on Google Colab, BF16
Run time	70 minutes to the end of training, including setup and baseline

The test set was not used for any decision. Both models used identical images, prompt, greedy decoding with no repetition penalty, and scoring.

Privacy

Once deployed on your own hardware, your documents don't need to leave it. That describes how the model can be deployed, not how it was made: training ran on Google Colab, and files were stored on Google Drive. The training data is public, and no Privoid customer data was used.

Licenses

• Base model: OpenMDW-1.1. The tokenizer is MIT with attribution. See the included LICENSE, THIRD_PARTY_NOTICES.md and TOKENIZER_ATTRIBUTION.md.
• Training data: docling-project/FinTabNet_OTSL is tagged license: other. A secondary source describes the original FinTabNet as CDLA-Permissive, but that could not be confirmed from IBM's retired page. The source pages are company reports. Check that this suits your use.