LightPFN

LightPFN is a small tabular foundation model for classification: a 4,603,088-parameter in-context learner pretrained only on synthetic data. fit stores the training set as context and predict_proba answers in one forward pass, with no training on your data and no hyperparameters to tune. It runs on CPU, CUDA, ROCm and, through its own Vulkan kernels, on AMD, Intel and NVIDIA GPUs.

• Code, documentation and training pipeline: github.com/GioOtto/LightPFN
• Package: pypi.org/project/LightPFN
• Technical report: LightPFN_report.pdf
• License: Apache 2.0, code and weights

Usage

pip install LightPFN

from lightpfn import LightPFNClassifier

clf = LightPFNClassifier(n_estimators=4, random_state=0)
clf.fit(X_train, y_train)          # downloads these weights at a pinned commit on the first fit
proba = clf.predict_proba(X_test)

X can be a NumPy array or a pandas DataFrame with numeric, categorical, string and boolean columns and missing values. device="auto" uses a CUDA or ROCm GPU, else a Vulkan GPU (pip install "LightPFN[vulkan]"), else the CPU. The user guide covers every option.

Model

Task	classification, 2 to 10 classes
Parameters	4,603,088 (18 MB, float32)
Architecture	cell embedding (value, rank, missing flag), two induced column stages, row refinement with four summary tokens, row compression, seven in-context blocks, retrieval decoder
Pretraining data	synthetic only: 90% structural causal graph prior, 10% rule prior (XOR, parity, lookup, trees); 7.68 million task draws from 4.03 million distinct tasks
Training	120,000 steps of 64 tasks on two RTX 5090 (tables up to 2,048 rows), then 39,250 steps on tables up to 60,000 rows
Not used in training	real datasets, distillation, weights or outputs of other tabular foundation models

Evaluation

The official TabArena leaderboard includes default, tuned and ensembled baselines. Other evaluations use default baselines; AUC differences have 95% paired bootstrap intervals.

Benchmark	Result
Official TabArena leaderboard, 38 classification datasets, default configuration (four estimators), run and verified by the TabArena maintainers, 11 October 2026	all splits: 21st of 92, Elo 1441 (+69 / -79); Lite: 20th of 92, Elo 1426 (+71 / -70); none imputed. Above every tree-based method, tuned and ensembled ones included (intervals overlap default CatBoost and the tuned entries); behind TabICLv2 (1575) and larger foundation models
55 OpenML-CC18 datasets outside TabArena (at most 1,000 rows, one estimator)	mean AUC 0.911, +0.86 [0.42, 1.40] over CatBoost, +1.8 to +2.1 over LightGBM, XGBoost and random forest
TabArena, 38 classification tasks, official splits, first repeat (our harness), four estimators	mean AUC 0.858, rank 2.50 of 7, behind TabICLv2 (0.864), lower error than CatBoost on 76% of tasks
13 OpenML datasets of 50,000 to 2.2M rows, 10,000 to 100,000 training rows	mean AUC difference from CatBoost between -0.34 and +0.28 points, intervals include zero

The leaderboard view is Classification with imputed models excluded: the default view also covers the 13 regression datasets, which LightPFN does not support. Full tables, timings and the official TabArena results: docs/en/RESULTS.md.

Limitations

• Classification only (2 to 10 classes); regression is planned for version 2.
• Categorical columns are read as ordinal codes. On tables dominated by high-cardinality categorical columns CatBoost is ahead; native categorical handling is planned for version 2.
• Above 20,000 training rows each estimator reads a stratified subsample (max_context).
• CPU time grows with the context length; on large tables a GPU is much faster.

Files

File	Content
model.safetensors, config.json	weights (float32) and architecture of the released model
provenance.json	hashes of the weights and of the source checkpoint
LightPFN_report.pdf	technical report
LICENSE, NOTICE	Apache License 2.0 and attribution notice

Citation

@techreport{ottoboni2026lightpfn,
  title  = {A Sling Against Giants: {LightPFN}, a 4.6M-parameter tabular in-context classifier designed to stay small},
  author = {Ottoboni, Giorgio},
  year   = {2026},
  url    = {https://github.com/GioOtto/LightPFN}
}