You need to agree to share your Hugging Face information to access this model
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
You agree to the terms of the Neuralk Seldon License.
Log in or Sign Up to review the conditions and access this model content.
Seldon TFM Family
Seldon is a tabular foundation model by Neuralk-AI. It predicts on new rows through in-context learning: you give it labeled example rows as context, and it predicts the target for new rows in a single forward pass, with no training or hyperparameter tuning on your data.
This repo contains seldon (213.5M parameters). For now, only the classification checkpoint is released.
Model family
| Variant | Parameters | Availability |
|---|---|---|
| seldon-mini | 27.4M | Open weights |
| seldon | 213.5M | Open Weights |
| seldon API | – | Neuralk API |
Quickstart
pip install neuralk-oss
from neuralk_oss import SeldonClassifier
clf = SeldonClassifier.from_pretrained("seldon", device="cuda")
clf.fit(X_train, y_train) # stores the context; no gradient updates
preds = clf.predict(X_test)
proba = clf.predict_proba(X_test)
Benchmark
Head-to-head win rate on TabArena classification (38 datasets, 594 folds), each model with its default settings.
| Baseline (params) | Seldon (213.5M) win rate |
|---|---|
| Kumo-Tabular Small (27.5M) | 75.8% |
| TabICLv2 (27.6M) | 82.3% |
| Kumo-Tabular Medium (61.5M) | 56.6% |
| TabPFN-3.5-Fast (83.5M) | 73.7% |
| Kumo-Tabular Large (213.7M) | 38.2% |
| TabPFN-3.5 (219.0M) | 63.5% |
| LimiX-2 (406.2M) | 40.6% |
| TabFM (1.64B) | 52.7% |
Architecture
Seldon follows the same design as Kumo Tabular and TabFM: the table is first compressed into one representation per row, and in-context learning then runs on these row representations.
- Tokenizer: each cell becomes a token through a Fourier feature projection of numerical values, a categorical mask, and a missing-value indicator. Features are grouped before projection.
- Cell-to-row embedding: interleaved column and row transformers (column attention across rows uses inducing points). The result is compressed into CLS readout tokens, giving one dense vector per row.
- ICL layers: a transformer stack over the compressed row representations. Labeled context rows serve as the training set, and the model predicts the query rows.
- Decoder: an MLP head maps each query row's representation to class probabilities.
Part Parameters Shape Cell/row embedding 9.93M 256 channels, 6 layer pairs, 256 inducing points, 4 readout tokens ICL stack 203.53M 24 layers, 1024 channels, 16 heads Total 213.46M
Pretraining: a synthetic prior
Seldon is pretrained only on synthetically generated datasets drawn from a prior developed by Neuralk. No real-world data was used in pretraining. The prior is built on structural causal models (SCMs): each synthetic task comes from a randomly sampled causal graph whose nodes are linked by randomly drawn functional relationships and noise, and the features and target are read off this graph. The main goal of the prior is diversity. Its new, richer generative mechanisms go beyond classical SCM priors and vary:
- the graph structure
- the kinds of functional dependencies
- feature types and distributions,
- noise, missingness, class balance and dataset shape...
As a result, the model sees a much wider range of tabular problems during pretraining than any collection of real datasets would provide, and learns a general way of doing in-context inference that carries over to real-world tables.
- Downloads last month
- 15