CardoAIDevs/Nori-Classifier-30M

Nori Classifier 30M (~29M) (in-context tabular classification head)

A thin classification head for Synthefy's Nori tabular foundation model. Nori is a frozen tabular regression model that predicts in-context: you hand it labelled rows, it scores new rows in a single forward pass, with no per-dataset training. This repo adds a small head so the same frozen model does binary and small-multiclass classification.

This repo ships only the head we trained (6 tensors). The Nori base is not included here: it is downloaded from Synthefy's official repo Synthefy/Nori-30M at load time and stays under its own license.

Use it

pip install "synthefy-nori" huggingface_hub safetensors numpy torch
# grab the loader from this repo:
huggingface-cli download CardoAIDevs/Nori-Classifier-30M nori_classifier.py --local-dir .

from nori_classifier import NoriClassifier

clf = NoriClassifier("CardoAIDevs/Nori-Classifier-30M").fit(X_train, y_train)   # stores context, no training
proba = clf.predict_proba(X_test)                      # (n_test, n_classes)
pred  = clf.predict(X_test)

X is a 2-D float array (rows x features), y is a 1-D label array. On first run it downloads the frozen Nori base from Synthefy.

Use it as a feature extractor

Beyond predicting, this model gives you a classification-tuned 128-d embedding per row (the target-token representation the head reads), with the same fit-then-extract ergonomics as Synthefy's embeddings (blog):

emb = NoriClassifier("CardoAIDevs/Nori-Classifier-30M").fit(X_context, y_context)
Z = emb.get_embeddings(X_query)      # (n_query, 128), classification path

For general-purpose (regression-path) embeddings, use the base Nori model directly, as in the blog:

from synthefy_nori.embedding import NoriEmbedding
Z = NoriEmbedding(n_fold=5).fit_transform(X, y)   # leak-free out-of-fold, 128-d

Where it is good, and where it is not

Best on small, IID tabular problems, roughly a few hundred to ~2000 labelled rows. On the BeyondArena non-IID benchmark it is genuinely competitive on the tiny-IID slice (rank 4 of 29, ROC AUC 0.857), behind only the TabPFN / TabICL foundation models and ahead of tuned trees and MLPs.

Honest limits, so you use it where it wins:

• The edge narrows as the dataset grows. The in-context signal saturates at about 2000 rows, so on larger tables gradient-boosted trees and larger in-context models pull ahead.
• Weak under distribution shift (train-past / test-future, or train/test on different groups). Use it for in-distribution problems.
• Multiclass probabilities are not calibrated. Good for ranking, not as calibrated confidences.

Benchmarks

Numbers below are the 6M as an in-context binary classifier (direct-E), ROC AUC:

benchmark	slice	AUC	standing
BeyondArena	iid-tiny (16 sets)	0.857	4 / 29, behind only TabPFN-3 / TabICLv2 / TabPFN-2.6
TabArena binary	tiny (≤ 2k rows)	0.832	34 / 79
TabArena binary	all (30 sets)	0.820	~64 / 79
TabArena-29	held-out, leak-safe	0.810	recipe selected off-benchmark

Strongest on small, IID data; the edge narrows as datasets grow and drops under distribution shift (see above). The 30M matches on held-out TabArena-29 (0.810), so the classification ceiling is a recipe limit, not capacity.

How it was trained

• Base: frozen Nori 30M (~29M) (Synthefy/Nori-30M), never modified.
• What we trained: only the classification interface, the label encoder (cls_y_encoder) and the two-layer read-out head (cls_y_decoder), 6 tensors in total. The entire transformer body stays frozen, so the base model's regression behaviour is unchanged.
• Data: Synthefy's own synthetic prior (their in-package data generator), never real data. Cross-entropy on the query rows, Adam, about 4000 prior episodes.
• Context cap: trained and evaluated with the context capped at ~2000 rows (stratified). The model plateaus there and attention is O(N^2), so the loader applies the same cap at inference.

Authors

Genc Geci (CardoAI) led the model-development work behind this checkpoint, establishing that Nori can be used directly as an in-context classifier. Located the dormant classification pathway in the released weights, reconstructed the model's synthetic prior to generate training episodes, and built the adaptation that activates it: a retrained head and label encoder (as well as LoRA-based extension).

Tommaso Guerrini (CardoAI) led the representation and benchmarking work: he analysed Nori's internal embeddings, established that the frozen backbone already carries strong classification signal, ran and interpreted the downstream evaluations, and led the packaging, Hugging Face integration, and release.

Credit and license

Built on Nori by Synthefy. All credit for the base model is theirs. This head and loader are released by CardoAI under Apache-2.0, the same license as the upstream model. We do not redistribute any Synthefy weights.

Citation

@misc{cardoai_nori_classifier,
  title     = {Nori Classifier 30M (~29M): an in-context tabular classification head for Synthefy Nori},
  author    = {Geci, Genc and Guerrini, Tommaso},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/CardoAIDevs/Nori-Classifier-30M}
}