chetanxpatil/collapse-nli

CollapseNLI (+ alignment)

An attention-free Natural Language Inference classifier, trained from scratch on SNLI with no pretrained embeddings and no transformer. ~5.52M parameters. Given a premise and a hypothesis it predicts entailment / neutral / contradiction.

Instead of attention, sentences are pooled into a single difference vector u − v that is warped through a 4-layer collapse engine toward three learned point-attractors (E / N / C), with a lightweight Decomposable-Attention-style cross-sentence alignment step in front. The collapse dynamics are deliberately analyzable — see Interpretability below.

Honest by design. Numbers below are reported against the right baselines (the hypothesis-only artifact floor, not the 33% chance floor). Read the limitations before citing.

Results (SNLI, no pretrained embeddings)

Variant	SNLI dev	SNLI test	Params
CollapseNLI (mean-pool)	66.1%	—	5.25M
CollapseNLI + alignment (this model)	74.7%	74.4%	5.52M

Measured with the bundled eval_snli.py on the official, leak-free SNLI splits. Dev and test agree to within 0.2 points — no dev overfitting. Per-class (test): entailment 73.6%, neutral 69.1%, contradiction 80.7% (neutral lowest, as expected — it is the narrow boundary class).

Reference points on the same footing: chance ≈ 33%; unlexicalized-feature floor ≈ 50% (Bowman et al. 2015); hypothesis-only artifact floor ≈ 67%; the simplest SNLI-only no-embeddings baseline (a lexical-feature classifier) ≈ 78.2%; GloVe encoders 77–88%; BERT/RoBERTa 90–93% (with massive pretraining).

So this sits well above the artifact floor and ~3.5 points below the simplest strong baseline — and the gain over mean-pool came from a mechanism (cross-sentence alignment), not more data.

Note. An earlier write-up reported 72.7% (an under-estimate), and the raw checkpoint records an internal best of ~78.1% — but that was on a leaky train-carved slice (inflated, as the training code warns). The numbers above (74.4% test / 74.7% dev) are the reproducible, leak-free figures; rerun eval_snli.py to verify.

Usage

from modeling_collapsenli import load_model, predict

model, meta = load_model("nli_collapse_best.pt")        # CPU by default
label, scores = predict(
    model, meta,
    premise="A man is playing a guitar on stage.",
    hypothesis="A person is performing music.",
)
print(label, scores)
# entailment {'entailment': 0.82, 'neutral': 0.15, 'contradiction': 0.03}

Only torch is required. The checkpoint carries its own vocabulary (stoi), embeddings, engine anchors and alignment module — nothing else to download.

Architecture

• Embedding: 20,003-word vocab × 256-d, trained from scratch (no GloVe/word2vec).
• Alignment (AlignModule): premise↔hypothesis soft alignment + compare, then pool.
• Collapse engine (VectorCollapseEngine): 4 residual steps warping u − v toward unit-normalized anchors E/N/C; norm-clamped to ‖h‖ ≤ 10.
• Classifier head: none — logits are the cosine similarity of the collapsed vector to the three anchors, divided by a learned temperature.

Config: dim=256, layers=4, n_words=20003, pool=mean, align=True.

Interpretability ("not a black box")

The collapse mechanism is a glass box, even though the learned embeddings it operates on are not. You can read off the three named attractors' geometry (E and C are the polar opposites; N sits orthogonal as the boundary), track how the decision crystallizes layer by layer, measure the basin sizes, and prove stability: across sampled points >92% of the 256 dimensions are strictly contracting (singular values < 1), so the system is a stable contraction toward its attractors. Full analysis: docs/COLLAPSE_STRUCTURE_REPORT.md in the source repo.

Limitations

• Not a general model. Trained only on SNLI (~570k image-caption-style sentences); it understands only that narrow distribution.
• SNLI artifacts. A large share of any SNLI score comes from hypothesis-only annotation artifacts, not reasoning — which is why the right baseline is the ~67% artifact floor, and why this and all word-level methods fall to ~chance on adversarial benchmarks like ANLI.
• Mechanism ceiling. Accuracy plateaus near 72–73% on dev; closing the last ~5 points needs fuller cross-sentence interaction, a GloVe warm-start, or running the collapse to a true fixed point — not more epochs.

Citation / source

Source code, training scripts and the full honest write-up: https://github.com/chetanxpatil/livnium (reached/pure).

License: PolyForm Noncommercial 1.0.0 — free for individuals, students, researchers and nonprofits; commercial use requires a paid license.

CollapseNLI by Chetan Patil.