all-MiniLM-L6-v2-yat

all-MiniLM-L6-v2 with every feed-forward (FFN) block replaced by a sigmoid-gated Yat-kernel MLP, obtained by a two-phase, mostly data-free distillation: Phase 1 fits each block on random-token activations (no corpus); Phase 2 is a ~48s real-activation fine-tune. The swap preserves MTEB STS within ~0.3% of the base model.

from sentence_transformers import SentenceTransformer
m = SentenceTransformer("mlnomad/all-MiniLM-L6-v2-yat", trust_remote_code=True)
m.encode(["hello world"])

The Yat FFN computes (alpha * (x.W+b)^2/(||x-W||^2+eps) * sigmoid(gate(x))) @ A + c, a kernel-native replacement for the GELU MLP. Can be further fine-tuned to recover any residual gap.