🏛️ qu_ssm (QU-SSM-130M-MoE): Continuous Quasi-Unitary State Space Model with Sparse Mixture-of-Experts

qu_ssm (specifically qu_ssm-130Moe / QU-SSM-130M-MoE) is a continuous quasi-unitary state space sequence architecture co-designed with sparse SwiGLU Mixture-of-Experts (MoE). Designed and invented by Prannessh K.V.A. (Sole Architect & Inventor), qu_ssm eliminates monotonic exponential memory dissipation present in classical real-valued SSMs (e.g. Mamba-1/2) while delivering strictly constant O(1) step memory (0.19 MB) and 3.32x generation speedup over modern SOTA Transformers.

🔍 What is qu_ssm (QU-SSM-130M-MoE)?

qu_ssm is a state-of-the-art linear-time sequence engine that formulates state transitions as continuous Lie-group rotations over SO(2) ≅ U(1). Unlike classical state space models that suffer from exponential forgetting (e^(-α·t) → 0), qu_ssm preserves state norm (‖R(θ)‖₂ ≡ 1.00000) while dynamically decoupling memory damping (γ_t).

Key Advantages of qu_ssm-130Moe:

⚡ 3.32x Faster Inference: Generates at 5.55 tok/s (180.16 ms/tok step latency) compared to 1.67 tok/s for modern Transformers (SmolLM-135M).
💾 Constant O(1) Memory: Requires strictly 0.19 MB of state RAM regardless of context length (1,894x smaller than a 360 MB KV-cache at L = 8,192).
🧠 42% Lower Active Compute: Evaluates only 78.27M active parameters per token across 8 SwiGLU experts with Top-2 routing.
🌊 Lowest Spectral Phase Distortion: Demonstrates 1.9648 rad phase error on live financial market streams (BTC, ETH, SPY, NVDA).

🏆 SOTA Benchmark Leaderboard (~135M Scale)

Model Architecture	Total Params	Active Params / Token	Generation Speed	Step Latency	RAM at L=8,192
SmolLM-135M (Hugging Face)	134.52M	134.52M (Dense)	1.67 tok/s	597.86 ms	360.00 MB
Mamba-130M-HF (Albert Gu et al.)	129.14M	129.14M (Dense)	1.98 tok/s	506.18 ms	0.19 MB
qu_ssm-130Moe (QU-SSM-130M-MoE)	134.89M	78.27M (Top-2 Sparse)	5.55 tok/s (🥇 3.32x)	180.16 ms	0.19 MB (🥇 Constant)

🔬 Mathematical Formulation of qu_ssm

The core recurrence equation evolves according to:

h_t = γ_t · R(θ_t) · h_{t-1} + u_t

Where:

R(θ_t) ∈ SO(2) is a 2x2 rotation matrix with strict norm preservation: ‖R(θ_t)‖₂ ≡ 1.00000.
γ_t = σ(W_γ · x_t + b_γ) ∈ (0, 1) is the decoupled memory gate controlling damping independently.

Exact Real Dual-Component Parallel Prefix Scan:

S = cumsum(log γ_t).clamp(min=-12.0, max=0.0)
Φ = cumsum(θ_t)
h_t = exp(S) · [ cos(Φ) · cumsum(u_real) - sin(Φ) · cumsum(u_imag) ]

💻 How to Install and Run qu_ssm

Run qu_ssm-130Moe directly using standard Hugging Face transformers:

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load qu_ssm-130Moe checkpoint
model_id = "Prannesshkva/QU-SSM-130M-MoE"
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Inference with qu_ssm
prompt = "Once upon a time in a world of state space models,"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
output = model.generate(input_ids, max_new_tokens=50)
print(tokenizer.decode(output[0]))

🌐 Universal Multimodal Support in qu_ssm

The qu_ssm backbone supports four continuous and discrete sequence interfaces:

Modality	Python Class	Input Data Format
📝 Language & Code	`QUSSMForCausalLM`	BPE Token IDs (50,257 Vocab)
🎙️ Raw 16kHz Audio	`QUSSMForAudio`	Continuous 1D Waveform
📈 Financial & Sensors	`QUSSMForSensorTelemetry`	Continuous Time-Series Ticks
🖼️ 2D Spatial Vision	`VisionQUSSM`	16x16 Pixel Patches

❓ Frequently Asked Questions (FAQ) about qu_ssm

What makes qu_ssm different from Mamba?

Classical SSMs like Mamba enforce negative real decay (A ∈ ℝ⁻), creating a dissipative low-pass filter that loses oscillatory and high-frequency phase information (e^(-α·t) → 0). qu_ssm uses dynamic Lie-group rotations over SO(2) with ‖R(θ)‖₂ ≡ 1.00000, preserving state energy indefinitely while dynamically controlling damping via γ_t.

Is qu_ssm an MoE model?

Yes. qu_ssm-130Moe co-designs the quasi-unitary recurrent state space engine with an 8-expert SwiGLU Mixture-of-Experts layer using Top-2 routing, evaluating only 78.27M active parameters per step.

What is the pretraining dataset for qu_ssm-130Moe?

This checkpoint is an early-stage foundational prototype pre-trained on ~20 Million tokens of structured narrative reasoning (TinyStories) over 5 epochs, achieving a 70x perplexity reduction (3,358 → 48.21 PPL).

🔒 Intellectual Property & Citation

Sole Architect & Inventor: Prannessh K.V.A.
Permanent Research DOI: 10.5281/zenodo.22217820
License: Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0)

@software{prannesshkva_qu_ssm_2026,
  author       = {Prannessh K.V.A.},
  title        = {QU-SSM-MoE: Continuous Quasi-Unitary Lie-Group State Space Models with Sparse Mixture-of-Experts},
  month        = sep,
  year         = 2026,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.22217820},
  url          = {https://doi.org/10.5281/zenodo.22217820}
}

🔗 Related qu_ssm Models & Ecosystem Links

Downloads last month: 284

Safetensors

Model size

0.1B params

Tensor type

F32

Prannesshkva
/

QU-SSM-130M-MoE