🛡️ PHANTOM-SSM (130M) & `phantom-cache`

PHANTOM-SSM is an inference-accelerated State Space Model architecture paired with phantom-cache, delivering Radix state prefix caching (up to 2.76× TTFT speedup on shared prompts) and dynamic INT8 state quantization (75% recurrent memory reduction) evaluated across multi-turn serving and long sequence benchmarks.

Author: Prannessh K.V.A. (prannesshkva@gmail.com)
Official CERN Zenodo Record & DOI: 10.5281/zenodo.22177116
PyPI Package: pip install phantom-cache
License: Business Source License 1.1 (BSL 1.1)

⚡ CUDA-Synchronized Serving Benchmarks (Tesla P100 GPU)

Empirical measurements with CUDA hardware synchronization across 10 benchmark iterations:

Prompt Prefix Length	Cold Prefill Latency	Warm Cached TTFT	Serving Acceleration	Radix Lookup Overhead
128 tokens	9.47 ms	7.06 ms	1.34× Speedup	0.0182 ms (18.2 µs)
256 tokens	7.43 ms	6.69 ms	1.11× Speedup	0.0270 ms (27.0 µs)
512 tokens	14.75 ms	6.66 ms	2.22× Speedup	0.0494 ms (49.4 µs)
1,024 tokens	18.76 ms	6.80 ms	2.76× Speedup	0.0900 ms (90.0 µs)

Observation: While cold prefill latency increases with prompt length (9.47 ms → 18.76 ms), warm cached TTFT remains constant at ≈ 6.7 ms. Radix tree lookup overhead is sub-millisecond (< 90 µs).

💾 Memory & State Quantization Benchmarks

Metric	Full Precision (FP32)	Dynamic INT8 Quantized	Savings / Fidelity
Recurrent State Footprint (24 Layers)	18.0 MB	4.5 MB	75.0% Memory Saved
State Reconstruction Error (RMSE)	0.0000	0.01056	High-fidelity recovery (~1.0% deviation)
Long-Sequence Needle Recall (@ 100k)	41.2% (Decayed)	98.4% (Associative Accumulator)	Mitigates long-range norm loss

📦 Quickstart: Using `phantom-cache` (PyPI)

Install the standalone prefix-caching and state-quantization engine:

pip install phantom-cache

Radix Prefix Caching & Prompt Resumption

import torch
from phantom_ssm_cache import PhantomSSMCache

# Initialize Radix Prefix Cache for SSMs (with dynamic INT8 state quantization)
cache = PhantomSSMCache(
    d_model=768,
    d_state=16,
    n_layers=24,
    quantize_int8=True  # 75% recurrent state memory savings
)

# Shared system prompt token IDs
system_prompt_tokens = [15496, 11, 703, 1422]  # "You are a helpful AI assistant..."
h_ssm = torch.randn(24, 1, 1536, 16)
h_conv = torch.randn(24, 1, 1536, 4)

# Cache recurrent hidden state at prefix node
cache.insert_prefix(system_prompt_tokens, h_ssm, h_conv)

# In incoming dialogue turns, instantly restore matching state
cached_state, matched_len = cache.match_longest_prefix(system_prompt_tokens + [234, 567])
print(f"Matched prefix length: {matched_len} tokens (Skipped full prefill re-computation!)")

🔬 Mathematical Specifications

1. Radix Prefix Tree Traversal for Recurrent States

Given prompt token sequence $\mathbf{x}_{1:T} = (x_1, x_2, \dots, x_T)$, the Radix tree stores hidden states at prefix nodes:

$\mathcal{S}_K = \left\{ h_{\text{ssm}}^{(l)}(K), h_{\text{conv}}^{(l)}(K) \right\}_{l=1}^L$

$\text{Prefill Time} = \mathcal{O}(T - K) \quad \text{instead of } \mathcal{O}(T)$

2. Dynamic Symmetric INT8 State Quantization

Quantizes recurrent state vectors channel-wise with negligible reconstruction error ($\text{RMSE} \le 0.01056$):

$s_c = \frac{\max |H_c|}{127.0}, \quad H_{\text{int8}} = \text{clamp}\left( \left\lfloor \frac{H}{s_c} + 0.5 \right\rfloor, -128, 127 \right)$

3. Associative Quantum Density Memory

Mitigates long-sequence norm decay using continuous outer-product key-value accumulation:

$\rho_t = (1 - \alpha)\rho_{t-1} + \alpha(k_t \otimes v_t^\top)$

📜 Citation & Intellectual Property

@software{phantom_cache_2026,
  author       = {Prannessh, K.V.A.},
  title        = {PHANTOM-CACHE: Radix State Prefix Caching and Dynamic INT8 State Quantization for State Space Models},
  month        = aug,
  year         = 2026,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.22177116},
  url          = {https://doi.org/10.5281/zenodo.22177116}
}

Downloads last month: -

Safetensors

Model size

0.1B params

Tensor type

F32

🛡️ PHANTOM-SSM (130M) & phantom-cache