Instructions to use vtava/SmolLM2-135M-CeNN-Partition-V3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use vtava/SmolLM2-135M-CeNN-Partition-V3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="vtava/SmolLM2-135M-CeNN-Partition-V3")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("vtava/SmolLM2-135M-CeNN-Partition-V3", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use vtava/SmolLM2-135M-CeNN-Partition-V3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "vtava/SmolLM2-135M-CeNN-Partition-V3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vtava/SmolLM2-135M-CeNN-Partition-V3",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/vtava/SmolLM2-135M-CeNN-Partition-V3

SGLang

How to use vtava/SmolLM2-135M-CeNN-Partition-V3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "vtava/SmolLM2-135M-CeNN-Partition-V3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vtava/SmolLM2-135M-CeNN-Partition-V3",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "vtava/SmolLM2-135M-CeNN-Partition-V3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "vtava/SmolLM2-135M-CeNN-Partition-V3",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use vtava/SmolLM2-135M-CeNN-Partition-V3 with Docker Model Runner:
```
docker model run hf.co/vtava/SmolLM2-135M-CeNN-Partition-V3
```

SmolLM2-135M CeNN Partition V3

Validation-selected partition_conservative TinyCeNN Integrated Memory V3 checkpoint over HuggingFaceTB/SmolLM2-135M.

Architecture

Variant: cenn_partition
Replaced attention layers: [0, 29]
Feature dimension: 64
Block size: 32
Sink tokens: 4
Exact base revision: 93efa2f097d58c2a74874c7e644dbc9b0cee75a2
TinyCeNN source commit: 0efdb79575204e9f02624ef64e0e6286076ab227

The remaining Transformer layers retain standard attention. This is a partial hybrid research checkpoint, not a fully attention-free model.

Held-out evaluation

Context	Perplexity	PPL ratio vs original	Cache ratio	Prefill speedup	Decode speedup
256	16.421	0.9993	0.9857	0.830×	0.893×
512	14.900	1.0021	0.9595	0.849×	0.910×
1024	14.348	1.0034	0.9464	0.829×	0.892×
2048	14.690	1.0093	0.9399	0.830×	0.891×

Ratios below 1.0 are better for perplexity/cache. Speedups above 1.0 are faster. Current PyTorch CeNN kernels are experimental and are not yet optimized like GPU SDPA.

Load

import sys
from huggingface_hub import snapshot_download
folder = snapshot_download('vtava/SmolLM2-135M-CeNN-Partition-V3')
sys.path.insert(0, folder)
from load_model import load_model
model, tokenizer = load_model('vtava/SmolLM2-135M-CeNN-Partition-V3')

Reproducibility

Validation NLL: 2.8096204151709876
Before-joint validation NLL: 2.815813680489858
Joint updates: 300
Trainable TinyCeNN parameters: 124050
Training precision: bfloat16
Full benchmark evidence is under benchmark/.

Limitations

This checkpoint uses a limited held-out set and one training seed. It does not establish universal superiority over Transformer attention. Qualitative generations are not benchmark evidence.

Source and licenses

TinyCeNN-LM: https://github.com/vtavakkoli/TinyCeNN-LM Base model: https://huggingface.co/HuggingFaceTB/SmolLM2-135M SmolLM2 is Apache-2.0. TinyCeNN-LM source is MIT; its license copy is included as LICENSE-TinyCeNN-LM.

Downloads last month: -; Downloads are not tracked for this model. How to track

Model tree for vtava/SmolLM2-135M-CeNN-Partition-V3

Base model

HuggingFaceTB/SmolLM2-135M

Finetuned

(939)

this model