digitalsolutiosai/synthellion

SynthelionML

Learned per-token keep/drop prompt compressor for Synthelion.

A small, fully offline transformer encoder trained on Wikipedia corpora (39 languages) to predict which tokens can be dropped while preserving meaning. The model is trained using self-distillation: the rule-based SYNTACTIC compressor provides ground-truth labels, and the encoder learns to generalise beyond simple rules by attending to surrounding token context.

Architecture

Char-ngram hashing -> word embedding + feature embedding -> positional encoding
-> 2-layer TransformerEncoder (d=128, h=4) -> Linear(2) keep/drop logits

Parameter	Value
Parameters	10.4M
d_model	128
n_heads	4
n_layers	2
ffn_dim	512
Vocabulary	70,000 words + 8,192 char-ngram buckets
Max seq len	96
Input features	18 (stopword, capitalization, length buckets)
Output	2-class logits (drop / keep)

Formats

File	Format	Size	Use case
model.bin	PyTorch state_dict	39.8 MB	PyTorch inference (native)
model.safetensors	SafeTensors	39.8 MB	HuggingFace / cross-framework
synthelionml.onnx	ONNX opset 17	39.8 MB	ONNX Runtime, JS, C#, mobile

Quick start

PyTorch (native)

from synthelion.synthelionml import SynthelionMLCompressor

compressor = SynthelionMLCompressor.get_instance()
compressed = compressor.compress("Your long prompt text here...")

ONNX Runtime

import onnxruntime as ort
import numpy as np

sess = ort.InferenceSession("synthelionml.onnx")

word_ids = np.array([[...]], dtype=np.int64)       # (1, seq_len)
features = np.random.randn(1, seq_len, 18).astype(np.float32)
mask = np.ones((1, seq_len), dtype=np.bool_)

logits = sess.run(None, {
    "word_ids": word_ids,
    "features": features,
    "attention_mask": mask,
})[0]  # (1, seq_len, 2) — logits for [drop, keep]

SafeTensors (HuggingFace)

from safetensors.torch import load_file

weights = load_file("model.safetensors")
# keys use "/" separator: "_word_emb.weight", "_encoder.layers.0.self_attn.in_proj_weight", etc.

Training

• Data: Synthelion's Wikipedia corpora (74,100 training examples, 3,900 eval)
• Labels: Self-distillation from the aggressive rule-based compressor
• Languages: 39 (see config.json -> trained_languages)
• Min compression: 70% (rank-based ratio controller)
• Runtime: Fully offline, CPU-only

Files

File	Description
config.json	Model hyperparameters and training metadata
vocab.json	Word vocabulary (70k words)
tokenizer.json	HuggingFace-compatible tokenizer config
preprocessor_config.json	Feature extractor settings
model.bin	PyTorch weights
model.safetensors	SafeTensors weights
model.safetensors.index.json	SafeTensors shard index
synthelionml.onnx	ONNX export
MODEL_INFO.txt	Training provenance

License

MIT License. See LICENSE for details.