elvinmarkmv/minimal-ai-models

What is a .miniai Model?

A .miniai file is a single-file, self-contained binary container:

• Zero Python/PyTorch at Runtime: Everything required to execute the model (architectural hyperparameters, tokenizers or phonemizers, Mel filterbanks, tensor directory table, and 64-byte aligned weights) is packaged into a single contiguous file.
• Direct Memory-Mapping (mmap): Models load instantaneously with zero runtime heap duplication. Weights are mapped directly into user space and fed to vectorized compute kernels (AVX2+FMA or ARM NEON).
• Block-Quantized (Q8_0): Weight matrices are compressed into 32-element symmetric blocks with 32-bit floating-point scale factors (9 bits/element), preserving near-lossless numerical fidelity ($R > 0.9999$ vs FP32 PyTorch reference) while slashing memory bandwidth and disk footprint by ~70%.
• FP32 Sensitive Layers: LayerNorm / RMSNorm scales, 1D biases, embeddings, recurrent weights, and voice style vectors are preserved in IEEE 754 float32 to prevent catastrophic drift.

Model Catalog

All models in this repository are pre-converted to Q8_0 block quantization and tested warning-free on miniai:

Model File	Architecture	Base Model	Size	Tasks / Domain	Specifications
kokoro_82m_q8_0.miniai	Kokoro TTS	hexgrad/Kokoro-82M	299 MB	24kHz Single-Shot Text-to-Speech	12L / 12H / 512D, StyleTTS 2 + ISTFTNet vocoder, 6 voices, embedded 89k G2P lexicon
smollm2_135m_instruct_q8_0.miniai	LLaMA / SmolLM2	HuggingFaceTB/SmolLM2-135M-Instruct	176 MB	Conversational Chat & Assistant	30L / 9Q-Heads / 3KV-Heads (3x GQA), SwiGLU, RoPE (8k ctx), RMSNorm
smollm2_135m_q8_0.miniai	LLaMA / SmolLM2	HuggingFaceTB/SmolLM2-135M	176 MB	Autoregressive Text Completion	30L / 9Q-Heads / 3KV-Heads (3x GQA), SwiGLU, RoPE (8k ctx), RMSNorm
gpt2_q8_0.miniai	GPT-2 Base	openai-community/gpt2	176 MB	Autoregressive Text Generation	12L / 12H / 768D, 1024 context, BPE tokenizer, persistent KV cache
whisper_tiny_q8_0.miniai	Whisper Enc-Dec	openai/whisper-tiny	65 MB	Automatic Speech Recognition (ASR)	4 Enc / 4 Dec, 6H / 384D, built-in 80-mel filterbank & cross-attention
distilbert_squad_q8_0.miniai	DistilBERT	distilbert-base-cased-distilled-squad	70 MB	Extractive Question Answering	6L / 12H / 768D, 512 context, SQuAD span extraction head
minilm_l6_v2_q8_0.miniai	MiniLM / BERT	sentence-transformers/all-MiniLM-L6-v2	25 MB	Sentence Embeddings & Search	6L / 12H / 384D, WordPiece tokenizer, mean pooling, cosine similarity
yolos_tiny_q8_0.miniai	YOLOS (ViT)	hustvl/yolos-tiny	9.4 MB	Vision Object Detection	12L / 3H / 192D, 100 queries, 91 COCO classes, dynamic 2D bicubic interpolation