AI & ML interests
Ultra-Low Bit quantised and binary/ternary-based model architecture.
Recent Activity
OneBit AI
Ternary models for on-device language and speech.
We train and recover models whose weights are mostly {-1, 0, +1} (about 1.58 bits per weight), then pack them for real runtimes — not INT8 wrappers around cloud-scale checkpoints.
The bet is simple: if the body of the network is ternary from the start (or recovered to true ternary), you can fit useful models in megabytes and run them on a laptop CPU or a single consumer GPU.
What we build
| Program | What it answers | Status (honest) |
|---|---|---|
| FineBit | Can we pretrain a BitNet-compatible ternary LLM from scratch, export I2_S GGUF, and decode fast on CPU? | 30M → 70M on FineWeb-Edu. 70M I2_S is ~61 MB and ~617 tok/s on CPU. Open generation is still early (not a chat product). |
| Cloe | Can a fully ternary ~0.8B LLM be recovered from quantization damage into fluent English? | Fluency restored (PPL collapse reversed). Knowledge is still far behind the bf16 teacher. |
| BitVoice | Can TTS and STT be designed ternary from day one? | Packed TTS ~5.5 MB, STT ~11.5 MB. Libri test-clean STT is usable-but-not-Whisper; TTS is Griffin–Lim, not studio quality. |
| PTQ / QAT SLMs | How much task ability survives when Qwen3-class models go ternary? | Lossless lattice packing on Qwen3-4B/8B (PPL unchanged). A native ternary matmul kernel is still unbuilt. |
Hardware constraint by design: a single RTX 5070 12 GB. If it only works on an 8×A100 box, it is not our product story.
How we differ
- Train or recover ternary, with STE / BitLinear-style QAT — not “quantize once at export and hope.”
- Ship a runtime path — FineBit targets microsoft/BitNet
bitnet-b1.58I2_S and bitnet.cpp, not only a PyTorch demo. - Keep sensitive tips in higher precision (embeddings, norms, heads) so the body can compress without total collapse.
- Publish the hard metrics — PPL when it lies, WER/CER for speech, MMLU/GSM8K when they fail.
This org
Models, datasets, and Spaces will land here as we release them. Until then, this card is the public map of the stack.
Contact: Anirudh Malik · OneBit AI