slm-125m-base-onnx

ONNX export of DeependraVerma/slm-125m-base, dynamically quantized to int8, built specifically to run entirely inside a web browser via transformers.js (@huggingface/transformers v3) — no server, no API, no cost.

Same weights, same architecture, same training as the PyTorch base model — this repo just packages them for WebAssembly inference. See DeependraVerma/slm-125m-base for the full model card (training data, procedure, evaluation, limitations); this page documents the ONNX-specific details.

• Repo / full build: github.com/DeependraVerma/legal-slm-125M
• Live demo (Playground, In-browser mode): deependraverma-ai-legal-slm-125-m.vercel.app
• Author: Deependra Verma — Generative AI Researcher / AI Engineer (Hugging Face)
• Source (PyTorch) model: DeependraVerma/slm-125m-base

This is a base (completion) model, not an assistant. It continues text plausibly and, at 125M parameters, fabricates specifics (names, numbers, citations). Never use its output as legal, financial, or factual advice. For a Q&A-capable browser model, see legal-slm-125m-sft-onnx.

What's in this repo

File	What
onnx/model.onnx	fp32 ONNX export (exported via optimum)
onnx/model_quantized.onnx	dynamically quantized int8 — the one the browser demo loads; roughly a quarter the size of the fp32 file
tokenizer.json, tokenizer_config.json, special_tokens_map.json	the project's own 16,384-vocab byte-level BPE tokenizer
config.json	the same LlamaConfig as the source model (12L/768d/12h, RoPE θ=10,000, SwiGLU 3,072, 1,024 ctx, 16,384 vocab)

Why this exists

Hugging Face's free serverless Inference API does not host arbitrary custom models, so the only genuinely free, zero-infrastructure way to let the public run this model is to ship the weights to the client and run them there. This export powers the "In-browser" mode of the live demo's Playground: the model downloads once (cached by the browser afterward) and every generation runs on the visitor's own device via WebAssembly — no backend, $0 forever.

The export/quantization was verified end-to-end with a real transformers.js load-and-generate test in this project (not just "the export step exited 0").

How to use (transformers.js, in the browser)

This is the real pattern used in this project's own front end (web/app/lib/browserModelBase.ts):

import { AutoTokenizer, AutoModelForCausalLM, TextStreamer, env } from "@huggingface/transformers";

const ONNX_BASE_REPO = "DeependraVerma/slm-125m-base-onnx";
env.allowLocalModels = false;

const tokenizer = await AutoTokenizer.from_pretrained(ONNX_BASE_REPO);
const model = await AutoModelForCausalLM.from_pretrained(ONNX_BASE_REPO, {
  dtype: "q8",       // loads onnx/model_quantized.onnx
  device: "wasm",
  progress_callback: (p) => {
    if (p?.status === "progress" && p?.total) {
      console.log(`Loading: ${Math.round((p.loaded / p.total) * 100)}%`);
    }
  },
});

const prompt = "The plaintiff filed a motion for summary judgment, arguing that";
const inputs = tokenizer(prompt, { add_special_tokens: false });
const eos = tokenizer.model.tokens_to_ids.get("<|eos|>");

const streamer = new TextStreamer(tokenizer, {
  skip_prompt: true,
  skip_special_tokens: true,
  callback_function: (token) => process.stdout.write(token),
});

await model.generate({
  ...inputs,
  max_new_tokens: 150,   // keep modest — browser WASM is single-threaded
  do_sample: true,
  temperature: 0.8,
  top_k: 50,
  top_p: 0.95,
  eos_token_id: eos,
  streamer,
});

Evaluation

The underlying weights are unchanged from the PyTorch source model — the same held-out perplexity applies:

Metric	Value
Held-out perplexity (full 20.6M-token validation set)	7.76
Validation loss	2.049

Quantization to int8 was verified with a real transformers.js load+generate run producing coherent, on-domain output, not a numerical perplexity re-check of the quantized graph specifically — treat the number above as the pre-quantization reference (int8 dynamic quantization typically costs a small, usually imperceptible amount of quality at this scale).

Citation

@misc{verma2026legalslm125m,
  author = {Deependra Verma},
  title  = {legal-slm-125M: A 125M-Parameter Legal and Financial Language Model Trained From Scratch},
  year   = {2026},
  url    = {https://huggingface.co/DeependraVerma/slm-125m-base},
  note   = {Code: https://github.com/DeependraVerma/legal-slm-125M}
}

Author

Deependra Verma — Generative AI Researcher / AI Engineer. GitHub · Hugging Face

License

MIT — see LICENSE in the source repo. This is a research artifact, not a source of legal or financial advice.