dstefl/EuroLLM-1.7B-Instruct-q4f16_1-MLC

EuroLLM-1.7B-Instruct · q4f16_1 · MLC/WebGPU

utter-project/EuroLLM-1.7B-Instruct converted to the MLC format so it runs fully client-side in a browser via web-llm. No such build existed publicly; this repo is the conversion, not a fine-tune — the weights are the original model's, quantized to q4f16_1 (~932 MB).

What is in this repo

• Quantized weight shards + tokenizer + mlc-chat-config.json (context_window_size: 4096).
• EuroLLM-1.7B-Instruct-q4f16_1-MLC-webgpu.wasm — the model library compiled for WebGPU against the exact TVM generation web-llm 0.2.84 ships its runtime for (mlc-llm 2008fe8, 2026-05-11). A library built against a newer TVM fails at load with LinkError: TVMFFIGetCustomAllocator — that pin is the hard-won part.

How to use (web-llm)

import { CreateMLCEngine } from "@mlc-ai/web-llm";

const engine = await CreateMLCEngine("EuroLLM-1.7B-Instruct-q4f16_1-MLC", {
  appConfig: {
    model_list: [
      {
        model: "https://huggingface.co/dstefl/EuroLLM-1.7B-Instruct-q4f16_1-MLC",
        model_id: "EuroLLM-1.7B-Instruct-q4f16_1-MLC",
        model_lib:
          "https://huggingface.co/dstefl/EuroLLM-1.7B-Instruct-q4f16_1-MLC/resolve/main/EuroLLM-1.7B-Instruct-q4f16_1-MLC-webgpu.wasm",
        overrides: { context_window_size: 4096 },
      },
    ],
  },
});

Grammar-constrained JSON (XGrammar) works — it is the same runtime as every prebuilt web-llm model.

How it was built

TVM + mlc_llm compiled from source on a GitHub-hosted runner (no PyPI wheel combination could convert a Llama-family model on linux at the time — the nightly pair segfaulted in tvm::ffi::ReprPrint, the release pair had a tvm-ffi ABI mismatch). Build script and workflow: [dstefl/householdsim → .github/workflows/eurollm-build-source.yml] (private repo; the script is reproducible: LLVM 18, uv-managed CPython 3.12, 3rdparty/tvm submodule, editable --no-deps installs so pip can never pull the mismatched PyPI tvm-ffi, emsdk + web/prep_emcc_deps.sh, then the three standard MLC steps).

Honest quality note

Verified loading and generating on real hardware (RTX, web-llm 0.2.84): weights stream at network speed, reload from cache ~30 s. An 8-prompt Czech conversation A/B ran end-to-end (1.6–7.3 s per reply) — with the quality you should expect of an unaligned 1.7B: identity confusion, English leaking mid-reply, occasional repetition loops. This build is a pipeline proof and a small-footprint European option, not a Czech-quality answer. The base model's published figure is Arc-Challenge 0.3496 (no MMLU-Pro exists for the 1.7B).

License & attribution

Base model © UTTER project, released under Apache 2.0; this quantized conversion is distributed under the same license. Not affiliated with the UTTER project. Converted 2026-08-23 for wattburg.com (a browser household-electricity simulator that runs its AI fully client-side).