mryesiller/devtools-assistant-360m-q4f16_1-MLC

DevTools Assistant 360M — q4f16_1 MLC

MLC-LLM build of devtools-assistant-360m-fused, for running in the browser with WebLLM.

It routes a problem described in plain English to one of the 210 tools on devtools.tools. Measured accuracy, training setup and known weaknesses are on the base repo's card.

Use with WebLLM

No custom model library is needed. Fine-tuning changed the weights, not the architecture, so MLC's prebuilt SmolLM2-360M q4f16_1 wasm applies unchanged:

const appConfig = {
  ...webllm.prebuiltAppConfig,
  model_list: [
    ...webllm.prebuiltAppConfig.model_list,
    {
      model: "https://huggingface.co/mryesiller/devtools-assistant-360m-q4f16_1-MLC",
      model_id: "devtools-assistant-360m-q4f16_1-MLC",
      model_lib:
        "https://raw.githubusercontent.com/mlc-ai/binary-mlc-llm-libs/main/" +
        "web-llm-models/v0_2_84/base/SmolLM2-360M-Instruct-q4f16_1_cs1k-webgpu.wasm",
    },
  ],
}

The config here deliberately matches mlc-ai/SmolLM2-360M-Instruct-q4f16_1-MLC on every field the library depends on — llama, q4f16_1, context window 8192, prefill chunk 8192, hidden size 960, vocab 49152 — so that wasm is the right one. If you rebuild this, check those first; a mismatch shows up as a load failure, not as a warning.

conv_template is chatml, which is what the base model's own tokenizer template uses and therefore what the fine-tune was trained through.

Build

Converted with mlc_llm convert_weight + gen_config at q4f16_1. The build needs a native x86_64 Linux host and a specific dependency set; the reproducible script is scripts/chatbot-dataset/finetune/mlc_export.sh in the DevTools repo, which documents why each pin is there.

License

Apache 2.0, inherited from SmolLM2-360M-Instruct.