VishalMysore/CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct-q0f16-MLC

CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct — q0f16 MLC build

Real MLC-converted weights of cycloneboy/CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct, a text-to-SQL fine-tune of Qwen2.5-Coder-0.5B-Instruct released alongside the paper:

Lei Sheng & Shuai-Shuai Xu, "SLM-SQL: An Exploration of Small Language Models for Text-to-SQL", arXiv:2507.22478 (Jul 2025). Code & weights

This repo exists to run that checkpoint in-browser via WebGPU, using WebLLM, as the SQL-specialist model in nl2sqlBenchMark.

What's in this repo

Converted with mlc_llm convert_weight at q0f16 (plain fp16 cast, no group quantization): mlc-chat-config.json, tensor-cache.json, params_shard_0.bin–params_shard_24.bin, and the base model's tokenizer files. No architecture or weight values were altered — this is a straight precision-cast conversion of the original checkpoint into MLC's runtime format, not a re-trained or distilled model.

Why q0f16 instead of the usual q4f16_1

q4f16_1 (4-bit group-quantization) is what most WebLLM prebuilt models use, and was the original target here. mlc_llm convert_weight --quantization q4f16_1 currently segfaults during group-quantization kernel compilation on the public mlc-ai-nightly-cpu / mlc-llm-nightly-cpu wheels — a confirmed, unresolved upstream bug: mlc-ai/mlc-llm#3283. q0f16 sidesteps that code path entirely, converts cleanly, and is strictly more faithful to the source weights than 4-bit would have been — at the cost of a larger download (~960 MB).

Because CscSQL-Merge is a full fine-tune of the exact base architecture (Qwen2.5-Coder-0.5B-Instruct → treated as Qwen2 0.5B by MLC), WebLLM's existing precompiled q0f16 WebGPU shader library for that architecture is reused as-is — no custom shader/wasm compile was needed for this repo, only the weight conversion.

Use

import { CreateMLCEngine } from "@mlc-ai/web-llm";

const appConfig = {
  model_list: [{
    model: "https://huggingface.co/VishalMysore/CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct-q0f16-MLC",
    model_id: "CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct-q0f16-MLC",
    model_lib: "https://raw.githubusercontent.com/mlc-ai/binary-mlc-llm-libs/main/web-llm-models/v0_2_84/base/Qwen2-0.5B-Instruct-q0f16_cs1k-webgpu.wasm",
    low_resource_required: true,
    vram_required_MB: 1624.12,
    overrides: { context_window_size: 4096 },
  }],
};

const engine = await CreateMLCEngine(
  "CscSQL-Merge-Qwen2.5-Coder-0.5B-Instruct-q0f16-MLC",
  { appConfig }
);

Full reproducible conversion steps: tools/convert-cscsql-to-mlc.md in the nl2sqlBenchMark repo.

License

MIT — matching the base checkpoint's release. Original weights and training by the SLM-SQL / CSC-SQL authors; this repo only contains a format conversion.

Citation

@article{sheng2025slmsql,
  title   = {SLM-SQL: An Exploration of Small Language Models for Text-to-SQL},
  author  = {Sheng, Lei and Xu, Shuai-Shuai},
  journal = {arXiv preprint arXiv:2507.22478},
  year    = {2025}
}