esilva/SlopCoder-Mongo-1.5B-full-ONNX

SlopCoder-Mongo-1.5B-full · ONNX (CPU and DirectML GPU)

ONNX Runtime GenAI builds of esilva/SlopCoder-Mongo-1.5B-full, a MongoDB autocomplete / editor-rewrite model distilled from a DeepSeek Coder 6.7B teacher into Qwen2.5-Coder-1.5B. Prompt format, training data, and limitations are described in the main model card.

Folder	Hardware	Quantization	Size	Use it for
dml-fp16/	GPU (DirectML)	FP16	3.10 GB	recommended with a GPU: bf16 accuracy, ~156 ms to first token
dml-int4/	GPU (DirectML)	INT4 RTN on MatMul only (embeddings kept in FP16)	1.34 GB	GPUs with less free memory; more tokens/s
int8/	CPU	INT8 (MatMulNBits, block 32)	2.69 GB	recommended on CPU: no measurable loss on free-form requests
int4/	CPU	INT4 k-quant (MatMulNBits, block 32, accuracy level 4), lm_head in INT8	1.13 GB	smaller, but loses accuracy on free-form requests

Every folder is self-contained: model, genai_config.json (which selects the execution provider), tokenizer, slopstudio-model.json (declares hardware: cpu or gpu) and a copy of the license files. Exported with the ONNX Runtime GenAI 0.15.2 builder; CPU builds use the cpu execution provider, dml-* builds the dml provider with DirectML graph capture (all nodes on the GPU, so the INT4 build does not quantize the tied embeddings).

Slop Studio: Preferências → Autocomplete → Baixar modelo with this repository lists the four folders, downloads the chosen one and verifies every file against the hashes published here.

Evaluation

Programmatic benchmark: 600-example subset (bf16 on the same examples: APT 3.28, rewrite intent 94.1%). Free-form: 120 handwritten requests (bf16: 87) and a 40-case blind set (bf16: 29). CPU: ONNX Runtime GenAI 0.15.2, AMD Ryzen 9 7900. GPU: ONNX Runtime GenAI WinML 0.15.2, DirectML, AMD Radeon RX 7800 XT, one model per process. Greedy decoding.

Variant	APT	Syntax valid	MongoDB valid	Rewrite intent	Free-form (120)	Blind (40)	TTFT mean / p95	Tokens/s
dml-fp16 (GPU)	3.28	93.4%	92.7%	92.6%	87	29	156 / 178 ms	67
dml-int4 (GPU)	3.32	92.1%	91.2%	93.1%	85	27	168 / 202 ms	100
int8 (CPU)	3.26	93.8%	92.9%	92.6%	87	29	516 / 846 ms	31
int4 (CPU)	3.40	92.3%	91.9%	92.6%	80	—	576 / 945 ms	44

• dml-fp16 and int8 answer exactly the same free-form cases as the bf16 model.
• With a GPU the 1.5B-full answers faster than SlopCoder-Mongo-0.5B-ONNX/int4 on CPU (215 ms) while keeping its free-form advantage (vs the 0.5B dml-fp16: 20 better / 5 worse, p = 0.004).
• Peak process RAM: 3.83 GB (int8), 2.59 GB (int4), ~0.6 GB for the DirectML builds (weights live in GPU memory, roughly the folder size; GPU memory was not measured by the benchmark).

Usage — GPU (DirectML, Windows)

hf download esilva/SlopCoder-Mongo-1.5B-full-ONNX --include "dml-fp16/*" --local-dir SlopCoder-Mongo-1.5B-full-ONNX
pip install onnxruntime-genai-winml==0.15.2

Use a clean environment: if the CPU onnxruntime Python package is also importable, the WinML package may load its DLL and refuse DirectML. genai_config.json already selects DirectML:

import onnxruntime_genai as og

model = og.Model("SlopCoder-Mongo-1.5B-full-ONNX/dml-fp16")
tokenizer = og.Tokenizer(model)
STOP = {151643, 151645, 151659, 151660, 151661, 151662}  # endoftext, im_end, fim_prefix, fim_middle, fim_suffix, fim_pad

prompt = '<|fim_prefix|>db.getCollection("orders").find({ status: "paid", total: { $gte: <|fim_suffix|> } })<|fim_middle|>'
ids = tokenizer.encode(prompt)
params = og.GeneratorParams(model)
params.set_search_options(do_sample=False, max_length=len(ids) + 32)
generator = og.Generator(model, params)
generator.append_tokens(ids)
out = []
while not generator.is_done():
    generator.generate_next_token()
    token = generator.get_next_tokens()[0]
    if token in STOP:
        break
    out.append(token)
print(tokenizer.decode(out) if out else "")  # decode([]) raises in 0.15.2

Known issue (ONNX Runtime GenAI WinML 0.15.2): loading a second DirectML model in the same process after releasing the first one crashed the process in our tests; loading it in a new process works.

Usage — CPU

hf download esilva/SlopCoder-Mongo-1.5B-full-ONNX --include "int8/*" --local-dir SlopCoder-Mongo-1.5B-full-ONNX
pip install onnxruntime-genai==0.15.2

The code is identical; load SlopCoder-Mongo-1.5B-full-ONNX/int8 instead:

import onnxruntime_genai as og

model = og.Model("SlopCoder-Mongo-1.5B-full-ONNX/int8")
tokenizer = og.Tokenizer(model)
STOP = {151643, 151645, 151659, 151660, 151661, 151662}

prompt = '<|fim_prefix|>db.getCollection("orders").find({ status: "paid", total: { $gte: <|fim_suffix|> } })<|fim_middle|>'
ids = tokenizer.encode(prompt)
params = og.GeneratorParams(model)
params.set_search_options(do_sample=False, max_length=len(ids) + 32)
generator = og.Generator(model, params)
generator.append_tokens(ids)
out = []
while not generator.is_done():
    generator.generate_next_token()
    token = generator.get_next_tokens()[0]
    if token in STOP:
        break
    out.append(token)
print(tokenizer.decode(out) if out else "")

For the editor-context header and the rewrite prompt, see the main model card.

License

This is a quantized copy of a Derivative of DeepSeek Coder under the DeepSeek License Agreement.

• DeepSeek License Agreement — LICENSE: its use-based restrictions (paragraph 5 and Attachment A) apply and must be passed on to anyone you redistribute to, together with a copy of the agreement.
• Apache License 2.0 — LICENSE-APACHE-2.0, for the Qwen2.5-Coder-1.5B initial weights.
• Attribution details: NOTICE.md. Each variant folder carries the same three files.

---

Resumo em português

Versões ONNX Runtime GenAI do SlopCoder-Mongo-1.5B-full, a opção orientada a chat:

• GPU DirectML: dml-fp16/ (3,10 GB, recomendado com GPU: mesma qualidade do bf16 — 87 de 120 pedidos livres — e TTFT ~156 ms) e dml-int4/ (1,34 GB, 85 de 120, mais tokens/s).
• CPU: int8/ (2,69 GB, recomendado na CPU, 87 de 120, TTFT ~516 ms) e int4/ (1,13 GB, 80 de 120).
• Com GPU, o 1.5B-full responde mais rápido que o 0.5B INT4 em CPU e mantém a vantagem nos pedidos livres.
• No Slop Studio, Preferências → Autocomplete → Baixar modelo com este repositório lista as quatro pastas.

O modelo foi destilado de um professor baseado no DeepSeek Coder 6.7B e por isso segue a DeepSeek License Agreement, com as restrições de uso do Anexo A, além da Apache-2.0 do Qwen2.5-Coder; cada pasta traz uma cópia das licenças. Formato de prompt e limitações: card principal esilva/SlopCoder-Mongo-1.5B-full.