dogtooting/Thoth-Q2_K-GGUF-split

Thoth Q2_K — split GGUF for in-browser inference

This repo contains Thoth.Q2_K.gguf from mradermacher/Thoth-GGUF, split into 6 shards of <480 MB each with llama-gguf-split so it can be loaded in the browser by wllama (which has a 2 GB per-file ArrayBuffer limit).

The original model, Thoth, is a mergekit merge of Mistral-7B-family models (32k context, SPM tokenizer). The base repo CoprolaliacPress/Thoth has since been removed, so these quants from mradermacher are the only surviving copy.

Files

File	Size
Thoth.Q2_K-00001-of-00006.gguf	472 MB
Thoth.Q2_K-00002-of-00006.gguf	479 MB
Thoth.Q2_K-00003-of-00006.gguf	476 MB
Thoth.Q2_K-00004-of-00006.gguf	474 MB
Thoth.Q2_K-00005-of-00006.gguf	461 MB
Thoth.Q2_K-00006-of-00006.gguf	357 MB

Total ≈ 2.72 GB, byte-identical to the source Thoth.Q2_K.gguf when merged.

Usage

Browser (wllama): pass the URL of shard 00001 — it pulls the rest automatically.

await wllama.loadModelFromUrl(
  "https://huggingface.co/dogtooting/Thoth-Q2_K-GGUF-split/resolve/main/Thoth.Q2_K-00001-of-00006.gguf"
);

llama.cpp: point -m at shard 00001, or merge first:

llama-gguf-split --merge Thoth.Q2_K-00001-of-00006.gguf Thoth.Q2_K.gguf

Prompt format

ChatML (as plain text — these are not special tokens in the vocab):

<|im_start|>system
{system}<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Stop on <|im_end|>.

Live demo: dogtooting/Thoth-GGUF-Chat