FoolDev Claude Opus 4.7 commited on
Commit
83022eb
·
1 Parent(s): 1c88b41

docs: collapse qwen36 workaround narrative now that bundle is qwen35-stamped

Browse files

Follow-up to 1c88b41, which deferred the deeper docs cleanup
("intentionally stale in this commit, follow-up will collapse").
This is that follow-up.

Touched three doc surfaces:

- README "Architecture" section: replaced the long
"qwen36-stamped, here's why upstream is the lagging side, here
are two one-liner workarounds, here's the manual rebadge
recipe, once-upstream-adds-qwen36-the-workaround-disappears"
block with a one-paragraph history note: bundle is qwen35,
loads directly, was qwen36 through v0.6.0, reverted in 964e418.
`scripts/rename_arch.py` example flipped from qwen36 -> qwen35
to qwen35 -> qwen36 to match its new "tool kept around for
future flips" framing.
- README Quick start Ollama option A: dropped the "fails today
with `unknown model architecture: 'qwen36'`" warning and the
`make heal-hf && ollama run ...` recovery line. A is now just
`ollama run hf.co/FoolDev/Thanatos-27B` — 17 GB Q4_K_M,
qwen35-stamped. B's comment lost the
"rebadges qwen36 -> qwen35" detail (irrelevant on a qwen35
bundle).
- examples/README.md Ollama setup: similar collapse — pull + run
is now the lead recipe with no intervening heal step. heal-hf
kept as a one-paragraph callout for anyone who pulled before
the 964e418 re-stamp.
- Modelfile preamble: shrunk from a 15-line qwen36-stamp +
rebadge-recipe block to a 6-line "bundle is qwen35, loads
directly, see README Architecture for history" note.

Net -39 lines across the three doc surfaces. No tooling change;
heal-hf, load-bundle, rename_arch.py all stay wired up exactly as
shipped.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

Files changed (4) hide show
  1. CHANGELOG.md +11 -5
  2. Modelfile +6 -15
  3. README.md +38 -64
  4. examples/README.md +12 -16
CHANGELOG.md CHANGED
@@ -31,16 +31,22 @@ and documentation**, not the underlying base model.
31
  stock Ollama. `make heal-hf` is mentioned as the recovery path
32
  for anyone who pulled the v0.6.0 bundle and still has the qwen36
33
  blob in their store.
 
 
 
 
 
 
 
 
 
 
34
 
35
  ### Note
36
  - `make heal-hf`, `make load-bundle`, and `scripts/rename_arch.py`
37
  stay in the repo and stay useful. heal-hf is idempotent on qwen35
38
  (skips early); load-bundle does a no-op rebadge on a qwen35 bundle
39
- and runs `ollama create` as normal. The deeper Architecture /
40
- Quick start / examples-README narrative still talks about the
41
- qwen36 stamp and the workaround dance; left intentionally stale
42
- in this commit for follow-up cleanup, since the immediate user
43
- pain (load 500 on fresh pull) is what the upload commit fixed.
44
 
45
  ### Added
46
  - `scripts/check.sh` now greps for the `VAR="$(cmd 2>/dev/null | filter)"`
 
31
  stock Ollama. `make heal-hf` is mentioned as the recovery path
32
  for anyone who pulled the v0.6.0 bundle and still has the qwen36
33
  blob in their store.
34
+ - README "Architecture" section, Quick start option A,
35
+ `examples/README.md` Ollama setup, and `Modelfile` preamble all
36
+ collapsed back to the pre-qwen36 single-path narrative. The
37
+ long "qwen36 is unloadable / here are the workarounds" prose
38
+ was replaced with a one-paragraph history note pointing at
39
+ commit 964e418 + the [CHANGELOG](CHANGELOG.md) entry. Net -39
40
+ lines across the three doc surfaces. `scripts/rename_arch.py`
41
+ example in Architecture flipped from qwen36 → qwen35 to qwen35
42
+ → qwen36 to match its new "tool kept around for future flips"
43
+ framing.
44
 
45
  ### Note
46
  - `make heal-hf`, `make load-bundle`, and `scripts/rename_arch.py`
47
  stay in the repo and stay useful. heal-hf is idempotent on qwen35
48
  (skips early); load-bundle does a no-op rebadge on a qwen35 bundle
49
+ and runs `ollama create` as normal.
 
 
 
 
50
 
51
  ### Added
52
  - `scripts/check.sh` now greps for the `VAR="$(cmd 2>/dev/null | filter)"`
Modelfile CHANGED
@@ -7,21 +7,12 @@
7
  # image input, or wait for the fix. See the Vision section in README.md.
8
  #
9
  # This repo bundles a single GGUF: Thanatos-27B.Q4_K_M.gguf (~17 GB),
10
- # stamped `general.architecture: 'qwen36'`. The FROM line below points
11
- # at it, but no released llama.cpp / Ollama recognizes `qwen36` yet, so
12
- # `ollama create thanatos-27b -f Modelfile && ollama run thanatos-27b`
13
- # fails today with `unknown model architecture: 'qwen36'`. To load now,
14
- # rebadge the bundle to `qwen35` first:
15
- #
16
- # python3 scripts/rename_arch.py --from-arch qwen36 --to-arch qwen35 \
17
- # Thanatos-27B.Q4_K_M.gguf Thanatos-27B.Q4_K_M.qwen35.gguf
18
- # # then temporarily point FROM at the qwen35 file, or use:
19
- # # echo "FROM $PWD/Thanatos-27B.Q4_K_M.qwen35.gguf" > /tmp/Modelfile.qwen35
20
- # # ollama create thanatos-27b -f /tmp/Modelfile.qwen35
21
- #
22
- # Once upstream adds the qwen36 arch entry the workaround disappears
23
- # and the FROM line below works as-is. See README "Architecture" for
24
- # the full story.
25
  #
26
  # For other quants (Q3_K_S, Q5_K_M, Q6_K, etc.), `make build QUANT=Q3_K_S`
27
  # downloads the chosen quant from unsloth/Qwen3.6-27B-GGUF and patches
 
7
  # image input, or wait for the fix. See the Vision section in README.md.
8
  #
9
  # This repo bundles a single GGUF: Thanatos-27B.Q4_K_M.gguf (~17 GB),
10
+ # stamped `general.architecture: 'qwen35'` — the arch entry every
11
+ # released llama.cpp / Ollama loads under for the Qwen 3.6 hybrid
12
+ # SSM + attention family. `ollama create thanatos-27b -f Modelfile &&
13
+ # ollama run thanatos-27b` loads it directly. See README
14
+ # "Architecture" for the qwen35 -> qwen36 -> qwen35 history through
15
+ # v0.6.0.
 
 
 
 
 
 
 
 
 
16
  #
17
  # For other quants (Q3_K_S, Q5_K_M, Q6_K, etc.), `make build QUANT=Q3_K_S`
18
  # downloads the chosen quant from unsloth/Qwen3.6-27B-GGUF and patches
README.md CHANGED
@@ -176,65 +176,43 @@ If you want the safetensors for `transformers`, fetch them from [`Qwen/Qwen3.6-2
176
  current loader compatibility.
177
  - Multi-token prediction (MTP) head trained for speculative decoding
178
 
179
- **The bundled GGUF declares `general.architecture: 'qwen36'`** — the
180
- architecturally-honest stamp. Upstream `ggml-org/llama.cpp` and
181
- `ollama/ollama` currently only register the hybrid SSM + attention
182
- stack under `qwen35` / `qwen35moe`; no `qwen36` arch entry exists yet
183
- (reconfirmed 2026-05-19 against llama.cpp 389ff61 and Ollama 0.24.0).
184
- Neither repo has an open PR or tracking issue for a `qwen36` arch
185
- string, and the actual Qwen 3.6-family upstream weights already load
186
- under `qwen35moe` — so the bundle's `qwen36` stamp may stay
187
- ahead-of-upstream indefinitely rather than for a few releases.
188
- Consequence: **the bundle is unloadable on current stock loaders.**
189
- `ollama run hf.co/FoolDev/Thanatos-27B` and `llama-server -m ...` both
190
- fail with `error loading model architecture: unknown model
191
- architecture: 'qwen36'` (Ollama 0.24 surfaces it as a 500 wrapping a
192
- generic `unable to load model: <blob>` — check `journalctl --user -u
193
- ollama` for the underlying line).
194
-
195
- This is intentional. The bundle is the *correct* metadata; the
196
- loaders are the lagging side. The flip is reversible until upstream
197
- catches up.
198
-
199
- Two one-liners, depending on whether you've already typed
200
- `ollama run hf.co/FoolDev/Thanatos-27B` and gotten the 500:
201
-
202
- ```bash
203
- # Fresh: build a local `thanatos-27b` tag from this repo's bundle
204
- # (LFS smudge + rebadge + `ollama create`, idempotent):
205
- make load-bundle
206
- ollama run thanatos-27b
207
-
208
- # Already pulled and hit the 500: rebadge the already-downloaded
209
- # blob in place + rewrite the manifest digest, so the same
210
- # `hf.co/...` tag becomes loadable without switching tag names:
211
- make heal-hf
212
- ollama run hf.co/FoolDev/Thanatos-27B
213
- ```
214
-
215
- What that wraps, if you want to do it by hand
216
- (`scripts/rename_arch.py` is metadata-only — tensors stay
217
- byte-identical):
218
 
219
  ```bash
220
  python3 scripts/rename_arch.py \
221
- --from-arch qwen36 --to-arch qwen35 \
222
  Thanatos-27B.Q4_K_M.gguf \
223
- Thanatos-27B.Q4_K_M.qwen35.gguf
224
- # Then either build a local Ollama tag (note absolute path —
225
- # `ollama create` resolves a relative FROM against the Modelfile's
226
- # directory, not your CWD):
227
- echo "FROM $PWD/Thanatos-27B.Q4_K_M.qwen35.gguf" > /tmp/Modelfile.qwen35
228
- ollama create thanatos-27b -f /tmp/Modelfile.qwen35
229
- ollama run thanatos-27b hi
230
- # …or point llama-server at the qwen35 file directly:
231
- llama-server -m Thanatos-27B.Q4_K_M.qwen35.gguf -ngl 99 -c 8192
232
  ```
233
 
234
- If upstream ever adds a `qwen36` arch entry — patch landed in
235
- `ggml-org/llama.cpp` and propagated into Ollama — `make load-bundle`,
236
- `make heal-hf`, and the whole workaround can be deleted; the bundle
237
- works as-is.
238
 
239
  ## Quick start
240
 
@@ -243,17 +221,13 @@ works as-is.
243
  Three paths:
244
 
245
  ```bash
246
- # A. Pull straight from HF (uses the bundled Q4_K_M + root-level
247
- # template / system / params files). Fails today with
248
- # `unknown model architecture: 'qwen36'`. If you've already run
249
- # this and got the 500, `make heal-hf` rebadges the pulled blob
250
- # in place so the same command then loads. Or use B / C below
251
- # to avoid the broken pull entirely.
252
- ollama run hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M, qwen36-stamped
253
- make heal-hf && ollama run hf.co/FoolDev/Thanatos-27B # heal + re-run after the first fails
254
-
255
- # B. Load THIS repo's bundle via the rebadge helper (smudges LFS if
256
- # needed, rebadges qwen36 → qwen35, runs `ollama create`):
257
  make load-bundle # creates local tag thanatos-27b
258
  ollama run thanatos-27b
259
 
 
176
  current loader compatibility.
177
  - Multi-token prediction (MTP) head trained for speculative decoding
178
 
179
+ **The bundled GGUF declares `general.architecture: 'qwen35'`** — the
180
+ arch entry every released llama.cpp / Ollama loads the Qwen 3.6
181
+ hybrid SSM + attention family under. Upstream `ggml-org/llama.cpp`
182
+ and `ollama/ollama` register the stack as `qwen35` / `qwen35moe`,
183
+ including for actual Qwen 3.6-family weights (the upstream
184
+ `Qwen3.6-35B-A3B_Q8_0` GGUFs load under `qwen35moe`); no `qwen36`
185
+ arch entry exists, and there's no open PR or tracking issue for
186
+ one — `qwen35` is the canonical load-time label.
187
+
188
+ `ollama run hf.co/FoolDev/Thanatos-27B` and `llama-server -m
189
+ Thanatos-27B.Q4_K_M.gguf` both load directly on current stock
190
+ loaders.
191
+
192
+ **History.** Through v0.6.0 the bundle was stamped `qwen36` — the
193
+ architecturally-honest label, but unloadable on stock loaders.
194
+ Users needed `make load-bundle` or `make heal-hf` to rebadge
195
+ before loading. The stamp was reverted to `qwen35` in commit
196
+ `964e418` to stop forcing the workaround on every fresh pull;
197
+ tensor data was byte-identical across both stamps, only the
198
+ `general.architecture` KV (and namespaced KV keys) flipped. See
199
+ the [CHANGELOG](CHANGELOG.md) entry for the full rationale.
200
+
201
+ `scripts/rename_arch.py` is kept in the repo as a generic GGUF
202
+ arch renamer (metadata only, tensors byte-identical) for future
203
+ flips:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
204
 
205
  ```bash
206
  python3 scripts/rename_arch.py \
207
+ --from-arch qwen35 --to-arch qwen36 \
208
  Thanatos-27B.Q4_K_M.gguf \
209
+ Thanatos-27B.Q4_K_M.qwen36.gguf
 
 
 
 
 
 
 
 
210
  ```
211
 
212
+ If you pulled the bundle while it was still qwen36-stamped (v0.6.0
213
+ through 2026-05-19) and have the broken blob in your local Ollama
214
+ store, `make heal-hf` rebadges it in place so the same
215
+ `hf.co/...` tag becomes loadable without re-pulling.
216
 
217
  ## Quick start
218
 
 
221
  Three paths:
222
 
223
  ```bash
224
+ # A. Pull straight from HF (gets the bundled Q4_K_M GGUF + the
225
+ # root-level template / system / params files in one step):
226
+ ollama run hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M, qwen35-stamped
227
+
228
+ # B. Build a local `thanatos-27b` tag from THIS repo's bundle
229
+ # (LFS smudge if needed, then `ollama create`). Useful if you
230
+ # want a bare local tag rather than the `hf.co/...` path:
 
 
 
 
231
  make load-bundle # creates local tag thanatos-27b
232
  ollama run thanatos-27b
233
 
examples/README.md CHANGED
@@ -21,37 +21,33 @@ from the `Modelfile` / bridge files.
21
 
22
  Pull straight from HF (gets the bundled Q4_K_M GGUF + this repo's
23
  root-level `template` / `system` / `params` files via HF's Ollama
24
- bridge), then heal the bundle's `qwen36` arch stamp so Ollama can
25
- actually load it. The pull itself succeeds; the first inference
26
- fails with `unable to load model` until heal — see parent README's
27
- [Architecture](../README.md#architecture) for why:
28
 
29
  ```bash
30
  ollama pull hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M (only bundled quant)
31
- cd .. && make heal-hf && cd examples # rebadges the pulled blob qwen36 -> qwen35 in store
32
  pip install requests
33
  MODEL=hf.co/FoolDev/Thanatos-27B python ollama_chat.py
34
  ```
35
 
36
- For the smaller-footprint Q3_K_S (~12 GB) or other quants, build
37
- locally instead — see the parent repo's `make build QUANT=...` flow.
 
 
38
 
39
- Or build locally from this repo without going through the HF pull
40
- (uses the bundled `Thanatos-27B.Q4_K_M.gguf` via `make load-bundle`,
41
- which handles the qwen36 → qwen35 rebadge in one shot and creates a
42
- local `thanatos-27b` tag):
43
 
44
  ```bash
45
- cd .. && make load-bundle && cd examples
46
  MODEL=thanatos-27b python ollama_chat.py
47
  ```
48
 
49
- For a non-bundled quant (e.g. Q5_K_M, Q3_K_S), `make build QUANT=...`
50
- downloads a qwen35-stamped GGUF from `unsloth/Qwen3.6-27B-GGUF` and
51
- creates the same local tag — bypasses the bundle and the heal entirely:
52
 
53
  ```bash
54
- cd .. && make build QUANT=Q5_K_M && cd examples
55
  MODEL=thanatos-27b python ollama_chat.py
56
  ```
57
 
 
21
 
22
  Pull straight from HF (gets the bundled Q4_K_M GGUF + this repo's
23
  root-level `template` / `system` / `params` files via HF's Ollama
24
+ bridge):
 
 
 
25
 
26
  ```bash
27
  ollama pull hf.co/FoolDev/Thanatos-27B # 17 GB Q4_K_M (only bundled quant)
 
28
  pip install requests
29
  MODEL=hf.co/FoolDev/Thanatos-27B python ollama_chat.py
30
  ```
31
 
32
+ If you pulled before commit `964e418` (the qwen35 re-stamp) and
33
+ still have the broken qwen36 blob in your Ollama store, run
34
+ `cd .. && make heal-hf` once to rebadge it in place. Fresh pulls
35
+ after the re-stamp go straight through.
36
 
37
+ For a non-bundled quant (e.g. Q3_K_S ~12 GB, Q5_K_M ~20 GB),
38
+ `make build QUANT=...` downloads from `unsloth/Qwen3.6-27B-GGUF`
39
+ and creates a local `thanatos-27b` tag:
 
40
 
41
  ```bash
42
+ cd .. && make build QUANT=Q3_K_S && cd examples
43
  MODEL=thanatos-27b python ollama_chat.py
44
  ```
45
 
46
+ Or build a local tag from this repo's bundled GGUF without going
47
+ through the HF pull:
 
48
 
49
  ```bash
50
+ cd .. && make load-bundle && cd examples
51
  MODEL=thanatos-27b python ollama_chat.py
52
  ```
53