SectileLabs/qwen36_35b_a3b_smol

Qwen3.6-35B-A3B, GInfer SMOL 2-bit mix (CPU)

This repo holds the GInfer package of Qwen3.6-35B-A3B in SMOL 2-bit mix: a single-file package for the GInfer inference engine (and GChat, which is built on it). It runs only on GInfer / GChat. It is not a safetensors or GGUF file and other runtimes cannot load it.

This package is for computers without an NVIDIA graphics card: GInfer runs it on the processor.

Package

File	Tensor-parallel size	Size in bytes	SHA-256
qwen36_35b_a3b_smol_autoround_down37.ginfer	1	14,446,362,368	a7af668a8b8f63f37a0951efd944f38777fb30992c65b09abd9da321557be8a9

Format: SMOL 2-bit mix (routed expert gate and up matrices in 2-bit, calibrated with AutoRound; routed down matrices in 4-bit except layers 0, 4 and 6 in 2-bit; every other matrix in 8-bit groupwise integers).

Check the download with sha256sum qwen36_35b_a3b_smol_autoround_down37.ginfer.

Upstream models and licenses

• Base model: Qwen/Qwen3.6-35B-A3B, revision 995ad96eacd98c81ed38be0c5b274b04031597b0. License field apache-2.0; the repo LICENSE file is the Apache License 2.0 (full text in LICENSE here).

Quantized by Sectile Labs to SMOL 2-bit mix from the official BF16 weights of Qwen/Qwen3.6-35B-A3B (revision 995ad96eacd98c81ed38be0c5b274b04031597b0). The routed gate and up weights were calibrated by Sectile Labs' own implementation of the AutoRound method on a mix of general web text, books and source code; the other weights use Sectile Labs' round-to-nearest conversion. The result was converted to a .ginfer package. No third-party quantized weights are used. The package includes the model's Vision tower weights; GInfer does not run Vision on this model yet.

This package is a conversion of the upstream weights into the GInfer format; GInfer does not change the upstream license. The upstream license text is in LICENSE.