csdhall/Nanbeige4.3-33B-MLX-8bit

Nanbeige4.3-33B community MLX 8-bit conversion

This is a 8-bit affine MLX conversion, with group size 64, made directly from the official BF16 Nanbeige/Nanbeige4.3-33B checkpoint at c1093c501cc6460427c0ac634db7223d0450c0ee. This card describes its construction, local Apple Silicon usage, and tested limits. The converted payload is approximately 35.21 decimal GB. Upload completion is established by the presence of every file in conversion_manifest.json; this card may appear before the weight transfer finishes.

Usage

The validated software versions are MLX 0.32.4 and mlx-lm 0.32.2 on Apple Silicon. Use the included configuration, tokenizer, and chat template with the weights.

python -m pip install mlx==0.32.4 mlx-lm==0.32.2
python -m mlx_lm.generate --model csdhall/Nanbeige4.3-33B-MLX-8bit --prompt "Explain how a binary search works." --max-tokens 256

This invocation downloads the model on first use. File size is not the complete unified-memory requirement: allow additional memory for the operating system, KV cache, and runtime buffers. Native MLX tokenizer validation passed; the GGUF-only llama.cpp patch is not required here. MLX CUDA operation was not tested.

Construction and validation

This variant was independently quantized from the official BF16 source using affine quantization, 8 bits, and group size 64. It was not converted from a lower-bit quantization or fine-tuned. Original BF16 Safetensors remain available from the publisher.

Recorded Apple Metal checks passed correctness, tokenizer parity, numerical comparisons against native BF16, and retrieval tests at configured capacities of 16,384, 32,768, and 65,536 tokens on a Mac with 128 GiB unified memory. Tests used BF16 KV caches without sliding or rotating caches and reserved output tokens. These are limited validation fixtures, not comprehensive downstream benchmarks or claims that every prompt filled every token.

The source's advertised 524,288-token context metadata is preserved, but 524,288-token operation is unverified. Larger contexts need separate feasibility and quality testing.

Integrity, privacy, and limitations

The SHA-256 and byte size of every model/configuration/tokenizer file are recorded in conversion_manifest.json. Verify downloaded files against it. No private company data or fine-tuning dataset was added by this conversion. That does not establish what the original training data contained. Responses can be inaccurate or biased; this release is not a behavioral safety certification or a publisher-issued release.

Inference can run locally on Apple Silicon. Initial model acquisition contacts Hugging Face, and any hosted inference service has its own privacy terms. No generated code or real tool action was executed during validation.

License and source

The source publisher declares Apache-2.0. See LICENSE and conversion_NOTICE.txt for the license and conversion notice. The source is Nanbeige/Nanbeige4.3-33B.