sprappcom/oaica-35b-malay-260827

Oaica 35B Malay 260827

Formerly: sprappcom/malay35b-pqm

malay35B — a Malay-focused continued-pretrain + SFT of the Qwen3.6-35B-A3B MoE (Gated DeltaNet hybrid attention), exported to .pqm, a Marlin-packed mixed-precision format for the prism-engine inference server. This is the base variant (no merges).

License & attribution

This repository distributes a composite work under a dual arrangement:

• Proprietary layer — © 2026 BCZ Singapore Pte Ltd. All rights reserved. The .pqm container format and packaging, the Malay continued-pretrain + SFT recipe, and the prism-engine inference server (Rust/CUDA) are proprietary and are not licensed under Apache-2.0.

• Base model weights — Apache-2.0. The underlying model weights are a modified derivative of an Apache-2.0 upstream:

  Component	Source	Copyright	License	Modification
  Backbone + vision tower	Qwen/Qwen3.6-35B-A3B	© 2026 Alibaba Cloud	Apache-2.0	Malay continued-pretrain + SFT (modified); weights repacked to .pqm

  These base weights remain licensed under Apache-2.0. The full Apache-2.0 license text and the per-component attribution are in the NOTICE file in this repo.

This is not an original, trained-from-scratch foundation model — it is a modified derivative of the Apache-2.0 base named above. Use of the base weights is governed by Apache-2.0; use of the proprietary .pqm packaging, the CPT/SFT recipe, and prism-engine requires a separate license from BCZ Singapore Pte Ltd.

Requirements — read before downloading

• Inference engine: .pqm is a proprietary container and is not readable by vLLM, llama.cpp, Ollama, or transformers. It runs only on prism-engine (proprietary Rust/CUDA server, not included in this repo). Contact sprappcom for engine access.
• Tokenizer sidecar: .pqm does not embed a tokenizer. Booting requires a separate .tok file passed via PRISMX_TOKENIZER. Use the malay35b.tok shipped in this repo (the same tokenizer serves all malay35B variants).
• Hardware: A100 (sm_80) or Ada/Ampere (sm_86/sm_89) validated. Full VRAM residency needs ~19 GB weights + batch scratch; with CPU-RAM expert offload it runs on an 8 GB GPU (see below).

Architecture

• Qwen3.6-35B-A3B MoE, hidden_size=2048, 40 layers (30 Gated DeltaNet linear-attention + 10 full attention)
• 256 routed experts + 1 shared expert per MoE layer, top-8 routing
• vocab_size=248077 (243 tokens extended over the stock base during CPT)

Quantization

Mixed-precision .pqm (Marlin-packed, GGUF-free standalone boot):

Tensor group	Type
token embedding, attn_qkv/attn_output/attn_gate, routed experts (ffn_*_exps)	Q4_K
shared expert (ffn_*_shexp), output.weight	Q6_K
norms, SSM gates	F32

Effective ~4.56 bits/weight (BPW). The container is ~39.9 GB because routed-expert bytes are stored raw (--cpu-moe-passthrough) so the same file supports CPU offload.

Usage

export PRISMX_PQM_STANDALONE=1
export PRISMX_PQM=/path/to/malay35b.pqm
export PRISMX_TOKENIZER=/path/to/malay35b.tok
export PRISMX_MIN_TEMPERATURE=0.15      # greedy decoding can collapse on this stack
export PRISMX_STRIP_THINK_OPENAI=1      # hide <think>…</think> reasoning from message.content
prism_server /path/to/malay35b.pqm 0.0.0.0:8080 --max-batch 1 --max-seq 8192

Argument order: the first positional argument is the model path (in standalone mode it only names the served model — the weights come from PRISMX_PQM), the second is the bind address. If you pass the address first it is taken as the model path and the server binds the default 0.0.0.0:8080 regardless.

OpenAI-compatible POST /v1/completions and POST /v1/chat/completions (streaming supported). The model always reasons in a <think> block first (~100–900 tokens depending on the prompt); with PRISMX_STRIP_THINK_OPENAI=1 that reasoning is removed from message.content, so send max_tokens large enough to cover it (>= 1000 for prose/Malay answers, >= 2000 with images — reasoning length varies run to run) or the answer comes back empty with finish_reason: "length".

Small-GPU / CPU-RAM offload

Keep the first N layers' routed experts in host RAM:

prism_server /path/to/malay35b.pqm 0.0.0.0:8080 --n-cpu-moe 38 --moe-cache-experts 512 --max-batch 1 --max-seq 8192

Measured on an RTX 4060 Laptop (8 GB): ~5.8 GB VRAM, ~16 GB host RAM, coherent output. On an A100 80 GB, --n-cpu-moe 36 --moe-cache-experts 2048 --max-batch 1 --max-seq 8192 measures 10.9 GB VRAM + 15.2 GiB page-locked host RAM at ~40 tok/s (this is the production configuration). --moe-cache-experts is a performance dial, not a correctness requirement.

Recommended sampling

Use temperature >= 0.15. Greedy decoding (temperature=0) can trigger repetition/garbage-token collapse on this serving stack.

Verification

Boot-verified GGUF-free on A100 (sm_80) and RTX 4060 (sm_89): coherent English and Malay completions, correct arithmetic reasoning. SHA-256 of malay35b.pqm matches the production-served file.

Known limitations

• Text-only by default; vision requires the separate sprappcom/qwen36-vision-tower bridge (see Vision below).
• Intermittent sub-word token drop/duplication on some code and arithmetic prompts is a known open issue under investigation; strict-JSON output may occasionally be malformed. Validate structured output downstream.

Vision

Image input is available by grafting the Qwen3.6-35B-A3B vision tower onto this model via the prism-engine vision bridge: sprappcom/qwen36-vision-tower (446.6 M params BF16; measured 1.3 GB VRAM for the bridge alone on an A100, 12.5 GB total VRAM for bridge + this model with --n-cpu-moe 36 during an image request; ~7.0 GB total with this model offloaded on an 8 GB 4060).

How to enable: run that repo's server.py (see its README for env vars), then start prism_server with PRISMX_VISION_BRIDGE_SOCK=<the bridge's socket path> added to the environment above and send OpenAI-style image_url content parts (data:image/...;base64,...) to /v1/chat/completions with max_tokens >= 2000. Verified end-to-end 2026-08-26 on an A100 from these exact HF artifacts: a real newspaper photo is described correctly (masthead, slogan, edition line; prompt_tokens: 312), a 4000x3000 solid-colour JPEG is identified correctly after the bridge's 1024 px resize cap (prompt_tokens: 780, no OOM), Malay answers require a Malay system message, text-only requests are unaffected, and streaming works. Full numbers and request shape in the vision-tower README.