ProCreations/Qwen3.8-27B-Escha-W2-Vision

Qwen3.8-27B Escha W2 — restored vision

An experimental vision-enabled derivative of EschaLabs/Qwen3.8-27B-Escha-W2. This adds the original Qwen vision encoder and merger/projector to Escha's compressed language model. No training was performed. Synthetic image and tool-call checks passed; broad visual or agent benchmark quality has not been established.

The language shards and optional MTP head are byte-for-byte identical to the pinned Escha export. All 333 restored visual tensors were checked for exact equality with the pinned Qwen/Qwen3.8-27B checkpoint. Source revisions, checksums, and assembly details are in assembly-provenance.json.

Component	Storage and serving
Language projections	Original Escha mixed 2/3-bit format, approximately 2.469 bits per projection weight
Token embeddings and LM head	Original INT8 + scales, kept compact in GPU memory
Vision encoder and merger	Original BF16 tensors; the tested server uses FP16 computation
MTP head	Original FP16, optional native NEXTN speculation

This is not GGUF. Q8_0 and a separate llama.cpp mmproj are not used. The included runtime patch enables Escha's existing INT8 embedding row lookup for the dense model, saving 1,270,405,120 bytes (1.18 GiB) of persistent storage. Every embedding row was tested against the prior FP16 dequantization, with exact equality across 1,271,398,400 values; the MTP model shares the same buffers. No extra quantization of the vision or MTP tensors was needed.

Runtime and installation

Use Linux x86-64, Python 3.12, NVIDIA Ampere or newer, a CUDA 12.8-compatible driver, a C compiler, and Python development headers. WSL2 is an upstream runtime option but was not tested for this derivative. AMD and Apple runtimes, stock Transformers, stock SGLang, llama.cpp, and Ollama cannot load this Escha format through these instructions.

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install huggingface_hub
hf download ProCreations/Qwen3.8-27B-Escha-W2-Vision --local-dir ./escha-vision
python -m pip install -r ./escha-vision/requirements.txt
python ./escha-vision/serve_vision.py

The launcher applies small, hash-checked patches inside this virtual environment. They enable compact dense embeddings and suppress API keys in the runtime configuration log. The launcher only accepts the pinned Escha runtime and supports one GPU (TP=1). Do not install a second SGLang distribution over the vendor wheel. PyTorch 2.9.1+cu128, torchvision 0.24.1+cu128, and Transformers 5.10.2 are pinned. Transformers 5.17.0 failed to register the image processor in this runtime; using an arbitrary newer version is not supported.

The launcher explicitly enables multimodal processing, keeps embeddings/head INT8, uses Triton attention, and defaults to one concurrent request and a 65,536-token context with FP8 E4M3 KV cache. Omitting multimodal enablement can silently produce image-independent answers.

Optional MTP and larger contexts:

# 24 GB or larger: 64K default, FP8 E4M3 KV cache, optional native MTP.
python ./escha-vision/serve_vision.py --mtp

# Tested 128K context profile on the PRO 6000 with a 24 GiB allocation budget.
python ./escha-vision/serve_vision.py --mtp --context 131072

The launcher targets nominal 24 GB GPUs and above. For 16 GB hardware, BetterWright keeps Ornith 1.5 9B as its default to leave more room for agent history. Earlier 16 GiB experiments are retained as diagnostics, not recommendations.

The default GPU allocation budget is the smaller of physical VRAM and 24 GiB. --budget-gib reproduces a lower allocation profile on a larger GPU. It is not a hardware-level memory limit. The final MTP profile reserves 5 GiB outside SGLang's static pool; measured total-device peaks are below. Desktop usage, image resolution, and other processes still affect memory. The context window includes input, image tokens, and generated output; agent harnesses must compact history before exhausting it.

FP8 E4M3 KV cache is enabled by default. This is cache quantization, independent of the language/vision weight formats. This checkpoint has no separately calibrated KV scales; the runtime uses its default scale of 1.0. Image, tool-history retrieval, and follow-up tests passed at 64K and 128K, but that does not establish unchanged reasoning quality across tasks. --kv-cache-dtype auto --context 32768 selects the FP16 cache fallback; its available token capacity differs. DFlash2 was not tested or enabled.

Client usage

OpenAI-compatible base URL: http://127.0.0.1:30000/v1; model ID: escha-qwen38-vision. Send images using standard image_url content blocks (including base64 data URLs). Function calls and tool-response follow-ups were exercised.

For responsive agent requests, the tested request body includes:

{
  "model": "escha-qwen38-vision",
  "temperature": 0,
  "max_tokens": 256,
  "chat_template_kwargs": {
    "enable_thinking": false,
    "reasoning_effort": "medium"
  }
}

Use system, user, assistant, and tool roles. The inherited chat template does not accept developer; a client must map that role to system. Enabling thinking requires consuming reasoning_content as well as content and providing enough output budget.

Validation

All measurements below were made on one RTX PRO 6000 Blackwell Workstation Edition, sm_120, driver 595.84, using reduced SGLang allocation budgets. Peak memory samples include the desktop and other device allocations, sampled every 0.5 seconds. These are not physical RTX 4090 or 16 GB card tests. Original Qwen BF16 weights, served through the same runtime in FP16, also passed the same 16 image cases. This demonstrates smoke-test parity only.

Profile (24 GiB allocation budget)	Context cap	Image/tool checks	Long tool-history prompt	Peak total-device VRAM
Ordinary decoding / FP8 KV	65,536	16/16	61,093; follow-up passed	22.18 GiB
MTP / FP8 KV	65,536	16/16	61,093; follow-up passed	22.51 GiB
MTP / FP8 KV	131,072	16/16	126,629; follow-up passed	22.51 GiB

The suite checks four solid colors, four OCR strings, three left/right shape-color observations, three object counts, and two synthetic checkout totals. Each requires an actual structured function call; it also checks a tool-response follow-up, text arithmetic, and image OCR after a long filler prompt. It does not establish full benchmark parity, robust browser-agent performance, video support, or multi-user throughput.

The optional three-prompt generation timing is measured end-to-end with 512 output tokens, including prefill and HTTP overhead. It is a small local throughput probe, not a cross-GPU performance claim. Full responses, memory peaks, source, fixtures, and failure notes are included under validation/ and reproduction/.

Provenance and licensing

• Language model and MTP: EschaLabs/Qwen3.8-27B-Escha-W2 at 561dae0aeea1efbdafe122f2e385fdeaa99127e7.
• Vision encoder, merger, and processor configs: Qwen/Qwen3.8-27B at 1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0.
• Runtime: EschaLabs/escha-runtime-qwen3dense at e926dbfc443243f891f4758951be0e61348795c5, plus the included dense INT8 embedding loader patch.

Apache-2.0 model licensing and upstream notices are retained in LICENSE and THIRD_PARTY_LICENSES/. Qwen and Escha Labs deserve credit for the pretrained model and compression. ProCreations assembled and tested this derivative; it is not an official Qwen or Escha Labs release. See SHA256SUMS.json for the published file manifest (excluding that manifest itself).