UMoX — Unified Mixture of Executors

UMoX is a quality-gated runtime architecture for making heterogeneous local hardware useful as a coordinated system. A large GPU-resident model remains the authoritative reasoning lane while a smaller NPU-resident model handles bounded utility work concurrently. The CPU runs tools, retrieval, validation, routing, and I/O instead of redundantly recomputing the same transformer layers.

Current status: admitted research prototype. The utility-concurrency lane passed its registered AC admission benchmark. The distributable 0.1.0 release is still being qualified; this repository does not claim that the production release gates or 48-hour soak have completed.

UMoX is a runtime design, not a new set of model weights. This repository does not redistribute Ornith, Qwen, QAIRT, GenieX, or llama.cpp binaries.

What was measured

On one ASUS Zenbook A16 with a Snapdragon X2 Elite Extreme, 48,758 MB usable memory, an Adreno X2-90 GPU, and a Hexagon NPU, UMoX paired:

• GPU: local/ornith-1.0-35b:Q4_K_M as the primary reasoning model;
• NPU: local/qwen3-4b-instruct-2507-x2 as the allowlisted utility model;
• CPU: routing, HTTP/SSE handling, evidence capture, and workload control.

The registered benchmark used 12 primary requests, 21 utility-corpus requests, four alternating candidate/control pairs, 128 maximum output tokens, AC power, resident model services, and an all-GPU control with the same corpus and client concurrency.

Result	Measured value
Median mixed-workload wall-time improvement	17.47%
Individual pair improvements	16.53–18.15%
Primary GPU degradation during UMoX	1.17%
GPU-alone generation median	24.95 tok/s
GPU generation median under UMoX	24.66 tok/s
NPU utility generation median	25.15 tok/s
Utility-corpus median completion latency	197.04 s → 6.48 s
Utility-corpus median latency ratio	30.4× faster
Utility-corpus first-content median	178.86 s → 4.90 s
First-content latency ratio	36.5× faster
Route correctness	84/84 (100%)
Request-interval overlap	99.999% of NPU utility union time
Recorded request errors	0

The 30.4× and 36.5× figures are queue/response-latency improvements under the registered mixed-load experiment. They are not claims that the NPU generates tokens 30× faster. The product-level result is the 17.47% reduction in total mixed-workload wall time.

See BENCHMARKS.md and the sanitized admission summary.

Why this architecture survived

Earlier UMoX research tested three ways to insert the NPU directly into the 35B token path: per-token decode work, prompt prefill, and speculative verification. Each lane was closed when measured boundary/session overhead failed its registered break-even rule. Those negative results changed the architecture rather than being hidden.

The surviving thesis is simpler: do not force the NPU to race the GPU at the GPU's best task. Give both engines independent useful work and overlap it.

                         UMoX loopback API
                                |
                  static, safety-first classifier
                      /                       \
        primary / tools / high risk       bounded utility
                    |                           |
          Ornith 35B on Adreno GPU       Qwen3 4B on Hexagon NPU
                    |                           |
                    +---- CPU tools, MCP, validation ----+
                                      |
                             authoritative result

The route is a pure function of request fields and static configuration. Model output cannot promote itself to another lane. Tools, structured/high-risk actions, credentials, and Genetec-sensitive requests stay on the primary GPU lane. A failed NPU request may fall back only before streaming begins.

What UMoX does not claim

• It does not split one transformer token across CPU, GPU, and NPU in the admitted product path.
• It does not prove that an NPU is universally faster than a GPU.
• It does not turn TOPS into a direct tokens-per-second prediction.
• It does not inherit upstream SWE-bench, LiveCodeBench, reasoning, or agentic scores as UMoX results.
• It does not yet qualify a 70B-class model resident beside both admitted models on this 48 GB system.
• It does not yet constitute a production security certification.

Project documents

• Architecture and safety model
• Benchmark protocol and current evidence
• Release qualification status
• UMoX 0.2 performance plan
• Independent admission-audit record
• How to contribute
• Historical Qwen3.6 research

Source availability

The admitted prototype source and adversarial test harness exist locally, but the canonical distributable package is intentionally withheld until its release-integrity, owned-startup, smoke/fallback, rollback, vocabulary, and soak gates complete. Publishing an unaudited launcher as a production package would contradict the project methodology.

The current accepted release-tooling milestone is Unit 1 rev 5: 69/69 tests, with builder/verifier identity logic accepted. Unit 2 (owned orchestration and live smoke/fallback) is next. See RELEASE_STATUS.md.

A note to contributors

UMoX is young, hardware-specific, and deliberately open about unfinished work. If the idea interests you, please reproduce it, try another model/hardware pair, challenge an assumption, or improve a benchmark. A carefully documented failure is as useful as a speedup. Please be kind to other contributors and severe only with the evidence.