Rootformer Stage 15.2 (rootformers_stage_v15.2) — Bāb al-Naẓm wa-Ḥidhāʾ al-Alfāẓ (باب النظم وحذاء الألفاظ)

Rootformer Stage 15.2 is a self-contained, BPE-Free Arabic-to-English scholarly translation and morphological-syntactic intelligence engine grounded in the non-concatenative morphological theory of al-Khalīl ibn Aḥmad al-Farāhīdī, the 16-state syntactic governance (ʿAmal) of Sībawayh, and ʿAbd al-Qāhir al-Jurjānī's theory of al-Naẓm (Dalāʾil al-Iʿjāz).

This repository is a 100% turnkey standalone deployment package: clone or download it onto any Linux GPU or CPU server and run two commands (./install.sh && ./run_standalone_server.sh) to launch the full Widescreen Desktop & Mobile Sovereign Workbench UI, REST JSON API, CLI, and Python SDK.

---

What Can Rootformer Stage 15.2 Do?

1. Asymmetric 64-Layer Maʿnā Prefill + Fast Hub-Guided Lafẓ BPE Decode (Stage 15.2)

In al-Jurjānī's Dalāʾil al-Iʿjāz, deep relational deduction (ترتيب المعاني في النفس — Tartīb al-Maʿānī fī al-Nafs) occurs when comprehending the Arabic clause and its syntactic dependencies (Taʿalluq), whereas uttering the target words (حذاء الألفاظ — Ḥidhāʾ al-Alfāẓ) follows the already-constructed semantic blueprint:

• During Arabic Prefill (seq_len > 16): Executes all 64 layers (100% of Qwen3.8-27B) together with the 12-Slot Stage 13.2 Books Mastery Bridge (scale = 0.65) + 6-Pillar GSCA MoE + Sabk Anaphora so the KV/recurrent cache holds the exact 64-layer semantic state $Z_{\text{maʿnā}}$.
• During English BPE Decode & Speculative Verification (seq_len <= 16): Elides $K \in {4, 8, 12}$ upper linear-attention blocks and replaces each with a lightweight 64-rank HidhaAlAlfazResidualAdapter (120 MB total across 24 layers) conditioned on a cached Semantic Hub Summary ($z_8, z_{12}$) from Layers 31 and 47.
• Intact English BPE Vocabulary (SibawayhSlicedLMHead): Preserves 100% of the active English & Arabic tokens (148,891 tokens, 0.000000 logit divergence) while pruning 99,429 unused CJK tokens (-1.95 GB VRAM saved).
• 100% Pure Original Arabic Script (الحروف الأصلية العربية): Technical parenthetical anchors are always emitted in original Arabic script ((الصانع), (حادث), (ترجيح بلا مرجح), (تمهيد), (ضرورة), (قياس)), never Latin transliterations.
• Bundled Native Triton causal_conv1d Kernel: Includes a zero-compile Triton causal_conv1d.py drop-in module so Qwen3.8-27B's GatedDeltaNet layers run at full hardware speed without C++ wheel compilation errors or slow fallback warnings.

2. Measured Benchmark Performance (Qwen3.8-27B 4-Bit NF4 on NVIDIA A40)

Decode Mode	Active Decode Depth	Mean Speed (tok/s)	Peak Speed (tok/s)	Word Throughput (words/s)	Total Time (10 Gold Passages)	Wall-Clock Speedup	Classical Gold chrF++	Classical Gold BLEU	Modern MSA-12 chrF++
Pure Autoregressive Baseline (K=0)	64L / 64L	11.03 tok/s	11.24 tok/s	8.59 words/s	124.03s	1.00x	60.102	32.52	92.090
Stage 15.2 Ultra-Fidelity (--ultra-fidelity, K=4)	64L Prefill / 60L Decode	28.82 tok/s	52.87 tok/s	22.49 words/s	50.02s	2.48x	60.599	33.67	92.090
Stage 15.2 Balanced Default (--27b, K=8)	64L Prefill / 56L Decode	30.36 tok/s	38.61 tok/s	23.53 words/s	43.77s	2.83x	59.734	32.86	92.090 (81.85 BLEU)
Stage 15.2 Turbo Mode (--turbo, K=12)	64L Prefill / 52L Decode	34.57 tok/s	49.50 tok/s	26.90 words/s	37.32s	3.32x	58.822	33.39	92.090

3. Core Capabilities Across Classical & Modern Arabic

1. BPE-Free Non-Concatenative Morphological Factorization ($\phi(w_t) = (r_t, w_t, p_t, s_t)$):
   • Deterministically decomposes Arabic surface words into 10,899 trilateral/quadrilateral roots (Judhūr), 160 morphological templates (Awzān), proclitics, and enclitics via the 329,695-entry Farāhīdī Lookup Table (O(1) latency) plus algorithmic fallback.
2. 100% Ṣarfī Active vs. Passive Participle Disambiguation (25/25):
   • Distinguishes unvocalized homographs such as محدث (muḥdith [Originator] vs. muḥdath [originated]), مؤثر (muʾaththir [Effective Cause] vs. muʾaththar [effected]), and مرجح (murajjiḥ [Determiner] vs. murajjaḥ [preponderated]) via Sībawayhian governance and theological constraints (100.0% accuracy).
3. 16-State Sībawayhian Syntactic Governance ($B_{\text{Sībawayh}}$) & Passive Voice (al-Mabnī lil-Majhūl):
   • Tracks Ibtidāʾ, Khabar, Fāʿil, Mafʿūl bihi, Nāʾib al-Fāʿil, Iḍāfah, and ʿAṭf transitions across clauses (100.0% passive voice accuracy).
4. 6-Pillar Classical Mastery & Zero Modern Polysemy Drift (0.00% Trap Leakage):
   • Separates Classical Kalām, Uṣūl al-Fiqh, Mālikī Fiqh, Taṣawwuf (Sīdī Aḥmad Zarrūq), Manṭiq, and Balāghah from Modern Standard Arabic (92.09 chrF++ on MSA-12; 0.00% modern drift on classical polysemous terms like العالم, الدور, الشارع, الفاعل, الحال, العين, القدر, المحل).
5. Cross-Boundary Pronominal Anaphora (Sabk — 100% on 8/8 Multi-Clause Chains):
   • Tracks enclitic pronouns (ـه, ـها, ـهم) back to their exact antecedents across long multi-sentence passages.

---

Quickstart (2 Commands on Any GPU or CPU Server)

git clone https://huggingface.co/enver/rootformers_stage_v15.2
cd rootformers_stage_v15.2
chmod +x install.sh run_standalone_server.sh
./install.sh
./run_standalone_server.sh

Or download with huggingface_hub (automatically tracked by Hugging Face download metrics):

from huggingface_hub import snapshot_download

local_dir = snapshot_download(repo_id="enver/rootformers_stage_v15.2")
print("Downloaded standalone release to:", local_dir)

Then open http://localhost:17800 in your desktop or mobile browser.

Server Launch Modes (./run_standalone_server.sh)

Command	Hardware / Backbone Loaded	RAM / VRAM	Description
./run_standalone_server.sh	Auto (27B 4-Bit K=8 Naẓm on GPU / 1.7B on CPU)	~14.2 GB VRAM (27B) or ~4.5 GB RAM (CPU)	Automatically loads Stage 15.2 Qwen3.8-27B 4-Bit NF4 + Asymmetric Naẓm (K=8, 30.36 tok/s, 2.83x speedup) on CUDA GPUs, or Qwen3-1.7B Sovereign (3.3 GB) + BPE-Free Bridge on CPU.
./run_standalone_server.sh --ultra-fidelity	Qwen3.8-27B 4-Bit NF4 (K=4 Naẓm, 64L→60L)	~14.2 GB VRAM	Highest Classical Gold fidelity (60.599 chrF++ / 33.67 BLEU) at 28.82 tok/s mean (52.87 tok/s peak, 2.48x speedup).
./run_standalone_server.sh --27b	Qwen3.8-27B 4-Bit NF4 (K=8 Naẓm, 64L→56L)	~14.2 GB VRAM	Balanced default: runs all 64 layers + 12-Slot Bridge on Arabic prefill and 56 layers + Ḥidhāʾ al-Alfāẓ Residual Adapters on English BPE decode (30.36 tok/s, 23.53 words/sec, 59.734 chrF++, 92.09 chrF++ on MSA-12).
./run_standalone_server.sh --turbo	Qwen3.8-27B 4-Bit NF4 (K=12 Naẓm, 64L→52L)	~14.2 GB VRAM	Turbo mode: 34.57 tok/s mean (49.50 tok/s peak, 26.90 words/sec, 33.39 BLEU, 3.32x speedup).
./run_standalone_server.sh --8gb	Strict 8GB VRAM Mode (27B 4-Bit)	< 7.2 GB Peak VRAM	Runs the 27B model on 8 GB consumer GPUs (RTX 3070/4060) using Dual-Stream Pinned-Memory Asynchronous Layer Prefetch (Taqdīm al-ʿĀmil).
./run_standalone_server.sh --1.7b	Force Qwen3-1.7B Sovereign (GPU or CPU)	~4.0 GB VRAM/RAM	Loads the bundled 3.3 GB 1.7B safetensors + 12-Layer BPE-Free Foundation + Tri-Layer Bridge.
./run_standalone_server.sh --cpu	Force CPU Full Neural (1.7B)	~4.5 GB System RAM	Runs full neural translation + 6-tab workbench 100% on CPU (zero GPU required).
./run_standalone_server.sh --fast-cpu	Instant Light CPU (<5 ms)	< 800 MB RAM	Runs the 12-Layer BPE-Free Foundation Model + Khalīlian Root/Wazn Analyzer + Sībawayh 16-State Syntax + Ṣarfī Disambiguator in milliseconds on any laptop CPU.

---

Python SDK & REST API Usage

1. Python SDK (rootformer_sdk.py)

from rootformer_sdk import RootformerStage12

# Mode A: Instant BPE-Free Analysis + 12-Layer Foundation Encoding (CPU or GPU, <5 ms)
rf = RootformerStage12(device="cpu", load_foundation=True, load_qwen=False)
analysis = rf.analyze(
    "العالم حادث وكل حادث فلا بد له من محدث ومؤثر تام بالذات يرجح وجوده على عدمه، لأن الترجيح بلا مرجح محال في العقل."
)
print("Rational Triad:", analysis["rational_triad"])
print("Sarfi Homographs:", analysis["sarfi_homographs"])

# Mode B: End-to-End Scholarly Translation
rf_translator = RootformerStage12(device="cuda", load_foundation=True, load_qwen=True)
result = rf_translator.translate(
    "ندعي أن محدث العالم قادر، لأن العالم فعل محكم مرتب متقن منظوم مشتمل على أنواع من العجائب والآيات"
)
print("English Translation:\n", result["translation"])

2. HTTP JSON API (POST /api/translate)

When ./run_standalone_server.sh is running on port 17800:

curl -s http://localhost:17800/api/translate \
  -H "Content-Type: application/json" \
  -d '{
    "text": "العالم حادث وكل حادث فلا بد له من محدث ومؤثر تام بالذات يرجح وجوده على عدمه، لأن الترجيح بلا مرجح محال في العقل.",
    "mode": "translate"
  }' | jq .

---

Repository Contents & Checkpoints

rootformers_stage_v15.2/
├── config.json                        # Root Hugging Face model configuration (enables Hub download tracking)
├── generation_config.json             # Root generation configuration
├── tokenizer_config.json              # Root tokenizer configuration
├── tokenizer.json                     # Root tokenizer vocabulary & merges
├── chat_template.jinja                # Root chat template
├── model.safetensors                  # Root Qwen3-1.7B Sovereign safetensors (3.3 GB)
├── install.sh                         # One-click dependency & checkpoint verifier (27B GPU & 1.7B CPU/GPU)
├── run_standalone_server.sh           # One-click server launcher (--ultra-fidelity, --27b, --turbo, --8gb, --1.7b, --cpu, --fast-cpu)
├── server_stage15_2.py                # Stage 15.2 unified 27B 4-bit GPU & 1.7B CPU/GPU HTTP server & UI host
├── causal_conv1d.py                   # Native Triton causal_conv1d kernel (zero C++ wheel compilation needed)
├── rootformer_sdk.py                  # Self-contained Python SDK, CLI & Fast-CPU server
├── requirements.txt                   # Minimal Python dependencies
├── README.md                          # Hugging Face Model Card & Quickstart Guide
├── aynengine_web_ui_11/               # Responsive 3-Column Desktop IDE & Mobile Web UI (6 Interactive Tabs)
├── checkpoints/                       # All safetensors & bridge weights included
│   ├── qwen3_8_27b_4bit/              # Qwen3.8-27B 4-Bit NF4 safetensors + tokenizer (21 GB, for GPU)
│   ├── stage15_2/                     # Stage 15.2 Asymmetric Naẓm 24-Layer Residual Adapters (120 MB)
│   ├── stage14_5/                     # Stage 14.5 Learned Neural BPE-Free-to-BPE Drafter (50 MB)
│   ├── stage13_2/                     # Stage 13.2 Books Mastery 12-Slot Bridge + GSCA MoE (142 MB)
│   ├── stage7_3_mujtahid_sovereign/   # Qwen3-1.7B Sovereign safetensors + tokenizer (3.3 GB, for CPU/GPU)
│   └── stage12_bpe_free_foundation_best.pt # 12-Layer (101.55M) BPE-Free Foundation Model (388 MB)
├── data/                              # 329K Farāhīdī NMFE LUT, 2,469 Lexicon, Qur'an & Consensus Graph
├── engine/                            # Deterministic Khalīlian, Sībawayhian, Ṣarfī & Sabk engines
├── scripts/                           # Self-contained Stage 11–15.2 architecture & evaluation modules
└── stage15_2/                         # Stage 15.2 evaluation suites, training scripts & benchmark logs