Laya-LoRA Coding Companion ($r=8$) & GWAYA Dual-Process AI

Non-Autoregressive System 1 Code-Quality Gatekeeper, Specialist Router, and Deliberative Co-Pilot Verifier for autonomous neuro-symbolic coding pipelines (ANSE & AutoevolveAI).

📄 Full Manuscript PDF: Download Camera-Ready Paper (12 pages, v3 Peer-Reviewed)
🏛️ Permanent Archival DOI: 10.5281/zenodo.23076256 (Zenodo Open-Access Deposition)
🗄️ Curriculum Dataset Hub: callensxavier/laya-coding-curriculum-78k (78,503 records)
⚙️ Source Code & Invariants: GitHub / AutoevolveAI

1. Architectural Summary & Parameter Accounting

Operating over a 149M-parameter ModernBERT-base encoder adapted via Low-Rank Adaptation (LoRA, rank $r=8, \alpha=16$) on fused query-key-value projections and four multi-task heads, Laya achieves microsecond non-autoregressive decision inference (~45 ms CPU latency).

$\Delta W_{qkv} = \frac{\alpha}{r} (A_{qkv} B_{qkv}), \quad A_{qkv} \in \mathbb{R}^{d \times r}, B_{qkv} \in \mathbb{R}^{r \times 3d}$

Component	Layer / Shape	Trainable Parameters	Description
LoRA Attention Adapter	$22 \text{ layers} \times 2 \times (768 \times 8 + 8 \times 768)$	540,672	Rank $r=8, \alpha=16$ on $W_{qkv}$ projections
NoulHead	$\text{Linear}(768, 1) + \sigma$	769	Binary gate: pass ($1$) or block ($0$)
ChoiceHead	$\text{Linear}(768, 46)$	35,374	46-class specialist agent and tactic router
ScoreHead	$\text{Linear}(768, 1) + \text{ReLU}$	769	Continuous physical energy regression $E(x) \in [0, \infty)$
QualityGateHead	$\text{Linear}(768, 1) + \sigma$	769	AR-H4 Non-Autoregressive token quality filter
Total Trainable	—	578,353	0.388% of 149M backbone ($

2. Empirical Benchmark Verification (50-Case Comprehensive Suite)

Coupled with Qwen3.8-27B in an asymmetric dual-process configuration:

System Metric	Standalone Laya (System 1)	Standalone Qwen3.8-27B (System 2)	Asymmetric Dual-Process (Laya + Qwen3.8)
Gate Accuracy	54.0% (Fast reflex)	76.0% (Standalone)	96.0% (Wilson 95% CI: [86.5%, 98.9%])
Precision	52.1%	85.7%	92.6%
Threat Recall	100.0% (25/25 blocked)	88.0% (22/25 blocked)	100.0% (25/25 blocked, 0 False Negatives)
F1 Score	0.685	0.869	0.962
Fast-Path Escalation	100% locally	0% (always autoregressive)	54.0% reflex resolved in ~45 ms CPU
P50 Latency	45.2 ms (CPU)	104.1 ms (GPU T4)	45.2 ms (Fast-path) / 104.1 ms (Escalated)
Energy Consumption	0.38 Wh / 1k queries	24.50 Wh / 1k queries	11.48 Wh / 1k queries (53.2% energy saved)

Cryptographic Provenance: dual_process_results.json
SHA-256: 1c1af2dc6742a1efe4270eaf6be7a0695cc8e0db1e98a48d23213bcb1f7bcc41

3. GWAYA (Qwen + Laya) Parallel Companion & Verifier

GWAYA integrates Laya's non-autoregressive reflex gate with Qwen deliberative reasoning to serve as a parallel advisor and zero-trust verifier for frontier agent outputs (e.g., Gemini 3.1 Pro and Gemini 3.8 Flash):

from anse.gwaya import GwayaAdvisor, verify_gemini_output, assert_zero_hallucination

# 1. Zero-trust anti-hallucination verification (< 2 ms)
report = verify_gemini_output("""
def solve(x: int) -> int:
    pass  # Incomplete stub
""")
print(report.passed)         # False
print(report.anti_stub_ok)   # False -> Empty stub detected!
print(report.energy_penalty) # 1,000,000.0 (Maximum Pain)

# 2. Dual-process deliberative advisory
advisor = GwayaAdvisor()
advice = advisor.advise("Implement symplectic Verlet integrator in Rust with zero heap allocations")
print(advice.specialist)      # "rust_systems_engineer"
print(advice.recommended_action) # "ALLOW"
print(advice.latency_ms)      # ~45.0 ms

4. Standalone Laya Quickstart

from modeling_laya import LayaCodingCompanion
from transformers import AutoTokenizer

repo_id = "callensxavier/laya-lora-modernbert-r8"
tokenizer = AutoTokenizer.from_pretrained("answerdotai/ModernBERT-base")
model = LayaCodingCompanion.from_pretrained(repo_id)

code = '''
def execute_query(user_input):
    import os
    os.system(f"echo {user_input}")  # Shell injection risk!
'''

inputs = tokenizer(code, return_tensors="pt", max_length=512, truncation=True)
decision = model.predict(inputs["input_ids"], inputs["attention_mask"])

print("Blocked:", decision.blocked) # True
print("Threat class:", decision.choice) # "vulnerable"
print("Energy penalty:", decision.total_energy) # 1,000,000.0

5. Serverless Scale-to-Zero Deployment

Deployable with zero idle cost on Google Cloud Run (--min-instances 0):

# Health check & inference
curl -X POST https://laya-serverless-endpoint/v1/audit/anti-hallucination \
     -H "Content-Type: application/json" \
     -d '{"code": "def test(): pass"}'

6. Citation

@article{callens2026laya,
  title={Laya-LoRA Coding Companion: Asymmetric Dual-Process Test-Time Compute, Full-Scale Curriculum on 10 Structured Coding Datasets, and Serverless Multi-Tier Inference Infrastructure on GCP},
  author={Callens, Xavier},
  journal={Systems for Machine Learning / Compound AI Systems},
  year={2026},
  doi={10.5281/zenodo.23076256},
  url={https://huggingface.co/callensxavier/laya-lora-modernbert-r8}
}

Downloads last month: 11

Model tree for callensxavier/laya-lora-modernbert-r8

Base model

answerdotai/ModernBERT-base

Adapter

(45)

this model