🏛️ DeepNarra Hiraya-1-H-MoE 1.0 (0.91B-A100M Ascended H-MoE)

Phase 2 Scaled Architecture — Two-Tier 92 Sub-Experts per Layer (736 Total Experts) + Persistent Shared Foundation Expert

Organization: DeepNarra
Pipeline Stage: Phase 2 — Two-Tier 92-Expert Hierarchical MoE Expansion
Architecture: H-MoE-Zamba2-92-Expert-Nested-Hierarchical (908,645,372 total parameters across 1,331 tensors)
Base Model Upcycled: DeepNarra/Hiraya-1-Base
Evaluation Governance: Zero Fake Scores Policy — 100% Verifiable Engineering Facts Only

---

🏛️ Verified Technical Specifications (Exact Repository Match)

Architectural Metric	Exact Verified Value in This Repository
Official Model Name	DeepNarra Hiraya-1-H-MoE 1.0 (0.91B-A100M)
Exact Safetensors Parameter Count	908,645,372 parameters (3,466.79 MB / 3.46 GB in model.safetensors, 1,331 tensors)
Active Parameters / Token	~100M Active per Token (macro_k = 2, micro_k = 2 -> 4 active sub-experts + Shared Foundation Expert)
Hierarchical Expert Topology	23 Macro-Engines × 4 Micro-Experts = 92 Sub-Experts per layer (736 total sub-experts across 8 layers)
Shared Foundation Expert	Persistent full-rank expert active on 100% of tokens to anchor universal grammar and baseline logic
MoE Load Balancing	Sinkhorn-Knopp Optimal Transport + Aux-Loss-Free Dynamic Bias Balancing
Backbone Dimensions	d_model = 768, num_layers = 8, vocab_size = 48,760 (tokenizer.json included)

---

🧬 DeepNarra Cumulative Training Lineage

1. ✅ DeepNarra/Hiraya-1-Base (118M / 450.12 MB) — Phase 1 Foundation Pre-Training
2. 👉 DeepNarra/Hiraya-1-H-MoE (908.6M / 3.46 GB) — You are here (Phase 2 92-Expert H-MoE Upcycling)
3. ⬆️ DeepNarra/Hiraya-1-Instruct (908.6M / 3.32 GB) — Phase 3 14-Horizon Streaming SFT Super-Agent
4. ⬆️ DeepNarra/Hiraya-1-Reasoning (908.6M / 1.73 GB FP16) — Phase 4 Autonomous SymPy CAS & DeepSeek-R1 GRPO Reasoning

---

🚀 Quickstart Usage (Python)

from transformers import AutoTokenizer
from safetensors.torch import load_file
from huggingface_hub import hf_hub_download

repo_id = "DeepNarra/Hiraya-1-H-MoE"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
weights_path = hf_hub_download(repo_id=repo_id, filename="model.safetensors")
state_dict = load_file(weights_path)
print(f"Loaded {len(state_dict):,} H-MoE tensors | Total Params: {sum(v.numel() for v in state_dict.values()):,}")

---

⚖️ License, Dataset Attributions & Responsible AI Compliance

1. Model License (Apache-2.0)

This repository, including all model weights (.safetensors and .gguf), configuration manifests, and custom architectural code, is licensed under the Apache License, Version 2.0 (http://www.apache.org/licenses/LICENSE-2.0). See the LICENSE and NOTICE files in the root of this repository for full legal terms.

2. Upstream Dataset & Open-Data Attributions (ODC-By-1.0 & Apache-2.0)

In strict compliance with open-data attribution requirements:

• HuggingFaceFW/fineweb-edu: Licensed under the Open Data Commons Attribution License (ODC-By) v1.0.
• CohereForAI/aya_collection: Licensed under the Apache License 2.0.
• m-a-p/CodeFeedback-Filtered-Instruction: Licensed under the Apache License 2.0.
• HuggingFaceH4/no_robots: Attributed under CC-BY-NC-4.0 for instruction-following research evaluation.
• SymPy CAS Procedural Reasoning Stream: Synthetically generated and formally verified via SymPy (BSD-3-Clause).

3. Intended Use, Safety & Limitations

• Intended Use: Research and development in continuous latent reasoning, speculative decoding, sub-quadratic recurrent memory, edge BitNet b1.58 GGUF deployment, and sandboxed autonomous agents.
• Safety & Execution Airlock: Code execution and tool-calling capabilities are designed to run strictly inside the DeepNarra Airlock Sandbox with AST static verification and SSRF network firewalling (169.254.169.254 and RFC-1918 private subnets blocked).
• Limitations & Disclaimer: As an experimental sovereign architecture trained on compact compute budgets, outputs should be independently verified for high-stakes medical, legal, financial, or safety-critical systems. Provided "AS IS" without warranties of any kind under Section 7 of the Apache 2.0 License.