Mark-Experimental-V3-8B

Overview

Mark-Experimental-V3-8B is an 8B-class sparse model packing 32 layers, 32 routed experts, 2 shared experts, and full H-MLA attention into 8,168,001,024 total parameters (2,002,374,144 active per token).

Quantized in IQ4_NL, the binary occupies 4382.55 MiB (4595.44 MB / 4,595,438,848 bytes), fitting within an 8 GB consumer GPU while leaving over 3.4 GB of VRAM for KV cache.

Specifications

Field	Value
Architecture	deepseek2
Weight State	UN-TRAINED_ARCHITECTURAL_BASE
Binary Size	4382.55 MiB (4595.44 MB / 4,595,438,848 bytes)
Total Parameters	8,168,001,024 (8168001024 physical)
Active Parameters	2,002,374,144 (2002374144 active)
Vocabulary Size	32032
Layers	32
Hidden Dimension	2560
Attention Heads	32
MLA Ranks	q_lora_rank=1024, kv_lora_rank=512, qk_rope_head_dim=32
MoE Experts	32 routed, 2 shared (4 active/token)
Expert FFN Dim	896
Context Length	32768 tokens
Tensor Count	579
Quantization Types	{'IQ4_NL': 418, 'F32': 161}
SHA-256	792e127a71f62145f87c3775c4a03a0dc7bb68b8ee355acb74602348cf748b4a

Architecture

Multi-Head Latent Attention (MLA)

The model projects key and value states into a shared low-rank latent representation to reduce KV cache memory:

c_KV = h_t * W_DKV          (dim: 512)
c_Q  = h_t * W_DQ           (dim: 1024)
q_C  = RMSNorm(c_Q * W_UQ)
k_C  = RMSNorm(c_KV * W_UK)

Decoupled rotary position embeddings are concatenated along a separate dimension to track token positions:

k_R  = RoPE(h_t * W_KR)     (dim: 32)
q_t  = [q_C ; q_R]
k_t  = [k_C ; k_R]

KV Cache Memory Usage

Physical memory consumption of the KV cache per token:

KV_bytes_per_token = 32 * (512 + 32) * 2 = 34,816 bytes/token

Context Window	MLA KV Cache	Standard MHA KV Cache	Memory Savings
4,096 tokens	136.00 MiB	1280.00 MiB	9.4x
8,192 tokens	272.00 MiB	2560.00 MiB	9.4x
32,768 tokens	1088.00 MiB	10240.00 MiB	9.4x

Mixture-of-Experts (MoE)

• Shared Experts (2S): Constantly active path for common features: F_shared(x) = (SiLU(x * W_gate) * (x * W_up)) * W_down
• Routed Experts (32R, Top-4): Routed using learned bias offsets without auxiliary load balancing loss: s_{t,e} = Softmax(x_t * W_router + b_e) y = x + F_shared(x) + sum_{i in TopK} g_i * F_routed_i(x)

V3 Architecture Details

1. Hadamard-Transformed MLA (H-MLA): Suppresses latent activation outliers by absorbing the normalized Walsh-Hadamard matrix: H_256 = (1/16) * H_2^(x8), W_fused = H_256 * W_proj directly into key-value projection weights (W_wkv_a and W_wkv_b) during compilation. This absorption eliminates activation spikes without adding runtime execution overhead.

2. Partitioned-Balancing Mixture-of-Experts (PB-MoE): The 32 routed experts are partitioned into 4 functional clans:

   • Clan 0 (Experts 0-7): Systems code, kernel semantics, and concurrency.
   • Clan 1 (Experts 8-15): Mathematics, formal logic, and STEM.
   • Clan 2 (Experts 16-23): Security boundaries, protocols, and validation.
   • Clan 3 (Experts 24-31): Multimodal and spatial reasoning. Routing enforces soft-pinning constraints ensuring balanced domain representation per token.

3. Latent Recursive Depth (LRD) & Tied Embeddings: Enables recurrent intermediate latent steps while enforcing weight-tying between input and output representations to minimize physical disk and VRAM footprints.

Quantization

The model is quantized to GGML_TYPE_IQ4_NL (4-bit non-linear) within the native deepseek2 GGUF container:

• Linear weights are structured in 18-byte blocks for each group of 32 weights:
  • 2 bytes: Half-precision (FP16) scale factor d.
  • 16 bytes: 32 four-bit nibbles referencing the non-linear lookup table kvalues_iq4nl: [-127, -104, -83, -65, -49, -35, -22, -10, 1, 13, 25, 38, 53, 69, 89, 113]
• 1D normalization weights (*_norm.weight) and router bias vectors (exp_probs_b.bias) remain in full precision FP32 to preserve numerical accuracy at layer boundaries.
• Routed MoE weights are serialized as contiguous 3D tensors (blk.{i}.ffn_gate_exps.weight, blk.{i}.ffn_up_exps.weight, blk.{i}.ffn_down_exps.weight), avoiding tensor splitting and kernel overhead.

Training Status

Untrained architectural base (UN-TRAINED_ARCHITECTURAL_BASE). The GGUF file contains initialized weights for testing binary graph compatibility with llama.cpp, measuring memory usage on target hardware, and serving as a base for downstream pre-training and fine-tuning.

All reported metrics represent physical parameter counts and deterministic SHA-256 checksums from the GGUF binary.

Usage

Download

# Using huggingface-cli
huggingface-cli download miceworks/Mark-Experimental-V3-8B mark_experimental_v3_8b_iq4_nl.gguf --local-dir .

# Using Python
from huggingface_hub import hf_hub_download
hf_hub_download(repo_id="miceworks/Mark-Experimental-V3-8B", filename="mark_experimental_v3_8b_iq4_nl.gguf")

Cryptographic Verification

echo "792e127a71f62145f87c3775c4a03a0dc7bb68b8ee355acb74602348cf748b4a  mark_experimental_v3_8b_iq4_nl.gguf" | sha256sum -c -

Ollama

# Modelfile
FROM ./mark_experimental_v3_8b_iq4_nl.gguf
TEMPLATE """{{ if .System }}<|bos|>System: {{ .System }}<|eos|>
{{ end }}{{ if .Prompt }}<|bos|>User: {{ .Prompt }}<|eos|>
{{ end }}<|bos|>Assistant:
<think>
"""

PARAMETER stop "<|eos|>"
PARAMETER temperature 0.3
PARAMETER top_p 0.95
PARAMETER repeat_penalty 1.1

SYSTEM """You are Mark-Experimental-V3-8B, a systems-focused neural model."""

ollama create mark-experimental-v3-8b -f Modelfile
ollama run mark-experimental-v3-8b

llama.cpp

# CLI generation
llama-cli -m mark_experimental_v3_8b_iq4_nl.gguf \
  -p "<|bos|>System: Systems mode active.<|eos|>\n<|bos|>User: Explain Linux memory barriers.<|eos|>\n<|bos|>Assistant:\n<think>\n" \
  -n 512 --temp 0.2

# HTTP server
llama-server -m mark_experimental_v3_8b_iq4_nl.gguf --port 8080 -c 32768

Python Inspection

import gguf

reader = gguf.GGUFReader("mark_experimental_v3_8b_iq4_nl.gguf")
print("Architecture:", reader.get_field("general.architecture"))
print("Total Tensors:", len(reader.tensors))
total_params = sum(int(t.shape.prod()) for t in reader.tensors)
print(f"Total Parameters: {total_params:,}")