Sparse-AST-BWM-200M-32

Large 204.4M parameter Sparse-AST architecture with 28 layers trained for deep procedural geometry reasoning in Blender Python.

Model Details

• Architecture: Sparse-AST (Recurrent Leaky Gated Transformer)
• Parameters: 204,372,272
• Serialization: SafeTensors (model.safetensors)
• Tied Embeddings: Yes (e.weight tied to h_out.weight, preserved via SafeTensors metadata)
• Vocabulary: 512 (UTF-8 byte-level tokenization)
• Domain: Procedural 3D Mathematics, Blender Python (bpy, mathutils, bmesh, numpy, gpu)

Context Window Specification

• Native Training Context: 32 tokens
• Inference-Time Context Interpolation: Supported up to 4,096 tokens via 1D linear positional interpolation in from_pretrained(..., target_context=4096).

This model was trained natively with a 32-token context window. For longer sequences, positional interpolation is applied at inference time.

Experimentally Measured Benchmarks

Evaluated on the standardized Blender 3D Math & Python Curriculum suite:

Metric	Measured Result
Curriculum Cross-Entropy Loss	4.4454
Perplexity	85.24
Evaluation Latency	7.9s

Verification & Numerical Integrity

This SafeTensors distribution underwent full CPU verification against the original PyTorch checkpoint:

• Checked Tensors: 480
• Tied Weights Handled: 1
• Max Absolute Error: 0.0 (Exact 0.0 bitwise equality)
• Verification Status: PASS

Quick Start Inference

from model import SparseAST
model = SparseAST.from_pretrained('.')
from generate import generate
prompt = 'import mathutils\nfrom mathutils import Vector\nv = Vector(('
output = generate(model, prompt, max_new_tokens=30)
print(output)