Sparse-AST-BWM-100M-32

High-accuracy 103.4M parameter Sparse-AST foundation model for 3D mathematics, vectors, matrices, quaternions, and bmesh operations.

Model Details

• Architecture: Sparse-AST (Recurrent Leaky Gated Transformer)
• Parameters: 103,393,784
• Serialization: SafeTensors (model.safetensors)
• Tied Embeddings: Yes (e.weight tied to h_out.weight, preserved via SafeTensors metadata)
• Vocabulary: 512 (UTF-8 byte-level tokenization)
• Domain: Procedural 3D Mathematics, Blender Python (bpy, mathutils, bmesh, numpy, gpu)

Context Window Specification

• Native Training Context: 32 tokens
• Inference-Time Context Interpolation: Supported up to 4,096 tokens via 1D linear positional interpolation in from_pretrained(..., target_context=4096).

This model was trained natively with a 32-token context window. For longer sequences, positional interpolation is applied at inference time.

Experimentally Measured Benchmarks

Evaluated on the standardized Blender 3D Math & Python Curriculum suite:

Metric	Measured Result
Curriculum Cross-Entropy Loss	3.9078
Perplexity	49.79
Evaluation Latency	4.1s

Verification & Numerical Integrity

This SafeTensors distribution underwent full CPU verification against the original PyTorch checkpoint:

• Checked Tensors: 310
• Tied Weights Handled: 1
• Max Absolute Error: 0.0 (Exact 0.0 bitwise equality)
• Verification Status: PASS

Quick Start Inference

from model import SparseAST
model = SparseAST.from_pretrained('.')
from generate import generate
prompt = 'import mathutils\nfrom mathutils import Vector\nv = Vector(('
output = generate(model, prompt, max_new_tokens=30)
print(output)