pizdecM2

pizdecM2 is an experimental micro-model (~520 parameters) built to evaluate character-level arithmetic representations, custom Llama-compatible GGUF metadata mapping, and PyTorch Safetensors export pipelines.

Despite its tiny size, the architecture achieves 100% accuracy on its specialized target domain.

---

⚠️ Important Note & Model Scope

CRITICAL: This model is designed strictly for a single specific task and WILL NOT understand regular text, multi-digit operations, or non-addition prompts.

• Supported Input Format: Exactly 4-character prompts formatted as x+y= where $x$ and $y$ are single digits such that $x + y < 10$.
  • Valid examples: 2+3=, 0+0=, 1+8=, 4+5=
  • Unsupported inputs: 5+5=, 10+2=, 2*3=, hello
• Context Length: Fixed at 4 tokens.
• Output: Predicts a single character token representing the result ($0 \dots 9$).

---

Model Metrics & Specs

• Model Name: pizdecM2
• Total Parameters: 520 (100% Trainable)
• Accuracy: 100.00% (55/55 evaluated test samples)
• Vocabulary Size: 13 tokens (0-9, +, =, \n)
• Embedding Size: 10
• FFN Hidden Dimension: 14
• Available Formats:
  • model.safetensors (~2.52 KB)
  • model.gguf (~2.97 KB)
  • tokenizer.json (~0.13 KB)

---

Architecture Breakdown

Layer	Shape	Param Count
embed_tokens.weight	[13, 10]	130
pos_emb	[4, 10]	40
ffn.weight	[14, 10]	140
ffn.bias	[14]	14
lm_head.weight	[13, 14]	182
lm_head.bias	[13]	13
pad	[1]	1
Total		520

---

How to Run Inference

import json
import torch
import torch.nn as nn
from safetensors.torch import load_file

# Define architecture
class LlamaCompatible500P(nn.Module):
    def __init__(self):
        super().__init__()
        self.embed_tokens = nn.Embedding(13, 10)
        self.pos_emb = nn.Parameter(torch.randn(4, 10) * 0.1)
        self.ffn = nn.Linear(10, 14)
        self.act = nn.SiLU()
        self.lm_head = nn.Linear(14, 13)
        self.pad = nn.Parameter(torch.zeros(1))

    def forward(self, x):
        B, T = x.size()
        h = self.embed_tokens(x) + self.pos_emb[:T, :]
        h_flat = h.mean(dim=1)
        feat = self.act(self.ffn(h_flat))
        return self.lm_head(feat)

# Load Tokenizer & Model Weights
with open("tokenizer.json", "r", encoding="utf-8") as f:
    vocab = json.load(f)["vocab"]

char_to_id = {ch: i for i, ch in enumerate(vocab)}
id_to_char = {i: ch for i, ch in enumerate(vocab)}

model = LlamaCompatible500P()
model.load_state_dict(load_file("model.safetensors"))
model.eval()

# Run Prediction
prompt = "3+5="
tokens = torch.tensor([[char_to_id[c] for c in prompt]])

with torch.no_grad():
    logits = model(tokens)
    pred_id = torch.argmax(logits, dim=-1).item()
    print(f"Input: {prompt} | Output: {id_to_char[pred_id]}")

License

MIT License