MathCore

A ~40M-parameter neural network that learns integer addition and subtraction from digit tokens. It uses bidirectional attention, Abacus place embeddings, and weight-shared recurrence with parallel (non-autoregressive) answer decoding.

Spec	Value
Parameters	39.8M
Architecture	d=640, 10 heads, 2 prelude + 4×R core + 2 coda
Training range	up to 15-digit operands
Weights	Hugging Face

Benchmark

Evaluated on 10,000 random add/sub problems per operand length (CUDA, mathcore_ckpt.pt).

Digits	Accuracy (%)	Correct	Total	Failed	Time (s)
3	100.00	10000	10000	0	109.81
6	99.99	9999	10000	1	109.55
9	99.84	9984	10000	16	109.76
12	99.32	9932	10000	68	110.40
15	98.45	9845	10000	155	109.60
18	96.97	9697	10000	303	109.56
21	97.03	9703	10000	297	109.69
24	93.65	9365	10000	635	109.64
27	68.24	6824	10000	3176	111.36

Accuracy stays near-perfect in-distribution (≤15 digits) and degrades on longer out-of-distribution operands, with a sharp drop beyond ~24 digits.

Setup

git clone https://github.com/notabdo/mathcore.git
cd mathgo
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt

Download weights from Hugging Face

pip install huggingface-hub

# PyTorch checkpoint (~152 MB)
huggingface-cli download not-abdo/mathcore mathcore_ckpt.pt --local-dir .

# Optional: pre-exported ONNX (~153 MB, for CPU-only inference)
huggingface-cli download not-abdo/mathcore mathcore.onnx --local-dir .

Or in Python:

from huggingface_hub import hf_hub_download
hf_hub_download(repo_id="not-abdo/mathcore", filename="mathcore_ckpt.pt", local_dir=".")

Usage

Chat (terminal REPL)

python mathcore.py --chat
# Examples: 1+2-3+10+5   999+1000-20+5

HTTP service

python mathcore.py --service --port 8000
curl "http://localhost:8000/solve?expr=123+456"

Backends

Backend	Requires	Command
PyTorch	torch	python mathcore.py --chat --backend torch
ONNX	onnxruntime only	python mathcore.py --chat --backend onnx

Export ONNX yourself (one-time, needs PyTorch):

python mathcore.py --export-onnx

Training

Set MODE at the top of training.py:

MODE = "train"   # "smoke" | "train" | "eval" | "diag" | "chat"
CKPT = "mathcore_ckpt.pt"

python training.py   # GPU recommended (BF16 on CUDA)

Project layout

mathgo/
├── mathcore.py       # Inference: chat, API, ONNX export
├── training.py       # Training, evaluation, diagnostics
├── benchmark.png     # Accuracy plot
├── requirements.txt
└── README.md

Weights (mathcore_ckpt.pt, mathcore.onnx) are not in this repo — download from Hugging Face.

License

CC0 1.0