suii0x/mac-dlp-grokking-test

Modular Arithmetic Challenge submission

This directory is a minimal HuggingFace submission artifact for the Modular Arithmetic Challenge. It is intentionally separate from the development repository so only the files required by the evaluator need to be uploaded.

Files

• manifest.json: evaluator manifest. This is required by modchallenge.
• model.py: entry class declared by manifest.json.
• weights.pt: trained PyTorch checkpoint loaded by model.py.
• .gitattributes: Git LFS rules for model weights.
• .gitignore: local Python cache ignores for the standalone HF repo.

The official evaluator expects manifest.json, not config.json. A HuggingFace model card (README.md) is useful but not part of the evaluator contract.

Local checks

Run these commands from the development repository root:

.\.venv\Scripts\modchallenge.exe check hf_submission
.\.venv\Scripts\modchallenge.exe evaluate hf_submission --total 1100 --timeout 300

Recorded public-benchmark baseline for this checkpoint:

• Tier 1: 100/100
• highest_tier_above_90: 1
• overall_accuracy: about 0.127 on the fixed public benchmark seed

For exact reproduction of the recorded public benchmark result:

.\.venv\Scripts\modchallenge.exe evaluate hf_submission `
  --total 1100 `
  --seed 6d6f646368616c6c656e67652d7075626c69632d62656e63686d61726b2d7631 `
  --timeout 300

HF upload shape

Create the HuggingFace repository from this directory, not from the development repository root:

cd hf_submission
git init
git lfs install
git add .gitattributes .gitignore README.md manifest.json model.py weights.pt
git commit -m "Initial modular arithmetic submission"

After pushing to HuggingFace, use the resulting 40-character commit SHA as the revision for local evaluate-hf testing and official submission.

Note: the development repository ignores *.pt files by default. If you check status from the parent repository root, hf_submission/weights.pt may not appear even though it exists. Initializing and pushing from inside hf_submission avoids that parent ignore rule.