Osam-2025/quantosimo-maths-marking-v1.1

QuantOsimo Maths Marking Model v1.1

Model Description

A LoRA fine-tuned version of Phi-3.5-mini-instruct for marking GCSE, AS-Level and A-Level Mathematics answers against official mark schemes from AQA, Edexcel and OCR.

Built by QuantOsimo — an AI-powered Maths and Science tutoring platform for UK students KS2 to A-Level.

This is v1.1 — trained without 2023-2024 papers to allow genuine held-out evaluation. See v1 for the full 2018-2025 trained model.

Evaluation Results — Genuine Held-out

Evaluated on 50 questions from 2023-2024 papers not seen during training.

Overall Performance

• Attempted correctly: 43/50 (86%)
• Wrong or unreadable input: 7/50 (14%)

By Exam Board

Board	Score
AQA	82% (14/17)
Edexcel	100% (17/17)
OCR	75% (12/16)

By Level

Level	Score
GCSE	88% (30/34)
AS-Level	100% (10/10)
A-Level	50% (3/6)

Notes

• v1 in-distribution score: 88% vs v1.1 held-out: 86% Only 2% drop — model generalises well to unseen papers
• 5 of 7 wrong answers had garbled PDF input text (unreadable questions — not model errors)
• OCR lower score reflects PDF extraction quality issues
• A-Level based on only 6 questions — too small to be conclusive
• Edexcel and AS-Level: perfect scores across all questions

Training Details

• Base model: microsoft/Phi-3.5-mini-instruct
• Method: LoRA (r=16, alpha=16)
• Epochs: 3
• Training examples: 4,066
• Years: 2018-2022 and 2025 (2023-2024 held out for evaluation)
• Boards: AQA, Edexcel, OCR
• Levels: GCSE (Foundation and Higher), AS-Level, A-Level
• Subject: Mathematics
• Training time: 116 minutes on Kaggle T4 x2

Training Data

Fine-tuned on 4,066 examples extracted from official past examination papers and mark schemes published by AQA, Pearson Edexcel and OCR for GCSE, AS-Level and A-Level Mathematics, covering the years 2018-2022 and 2025.

Training data is not published due to exam board copyright. A sample of 100 examples is available at: uk-gcse-alevel-maths-markschemes-2018-2025

Intended Use

• Automated marking of student answers in EdTech platforms
• Research into automated assessment
• Educational AI development

Limitations

• Only trained on Mathematics
• UK curriculum only (AQA, Edexcel, OCR)
• Some OCR papers have garbled PDF extraction affecting results
• A-Level evaluation based on small sample (6 questions)

How to Use

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model = "microsoft/Phi-3.5-mini-instruct"
model = AutoModelForCausalLM.from_pretrained(
    base_model,
    torch_dtype=torch.float16,
    device_map="auto",
    attn_implementation="eager"
)
model = PeftModel.from_pretrained(
    model,
    "Osam-2025/quantosimo-maths-marking-v1.1"
)
tokenizer = AutoTokenizer.from_pretrained(base_model)

Citation

@misc{osagiede2026quantosimomarking,
  author = {Osagiede, Samuel},
  title  = {QuantOsimo Maths Marking Model v1.1},
  year   = {2026},
  url    = {https://huggingface.co/Osam-2025/quantosimo-maths-marking-v1.1}
}