EricAI03/math_token_classification

FOR TESTING PURPOSES TO SEE IF TOKEN CLASSIFICATION CAN BE USED FOR MATCHING MATH EXPRESSIONS TO MATH FIELDS

Math Token Classification (RoBERTa-base)

This model is a fine-tuned version of roberta-base optimized to perform Named Entity Recognition (NER) / Token Classification on mathematical texts. Its primary objective is to detect inline and block LaTeX mathematical expressions within standard English text and classify them into their respective mathematical domains.

Target Domains & Labels

The model uses a BIO (Beginning, Inside, Outside) tagging schema across 4 distinct mathematical disciplines:

• ALGEBRA (e.g., linear equations, polynomials, systems of equations)
• CALCULUS (e.g., integrals, derivatives, limits)
• GEOMETRY (e.g., shapes, coordinate planes, geometric theorems)
• TRIGONOMETRY (e.g., sine, cosine, identity equations)

Label Mapping Indices

{
  "0": "O",
  "1": "B-ALGEBRA", "2": "I-ALGEBRA",
  "3": "B-CALCULUS", "4": "I-CALCULUS",
  "5": "B-GEOMETRY", "6": "I-GEOMETRY",
  "7": "B-TRIGONOMETRY", "8": "I-TRIGONOMETRY"
}

Intended Uses & Limitations

This model is intended for educational tools, math tutoring software, or documentation engines to help parse, analyze, or visually highlight mathematical syntax automatically.

Limitations

• Tokenizer Fragility: While RoBERTa's Byte-Level BPE tokenizer handles rare math symbols effectively, highly non-standard or custom LaTeX macros not present in standard datasets might fragment predictions.
• Context Dependence: The expression classification depends heavily on the context words around the formula in the prompt sentence.

Training Data & Methodology

The model was fine-tuned on 1,553 highly complex mathematical problems extracted from the lighteval/MATH-Hard dataset (specifically Level 5 competition-grade questions).

• Sequence Preprocessing: Regular expressions isolated math spans wrapped in standard delimiters ($...$, $$...$$, \[...\], \(...\)).
• Subword Alignment: Character boundaries were aligned to RoBERTa token vectors via structural sequence index matching (char_to_token).

Training Hyperparameters

• Base Architecture: RoBERTa-base (~125M parameters)
• Learning Rate: 3e-5
• Batch Size: 8
• Epochs: 3
• Optimizer: AdamW with weight decay (0.01)
• Precision: Mixed Precision (FP16) where available

How to Use

You can run inference using the Hugging Face pipeline API:

from transformers import pipeline, AutoTokenizer, AutoModelForTokenClassification

model_id = "EricAI03/math_token_classification"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForTokenClassification.from_pretrained(model_id)

nlp_ner = pipeline("token-classification", model=model, tokenizer=tokenizer, aggregation_strategy="simple")

text = "Find the derivative of \(\int x^2\) dx and then solve the linear equation \(y = 2x + 3\)."
print(nlp_ner(text))