007superher/mlx_qwen_3.5_lora_latex

Qwen3.5-2B LaTeX OCR (MLX 4-bit)

MLX-optimized, 4-bit quantized vision-language model for image → LaTeX conversion.
It is the Apple Silicon deployment build of 007superher/unsloth_finetune, a LoRA fine-tune of unsloth/Qwen3.5-2B trained on math formula images. Linear layer weights were quantized to int4 with blockwise quantization to reduce memory use and speed up inference on Mac GPUs via MLX.

Model description

Given an image of a mathematical expression (printed or handwritten), the model generates LaTeX that represents the formula. It was trained to respond to:

Write the LaTeX representation for this image. Outputs are typically returned as raw LaTeX, often wrapped in $$...$$ delimiters.

Intended uses

• Local LaTeX OCR for math images on Apple Silicon (Mac)
• Converting screenshots, photos, or scans of formulas into editable LaTeX
• Serving via mlx-vlm's OpenAI-compatible API

Out-of-scope / limitations

• Not a general-purpose chat or reasoning model
• LaTeX equivalence is ambiguous (spacing, delimiters, macro choices) — different valid renderings exist for the same formula
• Best suited for single-formula images; dense multi-line documents may degrade
• MLX only — this checkpoint is not a standard Transformers/PyTorch weights bundle
• Quantization may introduce small accuracy loss vs. the full-precision fine-tune

Training pipeline

Stage	Details
Base model	unsloth/Qwen3.5-2B
Fine-tuning	LoRA (rank 16, alpha 16) with Unsloth + TRL SFT
Training data	007superher/quant-and-serve-vlm-train-data — 7,000 augmented image–LaTeX pairs from formatted + handwritten sources
Instruction	Write the LaTeX representation for this image.
Optimizer	AdamW, lr 2e-4, linear schedule
Steps	200 (effective batch size 8)
Merged weights	007superher/unsloth_finetune
MLX conversion	mlx_vlm convert with --q-bits 4 (blockwise int4)

Training data sources

• unsloth/LaTeX_OCR — formatted equations
• deepcopy/MathWriting-human — handwritten equations
  Images were augmented (rotation, scale, shear, color jitter, noise) before training.

Evaluation

Automatic string-match metrics are a poor fit for LaTeX OCR because equivalent formulas can be written in many ways ($...$ vs $$...$$, spacing, macro variants). A human preference study was run on a held-out set of 100 images (50 formatted, 50 handwritten), comparing this fine-tuned model against the unfinetuned base mlx-community/Qwen3.5-2B-MLX-4bit:

Preference	Count	Share
Finetuned preferred	20	20%
Tie	67	67%
Base preferred	4	4%
Both bad	9	9%
By source:
Source	Finetuned	Tie
--------	----------:	----:
Formatted (50)	6	41
Handwritten (50)	14	26
The fine-tune shows the largest gains on handwritten inputs. On clean formatted math, outputs are often tied with the base model. Average latency on Apple Silicon was ~1.3 s per image (finetuned vs base, comparable).

Requirements

• Apple Silicon Mac (M-series)
• Python 3.12 recommended
• mlx-vlm ≥ 0.6.3 Approximate memory: fits comfortably within ~3–6 GB unified memory for the 2B 4-bit model.