Model Card for zeroproof-ecommerce-1b

Frontier-approaching e-commerce intent detection in a 1B model. Given a live customer conversation, zeroproof-ecommerce-1b returns a single JSON object with the customer's payment intent and its structured details. It runs inline on live traffic, at a cost and latency where calling a frontier model is not an option.

On a hard, held-out benchmark it reaches 70% exact-match core-intent accuracy, 88% of GPT-5's, at roughly 1/1000th the parameters and $0.18 per 1M tokens.

Model Details

Model Description

• Developed by: ZeroProof
• Model type: E-commerce payment-intent classifier; structured JSON output over seven intent types
• Language: English
• License: Gemma (inherited from the base model)
• Finetuned from: google/gemma-3-1b-it, via 4-bit QLoRA

Model Sources

• Repository: https://huggingface.co/zero-proof-ai/zeroproof-ecommerce-1b

Uses

Direct Use

Drop-in intent detection for e-commerce and payments support. Feed the conversation so far; the model returns one JSON object per message: whether an actionable payment intent is present, which of seven types it is, and the extracted details. Cheap and fast enough to call on every turn of every conversation.

Out-of-Scope Use

English-language e-commerce and payments only. It is a narrow intent classifier, not a general assistant, and should not be the sole authority for executing a payment without a downstream verification step.

Bias, Risks, and Limitations

Trained and evaluated on synthetic role-play conversations distilled from frontier models under a locked labeling policy; live-traffic distribution may differ. Accuracy is currently strongest on refunds, exchanges, transfers, and subscriptions; checkout and bill-pay are the focus of the current data round.

Recommendations

Pair the model with a verification layer before any payment executes, and re-benchmark on your own traffic before relying on the reported numbers in production.

How to Get Started with the Model

from peft import AutoPeftModelForCausalLM
from transformers import AutoTokenizer

model = AutoPeftModelForCausalLM.from_pretrained("zero-proof-ai/zeroproof-ecommerce-1b")
tok   = AutoTokenizer.from_pretrained("zero-proof-ai/zeroproof-ecommerce-1b")

Use greedy decoding (temperature=0). The system prompt and per-intent detail schemas ship in the repo.

Training Details

Training Data

The differentiator is the data. ZeroProof builds e-commerce intent training data as randomized conversational simulations: role-played customers with independently sampled personas, tones, financial situations, life stages, devices, and behaviors, including adversarial actors, distilled from frontier models under a locked labeling policy. Generation is label-blind (the generating models never see the intent schema), and every candidate passes a structural data gate: deduplicated by message signature, with zero train/eval leakage.

Training Procedure

Training Hyperparameters

• Method: 4-bit QLoRA (via unsloth)
• Epochs: 1
• Effective batch size: 16
• Learning rate: 2e-4
• Sequence length: 12288 (long conversations fit)

Evaluation

Testing Data, Factors & Metrics

Held-out eval of 1,977 conversations, zero train/eval leakage, macro-averaged (equal weight per intent), scored by exact match against gold. The frontier panel is scored on a 412-row balanced subset of the same eval.

Results

• Fine-tuning takes core-intent accuracy from 12% to 70%, landing at 88% of GPT-5 and within 10 points of Opus 4.8.
• On structured detail extraction it edges GPT-5 (44% vs 41%).
• No easy rows: every conversation is adversarial (negation, ambiguity) or long. The model holds 65% on the hard slice and 78% on the long slice, where the base model scores 18% and 11%.

Technical Specifications

Model Architecture and Objective

A LoRA adapter over gemma-3-1b-it, trained response-only to emit one structured intent object per turn.

Compute Infrastructure

Served as an OpenAI-compatible endpoint (base + adapter) under vLLM. Measured on a Modal L4 GPU: $0.18 per 1M output tokens at batched capacity and ~1s per response (p50 984 ms, p95 1,191 ms), 55 to 140x below frontier list prices.

Model Card Contact

ZeroProof, https://huggingface.co/zero-proof-ai