Polish public model card for Stage 5

README.md

@@ -18,115 +18,157 @@ license: mit
 
 # Premove ITN v0.1.0
 
-Premove ITN is an English inverse text normalization system for voice-agent
-utterances. This release is the first frozen, inference-only artifact for its
-structured-value contextual scorer. It ranks deterministic Rust candidates
-using sentence context, then applies exact maximum-score decoding.
+Premove ITN is an open-weight contextual inverse text normalization system for
+English voice-agent transcripts. It turns spoken-form ASR text into structured
+written text:
 
-**Premove ITN is released as an open-weight contextual inverse text
-normalization model. The inference code and model weights are licensed under
-MIT.**
+```text
+call me at four thirty  →  call me at 04:30
+the total is twenty dollars  →  the total is $20
+```
+
+Deterministic Rust realizers propose valid written forms. A
+DeBERTa-v3-large contextual scorer uses the complete sentence to score those
+candidates, and an exact decoder selects compatible, non-overlapping edits.
 
-The artifact contains the complete trained parameters in `model.safetensors`,
-the exact DeBERTa configuration, and the tokenizer files used by the selected
-production checkpoint. Optimizer state, scheduler state, training counters,
-training data, and evaluation rows are not included.
+This repository contains the frozen, inference-only v0.1.0 model artifact. It
+does not contain optimizer state, scheduler state, training counters, training
+data, or evaluation rows. Source code and retained evaluation evidence are in
+[`premove-ai/premove-itn`](https://github.com/premove-ai/premove-itn).
 
-The artifact is loaded through the `premove-itn` package. It is not a generic
-Transformers model; use `PremoveITN.from_pretrained()` so candidate generation,
-scoring, and decoding stay aligned with the release implementation:
+## Loading the model
+
+This is a custom candidate-scoring architecture. Do not load it with
+`AutoModel.from_pretrained()`.
 
 ```python
 from premove_itn import PremoveITN
 
-itn = PremoveITN.from_pretrained(
-    "premove-ai/premove-itn",
-    revision="v0.1.0",
-)
-print(itn.normalize("call me at four thirty"))  # call me at 04:30
+itn = PremoveITN.from_pretrained()
+print(itn.normalize("call me at four thirty"))
+# call me at 04:30
 ```
 
-Source code, deterministic Rust realization rules, the Python API, benchmark
-code, and retained results are available in the
-[`premove-ai/premove-itn`](https://github.com/premove-ai/premove-itn)
-repository.
+The `premove-itn` PyPI release is not published yet. Until the public package
+release, contributors can build and install the release wheel from the GitHub
+repository. Create one `PremoveITN` instance and reuse it; model initialization
+is expensive compared with warm normalization.
+
+`device="auto"` selects CUDA when available, then Apple MPS, then CPU. The
+current release candidate has been validated end-to-end only on macOS Apple
+Silicon with Python 3.11 and MPS. Other environments require release
+validation.
+
+## Architecture
+
+```text
+Spoken ASR text
+      ↓
+deterministic Rust candidates
+      ↓
+DeBERTa-v3-large contextual scores
+      ↓
+exact maximum-score decoder
+      ↓
+written transcript
+```
 
-The model uses `microsoft/deberta-v3-large` at the exact revision recorded in
-`provenance.json`. The artifact includes the tokenizer and base configuration;
-the base model is not separately redistributed. Microsoft lists the base model
-under the MIT license; retain that attribution when redistributing this
-artifact.
+The scorer has 435,594,145 parameters. The artifact contains the complete
+trained state in `model.safetensors`, the pinned DeBERTa configuration, and the
+tokenizer files required by the release.
 
-This release is tagged `v0.1.0`. Do not move or overwrite that tag after
-publication. The artifact digest and source checkpoint lineage are recorded in
-`provenance.json`.
+Supported candidate kinds are `DIGIT_SEQUENCE`, `CARDINAL`, `TIME`, `DATE`,
+`MONEY`, `DECIMAL`, `PHONE`, `ELECTRONIC`, `MEASUREMENT`, `ORDINAL`,
+`PUNCTUATION`, `WHITELIST`, and `WORD`.
 
-## Intended use
+## First Evaluation
 
-Use this artifact for English voice-agent text normalization where spoken
-numbers, dates, times, money, phone values, identifiers, URLs, and related
-structured values require context-sensitive disambiguation. The model receives
-only the utterance text at inference time. Candidate metadata is generated by
-the package's deterministic Rust layer.
+The retained First Evaluation used a frozen, balanced synthetic stress suite.
+Semantic entity accuracy is the primary structured-value metric. Strict exact
+match separately measures the complete canonical output.
 
-## Supported span kinds
+### Dedicated voice-agent rows
 
-The supported span kinds are listed in `config.json`: `DIGIT_SEQUENCE`,
-`CARDINAL`, `TIME`, `DATE`, `MONEY`, `DECIMAL`, `PHONE`, `ELECTRONIC`,
-`MEASUREMENT`, `ORDINAL`, `PUNCTUATION`, `WHITELIST`, and `WORD`.
+| Backend | Correct entities | Semantic accuracy | Mean latency |
+| --- | ---: | ---: | ---: |
+| **Premove ITN** | **398/400** | **99.50%** | 57.41 ms |
+| Thutmose | 268/400 | 67.00% | 16.04 ms |
+| text-processing-rs | 273/400 | 68.25% | 0.15 ms |
 
-## Runtime
+### Overall 1,500-row benchmark
 
-The contextual API requires Python 3.11 or newer, PyTorch, Transformers,
-SentencePiece, safetensors, and huggingface_hub. `device="auto"` selects CUDA,
-then Apple MPS, then CPU. Pass `device="cpu"`, `device="mps"`, or
-`device="cuda"` to select a device explicitly. The retained latency result was
-measured with batch size one on Apple MPS. Other runtimes and devices can have
-different latency and memory use.
+| Backend | Semantic accuracy | Strict exact | Mean latency |
+| --- | ---: | ---: | ---: |
+| **Premove ITN** | **89.70%** | **40.53%** | 56.49 ms |
+| Thutmose | 59.39% | 22.13% | 15.98 ms |
+| text-processing-rs | 55.79% | 16.53% | 0.14 ms |
 
-## First Evaluation
+Premove led measured semantic accuracy overall and on the 400 dedicated
+voice-agent rows. It did not lead latency.
 
-The retained First Evaluation used 1,500 frozen VoiceAgent ITN rows and three
-backends. The metrics below are semantic entity metrics unless marked strict.
-Formatting-only differences can reduce strict exact match while preserving the
-same semantic value.
+Latency used sequential batch-one requests on an Apple M4 MacBook Air with
+MPS, an optimized Rust extension, and eight Rayon workers. Models were loaded
+and warmed before request latency was measured. Download and initialization
+are excluded. The result is not an estimate of production-traffic accuracy,
+and the evaluation was not blind.
 
-| Backend | Entity micro | Entity macro | Voice rows | Mean latency |
-|---|---:|---:|---:|---:|
-| Premove ITN | 89.70% | 90.71% | 398/400 (99.50%) | 56.49 ms |
-| Thutmose | 59.39% | 57.61% | 268/400 (67.00%) | 15.98 ms |
-| text-processing-rs | 55.79% | 55.05% | 273/400 (68.25%) | 0.14 ms |
+See the
+[`First Evaluation report`](https://github.com/premove-ai/premove-itn/blob/main/eval/voice_agent_itn/results/first-evaluation/REPORT.md)
+and
+[`detailed tables`](https://github.com/premove-ai/premove-itn/blob/main/eval/voice_agent_itn/results/first-evaluation/DETAILS.md).
 
-Premove leads the measured semantic accuracy overall and on the 400
-voice-agent-domain rows. It does not lead latency. This is a balanced synthetic
-stress benchmark, not an estimate of production traffic accuracy, and the run
-was not blind.
+## Intended use
+
+Use Premove for English voice-agent transcripts in which numbers, dates,
+times, money, phone values, identifiers, URLs, and related structured values
+need sentence-level disambiguation. The runtime receives only transcript text.
+Candidate metadata is generated internally.
+
+## Model lifecycle
+
+- The first use downloads about 1.6 GB; duration depends on the network.
+- Cached initialization takes several seconds on the tested system.
+- Warm normalization averaged 56.49 ms in the retained MPS benchmark.
+- Services and transcript streams should keep one normalizer resident.
 
 ## Limitations
 
-- This custom scorer is not directly loadable with `AutoModel.from_pretrained`.
-  Use `PremoveITN.from_pretrained()` and the matching `premove-itn` package
-  version.
-- The artifact is English-only and depends on the package's candidate graph and
-  decoder for end-to-end normalization.
-- Benchmark results are not a guarantee for unseen domains or formatting
-  policies. Human gold adjudication and broader production-distribution testing
-  remain separate requirements.
+- English only.
+- A 435.6M-parameter model with an approximately 1.6 GB download.
+- Multi-second initialization.
+- Not compatible with generic `AutoModel.from_pretrained()` loading.
+- The benchmark is synthetic and does not measure live production traffic.
+- Weaker measured categories include URL, MONEY, CARDINAL, TIME,
+  REFERENCE_ID, and VERSION.
+- Blind human gold adjudication and broader contamination checks remain
+  incomplete.
+- End-to-end release validation currently covers macOS Apple Silicon, Python
+  3.11, and MPS only.
 
-## Release identity
+## Release identity and provenance
 
 - Artifact version: `v0.1.0`
 - Architecture: `premove-candidate-scorer-v1`
-- Package version: `0.1.0`
+- Required package version: `0.1.0`
 - Hub repository: `premove-ai/premove-itn`
-- Original artifact namespace: `premove-itn/premove-itn-contextual`
-- Hub revision: `v0.1.0`
-- Hub commit: `80bda5e2e1fe9542aa628597090242df57c1a157`
+- Immutable model commit: `80bda5e2e1fe9542aa628597090242df57c1a157`
 - Base model: `microsoft/deberta-v3-large`
 - Base model revision: `64a8c8eab3e352a784c658aef62be1662607476f`
-- Source checkpoint SHA-256: `9021fa11a028faefb31ef67878170cbe29ed25e68a9a78999f37b120c2ad00d5`
-- Inference artifact SHA-256: `119c0f19767b61446e04da1f8f01a001edf97a47a66965e7146db2483b4937a1`
+- Model SHA-256: `119c0f19767b61446e04da1f8f01a001edf97a47a66965e7146db2483b4937a1`
 
-## Licensing
+The package pins the immutable model commit and verifies its release metadata,
+base-model identity, and model digest before inference. Full training
+composition and checkpoint selection evidence are in the
+[`production model record`](https://github.com/premove-ai/premove-itn/blob/main/docs/model-provenance.md).
 
-Premove ITN is MIT licensed. The pinned base model is [`microsoft/deberta-v3-large`](https://huggingface.co/microsoft/deberta-v3-large), whose model card lists an MIT license. The architecture derives from [DeBERTaV3](https://arxiv.org/abs/2111.09543). Review and retain both notices when redistributing this artifact.
+## License and attribution
+
+Premove ITN source code and model weights are MIT licensed. The scorer uses
+[`microsoft/deberta-v3-large`](https://huggingface.co/microsoft/deberta-v3-large)
+at the revision above. Its architecture derives from the
+[`DeBERTaV3` paper](https://arxiv.org/abs/2111.09543).
+
+The Rust realization layer uses
+[`text-processing-rs`](https://github.com/FluidInference/text-processing-rs),
+which is Apache-2.0 licensed. Required notices are retained in the source
+repository.