Aviadara/phone-accentedness-scorer

Phone-Level Accentedness Scorer — E16

This repository contains the exact promoted E16 checkpoint for the Phone-Level Accentedness Scoring challenge. Given a speech recording and its expected phoneme sequence, the model returns one continuous 0–100 American-English accentedness score per phoneme.

Try it

• Open the live Gradio demo
• Open the verified Colab notebook
• View the complete source, experiments, and write-up
• Open the project Space

The checkpoint uses a frozen Whisper-tiny encoder, constrained CTC phone alignment, acoustic and alignment features, a bidirectional GRU, and an ordinal prediction head with per-phone class weights proportional to n_c^-0.54.

Validation results

Metric	E16 result
Balanced MAE	21.8496
MAE	18.0081
Quadratic-weighted kappa	0.5786
Macro-F1	0.5682
Balanced accuracy	0.6642
Spearman correlation	0.5583

Balanced MAE improved by 0.7248 over the E01 incumbent, with paired utterance-bootstrap 95% CI [-1.3549, -0.0812]. The supplied validation split has substantial inferred-speaker and prompt overlap with training, so these numbers must not be interpreted as new-speaker performance.

Artifact integrity

The model file SHA-256 is:

ead3144c82ab87ad9d6406511c6348a99c944a9f8ac1097756a6a61d78e80338

deployment_manifest.json binds the promoted checkpoint to its accepted confirmation evidence and records hashes for every deployed file. The nine artifact files in this repository are unchanged from submission/model/ at Git commit e05549fc0b8e89fef05d6f111a623b3f397240e1.

Usage

This is a custom architecture and is not compatible with transformers.AutoModel. The simplest supported path is the linked Colab notebook. For local use, clone the source repository, install the environment from submission/, download this model snapshot, and point inference at it:

git clone https://github.com/aviadarn/Accentedness-Scoring-Challenge.git
cd Accentedness-Scoring-Challenge/submission
uv sync --python 3.11
uv run python -c 'from huggingface_hub import snapshot_download; print(snapshot_download("Aviadara/phone-accentedness-scorer"))'
export ACCENT_MODEL_DIR=/path/printed/by/the/previous/command
uv run python -c 'from inference import score_phonemes; print(score_phonemes("sample.wav", ["n", "oʊ", "s", "ɝ"]))'

The public API is score_phonemes(audio_path, phonemes) -> list[float]. Input phones must use the model vocabulary recorded in accent_model_config.json.

Intended use and limitations

This artifact is for research, challenge evaluation, and formative pronunciation feedback. “Native-like” is a subjective annotation target, not a measure of intelligence, identity, employability, nationality, or general English proficiency. The challenge data lacks documented speaker IDs, rater agreement, consent, and authoritative continuous calibration. Do not use the model for high-stakes decisions. The private challenge audio and labels are not included in this repository.