mdonigian/mimir-vote-predictor

Mimir voice-to-vote predictor

Predicts the probability that a Supreme Court justice votes for the petitioner using vocal measurements from oral argument and justice identity. The selected model uses identity + all scalar features, trained on Court terms 2010–2015 and 2022–2023. model.json contains the selected logistic model, fitted preprocessing, and a nonvoice fallback.

The selected model improved all-vote probability loss over the nonvoice baseline on the final 2024-term evaluation, with a case-bootstrap interval below zero.

The evaluation did not establish added predictive value beyond speaking amount alone. An overall gain over justice identity therefore should not be taken as evidence that vocal delivery adds value beyond how much each justice speaks.

Final evaluation

The 2024 Court term was scored after selecting the training population, feature family, and regularization on the 2023 term and freezing the fitted model and target-free predictions. The primary result includes 537 votes across 60 case groups. Two-sided pitch is available for 342 votes (63.7%); the remaining votes use the separately fitted justice-identity baseline.

Model	Accuracy	Log loss	Brier score
Justice identity	62.8%	0.6663	0.2364
Identity + speaking amount	67.0%	0.6103	0.2114
Identity + pitch	63.9%	0.6417	0.2252
Identity + pitch, rate and pauses	64.1%	0.6397	0.2244
Identity + pitch, rate, pauses and amount	67.8%	0.6004	0.2071
Identity + pitch and other prosody	62.8%	0.6508	0.2289
Identity + all scalar features (selected)	67.0%	0.6061	0.2097

Selected voice minus nonvoice log loss: -0.0602, 95% paired whole-case bootstrap interval [-0.0888, -0.0326]. Negative differences favor voice. Intervals condition on the fitted models and do not include uncertainty from training or model selection. Secondary family/mask comparisons are exploratory. Accuracy uses a 0.5 threshold; case classification counts predicted justice votes without assuming that their probabilities are independent.

The accuracy difference is 4.3 percentage points, with a 95% case-bootstrap interval of [-0.6, 9.1]. The prespecified acceptance criterion concerns probability loss, not accuracy alone.

Selected model minus speaking-amount control log loss: -0.0042, 95% paired case-bootstrap interval [-0.0133, 0.0044].

A secondary identity-only ablation, declared after inspecting 2023 selection but frozen before 2024 scoring, uses exactly the speaking-amount model's fitting votes, training scope and regularization, with the original fallback. It reaches 62.8% accuracy and 0.6680 log loss. Selected model minus this matched identity control: -0.0619 log loss, exploratory 95% case-bootstrap interval [-0.0908, -0.0337]. This checks whether differences in the fitting cohort account for gains over the original baseline; it did not change primary model selection.

The unchanged older model was also evaluated on 2015: pitch accuracy was 64.5% on 152 pitch-available votes and 50.7% on all 349 labeled votes in 43 case groups. That replication did not establish a clear improvement from timing over pitch. It demonstrates why the feature-available subset cannot substitute for all-vote coverage.

Training and selection

The candidates were justice identity, speaking amount alone, pitch, pitch plus rate/pauses, pitch plus rate/pauses/speaking amount, pitch plus other prosody, and all scalar features. Training either used 2022 alone or added the existing 2010–2015 historical cohort; 2023 selected the population and regularization C from 0.01, 0.1, 1, and 10. Each specification was refitted on its training data plus 2023 before the final 2024 evaluation. Consolidated cases, rearguments and shared recordings are grouped across splits. Fitting labels must have been available before the next period. No 2024 labels enter fitting.

Imputation medians and justice/channel scaling come only from fitting examples. An unseen justice uses the fitted intercept and available channel/global scaling. The selected raw feature contrasts are: pitch_mean_diff, pitch_log_ratio, f0_sd_diff, f0_range_semitones_diff, f0_slope_diff, voiced_frac_diff, intensity_mean_diff, intensity_sd_diff, hnr_mean_diff, jitter_local_diff, shimmer_local_diff, speech_rate_diff, articulation_rate_diff, pause_fraction_diff, mean_turn_duration_diff, log_turn_count_diff. Neural emotion scores, text content, WavLM arrays, and decision-coded legal features do not enter this model. Voice candidates and the speaking-amount control use the same pitch-available cohort; the fallback learns from all eligible fitting votes in its selected training population.

Inputs and inference

The model expects one JSON record per case and justice, including justice_id, channel, pitch_mean_diff, pitch_log_ratio, and the additional raw feature contrasts above. Differences are petitioner minus respondent. Do not supply whole-term normalized values. Absent extra features use fitted medians. Set pitch_available to false when either side lacks usable pitch; the output explicitly reports whether voice or fallback was used.

For pitch and other per-turn vocal measurements, take the equal-weight mean of finite turns for each addressed side before forming the contrast. pitch_log_ratio is the natural log of petitioner mean pitch divided by respondent mean pitch. Pitch means and standard deviations use Hz; pitch slope uses Hz/second and range uses semitones. Intensity and HNR use dB. Rates use ASR words/second; pause fraction, voiced fraction, jitter and shimmer are fractions. Speaking amount uses mean mapped turn duration in seconds and the difference between the two sides' natural-log 1 + turn count values. The extraction pipeline defines the timing and mask eligibility of those turns.

Download model.json and predictor.py from this repository, then run:

python predictor.py --model model.json --input example-inputs.json --output predictions.json

The prediction runtime requires only Python 3.11 or later and its standard library. example-inputs.json and example-predictions.json provide a verified runnable example. Raw MP3 files require the documented Mimir acoustic extraction pipeline first; this logistic predictor does not consume MP3 bytes directly.

Data and reproducibility

Complete supporting dataset contains row-level fitting data, separate final inputs/targets, all candidate models, selection scores, predictions, calibration bins, subgroup and coverage reports, both audio masks, new diarization/word caches, original-order word outputs and their row maps, derived clock mappings, turn-level measurements, source inventories, and audit receipts. Existing source audio and historical artifacts are referenced by immutable commit and content hash, so identical original audio is not duplicated.

Original Oyez sources are preserved. Derived alignment uses disjoint fit/check phrases; recordings that do not pass those checks retain unavailable measurements. This does not establish that every excluded clock is wrong. The anchored mask is the prespecified primary choice. Overlap-only results are reported as a sensitivity analysis; superiority of the stricter mask has not been established. Stable sorting of Whisper words changes row order and derived indices only; raw words and timestamps are retained.

The protocol was committed before new predictive results in Git commit 70ba1f0. Code: mdonigian/mimir at 692117efd1872eee605f916ec08395837f21eda7. All publication files are inventoried with SHA-256 hashes. An anonymous reader check downloads the released artifact and independently replays the selected probabilities.

Intended use and limits

This is a retrospective research model for vote prediction after oral argument, using Oyez speaker annotations and derived addressed-side labels. The 2024 term was previously unscored in this project, but its public labels had been ingested for source coverage checks; this is not a prospectively collected trial. Results concern the evaluated Court and terms, and do not guarantee performance on a future case or a changed Court.

Speaking amount reflects interaction patterns as well as vocal delivery. A gain from the timing bundle alone does not establish that emotion caused or predicted the vote; the speaking-amount control addresses that distinction. Compare the published volume contrast before claiming added value from delivery. There is no matched-case result establishing superiority over Dietrich, Enos and Sen (2019).

Automatic speaker/side attribution before Oyez transcripts arrive is not evaluated by this model card. Saved human reviews and credentials are excluded from all artifacts.