DhawalM/mira-invocation-hubert-base

Mira invocation detector — HuBERT Base

An audio-only binary classifier for direct address to Mira versus incidental mentions. It uses a pretrained speech encoder, attention pooling, and a small classifier. No transcription, tokenizer, or text decoder is required for inference.

The main checkpoint is the earlier 2,002-window real-microphone baseline. Only the final four encoder blocks and pooling/classifier are fine-tuned. The later 2,512-window experiment is preserved under experiments/synthetic-expansion/; its validation recall was slightly lower.

The intended positive is the glasses wearer directly addressing Mira. The current model does not verify who is speaking; synthetic labels assume the wearer.

Checkpoints

File	Training clips	Validation recall	Validation FPR	Saved threshold
best.pt	2002	72.13%	4.17%	0.83004373
experiments/synthetic-expansion/best.pt	2512	70.49%	4.17%	0.77046478

These are the original best.pt files, with encoder weights/configuration, attention pooling, classifier, feature-extractor settings, and the saved cutoff. They are project-specific PyTorch checkpoints, not standard Transformers from_pretrained packages. Preserve run.csv beside each checkpoint so the loader can recover its BF16 training precision.

Each checkpoint directory includes the original run.csv, run-specific manifest.csv, epoch history, validation predictions, threshold curve, metrics, augmentation records, and learning-rate records where produced. checkpoints.csv provides a compact model comparison. Checkpoint files have not been converted or retrained for this upload.

Load and score audio

Use Python 3.12 and the dependencies from the training code. From that checkout, download the main run:

hf download DhawalM/mira-invocation-hubert-base --exclude 'baselines/*' 'experiments/*' --local-dir runs/window_hubert_base_top4_real_mic

For a single selected window of mono audio:

from pathlib import Path
import soundfile as sf
import torch
from src.model.inference import load_inference, predict, prepare_window

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
checkpoint = Path("runs/window_hubert_base_top4_real_mic/best.pt")
model, extractor, threshold, use_bf16 = load_inference(checkpoint, device)
audio, sample_rate = sf.read("clip.wav", dtype="float32")
if audio.ndim != 1:
    raise ValueError("Provide mono audio")
audio = audio[-16 * sample_rate:]
padded, _ = prepare_window(audio, sample_rate)
score = predict(model, extractor, padded, 16000, device, use_bf16)
print(score, score >= threshold)

The live demo selects the most recent 16 seconds, resamples to 16 kHz, and left-pads to 30 seconds. Feature extraction is restored from the checkpoint and differs between Whisper and HuBERT. CPU fallback uses FP32; its scores can differ from recorded CUDA/BF16 results. Notification cooldown is application policy, not part of these weights.

Data and limitations

The exact input WAVs are in the Mira dataset. The recorded training data includes 167 derivatives from one real microphone session with ASR-assisted, not fully human-verified labels. Both checkpoints use the same 157-clip synthetic validation set to select their best epoch and threshold. The reported metrics are not independent field performance, wearer verification, or false activations per hour. Stored training windows have mixed real-context lengths, while the demo uses a 16-second crop. No new training or test-set evaluation was performed for this handoff.

The pretrained encoder comes from facebook/hubert-base-ls960; see that upstream repository for its license and attribution. This handoff does not assign a new license to the fine-tuned additions or training data.