AdityaAA2004/dagger-phase2-final-model

dagger — Phase 2 DoD extractor (scratch-trained 3/4/5-speaker curriculum, LibriMix)

The checkpoint dagger's Phase 2 final numbers are reported from: a TF-GridNet + cross-attention extractor G(x_O, ē_i) conditioned on a TitaNet speaker embedding, which extracts every speaker from the untouched original mixture rather than a running residual (CLAUDE.md §1).

Unlike the earlier Phase 1 / Phase 2 checkpoints, this one is trained from random init by a single command — no warm-start chain — so every reported number is regenerable from one config.

• Repo / code: https://github.com/RohanBanerjee88/dagger
• Training config: configs/phase2/dod/phase2_librimix_curriculum_3_4_5_train_scratch.yaml (--system proposed). Multi-depth curriculum: batches interleaved from three loaders (n_src 3, 4, 5; 800 scheduled-placement Libri3Mix train-360 scenes each, 2400 total), 10 epochs, batch 4, lr 3e-4, grad_clip 50, one T4, ~6.5 h. "Curriculum" here means several overlap depths in one run, not easy-to-hard ordering.
• Eval configs: configs/phase2/dod/phase2_librimix_{3,4,5}spk_eval_scratch.yaml — 150 test scenes each, oracle diarization, tag scratch345clip50.
• Sample rate: 8 kHz.
• Speaker counts seen in training: 3, 4, 5 (recorded in the checkpoint as trained_n_src).

Results

Phase 2's claims are relative: every comparison holds scenes, mixture and checkpoint fixed and varies only the reconstruction strategy.

• Ordering, 9 of 9 slices: coarse_to_fine > gated_deflation > ungated_deflation at every overlap depth 2–5 across all three eval sets. Holds on both scratch checkpoints (18/18) and on every warm-started checkpoint before them.
• Accumulation (ungated_deflation, SI-SDR vs. number of prior estimates subtracted into the residual; m=5 at depth 5, n=150 per level): -4.97 → -5.89 → -6.31 → -7.00 → -6.78, -1.81 dB total, monotone through the body of the chain. The terminal step is the one-and-rest endpoint and flattens — a benign case of Theorem 2's ‖E_m‖ ≤ m·ε upper bound.
• Absolute SI-SDR (no_recursion, deepest depth per set): 3spk -1.29, 4spk -3.47, 5spk -4.87 dB. Negative at depths 4–5 for every system, including the one with no deflation logic — this is the extractor's operating point under a small training budget (~2,000 steps at n_src=3 vs. Phase 1's 15,000), not the reconstruction strategy.

Full tables, spread, n and diagnostic counts: results/phase2/dod_final/.

Caveats

• The ordering is checkpoint-robust; the accumulation magnitude is not. It measures ~1.8 dB on scratch-trained curriculum checkpoints (replicated twice, agreeing to 0.08 dB per level) and 5.30 dB on the earlier warm-started chain trained on the same curriculum recipe. Quote the magnitude as a range with the checkpoint named.
• Oracle diarization only. Real-diarization robustness is Phase 3 and not yet measured.
• Synthetic 8 kHz LibriMix (English read speech, hard boundaries, no reverb). Real-corpora numbers are Phase 4.
• Set refine.rounds: 0 unless you have contaminated enrollment. Embedding refinement was measured net-harmful under clean enrollment (-0.07 to -0.41 dB across 9 slices, gate healthy).
• Research artifact for reproducing the Phase 2 experiment — not a production separation model.

Files

• proposed_librimix_curriculum_3_4_5_scratch_clip50.pt — a torch.save dict with state_dict, model_config, system, trained_n_src. The architecture must match exactly or load_state_dict raises: hidden_channels 64, n_blocks 6, n_fft 256, hop_length 64, n_heads 4, n_tokens 8, cross_attn_blocks 6.

License

Apache-2.0, matching the dagger repository.

Dependency note (same as the Phase 1/2 checkpoints): at inference dagger loads NVIDIA's TitaNet-Large (nvidia/speakerverification_en_titanet_large) as the frozen speaker encoder φ, via NeMo (Apache-2.0). TitaNet-Large is CC-BY-4.0 by NVIDIA, a separate model, and is NOT bundled here — this repo's Apache-2.0 covers only the extractor weights uploaded here. Canonical text: NOTICE.

Usage

from huggingface_hub import hf_hub_download
import torch

ckpt_path = hf_hub_download(
    repo_id="AdityaAA2004/dagger-phase2-curriculum-3-4-5-scratch",
    filename="proposed_librimix_curriculum_3_4_5_scratch_clip50.pt",
)
state = torch.load(ckpt_path, map_location="cpu")
state["state_dict"], state["model_config"], state["trained_n_src"]
# load into dagger.extract.tfgridnet_crossattn — see the repo for the full
# pipeline (diarization -> enrollment -> extraction -> reconstruction).