APINAJA/inverse-fx-phase1-logmel-transformer-v2

Inverse FX Chain Predictor — Phase 1: Log-Mel + Transformer (effect-type prediction)

Predicts which of EQ, Compressor, Reverb were applied to a signal, from the dry and wet audio only (no effect-chain or parameter metadata is used as input). Multi-label: 3 sigmoid outputs trained with BCEWithLogitsLoss.

Dataset: APINAJA/inverse-fx-phase1-effect-type — existing train / validation / test splits. Test was used once, for the final evaluation of the best-validation checkpoint (selected by validation macro_f1).

Architecture

dry_audio [B, 441000] -> Log-Mel -> [B, 128, T] -+
                                                         +-> concat (feature axis) -> [B, T, 256]
wet_audio [B, 441000] -> Log-Mel -> [B, 128, T] -+
  -> Linear(256 -> 256) + LayerNorm -> prepend [CLS] + sinusoidal positions -> [B, T+1, 256]
  -> TransformerEncoder(layers=4, heads=4, ff=512, dropout=0.1, pre-LN, GELU)
  -> [CLS] pooling [B, 256] -> Linear-GELU-Dropout-Linear -> logits [B, 3] = [EQ, Compressor, Reverb]

T = 862 frames. Trainable parameters: 2,208,771.

Preprocessing / Log-Mel

setting	value
sample rate	44100 Hz, mono (channel mean), float32
length	10.0 s fixed (crop from start / zero-pad end)
normalisation	none on audio; log-mel standardised with global train stats (mean -6.3604, std 4.8448)
n_fft / hop / window	2048 / 512 / Hann
n_mels / f_min / f_max	128 / 20.0 Hz / 20000.0 Hz (Slaney mel)
log	log(mel_power + 1e-06)

Label mapping

Output index 0 = EQ (eq_label), 1 = Compressor (compressor_label), 2 = Reverb (reverb_label). Prediction: sigmoid(logit) >= 0.5. 7-combination view (reporting only; a [0,0,0] prediction is mapped to the single most probable effect):

#	combination	vector
1	EQ	[1, 0, 0]
2	Compressor	[0, 1, 0]
3	Reverb	[0, 0, 1]
4	EQ + Compressor	[1, 1, 0]
5	EQ + Reverb	[1, 0, 1]
6	Compressor + Reverb	[0, 1, 1]
7	EQ + Compressor + Reverb	[1, 1, 1]

Training

AdamW (lr 0.0003, wd 0.05), linear warmup 2 epochs + cosine, 40 epochs, per-GPU batch 16, FP16 AMP=True, grad clip 1.0, seed 42, PyTorch DDP (torchrun) on Kaggle.

Validation (best epoch 6)

macro F1 0.8571, micro F1 0.8428, exact-match 0.4907, 7-combination accuracy 0.4907, loss 0.3368

Test (best-validation checkpoint, evaluated once)

metric	value
loss	0.3705
exact-match accuracy	0.4740
7-combination accuracy	0.4740
macro F1	0.8496
micro F1	0.8355
examples	462

effect	precision	recall	F1
EQ	0.5772	0.9205	0.7095
Compressor	0.9960	0.9318	0.9628
Reverb	0.8937	0.8598	0.8764

Confusion matrix: results/plots/test_confusion_matrix.png, raw numbers in results/test_metrics.json.

Checkpoint history

epoch	train loss	val loss	val macro F1	val 7-comb acc	best
001	0.6531	0.6303	0.7034	0.2277	yes
002	0.5488	0.4392	0.8230	0.4037	yes
003	0.4198	0.3862	0.8328	0.4700	yes
004	0.3857	0.3863	0.8208	0.4700
005	0.3579	0.3675	0.8342	0.4638	yes
006	0.3366	0.3368	0.8571	0.4907	yes
007	0.3175	0.3382	0.8545	0.5217
008	0.3113	0.3413	0.8172	0.5010
009	0.2986	0.3376	0.8527	0.5135
010	0.2960	0.3501	0.8279	0.5031
011	0.2821	0.4265	0.7997	0.4907
012	0.2844	0.3305	0.8493	0.5197
013	0.2770	0.3592	0.8080	0.5093
014	0.2799	0.3510	0.8403	0.5300
015	0.2661	0.3453	0.8298	0.5259
016	0.2588	0.3543	0.8479	0.5487

Repository layout

checkpoint_epoch_XXX/ full resumable state (model, optimizer, scheduler, scaler, RNG, config, history), best/ best-validation checkpoint, latest_checkpoint.json pointer, final/ inference bundle, results/ history CSV/JSON + plots + test metrics, runs/ TensorBoard event files.

Loading

import torch, json
cfg = json.load(open("final/config.json"))["experiment_config"]
from fxp1.model import build_model      # code: fxp1/ in this repo
model = build_model(cfg); model.load_state_dict(torch.load("final/model.pt")); model.eval()
probs = torch.sigmoid(model(dry_wave, wet_wave))   # [B, 3] = [EQ, Compressor, Reverb]