Learnable Fractional Superlets with a Spectro-Temporal Emotion Encoder for Speech Emotion Recognition
Authors: Alaa Nfissi
Wassim Bouachir
Nizar Bouguila
Brian L Mishara
Venue: ICLR 2026 Poster (2026)
Keywords: Speech Emotion Recognition
Time–Frequency Analysis
Learnable Fractional Superlets
Spectro-Temporal Encoding
Representation Learning
End-to-End Neural Networks
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose the Learnable Fractional Superlet Transform (LFST), a principled differentiable time–frequency representation integrated with a Spectro-Temporal Emotion Encoder (STEE), enabling end-to-end speech emotion recognition from raw waveforms.

Abstract: Speech emotion recognition (SER) hinges on front-ends that expose informative time-frequency (TF) structure from raw speech. Classical short-time Fourier and wavelet transforms impose fixed resolution trade-offs, while prior ”superlet” variants rely on integer orders and hand-tuned hyperparameters. We revisit TF analysis from first principles and formulate a learnable continuum of superlet transforms. Starting from DC-corrected analytic Morlet wavelets, we define superlets as multiplicative ensembles of wavelet responses and realize learnable fractional orders via softmax-normalized weights over discrete orders, computed as a logdomain geometric mean. We establish admissibility (zero mean) and continuity in order and frequency, and characterize approximate analyticity by bounding negative-frequency leakage as a function of an effective cycle parameter. Building on these results, we introduce the Learnable Fractional Superlet Transform (LFST), a fully differentiable front-end that jointly optimizes (i) a monotone, logspaced frequency grid, (ii) frequency-dependent base cycles, and (iii) learnable fractional-order weights, all trained end-to-end. LFST further includes a learnable asymmetric hard-thresholding (LAHT) module that promotes sparse, denoised TF activations while preserving transients; we provide sufficient conditions for boundedness and stability under mild cycle and grid constraints. To exploit LFST for SER, we design a compact Spectro-Temporal Emotion Encoder (STEE), achieving strong performance with a parameter budget that is orders of magnitude smaller than large self-supervised models, at the cost of additional frontend computation compared to STFT- or LEAF-based baselines. STEE consumes two-channel TF maps, magnitude S and phase-congruency κ, through a compact multi-scale stack with residual temporal and depthwise-frequency blocks, Adaptive FiLM gating, axial (time-axis) self-attention, global attentive pooling, and a lightweight classifier. The full LFST+STEE system is trained in a standard train-validate-test regime using focal loss with optional class rebalancing, and is validated on IEMOCAP, EMO-DB, and the private NSPL-CRISE dataset under standard protocols. By unifying a principled, learnable TF transform with a compact encoder, LFST+STEE replaces ad hoc front-ends with a mathematically grounded alternative that is differentiable, stable, and adaptable to data, enabling systematic ablations over frequency grids, cycle schedules, and fractional orders within a single end-to-end model. The source code of this paper is shared on the GitHub repository: https://github.com/alaaNfissi/LFST-for-SER.

Published as a conference paper at ICLR 2026
LEARNABLE FRACTIONAL SUPERLETS WITH A TEMTRRL EMOTION ENCODER FOR
SPEECH EMOTION RECOGNITION
Alaa Nfissi2.2.4 Wassim Bouachir Nizar Bouguila" Brian Mishara 3.4
Data Science Laboratory (DOT-Lab)), Université TÉLUQ, Montréal, Canada. Concordia Institute for Information Systems Engineering, Concordia University, Montréal, Canada Ppsychology Department, University of Québec Montréal, Montréal, Canada
*Center for Research and Intervention on Suicide, Ethical Issues and End-of-Life Practices, Montréal, Canada alaa.nfi @mail concordia.....o..o......o....... ca, wass im, Bouachi werelli.rrelu...... ni zar. wwwguicuuuuuu laf concordia ca, mishara. bri brian@uqam.cc
ABSTRACT
Speech emotion recognition (SER) hinges on front-ends that expose informative timeffrqueecy (TF) structure from raw speech. Classical short-time Fourier and wavelet transforms impose fixed resolution trade-offs, while prior "superlet" vari- ants rely integer orders and hand-tuned hyperparameters. We revisit TF anal- ysis from first principles and formulate a learnable continuum of superlet trans- forms. Starting from DC-corrected analytic Morlet wavelets, we define superlets as multiplicative ensembles of wavelet responses and realize learnable fractional orders via sestmax-normalizg weights over discrete orders, computed domain geometric mean. We establish admissibility (zero mean) and continu- ity in order and frequency, and characterize approximate analyticity by bounding Srgative-rrequece leakage as a function of an effective cycle parameter. Build- ing on these results, introduce the Learnable Fractional Superlet Transform (LFST), fully differentiable front-end that jointly optimizes (i) monotone, log- spaced frequency grid, (ii) reequency-dependers base cycles, and (iii) learnable fractional-order weights, all trained end-to-end. LFST further includes learn- able asymmetric hard-thresholding (LAHHT) module that promotes sparse, de- noised TF activations while preserving transients; we provide sufficient conditions for boundedness and stability under mild cycle and grid constraints. To exploit LFST for SER, we design compact Spectro-Temporal Emotion Encoder (STEE), achieving strong performance with a parameter budget that orders of magni- tude smaller than large self supervupeee models, the cost of additional front- end computation compared to STFT- or LEAF-based baselines. STEE consumes two-channel TF maps, magnitude and phase-congruenccy k, through compact multi-scale stack with residual temporal and depthwise-freqquecyy blocks, Adap- tive FiLM gating, axial (time-axis) self-attention, global attentive pooling, and a lightweight classifier. The full LFST+STEE system is trained in a standard train hain-vaidatte-est regime using focal loss with optional class rebalancing, and is validated on IEMOCAP, EMO-DB, and the private NSPL-CRISE dataset un- der standard protocols. By unifying a principled, learnable TF transform with a compact encoder, LFST+STEE replaces ad hoc front-ends with mathematically grounded alternative that is differentiable, stable, and adaptable to data, enabling systematic ablations over frequency grids, cycle schedules, and fractional orders within a single end-to-end model. The source code of this paper is shared on the FSRR.....RR
log-
GitHub repository:
INTRODUCTION
Human speech carries dense affective information that conveys intent, mood, and social cues. Au- tomatic speech emotion recognition (SER) aims to infer this affective state from acoustic signals

---

Published as a conference paper at ICLR 2026
and underpins applications in conversational agents, mental-health monitoring, and human-robot interaction. A core scientific difficulty is the non-stationary nature of speech: emotionally salient patterns emerge across disparate time scales, from rapid pitch modulations and mnicoo-prosode cues to slower spectral-envelope dynamics, often with overlap, masking, or background noise (Rosen, 1992; EI Ayadi et al., 2011; Wani et al., 2021; Schuller, 2018). Effective front-ends must therefore expose Lime-frequency (TF) structure that balances temporal precision with spectral clarity, while remaining robust and learnable from data.
Traditional SER pipelines rely on fixed TF representations such as short-time Fourier transform (STFT) spectrograms or mel-spectrograms, which enforce a window-dependent trade-off: longer windows sharpen frequency resolution while smearing short events, whereas shorter windows do the opposite. Wavelet transforms partially alleviate this by analyzing low frequencies with long wavelets and high frequencies with short ones, yielding a multiresolution analysis (Mallat, 1989). Yet, the effective number of cycles is typically fixed across the spectrum, so frequency resolution degrades at high frequencies; more importanty, both STFT and classical wavelet front-ends bake in a priori TF compromises that cannot adapt to the signal statistics or task demands (Rosen, 1992; El Ayadi et al., 2011). In SER practice, these front-ends are often followed by deep classifiers, or replaced with raw- www-worefrmm models, but the cont-enndenccoer interface remains largely heuristic (Fayek al., 2017; Dai et al., 2017; Nfissi et al., 2022).
We revisit TF analysis from first principles and consider multiplicative ensembles of DC-corrected, approximately analytic Morlet responses at a common center frequency. Intuitively, combining short (few-cycle) and long (many-cycle) wavelets by log-domain geometric mean can approach the temporal acuity of short wavelets while recovering the frequency concentration afforded by long wavelets Moca et al. (2021). Extending the order of this ensemble beyond integers yields a contin- uum that avoids discrete "banding" artifacts and permits smooth trade-offs across frequency Bårzan et al. (2021). Concretely, we realize fractional orders via softmax- normalized weights over dis- crete orders; this convex combination produces an effective (learndd) order per frequency. We study three properties crucial for principled front-end: (i) admissibility via zero-mean, DDC-corected wavelets; (ii) continuity with respect to order and frequency grid; and (iii) approximate analytic- ity, in the sense that
leakage decays with an effective cycle parameter, yielding well-behaved magnitude and phase.
Motivated by these considerations, we introduce the Learnable Fractional Superlet Transform (LFST), a fully differentiable TF front-end that optimizes: (i) a monotone, log- log-spaced frequency grid with learnable positive increments and anchored endpoints; (ii) frequency-dependent base cy- cles (ensured 2 1 by softplus parameterization): and (iii) fractional order weights (softmax over order logits). For each order and frequency, LFST uses DC-corrected Morlet filters with magni- tude LI-normalization, aggregates responses multiplicatively via a weighted nog-su//xxponential, and computes phase-congruency map by summing order-weighted unit phasors. A learnable asymmetric hard-thresholding (LAHT) module acts on the magnitude map to promote sparse, de- noised TF activations while preserving transients. Practical stability is enforced through safe pa- rameterizations and numerics (e.g.. capped exponents, log-domain accumulation, bounded gates). All parameters are trained end-to-end by backpropagation together with the downstream network, turning the TF compromise from a fixed design choice into data-driven inductive bias. To ex- ploit LFST representations, we design a compact Opectro-Tempord Emotion Encoder (STEE) that consumes two-channe TF maps, magnitude S and phase-congruence, K, and processes them with depthwise-temport stem, spectral residual blocks and hybrid TF blocks (depthwise along frequency and time with pointwise mixing), gqueeze-exciiatiinn
and Adaptive FiLM frequency gate. The FiLM gate derives per-sample channel weights from per-frequency statistics of S and k (means and log-stds over time) fused with the effective order, enabling content- and order-aware modulation. We further apply axial (time-axis) self-attention (local windowed by default) and conclude with atten- tive statistics pooling (leared weighted mean and standard deviation) and a lightweight classifier. Variable-duratio utterances are handled by dynamic padding and explicit masks passed through LFST and the encoder. While STEE itself is compact in terms of parameters, LFST trades extra front-end computation (multi-order complex convolutions per frequency band) for a more structured and interpretable time-frequency representation. Section complexity $D in the appendix quantifies this trade-off in FLOPs, latency, and memory against STFT, LEAF, SincNet, and a way2vec2-style convolutional feature encoder.
2

---

Published as a conference paper at ICLR 2026
Contributions. (1) We formulate continuum of multiplicative wavelet ensembles and develop LFST, a mathematically grounded, differentiable TF transform with fractional-order weighting, a learnable monotone log-frequency grid, and deequency-dependee eycle schedules. (2) We provide regularity conditions (admissibiliity, continuity, approximate analyticity) and numerically stable pa- rameterizations (softplus, bounded gates, log-domain aggregation) that justify stable optimization and bounded activations. (3) We integrate LFST with a simple yet effective STEE encoder, com- bining hybrid TF convolutions, Adaptive FiLM gating driven by (S, K, Ooft) ). axial time-attention, and attentive statistics pooling, for end-to-end SER from raw waveforms using focal loss with op- tional class rebalancing. (4) We evaluate on IEMOCAP (Busso al., 2008), EMO-DB Burkhardt et al. (2005), and the private NSPL-CRISE dataset under standard protocols, reporting accuracy, macro-F1, precision, and recall for fair comparability (Tharwat, 2020; Hossin Sulaiman, 2015).
RELATED WORK
Early speech emotion recognition (SER) relied on handcrafted acoustic features (prosody, spectral, voice quality) paired with conventional classifiers (SVMs, GMMs, HMMs). These pipelines estab- lished feasibility but required expert feature selection and often failed to generalize across speakers and contexts, leading to performance plateaus that motivated learned representations (El Ayadi et al., 2011; Schuller, 2018).
Deep learning models enabled end-to-end feature learning from spectrograms raw audio. Trige- orgis et al. (2016) introduced an early waw-wavefomn CNN-RNN system, while subsequent work combined 1D/2D CNNs with LSTM/GRU layers to capture spectro-temporal patterns and long- range dependencies Zhao et al. (2019). Raw-waveform architectures (e.g., CNN-n-GRU) further showed that learned time-domain filters with gating can surpass fixed spectral features Nfissi et al. (2022). Collectively, deep learning improved accuracy and reduced reliance on manual features.
Emotion-relevant cues are inherently spectro-temporal, making TF analysis central SER. STFT/mel representations impose a fixed window trade-off between temporal precision and fre- quency resolution, whereas wavelet transforms offer multiresolution analysis via scale dilation. Wavelet TF features (DWT, wavelet packets) have aided SER and can outperform STFT in some regimes Eesquez-Corres et (2016); nevertheless, conventional wavelets lose frequency discrimi- nation at higher bands because shorter wavelets contain fewer cycles, motivating more flexible TF front-ends.
Superlets (Moca et al., 2021) geometrically combine multiple wavelets with increasing cycles at common center frequency, preserving temporal acuity while sharpening frequency resolution; this super-resolution proved effective for detecting fast neural oscillations. Fractional superlets extend the order beyond integers via weighted geometric means, avoiding discrete order jumps and reduc- ing banding artifacts (Bårzan et al., 2021). However, prior superlet formulations fixed parameters (cycles, weights) heuristically and were not designed as differentiable, learnable front-ends for end- to-end training, leaving a gap our approach addresses.
Differentiable front-ends parameterize TFF decompositions and learn them jointly with the classifier. LEAF uses parametric Gabor filters and compressive pooling to approximate and then refine mel- like representations (Zeghidour & Grangier, 2021), while SincNet employs learnable sinc-based bandpass filters as transparent Fourier front-end (Ravanelli & Bengio, 2018). Wavelet-inspired layers push this further: SigWavNet learns FDWT wavelets and coefficient thresholding for SER (Nfissi et al., 2025). and multi-level wavelet packet transform with CNN/GRU proved effective for high-risk suicide calls (Nfissi et al., 2024). In contrast, our LFST leverages the superlet principle to learn pen-band frequency grids, base cycles, and fractional-order weights with multiplicative (log- domain) aggregation, yielding more flexible TF tiling than fixed bases globally parameterized filterbanks.
Large self-supervised encoders such as wav2vec 2.0 and HuBERT achieve strong SER performance after fine-tuning (Baevski et al., 2020; Hsu et al., 2021), with comprehensive evaluations reporting compute-intensive intensive and comparatively
notable gains (Wagner al., 2023). Yet these models opaque. Our physics-ing irssics-inspired LFST-STEE offers a complementary, parameter-efffiient,, and inter- pretable alternative that can operate standalone or alongside such encoders.
3

---

Published as a conference paper at ICLR 2026
Fixed front-ends (STFT, mel, CWT) impose single resolution; classical wavelets remain hand- tuned; and CNN/RNNs on fixed spectrograms inherit these compromises. Learnable front-ends (LEAF, SincNet) improve frrquency modeling but still lack continuously adaptable super- resolution mechanism across bands (Zeghidour & Grangier, 2021; Ravanelli & Bengio, 2018). Prior wavelet-hased neural approaches often predefine filter shapes or levels (Nfissi al., 2025; 2024), and traditional superiets were not differentiable within GPU-centric training (Moca et al., 2021; Bärzan al., 2021). Our work formulates fractional superlets fully differentiable, end-to-end learnable front-end. LFST thus learns amotion-tailored TF patterns with continuous trade-off be- tween time and frequency resolution and integrates with compact STEE encoder to directly serve the classification objective.
3 PROPOSED METHOD
3.1 PROBLEM SETUP AND NOTATION
We consider supervised SER from raw audio. Let R-+R be finite-energy waveform (r E L2²) sampled at rate ra; continuous time is / and the discrete index is n. Our goal is time-ffrquency (TF) representation with F bands over [mminy fmax], denoted (S,, (4) Angular frequency w=2m// convolution is *; complex conjugation is (-)". Symbols are summarized in Appendix Table 4.
Morlet CWT foundation. We use DC-corrected, approximately analytic Morlet wavelets with Gaussian envelope. For frequency I and cycle count c, with o c/ (kisa)) (default hisd his
(t)=(t) g(t: 22 - - (2*f0)' g(t:o),
g(t: exp( (xp(-+)((20))),
(1)
which enforces
(admissibility). In implementation, Morlets magnitude- normalized (L1 by default; L2 optional) and built with o c/ikaf)), exactly as the papers' modified Morlet parameterization. Wavelet coefficients and the classical scalogram are:
(t, = (x (t) E C,
[Wr,o(t)22
(2)
Using one global e fixes a uniform TF trade-off (small c: temporal acuity; large c: frequency selec- tivity), motivating multi-c constructions.
Classical superlets (integer order). (Moca et al., 2021)
(multiplicative) superlet of order o at center frequency f is the set:
SLy.0 f c c1, Co c.c... <Cp.
typically under multiplicative cycle schedule mean (GM) of the individual wavelet responses; with analytic Morlets the per-wavelet response includes the usual /2 factor (immaterial for relative magnitudes):
c1- k. The superlet response is the geometric
9
R SL
II Rleyy.u Rlpr,al = V2x /2x+v/oac
(4)
To form magnitude TF map (the SLT),
1/0
s/0(0)
(1)
II [ (0)
Lz,r, = (STT (t,w))
(5)
1=1
Integer adaptive superlets (ASLT) increase o with frequency via a rounded schedule, producing the well-known "banding."
Fractional superlets (adjacent-order mixing; not fully continuous). (Bärzan et al., 2021)
To reduce banding, fractional superlets define fractional order of o; with or E N. E [0, 1),
and mix only orders (1,..., 0i, +1:
1/05
FSLT (t,w) 11(cs
II
Ra(crktt,u)
(6)
4

---

Published as a conference paper at ICLR 2026
Magnitude per order
Geometric S[B
LAHT on S (optional)
Analytic Moriet conv for a=1.0
Mask by length provided)
Order weights W[F:O]: compute o. eff[F]
kappa[B.F.] from phase sum
X[B,1,T]
+ base cycles
phasors; weighted sum
channels:
Forward o for FILM
S2[B.2.F.T]
Kappa
Figure 1: LFST front-end. Learnable log-spaced frequencies and softmax order weights yield an
effective order Out. Magnitudes are geometrically aggregated into $ E B FxT,, phase congruency [0, 1 BxFXx comes from weighted unit phasors. length mask is applied: LAHT is applied only to S; channels are stacked as $2 = [S, ki,, and Oeff is forwarded for FiLM.
with Ry the analytic wavelet response magnitude. The order schedule oy(w) (w:) is linear without round- ing, the representation is smooth within each interval of [0 0,+1), but the participating set of cycles still changes discretely at integers; hence FSLT is adjacent-ordet piecewise, not fully continuous mixture across all orders.
3.2 LEARNABLE FRACTIONAL SUPERLET TRANSFORM (LFST)
We go beyond FSLT by learning (i) a per-band fractional mixture over orders via a simplex of weights, (ii) strictly monotone log-frequency grid with exact endpoints, and (iii) per-band base cycle count. We also produce a weighted, differentiable phase-congruency channel and apply learnable asymmetric hard-threshold (LAHT) denoiser to magnitudes only, as illustrated in Fig.
ii) Learned order weights and geometric aggregation. For each hand f and order o E (I...., O} learn logits li and softmax weights:
exp(0i.0) Wio Ed exp(Or.r)
Σ Wioo 1, wioo 0,
The LFST magnitude is the log- log-domain weighted GM: and define (t. (x 1 (t)))
(t) exp Σ wi log log(lW... ++)
Oeff (fi) Σ o E [1, O),
which strictly generalizes FSLT: instead of adjacent-order mixing, LFST learns full simplex over orders each band. Implementation details: we accumulate E. Wio log [W,,, stably (per-order streaming; no F, tensors), then exponentiate with a capped exponent (e.g.. S 20) to avoid overflow.
Complex convolution and numerics. We implement analytic convolution with real 1-D convs by convolving with (Re. S (cross-correlation equivalence), align length ("same" padding plus symmetric crop/pad), and compute magnitudes with small floor (e.g., 10-12, to avoid division by zero in unit-phasor caleulations used for k; all steps are in the released code.
(ii) Learned log-frequency grid. tributing positive deltas log-frequency:
We learn strictly increasing grid with exact endpoints by dis-
log fa log f Fmiu
85
8y x (oftplus(aa)). fi fmin- fF fmax, fis <fp. (9)
This is implemented by softplus-positiviyy normalization to (log fmax - log fmin), and a cumulative
sum; fi exp(log f.) is returned.
(iii) Learned cycle schedule. We preserve the classical multiplicative structure but learn the per- band base cycles:
(f.)) + softplus(O..)) 2 1, co(f)) = 0- 00 (f.). o= 1,....,
(10)
The Morlet time- spread a then follows o = c/ (kun)) (Eq. 15), and wavelets are DC-corrected and L1/L2 normalized before convolution.
5

---

Published as a conference paper at ICLR 2026
DW- WW--Tempral stem
Spectral ResBlock
SE20 (squezz-eexiiallon
Adaptive FILM: uses 5,kapp,,0 eef
Tomporal downsample
Axial self ittttttttiion over
Mean over frequency
Attentive pooling (mean,sto) stdd
Projection embedding
Classifter togits
S2
TF Hybrid #1
TF Hybrid
Figure 2: Encoder. DW/PW conv stem - spectral residual block - two TF-hybrid blocks with SE2D. An adaptive FiLM gate conditions on stats of S, K, and Oeff Features are temporally down- sampled, passed through axial self-attention, mean-pooled over frequency, then pooled by attentive stats yield an embedding; a linear head outputs logits.
(iv) Weighted phase congruency. same learned weights wi
We quantify cross-order phase alignment at each (fa, t) via the
(t) Σ wio (t) +
K (t)
E [0, 1].
(11)
code we compute unit phasors per order and accumulate weighted real/imag parts; the final norm is clamped to [0, 1]. к is concatenated with S to form two-channel TFF input and, together with Oeff. conditions freefrreuuncyy Adaptive FiLM gate in the encoder.
3.3 LEARNABLE ASYMMETRIC HARD THRESHOLDING (LAHT)..
Applied only to S (never k), LAHT is element-wiss smooth hard-threshold It has independent, learnable branches; since S2 0, only the positive branch is active in practice (the negative branch is kept for generality).
Thresholds. With raw parameters a, 3, b4bb..... b_ E I. bounded biases via tanh (scale mmax 20), and a small s>0,
T4 = soft
(anh(b4)) +e, T
lanh (b- )) +e,
(12)
then clamp т+ [E, Tmax]. Thresholds are shared across TF bins.
Gate. We use a stable fast-sigmoid with slope >>00: (2(2) 1 (tanh(3 2) + 1), which yields near-binary gating without discontinuities.
Mapping. For u E.R, E, let u+= max(u) u. 0) and = mwx.mux. 0). The asymmetric LAHT is:
LAHT(u) = a, - -+) u.+ - on (u. --)
(13)
Entrywise on S, small coefficients are driven toward O, while large coefficients with unit gain (since ay ++1 as u+ >> 74); because S 2,0, the second term vanishes.
3.4 SPECTRO- TEMPORAL. EMOTION ENCODER (STEE)
The LFST yiclds two-channel TF tensor S2
[S Bx2xXXX (Sec. 3.2). The STEE (Fig. 2), maps S2 to an utterance representation via lightweight, TF-aware blocks:
(1) Temporal depthwise stem. A depthwise 2D convolution along time only (1 x kt), followed by 1x1 mixing, BN, GELU, and dropout:
Xo
E a BXXXFXX
This extracts band temporal matto-pa without early brods-band mixing.
(2) Spectral residual block. A depthwise frequency convolution (ky with residual path and two 1x1 pointwise layers (BN+GELU+Dropon inside the block). This captures short-range band correlations while preserving T.

---

Published as a conference paper at ICLR 2026
eff[F]
Sigmold gate[B, C,1.1
Elementwise: gate
Linear(5->1 per-F g[B,F]
Concat per-F
features[B,FF F.5)
Linear F->C
Linear C->C
GELU
Stats_ Stats_t(S): mean, logstd
x[B,C,F,]
Stats t(kappa): mean, logstd
Figure 3: FiLM gate. Per-frequency features fuse {mean, log sodssde } of S and k with oaf via Linear(5->l), then project F C - multiplicatively modulates encoder activations.
with GELU and sigmoid to form a gate BRxxxxxx that
(3) TF-hybrid residual block + SE. A residual block that sums a depthwise (ky x1) with a depthwise (1xk) branch, followed by 1x1 mixing; then squeeze-eccitaton (SE) across (F,T) to reweight channels globally, and second identical Ty-hhbrid block. These steps learn local TF motifs (e.g., short vertical/hooizzontal ridges) and calibrate channel salience.
(4) Adaptive FiLM frequency gating (see Fig. side-information. For each frequency f form:
3). We modulate channels using LEST
o(f) [.((f, (o)o) (e(f), logor(n)(ff,, oun (f)
where (-) and a+(-) are time-mean and (unbiased) std. small MLP fuses {d(s)}f- into channel gate E (0, 1)C via a per-frequency linear, a projection R -RC GELU, and sigmoid; apply X + g o X. This conditions processing on the LFST's band-wise analysis regime (via Oeft) and order-aligned phase reliability (via k).
(5) Temporal downsampling and time-only atfention. We reduce sequence length by fixed striding along time (t > l:::;; non-learnable subsampling), then apply local multi-head self-attention along time only with window Wy. Concretely, we first average over frequency, X = (X) E BxcxT", apply ID attention on T'. This captures long-range temporal dependencies at linear cost in T" and F.
(6) Attentive statistics pooling and projection. We average over frequency, X, m E BBCCT'' then use attentive statistics pooling (ASP) over time: frame weights as softmax(w' X,) yield:
# Σ a X.,
Σ at (X; - p2 tE,
and we form [pu; Ø] z E R,,, followed by linear classifier.
R2C, A Linear- (with dropout) projects h to
Complexity. All convolutions are depthwise or 1x1. Attention is ID and local, so the dominant cost scales as OCC, F, T' + C, T' T',;), far below 2D attention's ((FT))). See Appendix Table 5 for end-to-end FLOOP//atency/ppak-meeet
profiling of front-end+STEE pairs under a shared setup.
EXPERIMENTS & RESULTS
4.1 DATASETS
IEMOCAP Busso et al. (2008): approximately 12 hours of 16 kHz multimodal dyadic interactions across 5 male-female sessions (10,039 utterances; average s). Labels include anger, happiness, sadness, and neutral (plus others), with dimensional ratings and class imbalance. Following prior work Jin et al. (2015); Kim et al. (2013), we merge happy+exciiie and exclude rare classes (disgust, fear, surprise). EMO-DB Burkhardt al. (2005): 535 studio-quality German utterances (average

---

Published as a conference paper at ICLR 2026
Table 1: Classification reports on IEMOCAP, EMO-DB, and NSPL-CRISE.
(b) EMO-DB
(c) NSPL-CRISE
(a) IEMOCAP
Class
Prec.
Rec.
FI
Class
Prec.
Roc.
F1
Anger
1.000
0.949
0.974
Class
Prec.
Roc.
Angry
0.714
0.864
0.782
/Feareea
Boredom
0.905
0.952
0.905
0.833
0.889
0.905
FCW
Angry
0.711
0.767
0.757
0.727
0.762
0.719
Neutral
Happy
0.964
0.977
0.780
0.936
0.868
0.950
Happiness
Disgust
0.909
0.824
0.952
0.933
0.930
Happy
Neurral
0.753
0.802
0.777
Snd
0.821
0.935
0.874
Neutral
0.917
0.917
0.917
0.704
0.760
0.731
Acc.
Macro avg
0.869
0.879
0,875
0.868
Sadness
0.800
0.889
0,842
Acc.
0.769
Woighted avg
0.890
0.875
0.877
Acc.
0.914
Macro avg
0.771
0.765
0,766
Macro avg
Weighted avg.
0.776
Weighted avg
0.918
0.914
0.914
5) from 10 professional actors (5 male, 5 female) simulating seven emotions: anger, boredom, dis- gust, anxiety/fear, happiness, sadness, and neutral. NSPL--CRIS:: real telephony segments (8 kHz) from one month of National Suicide Prevention Lifeline calls. With IRB approval and anonymiza- tion, trained researchers annotated the first and last calls per high-frequency caller (confidence 1-5), yielding 738 angry, 435 fearful/comeereeewvoorre (FCW), 753 happy, 738 sad, and 909 neutral.
4.2 TRAINING AND EVALUATION PROTOCOL
Setup. Audio is resampled 16 (ΣMOCCА/ЕEMO-O) or kKZz (NSPL-CRISE) and peak- normalized. LFST is initialized with Smin ss 50-60 Hz and mmax just below Nyquist Por et al. (2019) (7,600 at 16 k6z 3,800-4,000 at kHz); its exponential parametrization enforces 0 Smin S fmax < Nyquist. Variable-length inputs are batch-wise time-padded with masks padding does not affect (S, k)). LFST: K = 96 log-spaced bands, O = 8, Ked = 5, window = 1024. STEE: dy = 128; kernels h2- 9, ky- 5; three spectral residual blocks, one TF-hybrid (SE), Adaptive FiLM, axial self-attention (4 heads, window 128) after stride-8 downsampling, dropout = 0.10 in conv blocks and ASP. Training: AdamW (Ir 10-3 cosine decay; wd 10 10-3), mixed precision, elementwise clipping 1.00; focal loss (7=2) with ot x 1 1/freq(y); gradients flow through all LFST parameters and LAHT. Results averaged over 10 seeds (td) We report accuracy and Fl-ssore/precisioonreaaal on held-out 10% tests, with 80% for training and 10% for validation.
4.3 RESULTS& & STATE-OF-THE-ART (SOTA) COMPARISION
Summary, Across all corpora, LFST+STEE is accurate and well-calibrated (Table 1, Fig. 4). IEMO- CAP (4-class): Acc = 0.875, F1 0.868, 95% CIs (Ace [0.846, 0.902]. F1 [0.839, 0.897). Co- hen's 0.833. EMO-DB (7-class): Acc 0.914, Fl= 0.904, Cls (Acc [0.864, 0.957]. Fl [0.847, 0.947), ) = 0.898. NSPL-CRISE (5-class, telephony): Acc 0. 769, FI 0.766, Cls (Acc (0.725, 0.811]. Fl (0.722, 0.811), = 0.708 Small gaps between macro and weighted averages indicate the class- bias. Thus, LFST+STEE outperforms other SOTA SER methods as presented in Table 2 on the three datasets.
Per-class trends. IEMOCAP: Neutral/Sad recalls 0.936/0.935; Happy recall 0.780 with confusion to Angry (17.7%; Fig. 4b). EMO-DB: class-wise performance is uniformly strong; mild Bore- domt-Neutral ambiguity persists (e.g., 4.2% boredom nneurr),)eutral),, yet all classes exceed 0.83 F1. NSPL-CRISE: narrowband/noisy conditions lower scores; main confusions are Nea (15.9%/6.8%) and Bagry--NuutrulSSer (12.2% (Fig. 4c).
Ablation-driven reading. The learned fractional order-mixture sharpens narrowband, quasi- stationary content (boosting Neutral/Sad) while preserving temporal acuity for transients (An- grylHappy). The phase-congruency channel (k) discounts broadband impulses (fewer Happy false positives), and LAHT suppresses low-SNR TF activations, especially helpful NSPL-CRISE. The learned log- -egfrequency grid concentrates resolution near pitch/formants, aligning with strong Happy/Neutral precision IEMOCAP/EMO-DB.
Statistical validation. McNemar tests vs. a majority- baseline are decisive: p < 10 80 (IEMO- CAP), p < 10-30 (EMO-DB), p 10-10 (NSPL.), confirming that gains are not priors attributable.
8

---

Published as a conference paper at ICLR 2026
(a) EMO-DB
(b) IEMOCAP
(c) NSPL-CRISE
Anger
94.9
0.0
0.0
2.6 0.0
0.0
2.6
AMY
75.7
0.0
0.0
12.2
12.2
AraketyFear
0.0
90.5
0.0
0.0
0.0
9.5
0.0
7CW
4.5
72.7
0.0
6.8
15.9
Henpy
17.7
78.0
0.0
4.3
360
0.0
0.0
0.0
93.3
6.7
0.0
10.5
3.9
77.6
7.9
Neurral
0.0
0.0
0.0
4.2
91.7
0.0
4.2
Hassivess
0.0
0.0
0.0
0.0
95.2
0.9
0.0
5.6
93.5
4.0
12.0
0.0
8.0
76.0
Prod Lied lubel
Prud
Figure 4: Confusion matrices for emotion recognition. (a) EMO-DD, (b) IEMOCAP, and (c) NSPL- CRISE datasets. Values are in %.
Compared methods on NSPL-CRISE (DI), IEMOCAP (D2), and EMO-DB (D3). Best results in bold.
Table 2:
(a) SOTA comparison across and D2
(b) SOTA comparison on EMO-DB (D3)
Metric
Accuracy (%)
F1-seore (%)
Method
Accuracy (%)
Fl-score (%)
Dataset
D1
D2
D1
D2
Liu ctal.
Mirsamadi al.
89.13
89.4
51.3
63.5
52.1
63.8
Liu & Koxin (2022)
Minsamadi (2017)
Tuncer ct.
Li ot al.
88.35
88.35
82.1
Tuneer (2021)
Li et al. (2019)
Parlak et al.
Chen
87.2
N/A
59.6
64.8
60.2
65.2
Parlak et al. (2014)
Chon et al. (2018)
Ancilin
Zhao et al.
81.5
N/A
Ancilin (2021)
Zhao al. (2019)
LEST+STEE (ours)
91.4
90.4
LEST+STEE (ours)
4.4 BASELINES (SAME STEE)
To isolate the contribution of the front-end, all time-frequency representations (STFT, CWT, LEAF, fixed superlets, and LFST) are fed into the same STEE encoder with identical hyperparameters. This experimental design does not aim to maximize the performance of each baseline individu- ally, but instead controls the downstream capacity Se ily attributed to the front-end. Accordingly, we do not present results for fully fine-tuned self- supervised speech models (e.g., wav2vec 2.0, HuBERT), which operate under different pretrain- ing and capacity regime; integrating LFST into such SSL pipelines left for future work. Under the same STEE backbone (Table 3), the choice of front-end induces characteristic error profiles. With STFT+STEE the lower joint time-frequency concentration increases confusions between Happy+Anngry on IEMOCAP, Boredom Using STve (Morlet = 3) improves harmonic tracking but offers poorer burst acuity; consequently, pitch-driven emrors (notably Happy/Neutral) are reduced, while transient "Angry" er- rors rise. A Fixed superlet+STER front-end yields tighter TF tiles than CWT yet lacks learned order weights, leading to behavior that falls between Wavelet and L.STT. Finally, LEAF (Zeghi- dour & Grangier, 2021) +STEE, generic learnable filterbank, tends, under our compact STEE, to behave similarly to STFT.
that performance differences
be primar-
EMO-DB, and FCW-sSad/Neurral on NSPL.
CONCLUSION
We introduced LFST, learnalle fractional superlet transform front-end, paired with a compact STEE encoder for speech emotion recognition. By jointly learning the log-frequency grid, fractional order mixture, and cluse-congruency weighting under physically motivated constraints, the model adapts time- frequency resolution speech structure while remaining fully differentiable end-to- end. Capacity-matee matched ablations (STFT. CWT, fixed superlets, LEAF) indicate consistent gains, especially in challenging telephony conditions, driven sharper quasi-stationary cues, preserved

---

Published as a conference paper at ICLR 2026
Table 3: Comparison of LFST+STEE with capacity-matched baselines across three datasets.
DI NSPL
D2 IEMOCAP
D3 EMO-DB
Method
F1
Acc
STFTASTEE
73.1
72.7
84.8
84.0
89.0
88.2
WaveleeSSTE (Morlet)
74.5
74.6
84.8
90.1
89.5
Fixed superlat+STEE
74.9
74.7
86.0
85.1
90.1
89.8
LEAF-STEE
72.5
72.1
84,9
84.1
89.0
88.2
(ours)
temporal acuity, and reduced broadband artifacts. These findings suggest that learning the analysis front-end itself is an effective and interpretable route to robust SER, with promising extensions to in-the-wild data, cross-lingual transfer, and broader paralinguistic tasks.
Reproducibility Statement. All components are fully specified in the main paper: $3 details the LFST front-end, the LAHT module, and the STEE architecture. Training and evaluation settings are consolidated in $4.2 (Training and Evaluation Protocol), while metrics and statistical procedures are reported in $4.3 (Results). Additional derivations, analysis, and supporting details provided in the Appendix ($5). Source code at: https://gituub..omaaaaaiiiiiii http.aaaaoisissiiaa........ FSS-forSSE...
Acknowledgment. This work supported by the Canada Research Chair in Artificial Intelligence for Suicide Prevention (CRC-2023-00036))
LLM Usage Statement. We used large language model writing assistant for minor revisions.
All changes were reviewed and validated by the authors.
REFERENCES
J Ancilin and Milton. Improved speech emotion recognition with mel frequency magnitude coef- ficient. Applied Acoustics, 179:108046, 2021.
Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A frame- work for self-supervised learning of speech representations. Advances in neural information processing systems, 33: 2449-12460, 2020.
Harald Bårzan, Vasile Moca, Ana-Maria Ichim, and Raul Muresan. Fractional superlets. In 2020 28th European Signal Processing Conference (EUSIPCO), pp. 2220-2224. IEEE, 2021.
Felix Burkhardt, Astrid Paeschke, Miriam Rolfes, Walter F Sendlmeier, Benjamin Weiss, et al. A database of german emotional speech. In Interspeech, volume pp. 1517-1520, 2005.
Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jean- nette N Chang, Sungbok Lee, and Shrikanth Narayanan. lemocap: Interactive emotional dyadic motion capture database. Language resources and evaluation, 42:335-3550 2008.
Mingyi Chen, Xuanji He, Jing Yang, and Han Zhang, 3-d convolutional recurrent neural networks with attention model for speech emotion recognition. IEEE Signal Processing Letters, 25(10):
1440-1444, 2018.
Wei Dai, Chia Dai, Shuhui Qu, Juncheng Li, and Samarjit Das. Very deep convolutional neural networks for raw waveforms. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 421-425. IEEE, 2017.
Moataz Ayadi, Mohamed S Kamel, and Fakhri Karray. Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern recognition, 44(3):572-55) 587, 2011.
Haytham M Fayek, Margaret Lech, and Lawrence Cavedon. Evaluating deep learning architectures for speech emotion recognition. Neural Networks, 92:60-68, 2017.
Mohammad Hossin and Md Nasir Sulaiman. A review evaluation metrics for data classification evaluations. International journal of data mining & knowledge management process, 5(2):1, 2015.
10

---

Published as a conference paper at ICLR 2026
Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. Hubert: Self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM transactions audio, speech, and language processing, 29:3451-3460, 2021.
Qin Jin, Chengxin Li, Shizhe Chen, and Huimin Wu. Speech emotion recognition with acoustic and lexical features. 2015 EEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4749 4753, 2015. doi: 10.1 101100//CCASPP2205577777
Yelin Kim, Honglak Lee, and Emily Mower Provost. Deep learning for robust feature generation in audiovisual emotion recognition. In 2013 IEEE international conference acoustics, speech and signal processing, pp. 3687-3691, IEEE, 2013.
Yuanchao Li, Tianyu Zhao, Tatsuya Kawahara, et al. Improved end-to-end speech emotion recog- nition using self attention mechanism and multitask learning. In Interspeech, pp. 2803-2807, 2019.
Yunxiang Liu and Zhang Kexin. Speech emotion recognition system based on wavelet transform and multi-task learning. In 2022 7th International Conference on Intelligent Informatics and Biomedical Science (ICIIBMS), volume 7, pp. 141-149. IEEE, 2022.
Stephane Mallat, A theory for multiresolution signal decomposition: the wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 11(7):674 693, 1989.
Scyedmahdad Mirsamadi, Emad Barsoum, and Cha Zhang. Automatic speech emotion recognition using recurrent neural networks with local attention. In 2017 IEEE International conference on acoustics, speech and signal processing (ICASSP), pp. 2227-2231 IEEE, 2017.
Vasile V Moca, Harald Bårzan, Adriana Nagy- Nagy-Dābåcan, and Raul C Muresan. Time- fime-freqeency super-resolution with superlets. Nature communications, 12(1):337, 2021.
Alaa Nfissi, Wassim Bouachir, Nizar Bouguila, and Brian Mishara. Cnn-n-gru: end-to-end speech emotion recognition from raw waveform signal using enns and gated recurrent unit networks. Proceedings of the 21 IEEE International Conference on Machine Learning and Applications (ICMLA 2022). IEEE., 2022.
Alaa Nfissi, Wassim Bouachir, Nizar Bouguila, and Brian Mishara. Deep multiresolution wavelet transform for speech emotion assessment of high-risk suicide callers. IAPR Workshop on Artificial Neural Networks in Pattern Recognition, pp. 256-268. Springer, 2024.
Alaa Nfissi, Wassim Bouachir, Nizar Bouguila, and Brian Mishara. Sigwavnet: Learning multires- olution signal wavelet network for speech emotion recognition. IEEE Transactions Affective Computing, 2025.
Cevahir Parlak, Banu Diri. and Fikret Gürgen. A cross-corpus experiment in speech emotion recog- nition. SLAM@ INTERSPEECH, pp. 58-61, 2014.
Emiel Por, Maaike Van Kooten, and Vanja Sarkovie. Nyquist- sampling theorem. Leiden University, 1(1):1-2, 2019.
Mirco Ravanelli and Yoshua Bengio. Speaker recognition from waveform with sincnet. In 2018 IEEE spoken language technology workshop (SLT), pp. 1021-1028. IEEE, 2018.
Stuart Rosen. Temporal information in speech: acoustic, auditory and linguistic aspects. Philo- sophical Transactions of the Royal Society of London. Series B: Biological Sciences, 336 (278): 367-373, 1992.
Bjöm W Schuller. Speech emotion recognition: Two decades in nutshell, benchmarks, and ongoing trends. Communications of the ACM, 65()9990, 2018.
Alaa Tharwat. Classification assessment methods. Applied computing and informatics, 17(1):168-
192, 2020.
11

---

Published as a conference paper at ICLR 2026
George Trigeorgis, Mihalis A. Nicolaou, Stefanos Zafeiriou, and or W. Schuller. Adieu fea- tures? and-to-end speech emotion recognition using a deep convolutional recurrent network. JEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pp. 5200-5204, 2016.
Turker Tuncer, Sengul Dogan, and U Rajendra Acharya, Automated accurate speech emotion recog- nition system using twine shuffle pattern and iterative neighborhood component analysis tech- niques. Knowledge-Basse Based Systems, 211:106547, 2021.
Juan Camilo Vasquez-Correa, Tomas Vergar,, Juan Rafael Brozco-Arroyave Jesús Francisco Vargas-Bonilla and Elmar Noeth. Wavelet-based time-frequency representations for automatic recognition of emotions from speech. In Speech Communication; 12. ITG Symposium, pp. 1-5.
VDE, 2016.
Johannes Wagner, Andreas Triantafyllopoulos, Hagen Wierstorf, Maximilian Schmitt, Felix Burkhardt, Florian Eyben, and Bjöm W Schuller. Dawn of the transformer in speech emo- tion recognition: closing the valence gap. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(9):10745-10733 2023.
Taiba Majid Wani, Teddy Surya Gunawan, Syed Asif Ahmad Qadri, Mira Kartiwi, and Eliathamby Ambikairajah. A comprehensive review of speech emotion recognition systems. IEEE Access, 9; 47795-47814, 2021. doi: A.L10//ACCESSSS221..06333
Neil Zeghidour and David Grangier. LEAF: learnable frontend for audio classification. In Inter- national Conference on Learning Representations (ICLR), 2021.
Jianfeng Zhao, Xia Mao, and Lijiang Chen. Speech emotion recognition using deep 1d & 2d enn Istm networks. Biomedical signal processing and control, 47:312-333 11122,23,,-22 2019.
12

---

Published as a conference paper at ICLR 2026
TECHNICAL APPENDICES AND SUPPLEMENTARY MATERIAL
A NOTATION AND PRELIMINARIES
This appendix provides all technical details necessary to reproduce the Learnable Fractional Su- perlet Transform (LFST) and the Epectro-Tempora Emotion Encoder (STEE) used in our work. We first summarize our notation and assumptions, then derive the LFST from first principles, pro- vide gradient derivations, give pseudocode for the full system, and describe the datasets, training protocol, and reproducibilityy package. The source code of this work is shared in this repository: http://gghbbboommllaaaiis//////t
A.1 SIGNAL MODEL AND CONVENTIONS
Let æ R + IR be a velled, finite-energy speech waveform. Throughout we assume æ E (2R and is sampled at rate T, (Hz). Continuous time is denoted by R and discrete sample index by n E Z; the two are related by t = n//rs- We consider a finite analysis window of length L samples centred at = O. Frequencies f are in Hertz and angular frequency w fr We write convolution by (f (t)
[(T)a(( T) dr and complex conjugation by z.
The goal of LFST is to produce, for a set of F frequency bands (Ai) two maps S E BxFTT and & E [0, 1x for a batch of B waveforms of (possibly varying) length T. Here a magnitude map highlighting spectro-tempora energy and quantifies phase congruency across orders. A length mask m E {0, Bx xx' can optionally be supplied to zero-out padded positions; operations in LFST and STEE obey the mask.
Throughout we normalise Morlet wavelets by either their & or f2 norm, as specified. We denote the order of superlet by {1....; O), with O the maximum order. Table summarises all symbols.
A.2 SYMBOL TABLE
Table 4: Summary of notation. Shapes refer to the implementation with batch size B, number of frequencies F and time steps T.
Symbol
Definition/MManing
Domain/Shape
Real valued input waveform Sampling rate
or R BxIxX
positive scalar (Hz)
L.
Wavelet kernel length
odd integer samples
(t)e(t)
DC-corrected Morlet wavelet with centre frequency f and cycles
C; see Eq. (15)
g(t;o)
Gaussian envelope exp(-t² exp(-t"/(2o²) /(2o²)
Time spread of Morlet; = c/ [kea f)
positive scalar
kad
Bandwidth constant controlling trade-off
fixed constant
analysis frequency
R. strictly increasing integer
Number of frequency bands Base cycles at band Cycles of order band é oc1 Maximum order
eiffi)
2 1 (Icarnable)
co(fi)
21
O
integer (8 experiments)
weo
noomaliedo weight order o band i Effective order Σ., Wio
2. Wio
(a
1
W.,(()
Analytic wavelet response (()..)))
CBxPX
(1)
LFST magnitude at (/s, l): geometric mean of [W..ol weighted by w
R BxFT
20
(t)
Phase congruency (ft. f)
[0, 8XFXT
Optional length mask LAHT threshold hyper-parameters
{0. 1. 1xIXXT
a, B,bb
real scalars (leamed)
Number channels in STEE.
integer (128 experiments) odd integers
k., ky
Kernel sizes (timeffrequency)
Slope of LAHT sigmoid
positive scalar (fixed)
13

---

Published as a conference paper at ICLR 2026
B DERIVATION OF THE LEARNABLE FRACTIONAL SUPERLET TRANSFORM
We derive LFST starting from analytic Morlet wavelets, then classical (integer) superlets, fractional superlets, and finally our learnable construction. We give explicit admissibility, normalization, sta- bility, and diffcrentiability results, and align each step with the implemented model.
B.1 ANALYTIC MORLET WAVELETS
For target frequency > and number of cycles 1, let:
((t;o) = exp 20
kidd > 0 (we use ksd = 5).
(14)
kisd S'
Define the DC-corrected analytic Morlet:
(t),(t) g(t σ) @22s
e- ((255))² g(t;o).
(15)
=(ffa)
and Sn o(t(t) dt Admissibility (zero-mean). Using Sa g(t; e2aftt di ((2 (223/))² 2 a, we have:
J (sral)) di V2TOe )(2m5a)² - xk(s V2T 2 =0
Hence the is zero-mean (admissible). In practice, we discretize with odd window L samples: [y.c[n] = ( for n (I. 1)/2 at sampling rate rs-
Normalization.
We normalise each discrete wavelet to unit & or unit &g norm:
Fhe
p (1,
1
This equalises the fer-filter gain.
B.2 CONTINUOUS WAVELET TRANSFORM AND SCALOGRAM
For real signal æ, the analytic CWT (f, is:
Wr,o(t) (x (1))
scalogram: (()
(16)
Implementation note. The code uses 1D cross-correlation (no time-reversal) with real and imaginary parts handled separately. For analytic Morlets with nea--symmetric envelope, this differs from true convolution by negligible phase offset; exact convolution can be obtained by time-reversing the kernel.
B.3 CLASSICAL SUPERLETS (INTEGER ORDER)
Let C1 c2 < ... < (typically C c1 k). Define Rf,cu (t) = (x (t).. superlet response the (complex) geometric mean:
The order-o
1/0
R R(SL.,))
I R Rr.aa(() 1=1
S R(ST.,,))()
[ 1=1
R 1 (1) (17)
This emphasizes components consistently present across scales and suppresses scale-inconsistent energy.
B.4 FRACTIONAL SUPERLETS
Let E R and write 0f = o, + a with o; == [or] E N, (0, 1). fractional superlet interpolates between orders o, and +:
1/09
04 II
FSLT (t) (Bi,aioutn())
(ssea(t))
(18)
k=1
This removes integer-order banding but still changes the participating integer boundaries.
14

---

Published as a conference paper at ICLR 2026
B.5 LEARNABLE FRACTIONAL SUPERLET TRANSFORM (LFST)
To learn the time-frequency tradeoff from data and avoid piecewise mixing, we introduce smooth, learnable order weights.
Learnable order weights.
At each band f: we learn logits 0,,1:0 and set:
8..
Wio
Eo= eii
Σ
wio0 1, wio 2 0.
(19)
This yields smooth weighted geometric mean (below) and an effective order:
Oct (fi)
Σ o wi [1,0].
(20)
Learnable frequency grid (monotone log-spacing). crements on the log-scale. Given parameters 0s.1:(F-1)> 1), define 6j softplus (0ft)) > 0 and:
Let fmin fmax and learn positive in-
A =
(log fmax - log funi);
= 1, F 1,
(21)
8x
k=1
log ft = log fmin +
Σ Dイ i= 1 F,
(empty sum = 0),
(22)
f: exp fi).
(23)
Lemma (monotonicity and exact endpoints). If fmax > fmin. then A, > 0 and log fi log fmins log fr = log fmax. and log fi <... < log fF. Proof. 8, > + Aj >> 0 and the cumulative sum telescopes to log fmax at = F; strictly positive increments ensure strict monotonicity.
Learnable cycle schedule. For band i, set:
(f)) + softplus(Pes) (2 1),
= -
(24)
LEST magnitude and phase congruency.
Let Wi,(() = (o * (f)oo With e > 0 small,
0
Σ
wio log (W +5) (s)
St. = exp
(25)
(r,(() WW. (t) +E
Ky (t)
Σ Wio0
E [0, 1].
(26)
Equation (25) is the weighted geometric mean of stabilized magnitudes; (26) measures phase align- ment across orders.
Basic properties.
(Scaling) For any A > O, W.. scales linearly: [r,] A A Hence log( W|++) shifts by log A (for [W] > e), and 5);(1) (t) scales approximately by A:
s/A=0 (1) exp Σ wio log( AlW + +s)) ~ (1)
(Range of k) Each summand in (26) is a vector of norm in (0, 1]. By the triangle inequality and Σ. Wi.o 1, < 1. Nonnegativity clear.
(Concentration) By Jensen, exp(>, Wioo log m.) S E. w. for m. 0, the geometric mean never larger than the arithmetic mean; this penalizes outlier magnitudes and concentrates persistent energy.
15

---

Published as a conference paper at ICLR 2026
B.6 DIFFERENTIABILITY AND GRADIENTS
All parameterizations use smooth maps (exponentil/sstill///sssmmma)))) so S and are C in both and We record useful derivatives.
Gradients w.r.t order logits. Let (i,o(t) log( (i,((t) E) and denote (g); E. Wi.oDi.o- Then:
(1 (1) wio (glo(!) (1): a
asy, (1)
S (5) Ag
(27)
i8.,o
where Ag max, Ii,o min, Ii.o is finite when z and wavelets are bounded. This is the standard softmax-log-mest lealllog-mmnngomm gradient; it numerically stable.
Gradients w.r.t. convolutional parameters.
For any scalar parameter Ç of sn.co (e.g. fi)
aS Sy, (t)
1
alWi,o(!)
wio
(28)
O
[W),(1)] +E
Oç
and
RW
OW o
oWi.o
(for W > 0),
x *
(29)
O
W
Oc
O
Thus only 06/0C is needed. Writing w = 2nf, n expl
(w we obtain:
au
9 (j2rt e )
8k
as
g.
(30)
- (2x) w
do
00
(g (clvt -x) k3
9
0x
ae
do
0g
t
(31)
Chain rule handles 61 and f via Kaff with co oct with (ft) = + softplus(B... (Be.). The frequency-grid derivatives follow from fa exp(log fi) and the cumulative-softplus construction of log fi.
dc f and 0f
k f and
Differentiability of K. Using K. -1200 w/o Unol12 with U² = Wi.o/ ((W + E),
Ōk
R E. wio U2,0 E. wio
8U.m) dc
OU
(IWI + E) aw
W W oc
8c
E. Wioo U2,ol12
Oç =
( + E)2
All terms smooth due to 0.
B.7 STABILITY (LIPSCHITZ BOUNDS)
Let || be the sup norm and assume unit-l, wavelet normalization (the Eg case is analogous). By Young's 's inequality, ||Wtollos < || || log I 1/c, so for each oi,o,t) t).
|| Moreover, on [e, || + e] the slope of
Lip(z + log(IW... e)) :
As E. wioo 1, the weighted sum has the same bound, and exp has slope at most || on the image interval. Therefore:
||0||00 +
Lip(x Sy) <
(32)
and x + Ky,, (t) also Lipschitz due to bounded, smooth composition. Choosing not too small improves worst-case constants while preserving sensitivity.
16

---

Published as a conference paper at ICLR 2026
B.8 COMPLEXITY AND MEMORY
For batch size B, F bands, order cap O, and T time steps, LFST performs, per order, two real ID correlations for F filters, i.e. O(BFT) MACs per order; streaming over O gives O(BFOT) time and O(BF T) activation memory (since order accumulation is in-place). This matches the implementation, which allocates reusable buffers and collapses the order loop.
B.9 IMPLEMENTATION ALIGNMENT AND NUMERICAL NOTES
Cross-correlation vs. convolution. The code uses cross-correlation with (Ry) - -Sa);; ex- act convolution be emulated by time-reversing tb. The effect on analytic Morlets is negligible in practice.
Stabilization. The code adds small constant (10 (10-12) 12) inside magnitude computations and caps the exponent in (25) before exponentiation to avoid overflow; these are now docu- mented.
Normalization. Both 4 and lz wavelet normalizations supported; experiments use & unless stated otherwise.
Frequency endpoints. The log-grid construction ensures monotonicity and exact end- points provided log fmax >> log fmin. In practice, parameterize log fmax log fmin + softplus(7) (7) to guarantee this.
Phase congruency channel. x is computed from unit phasors and not denoised by LAHT; only passed through LAHIT in the implementation (as assumed here).
Learnable Asymmetric Hard Thresholding (LAHT). To denoise 5 we use smooth hard- threshold with asymmetric positive/negative thresholds. Let u element of S). Given raw learnable parameters (a, B, b,, b-) and fixed constants bmax 0, > 0. define:
a sealar input (here, an
T+ = softplus softplus(a) + bmax tanh(64)) +E,
(33)
T_ = softplus softplus(33) + bmax tanh(b-)) + +
(34)
so that T+ > 0 always. Write u4 = max(1, 0), u max(-u, 0) and let:
(2(2)= 1/2 (tanh(3=) +1).
d4(x)) 7 sech (3²).
with slope parameter 7 > 0. The LAHT mapping is:
LAHT(u) = o. ( T+)
o - u_
(35)
branch
negative branch
Since S>0 only the positive branch is active practice.
BASIC PROPERTIES
Lemma 1 (Positivity, asymmetry, and boundedness). For any E R, > 0. and > 0, the map LAHT satisfies: (i) LAHT(") E [-|u], |u]] and sigh(LAHT(uu) = sign(u) for u + 0; (ii) [AA is nondecreasing in on [0, oo) and nonincreasing on (-o0, 0]: (iii) if 7+ T_ then
LAHT is an odd map, LAHT(- (u) = = -LAHT(u).
Sketch. (i) o.y [0, and d 2 hence each branch has magnitude at most |u] and correct sign.
(ii) On u 2 0, LAHT(2)= = (u- u with derivative (below) nonnegative; symmetry yields the negative side. (iii) follows by replacing T-) with a common value and noting the symmetric form of (35).
Proposition tHand-threshold limit), As 2 + 0c,
u, u 2 T42
0, 0 < u < T+ -T- 0,
LAHT(w)
u, u S -T-,
17

---

Published as a conference paper at ICLR 2026
i.e. LAHT converges pointwise to asymmetric hard threshold.
Proof. 0z(2) + = > 0} pointwise as + oc. Substitute into (35).
GRADIENTS W.R.T. THE INPUT
Using u4 = max(u, 0) and u = max - 0), for u > O,
i (LHTT(u) 0, T-) u + (2 T+);
(36)
Ou
ST/4
and for < 0 (recall u_ -().
0 LLAHT(u) du --------- -T- (y(uu-T)) T-).
(37)
Ou
At = 0, both branches vanish and the one-sided derivatives equal 0 (from the right) and 0,( T-) (from the left); thus LAHT is continuous at 0 and C* at 0 iff 74 = T-. In our use (S 2 0) only (36) matters.
Slope bounds and Lipschitz constant. Since of S 7/4 4 for all a and 0 S S 1.
@LAHT(u) 0 Σ
$ 1 ?
4 44
(u 0).
du
If inputs satisfy |u| s Umax (as they do when = (t) with bounded signals), then:
Lip(LAHT) s 1+ 4 Umax-
2
Moreover, far above threshold (u T+) the derivative approaches far below, it approaches 0, so LAHT behaves like a near-identity on strong components and a near-zero map on weak ones.
GRADIENTS W.R.T. THRESHOLD PARAMETERS
Let 24 U4 T4 and z_ = u T.. From (35),
LAHT (2+)))
LAHT
ÔT_ + (2-
(38)
θT+
Now differentiate thresholds by the chain rule. Writing s(x) softplus(x) and (zx) Tte-0
θT+ of (((()) bmax tanh(b+)) a(o),
0T+ (s(()) + bmax tanh(b-)) bmax sech bech2(b+);
da
Ob+
(39)
Ør_ = (s((3)= bmax tanh(b-)) o(3),
Ør_ o(s(3) + Umax tanh(b-)) bmax sech (bch?(b.)..
03
ab_
(40)
Combining with (38) yields
OLAHT
- 02,(2+) u4 of (s(()= + bmax tanl((b4)) o(a),
(41)
da
i LAHT
= d2(2+) u4 ((s(a) + Imax tanh(b+)) bmax sech (b+),
(42)
0b
LAAT
+ d, u of ((() bmax tanh o(3),
(43)
03
LLAHT
ab_ = d2(22- of s(3) bmax tanh(b b soch 2 (b.).
(44)
Implementation alignment. The code uses exactly this "double- structure and tanh- bounded bias terms (clamped in bmaxe bmax)) to keep thresholds positive, smooth, and numerically stable.
18

---

Published as a conference paper at ICLR 2026
INTERPRETATION AND EFFECT ON S
For S 2 0, LAHT reduces S - -
Bias-variance tradeoff: 7+ sets the denoising boundary; larger 74 suppresses low- energy bins (lower variance) at the risk of discarding faint but real structure (higher bias).
Soft hard threshold: Near threshold, the multiplicative gate a.y -T4) rapidly transitions from 0 to 1; away from threshold, the map near-identity.
Asymmetry capacity: Although S 2 0 in our pipeline, LAHT supports different 7
useful in contexts with signed u.
IMPLEMENTATION ALIGNMENT
The code uses the exact double-softluus tanh parameterization above (with thresholds softly bounded and & added) and fixes "y.
LAHT applied to S (nonnegative), the к channel bypasses LAHT (as used by FiLM).
Thresholds are clamped to safe numerical range, preventing exploding gates.
C ALGORITHMIC SPECIFICATION
We provide pseudocode for LFST (Algorithm 1), LAHT (Algorithm 2) and the STEE encoder (Al- gorithm 3). All loops are over orders and frequency bins; the implementation streams over orders to avoid storing tensors of shape [B, F, O,T].
Algorithm LFST forward pass (single batch of B signals)
Input Batch E @8xIxT length mask m E {0, 1x parameters θ defining frequenciess [fi base cycles {cr (ff)] and logits 0,.0-
Output: Magnitude map S BBxxx effective orders oet RF. phase congruency
[0, 1] 1]
Compute frequencies {fi} via softplus-normalieed increments (Sec. B). Compute base cycles (1... + softplus(We.)) and order cycles co(fi) oci(fi) for
1, O.
Compute weights Wi.o softmax,(0s..) and effective orders oct (fi) = E. 0i Initialise accumulators log + 0 E BxFxxT Re +- Im n. 0.
for o 1 to O do
for i to F
L Construct analytic Morlet filter (f.e)(f)) (Eq. (15)) of length L and normalise.
Convolve with Wh.e, using real ID convolutions to obtain W. E CB (real part
RW, imaginary part SW). Align output length by symmetric cropping or padding.
Compute magnitude !W..ol 2 + ( + E and unit phasors Uioo !...l"
W...
Accumulate log- maggittttuds:: log w log t log W (broadcast weights over B and
7).
Accumulate phasor components: Re Re + w. Re Imx Im + w. Im o.
Compute 5 exp (eppmmn((e) log)) (cap exponent to avoid overflow). Compute к = min( / /RRck) + (Im k)? 1).
Apply mask m to S and i (elementwise multiplication).
Return (S,
Ocff, (S,0etf, x)).
D COMPLEXITY BENCHMARK (FLOPs, LATENCY, MEMORY)
To quantify the computational trade-off of the proposed LFST front-end, benchmark several front-end + STEE configurations under a shared profiling setup. We report FLOPs (multiply-adds), mean inference latency (averaged over 100 runs after warm-up), and peak GPU memory during
19

---

Published as a conference paper at ICLR 2026
Algorithm 2 LAHT mapping (vectorised over tensor U)
Input Tensor E I (arbitrary shape), learnable raw parameters (a, 3, b4, b-), slope y and
bounds E
Output: Thresholded tensor V of same shape
Compute T. = softplus(softplus((a)) + Umax tanh(b+)) + and T_ softplus(softplus(() +
mmax tanh(b_)) +
Split into positive and negative parts: U+ = max(U, 0), U- = max(-U,,))
Define fast sigmoid (z)(2) 2 (Lanh( 22) 1).
Compute gating functions: G = 0g (U+ - T+), G- = 0,(U- - T-)-
Return V
G+ U+ G U_.
Algorithm Spectro-Tempora Emotion Encoder (STEE) with FiLM and Axial Attention Input Magnitude S E BxFT" phase congruency | 1]_xPXT and encoder parameters.
effective orders Oeff E R
Output: Utterance embedding RR and logits y E RBx for C classes.
Stack magnitude and phase channels: S2 cat(S, k) Bx2xPxT.
(1) Temporal depthwise stem. Apply a depthwise x k, convolution on S2 followed by pointwise
1 mixing, batch normalisation, GELU, and dropout obtain Xo BB
(2) Spectral residual block. Apply a depthwise (ky x 1) convolution, pointwise mixing, batch normalisation, GELU and dropout; add the residual input to obtain X1,
(3) TF-hybrid blocks and squeeze -xcitattenn Apply a depthwise (ky 11) branch and a depthwise
(1 ke)) branch in parallel, sum the outputs, project by a pointwise layer, add the residual (X₃ and apply batch normalisation, GELU and dropout; call this X2. Apply squeeze-excitation (two 1 x convolutions with GELU and sigmoid) to obtain X3; apply a second TF-hybrid block to obtain X (4) Adaptive FiLM gate. Compute per- per-frequency statistics S, log (S), log o(x) over time and fuse them with Oed via linear layer to produce a gate g E 1) BXC, Multiply X, by g broad- casting over F and T.
(5) Temporal downsampling and axial self-attention. Subsample along time by a fixed stride; average over frequency to obtain E IR BxCxT"; apply local multi-head self-attention along time to capture long-range dependencies; expand the attended features back over frequency. (6) Attentive statistics pooling and projection. Average the attended features over frequency to obtain X, E I BxCxT"; compute per-frame weights by a ID convolution and softmax; form weighted mean μ and standard deviation across time; concatenate [μ; o] Bx2x, project through linear layer, layer normalisation, GELU and dropout to obtain the embedding z E IR BxD, if classifier is present, project logits E R BxC
return (z, y).
forward pass, using a s input at 16 kHz All front-ends are paired with the same STEE encoder to keep downstream capacity fixed.
Table 5: Complexity benchmark (I input at 16kHz; shared profiling setup) for front-end-STEE pairs.
Model
FL.PPs (GF)
Peak memory (MB)
Latency (ms)
STFT STEE
0.36
18.7
2.2
SincNet + STEE
19.8
504.6
8.6
LEAF STEE
44.5
1156.0
15.7
Wav2Voc2-fcat + STEE
15.4
514,9
3.3
LFST STEE
201.5
4532.8
74,9
Wavelet + STEE
179.6
4533.2
109.4
FixedSuperlet + STEE
202.7
4533.2
75.4
Table 5 makes the computational positioning explicit: LFST+STEE is not "lightweight" in FLOPs, latency, or memory when compared to standard and learned front-ends. Any "lightweight" charac- terization should be understood referring the parameter budget of the STEE encoder (relative
20

---

Published as a conference paper at ICLR 2026
to large supe--se backbones), while LFST provides richer, structured time time--imeeeeee frequency rep- resentation at a substantially higher computational cost.
Key takeaways from the benchmark.
LFST vs. STFT (classical baseline). LFST+STEE requires 201.5 GFLOPs versus 0.36 GFLOPs for STFT+STEE, with 74.9 ms vs 2.2 ms latency, and 4532.8 MB vs 18.7 peak memory. This quantifies the cost of replacing FFT-based front-end with multi- order waveledssperrlet-sty.. TF analysis.
LFST vs. other learnable front-ends (SineNet, LEAF). Relative to LEAF+STEE, LFST+STEE uses 4.5x more FLOPs (201.5 vs 44.5), 4.8x higher latency (74.9 vs 15.7 ms). and ~ 3.9x higher peak memory (4532.8 vs MBB.. Relative to Sinc- Net+STEE, LFST is about 10.2 higher in FLOPs and 8.7x higher in latency, with much larger memory footprint. This situates LFST as high-cost, high-structure TF front-end.
Peak memory as the main practical constraint for TF-map-based wavelet/superlet front-ends. L.FS//Wavele/FixedSupeeee all reach R 4.53 GB peak GPU memory, indi- cating that the dense TF representation and multi-order computation dominate memory usage under fixed STEE trunk. In practice, this directly impacts feasible batch size and hardware requirements.
Where LFST's cost comes from (Wavelet and FixedSuperlet comparisons). Wavelet+STEE GFLOPs, ms) and FixedSuperlet+STED (202.7 GFLOPs, 75.4 ms) fall the same compute regime as LFST+STEE (201.5 GFLOPs, 74.9 ms), and all three share nearly identical peak memory (= 4.53 GB). The near-matching latency be- tween LFST and FixedSuperlet suggests that much the runtime cost is intrinsic to multi- order superlet-style TF computation (and the resulting TF map size), rather than being introduced solely by LFST's learnability.
(2LLS
Interpreting Vaw2 in this table. Wav2Vec2-feat-STT (15.4 GFLOPs, ms. 514.9 MB) is included here front-end feature extractor paired with the same STEE encoder, not as a full end-to-end self-supervised pipeline comparison. Its lower cost highlights that LFST targets different design point: mathematically grounded, interpretable TF front-end whose contribution is assessed under controlled downstream capacity.
Table 5 clarifies the cost/structure trade-off: LFST+STEE is more expensive than STFT//ineNew/LEAF in FLOPs, latency, and peak memory under the same encoder, while Wavelet and FixedSuperlet baselines show that this cost reflects the computational nature of multi-order waveletsuuperlet-styll TF analyses. These measurements provide an explicit resource context alongside the performance results.
E INTERPRETABILITY VISUAL: LEARNED FRACTIONAL ORDER DISTRIBUTION vs. FREQUENCY
This section visualizes the learned frequency-dependeet fractional-order behavior LFST, by di- rectly inspecting the trained LFST parameters. Concretely, LFST learns (i) frequency grid (fi) and (ii) order-mixing weights {w (a softmax distribution over discrete orders each fre- quency). For each frequency fú, we summarize the learned order distribution via the effective order:
)
Oct (Jt) = Σ ow
Wf.oo = softmax(00,..00...
(45)
ie., the expectation of the discrete order under the learned softmax weights. This produces a single, interpretable scalar per frequency that captures how LEST allocates multi-order analysis capacity across the spectrum.
The plot is obtained by extracting the leamed LFST frequency grid and the order logits from trained variant checkpoint our model, converting the logits into probabilities via a softmax over orders, and then computing oct (f) in Eq. (45). We also show the full distribution wfo
21

---

Published as a conference paper at ICLR 2026
Distribution vs Frequency
10.3
10.2
Himmee 21
Hamor
96-
Effective
6.2
1.2
Order Weight Distribution (Model 1)
0.12
18
0.1
021
0.2
157
10a
Prequency (Hz)
Figure 5: Learned fractional-order behavior as function of frequency for a representative trained LFST model. Top: Effective order oen (f) (Eq. (45)) versus frequency (log-scale), with shaded bands indicating typical speech regions: pitch range (FO) and formant regions (F1-F3). Bottom: Full order-weight distribution wfoo as heatmap (order vs. frequency), where brighter values indicate higher probability mass assigned to that order at that frequency.
heatmap, providing complete view of how probability mass shifts across orders as function of frequency. The main experiments use O 8, this visualization uses a representative model trained with O 16 to show finer-grained order distribution. The qualitative trend is consistent.
Figure highlights structured, frequency-dependeet allocation of orders:
Non-uniform order allocation across frequency. The effective order varies substantially across the spectrum (rather than remaining flat), indicating that LFST does not converge to a trivial uniform-order behavior. Instead, learns a frequency-dependent resolution strategy.
Effective order increases from the low-frequency region into vowel/formant- dominated bands. The top panel shows comparatively lower our (f) at very low frequen- cies (within typical pitch/FO ranges) and a clear increase as frequency moves into mid-band regions associated with vowel/formant structure (FI-F3). This is consistent with allocating relatively more spectral concentration capacity in bands where speech structure benefits from sharper frequency localization (c.g., formant-related cues), while avoiding a uniform allocation across all frequeneies. Note that the annotated FO and Fl regions partially over- lap (c.g., 300-400 Hz), reflecting that harmonic and formant structure can co-occur in that range.
Lower-to-moderate effective orders in the very low-frequency region. In the lower- frequency region (including typical pitch/FO ranges), the effective order is comparatively lower than in the mid/high speech bands. This corresponds to shifting capacity toward better temporal precision (shorter effective analysis support), aligning with the importance of capturing prosodic dynamics such as pitch movements, voicing transitions, and timing- related emotional cues.
Heatmap reveals fractional mixtures and smooth redistribution of order mass. The bottom panel shows that the distribution over orders is generally not concentrated at a single order across all frequencies. Instead, probability mass is distributed over multiple orders and redistributes smoothly with frequency (forming coherent "ridges"), indicating that LFST learns fractional mixtures rather than selecting single discrete order per band.
22

---

Published as a conference paper at ICLR 2026
This behavior matches the intended continuous/fractional design of the LFST formulation and provides a direct window into how analysis resolution is adapted across the spectrum.
These order patterns global LFST parameters shared across all utterances in trained model (not per-speaker or per-utterance settings). As a result, the figure provides a direct, model-level explanation of how LFST adapts its multi-order fieeueryyy analysis across frequency bands: learns a non-uniform, prequency-dependee redistribution of order mass, increasing effective order in bands where sharper frequency localization is useful for speech structure (e.g.. formant-related regions), while keeping comparatively lower effective order at very low frequencies where temporal dynamics and prosody dominate. This makes the learned TF front-end behavior transparent and directly tied to known speech acoustics.
F ABLATION: PHASE CONGRUENCY VS. LAHT
This ablation disentangles the contributions of the phase-congruency channel R. and the learnable asymmetric hard-thresholding (LAHT) module on the NSPL-CRISE corpus. NSPL-CRISE a par- ticularly relevant stress-test for this analysis because it is a telephone-domain dataset with substantial channel variability and background noise, where (i) phase-aligned structure can provide complemen- mechanisms can directly impact robust-
tary cues beyond magnitude, and (ii) sparsifying/denossin ness. We remove each component turn while keeping the rest of the LFST+STEE architecture, training protocol, and evaluation procedure unchanged: (i) LFST without uses magnitude-only LFST representation with LAHT kept, (ii) LFST without LAHT keeps but disables thresholding, and (iii) the fiall model uses both к and LAHT. This design yields controlled assessment of the marginal utility of each component under otherwise identical conditions.
Table 6: NSPL-CRISE ablation isolating the phase-congruency channel (xs) and LAHT within LFST+STEE.
Variant
Acc (%)
F1 (%)
LFST without R. (LAHT kept)
67.2
66.9
LFST without LAHT (k kept)
74.3
74.1
LFST (full model: x LAHT)
76.9
76.6
Effect of adding x (phase congruency). Comparing LFST without x (67.2/ to full model (76.9/ 76.6) shows a gain of +9.7 pp in accuracy and +9.7 pp in F1 when introducing K on top of LFST+LAHT. This indicates that k accounts for a substantial portion of the overall improvement on NSPL-CRISE. Intuitively, к complements magnitude S by emphasizing locally phase-aligned structure in the LFST representation. Such phase-aligned structure can sharpen the representation of coherent speech events (e.g., voiced segments, harmonic stacks, and transient onsets) that often earry emotion-related prosodic signatures, while being less sensitive to diffuse, incoherent background noise. In this sense, k acts as an additional "structure" channel that makes discriminative TF patterns more salient to the STEE encoder under challenging acoustic conditions.
Effect of LAHT (thresholding). Comparing LFST without LAHT (74.3 / 74.1) to the full model (76.9 / 76.6) yields smaller but consistent gain of +2.6 pp in accuracy and +2.5 pp in F1 when adding LAHT on top of LFST+*. This supports the interpretation of LAHT as learned sparsifying denoiser operating on TF activations: it suppresses low-energy clutter and weak, spatially diffuse responses that are common in telephone noise and channel artefacts, while preserving higher-energy. coherent TF structures that are more likely encode emotion-relevant information. Importantly, the improvement suggests that LAHT is not over- wer-pruning informative content in this domain; instead, its bounded, learnable thresholds settle into a regime where representation contrast and robustness improve.
Interaction between x and LAHT. The progression of results also suggests a complementary in- teraction: yields the dominant improvement by enriching the representation with chase-conssstent structure, while LAHT provides an additional refinement step that increases the effective signal-to- noise ratio of the resulting two-channel map. In other words, x primarily increases informativeness
23

---

Published as a conference paper at ICLR 2026
of the representation, whereas LAHT primarily increases its robustness by removing residual low- energy artefacts. This division of roles is consistent with the magnitude of gains observed in Table 6.
The ablation indicates that is the dominant contributor among the two components NSPL- CRISE, providing a large boost by introducing phase-consistency information, while LAHT yields an additional incremental improvement by sharpening the TF representation through learned sup- pression of low-energy clutter. The complete LFST**+LAHT configuration achieves the best accu- racy and Fl on NSPL-CRISE.
G DATASETS AND PREPROCESSING
We evaluate method on three corpora: IEMOCAP, EMO-DB and NSPL-CRISE. All audio is converted to mono, normalised to peak magnitude and resampled to the task-specific sample rate (16 kHz for IEMOCAP and EMO-DB; 8 kHz for NSPL-CRISE). We summarise key statistics in Table
Table 7: Speech emotion recognition datasets used in our experiments. "Utters" denotes the number of utterances. We follow standard class mappings and test protocols from the literature.
Dataset
Sampling rate
Utters
Classes (after mapping)
IEMOCAP
(Busso
kHz
039
angry. happy (excited merged), neutral, sad
al., 2008)
EMO-DB (Burkhardt
kH
anger, boredom, disgust, anxiety/fear, happiness, sadness, neutral angry. fearfulcomccrmcd/woriill happy, sad, neutral
et al., 2005)
NSPL-CRISE
8kHz
2999
For NSPL-CRISE, labels were derived from the first and last calls of high- freg-fryy callers on the National Suicide Prevention Lifeline over one month, with IRB approval and anonymisation. Calls were annotated by trained raters on 5-point confidence scale; we discarded low-confidence samples.
Preprocessing. For each dataset, we pad shorter utterances with zeros to the longest length in the batch and provide a binary mask to LFST and STEE so that padding does not influence magnitude phase. On each utterance, length L, and kisd = 5. The minimal frequency fmin and maximal fiax are initialised in the range [50, 60] Hz and just below Nyquist, respectively; these endpoints are learnable but constrained to remain in [0, Nyquist). The base cycles ei (ft) are initialised to cycles, and orders Wioo are initialised uniformly.
compute the LFST with F frequency bands, O orders, base kernel
H TRAINING AND EVALUATION PROTOCOL
We train LFST and STEE end-to-end using the AdamW optimiser (learning rate 10-3 with cosine decay, weight decay 10 10-+). Training uses mixed precision and gradient clipping at #1 to prevent exploding gradients. The loss is the class-balanced focal loss with focusing parameter y 2 and per-class weights Cxy x 1/freq(y). For each dataset, we split the utterances into vain/validation/test in 80/10/10 ratio, stratified by class. We average results over 10 random seeds and report the mean and standard deviation. All reported metrics (accuracy, Fl-score, precision, recall, Cohen's 's x) computed the held-out test sets. Confidence intervals (95%) obtained by bootstrapping test predictions.
I REPRODUCIBILITY
To reproduce our results, clone the anonymous repository and install the required dependencies (PyTorch 2.2 later, Python 3.10, NumPy, SciPy, scikit-lear). Experiments were run NVIDIA A100 GPUs with CUDA 11.8. Table lists hyperparameter examples. We fix random seeds (c.g., 1234) for NumPy and Py Torch before data loading.
24

---

Published as a conference paper at ICLR 2026
Table Main hyper-paramelers used in our experiments.
Component
Parameter
Value
Notes
LFST
Number of bands F
96
log-spaced, learnable
Maximum order O
weights antmaxx-normalied odd, symmetric padding
Window length L
1024
ked
Morlet bandwidth constant
10
stability constant
Initial cq
1.5
cycles per band
LAHT
7
sigmoid slope bias bound base width
bmax
STEE
Channels C
128
Kernel sizes kr,
9,5
odd for symmetry local window 128 steps training only
Axial attention heads
Dropout rate
0.10
Training
Optimiser
Adam W
Ir 10 cosine decay
Batch size
16
variable per dataset early stopping on validation loss
Epochs
LIMITATIONS AND KNOWN FAILURE MODES
While LFST improves frequency flexibility relative to fixed front-ends, certain limitations re- main. (i) The multiplicative nature of the superlet aggregation emphasizes signals present across all orders; very short transients may be attenuated. Increasing the number orders O or learning order-dependent ç₁ mitigate this, but increases cost. (ii) The learnable frequency grid uses an explicit parameterization to enforce a valid interval (fmin and fmax > fuin) and strict mono- tonicity of the log- frequency tiling; nevertheless, extreme endpoint choices can still reduce coverage PyTorch
of informative bands, so we monitor learned endpoints during training. (iii) Convolution implemented as cross-correlation; for asymmetric wavelets, this differs from the mathematical convolution by time reversal. Our analytic Morlet is approximately symmetric in its envelope. so the effect is negligible, but true convolution could be implemented by reversing the filters. (iv) The system is trained on limited datasets; generalising to other languages or recording conditions may require retraining and careful data augmentation. (v) Our experiments are designed as controlled front-end study: all time-frequency representations are evaluated under the same compact STEE backbone. We therefore do not provide comprehensive comparison against fully fine-tuned self- supervised models (e.g., wav2vee 2.0, HuBERT), which follow different pretraining and capacity regime; evaluating LFST within such SSL pipelines (e.g., as a replacement or auxiliary front-end) is left for future work.
25


Reviews:
[{"review_id": "KtTbjiy5SZ", "reviewer": "Reviewer_9ouD", "summary": "This paper proposes a Learnable Fractional Superlet Transform (LFST) for speech emotion recognition, extending conventional superlets into a continuous fractional-order form. By applying a softmax-weighted geometric mean across multiple Morlet wavelet responses, LFST enables differentiable and data-driven time–frequency resolution adaptation. The authors jointly learn the log-frequency grid, base cycles, and order weights, combined with a phase-consistency measure and a Learnable Asymmetric Hard Threshold (LAHT) for denoising. A Spectro-Temporal Efficient Encoder (STEE) further integrates depthwise-separable convolutions, TF-hybrid residuals, Adaptive FiLM frequency gating, axial attention, and adaptive pooling. Experiments on IEMOCAP, EMO-DB, and NSPL-CRISE show consistent gains in accuracy and F1-score over prior handcrafted and learnable front-ends such as STFT, LEAF, and fixed superlets.", "questions": "1. How much does phase-consistency κ contribute independently of LAHT? Have the authors tested models without κ or LAHT separately to quantify their individual benefits?\n2. Are the learned thresholds stable across sampling rates and speakers, or do they overfit particular acoustic domains?\n3. The appendix mentions a softplus constraint to ensure fmax>fmin. Did early training ever show instability in frequency spacing, and how was it mitigated?\n4. Any evidence that the model adapts meaningfully rather than overfitting individual timbre patterns?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The fractional-order superlet design generalizes discrete-scale superlets into a learnable, continuous formulation with theoretically grounded differentiability and numerical stability.\n2. Joint optimization of frequency grid, base cycles, and order weights, coupled with Adaptive FiLM gating, leads to a lightweight yet expressive front-end that achieves strong results with modest computation.\n3. The work contributes a new class of differentiable TF front-ends that retain physical meaning, offering a bridge between signal processing and deep learning for audio emotion understanding.", "weaknesses": "1. While the paper claims interpretability, there is no visualization of learned order weights across frequency bands.\n2. Although lightweight in design, there are no quantitative FLOPs or latency comparisons with LEAF, SincNet, or wav2vec2.\n3. Comparing against a fixed fractional-order (non-learnable) baseline could isolate the effect of learnability versus the fractional formulation itself.\n4. The datasets are small and homogeneous. Additional results under domain shifts (e.g., cross-language or different SNR levels) would test the claimed generalization.\n5. Adding comparative TF heatmaps (STFT vs. fixed Superlet vs. LFST) for the same utterance would intuitively show the sharper band selectivity and phase consistency gained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TYEN28iJG8", "reviewer": "Reviewer_YGVV", "summary": "The paper proposes the Learnable Fractional Superlet Transform (LFST) combined with a Spectro-Temporal Emotion Encoder (STEE) for end-to-end speech emotion recognition (SER). LFST generalizes superlet theory to learnable fractional orders, enabling continuous trade-offs between time and frequency resolution. It also introduces a learnable asymmetric hard-thresholding (LAHT) module for sparse denoising. Extensive experiments on IEMOCAP, EMO-DB, and NSPL-CRISE demonstrate strong performance, surpassing state-of-the-art baselines with high interpretability and stability.", "questions": "1. How does the LAHT behave under low-SNR or noisy conditions? Could over-sparsification harm important emotional features?\n2. Would LFST work as a plug-in front-end for pretrained models such as wav2vec2.0 or HuBERT?\n3. Can the authors provide quantitative comparisons in terms of FLOPs or inference time against STFT/LEAF front-ends?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "• The paper introduces an innovative Learnable Fractional Superlet Transform (LFST) that enables a continuous and learnable trade-off between time and frequency resolution, extending conventional wavelet and STFT formulations.\n• The integration with the Spectro-Temporal Emotion Encoder (STEE) results in a coherent and interpretable end-to-end framework for speech emotion recognition (SER).\n• Theoretical derivations are solid, including proofs of differentiability, stability, and Lipschitz boundedness.\n• The model provides meaningful interpretability—fractional order adaptation and frequency visualization reveal how LFST captures emotional cues in a physically interpretable manner.", "weaknesses": "1. The proposed approach, while theoretically elegant, may incur higher computational cost due to multi-order fractional convolutions. No runtime or FLOPs comparison is provided.\n2. The Learnable Asymmetric Hard Thresholding (LAHT) module lacks a detailed ablation study to demonstrate its independent impact.\n3. The datasets used are relatively limited in scale; generalization to multilingual or real-world noisy speech remains unclear.\n4. Comparisons with large self-supervised models (e.g., wav2vec2.0, HuBERT) are qualitative rather than quantitative, weakening the positioning of the contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ElbHgfVVmq", "reviewer": "Reviewer_dU5Z", "summary": "This paper introduces Learnable Fractional Superlet Transform (LFST), a novel, fully differentiable time-frequency (TF) front-end for speech emotion recognition (SER). LFST generalizes traditional superlets by learning fractional orders via softmax-weighted geometric means across multi-scale Morlet wavelets, enabling data-adaptive TF resolution. It jointly optimizes a log-spaced frequency grid, frequency-dependent base cycles, and fractional-order weights end-to-end. A learnable asymmetric hard-thresholding (LAHT) module promotes sparse, denoised TF activations while preserving transients. Coupled with a lightweight Spectro-Temporal Emotion Encoder (STEE) that leverages magnitude and phase-congruency maps, LFST+STEE achieves state-of-the-art results on IEMOCAP, EMO-DB, and NSPL-CRISE datasets, outperforming fixed front-ends (STFT, CWT, LEAF) and prior superlet variants.", "questions": "1. Prohibitive Computational Cost, Yet Marketed as \"Lightweight\"\nAlthough the paper repeatedly uses the word “lightweight,” LFST is anything but. For every one of the F = 96 frequency bands it performs O = 8 separate convolutions with analytic Morlet kernels of length L = 1 024 samples. At 16 kHz this corresponds to 64 ms of context per kernel; when the batch size, number of bands and temporal steps are taken into account the total operation count is 8–10 times higher than a single FFT-based STFT of the same frame. The authors sidestep this issue by writing that they “stream over orders,” yet the released PyTorch code still instantiates O complex kernels and keeps their feature maps in memory until the geometric-mean reduction, so peak GPU memory grows linearly with O. Nowhere in the paper do we find wall-clock time, number of multiply-adds, or on-device latency, so the reader has no way of knowing whether the front-end can even run in real time on a mobile CPU. When accuracy is compared with genuinely cheap front-ends such as LEAF or SincNet the absence of any compute figure creates an “accuracy-only” narrative that unfairly disadvantages the baselines.\n2. Baseline Selection Is a Straw-Man Comparison\nThe strongest published numbers on IEMOCAP and EMO-DB come from self-supervised models like wav2vec 2.0 and HuBERT, but these are waved away with the phrase “compute-intensive and opaque.” No attempt is made to fine-tune, distill, or even reduce the dimension of such models; they are simply excluded from the comparison table. Inside the paper’s own ablation study the same large STEE encoder (128-D, residual TF-blocks, axial attention, FiLM gating) is kept for LFST, while STFT, CWT and LEAF are forced to feed the very same heavyweight trunk. This gives LFST an implicit capacity boost: the baselines are not allowed to use a smaller, faster backbone that might have been chosen had the authors started from a conventional spectrogram. Consequently the ablation measures “front-end + big network” versus “front-end + big network,” not the isolated contribution of the learnable superlet layer. The resulting delta in accuracy is therefore as much a consequence of parameter count as of representational power, something a fair comparison would have controlled by equalising FLOPs or memory.\n3. Perhaps the most pressing issue is that LFST has so far been evaluated only as a stand-alone front-end paired with a purposely lightweight backend. In reality it is nothing more than a differentiable time-frequency layer, and its true value can only be gauged when it is grafted onto the current heavy-weights of the field—namely wav2vec 2.0, HuBERT, or other large self-supervised speech models. These systems already encode rich paralinguistic information, but they still rely on conventional mel-spectrograms or raw-waveform convolutions for their initial acoustic impressions. Injecting LFST’s adaptive, fractional-order super-resolution representations into the very first layers (or using them as an auxiliary input) could reveal whether the extra computational cost actually yields gains on top of the SOTA baselines that matter. Without such an integration experiment, the paper leaves the community with an attractive but untested feature extractor rather than a proven upgrade to the best performing pipelines.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "LFST’s core strength is that it turns the classical “one-window-fits-all” spectrogram into a continuous, end-to-end-learnable surface: by simply back-propagating through a softmax-weighted geometric mean of Morlet responses, the network can allocate razor-thin time resolution to explosive anger bursts while giving melancholy vowels the long cycles they need for pitch precision.", "weaknesses": "Although the paper brands LFST as “lightweight,” it is anything but: for every one of its 96 frequency bands it performs eight separate 1 024-sample complex convolutions, giving 8–10× the FLOPs of an FFT-based STFT and a peak GPU memory that grows linearly with order O—yet no wall-clock, multiply-add, or mobile-device latency is ever quoted, so readers are trapped in an “accuracy-only” narrative that unfairly penalises genuinely cheap front-ends such as LEAF or SincNet. Inside its own ablation the authors keep the same heavy STEE trunk (128-D, residual TF-blocks, axial attention, FiLM) for LFST while forcing STFT/CWT/LEAF to feed that same over-parameterised backbone, turning the comparison into “big network versus big network” rather than an isolated probe of the learnable superlet layer. Meanwhile the strongest community baselines—wav2vec 2.0 and HuBERT—are dismissed as “compute-intensive” without a single fine-tune, distill, or plug-in experiment; consequently the paper leaves the field with an appealing but unvalidated feature extractor instead of proving that the extra computational cost still pays off once LFST is grafted onto the SOTA pipelines that actually matter.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)