Title: SRAMA: LEARNING SYLLABLE REPRESENTATIONS WITH ADAPTIVE LOCAL MONOTONIC ATTENTION FOR SYLLABLE STRESS DETECTION IN L2 SPEECH
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: Speech, Syllable Representations, AMA, Contrastive Loss, Automatic Syllable Stress Detection

Abstract: Syllable stress plays a key role in stress-timed languages such as English, where shifting prominence/stress (e.g., obJECT vs. OBject) alters meaning and can hinder communication. For second-language (L2) learners, correct placement of syllable stress is a challenge, creating a strong need for automatic stress detection methods in computer-assisted language learning (CALL) systems. Existing methods, however, depend on syllable boundaries, which are costly to manually annotate and error-prone when generated automatically. To overcome this limitation, we propose a boundary-independent framework that learns syllable representations directly from speech and predicts their stress without explicit boundaries. The framework has three main components: (1) an Adaptive local Monotonic Attention (AMA) mechanism that captures syllable representations by enforcing the natural left-to-right order of syllables, (2) contrastive loss that drives stressed and unstressed syllables apart in the embedding space, and (3) an end-to-end encoder–decoder pipeline that integrates these two components to map speech frames to syllable representations and then decode them autoregressively into stress predictions. To demonstrate the effectiveness of each component, we further conduct ablation studies by either removing AMA or contrastive loss, which validate their critical role in learning robust syllable representations. Experiments on the ISLE corpus of German and Italian learners of English show that our method outperforms boundary-dependent baselines, thereby overcoming the need for explicit boundaries in stress prediction.