Do You Hear What I Mean? Quantifying the Instruction-Perception Gap in
  Instruction-Guided Expressive Text-To-Speech Systems
Authors: name: Yi-Cheng Lin, name: Huang-Cheng Chou, name: Tzu-Chieh Wei, name: Kuan-Yu Chen, name: Hung-yi Lee

Instruction-guided text-to-speech (ITTS) enables users to control speech
generation through natural language prompts, offering a more intuitive
interface than traditional TTS. However, the alignment between user style
instructions and listener perception remains largely unexplored. This work
first presents a perceptual analysis of ITTS controllability across two
expressive dimensions (adverbs of degree and graded emotion intensity) and
collects human ratings on speaker age and word-level emphasis attributes. To
comprehensively reveal the instruction-perception gap, we provide a data
collection with large-scale human evaluations, named Expressive VOice Control
(E-VOC) corpus. Furthermore, we reveal that (1) gpt-4o-mini-tts is the most
reliable ITTS model with great alignment between instruction and generated
utterances across acoustic dimensions. (2) The 5 analyzed ITTS systems tend to
generate Adult voices even when the instructions ask to use child or Elderly
voices. (3) Fine-grained control remains a major challenge, indicating that
most ITTS systems have substantial room for improvement in interpreting
slightly different attribute instructions.