Patent Document ID: 8880409
Application ID: 12866159
Patent Status: 1

Claim One:
1. A system for automatic temporal alignment between music audio signal and lyrics, comprising a computer, the computer implementing: a dominant-sound signal extracting portion configured to extract, from the music audio signal of a musical piece including vocals and accompaniment sounds, a dominant sound audio signal of the most dominant sound including the vocal at each point of time; a vocal-section feature extracting portion configured to extract, from the dominant sound audio signal at each point of time, vocal-section features available to estimate vocal sections which include the vocals and non-vocal sections which do not include the vocals; a vocal section estimating portion configured to estimate the vocal sections and the non-vocal sections based on the vocal-section features and to output information on the vocal sections and the non-vocal sections; a temporal-alignment feature extracting portion configured to extract, from the dominant sound audio signal at each point of time, temporal-alignment features appropriate for temporal alignment between lyrics of the vocal and the music audio signal; a phoneme network storing portion configured to store a phoneme network including a plurality of phonemes and a plurality of short pauses in connection with lyrics of the musical piece corresponding to the music audio signal; an alignment portion including a phone model for singing voice capable of estimating phonemes corresponding to the temporal-alignment features based on the temporal-alignment features, and configured to perform temporal alignment between phonemes in the phoneme network and the dominant sound audio signals, the alignment portion receiving as inputs the temporal-alignment features outputted from the temporal-alignment feature extracting portion, the information on the vocal sections and the non-vocal sections, and the phoneme network, and performing the temporal alignment on conditions that no phonemes exist at least in the non-vocal sections; and a non-fricative section extracting portion configured to extract, from the music audio signal, non-fricative sound sections where no fricative sounds exist, wherein the alignment portion further receives information on the non-fricative sound sections as inputs, and performs the temporal alignment on further more conditions that no fricative sound phonemes exist in the non-fricative sound sections.