This exploration is being made possible by the new multimedia and interactive TV set-top technologies. For example, tone of voice, inflection, rate of speech, the timing of responses, volume, the texture or fluency of speech, the unique identity of a voice, and accent, all convey semantics and contribute to our understanding of the words being presented. Music, sound effects and silence also contribute a great deal to the underlying mood and semantics of a scene. When one or more of these elements is missing, there can be a reduction or loss in our understanding which may in turn result in a reduction of our appreciation of a broad and varied genre of television, video and movie content. Presently, Closed Captioning communicates the verbatim or paraphrased transcript of the spoken words.
Music is labeled and a small descriptor identifies the tune or the type of music.
Background sounds may be described with one or two words when highly significant. The audio channel can present a large amount of information in parallel, a single tone, a slight inflection can alter the interpretation of an entire conversation. In contrast, text is processed in a serial fashion. Text can therefore carry only a transcript of the spoken words because speaking speeds and reading speeds are inequitable. Information about emotions, music, sound effects must be inserted into the conversational pauses and is often not synchronized with the audio track.