Modeling the auditory organization of speech - a summary and some comments

Since these papers bear on several different aspects of auditory organization, we may begin the discussion by sketching a view of the entire problem. Starting with the broadest question of the purpose of hearing and our ability to separate different sources, we note that sound ‐ the transmission of air-pressure variations in a certain frequency range ‐ offers a potential source of information concerning events in the physical world which might lead to an adapatation in the listener’s behavior. There are particular sources and attributes that the listener cares about (starting, perhaps, with threats and including subtle nuances of communication), and thus to be useful, auditory perception needs to recover this information in the widest range of circumstances i.e. to separate interesting sources from one another and other interference. We see immediately that the definition of a source in this context is not so much based on some objective physical quality, but on far more elusive subjective considerations of a listener’s internal conception of the world, which can depend on many incidental factors. The sound of a car passing in the street is perceived as one source, if we distinguish it from the background at all, but the sound of my own car may be analyzed into separate percepts for the sound of the engine, tyres, and that worrying rattle from the rear suspension. Conversely, the physically distinct speech sounds made by the vibrating vocal chords (in voiced portions) and turbulant constrictions of the vocal tract (as in fricatives) are almost irresistably heard as a single source, presumably because of our strong and benficial inclination to treat the communication sounds of a given speaker as a single, co-ordinated source. The overall problem, then, is to separate arbitrary information from unconstrained mixtures of subjectivelydefined sources. The fact that we have a sense of hearing at all demonstrates that this is something that we accomplish quite successfully, in spite of the broad requirements. By the same token, most situations will afford only a limited ‘view’ of any individual source, and the auditory system has evolved both to distinguish the information that can be separated, and to tolerate the absence of the information that cannot be recovered in a particular situation. The complement of the auditory system’s ability to separate sources is its ability to ‘degrade gracefully’ (Marr, 1982) in demanding situations, manifested as a preconscious inference of hidden features; both these functions are required for a useful perceptual modality. Figure 1 shows the basic structure we use to model the auditory system. The incoming physical (sound) stimulus is converted by the transducers and processing in the front end to generate the intermediate representation in terms of various feature values. The value of such a mid-level representation comes from meeting the dual constraints of being directly and unambiguously computable from the explicit physical properties of the original signal, while at the same time constituting a more suitable domain for the more abstract and uncertain organization that is to follow (Ellis & Rosenthal, 1998). This representation can benefit from displaying multiple features in parallel ‘maps’ as advocated by Brown (1992); one particularly useful feature map is the correlogram (Slaney & Lyon, 1983; Meddis & Hewitt, 1991; strongly related to the processing described in the chapter by Todd & Lee, and used as the input to the system of Brown & Wang), which displays the physical property of amplitude modulation on an autocorrelation axis (as a function of time and channel), upon which