Categorization of sounds 2006 by

Four experiments were conducted to test between Decision-Bound, Prototype, and Distribution theories for the categorization of sounds. Sounds varying in either resonance frequency or duration were used as stimuli. Different experimental conditions were created by varying the variance and overlap of two stimulus distributions used in a training phase and varying the size of the stimulus continuum used in the subsequent test phase. When resonance frequency was the stimulus dimension, the pattern of categorization-function slopes was in accordance with the Decision-Bound theory. When duration was the stimulus dimension, however, the slope pattern gave partial support for the Decision-Bound and Distribution theories. A new categorization model combining aspects of Decision-Bound and Distribution theories is introduced which gives a superior account of the slope patterns across the two stimulus dimensions. Smits, Sereno & Jongman: Categorization of sounds 3 The categorization of sounds plays an important role in everyday life. On a daily basis we categorize sounds in our environment as belonging to such basic events as a telephone ringing, a baby crying or the doors of a train slamming shut. The correct categorization of sounds may even be of vital importance, such as the hooting of an approaching car. Finally, the recognition of the sounds and words of spoken language is a form of auditory categorization which occupies a significant portion of most people’s waking hours. Despite its importance, auditory categorization has received relatively little attention in the psychological literature. An overwhelming majority of studies on perceptual categorization has been devoted to the categorization of simple visual stimuli, such as line segments of variable lengths and orientations. Three distinct theories of visual categorization have featured most prominently in the recent literature. Prototype theory (Rosch, 1973; Smith & Minda, 2000) assumes that stimuli are categorized based on their similarity to category prototypes stored in memory. A category prototype is generally defined as the average, or most typical, member of a category. Exemplar theory (Nosofsky, 1986), on the other hand, denies the explicit use of category prototypes. In its extreme formulation, Exemplar theory assumes that categorization is based on a comparison of the stimulus to all previously categorized exemplars of all categories. Finally, Decision-Bound theory (Ashby & Perrin, 1988) assumes that categorization is based on the comparison of the perceptual effect of a stimulus to category boundaries stored in memory. Explicit connections between the fields of speech perception and visual categorization have been sparse. Nevertheless, the hypotheses for the categorization of phonemes that have been proposed over the years are similar to those for visual categorization, albeit more qualitative and mathematically less well developed. One of the most popular research methodologies in speech perception is the phoneme categorization experiment. In phoneme categorization experiments listeners are presented with Smits, Sereno & Jongman: Categorization of sounds 4 naturally produced or synthetic speech sounds and are asked to assign the sounds to phonetic categories. In the most widely employed version of the phoneme categorization experiment, synthetic stimuli are used in which one or more acoustical parameters of interest, such as formant frequencies or silence durations, are systematically varied in a number of discrete steps of equal size to form a stimulus continuum. Researchers from Haskins laboratories pioneered the use of stimulus continua for investigating speech perception, showing among other things that the perceived phoneme is generally influenced by many acoustic parameters distributed over a wide temporal window. Since these landmark studies, stimulus continua have been employed to investigate many basic aspects of speech perception, such as the unit of recognition (Nearey, 1997), dependencies in the categorization of successive phonemes (Massaro & Cohen, 1983; Smits, 2001), the influence of speaking rate on phonetic categorization (Volaitis & Miller, 1992), and the relative weights of various acoustic cues to particular phonetic distinctions (Ainsworth, 1968). Despite the popularity of the phoneme categorization paradigm, we still do not fully understand how phonetic categories are represented and what listeners actually do in phoneme categorization experiments. How do we categorize speech sounds? The present study takes a first step in answering this question, focusing on the categorization of synthetic non-speech sounds. In the past, several hypotheses concerning the representation and categorization of speech sounds have been proposed. Fueled by the ”categorical perception” controversy, early speech perception experiments were mainly analyzed and discussed in terms of the boundaries between phonetic categories (e.g., Liberman, Harris, Hoffman & Griffith, 1957). The suggestion was made that what listeners do in phonetic categorization experiments is evaluate on which side of the relevant phonetic boundary the perceptual effect of the incoming stimulus is located. This hypothesis can be viewed as a phonetic implementation of Decision-Bound Smits, Sereno & Jongman: Categorization of sounds 5 theory mentioned earlier. It has also been hypothesized that phoneme categories may be represented by prototypes (e.g., Oden & Massaro, 1978; Kuhl, 1991). In the context of speech perception, Prototype theory assumes that listeners in phonetic categorization experiments compute the similarity of an incoming stimulus to each of the relevant category prototypes and categorize the stimulus on the basis of these similarities. More recently, Nearey and Miller and colleagues support a more elaborate view of phonetic category representation which contains more information than just that found for prototypes. Miller (1994) claims that representations of phonetic categories are essentially graded. This claim is based on the finding that category members vary in their perceived category goodness. Miller’s position must be interpreted as mainly contrasting with the classical categorical perception concept, where members of the same category are thought to be perceptually entirely equivalent. In principle, graded category structure may derive from a basic prototype representation, as Miller (1984) acknowledges, where members close to the prototype are judged better exemplars than members further away from the prototype. Miller does suggest however, that phonetic categories actually incorporate distributional information. Nearey and colleagues (Nearey & Assman, 1986; Assman, Nearey & Hogan, 1982; Andruski & Nearey, 1992; Hillenbrand & Nearey, 1999) take a more quantitative stance. Using the Normal A-Posteriori Probability (NAPP) model, they modeled listeners’ representations of vowel categories as multidimensional Gaussian distributions and showed that a-posteriori vowel probabilities based on their model gave very good predictions of listeners’ categorization for a given set of vowel stimuli. Henceforth, we will indicate the class of theories which assume that phonetic categories are represented as distributions (of some form) as the Distribution theory. Finally, a contemporary hypothesis concerning phonetic categorization simply denies the Smits, Sereno & Jongman: Categorization of sounds 6 existence or involvement of a sublexical layer in human speech recognition, instead assuming that words are represented by many, possibly all, previously encountered exemplars of the word (Johnson, 1997a, b; Goldinger, 1997). Henceforth this categorization theory will be indicated as the Exemplar theory. From this perspective, phonemes are not explicitly represented at all and what listeners exactly do in phonetic categorization tasks is not of central importance in understanding how human speech recognition works. Although it is logically possible to formulate an exemplar theory of speech perception with a sublexical layer containing phoneme exemplars, such a theory has, to our knowledge, not been proposed. Despite twenty years of experimenting, the issue of the basic mechanisms underlying perceptual categorization has not been resolved. In speech perception research, the number of experiments addressing the basic mechanisms in phonetic categorization is small as compared to to research on visual categorization (see, e.g., Maddox & Ashby, 1998; Nosofsky, 1998). While there is currently no consensus, neither in the fields of speech perception nor of general perceptual categorization, theories of general perceptual categorization have the advantage that they are mathematically fully developed and the subject of intense experimental evaluation. The present research therefore attempts to apply some of the methods and models of the visual categorization literature to the problem of phonetic categorization. A major problem hindering a direct transfer of the methods to the speech domain is that we as experimenters do not have any control over the ”training corpus” that participants have been exposed to. Throughout their lives, while hearing other people speak, adult listeners have heard large numbers of instances of the phonemes in their language. Both the basic categorization mechanism employed by listeners, as well as a number of system ”parameters” (boundary locations, prototype locations, or distribution covariance matrices) may be based on this unknown training corpus. This poses two problems. First, the training corpus will differ among listeners, and second, we cannot freely manipulate the training corpus to test certain Smits, Sereno & Jongman: Categorization of sounds 7 theoretical predictions. One methodological approach capable of overcoming the difficulties stemming from the lack of control over the training corpus is to study the categorization of non-speech sounds. The p