Source: http://www.google.com/patents/US20030055654?ie=ISO-8859-1&dq=6,548,982
Timestamp: 2014-10-23 05:58:02
Document Index: 244914166

Matched Legal Cases: ['ART 84', 'ART 91', 'ART 78', 'ART 80', 'ART 86', 'ART 90', 'ART 79', 'ART 79']

Patent US20030055654 - Emotion recognition method and device - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsThe emotion recognition is performed by: extracting (70) a set comprising at least one feature derived from said signal, and processing (84, 90) the set of extracted feature(s) to detect an emotion therefrom. There is further provided a step of low-pass filtering (74) the voice signal prior to extracting...http://www.google.com/patents/US20030055654?utm_source=gb-gplus-sharePatent US20030055654 - Emotion recognition method and deviceAdvanced Patent SearchPublication numberUS20030055654 A1Publication typeApplicationApplication numberUS 10/194,848Publication dateMar 20, 2003Filing dateJul 12, 2002Priority dateJul 13, 2001Also published asEP1256937A2, EP1256937A3, EP1256937B1, US7451079Publication number10194848, 194848, US 2003/0055654 A1, US 2003/055654 A1, US 20030055654 A1, US 20030055654A1, US 2003055654 A1, US 2003055654A1, US-A1-20030055654, US-A1-2003055654, US2003/0055654A1, US2003/055654A1, US20030055654 A1, US20030055654A1, US2003055654 A1, US2003055654A1InventorsPierre OudeyerOriginal AssigneeOudeyer Pierre YvesExport CitationBiBTeX, EndNote, RefManReferenced by (36), Classifications (19), Legal Events (3) External Links: USPTO, USPTO Assignment, EspacenetEmotion recognition method and deviceUS 20030055654 A1Abstract The emotion recognition is performed by: extracting (70) a set comprising at least one feature derived from said signal, and processing (84, 90) the set of extracted feature(s) to detect an emotion therefrom. There is further provided a step of low-pass filtering (74) the voice signal prior to extracting therefrom at least one feature of the set. The cut-off frequency for the low pass filtering is typically centred around 250 Hz. The features are e.g. statistical quantities extracted from sampling a signal of the intensity or pitch of the voice signal. Images(14) Claims(26)
[0107] The preferred embodiment of the invention is chosen to be speaker dependant for the emotion recognition process. This is not necessarily a bad point from an industrial point of view, when the embodiment it is targeted at robotic pets, for instance, which may interact only with their minder. Also, the fact that robots only manage to recognise their minder could even be a positive feature, because it is a source of complicity between a robot and its minder). However, the principles of invention can of course accommodate for speaker-independent approaches, as shall be outlined further. [0108] The embodiment is based on a robotic pet application, with four basic emotional states to detect in a human voice: i) calm, ii) sad, iii) angry and iv) happy. [0109]FIG. 1 shows how these emotions are positioned in a chart which represents an �emotional space�, in which the parameters �valence� and �excitement� are expressed respectively along vertical and horizontal axes 2 and 4. The valence axis ranges from negative to positive values, while the excitement axis ranges from low to high values. The cross-point O of these axes is at the centre of the chart and corresponds to a calm/neutral state. From that point are defined four quadrants, each containing an emotional state, as follows: happy/praising (quadrant Q1) characterised by positive valence and high excitement, comfort/soothing (quadrant Q2) characterised by positive valence and low excitement, sad (quadrant Q3) characterised by negative valence and low excitement, and angry/admonishing (quadrant Q4) characterised by negative valence and high excitement. [0110] The constraints imposed on the embodiment were as follows: [0111] persons may talk very freely to the robotic pet, [0112] persons may utter very short sentences, such as �Aibo ! you're a really naughty robot!�, [0113] the utterances will have to be detected in a noisy environment, and [0114] several persons may have to interact with the robot, whereupon a choice of speaker dependency or independence needs to be made. [0115] Some of these constraints implied that the utterances be restricted to infant-directed type of speech. [0116] At an initial stage, a strategy for study was developed to determine how best to meet these objectives. This strategy can be summarised as follows: [0117] i) experiment with simple features, simple algorithms and small databases, [0118] ii) explore systematically features and learning algorithms for the speaker-dependent case: [0119] a) with many examples, using a database with samples from two speakers, one male, the other female, [0120] b) with only a few examples of the above two-speaker database, and [0121] iii) conduct experiments in real-life situations with fluent speech. [0122] Database for Study [0123] The study was conducted using very large database of two speakers containing informal short emotional utterances. All experiments were conducted using the freely available data mining software Weka which implements most of the standards data mining techniques, information on which can be found in the Weka web page at: http://www.cs.waikato.ac.nz/�ml/. [0124] In order to have sufficiently large databases, some compromises had to made (the recording conditions as described in Slaney et al, 1998 or Breazal 2000 are rather poor and unpractical). Accordingly, the database used for the experimental study used two Japanese professional speakers (respectively male and female), who are both voice actor/actress and worked on many radio/TV commercials, Japanese dubbing of movies and animations. They were asked to imitate everyday speech by pronouncing short sentences or phrases like �Umm, I don't know�, �Exactly!�, �See�, �Hello�, �I see�, �How are you?�, �What kind of food do you like?�, �Wonderful!�, �D'you know�. Before each utterance, they had to imagine themselves in a situation where they could produce the phrase, and which would correspond to one of the four emotional classes: joy/pleasure, sorrow/sadness/grief, anger, normal/neutral. If several emotions were compatible with the sentence meaning, then they were allowed to utter each phrase with the corresponding emotion. The resulting database comprised 200 examples per speaker and per emotion, making a total of 2000 samples. [0125] The database stored in digital form the sound recordings of the samples, so that each sample could be accessed and played back at will. [0126] The two main measurements that were undertaken concerning intonation were pitch and intensity, as in all the reported works reported mentioned supra. [0127]FIGS. 2a-2 c illustrate how the two physical parameters pitch and intensity are derived from a speech signal. The latter is illustrated in FIG. 2a as a waveform envelope, as obtained e.g. directly from a microphone output or from playing back a voice sample. The waveform is the fluctuation in signal amplitude (vertical axis) as a function of time (horizontal axis). In the representation, the signal strength is assigned a zero value at a median position, from which it can take on negative (low amplitude) and positive (high amplitude) values. [0128] Pitch is a subjective quality of a sound that determines its position in a musical scale or voice spectrum. It may be measured as the frequency of the pure tone of specified intensity that is judged by the average ear to occupy the same place in the musical scale or voice spectrum. FIG. 2b illustrates a variation of pitch with time. Here, pitch is measured in terms of a frequency (vertical axis) for a particular time value. [0129] Intensity is a measure the sound energy integrated over a given frequency spectrum. FIG. 2c illustrates the variation of intensity with time, with the intensity measured in terms of decibels (dB). [0130] Additionally, in accordance with a feature specific to the present invention, measurements were made on a low-pass filtered form the electronic signal conveying the utterances, hereafter referred to as the voice signal. [0131]FIG. 3 illustrates schematically a system 1 for evaluating emotion detection performance with the possibility of low-pass filtering the voice signal. [0132] The voice signal can be obtained either directly from a microphone 6 or from the above-mentioned database of recorded utterances 8. In the latter case, each of the samples can be selectively read out in audio format, e.g. from a hard disk drive, so as to obtain an audio signal of the corresponding utterance at an audio output 10. The signal from the microphone 6 or database audio output 10 is fed to an audio input 12 of an audio amplification unit 14 which provides impedance matching, preamplification, and controllable gain functions. The thus-prepared audio signal is delivered at an audio output 14, from which it is fed to a low-pass filter 18. In the system 1, the low-pass filter 18 has an adjustable cut-off frequency Fco, the value of which is selected by a cut-off frequency selection unit 20. [0133]FIG. 4 shows a typical frequency response of the low-pass filter 18 with a curve 22 showing the variation of transferred audio signal level against frequency for an input of constant amplitude over the frequency range. The cut-off frequency Fco is shown as the point of half attenuation along the frequency scale. Ideally, the cut-off frequency is the frequency value beyond which the low-pass filter does not transfer a signal. The sharpness depends on the filter characteristics, e.g. the order of filtering, to approach the ideal low-pass response 24. [0134]FIG. 5 shows how the frequency response changes according to the cut-off frequency Fco selected through the selection unit 20, with three curves illustrated 22 a, 22 b and 22 c corresponding to Fco values respectively of 200 Hz, 250 Hz and 320 Hz. The shape of the response curves are substantially the same. In the system of FIG. 3, the low-pass filter has a continually adjustable cut-off frequency Fco in view of obtaining fine level comparisons. The cut-off frequency selection unit 20 can be electronically controlled to vary a filter component, such as one or several varicap diodes. [0135] The filter 18 is constructed as an analog circuit based on operational amplifiers. Such adjustable cut-off frequency low-pass filters are well known in the art and shall not be described here for conciseness. It can of course equally well be envisaged to produce the low-pass filtering function entirely by digital filtering techniques using e.g. digital signal processors, which are also well known in the art. In this case, the input audio signal would be initially digitised for processing by the filter electronics, the latter reproducing the response curve 22 in a suitably digitally encoded form. [0136] The output from the low-pass filter 18 is then supplied to a first input 28 a of an audio interface 28 of a feature measurement station 30, which is represented as a functional unit cooperating with a PC based control unit 32. [0137] There is also a separate leading from the audio output 16 directly to a second input 28 b of the audio interface 28 of the feature measurement station 30. The two inputs 28 a and 28 b constitute separate input channels allowing the feature measurement station 30 to operate selectively on the low-pass filtered form and non filtered forms of the audio signal. [0138] The PC based control unit 32 is composed of a monitor 34 and keyboard 36 forming the system interface, and a complete PC type computer 38 for the processing, drive and data storage functions. The feature measurement station 30 is programmable to extract different features of the signal presented at its signal input 28. As feature extraction provided by the station 30 is based on digital processing techniques, its signal input 28 is connected to an internal analog-to-digital converter (ADC) 40. The filtered audio signal is then processed in terms of binary words to extract the required features, which are discussed below. [0139] The measurements on the different features produced by the station 30 are stored in a data logger 42 where they can be accessed by a functional unit 44 which performs emotion recognition algorithms. The algorithms aim to discern an emotion expressed in the voice signal on the basis of the signal features extracted by the station 30. Typically, an algorithm under evaluation in unit 44 will indicate the discerned emotion (calm, happy, . . . ) and various internal parameters. The recognition unit 44 can thus execute different emotion recognition algorithms and produce their respective results and internal parameters. [0140] This information is entered into a results analyser unit 46 where different algorithms and/or choices of features can be compared in view of determining an optimum selection. [0141] It will be appreciated the functional units composed of the feature measurement station 30, data logger 42, emotion recognition algorithm execution unit 44 and the results analyser 46 can be implemented physically by the PC based control unit 32, as indicated by the staggered double-arrows. [0142] The system further includes a programming unit PU whose function is to program different emotion recognition systems on the basis of the most appropriate results found. To this end, the programming unit PU is operationally connected to the results analyser 46 as well as to the PC based control unit 32 and other units, so as to record all the algorithms, parameter settings, features to extract and other optimisations. The recording of this information is established in accordance with a predetermined protocol so that the programming data can be downloaded into a suitable programming interface of the an emotion sensitive system, such a robotic pet or the like. The programming unit PU can be made detachable from the rest of the system and portable for on-field programming, and/or adapted to produce the programming data on a portable recording medium, such as a disk, semiconductor memory, or the like. [0143] The measurements performed by the feature measurement station 30 were based on the intensity of low-passed form of the voice signal, as obtained from the filter 18. (Separate experiments were also conducted on high-passed forms using a high-pass filter instead.) For the low-pass filtering, the cut-off frequency was typically set to 250 Hz (the particular value appears not to be crucial), the idea being to separate the signal into a pure prosodic component and a pure �ceptral� component. [0144] The samples from the database 8 thus provide sound patterns on which various measurements can be made at the station 30. Psycho-acoustic measurements have shown that the most important information is prosody (intonation), and in particular the pitch curve (f0) (cf. FIG. 2b) and the intensity (I0) curve (cf. FIG. 2c). [0145] In the evaluation, three different signals were processed by the station 30: [0146] i) the intensity of the signal after low-pass filtering, [0147] ii) the intensity of the signal without low-pass filtering, and [0148] iii) the pitch of the signal without low-pass filtering. [0149] The first signal i) is taken from input 28 a, while the last two signal is taken from input 28 b of the feature measurement station 30. [0150] Note that the pitch can also be measured on the signal after low-pass filtering either instead of on the non filtered signal iii), or in addition, in which case the number of signals in the evaluation total four. However, experiments have shown that the information provided by the pitch analysed from the low-pass filtered signal is generally not significantly different from that provided by the pitch of the unfiltered signal. Accordingly, only signals i) to iii) are actually used in the evaluation. [0151] After low pass filtering, the intensity curve of the resultant signal i) was measured in the same way as for the unfiltered signal. [0152] Finally, to be exhaustive, there was conducted a ceptral measurement on signals iv) computed from the norm of the absolute vector derivative of the first 10 MFCCs (mel-frequency ceptral components). In speech recognition, there is often performed a measurement at each time step of the ceptral content, which is a parameter equivalent to the excitation of different nerves in the cochlea. These can correspond to the different ceptral coefficients of the signal. The measurement establishes the mean of the local slope of the derivatives of the vectors in the cepters. Typically, these measurements yield ten coefficients with ten values that evolve in time, and establish the difference between the successive vectors, from which can be obtained the norm of the vectors. [0153] All these measurements i) to iv) above were performed at each 0.01s time frame, using a signal processing toolkit freely available under the name �Praat software�, information on which can be found in web page �http://www.praat.org�. [0154] In particular, the pitch was computed using the algorithm described in Boersma P. (1993) �Accurate short-term analysis of the fundamental frequency and the harmonics-to-noise ratio of a sampled sound�, in Proceedings of the Institute of Phonetic Sciences of the University of Amsterdam, 17, 97-110 1993, which is known to be very accurate. [0155] In what follows, the ceptral measurement signals (iv) are taken into account in the evaluation; only measurements on signals i) to iii) cited above shall be considered. [0156] Each of these measurements provides a series of values that were transformed to produce different points of view on the data. [0157] The procedure to obtain the series of values is as follows. For a given voice sample from the database 8, typically corresponding to a short phrase, the feature measurement station 30 stores the corresponding voice signal produced at the input (channel 28 a or 28 b depending on the signal i), ii) or iii) above to be analysed). The signals are initially 10 Hz smoothed. [0158] From that stored 10 Hz smoothed signal are computed each of the three time-varying signals i) to iii) mentioned above. [0159] Each thus-computed signal i) to iii) is digitally sampled at e.g. 0.01 second intervals. Each sampling gives an absolute instantaneous value of the intensity (signals i) and ii))) or of the pitch (signal iii)). [0160] For each sampled signal, there is thus obtained a sequence of sampled values, typically a few hundred, given that a phrase is a few seconds long. The sequence of values is stored in time order, starting from the value of the earliest part of the signal, so that the sequence of values expresses the time variation of signal i), ii) or iii) considered. [0161] For each signal, the corresponding time-ordered sequence of values is analysed to produce the following four series of values: [0162] 1) The series of local minimas. This involves determining the values in the sequence that correspond to a minima and constructing a series formed just of these minima values of the sequence. The series is stored in time order, starting from the earliest minima. A minima in the sequence of values is identified as a member of the sequence whose value is less than that of its neighbour immediately before and immediately after in the sequence. [0163] 2) The series of local maximas. This involves determining the values in the sequence that correspond to a maxima and constructing a series formed just of these maxima values of the sequence. The series is stored in time order, starting from the earliest maxima. A maxima in the sequence of values is identified as a member of the sequence whose value is greater than that of its neighbour immediately before and immediately after in the sequence. [0164] 3) The series of durations (time intervals) between local (adjacent) extremas (which models rhythmic aspects of the signal). An extrema is either a minima or a maxima. This involves first identifying in the sequence the values that correspond to a minima or a maxima, by comparing each value with its immediately adjacent neighbours as for series 1) and 2) above. Then, the positions in time of the identified extremas are analysed to determine the time intervals separating successive extrema. These time intervals are set out in a series, starting from the interval separating the earliest and next earliest extremas, to yield the series of time intervals. [0165] 4) The series itself. This simply involves forming a series from all the successive sampled values of the sequence, starting from the beginning of the sampled voice signal. [0166]FIG. 6 illustrates how these four series of values are obtained from the signal undergoing measurement. [0167] It will be noted that for each signal i) to iii) above measured, the four series of values 1) to 4) above are extracted, giving a total of 12 series of values per audio sample of the database. [0168] Finally, the following ten statistical characteristics are computed for each of the twelve series of values: [0169] 1) the mean, [0170] 2) the maximum (i.e. the overall highest value), [0171] 3) the minimum (i.e. the overall lowest vale), [0172] 4) the difference between the maximum and the minimum, [0173] 5) the variance, [0174] 6) the median, [0175] 7) the first quartile, [0176] 8) the third quartile, [0177] 9) the interquartile range, defined as the difference between the third and first quartiles, [0178] 10) the mean of the absolute value of the local derivative. [0179] All the data are initially normalised before calculating these statistical characteristics. [0180] The resulting combination of these ten characteristics applied to the four series gives rise to 3*4*10=120 features. [0181] Learning Algorithms [0182] Next, after all these values are obtained for each signal, an experiment was conducted using different automated learning methods. The latter are capable of learning and generalising from a certain number of examples. Many such methods are known, each having their advantages and disadvantages. The approach makes a fairly exhaustive use of these different algorithms. [0183] For each learning algorithm tested, there is input a base set of examples. The performance for each base set is then checked against another example base set. This allows to determine a percentage of correct generalisations made by each learning algorithm. [0184] There are many learning schemes that have been developed in the last 20 years (e.g. Witten I., Frank E. (2000) �Data Mining�, Morgan Kaufflan Publishers.), and they are often not equivalent: some are more efficient with certain types of class distributions than others, while some are better at dealing with many irrelevant features (which is the case here, as seen a posteriori) or with structured feature sets (in which this is the �syntactic� combination of the values of features which is crucial). As, by definition, the structure of the data and/or the (ir-)relevance of its features is unknown, it would be a mistake to investigate the problem with only very few learning schemes. As a consequence, it was chosen to use a set of the most representative learning schemes, ranging from neural networks to rule induction or classification by regression. Also, one of the best meta-learning scheme, was used, namely AdaBoostM1 (Witten and Frank, 2000 cited supra), which significantly improves the generalisation performance for unstable learning schemes like decision trees (an unstable learning algorithm is one that can sometimes produce very different recognition machines when only a slight change in the learning database has been performed). One reason for choosing the Weka software was that its code and executable are freely available, so that the experiment, though being large scale, can be easily reproduced. This software also provides means like automatic cross-validation, or the search of feature spaces with for e.g. genetic algorithms as shall appear later. The list of all learning algorithms is given in table I below. TABLE I list of learning algorithms investigated Name Description 1-NN 1 nearest neighbour 3-NN voted two nearest neighbours 10-NN voted ten nearest neighbours Decision tree/C4.5 C4.5 decision trees Decision Rules/PART PART decision rules Kernel Density Radial basis function�neural network KStan � Linear Regression Classification via linear regression LWR Classification via locally weighted regression Voted Perceptrons Committee of perceptrons SVM1 polynomial (deg. 1) support vector machine SVM2 polynomial (deg. 2) support vector machine SVM3 polynomial (deg. 3) support vector machine VFI Voted features interval M5Prime Classification via M5Prim regression method Naive Bayes Naive Bayes classification algorithm AidaBoost M1/C4.5 Ada boosted version of C4.5 AidaBoost M1/PART Ada boosted version of PART [0185] More details about these algorithms can be found in Witten and Frank, 2000 cited supra. [0186] In a first experiment, an evaluation was conducted in which all algorithms were attributed all the (normalised) features, and were trained on 90 percent of the database and tested on the remaining 10 percent. This was repeated 10 times, with each time a different 90/10 percent split (a 10-fold cross-validation thus being performed). [0187] Table II below gives the average percentage of correct classification for the 10 fold cross-validation for the different algorithms used. TABLE II results using all features for two different speakers (1 and 2) Emotion detection success rate (%) Algorithm Speaker 1 Speaker 2 1-NN 82 87 3-NN 84 87 10-NN 83 87 Decision tree/C4.5 84 93 Decision Rules/PART 84 94 Kernel Density 84 90 KStan 81 83 Linear Regression 88 91 LWR 87 90 Voted Perceptrons 70 76 SVM1 88 94 SVM2 89 94 SVM3 88 94 VFI 80 93 M5Prime 86 96 Naive Bayes 84 90 AidaBoost M1/C4.5 90 96 AidaBoost M1/PART 91 97 [0188] It can be observed from these results that very high success rate (between 92 and 97 percent), which is higher than any other reported result in the literature can be obtained thanks to the use of certain algorithms. However, the difference among algorithms is striking: whereas the best results are obtained with �Adaboosted� decision trees and rules, some others perform 10 percent below (like nearest neighbours, RBF neural nets or Support Vector Machines, which are the ones typically used in other studies), or even 20 percent below (Committees of perceptrons). This illustrates that care must be taken to try many different learning schemes when solving a problem about which there is very little by way of few prior or intuitive knowledge. [0189] Feature Selection [0190] After this first experiment, the next step is to see how the feature set could be reduced, for three reasons: [0191] 1) a small features set can be expected to provide better generalisation performance (see Witten and Frank, 2000 cited supra); [0192] 2) it is computationally cheaper to compute fewer features; [0193] 3) it is interesting to see if the most useful features for the machine learning algorithms are the ones that are traditionally put forward in the psychoacoustic literature. [0194] A first way of exploring the feature set is to look at the results of learning schemes like decision rules (PART), which are often used mainly as knowledge discovery devices. [0195] One specific choice of rules and parameterisation produced and tested by the Applicant is: EXAMPLE 1 [0196] If MEDIANINTENSITYLOW>0.48 and [0197] MINMINIMASPITCH<=0.07 and [0198] THIRDQUARTINTENSITY>0.42==>CALM [0199] ELSE If MEANINTENSITYLOW<=0.58 and [0200] MEDIANINTENSITYLOW<=0.29==>ANGRY [0201] ELSE If THIRDQUARTINTENSITYLOW>0.48==>SAD [0202] ELSE==>HAPPY [0203] (the symbol �<=� means �less than or equal to�). [0204] All the numerical values are normalised. [0205] The definition of declared variables in the above algorithm are as follows: [0206] MEDIANINTENSITYLOW: the median value of series of values of the intensity of the signal after low-pass filtering; [0207] MIWINIMASPITCH: the value smallest minima in the series of minima in the pitch of the signal without low-pass filtering; [0208] THIRDQUARTINTENSITY: the value of the third quartile in the series of values of the intensity of the signal without low-pass filtering; [0209] MEANINTENSITYLOW: the mean value of the of the series of intensity values of the signal after low-pass filtering; [0210] THIRDQUARTINTENSITYLOW: the value of the third quartile in the series of intensity values of the signal after low-pass filtering. [0211] These four and surprisingly simple rules allow a remarkably high percentage of correct classification in generalisation of 94 percent for the speaker 2 database. The striking fact is the repeated use of features related to the intensity of the low-pass signal. [0212] Different sets rules can be adapted for optimised performance with different speakers/languages. The choice of an appropriate set of rules for a given speaker/language can be determined empirically. [0213] Another view of the feature set can be obtained simply by visualisation. An example of such a visualisation is illustrated in FIGS. 7a-7 c, which each depict an example of a line of measurements for a feature i, j and k respectively taken among the 160 features investigated, as explained above. For each line is defined a space in a rectangular box 48 a-48 c within which are plotted a number of points. Each plotted point corresponds to an identification of the voice samples of the database 8. The position of a point corresponds to the associated measurement value for the feature considered (evolving on the horizontal scale). The emotion corresponding to the plotted samples is identified by the following key: dot=calm, cross=sad, dash=angry, circle=happy. In the complete experiment, 160 such lines are produced, one for each feature. [0214]FIG. 7a shows a typical haphazard distribution of identified emotion samples, showing that the feature i from which it is generated does not provide a good basis for discerning emotions. Such a distribution is frequent among features extracted from a voice signal that has not been low-pass filtered. [0215]FIG. 7b shows a significantly more clustered distribution of identified emotion samples, indicating that the feature from which it is generated is potentially good for discerning emotions. However, there is some overlap between neighbouring clusters. Such a distribution is quite typical of a feature extracted from a voice signal whose intensity has been low-passed filtered, the cut-off frequency (f0) being 250 Hz in the examples. [0216] Finally, FIG. 7c shows an even better distribution of identified emotion samples where the emotion samples are not only well clustered, but also clearly separated by empty regions. Such results are obtained with some features extracted from a voice signal whose intensity has been low-pass filtered. [0217] (Note: to avoid cluttering, only a representative fraction of the voice samples are plotted in the figure.) [0218] The impressive results obtained by low-pass filtering the intensity of the voice signal was confirmed with different speakers. [0219] In the experiment, six features were selected among the group, for having the best clustering of emotion samples with at least some empty regions separating them. [0220] Generally, the best results are obtained by considering the following features of the low-passed filtered intensity of the voice signals: [0221] quartiles, notably the first and third quartiles, [0222] maximas, [0223] minimas, and [0224] the mean. [0225] By contrast, the least conclusive results were obtained from features involving the variance or mean of the absolute slope of the low-pass filtered signal. [0226] To confirm the Applicant's intuition that low-passed intensity is crucial in the distinction of emotions, FIGS. 8a and 8 b show plots of identified emotions from the samples of the database 8. In theses plots, two features noted for yielding potentially good emotion discernment are expressed on respective orthogonal axes: the first quartile of intensity distribution on the vertical (ordinate) axis, and the third quartile of intensity distribution on the horizontal axis (abscissa). The symbols used to identify the four different emotions are the same as in FIGS. 7a-7 c. Similarly, only a representative fraction of symbols are inserted to avoid cluttering. [0227] The plots of FIG. 8a are produced from the voice samples of the database without low-pass filtering, while the plots of FIG. 8b are produced from the same voice samples, but after low-pass filtering with a cut-off frequency of 250 Hz using the filter 18 of FIG. 3. The voice samples in both cases came from the same speaker (speaker 2). The same very striking effect also takes place for speaker 1; but what is interesting is that the clusters are not situated at the same places (anger and happiness are 90 degrees rotated), which is an illustration of the great speaker variability initially suspected. However, it seems that the use of some well chosen features is very stable for each speaker. [0228] In order to quantify the individual relevance of features or attributes, there is a measure that is often used in the data mining literature, which is the expected information gain, or mutual information between class and attribute. It corresponds to the difference between the entropies H(class) and H(classlattribute) (see Witten and Frank, 2000 cited supra, for details about how it is computed). Table III below gives the 20 best attributes according to the information gain they provide. TABLE III Information gain of 20 best features Information Feature gain 1: MEDIANINTENSITYLOW 1.44 (median value of sampled low-pass filtered signal intensity values) 2: MEANINTENSITYLOW 1.40 (mean value of low-pass filtered signal intensity values) 3: THIRDQUARTINTENSITYLOW 1.35 (third quartile value of series of sampled low-pass filtered signal intensity values) 4: ONEQUARTINTENSITYLOW 1.34 (first quartile value of series of sampled low-pass filtered signal intensity values) 5: MAXINTENSITYLOW 1.23 (maximum value of series of sampled low-pass filtered signal intensity values) 6: MININTENSITYLOW 1.14 (maximum value of series of sampled low-pass filtered signal intensity values) 7: THIRDQUARTMINIMASPITCH 0.72 (third quartile value of series of minimas of sampled unfiltered pitch values) 8: THIRQUARTMAXIMASPITCH 0.72 (third quartile value of series of maximas of sampled unfiltered pitch values) 9: THIRDQUARTPITCH 0.69 (third quartile of series of sampled unfiltered pitch values) 10: MAXMINIMASPITCH 0.67 (largest value in series of minimas of sampled unfiltered pitch values) 11: MAXMAXIMASPITCH 0.67 (largest value in series of maximas of sampled unfiltered pitch values) 12: MAXPITCH 0.67 (maximum value of sampled unfiltered pitch values) 13: MINMINIMASPITCH 0.59 (smallest value in series of minimas of sampled unfiltered pitch values) 14: MEDIANMINIMASPITCH 0.57 (median value in series of minimas of sampled unfiltered pitch values) 15: MEDIANMAXIMASPITCH 0.57 (median value in series of maximas of sampled unfiltered pitch values) 16: MINPITCH 0.52 (minimum value of series of sampled unfiltered pitch values) 17: MEDIANPITCH 0.52 (median value of series of sampled unfiltered pitch values) 18: MEANMINIMASPITCH 0.48 (mean value in series of minimas of sampled unfiltered pitch values) 19: MEANMAXIMASPITCH 0.48 (mean value in series of maximas of sampled unfiltered pitch values) 20: MEANPITCH 0.48 (mean value of series of sampled unfiltered pitch values) [0229] (mean value of series of sampled unfiltered pitch values) [0230] This table confirms the great value of the features concerning the first six set of the distribution of intensity values involving the low-passed signals. It also shows something rather surprising: among the 20 most individually informative features, only 3 (the features numbered 12, 16 and 20 in the table) are part of the standard set put forward in psychoacoustic studies (Murray E., Arnott J. L., (1995) �Implementation and testing of a system for producing emotion-by-rule in synthetic speech�, Speech Communication, 16(4), pp. 369-390; Sendlmeier and Burkhardt 2000 cited supra; Stevens and Williams 1972 cited supra), or used in most of more application oriented research as in Slaney et al. 1998 cited supra, Breazal 2000 cited supra). [0231] Nevertheless, it must be noted that individual salience of a feature is only partially interesting: it is not rare that success comes from the combination of features. Accordingly, in a first experiment, the Applicant compared a feature set containing only the features 1 to 6 related to low-passed signal intensity (LPF), with a feature set composed of the standard features (SF) used in (Breazal 2000, or Slaneyet al. 1998): [0232] 1) mean, [0233] 2) min, [0234] 3) max, [0235] 4) max-min, [0236] 5) variance of pitch of the unfiltered signal, [0237] 6) intensity of unfiltered signal, and [0238] 7) mean length of syllabic segments. [0239] (Results are similar if jitter and tremor are added, as sometimes also used). [0240] Table IV summarises these experiments (each number corresponds again to the mean percentage of correct classification in generalisation in 10-fold cross-validation). TABLE IV comparison between �standard� features and �low-passed signal intensity� features Learning schemes (LPF)sp.1 (LPF)sp.2 (SF)sp.1 (SF)sp.2 1-NN 78 83 70 72 5-NN 84 82 72 75 10-NN 84 82 73 73 Decision Trees/C4.5 80 84 72 71 Decision Rules/PART 78 83 72 74 Kernel Density 82 85 71 74 Kstar 80 84 70 72 Linear Regression 63 68 72 78 LWR 75 71 75 80 Voted Perceptrons 51 70 60 58 SVM degree 1 63 68 73 78 SVM degree 2 71 70 77 50 SVM degree 3 76 85 78 82 VFI 78 76 64 70 M5Prime 83 85 76 80 Naive Bayes 82 81 74 72 AdaBoost M1/C4.5 80 81 80 78 AdaBoost M1/PART 80 83 79 78 [0241] In the above, the abbreviations are as follow: (LPF)sp.1=low-passed signal of speaker 1 voice samples, (LPF)sp.2=low-passed signal of speaker 2 voice samples, (SF)sp.1=standard, non-filtered signal of speaker 1 voice samples, and (SF)sp.2=standard, non-filtered signal of speaker 2 voice samples. [0242] This table shows that use of only the quartiles of the low-passed signal intensity, still outperforms the combination of features used traditionally. [0243] In order to attain the object of finding a very efficient small set of features, the Applicant implemented an automatic search method based on the use of genetic algorithms. Populations of features (limited to 30) were generated and evolved using as fitness the 10-fold cross-validation with two algorithms: Naive Bayes and 5-Nearest Neighbours (chosen mainly because they are fast to train). The exact genetic algorithm is the simple one described in Goldberg, D. E. (1989) �Genetic algorithms in search, optimization and machine learning, reading�, MA: Addison-Wesley. [0244] The outcome of this experiment was not obvious: within the selected feature set, there were features related to the quartiles of low-passed signal intensity and features related to the quartiles of pitch, but also features with relatively low individual information gain: those related to the quartiles of the minimas of the unfiltered smoothed intensity curve. [0245] Also, it is again noted that the machine learning algorithm tends to always neglect features related to the variance or the range of distributions, whatever the measurement. [0246] A final experiment using these 15 features along with all learning algorithms was conducted (max, min, median, 3rd quartile and 1st quartile of low-passed signal intensity, pitch and minimas of unfiltered signal intensity). [0247] The results are summarised in table V below. TABLE V results of learning scheme using the �optimal� feature set name speaker 1 speaker 2 1-NN 87 92 5-NN 90 92 10-NN 87 91 Decision Trees/C4.5 85 92 Decision Rules/PART 86 93 Kernel Density 87 91 Kstar 86 90 Linear Regression 83 89 LWR 87 89 Voted Perceptrons 65 78 SVM degree 1 87 91 SVM degree 2 90 96 SVM degree 3 89 94 VFI 83 92 M5Prime 88 95 Naive Bayes 89 93 AdaBoost M1/C4.5 90 96 AdaBoost M1/PART 90 96 [0248] It can be observed the results obtained are very similar to the best results initially obtained, but with over than 10 times fewer features. Moreover, and interestingly, the variation between learning schemes is less important and algorithms which performed badly, like nearest neighbours or Naive Bayes, now behave more satisfactorily. [0249] When Only Very Few Examples are Provided [0250] In the previous section, large training databases were used: this was optimal to explore feature and algorithmic spaces. However, when dealing with a speaker dependant task, this is not directly applicable to a real world robotic pet, for instance. Indeed, it is not conceivable that the owner of such a robot would give hundreds of supervised examples to teach it how to recognise its way of expressing basic emotions, even though this is what probably happens with human babies and real pets. Indeed, people are generally less inclined to spend such an amount of time with robotic pets. [0251] It then becomes natural to ask what the results become if only very few training examples are given. [0252] To this end, the Applicant produced an experiment using the �optimal� feature set fed earlier, but giving to each algorithms only 12 examples of each class. The algorithms were tested on the remaining items of the database. [0253] This experiment was repeated 30 times with different sets of 12 examples, and results were averaged (the standard deviation was rather low, typically around 1.1). [0254] Table VI summarises the results of the experiment. TABLE VI results of learning scheme when very few training examples are provided learning scheme speaker 1 speaker 2 1-NN 81 85 5-NN 78 74 10-NN 68 64 Decision Trees/C4.5 79 78 Decision Rules/PART 79 74 Kernel Density 82 84 Kstar 76 68 Linear Regression 75 74 LWR 75 79 Voted Perceptrons 40 51 SVM degree 1 81 83 SVM degree 2 83 85 SVM degree 3 83 85 VFI 77 76 M5Prime 80 79 Naive Bayes 85 87 AdaBoost M1/C4.5 79 84 AdaBoost M1/PART 79 90 [0255] It can be observed that some of the algorithms manage to keep a very reasonable level of performance (around 85 percent of success in generalisation). Among them are examples of very cheap algorithms like 1-nearest neighbour or Naive Bayes. These results are rather comparable (and in fact slightly superior) to what is described in Breazal, 2000 cited supra. However, in this case, learning was off-line with a larger database of several female speakers): what is important is that Breazal conducted experiments and showed that this level of success is sufficient to develop interesting interactions with a robotic pet. Also, Breazal showed how these results could be substantially improved when integrated into a larger cognitive architecture which is working in the real world. For example, linking this recognition module to an artificial lymbic/emotional system in which there is some kind of emotional inertia (one very rarely swap from happy to angry in half a second) might give some additional information or tell the system there is uncertainty about the result. As a consequence, the robot may for instance take a posture showing it is not sure of what is happening. The human then will often repeat its utterance with an even more exaggerated intonation. [0256] Conclusions on Overall Studies [0257] From the foregoing, it appears that with the right set of features, reasonable performance can be reached when only few examples are given, which might be the case in �real situation� robots. [0258] An aim is therefore to try to reduce the number of measurements while maintaining an acceptable level of performance, or possibly better. Indeed, some learning algorithms are not well suited to exploiting a large number of measurements, especially when are not all highly relevant. Some learning algorithms are of the constructive type. They are interesting inasmuch as they aim to generate a certain number of rules readable by the developer for classifying emotions. For example, with one or two databases, there is a learning algorithm which identified from the mass of measurements that is there is simply made a test on four chosen values, acceptable results can be obtained. [0259] This allows to discern a certain number of things which appear to be important. These are typically the value of the signal linked to the intensity of the low-passed signal. More particularly, it indicates whether the median of the intensity of that signal is greater than a given value (e.g. 25.75 dB). [0260] An experiment was repeated, by supplying to the learning algorithm only the different quartiles of the intensity of the low passed signal. Measurements show that the best learning algorithms achieve typically 80-85% success (compared with 90-95% for a complete analysis). However, the calculation is greatly simplified, notably by the fact that no pitch analysis is conducted. [0261] On the data base, experiments were made using values traditionally employed in the literature for comparison with either all the values or with the low passed signal intensity. It appears that using the same data base that the values obtained from known choices yield a success rate typically 70-75%. This is around 10% below the results obtained simply by using the low passed signal intensity. [0262] A certain number of experiments were conducted to determine the best compromise between a small number of measurements and the results. This involved generating a large number of measurement groups, e.g. using a genetic algorithm, and analysing the space. The conclusion was that the best results are obtained from the combination of the quartiles of the low passed signal intensity+quartiles of the pitch+the quartiles of the minima of the normal (unfiltered) signal, giving 15 parameters. These success rates in the region of 90-97%, which is substantially the same as the success rate obtained by taking the 120 parameters together. [0263] The next question is to determine whether a robot can generalise relatively well from a relatively small initial number of examples. To this end, an experiment was conducted in which the algorithms were supplied with just twelve examples per class�giving 48 examples in total�to determine to what extent the results would be different. It was observed that differences among the different learning algorithms became more pronounced. However, there are a certain number of standard algorithms which still achieve a success rate in the region of 85-90%. [0264] The general results of the study are illustrated by the bar graphs of FIGS. 9 to 13, which compare all the different cited learning algorithms in terms of correct classification of emotion. For each algorithm, two results are indicated, corresponding to each of two respective speakers of the database 8. More particularly: [0265]FIG. 9 shows performance with only low-passed signal intensity, for the correct classification in generalisation from an initial sample. The features are just the low-passed (250 Hz cut-off frequency) signal intensity; [0266]FIG. 10 shows performance with so-called �Kismet� features (Kismet being a system developed by MIT) namely: pitch+intensity+length segments; [0267]FIG. 11 shows performance for the best combination of features: low-passed signal intensity+pitch+minimas intensity; [0268]FIG. 12 show performance when very few samples are available, with the correct classification in generalisation, based on 12 training examples/class. The features used are: low-passed signal intensity+pitch+minimas intensity; and [0269]FIG. 13 is a chart summarising the results in terms of the best scores per experiment for each of the preceding charts. [0270] Practical Implementation [0271]FIG. 14 illustrates schematically how the findings in accordance with the invention can be implemented in an emotion sensitive system 60. The latter can be part of a robotic pet, humanoid, teaching device, interactive computer or the like. Typically, the system is centred on a microcomputer comprising all the necessary processing, data storage, driving and interfacing functions necessary for the emotion recognition process. As these units are in themselves known and readily adaptable to application requirements by the skilled person, they have been omitted from the diagram for the sake of simplicity. [0272] The system 60 presents an audio input section for receiving voice data from which an emotion is to be detected. The voice data can be obtained from two different sources: a built-in microphone 62 to pick up environmental sounds and thus detect a voice utterance from a nearby person on which to perform emotion detection, as in a real-life situation, and a database of recorded utterances, which can interface with the system 60 via an audio connector 64. The database can simply be the database 8 of FIG. 3, or an adapted version thereof. In this way, the system can be pre-trained, corrected, characterised, etc. This interfacing possibility is also useful to set up all the internal sound processing parameters, including the low-pass filtering characteristics. [0273] The voice signals from either of the two sources 62 and 8 are fed to an audio amplification and preprocessing unit 66 which adapts the signal levels to the system electronics. The unit 66 may also include some noise reduction stages to reduce background noise to a minimum when the microphone 62 is in operation. The audio output 68 of unit 66 is fed along first and second paths P1 and P2 to two respective first and second channel inputs IC1 and IC2 of a feature extraction unit 70. [0274] The first path P1 contains a first digital-to-analog converter (DAC) 72 between the audio output 68 and the first channel input IC1. The latter then simply receives the suitably digitised form of the amplified and preprocessed audio signal. [0275] The second path P2 contains a low-pass filter 74 whose input is connected to the audio output 68. In the example, the low-pass filter is similar to the filter 18 of FIG. 3, inasmuch as it has an adjustable cut-off frequency. Typically, the cut-off frequency (Fco) is set by default to pass signals from a frequency range of 0 to 250 Hz, with a rapid attenuation after that frequency. The cut-off frequency can be set to different values by means of a cut-off frequency selector 76. The latter can be manually controlled by means of a slider to produce different cut-off frequencies, and/or electronically adjusted by internal control circuitry, e.g. in feedback system to obtain optimal results during a characterisation phase. The typical range of possible values for the cut-off frequency Fco is from 150 Hz to 400 Hz. The adjustment in the cut-off frequency can be necessary to optimise the system 60 to different speakers, languages, environments, varieties of emotions to detect, etc. [0276] The low-pass filter 74 further includes a variable attenuation control device 78 which allow the slope of the frequency response curve (cf. FIG. 4) to be modified, e.g. by changing the order of the filter, to allow a further degree of freedom in optimising the low-pass filtering characteristics. [0277] However, the filter 74 need not be variable, nor have an adjustable slope, and simpler embodiments can be envisaged with a fixed low-pass filter set at a predetermined cut-off frequency, e.g. 250 Hz. [0278] As for the system of FIG. 3, the low-pass filter 74 can be implemented with analog circuitry (passive or active), or as a digital filter. In the latter case, the DAC 80 is no longer necessary, but a similar DAC would be provided at its input, unless the digital low-pass filter comprises its own digital-to-analog converter stage. In a practical implementation, the low-pass filter 74 and its control units 76 and 78 would be typically fully digital such that the control of filter characteristics, including of selection of cut-off frequency are accessible digitally via an interface of a PC based control unit at the heart of the system 60. [0279] The low-passed signal output from the low-pass filter 74 (analog in the figure) is supplied to a second analog-to-digital converter 80 which supplies the suitably digitised low-pass filtered signal to the second channel input IC2 of the feature extraction unit 70. [0280] The latter forms the first link in a processing chain whose function is to detect an emotion conveyed by inputted voice signals. [0281] The first part of the processing chain comprises the feature extraction unit 70, which is programmed to extract a selection of features from the digitised signal, of which some are to be extracted from the voice signal after low pass filtering (at channel input IC2) in accordance with the invention. [0282] The feature extraction unit 70 operates in a manner analogous to the feature measurement station 30 of FIG. 3. In particular, it selectively extracts a set of features taken from the 120 features described supra, i.e. obtained from the possible combinations of: [0283] i) the intensity of the signal after low-pass filtering (derived from the signal at channel IC2), [0284] ii) the intensity of the signal without low-pass filtering (derived from the signal at channel IC1), and [0285] iii) the pitch of the signal without low-pass filtering (derived from the signal at channel IC1). [0286] A 10 Hz smoothing of the above signals is performed. [0287] From that stored 10 Hz smoothed signal are computed each of the three time-varying signals i) to iii) mentioned above. [0288] Each thus-computed signal i) to iii) is digitally sampled at e.g. 0.01 second intervals. Each sampling gives an absolute instantaneous value of the intensity (signals i) and ii))) or of the pitch (signal iii)). [0289] For each sampled signal, there is thus obtained a sequence of sampled values, typically a few hundred, given that a phrase is a few seconds long. The thus-obtained sequences of values are stored in time order, starting from the value of the earliest part of the signal, so that the sequence of values expresses the time variation of signal i), ii) or iii) considered. [0290] For each sequence obtained from each of these three signals: intensity after low-pass filtering, intensity without low-pass filtering and pitch without low-pass filtering, the feature extraction unit 70 can calculate the four series of values described above in connection with the feature measurement station 30 of FIG. 3, i.e. [0291] 1) The series of local minimas, by determining the values in the sequence that correspond to a minima and constructing a series formed just of these minima values of the sequence. The series is stored in time order, starting from the earliest minima. A minima in the sequence of values is identified as a member of the sequence whose value is less than that of its neighbour immediately before and immediately after in the sequence. [0292] 2) The series of local maximas, by determining the values in the sequence that correspond to a maxima and constructing a series formed just of these maxima values of the sequence. The series is stored in time order, starting from the earliest maxima. A maxima in the sequence of values is identified as a member of the sequence whose value is greater than that of its neighbour immediately before and immediately after in the sequence. [0293] 3) The series of durations (time intervals) between local (adjacent) extremas�minimas and maximas�(which models rhythmic aspects of the signal) by identifying in the sequence the values that correspond to a minima or a maxima. This is achieved by comparing each value with its immediately adjacent neighbours as for series 1) and 2) above. Then, the positions in time of the identified extremas are analysed to determine the time intervals separating successive extrema. These time intervals are set out in a series, starting from the interval separating the earliest and next earliest extremas, to yield the series of time intervals. [0294] 4) The series itself, by forming a series from all the successive sampled values of the sequence, starting from the beginning of the sampled voice signal. [0295] On the resulting 3�4 (=12) possible series, the feature extraction unit can perform the following 10 statistical operations to produce corresponding features: [0296] 1) the mean, [0297] 2) the maximum (i.e. the overall highest value), [0298] 3) the minimum (i.e. the overall lowest vale), [0299] 4) the difference between the maximum and the minimum, [0300] 5) the variance, [0301] 6) the median, [0302] 7) the first quartile, [0303] 8) the third quartile, [0304] 9) the interquartile range, defined as the difference between the third and first quartiles, [0305] 10) the mean of the absolute value of the local derivative. [0306] Naturally, some of the above statistical features involve a re-ordering of the values in the series from a time developing series to a series sorted by increasing value, notably to determine the first quartile, median and third quartile. [0307] All the data are initially normalised before calculating these statistical characteristics. [0308] The choice of which of these 120 features are to be extracted depends on the algorithms to be used, and is programmable. In the embodiment, the feature extraction unit 70 can be programmed to extract any number of these features according to requirements. At least one of the features is derived from an intensity of a low-pass filtered voice signal on which an emotion recognition is to be performed. [0309] As an example, the feature extraction unit 70 can be programmed to extract at least one the following five statistical features among the ten listed above: [0310] the first quartile, [0311] the median, [0312] the third quartile, [0313] the maximum, and [0314] the minimum. [0315] This extraction can be performed for one or a mix of all three signals i), ii, and iii) listed above, giving up to 15 features that can extracted by unit 70 for each voice signal from which an emotion is to be detected, at least one of the features being extracted from a low-passed intensity signal. [0316] Also, the extraction unit 70 can be additionally programmed to extract any one of features listed 1 to 10 above from the pitch signal extracted from the voice signal after low-pass filtering. In this case, for the thus-obtained low-pass filtered pitch, there can be obtained the same four series of values listed 1 to 4 above, each giving rise to the ten statistical features listed 1 to 10 above. Thus, the extraction unit can further be programmed to extract any number up to 4�10=40 statistical features from the pitch signal, derived after low pass filtering by filter 74. [0317] The feature extraction unit 70 can also be programmed to extract a reduced number of statistical features which correspond e.g. to at least one of the first six features listed in table III above, i.e.: [0318] 1: the median value of sampled low-pass filtered signal intensity values, [0319] 2: the mean value of low-pass filtered signal intensity values, [0320] 3: third quartile value of series of sampled low-pass filtered signal intensity values, [0321] 4: the first quartile value of series of sampled low-pass filtered signal intensity values, [0322] 5: the maximum value of series of sampled low-pass filtered signal intensity values, and [0323] 6: the maximum value of series of sampled low-pass filtered signal intensity values. [0324] Finally, each of the signals listed i) to iii) and�as the case arises�the low-pass filtered pitch signal can be individually programmed to be smoothed, e.g. by 10 Hz smoothing (or some other value), or not smoothed. [0325] The extracted features are temporarily stored in a data logger 82 where they can be accessed by other functional units. [0326] The emotion sensitive system 60 is configured to operate selectively in one of two possible modes, these being respectively off-line training emotion recognition and on-line training emotion recognition. [0327] In the off-line training mode, the training to detect emotions is conducted outside the device, e.g. using the evaluation system as described with reference to FIG. 3. In this case, the programming unit PU of FIG. 3 delivers a complete emotion detection algorithm ready for use. The algorithm can be of the type set out in example 1, for instance, or some quite different algorithm. The emotion recognition can be frozen in its parameters and programming (closed algorithm), or it can be adaptable to some degree to update its parameters and programming on the basis of user feedback response to correct errors or to coach in emotion detection. [0328] In the on-line training mode, there is no initial set emotion recognition algorithm. Rather, there is provided a teaching algorithm which analyses the different features extracted from the feature extraction unit 70 and user feedback, or user indications of the emotions that are to be detected. From this input, the teaching algorithm develops an specific on-line emotion recognition algorithm, which can be continually modified, updated and improved with growing user input and feedback. [0329] In FIG. 14, each of these two modes has its own path of functional units: OFF-L and ON-L respectively for the off-line and on-line modes. [0330] For the off-line mode, there is provided an off-line emotion recognition algorithm unit 84 as mentioned above. The latter inputs the features extracted by unit 70, as present in the data logger 82 in accordance with its parameterisation and programming, and determines from these extracted features an emotion in accordance with its own predetermined settings. In this mode, the voice signals will normally only come from the microphone 62. [0331] For the on-line training mode, there is first constituted a database 86 containing extracted features from which different choices can be investigated by a learning algorithm unit 88. To this end, the database 86 is connected to the data logger 82 so as to accumulate all the extracted features and thus build a corresponding collection of extracted features. A feedback (loop FL1) from the learning algorithm unit 86 to the feature extraction unit 70 allows the latter to be instructed to change or select new features from the possibilities, in order to meet new needs deriving the most appropriate emotion detection algorithm. The teaching algorithm unit 88 can include a program allowing a user to input voice messages with different emotions, or be coupled to receive data from the a database of voices, such as database 8. [0332] Once the teaching algorithm unit 86 has developed the required on-line emotion recognition algorithm, the latter is loaded into an on-line emotion recognition algorithm unit 90. The latter performs the same emotion recognition as that of unit 84, but is produced internally and specifically tailored to a given speaker, such as the user. [0333] The types of emotion detected by either of the emotion detection algorithm units 84 or 90 can be as discussed earlier: calm, sad, happy, angry. Other emotions can of course be accommodated. [0334] The choice of operation in the off-line or on-line is determined by a selector 92 which selects from which emotion detection algorithm 84 or 90 the detected emotions are to be taken. The selection is through an external user input signal 94. [0335] Naturally, the following variants are possible: [0336] the system 60 is provided only with an off-line emotion recognition algorithm 84. The latter can then be totally rigid, or adaptable to a certain degree through feedback response, or [0337] the system is provided only with an on-line emotion algorithm system composed of the teaching algorithm and the on-line emotion recognition algorithm produced by the latter (units 86 to 90). [0338] The programming unit PU of FIG. 3 serves to supply the required information and code for the different functional units. This is achieved through a programming interface PI. In particular, the programming unit PU and its interface PI serve to program the feature extraction unit 70, load the off-line emotion recognition algorithm (unit 84), and/or the teaching algorithm (unit 88), as well as features to database 86 in order to provide additional samples or complementary data. [0339] The output from one or the other of the off-line and on-line emotion recognition algorithm units 84 and 90 is delivered to a detected emotion input unit 94 where the recognised emotion is acknowledged. From that unit 94, the detected emotion is delivered to an emotion response unit 96 for governing all the intended emotional responses. For instance, in the case of a robotic pet, the emotional response unit 96 will command the operation of different electromechanical actuators, lights and audio output devices to simulate an appropriate response. [0340] One response includes giving feedback to the human user 98 or any other controlling person or device to indicate the detected information. This is achieved through a detected emotion indicator 100 which delivers a visual, aural, electrical or mechanical signal indicative of the detected emotion. This makes it possible to check if the identification is correct or not and to take corrective measures as appropriate. [0341] In particular, it allows the user 98 to express a feedback response to a feedback response unit 102 that produces the suitable feedback messages to the various units concerned: off-line emotion recognition algorithm unit 84 (if the latter allows some flexibility), teaching algorithm unit 88, and on-line emotion recognition algorithm unit 90. [0342] From the foregoing, it is clear that the invention can be implemented with many different combinations of features extracted from the signal, at least one feature being extracted from the signal after low-pass filtering. [0343] The choice of algorithm for the emotion detection on the basis of these features can be made on an empirical basis. The algorithms cited in this specification are just some of many algorithms suitable for that purpose. It can also be envisaged to implement the invention with other algorithms, both for teaching and for emotion detection. Referenced byCiting PatentFiling datePublication dateApplicantTitleUS7013005Feb 11, 2004Mar 14, 2006Hewlett-Packard Development Company, L.P.System and method for prioritizing contactsUS7113636 *Aug 30, 2002Sep 26, 2006Lockheed Martin CorporationMethod and computer program product for generating training data for a new class in a pattern recognition classifierUS7260519 *Mar 13, 2003Aug 21, 2007Fuji Xerox Co., Ltd.Systems and methods for dynamically determining the attitude of a natural language speakerUS7584020Jul 5, 2006Sep 1, 2009Battelle Energy Alliance, LlcOccupancy change detection system and methodUS7587260Jul 5, 2006Sep 8, 2009Battelle Energy Alliance, LlcAutonomous navigation system and methodUS7620477 *Jul 5, 2006Nov 17, 2009Battelle Energy Alliance, LlcRobotic intelligence kernelUS7664627 *Oct 2, 2003Feb 16, 2010A.G.I. Inc.Inspirational model device, spontaneous emotion model device, and related methods and programsUS7668621Jul 5, 2006Feb 23, 2010The United States Of America As Represented By The United States Department Of EnergyRobotic guarded motion system and methodUS7801644Jul 5, 2006Sep 21, 2010Battelle Energy Alliance, LlcGeneric robot architectureUS7849034Jun 21, 2006Dec 7, 2010Neuric Technologies, LlcMethod of emulating human cognition in a brain model containing a plurality of electronically represented neuronsUS7914468Sep 21, 2005Mar 29, 2011Svip 4 LlcSystems and methods for monitoring and modifying behaviorUS7953279Jun 28, 2007May 31, 2011Microsoft CorporationCombining online and offline recognizers in a handwriting recognition systemUS7974738Jul 5, 2006Jul 5, 2011Battelle Energy Alliance, LlcRobotics virtual rail system and methodUS8001067 *Apr 7, 2007Aug 16, 2011Neuric Technologies, LlcMethod for substituting an electronic emulation of the human brain into an application to replace a humanUS8073564Jul 5, 2006Dec 6, 2011Battelle Energy Alliance, LlcMulti-robot control interfaceUS8160362Apr 19, 2011Apr 17, 2012Microsoft CorporationCombining online and offline recognizers in a handwriting recognition systemUS8166109 *Jun 21, 2007Apr 24, 2012Cisco Technology, Inc.Linking recognized emotions to non-visual representationsUS8204747May 21, 2007Jun 19, 2012Panasonic CorporationEmotion recognition apparatusUS8271132Mar 13, 2008Sep 18, 2012Battelle Energy Alliance, LlcSystem and method for seamless task-directed autonomy for robotsUS8321412 *Jul 20, 2010Nov 27, 2012National Taiwan UniversityDigital data processing method for personalized information retrieval and computer readable storage medium and information retrieval system thereofUS8340956 *May 23, 2007Dec 25, 2012Nec CorporationInformation provision system, information provision method, information provision program, and information provision program recording mediumUS8355818Sep 3, 2009Jan 15, 2013Battelle Energy Alliance, LlcRobots, systems, and methods for hazard evaluation and visualizationUS8363950Mar 21, 2012Jan 29, 2013Microsoft CorporationCombining online and offline recognizers in a handwriting recognition systemUS8386257Sep 13, 2007Feb 26, 2013Nippon Telegraph And Telephone CorporationEmotion detecting method, emotion detecting apparatus, emotion detecting program that implements the same method, and storage medium that stores the same programUS8473449Dec 22, 2009Jun 25, 2013Neuric Technologies, LlcProcess of dialogue and discussionUS8571853 *Feb 11, 2007Oct 29, 2013Nice Systems Ltd.Method and system for laughter detectionUS8687792Apr 22, 2004Apr 1, 2014Hewlett-Packard Development Company, L.P.System and method for dialog management within a call handling systemUS8738370Jun 2, 2006May 27, 2014Agi Inc.Speech analyzer detecting pitch frequency, speech analyzing method, and speech analyzing programUS20090287469 *May 23, 2007Nov 19, 2009Nec CorporationInformation provision system, information provision method, information provision program, and information provision program recording mediumUS20110022615 *Jul 20, 2010Jan 27, 2011National Taiwan UniversityDigital data processing method for personalized information retrieval and computer readable storage medium and information retrieval system thereofUS20120116186 *Jul 20, 2010May 10, 2012University Of Florida Research Foundation, Inc.Method and apparatus for evaluation of a subject's emotional, physiological and/or physical state with the subject's physiological and/or acoustic dataUS20130054090 *Aug 24, 2012Feb 28, 2013Electronics And Telecommunications Research InstituteEmotion-based vehicle service system, emotion cognition processing apparatus, safe driving service apparatus, and emotion-based safe driving service methodUS20130300645 *May 12, 2012Nov 14, 2013Mikhail FedorovHuman-Computer Interface SystemEP1901281A1 *Jun 2, 2006Mar 19, 2008A.G.I. Inc.Speech analyzer detecting pitch frequency, speech analyzing method, and speech analyzing programWO2006033104A1Sep 21, 2005Mar 30, 2006Shalon Ventures Res LlcSystems and methods for monitoring and modifying behaviorWO2007067878A2 *Dec 4, 2006Jun 14, 2007Ian BennettEmotion detection device & method for use in distributed systems* Cited by examinerClassifications U.S. Classification704/275International ClassificationA63H11/00, B25J13/00, A63H3/33, G10L13/00, G10L15/10, G10L17/26, G10L13/10, G10L13/033, G10L13/04Cooperative ClassificationG10L13/04, G10L13/10, G10L13/043, G10L13/033, G10L17/26European ClassificationG10L13/10, G10L13/033, G10L13/04U, G10L17/26Legal EventsDateCodeEventDescriptionApr 11, 2012FPAYFee paymentYear of fee payment: 4May 18, 2010CCCertificate of correctionSep 9, 2002ASAssignmentOwner name: SONY FRANCE S.A., FRANCEFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:OUDEYER, PIERRE YVES;REEL/FRAME:013265/0127Effective date: 20020717RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services©2012 Google