Patent ID: 8487867

Claim:
A behavior recognition system by combining an image and a speech, comprising: a database, for storing a plurality of image-and-speech relation modules, wherein each of the image-and-speech relation modules comprises a feature extraction parameter and an image-and-speech relation parameter; a data analyzing module, for substituting a gesture image and a speech data corresponding to each other into each feature extraction parameter to obtain a plurality of image feature sequences and a plurality of speech feature sequences, and substituting each image feature sequence and each speech feature sequence corresponding to a same image-and-speech relation module into each image-and-speech relation parameter, so as to calculate a plurality of image-and-speech status parameters, wherein each image feature sequence comprises a plurality of image frame data, and the image frame data forms a plurality of image frame status combinations; each speech feature sequence comprises a plurality of speech frame data, and the speech frame data forms a plurality of speech frame status combinations, when the data analyzing module calculates each one of the image-and-speech status parameters, the data analyzing module substitutes each image frame status combination and each speech frame status combination into the image-and-speech relation parameter corresponding to the same image-and-speech relation module to calculate a plurality of image-and-speech sub-status parameters and selects one image-and-speech sub-status parameter from the plurality of image-and-speech sub-status parameters to serve as the image-and-speech status parameter corresponding to the image-and-speech relation module; and a calculating module, for using the image feature sequences, the speech feature sequences, and the image-and-speech status parameters to calculate a recognition probability corresponding to each of the image-and-speech relation modules, and taking a target parameter from the recognition probabilities.