Patent ID: 6990639

Claim:
A multimodal electronic component control system comprising: an object selection subsystem; a gesture recognition subsystem; a speech control subsystem; and an integration subsystem into which the obiect selection, gesture recognition and speech control subsystems provide inputs, said integration subsystem integrating said inputs to arrive at a unified interpretation of what component a user wants to control and what control action is desired, and wherein the integration subsystem comprises, a dynamic Bayes network which determines from the individual inputs of the obiect selection, gesture recognition, and speech control subsystems, the identity of a component the user wants to control (i.e., the referent), a command that the user wishes to implement (i.e., the command), and the appropriate control action to be taken to affect the identified referent in view of the command, said dynamic Bayes network comprising input, referent, command and action nodes, wherein the input nodes include said individual inputs which provide information as to their state to at least one of a referent, command, or action node, said inputs determining the state of the referent and command nodes, and wherein the states of the referent and command nodes are fed into an action node whose state indicates the action that is to be implemented to affect the referent, and wherein said referent, command and action node states comprise probability distributions indicating the probability that each possible referent, command and action is the respective referent, command and action.