Patent Document ID: 7596767
Application ID: 11156873

Base Claim:
1. A multimodal system for controlling electronic components, comprising: a general purpose computing system which is in communication with said electronic components via a computer network, said electronic components being separate from the computing system; a computer program comprising program modules executable by the computing system, said program modules comprising: an object selection module that identifies an object selected by a user via a pointing device associated with at least one camera and at least one light-emitting diode (LED), a gesture recognition module that recognizes one or more motions of the pointing device in three-dimensional space, the pointing device associated with at least one accelerometer, and a speech control module that identifies a component selected by a user, each of the object selection module, the gesture recognition module, and the speech control module providing inputs to an integration module that integrates said inputs to arrive at a unified interpretation of what object the user wants to control and what control action is desired.

---

Claim 2:
2. The system of claim 1 , wherein the integration module comprises a dynamic Bayes network which determines from the individual inputs of the object selection module, gesture recognition module, and speech control module, the identity of a component the user wants to control (i.e., the referent), a command that the user wishes to implement (i.e., the command), and the appropriate control action to be taken to affect the identified referent in view of the command.