Patent ID: 7640160
Filing Date: 2009-12-29
Classification: G06F,G10L

Abstract:
1. A system for processing multi-modal natural language inputs, comprising: a multi-modal voice user interface configured to receive a multi-modal input, the multi-modal input including a natural language utterance and a non-speech input, wherein a transcription module coupled to the multi-modal voice user interface is configured to transcribe the non-speech input to create a non-speech-based transcription; a multi-pass speech recognition module configured to transcribe the natural language utterance into text; a merging module configured to merge the text of the transcribed utterance and the non-speech-based transcription to create a merged transcription; a plurality of domain agents, wherein a context description grammar includes one or more grammar expression entries that one or more of the plurality of domain agents are configured to use to process requests in respective contexts; a knowledge-enhanced speech recognition engine configured to determine a most likely context for the multi-modal input, the knowledge-enhanced speech recognition engine further configured to: a response generating module configured to identify one or more of the plurality of domain agents that are configured to process requests in the most likely context for the multi-modal input, the response generating module configured to: