Patent ID: 8457959
Filing Date: 2013-06-04
Classification: G09B,G10L

Abstract:
1. A system for recognizing and evaluating possible relationships between terms expressed during cross-communication activities, the system comprising: a memory; a processor in signal communication with the memory; a speech recognition system having a speech collection device arranged to receive an ambiguously delimited speech signal and then transcribe the speech signal to a first plurality of sequences of articulatory features related to a portion of the speech signal; an ink segmentation and recognition system having an ink input receiving device configured to receive an ambiguously delimited digital ink input while the speech recognition system receives the speech portion, the ink segmentation and recognition system further configured to segment ink input that constitutes sketches from those that constitutes handwriting and then identify a second plurality of sequences of articulatory features related to the handwriting; and a multimodal fusion engine in signal communication with the processor, the multimodal fusion engine having a search alignment system configured to substantially align the articulatory features derived from the ambiguously delimited speech signal and the ambiguously delimited ink input using a coherence measure across articulatory features representations of the ambiguous inputs as between a candidate portion of the speech signal a candidate portion of the ink input.