Patent ID: 7630901

Claim:
A multimodal input method comprising: a receiving step of receiving first input information acquired by a speech input source during a speech input period and second input information indicating at least one pointed item acquired by a pointing input source during the speech input period: a speech recognition step of recognizing a plurality of recognition candidates based on the first input information; a determining step of determining whether the number of unknown items included in a respective recognition candidate is equal to the number of the pointed items acquired by the pointing input source; if it is determined in the determining step that the number unknown items included in the respective recognition candidate is not equal to the number of the pointed items acquired by the pointing input source, repeating the determining step with a different respective recognition candidate in order to obtain a matching recognition candidate which is a respective recognition candidate whose number of unknown items is equal to the number of the at least one pointed items; and an output step of outputting a recognition result in which the at least one unknown item included in the matching recognition candidate is replaced with the at least one pointed item acquired by the pointing input source.