Patent Document ID: 9396252
Application ID: 14554200

Base Claim:
1. A method comprising: receiving a search query, wherein the search query comprises a first input modality and a second input modality, the first input modality being distinct from the second input modality; retrieving first modality search results for the search query using the first input modality of the search query; outputting, on a display and all at once, the first modality search results; retrieving, from a database, partial second modality search results based on the first modality search results and the second input modality as the second input modality is being entered; outputting, on the display, the partial second modality search results incrementally as the second input modality is still being entered; retrieving, from the database, final second modality search results based on the first modality search results and the second input modality upon the second input modality being completely entered; and outputting, on the display, the final second modality search results.

---

Claim 6:
6. The method of claim 1 , further comprising: receiving a rephrased multimodal search query after receiving the search query; incrementally recognizing the rephrased multimodal search query to identify rephrased search terms; retrieving additional search results based on the search query, the rephrased search terms, the first modality search results, and the final second modality search results; and outputting the additional search results on the display before the recognizing of the rephrased multimodal search query is complete.