Patent ID: 7809720

Claim:
A method of determining data content for a response to a query, comprising the steps of: obtaining a user query; selecting one or more of a plurality of modalities for outputting a response to the query; dynamically determining data content suitable for generating a response to the query, wherein data content determination is modeled as an optimization operation which attempts to balance context-based selection constraints; and outputting the response to the query using the selected one or more of the plurality of modalities; wherein the data content determination further comprises modeling the context-based selection constraints as feature-based metrics, the feature-based metrics measuring a presentation desirability value and one or more cost values associated with the selected one or more of the plurality of modalities; and wherein the selection comprises: determining a cost value associated with a visual presentation of at least a portion of the response; determining a cost value associated with a spoken presentation of the at least a portion of the response; comparing the cost value associated with the visual presentation and the cost value associated with the spoken presentation; and selecting at least one of the visual presentation and the spoken presentation based at least in part on the comparison of the cost value associated with the visual presentation and the cost value associated with the spoken presentation.