Patent ID: 11899677
Assignee: ELASTICSEARCH B.V.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 5:
6. An automated method of providing curated responses combined with organic search results to a query, comprising:
utilizing a client device to access a search engine via a network, the search engine interfacing with a relevance model executing on a query system, the query system comprising a specially configured computing resource including a curation engine and the relevance model, the relevance model accessing a target resource for providing responses to queries from the client device over the network;
obtaining, at the relevance model from the client device, user action metrics over a period of time, the user action metrics pertaining to user actions with respect to model responses generated by the relevance model;
determining, by the relevance model using the machine learning model, queries having a ranking above a threshold value;
flagging the queries as automatable;
displaying, at the client device, a list of the flagged queries along with controls for adjusting, using the curation engine, curations of the model responses for the flagged queries, and
identifying a portion of the model responses not having statistical significance, wherein the portion continues to be generated using the relevance model and displayed as model results along with the list of flagged queries and
determining a defined length for the period of time based on a volume of user actions.