GENERATING A SEMANTIC SEARCH ENGINE RESULTS PAGE

The present disclosure relates to generating semantic search engine results. Aspects of the present disclosure retrieve relevant information from a search engine based on user's search query. The query can be a classic search query (keyword or short phrase) or a conversational query (e.g., a chat messages between users and/or chatbots), a query based upon an email or other type of message, or a query generate based upon a content item (e.g., a webpage, image, video, document, etc.). Aspects of the disclosure leverage a large language model (LLM), such as, for example, a generative model, to summarizes the content according to the intent detected from the query. In some cases, aspects of the present disclosure may generate a direct answer to the query and provide relevant references to support the information.

BACKGROUND

Classical search engines typically only retrieve and rank relevant content based on the user's query, without providing additional information or analysis. Without additional information, users are required to navigate multiple results to determine information relevant to their query. It is with respect to these and other general considerations that embodiments have been described. Also, although relatively specific problems have been discussed, it should be understood that the embodiments should not be limited to solving the specific problems identified in the background.

SUMMARY

Aspects of the present disclosure relate to systems and methods which provide a sematic search engine that is capable of performing functions beyond the capabilities of a classical search engine, such as, for example, summarizing and generating answers to queries, as well as providing a brief overview of the main facts, aspects or other disambiguation related to the query. Aspects of the disclosure relate to organizing and summarizing information from a retrieval-based search engine into a semantically meaningful format, so the information is more comprehensible and navigable for search engine users.

This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Additional aspects, features, and/or advantages of examples will be set forth in part in the following description and, in part, will be apparent from the description, or may be learned by practice of the disclosure.

DETAILED DESCRIPTION

Aspects of the present disclosure relate to organizing, synthesizing, and summarizing information from a classical retrieval-based search engine into a semantically meaningful format, so that the results are more comprehensible and navigable for users. That is, aspects disclosed herein relate to synthesizing traditional search information in a way that satisfies an intent associated with a received query. As part of the synthetization, aspects of the disclosure may gather additional information from various different data sources, such as local document stores, third-party platforms, applications, and the like in order to address the query intent. Aspects of the disclosure may create a summary that provides an overview of the information initial search results, and then creates disambiguated subsections about different aspects of the original search query based on its intent. These subsections use citation links to attribute the summarized information to their sources to provide credibility. In some examples, rather than creating a summary, aspect disclosed herein may provide an entire document, webpage, dataset, etc. in addition to a summary or in alternative of providing a summary. Among other benefits, aspects of the present disclosure help users quickly find and understand the information they are looking for by providing a curated and structured view of the search engine results page (SERP).

Aspects of the present disclosure retrieve relevant information from a search engine based on user's search query. The query can be a classic search query (keyword or short phrase) or a conversational query (e.g., a chat messages between users and/or chatbots), a query based upon an email or other type of message, or a query generate based upon a content item (e.g., a webpage, image, video, document, etc.). Aspects of the disclosure leverage a large language model (LLM), such as, for example, a generative model, to summarizes the content according to the intent detected from the query. In some cases, aspects of the present disclosure may generate a direct answer to the query and provide relevant references to support the information. Additionally, aspects disclosed herein provide a brief overview of the main facts or aspects related to the user's query, using information from reference documents. The model has access to data such as the date and location of the query, as well as the top web results (e.g., top five result, top five results, top ten results, etc.) and surrounding information and/or contextual information for each result.

Among other technical benefits, aspects of the present disclosure provide capabilities beyond that of a classical search engine by summarizing and generating answers to queries, as well as providing a brief overview of the main facts, aspects or other disambiguation related to the query. Classical search engines typically only retrieve and rank relevant content based on the user's query, without providing additional information or analysis. Our system achieves the new capabilities by leveraging large language models. One of skill in the art will appreciate other technical benefits provided by the aspects disclosed herein.

FIG.1depicts an exemplary system100that includes a semantic search engine120. System100includes a computing device102, a semantic search engine120, and one or more data store(s)106which communicate via a network115. Computing device102may be any of a variety of computing devices, including, but not limited to, a mobile computing device, a laptop computing device, a tablet computing device, a desktop computing device, and/or a virtual reality computing device. Computing device102may be configured to execute one or more application(s)104and/or services and/or manage hardware resources (e.g., processors, memory, etc.), which may be utilized by users of the computing device102. The application(s)104may be a native application or a web-based application. For example, the application(s)104may be a web browser, a digital personal assistant, a file browser, etc. The application(s)104may be used for communication across the network150to submit queries to the semantic search engine120. While not shown, in alternate examples an instance of the semantic search engine120may reside locally on the computing device102.

In examples, the semantic search engine120receives a query from the computing device102and processes the query using query processor124. In one example, the query may be a query for information on a network, such as the Internet. For example, the query can be a query provided to search engine. In other aspects, the query may be generated based upon a user intent derived from a user interaction (e.g., a user interacting with a chatbot, a user selecting a web page or other type of content) and/or from other content items (e.g., emails, documents, web pages, presentations, etc.). In still further examples, aspects of the present disclosure may generate additional queries related to the received query (e.g., disambiguation queries, alternate queries, etc.). In examples, the additional queries may be generated by an associated search engine, by a machine learning model, such as one or more of the models that are part of the model repository130, etc. Query processor124, in examples, processes the query (or queries) and generates an initial set of results in response to receiving the query. For example, the query processor may be a search engine that will generate a set of web search results based upon the received query. The query and the set of search results may be provided to a machine learning model to process the initial sets of results. For example, one or more machine learning (ML) models may be stored in model repository130. The query processor124may provide the results to a model from the repository based upon the type of content retrieved in the search results. In one example, a generative large language model (LLM) may be used to process the search results generated by the query processor124. A generative model (also generally referred to herein as a type of ML model) used according to aspects described herein may generate any of a variety of output types (and may thus be a multimodal generative model, in some examples) and may be a generative transformer model and/or a large language model (LLM), a generative image model, in some examples. Example ML models include, but are not limited to, Generative Pre-trained Transformer 3 (GPT-3), BigScience BLOOM (Large Open-science Open-access Multilingual Language Model), DALL-E, DALL-E 2, Stable Diffusion, or Jukebox. Additional examples of such aspects are discussed below with respect to the generative ML model illustrated inFIGS.5A-5B. The generative LLM may process the search results and determine whether the initial set of results satisfies an intent or task associated with the query. If not, the generative LLM that is part of the semantic search engine120may generate additional searches for information that can be used to satisfy the intent and or task associated with the query. The generated searches may be provided to the query processor124and/or the data source search interface126in order to query one or more additional data sources based upon the generated queries. In examples, different types of data sources106may be searched, e.g., web pages, application data stores, document stores, databases, etc. The data source search interface126helps process the queries across the different data sources. For example, the data source search interface126may include APIs or libraries that can be leveraged to access data from different data sources (e.g., weather information, stock information, third-party databases, etc.) to gather additional information relate to the query and/or related to an intent determined based upon the query and/or user interaction.

Upon retrieving data required to answer the query intent and/or task, the machine learning model employed by the semantic search engine120may summarize the content found in the results. As will be discussed further below, the machine learning model may be prompted to generate the summary in a particular format. Prompt generator128may be used to generate one or more prompts and provide the generated prompts to the ML model. The one or more provided prompts may be used to format the query results summary into a format appropriate for the result summary. In examples, the prompts may include a template that can be used by the machine learning model to format the information.

FIG.2depicts an exemplary method200for generating semantic search engine results. Flow begins at operation202where a query is received. In one example, the query may be a query to search for content on the web, such as a query received by a web search engine. For ease of explanation, examples discussed herein are described with respect to a web search query, however, one of skill in the art will appreciate that the aspects disclosed herein may be used to process other types of queries such as, for example, local directory searches, database searches, document repository queries, social media queries, audio and/or visual search queries, etc. In examples, an intent may be derived from the query. For example, the query may be analyzed, using a rule base system, a heuristics algorithm, and/or a machine learning model, to determine an intent or task associated with the suer query. The intent and/or task may be provided in addition to the query at operation204.

Flow continues to operation204where, in examples, the query is executed the results of the query, or a subset of the results (e.g., top result, top ten results, top one hundred results, data from relevant sources (e.g., information from news sources, weather sources, shopping sources, etc.), or other relevant data sources), are provided to a machine learning model along with the received query. For example, the results may be provided to a generative model, such as a generative LLM. In one example, the underlying content of the search result (e.g., web page content, content from a database executing the query, documents, videos, audio files, etc. identified in response to the query, etc.) may be provided to the database. Alternatively, or additionally, rather than providing the entire content (e.g., an entire web page) a summary of the content may be provided. The summarized data related to the content may be previously generated and retrieved from the database. Alternatively, or additionally, the results may be summarized using one or more different machine learning models prior and the generated summaries may be provided to the generative model. In other aspects, one or more different types of generative machine learning models may receive the search results and the query. The type of model receiving the query may be determined based upon the type of results (e.g., content, format, such as image, text, video, etc.).

Flow continues to decision operation206, where a determination is made as to whether the initial search results are sufficient to respond to the query. For example, the one or more machine learning models that receive the query and the initial query results may determine whether the results answer the query. For example, as previously discussed the query may be analyzed to determine an intent and/or task associated with the query. The intent may be analyzed upon receipt of the query, or may be determined by the generative model at the time of processing the query and results at operation204. Based upon the determined intent and/or task, the search results may be analyzed to determine whether the intent and/or task associated with the query can be sufficiently addressed. If not, then flow branches “No” to operation208.

If the initial query results do not sufficiently satisfy the query (e.g., do not adequately satisfy an intent or task associated with the query), the one or more machine learning models may generate additional search queries. The additional queries may be directed towards information not explicitly requested by the query. As an example, the received initial query may be: “Is February a good time to visit Japan.” A machine learning model may determine that the intent of the query is to plan a vacation to Japan in February. While the initial search results, for example, generated by a web search engine, may provide links to articles about Japan in February, the one or more machine learning models may determine to that the intent requires a more comprehensive answer, which could require additional information. In making the determination, the machine learning model may generate additional queries, such as, for example, “Weather in Japan in February,” “Things to do in Japan in February,” “Things to do in Tokyo in February,” “Things to do in Tokyo in February,” “Flights to Japan,” etc. These additional queries may be executed, for example, using a search engine, to generate additional results that that can then be processed by the one or more machine learning models.

At operation208, the queries for additional information may be executed, for example, by a search engine, file system, database, etc., and the additional search results, and, optionally, the additional queries, may be provided to the one or more machine learning models. The additional information retrieved from these additional queries can be used to provide a comprehensive response to the initial query, thereby satisfying the intent and/or task determined for the initial query without requiring a multi-step process of communication with the user. Flow then continues to operation210.

Returning briefly to decision operation206, if the one or more machine learning models determine that the initial set of query results satisfies the intent and/or task determined based upon the query, flow branches Yes to operation210.

While examples provided herein relate to web search queries, the additional queries are not limited to web searches. For example, the additional queries generated by the one or more machine learning models may be queries to search a local device or datastore for information (in instances where the user has given the one or more ML models permission to search local data stores or may be directed to other data stores (e.g., API calls to query applications, database queries, calls to specific data repositories, such as stock data or weather data, etc.).

At operation210, one or more prompts may be provided to the ML model. The one or more provided prompts may be used to format the query results summary into a format appropriate for the result summary. In examples, operation210may be optional. That is, the one or more prompts to format the results may be provided earlier, for example, with the initial query and set of results, with the additional search results generated at operation208, etc. In examples, the one or more prompts may be templates that can be used to format or summarize the information generated by one or more generative models. In examples, the templates may be selected based upon the type of data generated by a generative model, based upon a task associated with the query, based upon an intent associated with the query, etc.

Aspects of the present disclosure are operable to utilize a general ML model, that is, a model that is not trained specifically to generate semantic search engine results. For example, a generative large language model may be employed by the method200. Generally, LLMs are not trained to perform specific tasks. Accordingly, the one or more prompts generated and provided at operation210instruct the generative LLM (or other types of generative machine learning models) to generate a summary of the results in a format that is appropriate to the originally received query, and/or appropriate based upon the determined intent and/or task associated with the query.

Flow continues to operation212, where a summary generated by the one or more machine learning models is received from the one or more machine learning models. While traditional search engines generally return a link of webpages or files that match the search query, aspects of the present disclosure generate a summary that provides a detailed summary of the content related to the query. In examples, the summary of the content is formatted based upon the one or more prompts generated at operation210. Further, in example, the summary includes citations to the underlying data source (e.g., webpages, documents, video, etc.) for the information included in the summary. The links may be selectable, such that a user may be redirected to the source material by selecting the citation. Alternatively, depending upon the determined intent, a direct answer may be generated. For example, if the query intent relates to specific information, such as a query “What is Abraham Lincoln's Birthday?”, a direct answer may be generated, such that the answer is provided without a summary. In still further aspects, a direct answer and a summary may both be generated and/or provided.

For example,FIG.4provides an exemplary user interface400depicting a summary of information generated by a sematic search engine. Turning now toFIG.4, an exemplary user interface400is provided, in the form of a chat interface, in which the query402about visiting Japan is received. In examples, the query may be received via the text box401. In alternate user interfaces, not shown inFIG.4, the query may be received via a different UI component, such as an address bar in an internet browser, via a search engine text box, via audio (e.g., a spoken query, etc. As shown inFIG.4, the query402may be a natural language query. The semantic search engine generates a detailed summary of information about Japan which can be used to answer the user's query. In examples, the detailed summary may be broken into different sections, based upon topic. For example, section404details general activities in Japan, section406shows things to do in Tokyo, section408details things to do in Kyoto, and section410details things to do in Niseko. Further, the generative LLM may generate a summary that include references points, which when activated by a user, may direct the user to the source of the information.

While a specific user interface is shown inFIG.4, alternate user interfaces may be employed, such as a search interface integrated into a browser or webpage, a search interface that is part of an operating system or application, etc. For example, in alternate aspects, the summary of information may be included in a web page that includes traditional search results. For example, a summary of the search results may be included before, in the middle, or after a traditional listing of results generated by a search engine. In still further examples, the summary may be displayed as part of other applications user interface (e.g., within a mobile application, a file browser, and operating system feature, etc.). In still further aspects, although not shown inFIG.4, the summary may include various types of content in addition to, or instead of, a textual summary. For example, the summary may include images, videos, animations, audio playback, other type of generated resources (e.g., documents, spreadsheets, presentations, etc.) that can be displayed as part of the summary. One of skill in the art will apricate that the summary can be included in a number of different user interfaces that are capable of receiving and displaying the information generated by aspects of the present disclosure.

Returning toFIG.2, at operation214, the semantic search engine results are provided. In one example, the results may be provided to an application which received the initial query (e.g., a web browser, a chat interface, etc.). In another example, providing the results may include displaying the results or causing the results to be displayed. In still further examples, the semantic search engine results may be stored for future use. That is, semantic search engine results may be indexed and stored for retrieval upon receiving subsequent queries that are similar or have a similar intent. In doing so, the response time for future queries can be decreased, as the summaries can be retrieved rather than generated in response to the query.

FIG.3is a block diagram illustrating a method300for generating prompts for a machine learning model that is leveraged by a semantic search engine. Flow begins at operation302, where a query is analyzed to determine an intent and/or task associated with the query. In one example, a machine learning model may be used to determine an intent or task associated with a query. Alternatively, or additionally, a rules-based process, a query parser, or other type of process may be utilized to determine an intent associated with the application.

Upon determining and intent and/or task for the query, flow continues to operation304, where a response format is determined. The response format may be based upon a predicted content summary responding to the query. For example, the predicted content type, content length, etc. may be used to determine an appropriate format to best present a response to the query.

At operation306, one or more prompts may be determined based upon the determined response format. As noted, a generative LLM may be leveraged by a sematic search engine to generate responses to a query. The generative LLM may not be trained to generate the specific type of response, or response format, required to satisfy the query intent and/or task. Rather than fine-tuning the generative LLM, which may be a long and expensive process, one or more prompts may be generated and provided to guide the LLM to produce responses in the desired format. In one example, the one or more prompts may be generated by selecting appropriate pre-defined prompt templates from a prompt data store. In another example, a machine learning model trained to generate prompts may be employed to generate one or more prompts based upon the received query (e.g., the query received at operation202ofFIG.2). Upon generating the prompts, flow continues to operation308where the one or more prompts are provided to the machine learning model that is leveraged by the semantic search engine.

FIGS.5A and5Billustrate overviews of an example generative machine learning model that may be used according to aspects described herein. With reference first toFIG.5A, conceptual diagram500depicts an overview of pre-trained generative model package504that processes an input and a prompt502to generate model output506aspects described herein.

In examples, generative model package504is pre-trained according to a variety of inputs (e.g., a variety of human languages, a variety of programming languages, and/or a variety of content types) and therefore need not be finetuned or trained for a specific scenario. Rather, generative model package504may be more generally pre-trained, such that input502includes a prompt that is generated, selected, or otherwise engineered to induce generative model package504to produce certain generative model output506. It will be appreciated that input502and generative model output506may each include any of a variety of content types, including, but not limited to, text output, image output, audio output, video output, programmatic output, and/or binary output, among other examples. In examples, input502and generative model output506may have different content types, as may be the case when generative model package504includes a generative multimodal machine learning model.

As such, generative model package504may be used in any of a variety of scenarios and, further, a different generative model package may be used in place of generative model package504without substantially modifying other associated aspects (e.g., similar to those described herein with respect toFIGS.1,2,3, and4). Accordingly, generative model package504operates as a tool with which machine learning processing is performed, in which certain inputs502to generative model package504are programmatically generated or otherwise determined, thereby causing generative model package504to produce model output506that may subsequently be used for further processing.

Generative model package504may be provided or otherwise used according to any of a variety of paradigms. For example, generative model package504may be used local to a computing device (e.g., computing device102inFIG.1) or may be accessed remotely from a machine learning service (e.g., semantic search engine120). In other examples, aspects of generative model package504are distributed across multiple computing devices. In some instances, generative model package504is accessible via an application programming interface (API), as may be provided by an operating system of the computing device and/or by the machine learning service, among other examples.

With reference now to the illustrated aspects of generative model package504, generative model package504includes input tokenization508, input embedding510, model layers512, output layer514, and output decoding516. In examples, input tokenization508processes input502to generate input embedding510, which includes a sequence of symbol representations that corresponds to input502. Accordingly, input embedding510is processed by model layers512, output layer514, and output decoding516to produce model output506. An example architecture corresponding to generative model package504is depicted inFIG.5B, which is discussed below in further detail. Even so, it will be appreciated that the architectures that are illustrated and described herein are not to be taken in a limiting sense and, in other examples, any of a variety of other architectures may be used.

FIG.5Bis a conceptual diagram that depicts an example architecture550of a pre-trained generative machine learning model that may be used according to aspects described herein. As noted above, any of a variety of alternative architectures and corresponding ML models may be used in other examples without departing from the aspects described herein.

As illustrated, architecture550processes input502to produce generative model output506, aspects of which were discussed above with respect toFIG.5A. Architecture550is depicted as a transformer model that includes encoder552and decoder554. Encoder552processes input embedding558(aspects of which may be similar to input embedding510inFIG.5A), which includes a sequence of symbol representations that corresponds to input556. In examples, input556includes input and prompt for generation502(e.g., corresponding to a skill of a skill chain).

Further, positional encoding560may introduce information about the relative and/or absolute position for tokens of input embedding558. Similarly, output embedding574includes a sequence of symbol representations that correspond to output572, while positional encoding576may similarly introduce information about the relative and/or absolute position for tokens of output embedding574.

As illustrated, encoder552includes example layer570. It will be appreciated that any number of such layers may be used, and that the depicted architecture is simplified for illustrative purposes. Example layer570includes two sub-layers: multi-head attention layer562and feed forward layer566. In examples, a residual connection is included around each layer562,566, after which normalization layers564and568, respectively, are included.

Decoder554includes example layer590. Similar to encoder552, any number of such layers may be used in other examples, and the depicted architecture of decoder554is simplified for illustrative purposes. As illustrated, example layer590includes three sub-layers: masked multi-head attention layer578, multi-head attention layer582, and feed forward layer586. Aspects of multi-head attention layer582and feed forward layer586may be similar to those discussed above with respect to multi-head attention layer562and feed forward layer566, respectively. Additionally, masked multi-head attention layer578performs multi-head attention over the output of encoder552(e.g., output572). In examples, masked multi-head attention layer578prevents positions from attending to subsequent positions. Such masking, combined with offsetting the embeddings (e.g., by one position, as illustrated by multi-head attention layer582), may ensure that a prediction for a given position depends on known output for one or more positions that are less than the given position. As illustrated, residual connections are also included around layers578,582, and586, after which normalization layers580,584, and588, respectively, are included.

Multi-head attention layers562,578, and582may each linearly project queries, keys, and values using a set of linear projections to a corresponding dimension. Each linear projection may be processed using an attention function (e.g., dot-product or additive attention), thereby yielding n-dimensional output values for each linear projection. The resulting values may be concatenated and once again projected, such that the values are subsequently processed as illustrated inFIG.5B(e.g., by a corresponding normalization layer564,580, or584).

Feed forward layers566and586may each be a fully connected feed-forward network, which applies to each position. In examples, feed forward layers566and586each include a plurality of linear transformations with a rectified linear unit activation in between. In examples, each linear transformation is the same across different positions, while different parameters may be used as compared to other linear transformations of the feed-forward network.

Additionally, aspects of linear transformation592may be similar to the linear transformations discussed above with respect to multi-head attention layers562,578, and582, as well as feed forward layers566and586. Softmax594may further convert the output of linear transformation592to predicted next-token probabilities, as indicated by output probabilities596. It will be appreciated that the illustrated architecture is provided in as an example and, in other examples, any of a variety of other model architectures may be used in accordance with the disclosed aspects.

Accordingly, output probabilities596may thus form model output506according to aspects described herein, such that the output of the generative ML model defines an output corresponding to the input. For instance, model output506may be associated with a corresponding application and/or data format, such that model output is processed to display the semantic search engine page, among other examples.

FIG.6is a block diagram illustrating physical components (e.g., hardware) of a computing device600with which aspects of the disclosure may be practiced. The computing device components described below may be suitable for the computing devices described above, including computing device102inFIG.1. In a basic configuration, the computing device600may include at least one processing unit602and a system memory604. Depending on the configuration and type of computing device, the system memory604may comprise, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories.

The system memory604may include an operating system605and one or more program modules606suitable for running software application620, such as one or more components supported by the systems described herein. As examples, system memory604may store semantic search engine624and/or machine learning model(s)626. The operating system605, for example, may be suitable for controlling the operation of the computing device600.

Furthermore, aspects of the disclosure may be practiced in conjunction with a graphics library, other operating systems, or any other application program and is not limited to any particular application or system. This basic configuration is illustrated inFIG.6by those components within a dashed line608. The computing device600may have additional features or functionality. For example, the computing device600may also include additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated inFIG.6by a removable storage device609and a non-removable storage device610.

As stated above, a number of program modules and data files may be stored in the system memory604. While executing on the processing unit602, the program modules606(e.g., application620) may perform processes including, but not limited to, the aspects, as described herein. Other program modules that may be used in accordance with aspects of the present disclosure may include electronic mail and contacts applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-aided application programs, etc.

The computing device600may also have one or more input device(s)612such as a keyboard, a mouse, a pen, a sound or voice input device, a touch or swipe input device, etc. The output device(s)614such as a display, speakers, a printer, etc. may also be included. The aforementioned devices are examples and others may be used. The computing device600may include one or more communication connections616allowing communications with other computing devices650. Examples of suitable communication connections616include, but are not limited to, radio frequency (RF) transmitter, receiver, and/or transceiver circuitry; universal serial bus (USB), parallel, and/or serial ports.

FIG.7is a block diagram illustrating the architecture of one aspect of a computing device. That is, the computing device can incorporate a system (e.g., an architecture)702to implement some aspects. In some examples, the system702is implemented as a “smart phone” capable of running one or more applications (e.g., browser, e-mail, calendaring, contact managers, messaging clients, games, and media clients/players). In some aspects, the system702is integrated as a computing device, such as an integrated personal digital assistant (PDA) and wireless phone.

One or more application programs766may be loaded into the memory762and run on or in association with the operating system764. Examples of the application programs include phone dialer programs, e-mail programs, personal information management (PIM) programs, word processing programs, spreadsheet programs, Internet browser programs, messaging programs, and so forth. The system702also includes a non-volatile storage area768within the memory762. The non-volatile storage area768may be used to store persistent information that should not be lost if the system702is powered down. The application programs766may use and store information in the non-volatile storage area768, such as e-mail or other messages used by an e-mail application, and the like. A synchronization application (not shown) also resides on the system702and is programmed to interact with a corresponding synchronization application resident on a host computer to keep the information stored in the non-volatile storage area768synchronized with corresponding information stored at the host computer. As should be appreciated, other applications may be loaded into the memory762and run on the mobile computing device700described herein (e.g., an embedding object memory insertion engine, an embedding object memory retrieval engine, etc.).

The system702has a power supply770, which may be implemented as one or more batteries. The power supply770might further include an external power source, such as an AC adapter or a powered docking cradle that supplements or recharges the batteries.

The system702may also include a radio interface layer772that performs the function of transmitting and receiving radio frequency communications. The radio interface layer772facilitates wireless connectivity between the system702and the “outside world,” via a communications carrier or service provider. Transmissions to and from the radio interface layer772are conducted under control of the operating system764. In other words, communications received by the radio interface layer772may be disseminated to the application programs766via the operating system764, and vice versa.

A computing device implementing the system702may have additional features or functionality. For example, the computing device may also include additional data storage devices (removable and/or non-removable) such as, magnetic disks, optical disks, or tape. Such additional storage is illustrated inFIG.7by the non-volatile storage area768.

Data/information generated or captured by the computing device and stored via the system702may be stored locally on the computing device, as described above, or the data may be stored on any number of storage media that may be accessed by the device via the radio interface layer772or via a wired connection between the computing device and a separate computing device associated with the computing device, for example, a server computer in a distributed computing network, such as the Internet. As should be appreciated such data/information may be accessed via the computing device via the radio interface layer772or via a distributed computing network. Similarly, such data/information may be readily transferred between computing devices for storage and use according to well-known data/information transfer and storage means, including electronic mail and collaborative data/information sharing systems.

FIG.8illustrates one aspect of the architecture of a system for processing data received at a computing system from a remote source, such as a personal computer804, tablet computing device806, or mobile computing device808, as described above. Content displayed at server device802may be stored in different communication channels or other storage types. For example, various documents may be stored using a directory service824, a web portal825, a mailbox service826, an instant messaging store828, or a social networking site830.

An application820(e.g., similar to the application620) may be employed by a client that communicates with server device802. Additionally, or alternatively, machine learning models821may be employed by server device802. The server device802may provide data to and from a client computing device such as a personal computer804, a tablet computing device806and/or a mobile computing device808(e.g., a smart phone) through a network815. By way of example, the computer system described above may be embodied in a personal computer804, a tablet computing device806and/or a mobile computing device808(e.g., a smart phone). Any of these examples of the computing devices may obtain content from the store816, in addition to receiving graphical data useable to be either pre-processed at a graphic-originating system, or post-processed at a receiving computing system.

In one example, aspects of the present disclosure relate to a system comprising: at least one processor; and memory storing instructions that, when executed by the at least one processor, cause the system to perform a set of operations, the set of operations comprising: receive a query; generate an initial set of query results; provide the query and initial set of query results to a generative large language model; receive at least one additional query from the generative large language model; execute the at least one additional query; provide the results from the at least one additional query to the generative large language model; receive semantic search engine results from the generative large language model; and provide the semantic search engine results.

In yet another example, the system further comprises instruction to generate one or more alternate queries based upon the query, and wherein generating the initial set of query results comprised generating alternate query results based upon the one or more additional queries.

In still another example, the semantic search engine results are included in a summary generated by the generative large language model.

In another example, a format for the summary is determined based upon a type of information included in the summary.

In yet another example, a format for the summary is determined based upon a template provided to the generative large language model.

In still further examples, the summary includes one or more citations, and wherein the one or more citations link to one or more underlying data sources for the summary.

In further examples, the system comprises, determining an intent or a task based upon the received query, wherein the intent or the task is provided to the generative large language model.

In still further examples, the system further comprises operations to determine, using the generative large language model, whether additional information is required, wherein the determination is based upon the intent or task.

In yet further examples, the at least one additional query is generated by the generative large language model when it is determined that an additional information is required.

In another example, aspects of the disclosure relate to a method for generating semantic search engine results, the method comprising: receiving a query; generating an initial set of query results; providing the query and initial set of query results to a generative model; determining, using the generative model, that additional information is needed; receiving at least one additional query from the generative model; execute the at least one additional query; provide the results from the at least one additional query to the generative large language model; receive semantic search engine results from the generative model; and provide the semantic search engine results.

In examples, the method further comprises analyzing the query to determine an intent or a task based upon the query, wherein analyzing the query comprises providing the query to at least one of the generative model or an alternate machine learning model.

In yet another example, the method further comprises determining a format for the semantic search engine results, wherein the format is determined based upon the query or the task.

In further examples, the method further comprises generating a prompt for the generative model, wherein the prompt is generated based upon the format.

In still further examples the prompt comprises a template associated with the format, wherein the template defines the format for the semantic search engine results.

In another example, the generative model is a generative large language model.

In still another example, the semantic search engine results are included in a summary generated by the generative large language model, and wherein the summary includes one or more citations, and wherein the one or more citations link to one or more underlying data sources for the summary.

In still further aspects, aspects of the present discloser relate to computer storage medium comprising computer-executable instructions that, when executed by at least one processing unit, performs a method for generating semantic search engine results, the method comprising: receiving a query; generating an initial set of query results; providing the query and initial set of query results to a generative large language model; determining, using the generative model, that additional information is needed; receiving at least one additional query from the generative large language model; execute the at least one additional query; provide the results from the at least one additional query to the generative large language model; receive semantic search engine results from the generative large language model; and provide the semantic search engine results.

In examples, the method further comprises: analyzing the query to determine an intent or a task based upon the query, wherein analyzing the query comprises providing the query to at least one of the generative model or an alternate machine learning model; and determining a format for the semantic search engine results, wherein the format is determined based upon the query or the task.

In yet another example, the method further comprises generating a prompt for the generative model, wherein the prompt is generated based upon the format.

In still further examples, the semantic search engine results are included in a summary generated by the generative large language model, and wherein the summary includes one or more citations, and wherein the one or more citations link to one or more underlying data sources for the summary.