Patent Description:
When the user is familiar with a source of an incoming call, the user may accept the call under the assumption that the call will have some amount of utility to themselves or the source of the call. However, during the call, users will often refer to other sources of information, such as other applications and/or devices, in order to refresh their recollection of any subject matter relevant to the call. Such referencing of other sources external to the computing device that is actively participating in the phone call can prolong the call and cause multiple overlapping layers of network activity. For instance, a user that is attempting to access, during a call, information that is relevant to the call, can cause requests for data over a network that is the same as, or different from, the network handling the call. This can result in excess usage of bandwidth during the call and may, where the network is the same, degrade the quality of the call. Moreover, the call can be prolonged as a result of the user needing to access, during the call, the information that is relevant to the call. <CIT> discloses a terminal and method for displaying caller information.

<FIG> illustrates a view <NUM> of a user <NUM> being presented pre-call content <NUM> when the user <NUM> is receiving an incoming call from another user <NUM>. The other user <NUM> can initialize the call using a computing device <NUM>, which can communicate with a computing device <NUM> over a network <NUM>. When the other user <NUM> initializes the call with the user <NUM>, the computing device <NUM> can provide pre-call content <NUM> and/or a notification <NUM> in anticipation of the user <NUM> indicating their acceptance of the call via the computing device <NUM>. The pre-call content <NUM> includes first pre-call content <NUM> and second pre-call content <NUM>, described in more detail below. The notification <NUM> can be presented at a display panel <NUM> of the computing device <NUM> in response to the other user <NUM> initializing the phone call to the user <NUM>. Additionally, or alternatively, the pre-call content <NUM> can also be presented to the user <NUM> and/or the other user <NUM> via a display panel <NUM> in response to the other user <NUM> initializing a call to the user <NUM>.

The pre-call content <NUM> can be generated based on data from one or more different sources. In some implementations, the pre-call content <NUM> can be generated using one or more trained machine learning models. For instance, an input can be applied to a trained machine learning model, and an output that is based on the input being applied to the trained machine learning model can be used as a basis for generating the pre-call content <NUM>. The pre-call content <NUM> can be generated prior to the other user <NUM> initializing the call to the user <NUM>, in order to eliminate latency that may otherwise be exhibited. For instance, prior to the other user <NUM> initializing the call to the user <NUM>, the various types of data can be applied to one or more trained machine learning model, such as calendar data and/or message data. Output data, resulting from these types of data being applied to the one or more trained machine learning models, can be used as a basis for generating first pre-call content <NUM> and second pre-call content <NUM>.

For example, in order to generate the first pre-call content <NUM>, calendar data and other data identifying the other user <NUM> can be applied to a trained machine learning model. Output data that is provided as a result can be used as a basis for generating the first pre-call content <NUM>, which can include calendar data that is associated with the other user <NUM>. Furthermore, message data and data identifying the other user <NUM>, can be applied to a trained machine learning model. Output data that is provided as a result can be used as a basis for generating the second pre-call content <NUM>, which can include message data that is also associated with the other user <NUM>. Subsequent to generating such pre-call content, should the other user <NUM> initialize a call to the user <NUM>, the pre-call content <NUM> can be presented to the user <NUM> and/or the other user <NUM> via the computing device <NUM> and/or the other computing device <NUM>, respectively.

As an example, and as provided in <FIG>, in response to the other user <NUM> initializing the call, the pre-call content <NUM> can be presented at the display panel <NUM>. As a result, the user <NUM> is able to review the pre-call content <NUM> prior to indicating acceptance of the call. For instance, the display panel <NUM> can be a touch display panel <NUM> that allows the user <NUM> to provide one or more gestures in order to scroll through the pre-call content <NUM> in order to view more of the first pre-call content <NUM> and/or view more of the second pre-call content <NUM>. In some implementations, the first pre-call content <NUM> can be initially displayed and the second pre-call content <NUM> can be partially displayed, or omitted, until the user <NUM> provides one or more gestures indicating the user <NUM> would be interested in viewing the second pre-call content <NUM>. For instance, the user <NUM> can be providing a tap gesture, slide gesture, single touch gesture, multi-touch gesture, spoken gesture, facial gesture, and/or any other gesture that can be provided as input to a computing device.

When the user <NUM> indicates acceptance <NUM> of the incoming call from the other user <NUM>, one or more communication channels can be established between the computing device <NUM> and the other computing device <NUM> via the network <NUM>. The call can be a voice call, video call, both video and voice call, and/or any other type of call that can be executed via computing devices. As an example, and as provided in <FIG>, the call can be a voice call in which the other user <NUM> and the user <NUM> can provide verbal inputs. For instance, the other user <NUM> can provide a first input <NUM> such as, "Hey, looking forward to the event," and the user <NUM> can provide a second input <NUM> such as, "Great. Tim wants to meet beforehand. " The first input <NUM> and/or the second input <NUM> can be dialog content of the call between the user <NUM> and the other user <NUM>.

The dialog content, with prior permission from the user <NUM> and the other user <NUM>, can be captured as data that can be used as a basis for generating subsequent pre-call content <NUM>, as provided in view <NUM> of <FIG>. For instance, the dialog content can be compared to the pre-call content <NUM> to determine whether one or more correlations between the dialog content and the pre-call content <NUM> were apparent. A correlation that can be identified may include the second pre-call content <NUM> identifying a message from "Tim" and the user <NUM> providing the second input <NUM> that identifies "Tim. " Additionally, or alternatively, a correlation that can be identified can include the other user <NUM> providing the first input <NUM> that identifies "the event," and the first pre-call content <NUM> that identifies the "reunion event. " In some implementations, one or more correlations can be identified by comparing data that is based on the dialog content, to data that is based on first pre-call content <NUM>, and also to other data that is based on the second pre-call content <NUM>. In some implementations, one or more correlations can be identified by applying, to one or more trained machine learning models, data characterizing the dialog content and other data characterizing the first pre-call content <NUM>. Also, one or more correlations can be identified by applying, to one or more trained machine learning models, data characterizing the dialog content and other data characterizing the second pre-call content <NUM>.

Other pre-call content <NUM> can be provided to the user <NUM> and/or the other user <NUM> via their respective computing devices in response to the user <NUM> and/or the other user <NUM> initializing a call. For example, as provided in <FIG>, when the computing device <NUM> is awaiting acceptance of the incoming call from the other user <NUM>, the display panel <NUM> can be caused to provide another notification <NUM> and other pre-call content <NUM>. The notification <NUM> can be an optional notification that the user <NUM> is receiving a request to participate in a call with the other user <NUM> (e.g., "Charlie"). The other pre-call content <NUM> can be provided at the display panel <NUM> and can include first pre-call content <NUM> and second pre-call content <NUM>.

The first pre-call content <NUM> and the second pre-call content <NUM> can be provided at the display panel <NUM> based on whether one or more correlations were identified between the dialog content and the first pre-call content <NUM>, and the dialog content and the second pre-call content <NUM>. Furthermore, the arrangement of the first pre-call content <NUM> and the second pre-call content <NUM> at the display panel <NUM> can be based on whether the one or more correlations were identified. For example, the "message" type of pre-call content can be assigned a higher priority than the "calendar" type of pre-call content when a correlation between the dialog content and the second pre-call content <NUM> is identified, but no correlation between the dialog content and the first pre-call content <NUM> is identified. Additionally, or alternatively, "calendar" type pre-call content can be assigned a lower priority relative to one or more other types of pre-call content when no correlation between the "calendar" type pre-call content and the dialog content is identified. As a result, a priority of "calendar" type pre-call content can have a lower priority relative to "social media" type pre-call content, as illustrated in <FIG>. As a result, the display panel <NUM> can present "message" type pre-call content and "social media" type pre-call content when the computing device <NUM> is awaiting acceptance <NUM> of the subsequent incoming call from the other user <NUM>.

For example, when the computing device <NUM> provides the notification <NUM> of the incoming call, the user <NUM> can be presented with the first pre-call content <NUM>, which can provide content corresponding to various activities of one or more messaging applications. For instance, content from two different messaging applications can be consolidated to provide the first pre-call content <NUM>, which can include a portion of a latest message from "Charlie," the other user <NUM> who initialized the call, and "Tim," someone that the user <NUM> mentioned in the dialog content. The user <NUM> can influence subsequent presentations of pre-call content by engaging with the other pre-call content <NUM> (e.g., thereby causing one or more trained machine learning models to reflect the positive engagement by the user), and/or by not engaging with the other pre-call content <NUM> (e.g., thereby causing one or more trained machine learning models to reflect the negative engagement by the user). Additionally, or alternatively, positive engagement can be indicated by the user and/or the other user <NUM>, providing one or more gestures before, during, and/or after the other pre-call content <NUM> is be presented to the user <NUM>.

For example, after acceptance <NUM> of the subsequent call, the other user <NUM> can provide a first input <NUM> such as "Hey, what's been going on?" to which the user <NUM> can respond with a second input <NUM>, "Tim is wanting to get together. Did you see Carol's photos?" Dialog content capturing the first input <NUM> and other dialog content capturing the second input <NUM> can be compared to the other pre-call content <NUM> to determine whether one or more correlations are apparent. Because the user <NUM> mentioned "Carol's photos" and a message from "Tim," each of the first pre-call content <NUM> and the second pre-call content <NUM> can have positive correlations to the dialog content. As a result, the content, type of content, and/or other data related to the other pre-call content <NUM> can be treated as encouraging positive engagement, thereby presumably assisting the user <NUM> before accepting a call. This can allow for more efficient calls between users, as users would no longer need to access separate applications to be reminded of certain information (e.g., recent calendar events and/or messages). This can also help to ensure that the information is available at a time of need, particularly where communication signals are intermittent. Additionally, because the information may be retrieved from one or more locations in secondary storage (for example, flash memory) and loaded into primary memory (e.g., volatile RAM) in advance of the user requesting that information, retrieval of that information occurs more quickly from perspective of the user. This can also result in a decrease in usage of computational resources that might otherwise be consumed while the user is on the phone, and attempting to remember the information, thereby prolonging the call longer than necessary. When pre-call content is provided before video calls, computational resources can be even more preserved, relative to audio calls, as many video calls require the exchange of high definition video over the period of the video call.

<FIG>, <FIG>, and <FIG> illustrate a user changing contexts and causing pre-call content to be generated according to the changes in context. For example, <FIG> illustrates a view <NUM> of a user <NUM> riding in a vehicle <NUM> with another user <NUM>. While riding in the vehicle, the user <NUM> can be engaging in one or more activities with a computing device <NUM>, which can include a location sensor from which a location of the computing device <NUM> can be determined. Additionally, the computing device <NUM> can include, or otherwise provide access to, an automated assistant <NUM>. The user <NUM> can engage with the automated assistant <NUM> via one or more gestures and/or one or more spoken utterances. For example, while riding in the vehicle <NUM>, the user <NUM> can provide a spoken utterance <NUM> such as, "Assistant, tell me the recipe for pecan pie. " In response, the automated assistant <NUM> can access client data <NUM> and/or one or more applications <NUM> in order to fulfill the request from the user <NUM>. In order to fulfill the request, the automated assistant <NUM> can access a browser application of the applications <NUM> and download recipe data, which can be stored as part of the client data <NUM>.

The computing device <NUM> can be in communication with a remote computing device <NUM>, such as computing device <NUM>, which can be a server device. The computing device <NUM>, with prior permission from the user <NUM>, can generate and/or otherwise access contextual data <NUM>, which can characterize activities of the user <NUM> over time. The contextual data <NUM> characterizing the activity of the other user <NUM> can be used to generate pre-call content <NUM>. The pre-call content <NUM> can be generated at any time, regardless of whether there is an incoming call coming from another user or not. For example, in response to the user <NUM> invoking the automated assistant <NUM> to download the recipe, pre-call content <NUM> can be generated in order to characterize the interaction with the automated assistant <NUM>, should the user <NUM> receive an incoming call that is associated with the interaction. For example, the user <NUM> may have a Derby event to attend on their calendar, and the Derby event may describe "pecan pie" as being served at the Derby event. Therefore, the pre-call content <NUM> can be provided to the user <NUM> in response to a Derby event invitee (e.g., "Virginia") calling the user <NUM>. In other words, when the computing device <NUM> is waiting for the user <NUM> to accept the call, pre-call content <NUM> can be provided at a display panel at the computing device <NUM>, in order to refresh a recollection of the user <NUM>, especially with respect to the pecan pie recipe and/or the Derby event. However, should the user <NUM> perform other activities that are temporarily separate from the activities in the vehicle <NUM>, those subsequent activities can be used as a basis for generating other pre-call content <NUM>.

For example, <FIG> illustrates a view <NUM> of the user <NUM> engaging in activities within their kitchen <NUM>, subsequent to the user <NUM> riding in the vehicle <NUM>. The user <NUM> can be engaged in activities such as baking a pecan pie using a first connected device <NUM>, washing dishes via a second connected device <NUM>, and playing a podcast <NUM> via a client device <NUM>, such as a standalone speaker device <NUM>. The user <NUM> can engage in each activity using one or more computing devices, and/or an automated assistant to interact with various devices.

The subsequent activities engaged in by the user <NUM> can also be characterized by contextual data <NUM>, and used as a basis from which to generate additional pre-call content <NUM>. For example, pre-call content <NUM> generated based on the activities depicted in <FIG> can include "operational" type data. In this way, should the user <NUM> receive an incoming call from the Derby event invitee (e.g., "Virginia"), the client device <NUM> can audibly render pre-call content <NUM> in order to refresh the recollection of the user <NUM> in furtherance of the user having a more efficient call related to the Derby event. For example, the client device <NUM> can render audio characterizing an operational status of the first connected device <NUM>, an operational status of the second connected device <NUM>, and a current operating status of the client device <NUM>, while also providing an audible sound indicating that an incoming phone call is awaiting acceptance by the user <NUM>. As an example, the client device <NUM> can render a ringing sound to indicate an incoming call is being received, while also rendering audio data such as, "The oven timer has <NUM> minutes left," since the user <NUM> previously requested a recipe that was related to the Derby event, and a Derby event invitee is calling the user <NUM>. However, if no call is received while the user is engaging in the activities, pre-call content <NUM> can be otherwise stored and available to be presented at another time in the future. It will be appreciated that the above operations may be executed in response to an incoming call being received at a different client device to the client device <NUM>. For example, the client device <NUM> may perform the above described operations in response to an incoming call being received at a smartphone device,. That is, pre-call content may be provided on a device other than the device at which the incoming call is received.

<FIG> depicts the user in a context, subsequent to those characterized in <FIG> and <FIG>. However, pre-call content generated based on the activities of the user <NUM> within each context can be stored and presented within the context depicted in <FIG>. For example, as provided in view <NUM> of <FIG>, first pre-call content can be generated based on data that resulted from the user <NUM> engaging in the activity depicted in <FIG>, in which the user <NUM> initialized their oven timer <NUM>. The timer could have been initialized the user <NUM> via a request to a client automated assistant <NUM> via an assistant interface <NUM> of the client device <NUM>. Therefore, when generating first pre-call content <NUM>, automated assistant data and/or device data can be accessed in order to provide relevant pre-call content such as, "Oven timer has <NUM> minutes left. " In some implementations, an incoming call can affect a current operating status of a device, such as the client device <NUM>, which can pause an audio output (e.g., podcast audio) in response to an incoming call from another user <NUM>. Therefore, any pre-call content that is generated based on operational data characterizing the client device <NUM> can be updated in response to the incoming call, and reflected at a display panel <NUM> of a computing device <NUM>. For instance, as provided in view <NUM>, pre-call content <NUM> provided to the user <NUM>, while the computing device <NUM> is awaiting acceptance of the call, can include first pre-call content <NUM> including "History podcast is paused.

Furthermore, in response to the other user <NUM> (e.g., a Derby event invitee) initializing a call to the user <NUM> via computing device <NUM>, second pre-call content <NUM> can be provided at the display panel <NUM> based on activities of the user <NUM> described with respect to the <FIG>. In some implementations, a synopsis of activities and/or content viewed by the user <NUM> can be presented as part of the pre-call content. For instance, because the user <NUM> was viewing a "recipe" via a web browser application of the computing device <NUM>, content of the recipe web page can be applied to a machine learning model for generating data from which a synopsis of the web page can be generated. Therefore, as a result, the pre-call content <NUM> can include second pre-call content <NUM> such as "Pre-Call Browsing Content" that provides a synopsis of a recent web page (e.g., "pecan pie Recipe") viewed by the user <NUM>. Furthermore, other browsing content can also be characterized by the second pre-call content <NUM>, such as "Directions to Churchill Downs," if the user <NUM> had recently requested that the automated assistant provide such directions.

Each of the first pre-call content <NUM> and the second pre-call content <NUM> can correspond to different sources and/or types of pre-call content. In this way, an amount of engagement with each type and/or source of content can be determined, in order that one or more trained machine learning models can be trained based on such engagement. Furthermore, the user <NUM> can engage with the pre-call content <NUM> in order to determine whether to accept the call from the other user <NUM> or not accept the call. Should the user <NUM> elect to indicate acceptance <NUM> of the call, the pre-call content <NUM> can enable the user <NUM> to have a more effective call, compared to if the user had not seen the pre-call content <NUM>. For example, the other user <NUM> can be the Derby event invitee, and can provide a first input <NUM> to the call, such as "Hey, how's the pie coming along?" and in response, the user <NUM> can provide a second input <NUM> such as, "Good. Got the recipe earlier, and oven has <NUM> minutes left. Listening to a history podcast while I wait!" By providing the pre-call content <NUM>, based on the association of the caller to the user's <NUM> calendar event, the user <NUM> is able to be more readily responsive during the call, thereby reducing the amount of the time the user <NUM> will be on the call, as well as the other user <NUM> being on their respective computing device <NUM>.

<FIG> illustrates a system <NUM> for providing pre-call content to one or more users that are expected to receive an incoming call. The pre-call content can be provided in order to eliminate the need for other application data to be accessed during a call in order to eliminate waste of energy and/or computational resources during a call. In some implementations, the pre-call content can be provided via an automated assistant <NUM> that is accessible to a client device, and/or via any other application or module that is access to the client device. The automated assistant <NUM> can operate as part of an automated assistant application that is provided at one or more computing devices <NUM>. A user can interact with the automated assistant <NUM> via one or more assistant interfaces <NUM>, which can include one or more of a microphone, a camera, a touch screen display, a user interface, and/or any other apparatus capable of providing an interface between a user and an application. For instance, a user can initialize the automated assistant <NUM> by providing a verbal, textual, and/or a graphical input to the assistant interface to cause the automated assistant <NUM> to perform a function (e.g., provide data, control a device (e.g., control an IoT device, access an agent, modify a setting, control an application(s), etc.). The computing device <NUM> can include a display device, which can be a display panel that includes a touch interface for receiving touch inputs and/or gestures for allowing a user to control applications of the computing device <NUM> via the touch interface. The touch inputs and/or other gestures (e.g., spoken utterances) can also allow a user to interact with the automated assistant <NUM>.

In some implementations, the computing device <NUM> can lack a display device but include an audio interface (e.g., a speaker and/or a microphone), thereby providing an audible user interface output, without providing a graphical user interface output, as well as providing a user interface, such as a microphone, for receiving spoken natural language inputs from a user. For instance, in some implementations, the computing device can include one or more tactile input interfaces, such as one or more buttons, and omit a display panel that would be provided graphical data from a graphics processing unit (GPU). In this way, energy and processing resources can be saved compared to a computing device that includes a display panel and GPU.

The computing device <NUM> can be in communication with a server device over a network, such as the internet. The computing device <NUM> can offload computational tasks to the server device in order to preserve computational resources at the computing device <NUM>. For instance, the server device can host an automated assistant, and the computing device <NUM> can transmit inputs received at one or more assistant interfaces, and/or a user interface, to the server device. However, in some implementations, the automated assistant <NUM> can be hosted at the computing device <NUM>. In various implementations, all or less than all aspects of the automated assistant <NUM> can be implemented on the server device and/or the computing device <NUM>. In some of those implementations, aspects of the automated assistant <NUM> are implemented via a local automated assistant and interface with the server device, which can implement other aspects of the automated assistant <NUM>. The server device can optionally serve a plurality of users and their associated assistant applications via multiple threads. In implementations where all or less than all aspects of the automated assistant <NUM> are implemented via a local automated assistant of the computing device <NUM>, the local automated assistant can be an application that is separate from an operating system of the computing device <NUM> (e.g., installed "on top" of the operating system) - or can alternatively be implemented directly by the operating system of the computing device <NUM> (e.g., considered an application of, but integral with, the operating system).

In some implementations, the automated assistant <NUM> can include an input processing engine <NUM>, which can employ multiple different engines for processing inputs and/or outputs for computing device <NUM>. For instance, the input processing engine <NUM> can include a speech processing engine <NUM> that can process audio data received at an assistant interface <NUM> to identify the text embodied in the audio data. The audio data can be transmitted from, for example, the computing device <NUM> to the server device in order to preserve computational resources at the computing device <NUM>.

The process for converting the audio data to text can include a speech recognition algorithm, which can employ neural networks and/or statistical models for identifying groups of audio data corresponding to words or phrases. The text converted from the audio data can be parsed by a data parsing engine <NUM> and made available to the automated assistant <NUM> as textual data that can be used to generate and/or identify command phrases from the user. In some implementations, output data provided by the data parsing engine <NUM> can be provided to an action engine <NUM> to determine whether the user has provided an input that corresponds to a particular action and/or routine capable of being performed by the automated assistant <NUM> and/or an application, agent, and/or device that is capable of being accessed by the automated assistant <NUM>. For example, assistant data <NUM> can be stored at the server device and/or the computing device <NUM>, and can include data that defines one or more actions capable of being performed by the automated assistant <NUM>, as well as parameters involved in performing those actions.

When the input processing engine <NUM> has determined that a user has requested a particular action or routine be performed, the action engine <NUM> can determine one or more parameter values for the particular action or routine, and an output generating engine <NUM> can then provide an output to the user based on the particular action, routine, and/or the one or more parameters. For instance, in some implementations, in response to a user input, such as a gesture directed at the assistant interface <NUM>, the automated assistant <NUM> can cause data, which characterizes the gesture, to be transmitted to the server device for determining the action that the user is intending the automated assistant <NUM> to perform.

The system <NUM> can further include a pre-call content engine <NUM> that generates pre-call content for one or more users in anticipation of the one or more users receiving a call from one or more other users. The pre-call content engine <NUM> can access client data <NUM>, assistant data <NUM>, and/or application data <NUM> at the computing device <NUM> and/or any other device for generating the pre-call content. For instance, the computing device <NUM> can include various applications with which a user can engage in a variety of different activities, such as, but not limited to, sending a message to another user and sharing media (e.g., photos and/or videos) with another user. When a user performs a particular activity, data associated with that activity can be processed, with permission from the user, by a contextual engine <NUM>. The contextual engine <NUM> can process the data in order to identify correlations between the data and one or more other users, which the user can receive an incoming call from. When the data, which is generated based on a context or particular activity, is identified as being associated with another user, the contextual engine <NUM> can interact with the pre-call content engine <NUM> in order to cause the pre-call content engine <NUM> to generate pre-call content based on the identified data. For example, if the activity includes the user sending a message and a video to another user, the message and video can be used as a basis from which to generate pre-call content.

In some implementations, multiple different instances of pre-call content can be generated, and each instance of pre-call content can be assigned a priority by a priority engine <NUM>. The assigned priority for a particular instance of pre-call content can indicate a predicted propriety of the pre-call content to a call that may occur between a user and another user. In other words, the assigned priority for the particular instance can indicate a prediction of how appropriate the pre-call content is to the call. For instance, when an instance of pre-call content is generated by the priority engine <NUM>, based on a most recent message between a user and another user, the instance of pre-call content can be assigned a highest priority relative to other instances of pre-call content that correspond to the user and the other user. Such assignment of the highest priority can be based at least in part on the most recent message being the most recent in time message between the user and the other user. In this way, should the other user call the user, the highest priority pre-call content can be presented to the user while the call is pending acceptance by the user. Should the user subsequently engage in another activity associated with the other user, and that other activity be different than sending a message (e.g., creating a calendar event that identifies the user and the other user), additional pre-call content can be generated by the pre-call content engine <NUM> based on the other activity.

The additional pre-call content can be assigned a priority by the priority engine <NUM> based on historical determinations of the propriety of the other activity to calls between the user and the other user. For instance, dialog content data and pre-call content data can be applied to one or more trained machine learning model <NUM> (e.g., trained based on the historical determinations) in order to determine a measure that indicates predicted propriety of the pre-call content to the dialog content. The measure can be used as a basis for adjusting an assigned priority of any other instances of pre-call content that correspond to the same activity as an activity corresponding to the pre-call content data. Furthermore, multiple instances of pre-call content data can be stored on the computing device <NUM>, and one or more instances of the pre-call content data can correspond to different other users that may call the user, or have called the user in the past. For instance, pre-call content data can be generated in response to a particular user calling the user for the first time, and in expectation that the particular user will call the user again.

In some implementations, the contextual engine <NUM> can determine a current context of a user and cause the computing device <NUM> to render pre-call content according to the current context. For instance, when the client data <NUM> indicates that the user is controlling the computing device <NUM> or another computing device from a headset, the contextual engine <NUM> can communicate with the pre-call content engine <NUM> in order to convert any suitable pre-call content to audio format. In this way, should the user receive an incoming call, the pre-call content can be rendered audibly while the computing device <NUM> is awaiting acceptance of the call by the user. In some implementations, the client data <NUM> can indicate that the user is driving a vehicle, and therefore may be difficult to look at a screen. Therefore, based on this indication, the contextual engine <NUM> can communicate with the pre-call content engine <NUM> in order to convert any suitable pre-call content to audio format, should the user receive a call while operating the vehicle.

In some implementations, an interaction engine <NUM> of the system <NUM> can generate data that characterizes an extent to which a user has interacted with pre-call content, and that data can be shared with the priority engine <NUM>. The priority engine <NUM> can then re-assign priorities of particular instances of pre-call content according to whether the user engaged with certain pre-call content or not. For instance, a user that receives a call from their best friend or spouse may never rely on pre-call content, and therefore never be perceived to interact with the pre-call content. As a result, the interaction engine <NUM> can share certain data with the priority engine <NUM> indicating that pre-call content for the best friend and/or the spouse can have a low priority or otherwise be removed. In some implementations, when pre-call content generated for particular anticipated callers is assigned a lower priority relative to other anticipated callers, and/or for a threshold period of time, pre-call content for those particular anticipated callers can be bypassed. In this way, the user will not be shown pre-call content for those anticipated callers from which the user has a history of not relying on such pre-call content.

In some implementations, the contextual engine <NUM> can determine a context of the user in order to promote privacy of the user when providing pre-call content. For instance, the contextual engine <NUM> can determine, with prior permission from the user, that the user is away from their home and/or in a public location. Furthermore, the interaction engine <NUM> can determine, based on information from the contextual engine <NUM>, whether the user typically engages with pre-call content when away from their home and/or in public places. When the user has a history of not engaging with pre-call content while away from their home and/or in public places, regardless of the caller, the interaction engine <NUM> and/or contextual engine <NUM> can communicate this preference of the user to the priority engine <NUM>. In response, the priority engine <NUM> can re-assign priorities of certain pre-call content when the user is determined to be away from their home and/or in a public place. Additionally, or alternatively, the priority engine <NUM> can assign a tag to certain pre-call content indicating that the pre-call content should not be shown until the tag is withdrawn, or should not be shown while the user is away from their home and/or in a public place.

In some implementations, various gestures performed by the user when pre-call content is being presented can be identified by the interaction engine <NUM>. The gestures can be captured at one or more assistant interfaces <NUM>, and can include voice input, motion gestures, touch gestures, two-dimensional gestures, three-dimensional gestures, and/or any other gestures that can be identified by a computing device. Furthermore, the interaction engine <NUM> can use gesture data provided by the assistant interface <NUM> to distinguish an amount of engagement a user has for certain pre-call content over other pre-call content. For instance, a gesture can cause a display interface of the computing device <NUM> to scroll away from first pre-call content in order to make second pre-call content appear for a longer period of time.

<FIG> illustrates a method <NUM> for providing pre-call content to a user according to one or more different approaches. The method <NUM> can be performed by one or more computing devices, applications, and/or any other apparatus and/or module capable of generating content for a user. The method <NUM> can include an operation <NUM> of determining whether an incoming call is being received by a user from another user. For example, the user can have a portable computing device, such as a cell phone, capable of communicating via a wireless communication protocol with a separate portable computing device associated with the other user. When the other user initializes a phone call to the user, one or more network connections can be established for providing a communication channel between the portable computing device and the other portable computing device. Data generated in response to the other user initializing the phone call can be used as a basis for much to determine whether an incoming call to the user is being received from the other user. When, at operation <NUM>, a determination is made that the user is receiving a call from another user, the method <NUM> can proceed to an operation <NUM>.

The operation <NUM> can include a determination of whether the user has accepted the call from the other user. In some implementations, the user can indicate acceptance of the call from the other user by providing a user input to the portable computing device, and/or any other computing device that is associated with the portable computing device. For example, the user can indicate acceptance of the call by tapping a graphical element presented at a touch display panel have the portable computing device. Alternatively, the user can indicate acceptance of the call by providing a user input to a computing device that is separate from the portable computing device, such as a stand-alone speaker device, which can receive spoken utterances for indicating a willingness to accept phone calls.

When the user has not accepted the phone call, the method <NUM> can proceed to operation <NUM> of presenting pre-call content at an interface of the computing device (e.g., the portable computing device). The pre-call content can be generated based on data that is associated with the user that is receiving the call and/or the other user that has initialized the call. Alternatively, or additionally, when the user is the one who initialized the call, the pre-content can be presented at the interface of the computing device while the user is waiting for the other user to indicate acceptance of the initialized call. The pre-call content can be generated for indicating various different subject matter. For instance, the pre-call content can be generated for indicating one or more predicted subjects that will be discussed during the phone call. Alternatively, or additionally, the pre-call content can be generated for attempting to refresh a recollection of the user and/or the other user prior to the users participating in the phone call. Alternatively, or additionally, the pre-call content can be generated in order to provide guidance to the user regarding whether to even accept the call or not, for instance, based on security concerns, past user experiences with calls from the other user, a current context of the user and/or the other user, and/or any other information from which a user can determine whether to accept an incoming call.

The method <NUM> can proceed from the operation <NUM> to the operation <NUM>, which can include determining user engagement with the pre-call content. In some implementations, the amount of engagement with pre-call content can be based on one or more user inputs provided to one or more computing devices that provided and/or are otherwise associated with the pre-call content. In some implementations, the amount of engagement with pre-call content can be determined based on other user activities that can be associated with the pre-call content. In some implementations, user engagement with pre-call content can be determined based on an amount of time that the user caused first pre-call content to be presented at an interface of the computing device, and a separate amount of time that the user caused second pre-call content to be presented at the interface of the computing device. In the aforementioned example, when the amount of time that the first pre-call content was presented is greater than the separate amount of time that the second pre-call content to be presented, the first pre-call content can be designated as more relevant to calls between the user and the other user than the second pre-call content. In some implementations, relevance of pre-call content to certain calls can be provided as positive feedback and/or negative feedback to a trained machine learning model that can be used as a basis for creating output from which call content can be based.

The method <NUM> can proceed from the operation <NUM> to the operation <NUM>, which can include determining whether the call has been terminated. For example, if the user is determined to have not accepted the call at operation <NUM>, and subsequently the user or the other user have terminated the call, the method <NUM> can proceed from the operation <NUM> back to the operation <NUM>. In other words, the user would not have accepted the call, and therefore the method <NUM> can return to determining whether there is an incoming call. In some implementations, if the user is determined to have engaged with the pre-call content prior to terminating the call, the pre-call content, and/or a source of the pre-call content can be determined as relevant for the user When making decisions about whether to accept or terminate incoming calls from the other user. However, if the user is determined to have not engaged with the pre-call content prior to terminating the call, the pre-call content, and/or the source of the pre-call content, can be determined to be irrelevant to the user when the user is making decisions about whether to accept or terminate incoming calls from the other user. In some implementations, each pre-call content, type of pre-call content, and/or source of pre-call content can be assigned a priority, which can be dynamically modified according to whether the user engages with such pre-call content. Therefore, determinations of engagement can influence whether the user will be presented with certain pre-call content, types of pre-call content, and/or content from certain sources of pre-call content. For example, if a user engages with pre-call content that is based on a source such as a calendar application, that source can be assigned a higher priority than another source, such as a GPS source of location data.

When the user has not terminated the call at operation <NUM>, the method <NUM> can continue at operation <NUM> where the determination is made as to whether the user has accepted the incoming call. When the user is determined to have accepted the call at operation <NUM>, the method <NUM> can optionally proceed to the operation <NUM>. This optional path can result in the user accepting the call, and subsequently awaiting the call to be discontinued. For example, if the call has been undertaken for a few minutes, the call can then be terminated at the operation <NUM>, and the method <NUM> can proceed to the operation <NUM>. When the method <NUM> proceeds in this way, certain pre-call content, types of pre-call content, and/or sources of pre-call content can be reprioritized according to user engagement with the pre-call content prior to accepting the call. In this way, the next time the user receives an incoming call from the other user, the relevancy of the pre-call content would be optimized compared to the previous pre-call content presented to the user.

In some implementations, the method <NUM> can proceed from the operation <NUM> to the operation <NUM>, after the user is determined to have accepted the call at the operation <NUM>. The operation <NUM> can include determining, with user permission, dialog content of the first call between the user and the other user. For example, audio data corresponding to spoken inputs by the user can be captured and stored as dialog content of the call between the user and the other user. Alternatively, or additionally, audio data corresponding to spoken inputs from both the user and the other user can be captured and stored as dialog content of the call between the user and the other user. In some implementations, the audio data can be transcribed and further processed as contextual data corresponding to call between the user and the other user.

The method <NUM> can proceed from the operation <NUM> to the operation <NUM>, which can include determining whether any of the dialogue content is related to the pre-call content presented to the user and/or the other user. For example, the audio data capturing the dialogue content can be compared to the pre-call content to determine whether one or more correlations are apparent. In some implementations, textual data transcribed from the audio data, with permission from the user, can be compared to the pre-call content in order to determine whether one or more correlations are apparent. As an example, the pre-call content can include a synopsis of a calendar event stored by a calendar application, and the calendar event can identify the user and the other user. When the dialogue content includes at least some amount of data characterizing at least some portion of the synopsis, the dialog content can be considered correlated to the pre-call content.

If one or more correlations are identified between the dialogue contact and the pre-call content, the method <NUM> can proceed to an operation <NUM> of providing positive feedback to one or more trained machine learning models. In this way, one or more machine learning models can be adapted to provide more relevant content prior to calls between the user and the other user. Furthermore, the method <NUM> proceed from the operation <NUM> to the operation <NUM>. The operation <NUM> include generating pre-call content based on one or more trained machine learning models. Pre-call content can be generated before or after first call has been terminated, but before a second call has been initialized. In this way, latency between a call being initialized and the pre-call content being presented can be eliminated.

In some implementations, the method <NUM> can proceed from the operation <NUM> to the operation <NUM>, when one or more correlations between the dialogue content and the pre-call content are not identified. For example, if the pre-call content included text from one or more correspondence between the user and the other user, and the dialog content does not mention any of the text, or anything synonymous with the text, no correlation would be identified. As a result, the method <NUM> can proceed to the operation <NUM> of providing negative feedback to one or more trained machine learning models. By providing negative feedback data to the one or more trained machine learning models, the one or more trained machine learning models can be adapted to provide more relevant data from which pre-call content can be generated. For instance, if a source from which pre-call content was generated results in no correlation being identified, the one or more trained machine learning models can be adapted to rely on that source data less often. In some implementations, the one or more trained machine learning models can receive as input, identifiers for participants on an incoming call, and available sources for pre-call content. Thereafter, the source that provides content that was most relevant can be assigned a higher priority than another source of content that was not relevant. In some implementation, priorities can correspond to one or more values for adjusting and/or otherwise training one or more trained machine learning models. For example, a node of a machine learning model associated with a particular type of pre-call content data source can have an adjustable priority value, which can influence a final arrangement of pre-call content that is generated for particular calls.

The method <NUM> can proceed from the operation <NUM> to the operation <NUM>, in which pre-call content is generated based on the one or more machine learning models. In some implementations, one or more activities that a user engages in prior to the next incoming call and/or outgoing call can be used as a basis from which to further train the one or more machine learning models. Alternatively, or additionally, one or more activities that the user engages in prior to the next incoming call and/or outgoing call can be used as a basis from which to generate pre-call content to be presented when the user is being notified of the next incoming call.

Storage subsystem <NUM> stores programming and data constructs that provide the functionality of some or all of the modules described herein. For example, the storage subsystem <NUM> may include the logic to perform selected aspects of method <NUM>, and/or to implement one or more of computing device <NUM>, computing device <NUM>, computing device <NUM>, computing device <NUM>, remote computing device <NUM>, client device <NUM>, computing device <NUM>, computing device <NUM>, computing device <NUM>, and/or any other device, operation, application, and/or engine discussed herein.

In some implementations, a method implemented by one or more processors is provided and includes determining that a first call between a user and another user has been initialized via a wireless communication protocol between a computing device and another computing device. The method further includes, when the computing device is waiting for the user and/or the other user to indicate acceptance of the first call, causing pre-call content to be presented at an interface of the computing device. The pre-call content is optionally determined based on data that is generated using one or more trained machine learning models. The method further includes determining, subsequent to determining that the first call was initialized between the user and the other user, dialog content of the first call. The dialog content is based on one or more natural language inputs provided by the user and/or the other user during the first call. The method further includes determining, based on the dialog content, whether the pre-call content included at least one correlation to the dialog content. The method further includes determining, subsequent to determining whether the pre-call content included at least one correlation to the dialog content, that a second call between the user and the other user has been initialized. The method further includes, when the computing device and/or the other computing device is waiting for the user and/or the other user to indicate acceptance of the second call: causing the interface of the computing device, and/or another interface of another computing device, to present or omit the pre-call content based on whether the pre-call content included the at least one correlation to the dialog content.

In some implementations, the method further includes, when the pre-call content is determined to include at least the one correlation to the dialog content: causing the one or more trained machine learning models to be further trained according to feedback data indicating a positive correlation between dialog content and the pre-call content; and when the pre-call content is determined to not include at least one correlation to the dialog content: causing the one or more trained machine learning models to be further trained according to other feedback data indicating a negative correlation between dialog content and the pre-call content.

In some implementations, the one or more trained machine learning models are stored on the computing device.

In some implementations, the method further includes, when the computing device and/or the other computing device is waiting for the user and/or the other user to indicate acceptance of the second call, and the pre-call content is presented at the interface of the computing device, and/or the other interface of the other computing device: causing the pre-call content at the interface of the computing device, and/or the other interface of the other computing device, to be arranged based on whether the pre-call content included the at least one correlation to the dialog content.

In some implementations, the pre-call content is generated in advance of the first call being initialized. In some versions of those implementations, the pre-call content is generated locally at the client device. In some additional or alternative versions of those implementations, the pre-call content is most recently generated pre-call content, generated in a most recent iteration of generating pre-call content relevant to the user and the other user. In some of those additional or alternative versions, the most recent iteration is performed at a time that is based on a measure of frequency and/or quantity of communications between the user and the other user and/or that is based on new content relevant to the user and/or the other user being received. By generating the content in advance of the first call being initialized, latency in presenting the first content is reduced, as is latency in accepting the first call. Moreover, when new pre-call content is iteratively generated for a given user based on frequency of communication with the given user and/or based on new content from and/or related to the user being received, new pre-call content is generated dynamically in a manner that is tailored to enable pre-call content to be fresh, but to conserve resources so that new pre-call content is not continuously generated for all users. For example, new pre-call content can be generated at a first higher frequency for a given user that a user typically communicates with daily, whereas pre-call content can be generated at a second lower frequency for an additional given user that the user typically communicates with only once a week.

In some implementations, a method implemented by one or more processors is provided and includes determining content for provisioning at an interface of a computing device when a user is receiving an incoming call from another user. The content includes first content data, which corresponds to a first assigned priority, and second content data, which corresponds to a second assigned priority. The first assigned priority and the second assigned priority indicate propriety of the content data, at least relative to the incoming call. The method further includes determining, subsequent to determining the content for provisioning at the interface of the computing device and in response to a wireless transmission initialized by the other user via another computing device, that the other user has initialized a call to the user via a wireless communication protocol. The method further includes, when the computing device is waiting for the user to indicate acceptance of the call: causing, in response to determining that the other user has initialized the call, the interface of the computing device to present the first content data and the second content data, where the first content data is prioritized over the second content data at least based on the first assigned priority and the second assigned priority. The method further includes determining, based on the provisioning of the first content data, engagement data that characterizes an amount of engagement exhibited by the user when the interface of the computing device provided the first content data, and another amount of engagement exhibited by the user when the interface of the computing device provided the second content data. The method further includes determining, based on the engagement data, a third assigned priority for the first content data and a fourth assigned priority for the second content data. The third assigned priority and the fourth assigned priority indicate a different propriety of the content data, at least relative to a subsequent incoming call from the other user. The method further includes determining, subsequent to determining the third assigned priority and the fourth assigned priority, that the other user has initialized the subsequent incoming call to the user. The method further includes, when the computing device is waiting for the user to indicate acceptance of the subsequent call: causing, in response to determining that the other user has initialized the subsequent incoming call to the user, the interface of the computing device to present or omit the first content data and/or the second content data, at least based on the third assigned priority and the fourth assigned priority.

In some implementations, the engagement data is based on one or more gestures received at the interface of the computing device when the interface of the computing device is providing the first content data and the second content data.

In some implementations, the amount of engagement exhibited by the user when the interface of the computing device provided the first content data is based on a first gesture of the one or more gestures, and the other amount of engagement exhibited by the user when the interface of the computing device provided the second content data is based on a second gesture of the one or more gestures. In some versions of those implementations, the method further includes determining that the first gesture caused the first content data to be provided at the interface of the computing device for a longer period of time than the second gesture caused the second content data to be provided at the interface of the computing device. In those versions, the third assigned priority is prioritized over the fourth assigned priority at least based on the first content of data being provided at the interface of the computing device for a longer period of time than the second content data.

In some implementations, the method further includes accessing, before determining the content for provisioning to the user, data that at least characterizes one or more features of a context of a user. In some of those implementations, the content is determined based on the one or more features of the context of the user.

In some implementations, determining the third assigned priority for the first content data and the fourth assigned priority for the second content data includes: providing, to one or more trained machine learning models, input data that is based on the first content data and the second content data; and generating, based on output data generated by processing the input data using the one or more trained machine learning models, the third assigned priority and the fourth assigned priority. In some of those implementations, the method further includes causing the one or more trained machine learning models to be further trained based on the engagement data and the other engagement data.

In some implementations, a method implemented by one or more processors is provided and includes generating first pre-call content for provisioning at an interface of a computing device when a user is receiving, via the computing device, an incoming call from another user. The first pre-call content is based on data that resulted from the user engaging in an activity that is associated with the other user. The method further includes determining, subsequent to generating the first content data, that the user has engaged in another activity, where the other activity is also associated with the other user and resulted in other data being generated. The method further includes generating, based at least on the other data that resulted from the user engaging in the activity, second pre-call content in furtherance of provisioning content at the interface of the computing device when the user is receiving, via the computing device, the incoming call from the other user. The method further includes determining, subsequent to generating the second pre-call content, that the other user has initialized a call to the user via a wireless communication protocol. The method further includes, when the computing device is waiting for the user to indicate acceptance of the call: causing, in response to determining that the other user has initialized the call to the other, the computing device to present the first pre-call content and/or the second pre-call content at the interface of the computing device.

In some implementations, generating the first pre-call content includes: providing, based on the data, an input to a trained machine learning model, where the first pre-call content includes a synopsis of the data and the synopsis is generated based on an output generated based on processing the input using the trained machine learning model. In some of those implementations, generating the second pre-call content includes: processing the other data using the trained machine learning model to generate another output based on the other data, where the second pre-call content includes another synopsis of the other data and is generated based on the other output of the trained machine learning model.

In some implementations, the first pre-call content corresponds to a first assigned priority and the second pre-call content corresponds to a second assigned priority, and the activity and the other activity are temporally distinct activities.

In some implementations, the method further includes, when the computing device is waiting for the user to indicate acceptance of the call: determining, based on the first assigned priority and the second assigned priority, whether to cause the interface of the computing device to prioritize a position of the first pre-call content over another position of the second pre-call content.

In some implementations, the interface of the computing device includes an audio interface and causing the computing device to present the first pre-call content and/or the second pre-call content at the interface of the computing device includes: causing the interface to render audio corresponding to the first pre-call content and/or the second pre-call content when the computing device is waiting for the user to indicate acceptance of the call.

In some implementations, the other user initialized the call via another computing device, and the method further includes, when the computing device is waiting for the user to indicate acceptance of the call: determining, in response to determining that the other user has initialized the call to the other, whether to cause the interface of the computing device to present any pre-call content.

In some implementations, the activity corresponds to the user interacting with an application and the other activity corresponds to the user interacting with another application that is different from the application.

Claim 1:
A method implemented by one or more processors, the method comprising:
generating first pre-call content for provisioning at an interface of a computing device when a user is receiving, via the computing device, an incoming call from another user (<NUM>), wherein the first pre-call content (<NUM>) is based on data that resulted from the user (<NUM>) engaging in an activity that is associated with the other user (<NUM>);
determining, subsequent to generating the first content data, that the user (<NUM>) has engaged in another activity, wherein the other activity is also associated with the other user (<NUM>) and resulted in other data being generated;
generating, based at least on the other data that resulted from the user (<NUM>) engaging in the other activity, second pre-call content (<NUM>) in furtherance of provisioning content at the interface of the computing device when the user (<NUM>) is receiving, via the computing device, the incoming call from the other user (<NUM>),
wherein the first pre-call content (<NUM>) corresponds to a first assigned priority and the second pre-call content (<NUM>) corresponds to a second assigned priority,
wherein the activity and the other activity are temporally distinct activities;
determining, subsequent to generating the second pre-call content (<NUM>), that the other user (<NUM>) has initialized a call to the user via a wireless communication protocol;
when the computing device is waiting for the user (<NUM>) to indicate acceptance of the call:
determining, based on the first assigned priority and the second assigned priority, whether to cause the interface of the computing device to prioritize a position of the first pre-call content (<NUM>) over another position of the second pre-call content (<NUM>), and
causing, in response to determining that the other user (<NUM>) has initialized the call to the user (<NUM>), the computing device to present the first pre-call content (<NUM>) and/or the second pre-call content (<NUM>) at the interface of the computing device.