Patent Publication Number: US-10768954-B2

Title: Personalized digital assistant device and related methods

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is a continuation-in-part of U.S. patent application Ser. No. 16/038,993, filed Jul. 18, 2018, entitled SYSTEM AND METHODS FOR CROWDSOURCED ACTIONS AND COMMANDS, and also claims the benefit of U.S. Provisional Patent Application No. 62/624,099, filed Jan. 30, 2018, entitled PERSONALIZED DIGITAL ASSISTANT. Each of the foregoing applications are assigned or under obligation of assignment to the same entity as this application, the entire contents of each being herein incorporated by reference. 
    
    
     BACKGROUND 
     The aspirational vision for digital assistants is to provide a conversational interface between a human user and an electronic device. Rather than having to learn commands specific to a particular device or a particular application running on the device, a user should ideally be able to provide input or instructions in language natural to the user. The ideal digital assistant would span different devices or different applications, so that a user can complete a variety of tasks without having to engage different devices or different interfaces. Current digital assistants often fall short of these aspirations. For example, digital assistants can require training that make them useful for highly repetitive tasks, but cumbersome for new tasks the digital assistant has not previously completed or variations of previously completed tasks. As another example, digital assistants can misunderstand new commands or variations on old commands, meaning that once a “natural language” command has been established, it must still be remembered and used consistently to maximize the effectiveness of the digital assistant. As yet another example, to reduce initial training requirements, digital assistants can start from arbitrary, pre-configured patterns or assumptions that might or might not serve any particular user, and can be difficult, time-consuming or even impossible to override. Digital assistants are typically more useful in locating information, documents or applications, rather than completing tasks, whereas users are oftentimes more interested in easily completing tasks. Privacy concerns can also be of concern, as consumers would prefer to retain contextual data collected regarding their phone or application usage away from digital assistant manufacturers or other third parties. There remains a significant gap between the capabilities of available digital assistants and the aspirational function of digital assistants. 
     SUMMARY 
     This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used in isolation as an aid in determining the scope of the claimed subject matter. 
     Embodiments described in the present disclosure are generally directed towards systems and methods relating to personalization of a crowd-sourced digital assistant for computing devices. In particular, embodiments facilitate the intuitive creation and distribution of action datasets that include reproducible computing events and associated commands that can be employed to invoke a reproduction of the computing events on computing devices having the crowd-sourced digital assistant installed and/or executing thereon. The described embodiments describe a digital assistant system and application that can perform any operation on a computing device by way of a received command, the operations being limited only by the various operations executable on the computing device. 
     In accordance with embodiments described herein, the described digital assistant and corresponding system provides an ever-growing and evolving library of dialects that enables the digital assistant to learn from its users, in contrast to the frustrating and limited interpretation features provided by conventional digital assistants. Further, because the digital assistant and corresponding system is configured with a framework for distributing improvements to its collection of actionable operations and understandable commands, and because the digital assistant utilizes applications existing on the computing device of each user, privacy concerns typically associated with conventional digital assistants is significantly reduced. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The present invention is described in detail below with reference to the attached drawing figures, wherein: 
         FIG. 1  is a block diagram of an exemplary computing environment for a crowd-sourced digital assistant system, in accordance with embodiments of the present invention; 
         FIG. 2  is a block diagram of an exemplary digital assistant device, in accordance with an embodiment of the present disclosure; 
         FIG. 3  is a block diagram of an exemplary digital assistant server, in accordance with an embodiment of the present disclosure; 
         FIG. 4  is an exemplary data structure of an action dataset, in accordance with an embodiment of the present disclosure; 
         FIG. 5  is a flow diagram showing a method for distributing actionable operations in a crowd-sourced digital assistant network, according to various embodiments of the present invention; 
         FIG. 6  is another flow diagram showing a method for distributing actionable operations in a crowd-sourced digital assistant network, according to various embodiments of the present invention; 
         FIG. 7  is a flow diagram showing a method for disambiguating a received command representation for selecting and executing a most relevant action dataset, according to various embodiments of the present invention; 
         FIG. 8  is another flow diagram showing a method for disambiguating a received command representation for selecting and executing a most relevant action dataset, according to various embodiments of the present invention; and 
         FIG. 9  is a block diagram of an exemplary computing environment suitable for use in implementing embodiments of the present invention. 
     
    
    
     DETAILED DESCRIPTION 
     The subject matter of the present invention is described with specificity herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the terms “step” and/or “block” can be used herein to connote different elements of methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described. 
     Aspects of the technology described herein are generally directed towards systems and methods for crowdsourcing actionable operations of a digital assistant. The described embodiments facilitate the creation, on-boarding, and distribution of action datasets to any number of computing devices having an instance of the digital assistant installed and/or executing thereon (hereinafter referenced as a “digital assistant device”). In accordance with the present disclosure, an “operation” can correspond to a final result, output, or computing operation that is generated, executed, or performed by a digital assistant device based on one or more action datasets selected and interpreted for execution by the digital assistant device, each action dataset being comprised of one or more reproducible computing events that can be invoked in response to a received command determined to correspond to the action dataset. In accordance with embodiments described herein, an “action” is described in reference to one or more operations performed in response to an action dataset selected and interpreted for execution. In this regard, an action or action dataset can be performed, invoked, initiated, or executed, among other things, and any reference to the foregoing can imply that a corresponding action dataset is selected and interpreted for execution by the digital assistant device to perform the corresponding operation. 
     In some embodiments, actions (or the action datasets corresponding thereto) can be created, by the digital assistant device, which can record a series of detected events (e.g., inputs) that are typically provided by a user of the digital assistant device when manually invoking the desired operation (e.g., with manual inputs via a touchscreen or other input method of the digital assistant device). That is, to create a new action dataset, the digital assistant device can invoke a recording mode where a user can simply perform a series of computing operations (e.g., manual touches, click inputs) within one or more applications to achieve a desired result or operation. After the recording is stopped by the user, via a terminating input, the action dataset can store and be associated with a set of command templates corresponding to commands that the user would preferably announce to the digital assistant device when an invocation of the operation is desired. In various embodiments, a command representation can be received as speech data and converted to text (e.g., by a speech engine of the digital assistant device), or received as text input data. In accordance with embodiments described herein, a “command” is referenced herein to describe data, received as speech data or as text data. A “command representation,” on the other hand is referenced to describe text data that is received, based on inputs (e.g., keyboard), received speech data converted to text data, or received text data communicated from another computing device. A “command template” is referenced herein to describe a portion of a command representation having defined parameter fields in place of variable terms. 
     In more detail, one or more terms or keywords in the received command can be defined as a parameter based on input(s) received from the user. A parameter, in accordance with the present disclosure, can be referenced as corresponding to one of a plurality of predefined parameter types, such as but not limited to, genre, artist, title, location, name or contact, phone number, address, city, state, country, day, week, month, year, and more. It is also contemplated that the digital assistant device can access from a memory, or retrieve (e.g., from a server), a set of predefined parameter types that are known or determined to correspond to the application or applications for which an action dataset is being created. In some embodiments, the set of predefined parameter types can be determined based at least in part on corresponding application identifying information. The digital assistant device can extract, based on the defined parameters, the corresponding keywords and generate a command template based on the remaining terms and the defined parameters. By way of example only, if the command was originally received as “play music by Coldplay,” and the term “Coldplay” is defined as a parameter of type “artist,” a resulting command template generated by the digital assistant device can appear as “play music by &lt;artist&gt;”. In this regard, a command template can include the originally received command terms if no parameters are defined, or can include a portion of the originally received command terms with parameter fields defined therein, the defined parameters corresponding to variable terms of a command. 
     The digital assistant device can receive, among other things, application identifying information, a recorded series of events, and a set command templates, among other things, to generate a new action dataset that can be retrieved, interpreted and/or invoked by the digital assistant device, simply based on a determination, by the digital assistant device, that a received command or command representation is associated with the action dataset. When an action is invoked based on a determination that a received command or command representation corresponds to an action dataset, the digital assistant device can reproduce (e.g., emulate, invoke, execute, perform) the recorded series of events associated with the corresponding action dataset, thereby performing the desired operation. Moreover, in circumstances where a received command or command representation includes a parameter term, and a determination is made that the received command or command representation corresponds to an action dataset having a parameter field that also corresponds to the parameter term, the parameter term can be employed, by the digital assistant device, to perform custom operations while performing the action. For instance, the digital assistant device can input the parameter term as a text input into a field of the application. 
     In some further embodiments, an action dataset, once created by the digital assistant device, can be uploaded (hereinafter also referenced as “on-boarded”) to a remote server for storage thereby. The action dataset can be on-boarded automatically upon its generation or on-boarded manually based on a received instruction, by the digital assistant device. It is contemplated that individuals may want to keep their actions or command templates private, and so an option to keep an action dataset limited to locally-storage can be provided to the user (e.g., via a GUI element). The server, upon receiving an on-boarded action dataset, can analyze the action dataset and generate an associated action signature based on the metadata, characteristics and/or contents of the action dataset. Contents of an action dataset can include, among other things, application identifying information, corresponding command templates and parameters, and a recorded series of events. The action signature can be generated by various operations, such as hashing the on-boarded action dataset with a hashing algorithm, by way of example. It is also contemplated that the action signature can be generated by the on-boarding digital assistant device, the generated action signature then being stored in or appended to the action dataset before it is uploaded to the server. 
     In one aspect, the server can determine that the on-boarded action dataset already exists on the server, based on a determination that the action signature corresponds to the action signature of another action dataset already stored on the server. The server can either dispose of the on-boarded action dataset or merge the on-boarded action dataset (or determined differing portion(s) thereof) with an existing action dataset stored thereby, preventing redundancy and saving storage space. In another aspect, the server can analyze the on-boarded action dataset to determine if its contents (e.g., the recorded events, command templates, metadata) comply with one or more defined policies (e.g., inappropriate language, misdirected operations, incomplete actions) associated with general usage of the digital assistant system. In another aspect, the server can employ machine learning algorithms, among other things, to perform a variety of tasks, such as determining relevant parameter types, generating additional command templates for association with an on-boarded or stored action dataset, comparing similarity of events between on-boarded action datasets to identify and select more efficient routes for invoking an operation, and more. 
     In some further embodiments, the server can distribute one or more stored actions datasets to a plurality of digital assistant devices in communication with the server. In this way, each digital assistant device can receive action datasets or portions thereof (e.g., command templates) from the server. The action datasets can be distributed to the digital assistant devices in a variety of ways. For instance, in an embodiment, the server can freely distribute any or all determined relevant action datasets to digital assistant devices. In an embodiment, an application profile including a list of applications installed on a digital assistant device can be communicated to the server. Based on the application profile for the digital assistant device, the server can distribute any or all determined relevant action datasets to the digital assistant device. As digital assistant devices can include a variety of operating systems, and versions of applications installed thereon can also vary, it is contemplated that the application profile communicated by a digital assistant device to the server can include operating system and application version information, among other things, so that appropriate and relevant action datasets are identified by the server for distribution to the digital assistant device. For a more granular implementation, an action dataset profile including a list of action datasets or action signatures stored on the digital assistant device can be communicated to the server. In this way, only missing or updated action datasets can be distributed to the digital assistant device. 
     In some embodiments, a user can simply announce a command to the digital assistant device, and if a corresponding action dataset is not stored on the digital assistant device, the digital assistant device can send the command (representation) to the server for determination and selection of a set of relevant action datasets, which can then be communicated to the digital assistant device. Provided that the digital assistant device has the corresponding application installed thereon, the digital assistant device can retrieve, from the server, a set of determined most relevant action datasets, without additional configuration or interaction by the user, also reducing server load and saving bandwidth by inhibiting extraneous transfer of irrelevant action datasets. A retrieved set of relevant action datasets can be received from the server for invocation by the digital assistant device. It is further contemplated that if two or more action datasets are determined equally relevant to a received command, each action dataset can be retrieved from the server, and the digital assistant device can provide for display a listing of the determined relevant action datasets for selection and execution. In some embodiments, contextual information (e.g., current location, location history, previously-received and/or executed actions, previously-received commands or parameters parsed therefrom, demographic user information, search history, application usage history, application metadata) collected and/or determined by the digital assistant device can be analyzed and factored into a selection of a most-relevant action dataset. In some aspects, the contextual information can be stored in a memory of the digital assistant device to maintain privacy, and in this regard, the digital assistant device can determine and select the most-relevant action dataset. In some further aspects, the contextual information can be communicated from the digital assistant device to the server, such that the contextual information, among other things is associated with a user and/or device profile (referred to herein as “profile information”) that is stored on the server and associated with the digital assistant device. In this way, if two or more action datasets are determined equally relevant to a command received from a device associated with the profile information, the server can analyze the contextual information to determine and select the most-relevant action dataset. In some aspects, each action dataset determined relevant to a received command can be retrieved from the server, and the digital assistant device can provide for display a listing of the determined relevant action datasets for selection and execution. In this regard, the selection corresponding to one of the displayed action datasets can be stored in association with the profile information, for future relevance determinations by any one of the digital assistant device or server. 
     In some further embodiments, a user of a digital assistant device can customize command templates associated with an action dataset corresponding to an application installed on their digital assistant device. Put simply, a user can employ the digital assistant (or a GUI thereof) to select an action dataset from a list of action datasets stored on the computing device, select an option to add a new command to the action dataset, and define a new command and any associated parameters for storage in the action dataset. In this regard, the user can add any custom command and parameter that can later be understood by the digital assistant device to invoke the action. In some aspects, the custom command and/or modified action can be on-boarded to the server for analysis and storage, as noted above. In some further aspects, based on the analysis, the server can distribute the custom command and/or at least a portion of the modified action dataset to a plurality of other digital assistant devices. In this regard, the list of understandable commands and corresponding actions can continue to grow and evolve, and be automatically provided to any other digital assistant device. 
     Accordingly, at a high level and with reference to  FIG. 1 , an example operating environment  100  in which some embodiments of the present disclosure can be employed is depicted. It should be understood that this and other arrangements and/or features described by the enclosed document are set forth only as examples. Other arrangements and elements (e.g., machines, interfaces, functions, orders, and groupings of functions, etc.) or features can be used in addition to or instead of those described, and some elements or features can be omitted altogether for the sake of clarity. Further, many of the elements or features described in the enclosed document can be implemented in one or more components, or as discrete or distributed components or in conjunction with other components, and in any suitable combination and location. Various functions described herein as being performed by one or more entities can be carried out by hardware, firmware, and/or software. For instance, some functions can be carried out by a processor executing instructions stored in memory. 
     The system in  FIG. 1  includes one or more digital assistant devices  110 ,  115   a ,  115   b ,  115   c , . . .  115   n , in communication with a server  120  via a network  130  (e.g., the Internet). In this example, the server  120 , also in communication with the network  130 , is in communication with each of the digital assistant devices  110 ,  115   a - 115   n , and can also be in communication with a database  140 . The database  140  can be directly coupled to the server  120  or coupled to the server  120  via the network  130 . The digital assistant device  110 , representative of other digital assistant devices  115   a - 115   n , is a computing device comprising one or more applications  112  and a digital assistant module  114  installed and/or executing thereon. 
     The one or more applications  112  includes any application that is executable on the digital assistant device  110 , and can include applications installed via an application marketplace, custom applications, web applications, side-loaded applications, applications included in the operating system of the digital assistant device  110 , or any other application that can be reasonably considered to fit the general definition of an application or mobile application. On the other hand, the digital assistant module  114  can provide digital assistant services installed on the digital assistant device  110  or provided by the server  120  via the network  130 , or can be implemented at least partially into an operating system of the digital assistant device  110 . In accordance with embodiments described herein, the digital assistant module  114  provides an interface between a digital assistant device  110  and an associated user (not shown), generally via a speech-based exchanged, although any other method of exchange between user and digital assistant device  110  (e.g., keyboard input, communication from another digital assistant device or computing device) remains within the purview of the present disclosure. 
     When voice commands are received by the digital assistant device  110 , the digital assistant module  114  can convert the speech command to text utilizing a speech-to-text engine (not shown) to extract identified terms and generate a command representation. The digital assistant module  114  can receive the command representation, and determine that the command representation corresponds to at least one command template of at least one action dataset stored on the digital assistant device. In some embodiments, the digital assistant module can generate an index of all command templates stored on the digital assistant device  110  for faster searching and comparison of the received command representation to identify a corresponding command template, and thereby a corresponding action dataset. Each indexed command template can be mapped to a corresponding action dataset, which can be interpreted for execution in response to a determination of a confirmed match with the received command representation. 
     By way of brief overview, a command template can include one or more keywords and/or one or more parameters that each have a corresponding parameter type. Each command template generally corresponds to an operation that can be performed on one or more applications  112  installed on a digital assistant device  110 . Moreover, a plurality of command templates can correspond to a single operation, such that there are multiple equivalent commands that can invoke the same operation. By way of example only, commands such as “check in,” check into flight,” “please check in,” “check into flight now,” “check in to flight 12345,” and the like, can all invoke the same operation that, by way of example only, directs the digital assistant module  114  to execute an appropriate airline application on the digital assistant device  110  and perform a predefined set of events or computer operations to achieve the same result. 
     The aforementioned commands, however, can lack appropriate information (e.g., the specific airline). As one of ordinary skill may appreciate, a user can have multiple applications  112  from various vendors (e.g., airlines) associated with a similar service (e.g., checking into flights). A digital assistant device  110  in accordance with embodiments described herein can provide features that can determine contextual information associated with the digital assistant device  110 , or its associated user, based on collected historical use (e.g., previously-received commands or parameters included therein, selected action datasets, executed actions) of the digital assistant device  110 , collected location history data (e.g., GPS data, Wi-Fi) of the digital assistant device  110 , collected search history data (e.g., web, memory, contacts, documents, photos) of the digital assistant device  110 , profile information stored on the digital assistant device  110  or server  120 , indexed messages (e.g., email, text messages) stored on the digital assistant device, and a variety of other types of data (e.g., visual data, screen shots, optical data, audio data, input data) collected by the digital assistant device and/or stored locally on the digital assistant device or remotely on a server, such as server  120 . In this way, the digital assistant device or server can employ this data to identify a most relevant parameter and supplement a command and/or select a most relevant action dataset. More specific commands, such as “check into Friendly Airline flight,” or “Friendly Airline check in,” and the like, where a parameter is specifically defined in the command, can be recognized by the digital assistant module  114 . As described herein, the determination of the most relevant action dataset can be performed locally on the digital assistant device or remotely on the server, in accordance with various embodiments. 
     One or more recognizable commands and corresponding action datasets can be received by the digital assistant device  110  from the server  120  at any time, including upon installation, initialization, or invocation of the digital assistant module  114 , after or upon receipt of a speech command by the digital assistant module  114 , after or upon installation of a new application  112 , periodically (e.g., once a day), when pushed to the digital assistant device  110  from the server  120 , among many other configurations. It is contemplated that the action datasets received by the digital assistant device  110  from the server  120  can be limited based at least in part on the applications  112  installed on the digital assistant device  110 , although configurations where a larger or smaller set of action datasets received are contemplated. 
     In the event an action dataset is determined not available for a particular application  112  installed on the digital assistant device  110 , digital assistant module  114  can either redirect the user to a marketplace (e.g., launch an application marketplace application) to install the appropriate application determined by the server  120  based on the received command, or can invoke an action training program that prompts a user to manually perform tasks on one or more applications to achieve the desired result, the tasks being recorded and stored into a new action dataset by the digital assistant device  110 . The digital assistant module  114  can also receive one or more commands from the user (e.g., via speech or text) to associate with the action dataset being generated. If the command includes variable parameters (e.g., optional fields), the action training program can facilitate a definition of such parameters and corresponding parameter types to generate command templates for inclusion in the action dataset being generated. In this way, a command template(s) is associated with at least the particular application designated by the user and also corresponds to the one or more tasks manually performed by the user, associating the generated command template to the task(s) and thus the desired resulting operation. 
     In some instances, the server  120  can provide a determined most-relevant action dataset to the digital assistant device  110  based on the received command. The server  120  can store and index a constantly-growing and evolving plurality of crowd-sourced action datasets submitted by or received from digital assistant devices  115   a - 115   n  also independently having a digital assistant module  114  and any number of applications  112  installed thereon. The digital assistant devices  115   a - 115   n  can have any combination of applications  112  installed thereon, and any generation of action datasets performed on any digital assistant device  110 ,  115 - 115   n  can be communicated to the server  120  to be stored and indexed for mass or selective deployment, among other things. In some aspects, the server  120  can include various machine-learned algorithms to provide a level of quality assurance on command templates included in on-boarded action datasets and/or the tasks and operations performed before they are distributed to other digital assistant devices via the network  130 . 
     When the digital assistant module  114  determines an appropriate action dataset (e.g., one or more tasks to achieve a desired result) having one or more command templates that corresponds to the received command, the digital assistant module  114  can generate an overlay interface that can mask any or all visual outputs associated with the determined action or the computing device generally. The generation of the overlay interface can include a selection, by the digital assistant module  114 , of one or more user interface elements that are stored in a memory of the digital assistant device  110  or server  120 , and/or include a dynamic generation of the user interface element(s) by the digital assistant module  114  or server  120  based on one or more portions of the received command and/or obtained contextual data (e.g., determined location data, user profile associated with the digital assistant device  110  or digital assistant module  114 , historical data associated with the user profile, etc.) obtained by the digital assistant device  110 , digital assistant module  114 , and/or server  120 . The selected or generated one or more user interface elements can each include content that is relevant to one or more portions (e.g., terms, keywords) of the received command In the event of dynamic generation of user interface elements, such elements can be saved locally on the digital assistant device  110  or remotely on the server  120  for subsequent retrieval by the digital assistant device  110 , or can be discarded and dynamically regenerated at any time. 
     Example operating environment depicted in  FIG. 1  is suitable for use in implementing embodiments of the invention. Generally, environment  100  is suitable for creating, on-boarding, storing, indexing, crowd-sourcing (e.g., distributing), and invoking actions or action datasets, among other things. Environment  100  includes digital assistant device  110 , action cloud server  120  (hereinafter also referenced as “server”) and network  130 . Digital assistant device  110  can be any kind of computing device having a digital assistant module installed and/or executing thereon, the digital assistant module being implemented in accordance with at least some of the described embodiments. For example, in an embodiment, digital assistant device  110  can be a computing device such as computing device  900 , as described below with reference to  FIG. 9 . In embodiments, digital assistant device  110  can be a personal computer (PC), a laptop computer, a workstation, a mobile computing device, a PDA, a cell phone, a smart watch or wearable, or the like. Any digital assistant device described in accordance with the present disclosure can include features described with respect to the digital assistant device  110 . In this regard, a digital assistant device can include one or more applications  112  installed and executable thereon. The one or more applications  112  includes any application that is executable on the digital assistant device, and can include applications installed via an application marketplace, custom applications, web applications, side-loaded applications, applications included in the operating system of the digital assistant device, or any other application that can be reasonably considered to fit the general definition of an application. On the other hand, the digital assistant module can be an application, a service accessible via an application installed on the digital assistant device or via the network  130 , or implemented into a layer of an operating system of the digital assistant device  110 . In accordance with embodiments described herein, the digital assistant module  114  can provide an interface between a digital assistant device  110  and a user (not shown), generally via a speech-based exchange, although any other method of exchange between user and digital assistant device can be considered within the purview of the present disclosure. 
     Similarly, action cloud server  120  (“server”) can be any kind of computing device capable of facilitating the on-boarding, storage, management, and distribution of crowd-sourced action datasets. For example, in an embodiment, action cloud server  120  can be a computing device such as computing device  900 , as described below with reference to  FIG. 9 . In some embodiments, action cloud server  120  comprises one or more server computers, whether distributed or otherwise. Generally, any of the components of environment  100  can communicate with each other via a network  130 , which can include, without limitation, one or more local area networks (LANs) and/or wide area networks (WANs). Such networking environments are commonplace in offices, enterprise-wide computer networks, intranets, and the Internet. The server  120  can include or be in communication with a data source  140 , which can comprise data sources and/or data systems, configured to make data available to any of the various constituents of the operating environment  100 . Data sources  140  can be discrete from the illustrated components, or any combination thereof, or can be incorporated and/or integrated into at least one of those components. 
     Referring now to  FIG. 2 , a block diagram  200  of an exemplary digital assistant device  210  suitable for use in implementing embodiments of the invention is shown. Generally, digital assistant device  210  (also depicted as digital assistant device  110  of  FIG. 1 ) is suitable for receiving commands or command representations, selecting action datasets to execute by matching received commands to command templates of action datasets, or determining that no action datasets correspond to received commands, interpreting a selected action dataset to execute the associated operation, generating new action datasets, and sending action datasets to or receiving action datasets from a digital assistant server, such as server  120 . 
     Digital assistant device  210  can include, among other things, a command receiving component  220 , an action matching component  230 , an action executing component  240 , a training component  250 , and a server interfacing component  260 . The command receiving component  220  can receive a command, either in the form of speech data or text data. The speech data can be received via a microphone of the digital assistant device  210 , or another computing device paired to or in communication with the digital assistant device  210 . The command receiving component  220 , after receiving the speech data, can employ a speech-to-text engine of the digital assistant device  210  to generate a command representation (e.g., a text string of the command). Text data received by command receiving component  220 , on the other hand, can be received via a virtual keyboard or other input method of the digital assistant device  210 , and similarly, can be received from another computing device paired to or in communication with the digital assistant device  210 . Received text data is already in the form of a command representation, and is treated as such. In various embodiments, command receiving component  210  can be invoked manually by a user (e.g., via an input to begin listening for or receiving the command), or can be in an always-listening mode. 
     Based on a command representation being received, action matching component  230  can determine whether one or more action datasets stored on the digital assistant device  210  include a command template that corresponds to or substantially corresponds (e.g., at least 90% similar) to the received command representation. In some aspects, a corresponding command template can be identified, and the action dataset of which the corresponding command template is stored in is selected for interpretation by action executing component  240 . In some other aspects, a corresponding command template cannot be identified, and either the training component  250  can be invoked, or the received command is communicated to the digital assistant server (depicted as server  120  of  FIG. 1  and digital assistant server  310  of  FIG. 3 ) via the server interfacing component  260 . For privacy reasons, users can employ a GUI of the digital assistant device  210  that presents options to opt-in or opt-out of sending their own command representations to the digital assistant server, so as to alleviate privacy concerns. In this regard, a selection corresponding to one of these options can be received, stored on the digital assistant device  210 , and employed by the digital assistant device  210  to select how relevant action datasets are to be determined. 
     After receiving the command representation, digital assistant device  210  can display or otherwise affirm the command representation while the command representation is being processed. The digital assistant device  210  can process the representation and execute a corresponding action. In an example, the command representation is “get boarding pass.” The digital assistant recognizes this as a request for an airline boarding pass. Utilizing various techniques described herein, the digital assistant device  210  can determine which airline and which boarding pass to present. The digital assistant device  210  can open a website and navigate to a page where a corresponding boarding pass would be displayed. If the digital assistant device  210  is unable to determine which airline or which boarding pass to present, digital assistant device  210  can present an alternate display that presents various options to clarify the command, such as a list of likely airlines, or a list of possible travel dates or flight numbers (e.g., relevant parameters required to complete generation of the command representation). In some aspects, the digital assistant device can verbally prompt a request (e.g., generate a verbal output prompt via a speaker of the digital assistant device) that corresponds to each of the required parameters and/or possible options. Upon receipt of verbal or physical inputs that correspond to the required parameters, the digital assistant device  210  can add the required parameters to the command representation, for processing and selection of a relevant action dataset. 
     In some aspects, if an obtained command representation is determined not to correspond to a relevant action dataset (e.g., directly or within a threshold confidence level), the digital assistant device  210  can attempt to disambiguate the command In some aspects, the digital assistant device  210  or digital assistant server can disambiguate the command without requesting additional input. 
     By way of non-limiting example, the digital assistant device  210  or digital assistant server can check to see if the received command representation corresponds to two or more relevant action datasets. In various embodiments, the digital assistant device  210  or digital assistant server can access stored profile information associated with the digital assistant device  210  to determine a similarity of the received command representation to a previously-received command representation, a most recently-received command representation, or a common command representation among recently-received command representation, among other things. The digital assistant device  210  can determine that the user, by way of the received command representation, likely intends one action over another possible action for the given command representation. If so, the digital assistant device  210  can automatically modify the received command representation with parameters extracted from other command representations corresponding to one or more previously-received command representations to select a relevant action dataset, or if necessary, communicate the modified command representation to the digital assistant server for making the determination. 
     In some aspects, if a received command is ambiguous and there is no locally-stored profile information or determined corresponding action dataset, the digital assistant device  210  can generate a query and search stored profile information that can include relevant data that corresponds to a most-likely action dataset. For example, if a user of digital assistant device  210  speaks “play my playlist by [ambiguous] Simon” and the digital assistant device  210  or profile information includes data corresponding to a play list for artist “Carly Simon,” but not “Paul Simon,” the digital assistant device  210  can determine a likely intent to play music by “Carly Simon,” and modify a received command representation to include parameters corresponding to “Carly Simon.” If there is no locally stored profile information, or if the profile information does not resolve the ambiguity (e.g., the playlist includes both Carly Simon and Paul Simon and both have been played recently), the digital assistant device  210  can, in some embodiments, communicate the command representation to the digital assistant server, and the digital assistant server can select a more frequently received command representation corresponding to “play my playlist by [ambiguous] Simon.” More specifically, if the digital assistant server receives command representations including “Paul Simon” with “play my playlist by [ambiguous parameter] Simon” more frequently than “Carly Simon,” the digital assistant server can determine that the more relevant action dataset corresponds to an action dataset that will invoke a set of instructions or operations on the digital assistant device  210  to play music by Paul Simon. 
     In some instances, the digital assistant device  210  or digital assistant server can determine that a received command representation is missing a parameter. For example, a determination can be made that the received command representation is missing a parameter—which song or artist to play. If a parameter is missing, the digital assistant device  210  or digital assistant server can similarly generate a query and/or search associated profile information or select a more frequently-received command representation to modify or replace the received command representation, as described above. As also noted, the digital assistant device  210  can generate prompts for the determined missing parameter(s). In various embodiments, the digital assistant device  210  can perform as much of the action as possible without the parameter. For example, if the user has a local playlist, the digital assistant device  210  can open the local playlist, or if the digital assistant device  210  has a music-related application installed thereon, the digital assistant device  210  can simply execute operations to open the music-related app. As an example, the digital assistant device  210  does not fully execute an action dataset by playing a particular song or music of a particular style or particular artist, but would follow the operations of a determined relevant action dataset until further steps could not be performed (e.g., selecting the desired album or artist and initiating music playback) without the determined missing parameter. 
     If any action dataset includes command template(s) determined relevant to a received command representation and does not require a parameter, the digital assistant device  210  can generate a prompt to inquire a user to select whether a particular action can be inferred from profile information. For example, if a received command representation includes “Play Music,” the digital assistant device  210  can check a list of applications installed on the device, determine, for example, that SPOTIFY and PANDORA both play music. The digital assistant device  210  can determine from the profile information that PANDORA was a most recent application executed on the digital assistant device  210 , select and execute an action dataset that corresponds to operations for playing music from PANDORA. If profile information is not available or does not support an inference determination of what action dataset to select, the digital assistant device  210  can communicate the command representation to the digital assistant server for performing the determination. If the digital assistant server cannot make such a determination (e.g., without meeting a defined confidence threshold), the digital assistant device can present possible actions or action datasets for selection, or can prompt the receipt of a new input (e.g., command). 
     In some aspects, one or more portions of (e.g., terms, keywords, parameters) received command representations and/or selected and executed action datasets can be stored in associated profile information with or without substantially contemporaneous contextual information detected by digital assistant device  210  for future use. In some aspects, these parsed and stored portions can be stored in profile information on the digital assistant device  210  or digital assistant server, and can further be associated with one of a plurality of corresponding application metadata, usage (e.g., time of use) history, corresponding location (e.g., location of use) history, or other like characteristics, which can be detected and stored by the digital assistant device  210  or digital assistant server in association with application or action dataset selection and/or usage. As such, in some aspects, if the digital assistant device  210  or digital assistant server determines that a plurality of action datasets are relevant to a received command representation, the digital assistant device  210  or digital assistant server can detect current contextual information (e.g., location, time, recent application usage history, currently-opened application) and determine whether a threshold confidence level can be determined that either of the determined relevant action datasets is related to one or more portions of stored profile information or application characteristics of such portions. Profile information and associated application characteristics can be employed by the digital assistant device  210  or digital assistant server to determine whether a received command representation corresponds to one of the determined relevant action datasets associated with a recently or commonly-utilized application, a similar type of recently or commonly-utilized application (e.g., airline applications, music applications, weather applications, financial applications, etc.), a recently or commonly-accessed digital asset (e.g., photo, song, document, web page), or any other recently or commonly-utilized feature or stored digital asset of a digital assistant device  210  that is accessed or executed. In some aspects, profile information and associated application characteristics can be employed to determine whether certain determined relevant action dataset(s) are temporally related, related to similar events, related by location, or more. It is contemplated that data stored in profile information can be associated with similar application characteristics even if they are semantically distinct. By way of example, received command representations “check in” and “boarding passes” can cause the digital assistant device  210  to trigger actions in the same application, as they can be temporally related (e.g., within a particular time frame, or typically associated with one another). Similarly, “boarding pass” is likely to be determined associated with the same airline application recently used for a check-in performed on the digital assistant device  210 , or a for a digital assistant device  210  frequently employed to retrieve and present a boarding pass to a flight shortly after performing a check-in operation on the digital assistant device  210 . Storing profile information with collected contextual information and/or application characteristics can enable the digital assistant device  210  or digital assistant server analyze and determine patterns or make confidence determinations through determined temporal relationships, location-based relationships, usage history relationships, and more. In some aspects, command-action combinations can be stored in profile information with or without additional contextual information, which can be employed for future analysis. In various embodiments, application characteristics and contextual data associated with profile information can be weighted as factors in relevance determinations by way of one or more defined rules that are stored and employed by the digital assistant device  210  or digital assistant server. 
     The action executing component  240  can receive a selected action dataset, either selected by digital assistant device  210  from local storage, by the digital assistant server from storage accessible thereto, or selected from a list presented by digital assistant device  210 . The action executing component  240  can, from the received action dataset, interpret event data, which can include executable code, links, deep links, references to GUI elements, references to screen coordinates, field names, or other pieces of data that can correspond to one or more tasks or events stored in the selected action dataset. When the event data is interpreted, the action executing component  240  can reproduce the events that were recorded when the action dataset was initially generated, by any digital assistant device such as digital assistant device  210 . In some aspects, the event data can include time delays, URLs, deep links to application operations, or any other operation that can be accessed, processed, emulated, or executed by the action executing component  240 . In some aspects, events like click or touch inputs, can be reproduced on the digital assistant device  210 , based on the interpreted event data stored in an action dataset. 
     The training component  250  can facilitate the generation of an action dataset, among other things. When the training component  250  is invoked, an indication, such as a GUI element, indicating that an action recording session has begun can be presented for display. A prompt to provide the tasks or events required to perform the desired operation can also be presented for display. In this regard, a user can begin by first launching an application for which the operation is associated with, and proceed with providing inputs to the application (i.e., (performing the requisite tasks). The inputs can be recorded by the digital assistant device  210 , and the training component  250  can listen for, parse, identify, and record a variety of attributes of the received inputs, such as long or short presses, time delays between inputs, references to GUI elements interacted with, field identifiers, application links activated based on received inputs (e.g., deep links), and the like. The recorded inputs and attributes (e.g., event data) can be stored, sequentially, in an event sequence, and stored into a new action dataset. The application launched is also identified, and any metadata or application identifying information, such as operating system, operating system version, application version, paid or free version status, and more, can be determined from associated application metadata and also stored into the new action dataset. When the desired operation is completed (i.e., all requisite tasks/events performed), a user can activate a training termination button, which can be presented as a floating button or other input mechanism that is preferably positioned away from an active portion of the display. Other termination methods are also contemplated, such as voice activated termination, or motion activated termination, without limitation. 
     The training component  250  can further request that the user provide a set of commands that correspond to the desired operation. A command can be received via speech data and converted to a command representation by a speech to text engine, or received via text input as a command representation, among other ways. When the set of commands is provided and stored as command representations, the training component  250  can further prompt the user to define any relevant parameters or variables in the command representations, which can correspond to keywords or values that can change whenever the command is spoken. In this regard, a user can select one or more terms included in the received command representations, and define them with a corresponding parameter type selected from a list of custom, predefined, or determined parameter times, as described herein. The training component  250  can then extract the selected one or more terms from a command representation defined as parameter(s), replacing them with parameter field identifier(s) of a corresponding parameter type, and store the resulting data as a command template. The training component  250  can then generate the action dataset from the recorded event sequence, the application identifying information, and the one or more defined command templates. In some embodiments, the training component  250  can generate an action signature or unique hash based on the generated action dataset or one or more portions of data included therein. The action signature can be employed by the digital assistant server to determine whether the action dataset or data included therein is redundant, among other things. 
     Looking now to  FIG. 3 , a block diagram  300  of an exemplary digital assistant server  310  suitable for use in implementing embodiments of the invention is shown. Generally, digital assistant server  310  (also depicted as server  120  of  FIG. 1 ) is suitable for establishing connections with digital assistant device(s)  210 , receiving generated action datasets, maintaining or indexing received action datasets, receiving commands or command representations to determine one or more most relevant action datasets for selection and communication to a sending digital assistant device of the command or command representation, and distributing action datasets to other digital assistant devices, such as digital assistant device  210 . Digital assistant server  310  can include, among other things, an on-boarding component  320 , an indexing component  330 , a maintenance component  340 , a relevant component  350 , and a distribution component  360 , among other things. 
     The on-boarding component  320  can receive action datasets generated by one or more digital assistant devices  210  in communication therewith. In some aspects, the on-boarding component can generate an action signature for a received action dataset, similar to how a digital assistant device can, as described herein above. Before storing the received action dataset, the action signature can be searched (e.g., utilizing a query generated therefrom) utilizing the indexing component  330 , which maintains an index of all action datasets stored by the digital assistant server  310 . The indexing component  330  facilitates quick determination of uniqueness of received action datasets, and reduces redundancy and processing load of the digital assistant server  310 . 
     On a similar note, the maintenance component  340  can determine whether any portion of a received action dataset is different than action datasets already stored on or by the server (e.g., in a database), and extract such portions for merging into the existing corresponding action datasets. Such portions can be identified in circumstances where only command templates are hashed in the action signature, or where each portion of the action dataset (e.g., application identifying information, command template(s), event sequence) is independently hashed either by training component  240  of  FIG. 2  or on-boarding component  310  of  FIG. 3 , to more easily identify changes or differences between action datasets. By way of example, in some embodiments, a received action dataset can include separate hashes for its application identifying information, event sequence, and command template(s). In this regard, the digital assistant server  310  can employ the indexing component  330  and maintenance component  340  to quickly identify, for instance, that the received action data corresponds to a particular application and operation, or that the command template(s) are different than those stored in the stored action dataset by virtue of the command template hashes being different. Similarly, the independent hash signatures for each portion of data included in an action dataset can facilitate efficient determination of changes or differences between any combination of data portions in a received action dataset and a stored action dataset. 
     Relevance component  350  can determine, based on commands or command representations received by a digital assistant device  210 , a likelihood that a particular command template corresponds to the received command or command representation. While a variety of relevance determining methods can be employed, a machine learning implementation can be preferable, though a ranking of determined most similar command templates to a command or command representation received from a digital assistant device  210  can also facilitate a determination of relevance and therefore one or more most relevant command templates. Determined most-relevant command templates can thereby facilitate the selection of a most relevant action dataset to be distributed to the command-sending digital assistant device  210 . 
     The distribution component  360  can distribute or communicate to one or more digital assistant devices  210 , determined relevant or most relevant action datasets, determined new action datasets, determined updated action datasets, any portion and/or combination of the foregoing, or generated notifications corresponding to any portion and/or combination of the foregoing, among other things, based on a variety of factors. For instance, the distribution component  360  can include features that determine, among other things, which applications are installed on a digital assistant device  210 . Such features can enable the digital assistant server  310  to determine which action datasets or portions thereof are relevant to the digital assistant device  210 , and should be distributed to the digital assistant device  210 . For instance, a digital assistant device  210  profile (not shown) describing all applications currently installed or executable by a digital assistant device  210 , can be maintained (e.g., stored, updated) by the digital assistant server  310 . The profile can be updated periodically, manually, or dynamically by a server interfacing component  260  of the digital assistant device  210  (e.g., whenever the digital assistant is in communication with and sends a command to the digital assistant server  310 , or whenever an application is installed or updated on the digital assistant device  210 ). The distribution component  360  can distribute or communicate notifications, action datasets, or portions thereof, in a variety of ways, such as pushing, sending in response to received requests for updates, sending in response to established communications with a digital assistant device  210 , or by automatic wide scale (e.g., all digital assistant devices) or selective scale (e.g., region, location, application type, application name, application version) distribution, among other things. 
     Turning now to  FIG. 4 , a data structure  400  of an exemplary action dataset  410  in accordance with some of the described embodiments is illustrated. The depicted data structure is not intended to be limiting in any way, and any configuration of the depicted data portions of information can be within the purview of the present disclosure. Further, additional data portions or less data portions can be included in an action dataset  410  also remaining within the purview of the present disclosure. 
     In the depicted data structure  400 , the action dataset  410  includes application identifying information  420 , recorded event sequence data  430 , and command templates  440 . In some embodiments, the action dataset  410  further includes hash(es)  450 , which can include a hash value generated based on the entire action dataset  410 , or hash values generated based on any portion of the aforementioned data portions  420 ,  430 ,  440 , among other things. The action dataset  410  can be generated by training component  250  of digital assistant device  210  of  FIG. 2  and/or received from distribution component  360  of digital assistant server  310  of  FIG. 3 . 
     The application identifying information  420  can include information about a particular application that is required for execution to perform a particular operation for which the action dataset  410  was created. Exemplary pieces of application identifying information  420  are depicted in identifying information  425 , which can include any one or more of an operating system (OS) name for which the particular application is executed on, an OS version of the aforementioned OS, a defined native language of the aforementioned OS, a name of the particular application, a version of the particular application, and the like. It is contemplated that the application identifying information  420  is required and checked (e.g., by the digital assistant server  310  of  FIG. 3 ), before an action dataset  410  is distributed to a digital assistant device (e.g., digital assistant device  210  of  FIG. 2 ) and employed by the digital assistant device to ensure that the action dataset  410  is compatible with, or can be correctly interpreted by action executing component  240  of  FIG. 2 , so that the corresponding and desired operation is performed by the digital assistant device  210 . 
     The recorded event sequence data  430  can include any or all task or event-related data that was obtained, received, or determined by the digital assistant device (e.g., via training component  250  of  FIG. 2 ) responsible for generating the action dataset  410 . As noted herein, the recorded event sequence data can include timing attributes of received inputs (e.g., delays before or in between successive inputs, duration of inputs, GUI elements interacted with, relative positions of GUI elements, labels or metadata of GUI elements, scroll inputs and distances, links or URLs accessed activated, detected activation of application deep links activated in response to received inputs, and more). In some instances, the recorded event sequence data  430  can include conditions that require actual user intervention before subsequent events or tasks are resumed. For instance, secured login screens can require that a user input username and password information before an application is executed. In this regard, the recorded event sequence data  430  can include a condition corresponding to when user authentication has occurred, and instructions (e.g., interpretable by action executing component  240 ) to proceed with the tasks or events in the recorded event sequence data  430  based upon an occurrence of the condition. In various implementations, it is contemplated that the action executing component  240  of  FIG. 2  can parse metadata, GUI elements, or other information from an executing application to determine when certain events occur or conditions are met. In this regard, additional conditions can be included in the recorded event sequence data  430  that require prior events or tasks to be completed, or certain GUI elements be displayed or interacted with, or any other conditions to be met, before subsequent events or tasks are performed by the action executing component  240  of  FIG. 2 . 
     Turning now to  FIG. 5 , an embodiment of a process flow or method  500  for distributing actionable operations (i.e., action datasets) for a crowd-sourced digital assistant network is described. At step  510 , a server device, such as digital assistant server  310  of  FIG. 3 , can receive a command transmitted from a digital assistant device, such as digital assistant device  210  of  FIG. 2 . The command can include, among other things, an alphanumeric text representation of a spoken command that was detected by the digital assistant device. To this end, the digital assistant device can employ a speech-to-text engine or 3 rd -party service to convert the detected spoken command to alphanumeric text before it is communicated to the server device. 
     At step  520 , the server device can determine, among other things, that the received command corresponds to an action dataset that is stored by the server device. The action dataset can be one of many action datasets stored by the server device, and can be stored in a local memory or a database, such as database  140  of  FIG. 1 , that is in communication with the server device. A stored action dataset can be generated by any computing device, such as a remote computing device or another digital assistant device, which can generate the action dataset and communicate it to the server device for storage, indexing, and maintenance thereof. An action dataset can include, among other things, one or more command templates that includes keywords and/or parameter fields, which can be employed by the server device to determine that the received command corresponds to the action dataset. For instance, the keywords can be compared to the received command, and parameters corresponding to the parameter fields can be extracted from the received command. In some other instances, ranking and/or comparison algorithm(s), or other machine learning technologies can be employed to determine a likelihood that a command template of an action dataset corresponds to a received command. As also described herein, each action dataset can include, among other things, a set of instructions that is interpretable by a digital assistant device in accordance with the present disclosure, to reproduce a particular set of operations that can be unique to the action dataset. 
     At step  530 , the server device can communicate the action dataset determined to correspond to the received command to the digital assistant device. In some aspects, the received command or portions thereof (e.g., extracted parameters) can also be communicated back to the digital assistant device. In this regard, when the digital assistant device receives the action dataset from the server device responsive to the command having been sent to the server device, the digital assistant device can, at step  540 , reproduce the set of operations included in the received action dataset. In other words, the communication of the action dataset from the server device to the digital assistant device can cause the digital assistant device, based on its receipt of the action dataset, to reproduce the set of operations included in the action dataset. In this regard, the digital assistant device&#39;s interpretation of the included set of operations can reproduce or execute a desired result (e.g., function) that is defined by the set of operations of the action dataset. 
     In some aspects, the set of operations can be interpreted by the digital assistant device immediately upon receipt of the action dataset. In some other aspects, the set of operations can be interpreted after a prompt is displayed or audibly reproduced by the digital assistant device and an affirmative confirmation (e.g., selection, user input, or verbal acknowledgement) is received by the digital assistant device. As it is contemplated that in some situations, multiple action datasets can be determined by the server device to correspond to the received command, each of the determined corresponding action datasets can be communicated to the digital assistant device, such that a selection of a preferred operation can be received by the digital assistant device. In other words, the digital assistant device can present or provide for display a list of corresponding applications of each received action dataset, so that a user can provide a selection (e.g., touch input or audible input) of one of the received action datasets so that it&#39;s included set of operations can be interpreted by the digital assistant device. 
     Turning now to  FIG. 6 , an embodiment of a process flow or method  600  for distributing actionable operations (i.e., action datasets) for a crowd-sourced digital assistant network is described. At step  610 , a server device, such as digital assistant server  310  of  FIG. 3 , can receive an action dataset generated by a remote computing device, such as a computing device  900  described in accordance with  FIG. 9 . In some aspects, the remote computing device can be any digital assistant device, such as described in accordance with digital assistant device  210  of  FIG. 2 . The received action dataset can include, among other things, a set of instructions that is interpretable by any digital assistant device in accordance with embodiments described herein, and can further include one or more command templates having one or more defined keywords and/or parameter fields. In some aspects, each command template corresponds to a spoken command or a command representation that can be determined, by the server device or digital assistant device, that the spoken command or command representation corresponds to a particular (or several particular) action dataset(s). 
     At step  620 , the server device can receive a command from a digital assistant device, such as digital assistant device  210  of  FIG. 2 . For purposes of this description, the digital assistant device from which the command was received can be a different digital assistant that had generated and communicated the action dataset to the server device, though the possibility that a single digital assistant device can both generate, send to the server, and receive the same action dataset is considered within the purview of the present disclosure. The command can include, among other things, an alphanumeric text representation of a spoken command that was detected by the digital assistant device. To this end, the digital assistant device can employ a speech-to-text engine or 3 rd -party service to convert the detected spoken command to alphanumeric text before it is communicated to the server device. 
     At step  630 , the server device can determine, among other things, that the received command corresponds to an action dataset that is stored by the server device. The action dataset can be one of many action datasets stored by the server device, and can be stored in a local memory or a database, such as database  140  of  FIG. 1 , that is in communication with the server device. An action dataset can include, among other things, one or more command templates that includes keywords and/or parameter fields, which can be employed by the server device to determine that the received command corresponds to the action dataset. For instance, the keywords can be compared to the received command, and parameters corresponding to the parameter fields can be extracted from the received command In some other instances, ranking and/or comparison algorithm(s), or other machine learning technologies can be employed to determine a likelihood that a command template of an action dataset corresponds to a received command. As also described herein, each action dataset can include, among other things, a set of instructions that is interpretable by a digital assistant device in accordance with the present disclosure, to reproduce a particular set of operations that can be unique to the action dataset. The server device can communicate the action dataset determined to correspond to the received command to the digital assistant device. In some aspects, the received command or portions thereof (e.g., extracted parameters) can also be communicated back to the digital assistant device. In this regard, when the digital assistant device receives the action dataset from the server device responsive to the command having been sent to the server device, the digital assistant device can, at step  640 , reproduce the set of operations included in the received action dataset. In other words, the communication of the action dataset from the server device to the digital assistant device can cause the digital assistant device, based on its receipt of the action dataset, to reproduce the set of operations included in the action dataset. In this regard, the digital assistant device&#39;s interpretation of the included set of operations can reproduce or execute a desired result (e.g., function) that is defined by the set of operations of the action dataset. 
     In some aspects, the set of operations can be interpreted by the digital assistant device immediately upon receipt of the action dataset. In some other aspects, the set of operations can be interpreted after a prompt is displayed or audibly reproduced by the digital assistant device and an affirmative confirmation (e.g., selection, user input, or verbal acknowledgement) is received by the digital assistant device. As it is contemplated that in some situations, multiple action datasets can be determined by the server device to correspond to the received command, each of the determined corresponding action datasets can be communicated to the digital assistant device, such that a selection of a preferred operation can be received by the digital assistant device. In other words, the digital assistant device can present or provide for display a list of corresponding applications of each received action dataset, so that a user can provide a selection (e.g., touch input or audible input) of one of the received action datasets so that it&#39;s included set of operations can be interpreted by the digital assistant device. 
     Turning now to  FIG. 7 , an embodiment of a process flow or method  700  for  FIG. 7  depicts an overview of the process to disambiguate a received command representation for determining, interpreting, and/or executing a most relevant action dataset and associated operations in accordance with various embodiments described herein. At step  705 , The command representation (depicted as “command”) can be obtained by a digital assistant device, such as digital assistant device  210  of  FIG. 2 . In some embodiments, the digital assistant device can obtain the command representation through speech received via a microphone thereof, recorded as speech data, and converted to an alphanumeric text representation via a speech-to-text conversion engine. In some other embodiments, the speech data or command representation can be received via another computing device in coupled communication (e.g., via a wired, wireless, or network connection) with the digital assistant device. The digital assistant device can determine  710  whether the command representation  710  matches or corresponds to a known (e.g., stored, available, indexed) command template associated with one or more action datasets, as described herein. A match or correspondence can be determined based on, among other things, a determined threshold similarity or calculated confidence level for similarity between the obtained command representation  710  and one or more command templates, each command template being associated with an action dataset, such as described in accordance with  FIG. 4 , stored in a repository of action datasets stored on the digital assistant device or digital assistant server. In some aspects, a match or correspondence can be determined based on a highest calculated correlation between the command representation and a command template of one or more action datasets. Command templates can include keywords and/or parameter fields that can correspond to an obtained command representation, either of which can be defined in a variety of manners. For example, a user can, via a GUI of an associated digital assistant device, provide user-specified commands  715  (e.g., keywords, parameters). That is, the user may record a step or sequence of steps associated with a desired result (e.g., an action) and define therewith a user-specified command  715 . As another example, a creator of a device, app, website, or the like, can provide creator-specified commands  720  that are accessible to the digital assistant device. For example, the creator of a weather application could provide a creator-specified command  720  to “get 10-day forecast,” which would cause the digital assistant device to open the weather app, determine the current location of the user&#39;s device, and navigate to a 10-day forecast for the current location of the user&#39;s device. As another example, the digital assistant device can retrieve cloud-sourced commands  725  (i.e., command templates), such as those associated with action datasets received from digital assistant server  310  of  FIG. 3 . User-specified commands  715  (i.e., command templates), creator-specified commands  720  (i.e., command templates), and any other known commands (i.e., command templates) can be aggregated, for example, on a remote server or servers, such as the digital assistant server. By accessing a pool (e.g., stored repository) of command templates utilized by other users or creators, the digital assistant device can obtain and interpret, with a high level of confidence, a command representation that has never been received by the digital assistant device from its associated user, based on how the same or similar command representation has been utilized by others via their respective digital assistant devices. In another example, the digital assistant device can receive unambiguous commands  730  (i.e., command representations). For example, a digital assistant device receiving the command representation “open Q App” can recognize “open” as a request to open an application on the digital assistant device and can recognize “Q App” as an application available (e.g., installed) on the digital assistant device (e.g., by comparing a parameter corresponding to an application name to a list of or metadata extracted from applications installed on the device). 
     If the command representation is determined to correspond to a known command template and thus, a corresponding action dataset, the digital assistant device can execute  735  the action (e.g., interpret operations defined in the corresponding action dataset) to perform operations on the digital assistant device that are determined to correspond to the obtained command representation. If the command representation does not correspond a known command template, the digital assistant device can attempt to disambiguate  740  the command representation, as will be described in more detail in  FIG. 8 . 
     Turning now to  FIG. 8 , another embodiment of a process flow or method  800  for  FIG. 8  depicts an overview of the process  800  to disambiguate a received command  810  (i.e., command representation) for determining, interpreting, and/or executing a most relevant action dataset and associated operations in accordance with various embodiments described herein. After receiving  805  a command representation  805  and determining that it cannot be determined as relevant (e.g., correspond to within a defined threshold confidence level) to an action dataset or command templates thereof by digital assistant device, such as digital assistant device  210  of  FIG. 2 , or digital assistant server, such as digital assistant server  310  of  FIG. 3 , the digital assistant device or server can attempt to disambiguate the command with or without requesting additional input for the user. It should be understood that the steps shown in  FIG. 8  are in largely arbitrary order for the purpose of illustration, and the disambiguation techniques could be employed in a different order, simultaneously, or in sub-groups simultaneously. The digital assistant device or server can perform an analysis (e.g., confidence or probability determination) to see if the received command representation matches or corresponds to  410  two or more possible action datasets based on the command templates associated therewith. In some aspects, a command representation commonly utilized by particular user to invoke a particular operation(s) of an action dataset can be utilized by other users to invoke completely different operation(s) (i.e., of different action datasets). For example, User A may say “Play Simon” to have her digital assistant device play music by Paul Simon, while User B may say “Play Simon” to have her device play music by Carly Simon. In this case, the command representation can correspond to (e.g., has a threshold similarity to) at least two possible command templates of different action datasets, and the digital assistant device can inquire  815  (i.e., generate a query to search) for a local or user-specified action dataset to associate with the command representation. 
     In the depicted flow and by way of non-limiting example, the digital assistant device has only reached step  815  because there was no determined probable correlation between the command representation at  810 , however, at step  815 , the digital assistant device can access and employ stored profile information and/or device usage data (e.g., utilized features, applications, operations executed on the digital assistant device) to determine, for instance, a last time the digital assistant device played or received a command representation to play music generally, or play music by Paul Simon, by way of example. Based on a determined similarity to a prior received command representation, a most recently-obtained command representation, or a determined frequent or common command representation among recently-obtained command representations, the digital assistant device can determine a higher probability (e.g., confidence level) that the user likely intends operations associated with one action dataset over another possible action dataset for an obtained command representation. If so, the digital assistant device can execute  835  the action dataset based on the analysis of stored profile information and/or device usage data. In some embodiments, if necessary, the digital assistant device can modify the command representation to include one or more portions (e.g., parameters) extracted from profile information and/or device usage data, to select a determined most-relevant action dataset, or in further embodiments, generate a query with the modified command representation and once again search (e.g., locally or via digital assistant server) for matches or most-relevant action datasets. 
     If a determination is made that there is no local or user-specified command representation or command template within a defined threshold confidence level (e.g., equal or greater than 50%, 60%, 70%, 80%, or 90%), the digital assistant device can generate a query and search  820  locally stored data that would suggest a relevant action dataset or command template. For example, if the user has a play list that includes Carly Simon but not Paul Simon, the digital assistant device can determine that the user likely intends to play music by Carly Simon, and select and execute  840  that determined relevant action dataset. If there is no relevant local data (e.g., profile information or device usage data), or if local data does not resolve the ambiguity—for example, if the user&#39;s playlist includes both Carly Simon and Paul Simon, and both have been played recently—the digital assistant device can generate a query and generate a query to search  825  for a dominant cloud-sourced action dataset (i.e., a most frequently selected action dataset or action dataset determined relevant to the command representation, via digital assistant server) corresponding to the received command representation. For example, if cloud-sourced action datasets  825  associate Paul Simon with “Play Simon” more frequently than Carly Simon, the digital assistant device can determine that the dominant action dataset is one interpretable to play music by Paul Simon, to retrieve and execute  845  that dominant action dataset. 
     If the command representation does not correspond to  810  or is not determined relevant to multiple (e.g., two or more) action datasets, the digital assistant device or server can determine that the command representation is ambiguous (e.g., is missing a parameter). For example, if no action datasets correspond to “Play Simon,” the digital assistant device or server can be able to determine that “Play” is generally associated with playing music (or other form of media), but is unable to determine which music to play based on “Simon”. In this case, the command representation is missing a parameter—which song or artist (or other characteristics of media) to play. If a parameter is determined missing or necessary to disambiguate the command representation, the digital assistant device or server can go through a similar process as described in steps  815 ,  820  and  825 , to determine whether the parameter can be hypothesized with reasonable certainty from stored profile information, detected contextual information, or cloud-sourced dominance Otherwise, in some embodiments, the digital assistant device can present a GUI prompt to request  855  the parameter from the user. In some other embodiments, the digital assistant device can perform a default or determined most probable (e.g., having a highest probability or confidence level) relevant action dataset as possible, without the parameter. For example, if the user has a local playlist, the digital assistant device can open the local playlist, or if the user has a music-related application on the device, the digital assistant device can open the music-related app. In this instance, the digital assistant device would not be able to fully execute the action dataset by playing a particular song or music of a particular style or particular artist, but would follow the action dataset operations through until further steps could not be performed with the missing parameter action (e.g., selecting the desired album or artist and initiating music playback). 
     If a determination is made that an action dataset includes a command template that does not require a parameter, the digital assistant device can inquire (e.g., generate a query and search)  865  profile information or device usage data to determine whether the desired action dataset can be inferred from the profile information or usage data (e.g., locally or remotely-stored user or device-specific data). As a non-limiting example, if the command representation includes “Play Music,” the digital assistant device or server can check a list or metadata of applications installed on the device, determine that SPOTIFY and PANDORA are applications associated with characteristics for applications that play music, and, because PANDORA was last utilized by this digital assistant device, execute  870  or communicate to digital assistant device for execution a corresponding action dataset to open PANDORA and play music without specifying a particular station. In various aspects, if local data is not available or does not support an inference of which action dataset to select and interpret, the digital assistant device can attempt to facilitate communications with the digital assistant server by generating a query including the command representation, which in some aspects can include profile information and/or usage data, to match  875  the command representation to a cloud-sourced action dataset (i.e., a command template associated with one of many action datasets stored by digital assistant server). If a closely related or relevant action dataset is thus identified, whether within a confidence threshold level or having a highest probability of relevance, the digital assistant device can retrieve the action dataset from the digital assistant server and execute  880  the cloud-sourced action dataset. If no cloud-sourced action dataset is matched  875 , the digital assistant device can present determined probable action datasets for user selection  830 , for instance, if any probable action datasets were identified  885  by the digital assistant device or server. In some aspects, digital assistant device can alternatively present a GUI prompt to request  890  a new command representation (e.g., via spoken command or text input). In various embodiments, any determination of an action dataset having an associated command template that is determined by a digital assistant device or server as matching, most probable, dominant, or within a threshold confidence level of relevance to, a received command representation can be selected by one of the digital assistant device or server and interpreted so that the operations defined therein are executed on the digital assistant device as a response to a received command or obtained command representation. 
     Having described various embodiments of the invention, an exemplary computing environment suitable for implementing embodiments of the invention is now described. With reference to  FIG. 9 , an exemplary computing device is provided and referred to generally as computing device  900 . The computing device  900  is but one example of a suitable computing environment and is not intended to suggest any limitation as to the scope of use or functionality of the invention. Neither should the computing device  900  be interpreted as having any dependency or requirement relating to any one or combination of components illustrated. 
     Embodiments of the invention can be described in the general context of computer code or machine-useable instructions, including computer-useable or computer-executable instructions, such as program modules, being executed by a computer or other machine, such as a personal data assistant, a smartphone, a tablet PC, or other handheld device. Generally, program modules, including routines, programs, objects, components, data structures, and the like, refer to code that performs particular tasks or implements particular abstract data types. Embodiments of the invention can be practiced in a variety of system configurations, including handheld devices, consumer electronics, general-purpose computers, more specialty computing devices, etc. Embodiments of the invention can also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices. 
     With reference to  FIG. 9 , computing device  900  includes a bus  910  that directly or indirectly couples the following devices: memory  912 , one or more processors  914 , one or more presentation components  916 , one or more input/output (I/O) ports  918 , one or more I/O components  920 , and an illustrative power supply  922 . Bus  910  represents what can be one or more busses (such as an address bus, data bus, or combination thereof). Although the various blocks of  FIG. 9  are shown with lines for the sake of clarity, in reality, these blocks represent logical, not necessarily actual, components. For example, one can consider a presentation component such as a display device to be an I/O component. Also, processors have memory. The inventors hereof recognize that such is the nature of the art and reiterate that the diagram of  FIG. 9  is merely illustrative of an exemplary computing device that can be used in connection with one or more embodiments of the present invention. Distinction is not made between such categories as “workstation,” “server,” “laptop,” “handheld device,” etc., as all are contemplated within the scope of  FIG. 9  and with reference to “computing device.” 
     Computing device  900  typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computing device  900  and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media can comprise computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVDs) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device  900 . Computer storage media does not comprise signals per se. Communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above should also be included within the scope of computer-readable media. 
     Memory  912  includes computer storage media in the form of volatile and/or nonvolatile memory. The memory can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical-disc drives, etc. Computing device  900  includes one or more processors  914  that read data from various entities such as memory  912  or I/O components  920 . Presentation component(s)  916  presents data indications to a user or other device. Exemplary presentation components include a display device, speaker, printing component, vibrating component, and the like. 
     The I/O ports  918  allow computing device  900  to be logically coupled to other devices, including I/O components  920 , some of which can be built in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, etc. The I/O components  920  can provide a natural user interface (NUI) that processes air gestures, voice, or other physiological inputs generated by a user. In some instances, inputs can be transmitted to an appropriate network element for further processing. An NUI can implement any combination of speech recognition, touch and stylus recognition, facial recognition, biometric recognition, gesture recognition both on screen and adjacent to the screen, air gestures, head and eye tracking, and touch recognition associated with displays on the computing device  900 . The computing device  900  can be equipped with depth cameras, such as stereoscopic camera systems, infrared camera systems, RGB camera systems, and combinations of these, for gesture detection and recognition. Additionally, the computing device  900  can be equipped with accelerometers or gyroscopes that enable detection of motion. The output of the accelerometers or gyroscopes can be provided to the display of the computing device  900  to render immersive augmented reality or virtual reality. 
     Some embodiments of computing device  900  can include one or more radio(s)  924  (or similar wireless communication components). The radio  924  transmits and receives radio or wireless communications. The computing device  900  can be a wireless terminal adapted to receive communications and media over various wireless networks. Computing device  900  can communicate via wireless protocols, such as code division multiple access (“CDMA”), global system for mobiles (“GSM”), or time division multiple access (“TDMA”), as well as others, to communicate with other devices. The radio communications can be a short-range connection, a long-range connection, or a combination of both a short-range and a long-range wireless telecommunications connection. When we refer to “short” and “long” types of connections, we do not mean to refer to the spatial relation between two devices. Instead, we are generally referring to short range and long range as different categories, or types, of connections (i.e., a primary connection and a secondary connection). A short-range connection can include, by way of example and not limitation, a Wi-Fi® connection to a device (e.g., mobile hotspot) that provides access to a wireless communications network, such as a WLAN connection using the 802.11 protocol; a Bluetooth connection to another computing device is a second example of a short-range connection, or a near-field communication connection. A long-range connection can include a connection using, by way of example and not limitation, one or more of CDMA, GPRS, GSM, TDMA, and 802.16 protocols. 
     Many different arrangements of the various components depicted, as well as components not shown, are possible without departing from the scope of the claims below. Embodiments of the present invention have been described with the intent to be illustrative rather than restrictive. Alternative embodiments will become apparent to readers of this disclosure after and because of reading it. Alternative means of implementing the aforementioned can be completed without departing from the scope of the claims below. Certain features and sub-combinations are of utility and can be employed without reference to other features and sub-combinations and are contemplated within the scope of the claims.