Patent Publication Number: US-2022229994-A1

Title: Operational modeling and optimization system for a natural language understanding (nlu) framework

Description:
CROSS-REFERENCE 
     This application claims priority from and the benefit of U.S. Provisional Patent Application No. 63/140,117, entitled “OPERATIONAL MODELING AND OPTIMIZATION SYSTEM FOR A NATURAL LANGUAGE UNDERSTANDING (NLU) FRAMEWORK,” filed Jan. 21, 2021, which is herein incorporated by reference in its entirety for all purposes. 
    
    
     BACKGROUND 
     The present disclosure relates generally to the fields of natural language understanding (NLU) and artificial intelligence (AI), and more specifically, to a hybrid learning system for NLU. 
     This section is intended to introduce the reader to various aspects of art that may be related to various aspects of the present disclosure, which are described and/or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. Accordingly, it should be understood that these statements are to be read in this light, and not as admissions of prior art. 
     Cloud computing relates to the sharing of computing resources that are generally accessed via the Internet. In particular, a cloud computing infrastructure allows users, such as individuals and/or enterprises, to access a shared pool of computing resources, such as servers, storage devices, networks, applications, and/or other computing based services. By doing so, users are able to access computing resources on demand that are located at remote locations and these resources may be used to perform a variety computing functions (e.g., storing and/or processing large quantities of computing data). For enterprise and other organization users, cloud computing provides flexibility in accessing cloud computing resources without accruing large up-front costs, such as purchasing expensive network equipment or investing large amounts of time in establishing a private network infrastructure. Instead, by utilizing cloud computing resources, users are able redirect their resources to focus on their enterprise&#39;s core functions. 
     In modern communication networks, examples of cloud computing services a user may utilize include so-called infrastructure as a service (IaaS), software as a service (SaaS), and platform as a service (PaaS) technologies. IaaS is a model in which providers abstract away the complexity of hardware infrastructure and provide rapid, simplified provisioning of virtual servers and storage, giving enterprises access to computing capacity on demand. In such an approach, however, a user may be left to install and maintain platform components and applications. SaaS is a delivery model that provides software as a service rather than an end product. Instead of utilizing a local network or individual software installations, software is typically licensed on a subscription basis, hosted on a remote machine, and accessed by client customers as needed. For example, users are generally able to access a variety of enterprise and/or information technology (IT)-related software via a web browser. PaaS acts an extension of SaaS that goes beyond providing software services by offering customizability and expandability features to meet a user&#39;s needs. For example, PaaS can provide a cloud-based developmental platform for users to develop, modify, and/or customize applications and/or automating enterprise operations without maintaining network infrastructure and/or allocating computing resources normally associated with these functions. 
     Such a cloud computing service may host a virtual agent, such as a chat agent, that is designed to automatically respond to issues with the client instance based on natural language requests from a user of the client instance. For example, a user may provide a request to a virtual agent for assistance with a password issue, wherein the virtual agent is part of a Natural Language Processing (NLP) or Natural Language Understanding (NLU) system. NLP is a general area of computer science and AI that involves some form of processing of natural language input. Examples of areas addressed by NLP include language translation, speech generation, parse tree extraction, part-of-speech identification, and others. NLU is a sub-area of NLP that specifically focuses on understanding user utterances. Examples of areas addressed by NLU include question-answering (e.g., reading comprehension questions), article summarization, and others. For example, a NLU may use algorithms to reduce human language (e.g., spoken or written) into a set of known symbols for consumption by a downstream virtual agent. NLP is generally used to interpret free text for further analysis. Current approaches to NLP are typically based on deep learning, which is a type of AI that examines and uses patterns in data to improve the understanding of a program. 
     As such, it is presently recognized that there is a need to improve the ability of virtual agents to apply NLU techniques to properly derive meaning from complex natural language utterances. For example, it may be advantageous to create a virtual agent capable of comprehending complex language and executing contextually relevant requests, which could afford substantial advantages in terms of reduced operational cost and increased responsiveness to client issues. Additionally, it is recognized that it is advantageous for virtual agents to be customizable and adaptable to various communication channels and styles. 
     SUMMARY 
     A summary of certain embodiments disclosed herein is set forth below. It should be understood that these aspects are presented merely to provide the reader with a brief summary of these certain embodiments and that these aspects are not intended to limit the scope of this disclosure. Indeed, this disclosure may encompass a variety of aspects that may not be set forth below. 
     A NLU framework may include a number of complex subsystems, such as vocabulary management subsystems, data augmentation subsystems, written-modality prosody subsystems, and so forth, and each of these subsystems may include a number of components (e.g., plugins, rules, forms, thresholds, coefficients, weight values). It is presently recognized that it would be advantageous to be able to model the complex interactions of these subsystems and components in order to thoroughly understand the operation of the NLU framework and provide explainability as to why the NLU framework inferenced a user utterance in a given way. This enables a more transparent understanding model, which aids users to better understand and use the NLU framework and aids designers to improve the configuration and operation of the NLU framework. Additionally, certain components may be configured to apply particular numerical values (e.g., threshold values, coefficient values, weight values) that substantially impact the operation of the component during inference. As such, it can be challenging for a designer to determine the impact of adjusting the configuration of one or more of these components in terms of overall NLU performance (e.g., latency time, precision, recall) and computational performance (e.g., processing time, memory usage, storage usage) of the NLU framework. As such, it is presently recognized that it would be advantageous to have a facility within the NLU framework that enables the configurations of the various components of the NLU framework, as well as the NLU framework itself, to be optimized to provide a desired level of performance to a client. 
     With this in mind, the disclosed NLU framework includes a modeling and optimization system that enables enhanced understanding and explainability to the operation of the NLU framework during inference. In general, the modeling and optimization system leverages ML-based techniques and linguistic theory to enhance the explainability of the NLU framework, which improves a user or designer understanding of the operation of the components at various levels of the NLU framework. The modeling and optimization system enables the operation of one or more components of the NLU framework to be systematically enabled, disabled, or modified, and then evaluates the impact of these changes to the overall performance of the NLU framework to provide enhanced understanding of the operation of the NLU framework. 
     Using performance metrics collected for a number of different NLU framework configurations, the modeling and optimization system may determine which components have a positive and important impact during interference, as well as which combinations components are strongly related and dramatically impact the performance of the NLU framework. The modeling and optimization system may also use the performance metrics to determine, for each configuration of the NLU framework, which paths test utterances traversed through the NLU framework during inference, and use this to construct an utterance flow model. The utterance flow model can be used to determine, for a given utterance, which potential paths through the NLU framework the utterance may take during inference to enable enhanced explainability within the framework. The modeling and optimization system can be leveraged to identify components of the NLU framework that should be improved (e.g., components that negatively impact performance metrics), as well as what impact that improvement might have to the overall performance of the framework. Furthermore, the modeling and optimization system may be leveraged to evaluate new components that are being considered for inclusion in the NLU framework to determine how these components will interact with the existing components of the NLU framework, as well as how they will improve or degrade the overall performance of the framework. 
     Additionally, in certain embodiments, the modeling and optimization system learns or automatically determines optimized configurations for the NLU framework. For example, the modeling and optimization system may automatically determine which components of the NLU framework should be activated or deactivated to enable a desired level of NLU and/or computational performance. In certain embodiments, the modeling and optimization system may, additionally or alternatively, automatically determine numerical values (e.g., threshold values, coefficients, weight values) for components applied during utterance inference to enable a desired level of NLU and/or computational performance. The optimization process is highly parallelizable, and can be performed in a distributed manner for enhanced performance (e.g., reduced runtime). As such, the modeling and optimization system substantially improves understanding of the NLU framework for users and designers, improves the continued development of the framework, and improves the operation of the NLU framework in accordance with the NLU and computation performance needs of the client. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       Various aspects of this disclosure may be better understood upon reading the following detailed description and upon reference to the drawings in which: 
         FIG. 1  is a block diagram of an embodiment of a cloud computing system in which embodiments of the present technique may operate; 
         FIG. 2  is a block diagram of an embodiment of a multi-instance cloud architecture in which embodiments of the present technique may operate; 
         FIG. 3  is a block diagram of a computing device utilized in a computing system that may be present in  FIG. 1 or 2 , in accordance with aspects of the present technique; 
         FIG. 4A  is a schematic diagram illustrating an embodiment of an agent automation framework including a NLU framework that is part of a client instance hosted by the cloud computing system, in accordance with aspects of the present technique; 
         FIG. 4B  is a schematic diagram illustrating an alternative embodiment of the agent automation framework in which portions of the NLU framework are part of an enterprise instance hosted by the cloud computing system, in accordance with aspects of the present technique; 
         FIG. 5  is a flow diagram illustrating an embodiment of a process by which an agent automation framework, including an NLU framework and a Reasoning Agent/Behavior Engine (RA/BE) framework, extracts intent/entities from and responds to a user utterance, in accordance with aspects of the present technique; 
         FIG. 6  is a block diagram illustrating an embodiment of the NLU framework including a meaning extraction subsystem and a meaning search subsystem, wherein the meaning extraction subsystem generates meaning representations from a received user utterance to yield an utterance meaning model and generates meaning representations from sample utterances of an intent/entity model to yield understanding model, and wherein the meaning search subsystem compares meaning representations of the utterance meaning model to meaning representations of the understanding model to extract intents and entities from the received user utterance, in accordance with aspects of the present technique; 
         FIG. 7  is a block diagram illustrating an embodiment of the meaning extraction subsystem using a combination of rules-based methods and machine-learning (ML)-based methods within a vocabulary subsystem, a structure subsystem, and a prosody subsystem, to generate an annotated utterance tree for an utterance, in accordance with aspects of the present technique; 
         FIG. 8  is a flow diagram illustrating an example process by which the meaning extraction subsystem performs error correction of an annotated utterance tree of an utterance before generating the corresponding meaning representation of the utterance, in accordance with aspects of the present technique; 
         FIG. 9  is a flow diagram illustrating an example process by which the meaning extraction subsystem generates a meaning representations of the understanding model or the utterance meaning model based on the annotated utterance trees and a compilation model template, in accordance with aspects of the present technique; 
         FIG. 10  is a block diagram illustrating an embodiment of the compilation model template, in accordance with aspects of the present technique; 
         FIG. 11  is a block diagram illustrating example operation of an embodiment of a tree substructure vectorization algorithm to generate a combined subtree vector for a subtree of an annotated utterance tree, in accordance with aspects of the present technique; 
         FIG. 12  is a flow diagram illustrating example process by which the meaning search subsystem searches the meaning representations of the understanding model for matches to the meaning representation of the user utterance, in accordance with aspects of the present technique; 
         FIG. 13  is a flow diagram illustrating an embodiment of a process by which a tree-model comparison algorithm compares an intent subtree of a first meaning representation to an intent subtree of a second meaning representation, based on the compilation model template, to generate an intent subtree similarity score, in accordance with aspects of the present technique; 
         FIG. 14  is a block diagram illustrating an embodiment of a process by which the agent automation system continuously improves a structure learning model, such as a recurrent neural network associated with a ML-based parser of the NLU framework, for improved domain specificity, based on a collection of utterances, in accordance with aspects of the present technique; 
         FIG. 15  is a block diagram illustrating an embodiment of a process by which the agent automation system continuously learns new words and/or refines word understanding for improved domain specificity based on a collection of utterances, in accordance with aspects of the present technique; 
         FIG. 16  is a diagram illustrating an embodiment of an annotated utterance tree, in accordance with aspects of the present technique; and 
         FIG. 17  is a diagram illustrating an embodiment of a meaning representation, in accordance with aspects of the present technique. 
         FIG. 18  is a block diagram illustrating how the vocabulary subsystem of  FIG. 7  operates within the NLU framework, in accordance with aspects of the present techniques; 
         FIG. 19  is a diagram illustrating what may be included in a vocabulary model template, in accordance with aspects of the present techniques; 
         FIG. 20  is a block diagram illustrating example operation of an embodiment of a multi-vector aggregation algorithm to generate a combined sub-phrase vector for a subtree of an annotated utterance tree, in accordance with aspects of the present techniques; 
         FIG. 21  is a flow diagram illustrating how the agent automation framework continuously improves a word vector distribution model, which may be plugged into the vocabulary subsystem of the meaning extraction subsystem shown in  FIG. 7 , in accordance with aspects of the present techniques; and 
         FIG. 22  is a flow diagram illustrating a process for receiving the user utterance, determining which meanings of one or more words or phrases appearing in the utterance were intended, and outputting one or more associated semantic word vectors, in accordance with aspects of the present technique. 
         FIG. 23  is a flow diagram illustrating an example process by which a model augmentation subsystem of the NLU framework augments one or more models (e.g., the utterance meaning model, the understanding model, or a combination thereof) before performing a meaning search operation, in accordance with aspects of the present technique; 
         FIG. 24  is a diagram of an embodiment of a model augmentation template storing generalizing rule-sets, refining rule-sets, and model applicability criteria used by the model augmentation subsystem to augment the one or more models, in accordance with aspects of the present technique; 
         FIG. 25  is a diagram illustrating an embodiment of model augmentation in which meaning representations of the one or more models are generalized and/or refined to yield an augmented model, in accordance with aspects of the present technique; 
         FIG. 26  is a flow diagram illustrating an embodiment of a process whereby the model augmentation subsystem performs rule-based generalization of the meaning representations of the one or more models, in accordance with aspects of the present technique; and 
         FIG. 27  is a flow diagram illustrating an embodiment of a process whereby the model augmentation subsystem performs rule-based refinement of the meaning representations of the one or more models, in accordance with aspects of the present technique. 
         FIG. 28  is a flow diagram illustrating an embodiment of the prosody subsystem digesting conversation logs into a number of different outputs for consumption by various components of the NLU framework, in accordance with aspects of the present technique; 
         FIG. 29  is a block diagram illustrating an embodiment of a process by which the agent automation system continuously improves a structure learning model, such as a recurrent neural network associated with a ML-based parser of the NLU framework, for improved domain specificity, based on intent segments identified by the prosody subsystem, in accordance with aspects of the present technique; 
         FIG. 30  is a block diagram illustrating an embodiment of a process by which the agent automation system continuously learns new words and/or refines word understanding for improved domain specificity, based on in-context utterances identified by the prosody subsystem, in accordance with aspects of the present technique; 
         FIG. 31  is a block diagram depicting a high-level view of certain components of the agent automation framework, including a semantic mining framework, in accordance with aspects of the present technique; 
         FIG. 32  is a block diagram of a semantic mining pipeline of the semantic mining framework illustrating a number of processing steps of a semantic mining process, in accordance with aspects of the present technique; 
         FIG. 33  is a diagram illustrating the prosody subsystem supporting the RA/BE in segmenting episodic context information from conversation logs, in accordance with aspects of the present technique; 
         FIG. 34  is a flow diagram illustrating how a persona of the RA/BE uses the prosody subsystem to manage episodic context within the agent automation framework, in accordance with aspects of the present technique; 
         FIG. 35  is a block diagram illustrating an embodiment of a NLU framework that includes NLU system and a modeling and optimization system, in accordance with aspects of the present technique; 
         FIGS. 36, 37 and 38  are flow diagrams illustrating embodiments in which the modeling and optimization system provides test utterances to a NLU system of a NLU framework to process based on a particular configuration vector to determine respective performance metrics for each configuration, in accordance with aspects of the present technique; 
         FIG. 39  is a flow diagram illustrating an embodiment of an optimization process whereby the modeling and optimization system applies ML-based techniques to automatically determine optimized configurations for components of the NLU framework, in accordance with aspects of the present technique; and 
         FIG. 40  is a flow diagram illustrating an embodiment of a distributed optimization process whereby the modeling and optimization system optimizes subsets of a configuration vector in parallel, in accordance with aspects of the present technique. 
     
    
    
     DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS 
     One or more specific embodiments will be described below. In an effort to provide a concise description of these embodiments, not all features of an actual implementation are described in the specification. It should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers&#39; specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another. Moreover, it should be appreciated that such a development effort might be complex and time consuming, but would nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill having the benefit of this disclosure. 
     As used herein, the terms “application”, “engine”, “program”, or “plugin” refers to one or more sets of computer software instructions (e.g., computer programs and/or scripts) executable by one or more processors of a computing system to provide particular functionality. Computer software instructions can be written in any suitable programming languages, such as C, C++, C#, Pascal, Fortran, Perl, MATLAB, SAS, SPSS, JavaScript, AJAX, and JAVA. Such computer software instructions can comprise an independent application with data input and data display modules. Alternatively, the disclosed computer software instructions can be classes that are instantiated as distributed objects. The disclosed computer software instructions can also be component software, for example JAVABEANS or ENTERPRISE JAVABEANS. Additionally, the disclosed applications or engines can be implemented in computer software, computer hardware, or a combination thereof. 
     As used herein, the term “framework” refers to a system of applications and/or engines, as well as any other supporting data structures, libraries, modules, and any other supporting functionality, that cooperate to perform one or more overall functions. In particular, a “natural language understanding framework” or “NLU framework” comprises a collection of computer programs designed to process and derive meaning (e.g., intents, entities, artifacts) from natural language utterances using one or more machine-learning (ML) components and one or more rule-based components. As used herein, a “behavior engine” or “BE,” also known as a reasoning agent or RA/BE, refers to a rule-based agent, such as a virtual agent, designed to interact with users based on a conversation model. For example, a “virtual agent” may refer to a particular example of a BE that is designed to interact with users via natural language requests in a particular conversational or communication channel. With this in mind, the terms “virtual agent” and “BE” are used interchangeably herein. By way of specific examples, a virtual agent may be or include a chat agent that interacts with users via natural language requests and responses in a chat room environment, or that provides recommended answers to requests or queries made in a search text box. Other examples of virtual agents may include an email agent, a forum agent, a ticketing agent, a telephone call agent, a search agent, a genius search result agent, and so forth, which interact with users in the context of email, forum posts, search queries, autoreplies to service tickets, phone calls, and so forth. 
     As used herein, an “intent” refers to a desire or goal of a user which may relate to an underlying purpose of a communication, such as an utterance. As used herein, an “entity” refers to an object, subject, or some other parameterization of an intent. It is noted that, for present embodiments, certain entities are treated as parameters of a corresponding intent within an intent-entity model. More specifically, certain entities (e.g., time and location) may be globally recognized and extracted for all intents, while other entities are intent-specific (e.g., merchandise entities associated with purchase intents) and are generally extracted only when found within the intents that define them. As used herein, “artifact” collectively refers to both intents and entities of an utterance. As used herein, an “understanding model” is a collection of models used by the NLU framework to infer meaning of natural language utterances. An understanding model may include a vocabulary model that associates certain tokens (e.g., words or phrases) with particular word vectors, an intent-entity model, an intent model, an entity model, a taxonomy model, other models, or a combination thereof. As used herein an “intent-entity model” refers to a model that associates particular intents with particular entities and particular sample utterances, wherein entities associated with the intent may be encoded as a parameter of the intent within the sample utterances of the model. As used herein, the term “agents” may refer to computer-generated personas (e.g. chat agents or other virtual agents) that interact with human users within a conversational channel. As used herein, a “corpus” may refer to a captured body of source data that can include interactions between various users and virtual agents, wherein the interactions include communications or conversations within one or more suitable types of media (e.g., a help line, a chat room or message string, an email string). As used herein, an “utterance tree” refers to a data structure that stores a representation of the meaning of an utterance. As discussed, an utterance tree has a tree structure (e.g., a dependency parse tree structure) that represents the syntactic structure of the utterance, wherein nodes of the tree structure store vectors (e.g., word vectors, subtree vectors) that encode the semantic meaning of the utterance. 
     As used herein, an “utterance” refers to a single natural language statement made by a user that may include one or more intents. As such, an utterance may be part of a previously captured corpus of source data, and an utterance may also be a new statement received from a user as part of an interaction with a virtual agent. As used herein, “machine learning” or “ML” may be used to refer to any suitable statistical form of artificial intelligence capable of being trained using machine learning techniques, including supervised, unsupervised, and semi-supervised learning techniques. For example, in certain embodiments, ML-based techniques may be implemented using an artificial neural network (ANN) (e.g., a deep neural network (DNN), a recurrent neural network (RNN), a recursive neural network, a feedforward neural network). In contrast, “rules-based” methods and techniques refer to the use of rule-sets and ontologies (e.g., manually-crafted ontologies, statistically-derived ontologies) that enable precise adjudication of linguistic structure and semantic understanding to derive meaning representations from utterances. As used herein, a “vector” (e.g., a word vector, an intent vector, a subject vector, a subtree vector) refers to a linear algebra vector that is an ordered n-dimensional list (e.g., a 300 dimensional list) of floating point values (e.g., a 1×N or an N×1 matrix) that provides a mathematical representation of the semantic meaning of a portion (e.g., a word or phrase, an intent, an entity, a token) of an utterance. As used herein, “domain specificity” refers to how attuned a system is to correctly extracting intents and entities expressed in actual conversations in a given domain and/or conversational channel (e.g., a human resources domain, an information technology domain). As used herein, an “understanding” of an utterance refers to an interpretation or a construction of the utterance by the NLU framework. As such, it may be appreciated that different understandings of an utterance may be associated with different meaning representations having different parse structures (e.g., different nodes, different relationships between nodes), different part-of-speech taggings, and so forth. 
     Agent Automation Framework 
     Present embodiments are directed to an agent automation framework that is capable of extracting meaning from user utterances, such as requests received by a virtual agent (e.g., a chat agent), and suitably responding to these user utterances. To do this, the agent automation framework includes a NLU framework and an intent/entity model having defined intents and entities that are associated with sample utterances. The NLU framework includes a meaning extraction subsystem that is designed to generate meaning representations for the sample utterances of the intent/entity model, as well as a meaning representation for a received user utterance. Additionally, the disclosed NLU framework includes a meaning search subsystem that is designed to search the meaning representations of the intent/entity model to locate matches for a meaning representation of a received user utterance. As such, present embodiments generally address the hard problem posed by NLU by transforming it into a manageable search problem. 
     In present embodiments, a meaning representation can be generated from an annotated utterance tree structure having a form or shape that represents the grammatical structures of the utterance, and having nodes that each represent words or phrases of the utterances as word vectors encoding the semantic meaning of the utterance. The meaning extraction subsystem includes a vocabulary subsystem, a structure subsystem, and a prosody subsystem that cooperate to parse utterances into the annotated utterance trees based on combinations of rule-based methods and machine learning (ML)-based (e.g., statistical) methods. Using one or more tree substructure vectorization algorithms and focus/attention/magnification (FAM) coefficients defined by a stored compilation model template, the meaning extraction subsystem subsequently generates subtree vectors for the annotated utterance tree structure, yielding the corresponding meaning representation for subsequent searching by the meaning search subsystem. 
     The disclosed NLU framework is also capable of detecting and addressing errors in an annotated utterance tree before the meaning representation is generated. For example, the meaning extraction subsystem can include a rule-based augmentation error detection subsystem that can cooperate with the vocabulary, structure subsystem, and prosody subsystems to iteratively parse and correct an utterance before meaning representations are generated for improved domain specificity. Additionally, present embodiments support entrenchment, whereby the NLU framework can continue to learn or infer meaning of new syntactic structures in new natural language utterance based on previous examples of similar syntactic structures. For example, components of the NLU framework (e.g., the structure subsystem or the vocabulary subsystem of the meaning extraction subsystem) may be continuously updated based on new utterances, such as exchanges between users and a virtual agent, to enhance the adaptability of the NLU framework to changes in the use of certain terms and phrases over time. 
     The meaning search subsystem of the disclosed NLU framework is designed to compare a meaning representation generated for a received user utterance to the set of meaning representations generated for the sample utterances of the intent/entity model based on the compilation model template. For example, the compilation model template defines one or more tree model comparison algorithms designed to determine a similarity score for two subtree vectors based on class compatibility rules and class-level scoring coefficients stored in the compilation model template. The class compatibility rules define which classes of subtree vectors can be compared to one another (e.g., verb subtree vectors are compared to one another, subject subtree vectors are compared to one another) to determine vector distances between the subtrees of the meaning representations. The class-level scoring coefficients define different relative weights that determine how much the different classes of subtree vectors contribute to an overall vector generated by the substructure vectorization algorithm for a given subtree (e.g., verb subtree vectors and/or direct object subtree vectors may be weighted higher and contribute more than subject subtree vectors or modifier subtree vectors). Using these algorithms, rules, and coefficients of the compilation model template, the meaning search subsystem determines similarity scores between portions of the meaning representation of the user utterance and portions of the meaning representations of the sample utterances of the intent/entity model. Based on these similarity scores, intents/entities defined within the intent/entity model are extracted from the user utterance and passed to a reasoning agent/behavior engine (RA/BE), such as a virtual agent, to take appropriate action based on the extracted intents/entities of the user utterance. 
     As mentioned, a computing platform may include a chat agent, or another similar virtual agent, that is designed to automatically respond to user requests to perform functions or address issues on the platform. There are two predominant technologies in NLU, namely traditional computational linguistics and newer machine learning (ML) methods. It is presently recognized that these two technologies demonstrate different strengths and weaknesses with respect to NLU. For example, traditional computational linguistic methods, also referred to herein as “rule-based” methods, include precision rule-sets and manually-crafted ontologies that enable precise adjudication of linguistic structure and semantic understanding to derive meaning representations. Traditional cognitive linguistic techniques also include the concept of construction grammars, in which an aspect of the meaning of a natural language utterance can be determined based on the form (e.g., syntactic structure) of the utterance. Therefore, rule-based methods offer results that are easily explainable and customizable. However, it is presently recognized that such rule-based methods are not particularly robust to natural language variation or adept at adapting to language evolution. As such, it is recognized that rule-based methods alone are unable to effectively react to (e.g., adjust to, learn from) data-driven trends, such as learning from chat logs and other data repositories. Furthermore, rule-based methods involve the creation of hand-crafted rules that can be cumbersome, wherein these rules usually are domain specific and are not easily transferable to other domains. 
     On the other hand, ML-based methods, perform well (e.g., better than rule-based methods) when a large corpus of natural language data is available for analysis and training. The ML-based methods have the ability to automatically “learn” from the data presented to recall over “similar” input. Unlike rule-based methods, ML-based methods do not involve cumbersome hand-crafted features-engineering, and ML-based methods can support continued learning (e.g., entrenchment). However, it is recognized that ML-based methods struggle to be effective when the size of the corpus is insufficient. Additionally, ML-based methods are opaque (e.g., not easily explained) and are subject to biases in source data. Furthermore, while an exceedingly large corpus may be beneficial for ML training, source data may be subject to privacy considerations that run counter to the desired data aggregation. 
     Accordingly, present embodiments are generally directed toward an agent automation framework capable of applying a combination rule-based and ML-based cognitive linguistic techniques to leverage the strengths of both techniques in extracting meaning from natural language utterances. More specifically, present embodiments are directed to generating suitable meaning representations for utterances, including received user utterances and sample utterances of an intent/entity model. These meaning representations generally have a shape that captures the syntactic structure of an utterance, and include one or more subtree vectors that represent the semantic meanings of portions of the utterance. The meaning representation of the utterance can then be searched against a search space populated with the meaning representations of the sample utterances of the intent/entity model, and one or more matches may be identified. In this manner, present embodiments extract intents/entities from the user utterance, such that a virtual agent can suitably respond to these intent/entities. As such, present embodiments generally address the hard NLU problem by transforming it into a more manageable search problem. 
     With the preceding in mind, the following figures relate to various types of generalized system architectures or configurations that may be employed to provide services to an organization in a multi-instance framework and on which the present approaches may be employed. Correspondingly, these system and platform examples may also relate to systems and platforms on which the techniques discussed herein may be implemented or otherwise utilized. Turning now to  FIG. 1 , a schematic diagram of an embodiment of a computing system  10 , such as a cloud computing system, where embodiments of the present disclosure may operate, is illustrated. Computing system  10  may include a client network  12 , network  18  (e.g., the Internet), and a cloud-based platform  20 . In some implementations, the cloud-based platform may host a management database (CMDB) system and/or other suitable systems. In one embodiment, the client network  12  may be a local private network, such as a local area network (LAN) having a variety of network devices that include, but are not limited to, switches, servers, and routers. In another embodiment, the client network  12  represents an enterprise network that could include one or more LANs, virtual networks, data centers  22 , and/or other remote networks. As shown in  FIG. 1 , the client network  12  is able to connect to one or more client devices  14 A,  14 B, and  14 C so that the client devices are able to communicate with each other and/or with the network hosting the platform  20 . The client devices  14 A-C may be computing systems and/or other types of computing devices generally referred to as Internet of Things (IoT) devices that access cloud computing services, for example, via a web browser application or via an edge device  16  that may act as a gateway between the client devices and the platform  20 .  FIG. 1  also illustrates that the client network  12  includes an administration or managerial device or server, such as a management, instrumentation, and discovery (MID) server  17  that facilitates communication of data between the network hosting the platform  20 , other external applications, data sources, and services, and the client network  12 . Although not specifically illustrated in  FIG. 1 , the client network  12  may also include a connecting network device (e.g., a gateway or router) or a combination of devices that implement a customer firewall or intrusion protection system. 
     For the illustrated embodiment,  FIG. 1  illustrates that client network  12  is coupled to a network  18 . The network  18  may include one or more computing networks, such as other LANs, wide area networks (WAN), the Internet, and/or other remote networks, to transfer data between the client devices  14 A-C and the network hosting the platform  20 . Each of the computing networks within network  18  may contain wired and/or wireless programmable devices that operate in the electrical and/or optical domain. For example, network  18  may include wireless networks, such as cellular networks (e.g., Global System for Mobile Communications (GSM) based cellular network), IEEE 802.11 networks, and/or other suitable radio-based networks. The network  18  may also employ any number of network communication protocols, such as Transmission Control Protocol (TCP) and Internet Protocol (IP). Although not explicitly shown in  FIG. 1 , network  18  may include a variety of network devices, such as servers, routers, network switches, and/or other network hardware devices configured to transport data over the network  18 . 
     In  FIG. 1 , the network hosting the platform  20  may be a remote network (e.g., a cloud network) that is able to communicate with the client devices  14 A-C via the client network  12  and network  18 . The network hosting the platform  20  provides additional computing resources to the client devices  14 A-C and/or client network  12 . For example, by utilizing the network hosting the platform  20 , users of client devices  14 A-C are able to build and execute applications for various enterprise, IT, and/or other organization-related functions. In one embodiment, the network hosting the platform  20  is implemented on one or more data centers  22 , where each data center could correspond to a different geographic location. Each of the data centers  22  includes a plurality of virtual servers  24  (also referred to herein as application nodes, application servers, virtual server instances, application instances, or application server instances), where each virtual server can be implemented on a physical computing system, such as a single electronic computing device (e.g., a single physical hardware server) or across multiple-computing devices (e.g., multiple physical hardware servers). Examples of virtual servers  24  include, but are not limited to a web server (e.g., a unitary web server installation), an application server (e.g., unitary JAVA Virtual Machine), and/or a database server, e.g., a unitary relational database management system (RDBMS) catalog. 
     To utilize computing resources within the platform  20 , network operators may choose to configure the data centers  22  using a variety of computing infrastructures. In one embodiment, one or more of the data centers  22  are configured using a multi-tenant cloud architecture, such that one of the server instances  24  handles requests from and serves multiple customers. Data centers with multi-tenant cloud architecture commingle and store data from multiple customers, where multiple customer instances are assigned to one of the virtual servers  24 . In a multi-tenant cloud architecture, the particular virtual server  24  distinguishes between and segregates data and other information of the various customers. For example, a multi-tenant cloud architecture could assign a particular identifier for each customer in order to identify and segregate the data from each customer. Generally, implementing a multi-tenant cloud architecture may suffer from various drawbacks, such as a failure of a particular one of the server instances  24  causing outages for all customers allocated to the particular server instance. 
     In another embodiment, one or more of the data centers  22  are configured using a multi-instance cloud architecture to provide every customer its own unique customer instance or instances. For example, a multi-instance cloud architecture could provide each customer instance with its own dedicated application server(s) and dedicated database server(s). In other examples, the multi-instance cloud architecture could deploy a single physical or virtual server and/or other combinations of physical and/or virtual servers  24 , such as one or more dedicated web servers, one or more dedicated application servers, and one or more database servers, for each customer instance. In a multi-instance cloud architecture, multiple customer instances could be installed on one or more respective hardware servers, where each customer instance is allocated certain portions of the physical server resources, such as computing memory, storage, and processing power. By doing so, each customer instance has its own unique software stack that provides the benefit of data isolation, relatively less downtime for customers to access the platform  20 , and customer-driven upgrade schedules. An example of implementing a customer instance within a multi-instance cloud architecture will be discussed in more detail below with reference to  FIG. 2 . 
       FIG. 2  is a schematic diagram of an embodiment of a multi-instance cloud architecture  40  where embodiments of the present disclosure may operate.  FIG. 2  illustrates that the multi-instance cloud architecture  40  includes the client network  12  and the network  18  that connect to two (e.g., paired) data centers  22 A and  22 B that may be geographically separated from one another. Using  FIG. 2  as an example, network environment and service provider cloud infrastructure client instance  42  (also referred to herein as a simply client instance  42 ) is associated with (e.g., supported and enabled by) dedicated virtual servers (e.g., virtual servers  24 A,  24 B,  24 C, and  24 D) and dedicated database servers (e.g., virtual database servers  44 A and  44 B). Stated another way, the virtual servers  24 A- 24 D and virtual database servers  44 A and  44 B are not shared with other client instances and are specific to the respective client instance  42 . Other embodiments of the multi-instance cloud architecture  40  could include other types of dedicated virtual servers, such as a web server. For example, the client instance  42  could be associated with (e.g., supported and enabled by) the dedicated virtual servers  24 A- 24 D, dedicated virtual database servers  44 A and  44 B, and additional dedicated virtual web servers (not shown in  FIG. 2 ). 
     In the depicted example, to facilitate availability of the client instance  42 , the virtual servers  24 A- 24 D and virtual database servers  44 A and  44 B are allocated to two different data centers  22 A and  22 B, where one of the data centers  22  acts as a backup data center. In reference to  FIG. 2 , data center  22 A acts as a primary data center that includes a primary pair of virtual servers  24 A and  24 B and the primary virtual database server  44 A associated with the client instance  42 . Data center  22 B acts as a secondary data center  22 B to back up the primary data center  22 A for the client instance  42 . To back up the primary data center  22 A for the client instance  42 , the secondary data center  22 B includes a secondary pair of virtual servers  24 C and  24 D and a secondary virtual database server  44 B. The primary virtual database server  44 A is able to replicate data to the secondary virtual database server  44 B (e.g., via the network  18 ). 
     As shown in  FIG. 2 , the primary virtual database server  44 A may back up data to the secondary virtual database server  44 B using a database replication operation. The replication of data between data centers could be implemented by performing full backups weekly and daily incremental backups in both data centers  22 A and  22 B. Having both a primary data center  22 A and secondary data center  22 B allows data traffic that typically travels to the primary data center  22 A for the client instance  42  to be diverted to the secondary data center  22 B during a failure and/or maintenance scenario. Using  FIG. 2  as an example, if the virtual servers  24 A and  24 B and/or primary virtual database server instance  44 A fails and/or is under maintenance, data traffic for client instances  42  can be diverted to the secondary virtual servers  24 C and/or  24 D and the secondary virtual database server instance  44 B for processing. 
     Although  FIGS. 1 and 2  illustrate specific embodiments of a cloud computing system  10  and a multi-instance cloud architecture  40 , respectively, the disclosure is not limited to the specific embodiments illustrated in  FIGS. 1 and 2 . For instance, although  FIG. 1  illustrates that the platform  20  is implemented using data centers, other embodiments of the platform  20  are not limited to data centers and can utilize other types of remote network infrastructures. Moreover, other embodiments of the present disclosure may combine one or more different virtual servers into a single virtual server or, conversely, perform operations attributed to a single virtual server using multiple virtual servers. For instance, using  FIG. 2  as an example, the virtual servers  24 A-D and virtual database servers  44 A and  44 B may be combined into a single virtual server. Moreover, the present approaches may be implemented in other architectures or configurations, including, but not limited to, multi-tenant architectures, generalized client/server implementations, and/or even on a single physical processor-based device configured to perform some or all of the operations discussed herein. Similarly, though virtual servers or machines may be referenced to facilitate discussion of an implementation, physical servers may instead be employed as appropriate. The use and discussion of  FIGS. 1 and 2  are only examples to facilitate ease of description and explanation and are not intended to limit the disclosure to the specific examples illustrated therein. 
     As may be appreciated, the respective architectures and frameworks discussed with respect to  FIGS. 1 and 2  incorporate computing systems of various types (e.g., servers, workstations, client devices, laptops, tablet computers, cellular telephones, and so forth) throughout. For the sake of completeness, a brief, high level overview of components typically found in such systems is provided. As may be appreciated, the present overview is intended to merely provide a high-level, generalized view of components typical in such computing systems and should not be viewed as limiting in terms of components discussed or omitted from discussion. 
     With this in mind, and by way of background, it may be appreciated that the present approach may be implemented using one or more processor-based systems such as shown in  FIG. 3 . Likewise, applications and/or databases utilized in the present approach may be stored, employed, and/or maintained on such processor-based systems. As may be appreciated, such systems as shown in  FIG. 3  may be present in a distributed computing environment, a networked environment, or other multi-computer platform or architecture. Likewise, systems such as that shown in  FIG. 3 , may be used in supporting or communicating with one or more virtual environments or computational instances on which the present approach may be implemented. 
     With this in mind, an example computer system may include some or all of the computer components depicted in  FIG. 3 .  FIG. 3  generally illustrates a block diagram of example components of a computing system  80  and their potential interconnections or communication paths, such as along one or more busses. As illustrated, the computing system  80  may include various hardware components such as, but not limited to, one or more processors  82 , one or more busses  84 , memory  86 , input devices  88 , a power source  90 , a network interface  92 , a user interface  94 , and/or other computer components useful in performing the functions described herein. 
     The one or more processors  82  may include one or more microprocessors capable of performing instructions stored in the memory  86 . Additionally or alternatively, the one or more processors  82  may include application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and/or other devices designed to perform some or all of the functions discussed herein without calling instructions from the memory  86 . 
     With respect to other components, the one or more busses  84  include suitable electrical channels to provide data and/or power between the various components of the computing system  80 . The memory  86  may include any tangible, non-transitory, and computer-readable storage media. Although shown as a single block in  FIG. 3 , the memory  86  can be implemented using multiple physical units of the same or different types in one or more physical locations. The input devices  88  correspond to structures to input data and/or commands to the one or more processors  82 . For example, the input devices  88  may include a mouse, touchpad, touchscreen, keyboard and the like. The power source  90  can be any suitable source for power of the various components of the computing system  80 , such as line power and/or a battery source. The network interface  92  includes one or more transceivers capable of communicating with other devices over one or more networks (e.g., a communication channel). The network interface  92  may provide a wired network interface or a wireless network interface. A user interface  94  may include a display that is configured to display text or images transferred to it from the one or more processors  82 . In addition and/or alternative to the display, the user interface  94  may include other devices for interfacing with a user, such as lights (e.g., LEDs), speakers, and the like. 
     It should be appreciated that the cloud-based platform  20  discussed above provides an example architecture that may utilize NLU technologies. In particular, the cloud-based platform  20  may include or store a large corpus of source data that can be mined, to facilitate the generation of a number of outputs, including an intent/entity model. For example, the cloud-based platform  20  may include ticketing source data having requests for changes or repairs to particular systems, dialog between the requester and a service technician or an administrator attempting to address an issue, a description of how the ticket was eventually resolved, and so forth. Then, the generated intent/entity model can serve as a basis for classifying intents in future requests, and can be used to generate and improve a conversational model to support a virtual agent that can automatically address future issues within the cloud-based platform  20  based on natural language requests from users. As such, in certain embodiments described herein, the disclosed agent automation framework is incorporated into the cloud-based platform  20 , while in other embodiments, the agent automation framework may be hosted and executed (separately from the cloud-based platform  20 ) by a suitable system that is communicatively coupled to the cloud-based platform  20  to process utterances, as discussed below. 
     With the foregoing in mind,  FIG. 4A  illustrates an agent automation framework  100  (also referred to herein as an agent automation system  100 ) associated with a client instance  42 , in accordance with embodiments of the present technique. More specifically,  FIG. 4A  illustrates an example of a portion of a service provider cloud infrastructure, including the cloud-based platform  20  discussed above. The cloud-based platform  20  is connected to a client device  14 D via the network  18  to provide a user interface to network applications executing within the client instance  42  (e.g., via a web browser of the client device  14 D). Client instance  42  is supported by virtual servers similar to those explained with respect to  FIG. 2 , and is illustrated here to show support for the disclosed functionality described herein within the client instance  42 . The cloud provider infrastructure is generally configured to support a plurality of end-user devices, such as client device  14 D, concurrently, wherein each end-user device is in communication with the single client instance  42 . Also, the cloud provider infrastructure may be configured to support any number of client instances, such as client instance  42 , concurrently, with each of the instances in communication with one or more end-user devices. As mentioned above, an end-user may also interface with client instance  42  using an application that is executed within a web browser. 
     The embodiment of the agent automation framework  100  illustrated in  FIG. 4A  includes a reasoning agent/behavior engine (RA/BE)  102 , a NLU framework  104 , and a database  106 , which are communicatively coupled within the client instance  42 . The RA/BE  102  may host or include any suitable number of virtual agents or personas that interact with the user of the client device  14 D via natural language user requests  122  (also referred to herein as user utterances  122 ) and agent responses  124  (also referred to herein as agent utterances  124  or agent confirmations  124 ). It may be noted that, in actual implementations, the agent automation framework  100  may include a number of other suitable components, including the meaning extraction subsystem, the meaning search subsystem, and so forth, in accordance with the present disclosure. 
     For the embodiment illustrated in  FIG. 4A , the database  106  may be a database server instance (e.g., database server instance  44 A or  44 B, as discussed with respect to  FIG. 2 ), or a collection of database server instances. The illustrated database  106  stores an intent/entity model  108 , a conversation model  110 , a corpus of utterances  112 , and a collection of rules  114  in one or more tables (e.g., relational database tables) of the database  106 . The intent/entity model  108  stores associations or relationships between particular intents and particular sample utterances. In certain embodiments, the intent/entity model  108  may be authored by a designer using a suitable authoring tool. However, it should be noted that such intent/entity models typically include a limited number of sample utterances provided by the designer. Additionally, designers may have limited linguistic knowledge and, furthermore, are constrained from reasonably providing a comprehensive list of all possible ways of specifying intents in a domain. It is also presently recognized that, since the meaning associated with various intents and entities is continuously evolving within different contexts (e.g., different language evolutions per domain, per cultural setting, per client, and so forth), authored intent/entity models generally are manually updated over time. As such, it is recognized that authored intent/entity models are limited by the time and ability of the designer, and as such, these human-generated intent/entity models can be limited in both scope and functionality. 
     With this in mind, in certain embodiments, the intent/entity model  108  may instead be generated from the corpus of utterances  112  using techniques described in the commonly assigned, co-pending U.S. patent application Ser. No. 16/179,681, entitled, “METHOD AND SYSTEM FOR AUTOMATED INTENT MINING, CLASSIFICATION AND DISPOSITION,” which is incorporated by reference herein in its entirety for all purposes. More specifically, the intent/entity model  108  may be generated based on the corpus of utterances  112  and the collection of rules  114  stored in one or more tables of the database  106 . It may be appreciated that the corpus of utterances  112  may include source data collected with respect to a particular context, such as chat logs between users and a help desk technician within a particular enterprise, from a particular group of users, communications collected from a particular window of time, and so forth. As such, the corpus of utterances  112  enable the agent automation framework  100  to build an understanding of intents and entities that appropriately correspond with the terminology and diction that may be particular to certain contexts and/or technical fields, as discussed in greater detail below. 
     For the embodiment illustrated in  FIG. 4A , the conversation model  110  stores associations between intents of the intent/entity model  108  and particular responses and/or actions, which generally define the behavior of the RA/BE  102 . In certain embodiments, at least a portion of the associations within the conversation model are manually created or predefined by a designer of the RA/BE  102  based on how the designer wants the RA/BE  102  to respond to particular identified intents/entities in processed utterances. It should be noted that, in different embodiments, the database  106  may include other database tables storing other information related to intent classification, such as tables storing information regarding compilation model template data (e.g., class compatibility rules, class-level scoring coefficients, tree-model comparison algorithms, tree substructure vectorization algorithms), meaning representations, and so forth, in accordance with the present disclosure. 
     For the illustrated embodiment, the NLU framework  104  includes a NLU engine  116  and a vocabulary manager  118  (also referred to herein as a vocabulary subsystem). It may be appreciated that the NLU framework  104  may include any suitable number of other components. In certain embodiments, the NLU engine  116  is designed to perform a number of functions of the NLU framework  104 , including generating word vectors (e.g., intent vectors, subject or entity vectors, subtree vectors) from word or phrases of utterances, as well as determining distances (e.g., Euclidean distances) between these vectors. For example, the NLU engine  116  is generally capable of producing a respective intent vector for each intent of an analyzed utterance. As such, a similarity measure or distance between two different utterances can be calculated using the respective intent vectors produced by the NLU engine  116  for the two intents, wherein the similarity measure provides an indication of similarity in meaning between the two intents. 
     The vocabulary manager  118 , which may be part of the vocabulary subsystem discussed below, addresses out-of-vocabulary words and symbols that were not encountered by the NLU framework  104  during vocabulary training. For example, in certain embodiments, the vocabulary manager  118  can identify and replace synonyms and domain-specific meanings of words and acronyms within utterances analyzed by the agent automation framework  100  (e.g., based on the collection of rules  114 ), which can improve the performance of the NLU framework  104  to properly identify intents and entities within context-specific utterances. Additionally, to accommodate the tendency of natural language to adopt new usages for pre-existing words, in certain embodiments, the vocabulary manager  118  handles repurposing of words previously associated with other intents or entities based on a change in context. For example, the vocabulary manager  118  could handle a situation in which, in the context of utterances from a particular client instance and/or conversation channel, the word “bike” actually refers to a motorcycle rather than a bicycle. 
     Once the intent/entity model  108  and the conversation model  110  have been created, the agent automation framework  100  is designed to receive a user utterance  122  (in the form of a natural language request) and to appropriately take action to address the request. For example, for the embodiment illustrated in  FIG. 4A , the RA/BE  102  is a virtual agent that receives, via the network  18 , the utterance  122  (e.g., a natural language request in a chat communication) submitted by the client device  14 D disposed on the client network  12 . The RA/BE  102  provides the utterance  122  to the NLU framework  104 , and the NLU engine  116 , along with the various subsystems of the NLU framework  104  discussed below, processes the utterance  122  based on the intent/entity model  108  to derive intents/entities within the utterance  122 . Based on the intents/entities derived by the NLU engine  116 , as well as the associations within the conversation model  110 , the RA/BE  102  performs one or more particular predefined actions. For the illustrated embodiment, the RA/BE  102  also provides a response  124  (e.g., a virtual agent utterance or confirmation) to the client device  14 D via the network  18 , for example, indicating actions performed by the RA/BE  102  in response to the received user utterance  122 . Additionally, in certain embodiments, the utterance  122  may be added to the utterances  112  stored in the database  106  for continued learning within the NLU framework  104 , as discussed below. 
     It may be appreciated that, in other embodiments, one or more components of the agent automation framework  100  and/or the NLU framework  104  may be otherwise arranged, situated, or hosted for improved performance. For example, in certain embodiments, one or more portions of the NLU framework  104  may be hosted by an instance (e.g., a shared instance, an enterprise instance) that is separate from, and communicatively coupled to, the client instance  42 . It is presently recognized that such embodiments can advantageously reduce the size of the client instance  42 , improving the efficiency of the cloud-based platform  20 . In particular, in certain embodiments, one or more components of the semantic mining framework discussed below may be hosted by a separate instance (e.g., an enterprise instance) that is communicatively coupled to the client instance  42 , as well as other client instances, to enable semantic intent mining and generation of the intent/entity model  108 . 
     With the foregoing in mind,  FIG. 4B  illustrates an alternative embodiment of the agent automation framework  100  in which portions of the NLU framework  104  are instead executed by a separate, shared instance (e.g., enterprise instance  125 ) that is hosted by the cloud-based platform system  20 . The illustrated enterprise instance  125  is communicatively coupled to exchange data related to intent/entity mining and classification with any suitable number of client instances via a suitable protocol (e.g., via suitable Representational State Transfer (REST) requests/responses). As such, for the design illustrated in  FIG. 4B , by hosting a portion of the NLU framework  104  as a shared resource accessible to multiple client instances  42 , the size of the client instance  42  can be substantially reduced (e.g., compared to the embodiment of the agent automation framework  100  illustrated in  FIG. 4A ) and the overall efficiency of the agent automation framework  100  can be improved. 
     In particular, the NLU framework  104  illustrated in  FIG. 4B  is divided into three distinct components that perform different aspects of semantic mining and intent classification within the NLU framework  104 . These components include: a shared NLU trainer  126  hosted by the enterprise instance  125 , a shared NLU annotator  127  hosted by the enterprise instance  125 , and a NLU predictor  128  hosted by the client instance  42 . It may be appreciated that the organizations illustrated in  FIGS. 4A and 4B  are merely examples, and in other embodiments, other organizations of the NLU framework  104  and/or the agent automation framework  100  may be used, in accordance with the present disclosure. 
     For the embodiment of the agent automation framework  100  illustrated in  FIG. 4B , the shared NLU trainer  126  is designed to receive the corpus of utterances  112  from the client instance  42 , and to perform semantic mining (e.g., including semantic parsing, grammar engineering, and so forth) to facilitate generation of the intent/entity model  108 . Once the intent/entity model  108  has been generated, when the RA/BE  102  receives the user utterance  122  provided by the client device  14 D, the NLU predictor  128  passes the utterance  122  and the intent/entity model  108  to the shared NLU annotator  127  for parsing and annotation of the utterance  122 . The shared NLU annotator  127  performs semantic parsing, grammar engineering, and so forth, of the utterance  122  based on the intent/entity model  108  and returns annotated utterance trees of the utterance  122  to the NLU predictor  128  of client instance  42 . The NLU predictor  128  then uses these annotated structures of the utterance  122 , discussed below in greater detail, to identify matching intents from the intent/entity model  108 , such that the RA/BE  102  can perform one or more actions based on the identified intents. It may be appreciated that the shared NLU annotator  127  may correspond to the meaning extraction subsystem, and the NLU predictor may correspond to the meaning search subsystem, of the NLU framework  104 , as discussed below. 
       FIG. 5  is a flow diagram depicting the roles of the reasoning agent/behavior engine (RA/BE)  102  and NLU framework  104  within an embodiment of the agent automation framework  100 . For the illustrated embodiment, the NLU framework  104  processes a received user utterance  122  to extract intents/entities  140  based on the intent/entity model  108 . The extracted intents/entities  140  may be implemented as a collection of symbols that represent intents and entities of the user utterance  122  in a form that is consumable by the RA/BE  102 . As such, these extracted intents/entities  140  are provided to the RA/BE  102 , which processes the received intents/entities  140  based on the conversation model  110  to determine suitable actions  142  (e.g., changing a password, creating a record, purchasing an item, closing an account) and/or virtual agent utterances  124  in response to the received user utterance  122 . As indicated by the arrow  144 , the process  145  can continuously repeat as the agent automation framework  100  receives and addresses additional user utterances  122  from the same user and/or other users in a conversational format. 
     As illustrated in  FIG. 5 , it may be appreciated that, in certain situations, no further action or communications may occur once the suitable actions  142  have been performed. Additionally, it should be noted that, while the user utterance  122  and the agent utterance  124  are discussed herein as being conveyed using a written conversational medium or channel (e.g., chat, email, ticketing system, text messages, forum posts), in other embodiments, voice-to-text and/or text-to-voice modules or plugins could be included to translate spoken user utterance  122  into text and/or translate text-based agent utterance  124  into speech to enable a voice interactive system, in accordance with the present disclosure. Furthermore, in certain embodiments, both the user utterance  122  and the virtual agent utterance  124  may be stored in the database  106  (e.g., in the corpus of utterances  112 ) to enable continued learning of new structure and vocabulary within the agent automation framework  100 . 
     As mentioned, the NLU framework  104  includes two primary subsystems that cooperate to convert the hard problem of NLU into a manageable search problem—namely: a meaning extraction subsystem and a meaning search subsystem. For example,  FIG. 6  is a block diagram illustrating roles of the meaning extraction subsystem  150  and the meaning search subsystem  152  of the NLU framework  104  within an embodiment of the agent automation framework  100 . For the illustrated embodiment, the right-hand portion  154  of  FIG. 6  illustrates the meaning extraction subsystem  150  of the NLU framework  104  receiving the intent/entity model  108 , which includes sample utterances  155  for each of the various intents/entities of the model. The meaning extraction subsystem  150  generates an understanding model  157  that includes meaning representations  158  of the sample utterances  155  of the intent/entity model  108 . In other words, the understanding model  157  is a translated or augmented version of the intent/entity model  108  that includes meaning representations  158  to enable searching (e.g., comparison and matching) by the meaning search subsystem  152 , as discussed below. As such, it may be appreciated that the right-hand portion  154  of  FIG. 6  is generally performed in advance of receiving the user utterance  122 , such as on a routine, scheduled basis or in response to updates to the intent/entity model  108 . 
     For the embodiment illustrated in  FIG. 6 , the left-hand portion  156  illustrates the meaning extraction subsystem  150  also receiving and processing the user utterance  122  to generate an utterance meaning model  160  having at least one meaning representation  162 . As discussed in greater detail below, these meaning representations  158  and  162  are data structures having a form that captures the grammatical, syntactic structure of an utterance, wherein subtrees of the data structures include subtree vectors that encode the semantic meanings of portions of the utterance. As such, for a given utterance, a corresponding meaning representation captures both syntactic and semantic meaning in a common meaning representation format that enables searching, comparison, and matching by the meaning search subsystem  152 , as discussed in greater detail below. Accordingly, the meaning representations  162  of the utterance meaning model  160  can be generally thought of like a search key, while the meaning representations  158  of the understanding model  157  define a search space in which the search key can be sought. Accordingly, the meaning search subsystem  152  searches the meaning representations  158  of the understanding model  157  to locate one or more intents/entities that match the meaning representation  162  of the utterance meaning model  160  as discussed below, thereby generating the extracted intents/entities  140 . 
     The meaning extraction subsystem of  FIG. 6  itself includes a number of subsystems that cooperate to generate the meaning representations  158  and  162 . For example,  FIG. 7  is a block diagram illustrating an embodiment of the meaning extraction subsystem  150  of the NLU framework  104  of the agent automation framework  100 . The illustrated embodiment of the meaning extraction subsystem  150  uses rules-based methods interleaved with ML-based methods to generate an annotated utterance tree  166  for an utterance  168 , which may be either a user utterance  122  or one of the sample utterances  155  of the intent/entity model  108 , as discussed above with respect to  FIG. 6 . More specifically,  FIG. 7  illustrates how embodiments of the meaning extraction subsystem  150  can include a number of best-of-breed models, including combinations of rule-based and ML-based (e.g., statistical) models and programs, that can be plugged into the overall NLU framework  104 . For example, because of the pluggable design of the illustrated meaning extraction subsystem  150 , the vocabulary subsystem  170  can include any suitable word vector distribution model that defines word vectors for various words or phrases. That is, since it is recognized that different word distribution models can excel over others in a given conversational channel, language, context, and so forth, the disclosed pluggable design enables the meaning extraction subsystem  150  to be customized to particular environments and applications. For the embodiment illustrated in  FIG. 7 , the meaning extraction subsystem  150  includes three plugin-supported subsystems, namely a vocabulary subsystem  170 , a structure subsystem  172 , and a prosody subsystem  174 , and the various outputs of these subsystems are combined according to the stored rules  114  to generate the annotated utterance tree  166  from the utterance  168 . 
     For the embodiment of the meaning extraction subsystem  150  illustrated in  FIG. 7 , the vocabulary subsystem  170  generally handles the vocabulary of the meaning extraction subsystem  150 . As such, the illustrated meaning extraction subsystem  150  includes a number of vocabulary plug-ins  176  that enable analysis and extraction of the vocabulary of utterances. For the illustrated embodiment, the vocabulary plug-ins  176  include a learned multimodal word vector distribution model  178 , a learned unimodal word vector distribution model  180 , and any other suitable word vector distribution models  182 . In this context, “unimodal” refers to word vector distribution models having a single respective vector for each word, while “multimodal” refers to word vector distribution models supporting multiple vectors for particular words (e.g., homonyms, polysemes) that can have different meanings in different contexts (e.g., a “bank” may refer to a place to store money, money itself, a maneuver of an aircraft, or a location near a river). The models  178 ,  180 , and  182  provide pluggable collections of word vectors that can be selected based on suitable parameters, such as language, conversation style, conversational channel, and so forth. 
     For example, the learned multimodal distribution model  178  and the learned unimodal distribution model  180  can provide word distributions (e.g., defined vector spaces of word vectors) that are generated using unsupervised learning or other general clustering algorithms, as discussed below with respect to  FIG. 15 . That is, appreciating that words commonly used in close proximity within utterances often have related meanings, the learned multimodal distribution model  178  and learned unimodal distribution model  180  can be generated by performing statistical analysis of utterances (e.g., from the corpus of utterances  112 ), and then defining vectors for words based on how the word is commonly used with respect to other words within these utterances. As such, these vocabulary plugins  176  enable the vocabulary subsystem  170  to recognize and address synonyms, misspelled words, encoded symbols (e.g., web addresses, network paths, emoticons, and emojis), out-of-vocabulary terms, and so forth, when processing the user utterance  122  and sample utterances  155 . In certain embodiments, the vocabulary subsystem  170  can combine or select from word vectors output by the various vocabulary plug-ins  176  based the stored rules  114  to generate word vectors for nodes of the annotated utterance tree  166 , as discussed below. Moreover, the word vector distribution models  178 ,  180 , and/or  182  can be continually updated based on unsupervised learning performed on received user utterances  122 , as discussed below with respect to  FIG. 15 . 
     For the embodiment illustrated in  FIG. 7 , the structure subsystem  172  of the meaning extraction subsystem  150  analyzes a linguistic shape of the utterance  168  using a combination of rule-based and ML-based structure parsing plugins  184 . In other words, the illustrated structure plug-ins  184  enable analysis and extraction of the syntactic and grammatical structure of the utterances  122  and  155 . For the illustrated embodiment, the structure plug-ins  184  include rule-based parsers  186 , ML-based parsers  188  (e.g., DNN-based parsers, RNN-based parsers, and so forth), and other suitable parser models  190 . For example, one or more of these structure plug-ins  184  enables class annotations or tagging (e.g., as a verb, a subject or entity, a direct object, a modifier, and so forth) for each word or phrase of the utterance. In certain embodiments, the structure subsystem  172  can combine or select from parse structures output by the various structure plug-ins  184  based on one or more rules  114  stored in the database  106 , which are used to define the structure or shape of the annotated utterance trees  166 , as discussed below. 
     For the embodiment illustrated in  FIG. 7 , the prosody subsystem  174  of the meaning extraction subsystem  150  analyzes the prosody of the utterance  168  using a combination of rule-based and ML-based prosody plugins  196 . The illustrated prosody plug-ins  192  include rule-based prosody systems  194 , ML-based prosody systems  196 , and other suitable prosody systems  198 . Using these plugins, the prosody subsystem  174  analyzes the utterance  168  for prosody cues, such as rhythm (e.g., speech rhythm, segmentations indicated by punctuation or pauses), emphasis (e.g., capitalization, bolding, underlining, asterisks), focus or attention (e.g., repetition of particular terms or styles), and so forth, which can be used to determine, for example, boundaries between intents, degrees of urgency or relative importance with respect to different intents, and so forth. As such, in certain embodiments, the prosody subsystem  174  can combine or select from prosody parsed structures output by the various prosody plug-ins  192  based on the rules  114  stored in the database  106  to generate the annotated utterance tree  166 , as discussed below. 
     As such, for the embodiment of the meaning extraction subsystem  150  illustrated in  FIG. 7 , the vocabulary subsystem  170 , the structure subsystem  172 , and the prosody subsystem  174  cooperate to generate the annotated utterance tree  166  from the utterance  168  based on one or more rules  114 . It may be appreciated that, in certain embodiments, a portion of the output of one subsystem (e.g., the prosody subsystem  174 ) may be provided as input to another subsystem (e.g., the structure subsystem  172 ) when generating the annotated utterance tree  166  from the utterance  168 . The resulting annotated utterance tree  166  data structure generated by the meaning extraction subsystem  150  includes a number of nodes, each associated with a respective word vector provided by the vocabulary subsystem  170 . Furthermore, these nodes are arranged and coupled together to form a tree structure based on the output of the structure subsystem  172  and the prosody subsystem  174 , according to the stored rules  114 . 
     For example,  FIG. 16  is a diagram illustrating an example of an annotated utterance tree  166  generated for an utterance  168 , in accordance with an embodiment of the present approach. As mentioned, the annotated utterance tree  166  is a data structure that is generated by the meaning extraction subsystem  150  based on the utterance  168 . For the example illustrated in  FIG. 16 , the annotated utterance tree  166  is based on an example utterance, “I want to go to the store by the mall today to buy a blue, collared shirt and black pants and also to return some defective batteries.” The illustrated annotated utterance tree  166  includes a set of nodes  202  (e.g., nodes  202 A,  202 B,  202 C,  202 D,  202 E,  202 F,  202 G,  202 H,  202 I,  202 J,  202 K,  202 L,  202 M,  202 N, and  202 P) arranged in a tree structure, each node representing a particular word or phrase of the utterance  168 . It may be noted that each of the nodes  202  may also be described as representing a particular subtree of the annotated utterance tree  166 , wherein a subtree can include one or more nodes  202 . 
     As mentioned, the form or shape of the annotated utterance tree  166  illustrated in  FIG. 16  is determined by the prosody subsystem  174  and the structure subsystem  172  and represents the syntactic, grammatical meaning of the example utterance. More specifically, the prosody subsystem  174  segments the utterance, while the structure subsystem  172  constructs the annotated utterance tree  166  from these segments. Each of the nodes  202  store or reference a respective word vector that is determined by the vocabulary subsystem  170  to indicate the semantic meaning of the particular word or phrase of the utterance. As mentioned, each word vector is an ordered n-dimensional list (e.g., a 300 dimensional list) of floating point values (e.g., a 1×N or an N×1 matrix) that provides a mathematical representation of the semantic meaning of a portion of an utterance. 
     Moreover, each of the nodes  202  is annotated by the structure subsystem  172  with additional information about the word or phrase represented by the node. For example, in  FIG. 16 , each of the nodes  202  has a respective class annotation. In particular, for the example annotated utterance tree illustrated in  FIG. 16 , certain subtrees or nodes (e.g., nodes  202 A,  202 B,  202 C, and  202 D) are annotated to be verb nodes, and certain subtrees or nodes (e.g., nodes  202 E,  202 F,  202 G,  202 H,  202 I, and  202 J) are annotated to be subject or object nodes, and certain subtrees or nodes (e.g., nodes  202 K,  202 L,  202 M,  202 N, and  202 P) are annotated to be modifier nodes (e.g., subject modifier nodes, object modifier nodes, verb modifier nodes) by the structure subsystem  172 . As discussed below, these class annotations are used by the meaning search subsystem  152  when comparing meaning representations that are generated from annotated utterance trees, like the example annotated utterance tree  166  illustrated in  FIG. 16 . As such, it may be appreciated that the annotated utterance tree  166 , from which the meaning representations are generated, serves as a basis (e.g., an initial basis) for intent/entity extraction. 
     It may also be noted that, in certain embodiments, the meaning extraction subsystem  150  includes rule-based error detection and correction mechanisms for improved domain specificity. For example,  FIG. 8  is a flow diagram illustrating an embodiment of a process  210  whereby the meaning extraction subsystem  150  can iteratively generate and then analyze the annotated utterance tree  166  for errors before a corresponding meaning representation  212  is generated for searching. In other words, to accommodate inaccuracies and unexpected output from ML-based models of the vocabulary subsystem  170 , the structure subsystem  172 , and/or the prosody subsystem  174 , the meaning extraction subsystem  150  is capable of performing a rule-based automated error detection process before the corresponding meaning representation  212  is generated. It may be appreciated that, when the utterance  168  is a user utterance  122 , the corresponding meaning representation  212  becomes part of the meaning representations  162  of the utterance meaning model  160 , and when the utterance is one of the sample utterances  155  of the intent/entity model  108 , the corresponding meaning representation  212  becomes part of the meaning representations  158  of the understanding model  157 , as discussed above with respect to  FIG. 6 . 
     For the embodiment illustrated in  FIG. 8 , the process  210  begins with the meaning extraction subsystem  150  of the NLU framework  104  generating (block  214 ) the annotated utterance tree  166  from the utterance  168  using one or more ML-based plugins (e.g., ML-based parsers  188  or ML-based prosody systems  196 ), as discussed above. In certain embodiments, this step may include a preliminary cleansing and augmentation step performed before the annotated utterance tree  166  is generated. For example, in certain embodiments, this preliminary cleansing and augmentation step may involve the vocabulary subsystem  170 , the structure subsystem  172 , and/or the prosody subsystem  174  modifying the utterance  168  based on the stored rules  114 . By way of specific example, during this step, the utterance  168  may be processed by the vocabulary subsystem  170  to modify words of the utterance (e.g., substitute synonyms, correct misspellings, remove punctuation, address domain-specific syntax and terminology, combine words, separate compounds words and contractions) based on the rules  114 . Then, the vocabulary subsystem  170 , the structure subsystem  172 , and the prosody subsystem  174  of the meaning extraction subsystem  150  can cooperate to generate the annotated utterance tree  166  from the utterance  168  based on the stored rules  114 . 
     Additionally, for the embodiment illustrated in  FIG. 8 , the process  210  includes a rule-based augmentation error and detection step (block  216 ) in which the generated annotated utterance tree  166  is analyzed for errors based on the stored rules  114 . These errors may include, for example, misclassification, misparses, and so forth, by one or more ML-based plugins of the meaning extraction subsystem  150 . When, during the rule-based augmentation error and detection step of block  216 , the meaning extraction subsystem  150  detects an error (decision block  218 ), then the meaning extraction subsystem  150  performs a rule-based correction (block  220 ) to generate a modified utterance  222  from the original or previous utterance  168  based on the stored rules  114 . 
     In situations in which errors are detected in block  218 , once the correction has been applied in block  220 , the annotated utterance tree  166  is regenerated in block  214  from the modified utterance  222  based on the rules  114 , as indicated by the arrow  224 . In certain embodiments, this cycle may repeat any suitable number of times, until errors are no longer detected at decision block  218 . At that point, the meaning extraction subsystem  150  generates (block  226 ) the corresponding meaning representation  212  to be processed by the meaning search subsystem  152 , as discussed below. In certain embodiments, information regarding the corrections performed in block  220  and the resulting annotated utterance tree  166  that is converted to the meaning representation  212  may be provided as input to train one or more ML-based plugins of the meaning extraction subsystem  150  (e.g., ML-based parsers  188  or ML-based prosody systems  196 ), such that the erroneous annotated utterance trees can be avoided when processing future utterances. 
     In certain embodiments, generating the corresponding meaning representation  212  for the annotated utterance tree  166  (block  226 ) may include determining compilation unit information (e.g., root nodes, parent root nodes, and subtree vectors) and optimizing the meaning representations for search. For example,  FIG. 9  is a flow diagram illustrating an embodiment of a process  240  whereby the meaning extraction subsystem  150  generates the corresponding meaning representation  212  from the annotated utterance tree  166 . To do this, the prosody subsystem  174  of the meaning extraction subsystem  150  takes the annotated utterance tree  166  and performs a segmentation step (block  242 ) based on one or more stored rules  114  (e.g., intent segmentation rules). During this segmentation step, the annotated utterance tree  166  is segmented or divided into individual intent subtrees, each representing an atomic intent of the annotated utterance tree  166 . This intent segmentation step may also involve information from a compilation model template  244 , which may be part of a compilation model template table or database (e.g., associated with the database  106  of  FIGS. 4A and 4B ). The compilation model template  244  stores data indicating how meaning representations  162  and  158  are to be generated by the meaning extraction subsystem  150  and compared to one another by the meaning search subsystem  152 , as is discussed below in greater detail. 
     For the embodiment illustrated in  FIG. 9 , for each intent subtree identified in block  242 , the meaning extraction subsystem  150  identifies (block  246 ) all corresponding subtrees that depend from each particular intent subtree. Then, for each of these intent trees and corresponding subtrees, the meaning extraction subsystem  150  generates (block  248 ) a respective compilation unit triple  250 . In particular, the illustrated compilation unit triple  250  includes: a reference  252  to a root node of a subtree, a reference  254  to a parent of the root node of the subtree, and a subtree vector  256  that is representative of the semantic meaning of the subtree. The aforementioned compilation model template  244  defines one or more tree substructure vectorization algorithms  258  that produce vectors for each of the corresponding subtrees, as discussed in greater detail below. 
     Once the compilation unit triples  250  have been generated for the annotated utterance tree  166 , the annotated utterance tree  166  is converted into the meaning representation  212 . In certain embodiments, certain information that is not relevant to the meaning search subsystem  152  (e.g., certain classes of nodes, certain annotation data) may be removed during this step to minimize the size of the meaning representation  212  for enhanced efficiency when searching. The generated meaning representation  212  subsequently becomes one of the meaning representations  162  of the utterance meaning model  160  or one of the meaning representations  158  of the understanding model  157 , depending on the origin of the utterance  168  represented by the annotated utterance tree  166 , as discussed above. 
     To more clearly illustrate,  FIG. 17  is a diagram presenting an example of a meaning representation  212  generated for the example annotated utterance tree  166  of  FIG. 16 , in accordance with an embodiment of the present approach. As mentioned, the meaning representation  212  is a data structure generated from the annotated utterance tree  166  by the meaning extraction subsystem  150 . As such, certain nodes of the meaning representation  212  include compilation unit triples  250  that were generated using the process  240  of  FIG. 9 . In particular, all of the intent subtrees (e.g., subtrees from nodes  202 A,  202 B,  202 C, and  202 D), and all of the subtrees that depend from these intent subtrees (e.g., subtrees  202 E,  202 F,  202 G,  202 H,  202 I,  202 J), include a respective compilation unit triple  250  (e.g., compilation unit triples  250 A,  250 B,  250 C,  250 D,  250 E,  250 F,  250 G,  250 H,  250 I, and  250 J). Further, as discussed above, each of these compilation unit triples  250  includes a respective subtree vector that is generated based the vectors (e.g., word vectors and/or subtree vectors) of depending nodes and/or subtrees. 
       FIG. 10  is a diagram that illustrates an example embodiment of the compilation model template  244  mentioned above. Data stored within the compilation model template  244  generally defines how the meaning extraction subsystem  150  generates subtree vectors for the annotated utterance trees  166  as part of the compilation unit triple  250  determined in block  248  of  FIG. 9 . Further, data stored within the compilation model template  244  generally defines how the meaning search subsystem  152  compares and scores similarity between the meaning representations  162  of the utterance meaning model  160  and the meaning representations  158  of the understanding model  157 , as illustrated in  FIG. 6 . In certain embodiments, the compilation model template  244  may be stored as one or more tables of the database  106  illustrated in  FIGS. 4A and 4B , or within another suitable data structure, in accordance with the present disclosure. 
     As mentioned with respect to  FIG. 9 , the compilation model template  244  illustrated in  FIG. 10  includes one or more tables identifying or storing one or more pluggable tree substructure vectorization algorithms  258  that generate the subtree vectors  256  of the compilation unit triples  250 . As illustrated, the tree substructure vectorization algorithms  258  may be associated with focus/attention/magnification (FAM) coefficients  270 . For such embodiments, these FAM coefficients  270  are used to tune how much relative focus or attention (e.g., signal magnification) should be granted to each portion of a subtree when generating a subtree vector. The tree-model comparison algorithms  272 , the class compatibility rules  274 , and the class-level scoring coefficients  276  of the compilation model template  244  illustrated in the compilation model template  244  of  FIG. 10  are discussed below. 
       FIG. 11  is a block diagram illustrating example operation of an embodiment of a tree substructure vectorization algorithm  258  to generate a subtree vector  256 , which is part of the compilation unit triple  250  determined for subtrees of the annotated utterance tree  166 , as discussed with respect to  FIG. 9 . As mentioned above, the vocabulary subsystem  170  provides word vectors for each node  202  of an annotated utterance tree  166 . For the illustrated embodiment, the vocabulary subsystem  170  generated four or more word vectors, represented as V 1 , V 2 , V 3 , and V 4 , which are respectively associated with four nodes of the annotated utterance tree  166 . That is, in certain embodiments, the NLU framework  104  may modify the annotated utterance tree  166  (e.g., the vocabulary subsystem  170  may replace individual words with phrasal equivalents, the structure subsystem  172  may expand contractions, and so forth), as discussed with respect to  FIG. 8 . As such, it is appreciated that, at one or more stages of intent/entity extraction, the number of nodes/subtrees of the annotated utterance tree  166  may be increased or decreased, along with the number of word vectors combined to calculate the subtree vector  256 , relative to an original utterance or an initially generated annotated utterance tree  166 . 
     As such, for the example illustrated in  FIG. 11 , the tree substructure vectorization algorithm  258  generates the subtree vector  256 , by first multiplying each of the word vectors by a respective one (e.g., α, β, γ, δ) of the FAM coefficients  270 , which increases or decreases the contribution of each word vector to the combined subtree vector  256 . After applying the FAM coefficients  270  to the word vectors V 1-4 , the results are combined using vector addition, as indicated by the “+” notation in  FIG. 11 . Additionally, for the illustrated embodiment, the resulting subtree vector  256  is subsequently normalized to ensure that the dimensions of the combined subtree vector are each within a suitable range after the multiplication and addition operations. It may be noted that the tree substructure vectorization algorithm  258  illustrated in  FIG. 11  is merely provided as an example, and in other embodiments, other suitable tree substructure vectorization algorithms may be used, in accordance with the present disclosure. 
     By way of example, in certain embodiments, verb words or subtrees may be associated with one of the FAM coefficients  270  (e.g., a) that is greater in value than another FAM coefficient (e.g., β) associated with a subject or direct object word or subtree vector. In certain embodiments, root node word vectors may be associated with a relatively higher FAM coefficient  270  than word vectors associated with other nodes. In certain embodiments, the combined subtree vector  256  is a centroid that is calculated as the weighted average of the word vectors associated with all nodes of the subtree. In other embodiments, the meaning extraction subsystem  150  may recursively perform subtree vectorization to a predefined depth or until a particular node class is identified (e.g., a subject node, a modifier node). In certain embodiments, one or more of the vectors (e.g., V 1 , V 2 , V 3 , and V 4 ) that are used to generate the combined subtree vector may itself be a combined subtree vector that is generated from other underlying word and/or subtree vectors. For such embodiments, subtrees with at least one depending node (e.g., non-leaf nodes/subtrees) may be associated with a higher FAM coefficient value than single-node (e.g., a leaf nodes/subtrees). 
     Once the meaning representations  158  and  162  have been generated, as illustrated in  FIG. 6 , the meaning search subsystem  152  can compare these meaning representations to extract intent/entities from the user utterance  122 .  FIG. 12  is a flow diagram illustrating an example embodiment of a process  280  whereby the meaning search subsystem  152  searches the meaning representations  158  of the understanding model  157  for matches to the meaning representation  162  of the user utterance  122  based on information stored in the compilation model template  244 . For the embodiment illustrated in  FIG. 12 , the meaning search subsystem  152  receives the at least one meaning representation  162  of the utterance meaning model  160  generated in  FIG. 9 , as discussed above. Using the prosody subsystem  174  discussed above, the meaning search subsystem  152  first segments (block  282 ) the meaning representations  162  into intent subtrees, each representing an atomic intent, based on one or more stored rules  114  (e.g., intent-segmentation rules). 
     For the embodiment illustrated in  FIG. 12 , for each intent subtree of the meaning representation  162  identified in block  282 , the meaning search system  152  compares (block  284 ) the subtree of the meaning representation  162  to the meaning representations  158  of the understanding model  157 , based on the contents of the compilation model template  244 , to generate corresponding intent-subtree similarity scores  285  using the tree-model comparison algorithm  272 . For the embodiment illustrated in  FIG. 12 , the meaning search system  152  then adds (block  286 ) the similarity scores calculated in block  284  to the utterance meaning model  160 , which may serve as the extracted intent/entities  140  that are passed to the RA/BE  102 , as illustrated in  FIG. 5 . In other embodiments, the meaning search system  152  may generate a different data structure (e.g., a simpler, smaller data structure) to represent the extracted intents/entities  140  that includes only the identified intents/entities from the user utterance  122  (or references to these intent/entities in the intent/entity model  108 ) along with the intent-subtree similarity scores  285  as a measure of confidence in the intent/entity extraction. In still other embodiments, the extracted intents/entities  140  may only include intents/entities associated with intent subtree similarity scores greater than a predetermined threshold value, which may be stored as part of the compilation model template  244 . 
     Returning briefly to  FIG. 10 , the illustrated compilation model template  244  includes one or more tables identifying or storing one or more tree model comparison algorithms  272  that are used to compare and score similarity between the meaning representations  162  of the utterance meaning model  160  and the meaning representations  158  of the understanding model  157 , as illustrated in  FIG. 6 . As discussed in greater detail, the tree model comparison algorithms  272  are pluggable modules defined or identified in the compilation model template  244  that are designed to determine a similarity score between two subtree vectors generated by the substructure vectorization algorithms  258 , based on class compatibility rules  274  that are also stored as part of the compilation model template  244 . The class compatibility rules  274  define which classes of subtree vectors can be compared to one another (e.g., verb word and subtree vectors are compared to one another, subject or object word and subtree vectors are compared to one another) to determine vector distances that provide measures of meaning similarity therebetween. 
     The illustrated embodiment of the compilation model template  244  also includes class-level scoring coefficients  276  that define different relative weights in which different classes of word/subtree vectors contribute to an overall similarity score between two subtrees, as discussed with respect to  FIG. 13 . For example, in certain embodiments, a verb subtree similarity score may be weighted higher and contribute more than a subject subtree similarity score. This sort of weighting may be useful for embodiments in which the agent automation system  100  tends to receive specific natural language instructions. Additionally, in certain embodiments, both the action being requested and the object upon which this action should be applied may be considered more important or influential to the meaning of an utterance than the subject, especially when the subject is the agent automation system  100 . For such embodiments, a verb subtree similarity score and a direct object subtree similarity score may be weighted higher and contribute more to the overall similarity score than a subject subtree similarity score. In certain embodiments, the class-level scoring coefficients  276  may be predefined, derived or updated using a ML-based approach, derived or updated using a rule-based approach, or a combination thereof. 
     As such, in certain embodiments, subtrees are considered a match (e.g., are afforded a higher similarity score) when they resolve to prescribed syntactic patterns found within a larger form. For instance, for an utterance determined to be in an active form (e.g., a subject-verb-any form, as detected by a rules-based parser  186  of the structure subsystem  172  using pre-defined pattern rules), a direct subject subtree (which could be a single word or a complete clause) of the verb may be treated as the subject argument to the verb-led form. Likewise, for an utterance determined to be in a passive form (e.g., a form with passive auxiliaries to the verb), then a prepositional object attached to a specific form of preposition attached to the verb may be treated as the subject equivalent. For example, certain subject (e.g., direct subject) or object (e.g., direct object, indirect object, prepositional object) subtrees are compatible with other subject or object subtrees and can be compared. As a specific example, a first utterance, “Bob ate cheese,” is in the active form and, therefore, “Bob” is the direct subject of a form of the verb “to eat.” In a second example utterance, “Cheese was eaten by Bob,” “was” is a passive auxiliary that indicates, along with the verb form, that the second utterance is in the passive form. For the second example utterance, “by Bob” is the prepositional phrase, with “Bob” being the prepositional object. Accordingly, “Bob” in the first utterance (e.g., as a direct subject in the active form) is compatible with “Bob” in the second utterance (e.g., as a prepositional object in the passive form) and can be compared as described. 
       FIG. 13  illustrates an embodiment of a process  290  in which an example tree-model comparison algorithm  272  of the meaning search subsystem  152  compares an intent subtree  292  of the meaning representations  162  (representing at least a portion of the user utterance  122 ) to an intent subtree  294  of the meaning representations  158  (representing at least a portion of one of the sample utterances  155  of the intent/entity model  108 ) to calculate an intent subtree similarity score  285 . As mentioned, the tree-model comparison algorithm  272  uses the class compatibility rules  274  and the class-level scoring coefficients  276  of the compilation model template  244  to calculate this intent subtree similarity score  285 . It may be noted that, in other embodiments, the process  290  may include fewer steps, additional steps, repeated steps, and so forth, in accordance with the present disclosure. 
     For the illustrated embodiment, the process  290  involves identifying (block  296 ) class compatible sub-trees  298  and  300  from the intent subtrees  292  and  294 , respectively, as defined by the class compatibility rules  274 . For the illustrated example, the first class compatible subtree  298  (of the first intent subtree  292 ) and the second class compatible subtree  300  (of the second intent subtree  294 ) are then compared to determine a respective class similarity score. More specifically, a respective class similarity score is calculated (block  302 ) for each node or subtree depending from the class compatible subtrees identified in block  296 . In particular, the class similarity score may be determined based on the vector distance between the subtree vectors  256  of the first and second class-compatible subtrees  298  and  300 . 
     As indicated by the arrow  304 , blocks  296  and  302  may be repeated until all class compatible subtrees have been identified and the class similarity scores  306  for all class compatible subtrees have been calculated. In an example, the class similarity score for a given class (e.g., a verb class, a subject class, a modifier class) is calculated to be the weighted average of all class-compatible similarity contributions by the constituent subtrees of the intent trees being compared. In other embodiments, the class similarity score for a given class may be calculated as an average similarity score (e.g., an average vector distance) of all nodes or subtrees of the class that are directly coupled to the root nodes of the class compatible subtrees  298  and  300 . In certain embodiments, each class similarity score value may be between 0 and 1, inclusively. For example, when comparing the intent subtrees  292  and  294 , a set (e.g., an array or matrix) of class similarity scores may include a first class similarity score corresponding to nodes and subtrees of a first class (e.g., verbs), a second class similarity score corresponding to nodes and subtrees of a second class (e.g., direct objects), a third class similarity score corresponding to nodes and subtrees of a third class (e.g., verb modifiers), and so forth. 
     Continuing through the process illustrated in  FIG. 13 , the class similarity scores  306  are subsequently combined (block  308 ) to yield an overall intent-subtree similarity score  285  between the first and second intent subtrees  292  and  294 . That is, in block  308 , the meaning search subsystem  152  uses the class-level scoring coefficients  276  of the compilation model template  244  to suitably weight each class similarity score generated in block  302  to generate the overall intent subtree similarity score  285 . For example, a first class similarity score corresponding to nodes and subtrees of a first class (e.g., modifiers) is multiplied by a class-level scoring coefficient associated with the first class, a second class similarity score corresponding to nodes and subtrees of a second class (e.g., verbs) is multiplied by a class-level scoring coefficient associated with the second class, a third class similarity score corresponding to nodes and subtrees of a third class (e.g., subjects), is multiplied by a class-level scoring coefficient associated with the third class, and so forth. Additionally, in certain embodiments, one class similarity score corresponds to the vector distance between the respective subtree vectors  256  associated with the root node of the first intent subtree  292  and the root node of the second intent subtree  294 , and this class similarity score is similarly multiplied by a respective class-level scoring coefficient (e.g., root node scoring coefficient). In certain embodiments, these products are summed and the result is divided by the number of class similarity scores. As such, for the illustrated example, the overall intent subtree similarity score  285  may be described as a weighted average of the class similarity scores  306  of the class compatible subtrees and the class similarity score of the root nodes. In certain embodiments, the intent subtree similarity score  285  may be normalized to have a value between 0 and 1, inclusive. 
     Additionally, it may be appreciated that present embodiments enable entrenchment, which is a process whereby the agent automation system  100  can continue to learn or infer meaning of new syntactic structures in new natural language utterances based on previous examples of similar syntactic structures to improve the domain specificity of the NLU framework  104  and the agent automation system  100 . As used herein, “domain specificity” refers to how attuned the system is to correctly extracting intents and entities expressed in actual conversations in a given domain and/or conversational channel. For example, in an embodiment, certain models (e.g., NN structure or prosody models, word vector distribution models) are initially trained or generated using generic domain data (e.g., such as a journal, news, or encyclopedic data source). Since this generic domain data may not be representative of actual conversations (e.g., actual grammatical structure, prosody, and vocabulary) of a particular domain or conversational channel, the disclosed NLU framework  104  is capable of analyzing conversations within a given domain and/or conversational channel, such that these models can be conditioned to be more accurate or appropriate for the given domain. 
     It is presently recognized that this can enable the agent automation system  100  to have a continuously learning grammar structure model capable of accommodating changes in syntactic structure, such as new grammatical structures and changes in the use of existing grammatical structures. For example,  FIG. 14  is a flow diagram illustrating an embodiment of a process  320  whereby the agent automation system  100  continuously improves a ML-based parser  188 , which may be plugged into the structure subsystem  172  of the meaning extraction subsystem  150 , as discussed with respect to  FIG. 7 . 
     For the example illustrated in  FIG. 14 , the ML-based parser  188  is specifically a recurrent neural network (RNN)-based parser that operates based on a RNN model  322 . As such, it is appreciated that, by adjusting signal weighting within the RNN model  322 , the ML-based parser  188  can continue to be trained throughout operation of the agent automation system  100  using training data generated from a continually growing corpus of utterances  112  of the database  106  illustrated in  FIG. 4A . For the example illustrated in  FIG. 14 , the corpus of utterances  112  may be a continually growing collection of stored user utterances  122  and agent utterances  124 , such as a chat log. 
     For the embodiment illustrated in  FIG. 14 , prior to operation of the agent automation system  100 , the RNN-based model  322  may initially have a set of weights (e.g., a matrix of values) that are set by training. For this example, the ML-based parser  188  may be trained using a first corpus of utterances having a particular grammatical style, such as a set of books, newspapers, periodicals, and so forth, having a formal or proper grammatical structure. However, it is appreciated that many utterances exchanges in different conversational channels (e.g., chat rooms, forums, and emails) may demonstrate different grammatical structures, such as less formal or more relaxed grammatical structures. With this in mind, the continual learning loop illustrated in  FIG. 14  enables the RNN-model  322  associated with the ML-based parser  188  to be continually updated and adjusted, such that the ML-based parser  188  can become more adept at parsing different (e.g., less-formal or less-proper) grammatical structures in newly received user utterances  122 . 
     The continual leaning process  320  illustrated in  FIG. 14  includes receiving and responding to the user utterance  122 , as discussed above with respect to the process  145  of  FIG. 5 . As mentioned, in certain embodiments, the user utterances  122  and the agent utterances  124  are collected to populate the corpus of utterance  112  stored in the database  106 , as illustrated in  FIG. 4A . As some point, such as during regularly scheduled maintenance, the prosody subsystem  174  of the meaning extraction subsystem  150  segments (block  323 ) the collection of stored user utterances  122  and agent utterances  124  into distinct utterances  324  ready for parsing. Then, different rule-based parsers  186  and/or ML-based parsers  188  of the structure subsystem  172  of the meaning extraction subsystem  150  parse (block  325 ) each of the utterances  324  to generate a multiple annotated utterance tree structures  326  for each of the utterances  324 . The meaning extraction subsystem  150  then determines (in decision block  328 ) whether a quorum (e.g., a simple majority consensus) has been reached by the different parsers. 
     For the example illustrated in  FIG. 14 , when the meaning extraction subsystem  150  determines in block  328  that a sufficient number (e.g., a majority, greater than a predetermined threshold value) of annotated utterance trees  326  for a particular utterance are substantially the same for a quorum to be reached, then the meaning extraction subsystem  150  may use the quorum-based set of annotated utterance trees  330  to train and improve a ML-model  322  associated with the ML-based parser  188 , as indicated by the arrow  331 . For example, the weights within the ML-model  322  may be repeatedly adjusted until the ML-based parser  188  generates the appropriate structure from the quorum-based set of annotated utterance trees  330  for each of the utterances  324 . After this training, upon receiving a new user utterance  122  having a grammatical structure similar to a structure from the quorum-based set of annotated utterance trees  330 , the operation of the ML-based parser  188 , the NLU framework  104 , and the agent automation system  100  is improved to more correctly parse the grammatical structure of the user utterance  122  and extract the intents/entities  140  therefrom. 
     Additionally, in certain embodiments, the agent automation system  100  can continue to learn or infer meaning of new words and phrases. It is presently recognized that this can enable the agent automation system  100  to have a continuously expanding/adapting vocabulary capable of accommodating the use of unfamiliar words, as well as changes to the meaning of familiar words. For example,  FIG. 15  is a flow diagram illustrating an embodiment of a process  340  whereby the agent automation system  100  continuously improves a word vector distribution model  342 , which may be plugged into the structure subsystem  172  of the meaning extraction subsystem  150 , such as the learned multimodal word vector distribution model  178  or the learned unimodal word vector distribution model  180  discussed above with respect to  FIG. 7 . As such, it is appreciated that, by expanding or modifying the word vector distribution model  342 , operation of the vocabulary subsystem  170 , the NLU framework  104 , and the agent automation system  100  can be improved to handle words with new or changing meanings using only training data that can be generated from a continually growing corpus of utterances  112  of the database  106  illustrated in  FIG. 4A . For the example illustrated in  FIG. 15 , the corpus of utterances  112  may be, for example, a collection of chat logs storing user utterances  122  and agent utterances  124  from various chat room exchanges, or other suitable source data. 
     For the embodiment illustrated in  FIG. 15 , prior to operation of the agent automation system  100 , the word vector distribution model  342  may initially be generated based on a first corpus of utterances that have a particular diction and vocabulary, such as a set of books, newspapers, periodicals, and so forth. However, it is appreciated that many utterances exchanges in different conversational channels (e.g., chat rooms, forums, emails) may demonstrate different diction, such as slang terms, abbreviated terms, acronyms, and so forth. With this in mind, the continual learning loop illustrated in  FIG. 15  enables the word vector distribution model  342  to be modified to include new word vectors, and to change values of existing word vectors, based on source data gleaned from the growing collections of user and agent utterances  122  and  124 , to become more adept at generating annotated utterance trees  166  that include these new or changing terms. 
     Like  FIG. 14 , the process  340  illustrated in  FIG. 15  includes receiving and responding to the user utterance  122 , as discussed above with respect to  FIG. 5 . As mentioned, the user utterances  122  and the agent utterances  124  can be collected to populate the corpus of utterance  112  stored in the database  106 , as illustrated in  FIG. 4A . As some point, such as during regularly scheduled maintenance, the prosody subsystem  174  of the meaning extraction subsystem  150  segments (block  343 ) the corpus of utterances  112  into distinct utterances  344  that are ready for analysis. Then, in block  345 , the meaning extraction subsystem  150  performs rule-augmented unsupervised learning to generate a refined word vector distribution model  346  containing new or different word vectors  348  generated from the segmented utterances  344 . 
     For example, as discussed above, the meaning extraction subsystem  150  may analyze the set of segmented utterances  344  and determine word vectors  348  for the words of these utterances based on how certain words tend to be used together. For such embodiments, two words that are frequently used in similar contexts within these utterances  344  are considered closely related and, therefore, are assigned a similar vector value (e.g., relatively closer in terms of Euclidean distance) in one or more dimensions of the word vectors  348 . In this manner, the meaning extraction subsystem  150  may adapt to changes in the meaning of a previously understood term based on new context in which the term is used. 
     As illustrated in  FIG. 15 , the refined word vector distribution model  346  is used to replace the existing word vector distribution model  342 , such that the vocabulary subsystem  170  can use this refined model to provide word vectors for the words and phrases of new user utterances  122  received by the agent automation system  100 . For example, an initial word vector distribution model  342  may have a word vector for the term “Everest” that is relatively close in one or more dimensions to other word vectors for terms such as, “mountain”, “Himalayas”, “peak”, and so forth. However, when a client creates a new conference room that is named “Everest,” the term begins to be used in a different context within user utterances  122 . As such, in block  345 , a new word vector would be generated for the term “Everest” that would be relatively close in one or more dimensions to word vectors for terms such as “conference”, “meeting”, “presentation”, and so forth. After updating the word vector distribution model, upon receiving a user utterance  122  having the revised term “Everest,” the operation of the vocabulary subsystem  170 , the NLU framework  104 , and the agent automation system  100  is improved to more provide more accurate word vectors, annotated utterance trees, and meaning representations, which result in more accurately extracted intents/entities  140 . 
     Technical effects of the present disclosure include providing an agent automation framework that is capable of extracting meaning from user utterances, such as requests received by a virtual agent (e.g., a chat agent), and suitably responding to these user utterances. The NLU framework includes a meaning extraction subsystem that is designed to generate meaning representations for the sample utterances of the intent/entity model, as well as a meaning representation for a received user utterance. To generate these meaning representations, the meaning extraction subsystem includes a vocabulary subsystem, a structure subsystem, and a prosody subsystem that cooperate to parse utterances based on combinations of rule-based methods and ML-based methods. Further, for improved accuracy, the meaning extraction subsystem includes a rule-based augmentation error detection subsystem that can cooperate with the vocabulary, structure subsystem, and prosody subsystems to iteratively parse and correct an utterance before meaning representations are generated. The meaning representations are a data structure having a form or shape that captures the grammatical structure of the utterance, while subtrees of the data structure capture the semantic meaning of the words and phrases of the utterance as vectors that are annotated with additional information (e.g., class information). 
     Additionally, the disclosed NLU framework includes a meaning search subsystem that is designed to search the meaning representations associated with the intent/entity model to locate matches for a meaning representation of a received user utterance. Conceptually, the meaning representation of the received user utterance is used like a search key to locate matching meaning representations in the search space defined by the collection of meaning representations generated from the intent/entity model. The meaning search subsystem is designed to determine a similarity score for portions of different meaning representations based on stored particular stored rules and weighting coefficients (e.g., class compatibility rules and class-level scoring coefficients). Additionally, the NLU framework can continue to learn or infer meaning of new syntactic structures in new natural language utterance based on previous examples of similar syntactic structures, and learn or modify its vocabulary based on a usage of a new term or an existing term in a new context. As such, components of the NLU framework (e.g., a neural network models, the word vector distributions) may be continuously updated based on new utterances, such as natural language exchanges between users and a virtual agent, to enhance the adaptability of the NLU framework to changes in the use and meaning of certain terms and phrases over time. 
     Vocabulary Management 
     Virtual agents may be implemented in a wide range of applications for a wide range of customers or clients. For example, virtual agents may be utilized by organizations in retail, manufacturing, accounting, consumer product development and/or sales, software development, information technology services, social media, consulting, engineering, banking, oil and gas, insurance, real estate, commercial equipment sales, media, healthcare, construction, legal services, transportation, pharmaceuticals, marketing, etc. Further, these virtual agents may engage with users within these organizations in a wide variety of roles, such as executives, information technology an professionals, assistants, engineers, attorneys, doctors, nurses, technicians, accountants, human resources professionals, analysts, software developers, janitors, etc. Dependent upon the particular application and the particular users, a given instantiation of the virtual agent may utilize vocabulary terms that may be specific to that application and/or the specific users of the virtual agent. As such, it is presently recognized that there is a need to customize the vocabulary of virtual agents to the particular industries and users they will serve. Present embodiments are directed to an agent automation framework that is capable of extracting meaning from user utterances, such as requests received by a virtual agent (e.g., a chat agent), suitably responding to these user utterances, and learning new vocabulary words, or new meanings for known words, as time passes and exchanges between the chat agent and the user occur. 
     The agent automation framework includes a NLU framework and an intent/entity model having defined intents and entities that are associated with sample utterances. The NLU framework includes a meaning extraction subsystem that is designed to generate meaning representations for the sample utterances of the intent/entity model, as well as a meaning representation for a received user utterance. Additionally, the disclosed NLU framework includes a meaning search subsystem that is designed to search the meaning representations of the intent/entity model to locate matches for a meaning representation of a received user utterance. As such, present embodiments generally address the hard problem posed by NLU by transforming it into a manageable search problem. 
     In present embodiments, a meaning representation can be generated from an annotated utterance tree structure having a form or shape that represents the grammatical structures of the utterance, and having nodes that each represent words or phrases of the utterances as word vectors encoding the semantic meaning of the utterance. The meaning extraction subsystem includes a vocabulary subsystem, a structure subsystem, and a prosody subsystem that cooperate to parse utterances into the annotated utterance trees based on combinations of rule-based methods and machine learning (ML)-based (e.g., statistical) methods. Using one or more tree substructure vectorization algorithms and focus/attention/magnification (FAM) coefficients defined by a stored compilation model template, the meaning extraction subsystem subsequently generates subtree vectors for the annotated utterance tree structure, yielding the corresponding meaning representation for subsequent searching by the meaning search subsystem. 
     In present embodiments, the virtual agent is capable of learning new words, or new meanings for known words, based on exchanges between the virtual agent and the user in order to customize the vocabulary of the virtual agent to the needs of the user or users. Specifically, the agent automation framework may have access to a corpus of previous exchanges between the virtual agent and the user, such as one or more chat logs. The agent automation framework may segment the chat logs into utterances using the prosody subsystem. The utterances may then be further segmented into words and/or phrases. The agent automation framework may then recognize new words and/or new meanings for known words. New word vectors may then be generated for these new words and/or new meanings for known words. The new word vectors may then be added to an existing word vector distribution model of the vocabulary subsystem to generate a refined word vector distribution model. The new word vector may be generated, for example, based on the context in which the new word or meaning was used over one or more uses in the chat logs, input from a user, or some other source. The NLU framework may then utilize the refined word vector distribution model to interpret and analyze user utterances and generate responses. 
     In interpreting and analyzing user utterances, the user utterance may include a word or phrase for which there are multiple word vectors corresponding to multiple respective known meanings for the word. In order to identify the intended meaning, the agent automation framework segments the utterance into words and/or phrases. The words and/or phrases may then be pre-processed by, for example, applying directives and/or instructions from the vocabulary model. Pre-processing may include checking spelling, correcting formatting issues, expanding contractions, expanding abbreviations, replacing acronyms with associated words, as well as other data-cleansing processes. If word usage context is available, the agent automation framework may determine which meaning was intended by performing context-based disambiguation via an ontology service and/or the structure service. If no context is available, the agent automation framework may extract word vectors matching the surface form or form derivatives. If no word vectors are found, the agent automation framework derives semantic word vectors according to null-word rules. The vectors are then post-processed before being output. Post-processing may include, for example, extracting a representative vector or vector set given one or more synonymic vector lists. As time passes and the virtual agent exchanges utterances with the user, the virtual agent learns new words, or new meanings for known words, and thus customizes its vocabulary to its specific application and users. 
     A computing platform may include a chat agent, or another similar virtual agent, that is designed to respond to user requests to perform functions or address issues on the platform. There are two predominant technologies in NLU, namely traditional computational linguistics and newer machine learning (ML) methods. It is presently recognized that these two technologies demonstrate different strengths and weaknesses with respect to NLU. For example, traditional computational linguistic methods, also referred to herein as “rule-based” methods, include precision rule-sets and manually-crafted ontologies that enable precise adjudication of linguistic structure and semantic understanding to derive meaning representations. Traditional cognitive linguistic techniques also include the concept of construction grammars, in which an aspect of the meaning of a natural language utterance can be determined based on the form (e.g., syntactic structure) of the utterance. Therefore, rule-based methods offer results that are easily explainable and customizable. However, it is presently recognized that such rule-based methods are not particularly robust to natural language variation or adept at adapting to language evolution. As such, it is recognized that rule-based methods alone are unable to effectively react to (e.g., adjust to, learn from) data-driven trends, such as learning from chat logs and other data repositories. Furthermore, rule-based methods involve the creation of hand-crafted rules that can be cumbersome, wherein these rules usually are domain specific and are not easily transferable to other domains. 
     On the other hand, ML-based methods, perform well (e.g., better than rule-based methods) when a large corpus of natural language data is available for analysis and training. The ML-based methods have the ability to automatically “learn” from the data presented to recall over “similar” input. Unlike rule-based methods, ML-based methods do not involve cumbersome hand-crafted features-engineering, and ML-based methods can support continued learning (e.g., entrenchment). However, it is recognized that ML-based methods struggle to be effective when the size of the corpus is insufficient. Additionally, ML-based methods are opaque (e.g., not easily explained) and are subject to biases in source data. Furthermore, while an exceedingly large corpus may be beneficial for ML training, source data may be subject to privacy considerations that run counter to the desired data aggregation. 
     Accordingly, present embodiments are generally directed toward an agent automation framework capable of applying a combination rule-based and ML-based cognitive linguistic techniques to leverage the strengths of both techniques in extracting meaning from natural language utterances. More specifically, present embodiments are directed to generating suitable meaning representations for utterances, including received user utterances and sample utterances of an intent/entity model. These meaning representations generally have a shape that captures the syntactic structure of an utterance, and include one or more subtree vectors that represent the semantic meanings of portions of the utterance. The meaning representation of the utterance can then be searched against a search space populated with the meaning representations of the sample utterances of the intent/entity model, and one or more matches may be identified. As the chat agent of the agent automation framework exchanges utterances with one or more users, a chat log or other corpus of utterances may be populated. The agent automation framework may then analyze the chat log to identify new words or new meanings for known words, and generate word vectors for these new words and/or meanings. The new word vectors can be used to better analyze user utterances and can also be used in agent utterances responding to user utterances. Accordingly, as time passes and utterances are exchanged with the user, the chat agent may learn new words and/or new meanings for known words, thus customizing the chat agent&#39;s vocabulary to the chat agent&#39;s specific application and users. 
       FIG. 18  is a block diagram illustrating how the vocabulary subsystem  170  of  FIG. 7  operates within the NLU framework  104 . As shown, the rule based meaning and extraction system  150  has access to a base meaning repository  400 . The base meaning repository  400  may be a vectorized word space. That is, the base meaning repository  400  may include a collection of word vectors for known vocabulary words. As shown, the base meaning repository  400  may have access to a number of word distribution databases  402 . In the illustrated embodiment, the word distribution databases  402  include a unimodal database  404 , a multimodal database  406 , and a lexical database  408 . As described above with regard to  FIG. 7 , unimodal refers to word vector distributions having a single vector for each word. Accordingly, the unimodal database  404  may include a single word vector for each word listed in the database  404 . Correspondingly, multimodal refers to having word vector distributions supporting multiple vectors for particular words (e.g., homonyms, polysemes) that can have different meanings in different contexts (e.g., a “bank” may refer to a place to store money, money itself, a maneuver of an aircraft, or a location near a river). As such, the multimodal database  406  may include multiple word vectors for each word listed in the database  406  based on the different possible meanings for the word. The lexical database  408  may be used to for disambiguation purposes to help select the correct word vector for a given word from the multimodal database  406 . In some embodiments, this determination may be made based on context with help from the structure subsystem  172 . In some embodiments, as is discussed in more detail below, the lexical database  408  may also be used to generate word vectors for new words that were previously unknown, or for new meanings for known words. It should be understood, however, that the word distribution databases  402  shown in  FIG. 18  are merely examples and that embodiments are envisaged that utilize additional databases, fewer databases, or different combinations of databases. 
     As shown, data from the word distribution databases  402  may be retrieved or plugged into the base meaning repository  400  to provide a repository of known vocabulary words and their meanings to the meaning extraction subsystem  150 . As previously described, the meaning extraction subsystem  150  may parse provided utterances and output one or more meaning representations  418 . Occasionally, the meaning extraction subsystem  150  may come across a new word that is not in the base meaning repository  400 , or use of a word in the base meaning repository  400  that does not comport with any of the meanings of the word stored in the base meaning repository  400 . In such instances, the meaning extraction subsystem  150  may alert a vocabulary modeling service  410 . In some embodiments, the meaning extraction subsystem  150  may provide the vocabulary modeling service  410  with both the new word and/or meaning, as well as the context in which the new word and/or meaning was used. The vocabulary modeling service  410  generates and/or maintains an agent-specific vocabulary model  412 . For example, the vocabulary modeling service  410  may store meanings for new vocabulary words and/or alternate meanings for known vocabulary words. For example, a company may have in its office a conference room called “Everest”. Accordingly, the vocabulary modeling service  410  may store the meaning of the word “Everest” as being a mountain and/or a conference room. Further, the vocabulary modeling service  410  may notice certain patterns for when different meanings of a word are intended and update meaning/derivation rules and/or synonym entries accordingly. For example, the character sequence “http” may indicate that the character string is a URL. Accordingly, as new words, or new meanings for existing words, are used, a patterns for what meaning is intended are discovered, the vocabulary modeling service  410  may update the agent-specific vocabulary model  412  to incorporate these changes. In some embodiments, the vocabulary modeling service  410  may update the base meaning repository  400  with new words and/or meanings. Accordingly, over time, as new words, or new meanings for existing words, are used, the agent-specific vocabulary model  412  may evolve such that the agent is more suited to its specific application. 
     As previously discussed, when the meaning extraction subsystem  150  notices a new word or a new meaning for an existing word, the meaning extraction subsystem  150  notifies the vocabulary modeling service  410  and provides the new word and/or meaning to the vocabulary modeling service  410 , as well as the context in which the new word and/or meaning was used. In some embodiments, the structure subsystem  172  may be used to analyze the context in which the new word and/or meaning were used. For example, as shown in  FIG. 18 , the structure subsystem  172  may include an ontology service  414  and a structure service  416 . The ontology service  414  may access the lexical database  408 , which may store metadata for words reflecting different possible forms of the word (e.g., noun, verb, adjective, etc.). The structure service  416  performs linguistic structure extraction (e.g., parsing the structure, tagging parts of speech, etc.) and may assist the ontology service  414  in disambiguation by analyzing the context of the new word and/or meaning. Accordingly, the ontology service  414  and the structure service  416  may work in concert, using data from the lexical database  408 , to analyze the use of the word, the context of the word&#39;s use, and determine what meaning of the word was intended. 
     As previously described, the prosody subsystem  174  analyzes the prosody of the utterance using a combination of rule-based and ML-based prosody plugins. Specifically, the prosody subsystem  174  analyzes the utterance for prosody cues, such as rhythm (e.g., speech rhythm, segmentations indicated by punctuation or pauses), emphasis (e.g., capitalization, bolding, underlining, asterisks), focus or attention (e.g., repetition of particular terms or styles), and so forth, which can be used to determine, for example, boundaries between intents, degrees of urgency or relative importance with respect to different intents, and so forth. Accordingly, the prosody subsystem  174  can combine or select from prosody parsed structures output by the various prosody plug-ins to help generate the meaning representations  418 . 
     The agent specific vocabulary model  412  may be developed using a collection of vocabulary model templates.  FIG. 19  is a diagram illustrating what may be included in a vocabulary model template  440 . As shown, the vocabulary model template  440  may include fields for base vector space  442 , constructed pattern synonyms  444 , constructed word synonyms  446 , context processing rules  448 , null word rules  450 , and multi-vector algorithms  452 . 
     The base vector space  442  may include data from the base meaning repository  400  and/or the databases  402 . Accordingly, the data may include one or more word vectors (e.g., a vector for each respective meaning), as well as data corresponding to word usage and methods for vector learning and/or derivation. As previously described, the base vector space  442  may be derived from some available corpus of data (e.g., one or more websites, or some other collection of writing) and act as a basis for subsequent modifications to the meanings of the word. 
     The constructed pattern synonyms  444  include one or more phrases or collections of words that may act as synonyms for the word in question or otherwise have the same or similar meanings as the word in question. Similarly, the constructed word synonyms  446  include words that may act as synonyms for the word in question or otherwise have the same or similar meanings as the word in question. Both the constructed pattern synonyms  444  and the constructed word synonyms  446  may be generated and/or maintained by the vocabulary modeling service  410  over time. In some embodiments, the constructed pattern synonyms  444  and the constructed word synonyms  446  include word vectors for the word synonyms and the pattern synonyms. 
     The context processing rules  448  include rules generated by the vocabulary modeling service  410  for how to process the context in which the word is used. The context processing rules  448  may be developed by the vocabulary modeling service  410  as new uses and/or meanings for words, or new words, are used in utterances. For example, the vocabulary modeling service may develop rules for determining when the word “return” is intended as a noun (e.g., “I submitted my tax return today”) or a verb (e.g., “I would like to return this pair of shoes that I bought”). The context processing rules  448  may be used to determine which of the known meanings for a word is intended, as well as how to determine an intended meaning for a word that does not comport with one of the known meanings for a word. Further, the context processing rules  448  may be used to determine an intended meaning for a new word based on context. In addition to generating new rules or modifying existing rules, the context processing rules  448  may also include combining multiple rules to process context of word usage. 
     The null word rules  450  include rules for determining meanings for words that cannot otherwise be determined via disambiguation and/or other word vector techniques based on data in the base meaning repository  400 , context, etc. For example, the null word rules  450  may include rules for deriving a word vector for a word based on the surrounding words. For example, if the base word vector database was learned via contextual approaches (i.e., learning a new word or a new meaning for an existing word based on the words that frequently surround it or predicting what words typically surround a word, or any other statistical method measuring co-occurrence of a word and its surrounding context), then the null word placeholder can be generated based on the word vectors of the surrounding words in an utterance. Alternatively, a model can be trained to directly generate word vectors given known word-surface-form-as-an-ordered-collection-of-characters to vector mappings (e.g., use a pre-existing word vector database as training data to derive an ML model that can be consulted to generate word vectors given an ordered collection of characters). 
     The multi-vector aggregation algorithms  452  include one or more algorithms for deriving a single word vector from a collection of word vectors. For example,  FIG. 20  illustrates how a multi-vector aggregation algorithm  452  combines multiple word vectors into a single sub-phrase vector  470 . As shown in  FIG. 20 , the vocabulary model  412  of the vocabulary subsystem  170  provides word vectors for each node  202  of an annotated utterance tree  166 . For the illustrated embodiment, the vocabulary subsystem  170  generated four or more word vectors, represented as V 1 , V 2 , V 3 , and V 4 , which are respectively associated with four nodes of the annotated utterance tree  166 . That is, in certain embodiments, the NLU framework  104  may modify the annotated utterance tree  166  (e.g., the vocabulary subsystem  170  may replace phrases including multiple word vectors with individual words or sub-phrases having single word vectors). As such, it is appreciated that, at one or more stages of intent/entity extraction, the number of nodes/subtrees of the annotated utterance tree  166  may be increased or decreased, along with the number of word vectors combined to calculate the sub-phrase vector  470 , relative to an original utterance or an initially generated annotated utterance tree  166 . In other embodiments, the multi-vector aggregation algorithm  452  may be used to generate a single word vector  470  from a group of synonyms by using weighted average. 
     As shown in  FIG. 20 , the multi-vector aggregation algorithm  452  generates the sub-phrase vector  470  by multiplying each of the word vectors by a respective focus/attention/magnification (FAM) coefficient  472  (e.g., α, β, γ, δ) associated with the word vector. The FAM coefficients  472  are used to tune how much relative focus or attention (e.g., signal magnification) should be granted to each portion (e.g., node) of a subtree when generating a sub-phrase vector  470 . Accordingly, the FAM coefficients  472  increase or decrease the contribution of each word vector to the combined sub-phrase vector  470 . After applying the FAM coefficients  472  to the word vectors V 1-4 , the results are combined using vector addition, as indicated by the “+” notation in  FIG. 20 . Additionally, for the illustrated embodiment, the resulting sub-phrase vector  470  is subsequently normalized to ensure that the dimensions of the combined sub-phrase vector  470  are each within a suitable range after the multiplication and addition operations. It may be noted that the tree substructure vectorization algorithm  452  illustrated in  FIG. 20  is merely provided as an example, and in other embodiments, other suitable tree substructure vectorization algorithms may be used, in accordance with the present disclosure. In some embodiments, the vector aggregation may be performed iteratively (e.g., via class-level component comparisons). In other embodiments, a general signal for all nodes in the tree or sub-tree may be requested and used for vector aggregation. 
     By way of example, in certain embodiments, verb words or subtrees may be associated with one of the FAM coefficients  472  (e.g., α) that is greater in value than another FAM coefficient (e.g., β) associated with a subject or direct object word or subtree vector. In certain embodiments, root node word vectors may be associated with a relatively higher FAM coefficient  472  than word vectors associated with other nodes. In certain embodiments, the combined sub-phrase vector  470  is a centroid that is calculated as the weighted average of the word vectors associated with all nodes of the subtree. In other embodiments, the meaning extraction subsystem  150  may recursively perform vector aggregation. In certain embodiments, one or more of the vectors (e.g., V 1 , V 2 , V 3 , and V 4 ) that are used to generate the combined sub-phrase vector  470  may itself be a combined sub-phrase vector  470  that is generated from other underlying word and/or subtree vectors. For such embodiments, subtrees with at least one depending node (e.g., non-leaf nodes/subtrees) may be associated with a higher FAM coefficient value than single-node (e.g., a leaf nodes/subtrees). 
     Returning to  FIG. 7 , as the virtual agent communicates with users (e.g., receives, processes, and responds to utterances), data may be generated and collected to further train the system to learn new words and/or refine word understandings. For example, the agent automation system  100  can continue to learn or infer meaning of new words and phrases. It is presently recognized that this can enable the agent automation system  100  to have a continuously expanding/adapting vocabulary capable of accommodating the use of unfamiliar words, as well as changes to the meaning of familiar words. For example,  FIG. 21  is a flow diagram illustrating an embodiment of a process  490  whereby the agent automation system  100  continuously improves a word vector distribution model  492 , which may be plugged into the vocabulary subsystem  170  of the meaning extraction subsystem  150 , such as the learned multimodal word vector distribution model  178  or the learned unimodal word vector distribution model  180  discussed above with respect to  FIG. 7 . As such, it is appreciated that, by expanding or modifying the word vector distribution model  492 , operation of the vocabulary subsystem  170 , the NLU framework  104 , and the agent automation system  100  can be improved to handle words with new or changing meanings using only training data that can be generated from a continually growing corpus of utterances  112  of the database  106  illustrated in  FIG. 4A . For the embodiment illustrated in  FIG. 21 , the corpus of utterances  112  may be, for example, a collection of chat logs  494  storing user utterances  122  and agent utterances  124  from various chat room exchanges, or other suitable source data. 
     For the embodiment illustrated in  FIG. 21 , prior to operation of the agent automation system  100 , the word vector distribution model  492  may initially be generated based on a first corpus of utterances that have a particular diction and vocabulary, such as a set of books, newspapers, periodicals, and so forth. However, it is appreciated that many utterances exchanges in different conversational channels (e.g., chat rooms, forums, emails) may demonstrate different diction, such as slang terms, abbreviated terms, acronyms, and so forth. With this in mind, the continual learning loop illustrated in  FIG. 21  enables the word vector distribution model  492  to be modified to include new word vectors, and to change values of existing word vectors, based on source data gleaned from the growing collections of user and agent utterances  122  and  124 , to become more adept at generating annotated utterance trees  166  that include these new or changing terms. 
     The process  490  illustrated in  FIG. 21  includes receiving and responding to the user utterance  122 , as discussed above with respect to  FIG. 5 . As mentioned, the user utterances  122  and the agent utterances  124  can be collected to populate the corpus of utterances  112  (e.g., chat logs  494 ) stored in the database  106 , as illustrated in  FIG. 4A . As some point, such as during regularly scheduled maintenance, the prosody subsystem  174  of the meaning extraction subsystem  150  segments (block  496 ) the chat logs  494  into distinct utterances  498  that are ready for analysis. Then, in block  500 , the meaning extraction subsystem  150  performs rule-augmented unsupervised learning to generate a refined word vector distribution model  502  containing new or different word vectors  504  generated from the segmented utterances  498 . 
     For example, as discussed above, the meaning extraction subsystem  150  may analyze the set of segmented utterances  498  and determine word vectors  504  for the words of these utterances based on how certain words tend to be used together. For such embodiments, two words that are frequently used in similar contexts within these utterances  498  are considered closely related and, therefore, are assigned a similar vector value (e.g., relatively closer in terms of Euclidean distance) in one or more dimensions of the word vectors  504 . In this manner, the meaning extraction subsystem  150  may adapt to changes in the meaning of a previously understood term based on new context in which the term is used. Accordingly, the agent automation system  100  may parse the chat logs  494  to evaluate how a word or phrase is used in the chat logs  494  and determine whether the usage is represented by one or more associated word vectors  504  of the word vector distribution model  502 . This may include, for example, considering the context in which the word or phrase is used to determine the intended meaning of the word, as described below with regard to  FIG. 22 . If not, the word vector distribution model  502  may be updated to add one or more new word vectors  504  representative of the new meaning, or replace one or more existing vectors  504  to match the new meaning. 
     As illustrated in  FIG. 21 , the refined word vector distribution model  502  is used to replace the existing word vector distribution model  492 , such that the vocabulary subsystem  170  can use this refined model to provide word vectors for the words and phrases of new user utterances  122  received by the agent automation system  100 . For example, an initial word vector distribution model  492  may have a word vector for the term “Everest” that is relatively close in one or more dimensions to other word vectors for terms such as, “mountain”, “Himalayas”, “peak”, and so forth. However, when a client creates a new conference room that is named “Everest,” the term begins to be used in a different context within user utterances  122 . As such, in block  500 , a new word vector would be generated for the term “Everest” that would be relatively close in one or more dimensions to word vectors for terms such as “conference”, “meeting”, “presentation”, and so forth. After updating the word vector distribution model  502 , upon receiving a user utterance  122  having the revised term “Everest,” the operation of the vocabulary subsystem  170 , the NLU framework  104 , and the agent automation system  100  is improved to more provide more accurate word vectors, annotated utterance trees, and meaning representations, which result in more accurately extracted intents/entities  140 . 
     As previously described, a word or phrase may be associated with multiple word vectors  504  associated with different meanings of the word. Using the “Everest” example from above, when the term “Everest” is used in a user utterance  122 , the user could have intended Mount Everest, the conference room named Everest, or some other meaning. Accordingly, the NLU framework  104  may analyze user utterances  122  to determine which word vectors  504  were intended when a word or phrase appears in an utterance  122 .  FIG. 22  is a flow chart illustrating a process  520  for receiving a user utterance  122 , determining which meanings of one or more words or phrases  522  appearing in the utterance  122  were intended, and outputting one or more associated semantic word vectors  524 . At block  526 , the utterance  122  is parsed and segmented into words and/or phrases  522 . This may include, for example, parsing the utterance  122  and generating an annotated utterance tree as shown and described above with regard to  FIGS. 8 and 9 , wherein each word or phrase  522  is represented by a node. 
     At block  528 , each word/phrase  522  is pre-processed. Pre-processing may include, for example, applying pre-processing directives or instructions from the vocabulary model  482 . These directives or instructions may provide guidance for checking spelling, correcting formatting issues, expanding contractions, expanding abbreviations, replacing acronyms with associated words, as well as other data-cleansing processes. 
     At decision  530 , the system determines whether there is any word context available. If word context is available, the process  520  proceeds to block  532  and performs context-based disambiguation. The surrounding words and/or phrases  522  in the utterance  122  may provide context for determining what meaning of the word or phrase  522  in question was intended. In some embodiments, the ontology service  414  and/or the structure service  416  may be utilized to perform the context-based disambiguation. Returning to the “Everest” example, if the utterance  122  is “I&#39;m not sure, but we have a meeting scheduled in Everest at 2:30 pm this afternoon to discuss what to do”, the other words  522  in the utterance  122  may be used to determine that the use of the word “Everest” in the utterance  122  was referring to the conference room. For example, the words “meeting” and reference to a time just a couple of hours in the future, with Mount Everest being thousands of miles away, may signal that Everest the conference room was intended, instead of Mount Everest. However, if the utterance had been “we&#39;re just going to Santa Cruz for the weekend, it&#39;s not like we&#39;re traveling to go climb Everest”, the other words  522  in the utterance  122  may be used to determine that the use of the word “Everest” in the utterance  122  was referring to Mount Everest and not the conference room. For example, the words “climb”, “traveling” and reference to various destinations, may signal that Mount Everest was intended, instead of Everest the conference room. Once the context-based disambiguation has been performed, word vectors are selected based on the context and extracted. 
     However, if the utterance  122  in question only includes a single word or phrase  522 , the system may determine that there is no context available and proceed to extract one or more vectors matching the surface form or form derivatives (block  534 ). For example, word vectors  484  for the word or phrase  522  may be selected from the vocabulary model  482  based solely on the surface form used in the utterance  122 , or form derivatives. Alternatively, if the utterance  122  in question only includes a single word or phrase  522 , the system may refer to other utterances that precede or proceed the utterance in question to determine whether any context is available. For example, the single word or phrase  522  of the utterance  122  may be a single word or phrase answer to a question. As such, considering the question may provide context as to what was meant by the single word or phrase  522  of the utterance  122 . 
     If there are no word vectors associated with the word or phrase  522 , then no word vectors are extracted. At decision  536 , the process  520  determines whether a vector list has been found (i.e., were any word vectors extracted?). If vectors were extracted in blocks  532  and/or  534 , then the process  520  proceeds to block  538  and post-processes the extracted vectors, resulting in the output semantic word vectors  524 . For example, the post-processing may include extracting a representative vector or vector set given one or more synonymic vector lists generated during blocks  532  and  534 . 
     However, if no vector word lists have been extracted in blocks  532  and  534 , the process  520  proceeds to block  540  and uses null-word rules (e.g., as described with regard to  FIG. 19 ) received from the vocabulary model  482  to generate word vectors  524  for the word or phrase  522 . In some embodiments, when null word rules are used to derive semantic vectors for a word or phrase, the unknown word or phrase may be flagged as an unknown word for future learning, and/or input may be requested from a user to help define the word or phrase. The generated word vectors  524  may then be post-processed (block  538 ) as described above. The NLU framework  104  may then insert the semantic vectors  524  output by the process  520  shown in  FIG. 22  into the annotated utterance tree and use the annotated utterance tree to determine the intent of the utterance  122  and generate a response. In some embodiments, learning may be triggered by one or more conditions. For example, in some embodiments, learning may be triggered by the magnitude of collected data (e.g., when chat logs reach a threshold size). In other embodiments, an unknown word or an unknown meaning for a known word being used a threshold number of times within some window of time may trigger learning. In some embodiments, learning may take place on a scheduled basis (e.g., weekly, monthly, quarterly, annually, etc.). It should be understood, however, that in some embodiments, learning may be triggered by one or more of multiple possible conditions. 
     Technical effects of the present section of the disclosure include a virtual agent that is capable of learning new words, or new meanings for known words, based on exchanges between the virtual agent and the user in order to customize the vocabulary of the virtual agent to the needs of the user or users. The agent automation framework may have access to a corpus of previous exchanges between the virtual agent and the user, such as one or more chat logs. The agent may segment the chat logs into utterances using the prosody subsystem, and then further segment the utterances into words and/or phrases. The agent automation framework may then recognize when new words and/or new meanings for known words appear in user utterances. New word vectors may be generated for these new words and/or new meanings for known words. The new word vectors may then be added to an existing word vector distribution model to generate a refined word vector distribution model. The new word vector may be generated, for example, based on the context in which the new word or meaning was used over one or more uses in the chat logs, input from a user, or some other source. The NLU framework may then utilize the refined word vector distribution model to interpret and analyze user utterances and generate responses. 
     When determining the intended meaning for a word used in an utterance that has multiple different meanings and multiple different respective word vectors, the agent automation framework segments the utterance into words and/or phrases. If word usage context is available, the agent automation framework may determine which meaning was intended by performing context-based disambiguation via the ontology service and/or the structure service. If no context is available, the agent automation framework may extract word vectors matching the surface form or form derivatives. If no word vectors are found, the agent automation framework derives semantic word vectors according to null-word rules. As time passes and the virtual agent exchanges utterances with the user, the virtual agent learns new words, or new meanings for known words, and thus customizes its vocabulary to its specific application and users. 
     Templated Rule-Based Data Augmentation for Intent Extraction 
     It is presently recognized that there is a need to improve the ability of virtual agents to apply NLU techniques to properly derive meaning from complex natural language utterances. For example, it may be advantageous to create a virtual agent capable of comprehending complex language and executing contextually relevant requests, which could afford substantial advantages in terms of reduced operational cost and increased responsiveness to client issues. Additionally, it is recognized that it is advantageous for virtual agents to be customizable and adaptable to various communication channels and styles. 
     Additionally, in the context of NLU and AI, it is recognized that data augmentation can add value to base data by adding information derived from internal and external sources within an enterprise. For example, data augmentation can help reduce manual intervention involved in developing meaningful information and insight from business data, as well as significantly enhance data quality. There are many approaches to augment data, for instance, adding noise or applying transformations on existing data and simulation of data. However, it is presently recognized that data augmentation can become increasingly challenging as the complexity of data increases. With this in mind, there appears to be a need to improve methods of applying semantic techniques for data augmentation within a NLU framework. 
     In present embodiments, a meaning representation can be generated from an annotated utterance tree structure having a form or shape that represents the grammatical structures of the utterance, and having nodes that each represent words or phrases of the utterances as word vectors encoding the semantic meaning of the utterance. In particular, the meaning extraction subsystem of the disclosed NLU framework includes a vocabulary subsystem, a structure subsystem, and a prosody subsystem that cooperate to parse utterances (e.g., received user utterances, sample utterances of the intent/entity model) into the annotated utterance trees based on combinations of rule-based methods and machine learning (ML)-based (e.g., statistical) methods. 
     The disclosed NLU framework also includes a model augmentation subsystem capable of performing rule-based augmentation to augment a model (e.g., the utterance meaning model and/or the understanding model) by generalizing and/or refining the model. For example, the disclosed NLU framework is capable of expanding a number of meaning representations in the model based on stored generalizing rules, wherein the additional meaning representations are re-expressions of the original meaning representations of the model. The disclosed NLU framework is also capable of refining the meaning representations of these models, for example, to prune substantially similar meaning representations from the model based on stored refining rules. Additionally, refining may also include applying substitution rules that modify meaning representations by replacing one word surface or phrasal form with another that may be more common in a given conversational style, discourse, or channel. As such, the disclosed technique enables the generation of an augmented understanding model and/or augmented utterance meaning model having generalized and/or refined meaning representations. By expanding and/or refining the understanding model and/or the utterance model, the disclosed augmentation techniques enable the NLU framework and the agent automation system to be more robust to variations and idiosyncrasies in discourse styles and to nuances in word surface form and usage. The disclosed techniques can also improve the operation of the NLU framework and agent automation system by reducing or optimizing processing and memory resource usage when deriving meaning from natural language utterances. 
     Once the meaning representations  158  and  162  have been generated, as illustrated in  FIG. 6 , the meaning search subsystem  152  can compare these meaning representations to extract intent/entities from the user utterance  122 . However, in certain embodiments, the NLU framework  104  may first augment the utterance meaning model  160  and/or the understanding model  157  to improve the operation of the meaning search subsystem  152  to extract the intents/entities  140 . For example,  FIG. 23  is a flow diagram illustrating an embodiment of an augmentation process  600  whereby the NLU framework  104  augments a model  602  that includes one or more meaning representations  604 . It should be noted that, in certain embodiments, the augmentation process  600  may be separately performed on the meaning representations  162  to augment the utterance meaning model  160 , or the meaning representations  158  to augment the understanding model  157 , or a combination thereof. For clarity, prior to augmentation, the model  602  may be referred to herein as an “original” model, and the one or more meaning representations  604  may be referred to herein as “original” meaning representations  604 . The augmentation process  600  may be executed as part of a model augmentation subsystem  606 , or another suitable subsystem, of the NLU framework  104 . Additionally, the model augmentation subsystem  606  may cooperate with other subsystems (e.g., the vocabulary subsystem  170 , the structure subsystem  172 , and/or the prosody subsystem  174 ) of the NLU framework  104  to perform the augmentation process  600 , as discussed below. 
     The embodiment of the augmentation process  600  illustrated in  FIG. 23  begins with model augmentation subsystem  606  performing (block  608 ) a rule-based generalization of the model  602 , which may be generated as discussed above. For example, based on one or more stored generalizing rule-sets  610 , model augmentation subsystem  606  generates a plurality of generalized meaning representations  612  for at least a portion of the original meaning representations  604  of the model  602 . As such, after the generalization step of block  608 , the model  602  is expanded to include the generalized meaning representations  612 , and the resulting model may be referred to herein as a generalized model  614  (e.g., a generalized utterance meaning model or a generalized understanding model). The generalized meaning representations  612  may be different structural permutations that are re-expressions of original meaning representations  604  and the underlying utterance. In general, the purpose of generalization is to expand the original model  602  to include additional forms related to the original meaning representations  604  already present in the model. It should be noted that, in certain embodiments, the augmentation process  600  only includes the generalization step of block  608 . For such embodiments, the augmentation process  600  concludes at block  608  and the generalized model  614  serves as the augmented model (e.g., an augmented meaning model or an augmented understanding model) that is used for the subsequent meaning search operation. 
     For the embodiment illustrated in  FIG. 23 , the augmentation process  600  continues with the model augmentation subsystem  606  performing (block  616 ) a rule-based refinement of the generalized model  614  generated in block  608 . For example, based on one or more stored refining rule-sets  618 , the model augmentation subsystem  606  modifies or eliminates certain meaning representations (e.g., original meaning representations  604  and/or generalized meaning representations  612 ) of the generalized model  614 . In general, the purpose of refinement is to adjust and focus the generalized model  614  to improve performance (e.g., improve domain specific performance, and/or reduce resource consumption) of the subsequent search operation by the meaning search subsystem  152 . Upon completion of the rule-based refinement step of block  616 , meaning representations  604  and/or  612  of the generalized model  614  are refined (e.g., modified and/or pruned) to generate refined meaning representations  620 , and the resulting model may be referred to herein as a refined model  622  (e.g., a refined utterance meaning model or a refined understanding model). For the illustrated embodiment, after both the generalization step of block  608  and the refining step of block  616 , the resulting refined model  622  may be referred to as the augmented model (e.g., the augmented utterance meaning model or the augmented understanding model) that is used for the subsequent meaning search operation. It may be appreciated that, in certain embodiments, the generalization step of block  608  may be skipped, and augmentation process  600  may include performing the rule-based refinement of block  616  on the original meaning representations  604  of the original model  602  to generate the augmented model. Once the augmentation process  600  is complete, the meaning search subsystem  152  can instead use the augmented model (e.g., a generalized model, a refined model, or a generalized and refined model) as the utterance meaning model  160  or the understanding model  157  when extracting intent/entities from the user utterance  122 , as illustrated and discussed with respect to  FIG. 6 . 
     The aforementioned generalizing rule-sets  610  and refining rule-sets  618  generally define how the augmentation process  600  is performed to generate the augmented model.  FIG. 24  illustrates an embodiment of a model augmentation template  640  that stores these generalizing rule-sets  610  and refining rule-sets  618 , as well as model applicability criteria  642  that defines when and how these rule-sets are to be applied. In certain embodiments, the model augmentation template  640  may be stored as one or more tables of the database  106  illustrated in  FIGS. 4A and 4B , or within another suitable data structure, in accordance with the present disclosure. 
     For the embodiment illustrated in  FIG. 24 , the model augmentation template  640  defines each rule of the generalizing rule-sets  610  and the refining rule-sets  618  based on particular model applicability criteria  642 . That is, for the illustrated embodiment, particular rules of the generalizing rule-sets  610  and the refining rule-sets  618  can be selectively applied to or executed against certain meaning representations having particular characteristics, as defined by particular model applicability criteria  642 . For example, the corresponding model applicability criteria  642  can indicate that particular generalizing rule-sets  610  and/or refining rule-sets  618  should only be applied to certain types of models (e.g., the utterance meaning model  160  or the understanding model  157 ), or to certain meaning representations (e.g., having a particular form or shape, having particular nodes, having particular classes of nodes, having particular word vectors, having particular subtree vectors, and so forth). As such, the model applicability criteria  642  can include language-specific conditions, discourse-style conditions, and so forth, which govern when and how these rule-sets should be applied during the augmentation process  600 . For the illustrated example, as indicated by the checkboxes  644  (e.g., checkboxes  644 A,  644 B,  644 C,  644 D,  644 E, and  644 F), only a portion of the generalizing rule-sets  610  and the refining rule-sets  618  have been activated for the augmentation process  600 . 
     For the illustrated embodiment, the generalizing rule-sets  610  include subject/object rules  646 , a passive/active rules  648 , as well as other suitable generalizing rules  650 . For example, the subject/object rules  646  may include a rule that consumes a meaning representation of a model and, from it, generates an alternative form of the meaning representation in which a subject and an object of the meaning representation (and of the underlying utterance) are interchanged. By way of specific example, one of the subject/object rules  646  may generate a meaning representation corresponding to the utterance, “She sent him the item” from a meaning representation corresponding to the utterance, “She sent the item to him.” Similarly, the passive/active rules  648  may include a rule that consumes a meaning representation of a model and, from it, generates an alternative form of the meaning representation that has been converted from an active form to a passive form, or from a passive form to an active form. By way of specific example, the passive/active rule  648  may generate a meaning representation corresponding to the utterance, “I sent him the item” from a meaning representation corresponding to the utterance, “The item was sent to him.” The other generalizing rules  650  may include any other grammatical rearrangement or transformation that generates a meaning representation having a similar (e.g., the same or closely related) meaning relative to an original meaning representation already present in the model. 
     For the illustrated embodiment, the refining rule-sets  618  include substitution rules  652 , pruning rules  654 , and any other suitable refining rules  656 . For example, the one of the substitution rules  652  may define how constituent portions (e.g., nodes, subtrees, word vectors, subtree vectors) of a meaning representation of a model should be replaced with other substitute portions prior to performing the search operation. For example, one of the substitution rules  652  may identify two phrases having similar surface forms and different meanings, and substitute a portion (e.g., a subtree) of the original meaning representation representing the first phrase with a different portion (e.g., a replacement subtree) representing the second phrase. For example, the substituted structure may include fewer nodes or more nodes relative to the original meaning representation. As such, using substitution rules  652 , certain word surface forms (e.g., words, acronyms, expressions, emojis, and so on) can be replaced with other word surface forms or phrasal forms that are more common in a given conversation style, discourse, and/or domain. As such, it should be appreciated that the substitution rules  652 , as well as other refining rules  656  expressed in the model augmentation template  640 , can be used to capture local conversation style or subject vertical idiosyncrasies, as well as address nuances in word surface form, for example, in cases involving polysemy or other word-usage nuances. 
     By way of particular example, individually, the meanings of the words “look” and “up” are substantially different from the resulting meaning when used in combination (“look up”). As such, in an embodiment, a substitution rule  652  may locate every representation of the word “look” that is associated with the word “up” within the meaning representations of a model, and then substitute the corresponding structure with suitable structure (e.g., nodes, subtrees, word vectors, subtree vectors) that instead represent the term “look-up” or “search”. For this example it may also be appreciated that, when performing comparisons during the later meaning search operation, “search” may be represented by a single tree node, while “look-up” may be represented by multiple tree nodes. As such, in certain cases, the substitution rule  652  can reduce the number of comparison operations and yield better match scores during the subsequent meaning search operation. However, it may be noted that, in certain embodiments, rather than locate and substitute specific words or phrases represented with the meaning representation, the substitution rule  652 , as well as other rules defined in the model augmentation template  640 , may instead rely on the shape of the meaning representation (e.g., the grammatical structure of the represented sentence or phrase) when applying generalizing and/or refining linguistic-rule-based manipulation, as set forth herein. 
     For the refining rule-set  618  illustrated in  FIG. 24 , the pruning rules  654  generally improve efficiency and reduce redundancy by removing sufficiently similar meaning representations from a model. As such, the pruning rules  654  may include a rule that defines a threshold level of similarity (e.g., in terms of similarity in structure, word vectors, and/or subtree vectors) that is acceptable between two meaning representations of a model, as well as which of the two meaning representations should be culled from the model. By way of particular example, one of the pruning rules  654  may identify two meaning representations (e.g., an original meaning representation and a generalized meaning representation, or two generalized meaning representations) that differ in limited ways (e.g., only in leaf nodes or in modifier nodes). In response, the rule may remove one of the two meaning representations from the model, reducing redundancy in the model and improving performance during the subsequent search operation. That is, by reducing the number of meaning representation of the utterance meaning model  160  and/or the understanding model  157 , the memory footprint and the number of comparison operations of the subsequent meaning search operation can be substantially reduced, improving the performance of the NLU framework  104  and the agent automation system  100 . 
       FIG. 25  provides another depiction of the augmentation process  600  whereby one or more original meaning representations  604  of the original model  602  (e.g., meaning representations  162  of the utterance meaning model  160  or meaning representations  158  of the understanding model  157 ) are generalized and/or refined to yield an augmented model  680  having augmented meaning representations  682  (e.g., generalized and/or refined meaning representations). As set forth above, during rule-based generalization (block  608 ), the model augmentation subsystem  606  of the NLU framework  104  cooperates with the structure subsystem  172  of the NLU framework  104  to generate alternative forms of at least a portion of the meaning representations  604  based on one or more active generalization rule-sets  610 . For the illustrated embodiment, the rule-based generalization of block  608  includes selectively applying subject/object rules  646 , passive/active rules  648 , and/or other suitable generalization rules  650  based on the model applicability criteria  642  that corresponds to these generalization rules. As such, the rule-based generalization of block  608  illustrates a single meaning representation  604 A, which meets certain model applicability criteria  642  defined for one or more of the generalizing rule-sets  610 , and which is used to generate at least generalized meaning representations  612 A,  612 B, and  612 C based on the corresponding generalizing rule-sets  610 . 
     For the embodiment illustrated in  FIG. 25 , during rule-based refining (block  616 ), the model augmentation subsystem  606  of the NLU framework  104  modifies the original meaning representation  604 A of the model  602  based on one or more active refining rule-sets  618 . In particular, for the illustrated embodiment, the model augmentation subsystem  606  cooperates with the vocabulary subsystem  170  of the NLU framework  104  to replace word vectors and subtree vectors associated with portions of the original meaning representation  604 A with alternative word vectors and subtree vectors based on one or more active refining rule-sets  618 . For the illustrated embodiment, the rule-based refining of block  616  includes applying substitution rules  652  and/or other suitable refining rules  656  based on the model applicability criteria  642  that corresponds to these refining rules. As such, the rule-based refinement of block  616  illustrates the original meaning representation  604 A, which meets certain model applicability criteria  642  defined for one or more of the refining rule-sets  618 , being used to generate the refined meaning representation  620 A based on the corresponding refining rule-sets  618 . In other embodiments, in block  616 , the model augmentation subsystem  606  also removes or prunes one or more of the meaning representations of the model  602  (e.g., original meaning representations  604 A, generalized meaning representations  612 A,  612 B,  612 C) based on one or more active refining rule-sets  618 . As mentioned, in certain embodiments, the steps of block  608  or  616  may be skipped, resulting in the augmented model  680  being only generalized or refined relative to the original model  602 . 
     For embodiment illustrated in  FIG. 25 , relative to the original model  602 , the augmented model  680  generally includes an expanded number of meaning representations  682 , a reduction in redundancy within meaning representations  682 , and/or an improvement in domain specificity. As such, when the original model  602  is the understanding model  157 , then the augmented model  680  enables an expanded and/or refined search space for the subsequent meaning search operation. When the original model  602  is the utterance meaning model  160 , then the augmented model  680  may be described as an expanded and/or refined search key for the subsequent meaning search operation. As such, by augmenting the utterance meaning model  160  and/or understanding model  157 , the meaning search subsystem  152  is more likely to correctly extract the intents/entities  140  from received user utterances  122 , as discussed above with respect to  FIG. 6 . 
       FIG. 26  is a flow diagram illustrating an embodiment of a generalization process  700  whereby the model augmentation subsystem  606  of the NLU framework  104  performs rule-based generalization of the original meaning representations  604  of the original model  602 . As mentioned, the original model  602  may be the utterance meaning model  160 , including meaning representations  162 , or the understanding model  157 , including meaning representations  158 , as illustrated in  FIG. 6 . The example generalization process  700  of  FIG. 26  may be executed by the model augmentation subsystem  606  or another suitable subsystem of the NLU framework  104 , and may involve the cooperation of other components (e.g., the structure subsystem  172  and the prosody subsystem  174 ) of the NLU framework  104 . For the illustrated embodiment, the generalization process  700  receives the model  602  as an input, as well as the compilation model template  244  and the model augmentation template  640  discussed above. 
     The embodiment of the generalization process  700  illustrated in  FIG. 26  begins with the model augmentation subsystem  606  identifying (block  702 ) intent subtrees for each of the meaning representations  604  of the model  602 , wherein each intent subtree represents a distinct (e.g., atomic) intent of a particular meaning representation (as well as the underlying utterance). For example, in certain embodiments, the model augmentation subsystem  606  may invoke the prosody subsystem  174  to use one or more stored rules to segment the meaning representations  604  into distinct intent subtrees. Once all of the intent subtrees have been identified, the generalization process  700  includes an outer “for loop” (indicated by block  704 ) in which each of the intent subtrees are individually, serially processed. Within the outer “for loop” of block  704 , there is an inner “for loop” (indicated by block  706 ) in which each of the activated rules of the generalizing rule-set(s)  610  of the model augmentation template  640  are conditionally or selectively applied, based on the corresponding model applicability criteria  642 . In other words, the “for loops” associated with blocks  704  and  706  conditionally applies each activated rule of the generalizing rule-sets  610  to each intent subtree of the original meaning representations  604  of the model  602  as part of the generalization process. 
     Within the inner “for loop” indicated by block  706 , the generalization process  700  includes the model augmentation subsystem  606  determining (decision block  708 ) whether the current activated generalizing rule (e.g., one of the subject/object rules  646  or passive/active rules  648 ) is applicable to the current intent subtree based on the model applicability criteria  642  of the model augmentation template  640 , as discussed above with respect to  FIG. 24 . When the model augmentation subsystem  606  determines, based on the model applicability criteria  642 , that the current rule does not apply, then it may proceed to the next rule in the “for loop” of block  706 , as indicated by the block  710 . When the model augmentation subsystem  606  determines, based on the model applicability criteria  642 , that the current rule is applicable to the current intent subtree, the model augmentation subsystem  606  generates (block  712 ) one or more generalized meaning representations  612  from the current intent subtree based on the current generalization rule. 
     For the embodiment illustrated in  FIG. 26 , the generalization process  700  continues with the model augmentation subsystem  606  determining (decision block  714 ) whether a user should review and provide input to validate the generalized meaning representations  612  generated in block  712 . For example, the model augmentation subsystem  606  may check the current generalization rule within the model augmentation template  640  to determine whether user validation and input should be sought. When the model augmentation subsystem  606  determines that user input should be requested, it queues (block  716 ) the generalized meaning representations  612  generated in block  712  for later user validation. When the model augmentation subsystem  606  eventually receives a valid response from the user (decision block  718 ), or when the model augmentation subsystem  606  determines in decision block  714  that user input is not involved with the current rule, then the model augmentation subsystem  606  responds by updating (block  720 ) the model  602  to include the generalized meaning representations  612  generated in block  712 . In response to the model augmentation subsystem  606  receiving an invalid response from the user responding in decision block  718 , or in response to the model augmentation subsystem  606  updating the model in block  720 , the model augmentation subsystem  606  proceeds (block  710 ) to the next rule in the “for loop” of block  706 . 
     Once the NLU framework  104  has processed each of the active rules of the generalization rule-set  610 , the model augmentation subsystem  606  then proceeds to the next intent subtree of the “for loop” of block  704 , and then repeats the “for loop” of block  706 , which again conditionally applies each of the active model generalization rules-sets  610  against the next intent subtree based on the corresponding model applicability criteria  642 . Accordingly, the illustrate generalization process  700  continues until all intent subtrees identified in block  702  have been processed in this manner to expand the model  602  into the generalized model  614  (e.g., a generalized utterance meaning model or a generalized understanding model). As mentioned, in certain embodiments, the generalized model  614  undergoes a subsequent refining step as a part of the augmentation process  600 . However, in certain embodiments, the generalized model  614  produced by the generalization process  700  may not undergo a refining step, and the generalized model  614  serves as an augmented model  680  (e.g., an augmented utterance meaning model or an augmented understanding model) for the subsequent meaning search operation. In certain embodiments, the generalization process  700  is executed separately for both the utterance meaning model  160  and the understanding model  157  to expand both models prior to the meaning search operation. 
       FIG. 27  is a flow diagram illustrating an embodiment of a refinement process  740  whereby the model augmentation subsystem  606  performs rule-based refinement of the one or more meaning representations  604  of the model  602 . As mentioned, in certain embodiments, the meaning representations  604  and the model  602  may be original meaning representations of an original (e.g., non-generalized, non-expanded) model  602  (e.g., the utterance meaning model  160  or the understanding model  157 ). In other embodiments, the model  602  may be the generalized model  614  (e.g., a generalized utterance meaning model or generalized understanding model) that is the product of the generalization process  700  of  FIG. 26 . The example refinement process  740  may be executed by the model augmentation subsystem  606  or anther suitable subsystem of the NLU framework  104 , and may involve the cooperation of other components (e.g., the vocabulary subsystem  170 , the structure subsystem  172 , and the prosody subsystem  174 ) of the NLU framework  104 . For the illustrated embodiment, the refinement process  740  receives the model  602  as an input, as well as the compilation model template  244  and the model augmentation template  640  discussed above. 
     The embodiment of the refinement process  740  illustrated in  FIG. 27  begins with the model augmentation subsystem  606  identifying (block  742 ) unrefined intent subtrees for each of the meaning representations  604  of the model  602 , wherein each unrefined intent subtree represents distinct (e.g., atomic) intents of meaning representations that have not been processed by the refining rule-sets  618 . For example, in certain embodiments, the prosody subsystem  174  may use one or more stored rules to segment the meaning representations  604  into these intent subtrees. Once all of the unrefined intent subtrees have been identified, the refinement process  740  includes an outer “for loop” (indicated by block  744 ) in which each of the unrefined intent subtrees are individually processed. Within the outer “for loop” of block  744 , there is an inner “for loop” (indicated by block  746 ) in which each of the activated rules of the refining rule-sets  618  of the model augmentation template  640  is selectively applied based on the model applicability criteria  642 . In other words, the “for loops” associated with blocks  744  and  746  ensure that each activated rule of the refining rule-sets  618  can be conditionally or selectively applied to each intent subtree of the meaning representations  604  of the model  602  as part of the refinement process. 
     Within the inner “for loop” indicated by block  746 , the refinement process  740  includes the model augmentation subsystem  606  determining (decision block  748 ) whether the current activated refining rule is applicable to the current intent subtree based on the model applicability criteria  642  of the model augmentation template  640 , as discussed above with respect to  FIG. 24 . When the current rule does not apply, then the model augmentation subsystem  606  may proceed to the next refining rule in the “for loop” of block  746 , as indicated by block  750 . When the model augmentation subsystem  606  determines that the current refining rule is applicable to the current intent subtree, the model augmentation subsystem  606  applies (block  752 ) the current refinement rule to generate a refined meaning representation  620 . 
     For the embodiment illustrated in  FIG. 27 , the refinement process  740  continues with the model augmentation subsystem  606  determining (decision block  754 ) whether a user should review and provide input to validate the refined meaning representation  620  generated in block  752 . For example, the model augmentation subsystem  606  may check the current refinement rule within the model augmentation template  640  to determine whether user validation and input should be sought. When the model augmentation subsystem  606  determines that user input should be requested, it queues (block  756 ) the refined meaning representation generated in block  752  for later user validation. When the model augmentation subsystem  606  eventually receives a valid response from the user (decision block  758 ), or when the model augmentation subsystem  606  determines in decision block  754  that user input is not involved for the current refinement rule, then the model augmentation subsystem  606  responds by updating (block  760 ) the model  602  using the refined meaning representation  620  generated in block  752 . In response to the model augmentation subsystem  606  receiving an invalid response from the user in decision block  718 , or in response to the model augmentation subsystem  606  updating the model  602  in block  720 , the model augmentation subsystem  606  proceeds (block  710 ) to the next rule the next rule in the inner “for loop” of block  706 . As noted, the refined meaning representation  620  may include one or more substituted structural portions (e.g., different nodes, subtrees, or relative organization of nodes) and/or one or more substituted semantic portions (e.g., a vector, such as a word vector or subtree vector) relative to the meaning representations of the model  602  prior to the refinement process  740 . 
     For the illustrated embodiment, once the model augmentation subsystem  606  has processed each of the active rules of the refining rule-set  618 , it then proceeds to the next intent subtree of the outer “for loop” of block  744 , and then repeats the “for loop” of block  746 , which conditionally applies each of the active model refinement rules-sets  618  against the next intent subtree based on the corresponding model applicability criteria  642 . The refinement process  740  continues until all intent subtrees identified in block  742  have been processed in this manner. As mentioned, in certain embodiments, after the refinement process  740 , the resulting refined model  622  may serve as an augmented model  680  (e.g., an augmented utterance meaning model or an augmented understanding model) for the subsequent meaning search operation. 
     Technical effects of the present section of this disclosure include providing an agent automation framework that is capable of extracting meaning from user utterances, such as requests received by a virtual agent (e.g., a chat agent), and suitably responding to these user utterances. The disclosed NLU framework includes a model augmentation subsystem capable of performing rule-based augmentation of an utterance meaning model and/or an understanding model, whereby the model is augmented by expanding and/or refining of the model based on a model augmentation template. For example, the disclosed NLU framework is capable of expanding a number of meaning representations in the model based on stored generalizing rules, wherein the additional meaning representations are re-expressions of the original meaning representations of the model. The disclosed NLU framework is also capable of refining the meaning representations of these models, for example, to remove substantially similar meaning representations based on stored refining rules, and to modify meaning representations to replace word surface or phrasal forms. As such, the disclosed technique enables the generation of an augmented understanding model and/or augmented utterance meaning model having generalized and/or refined meaning representations. By expanding and/or refining the understanding model and/or the utterance model, the disclosed augmentation techniques enable the NLU framework and the agent automation system to be more robust to variations in discourse styles and to nuances in word surface form and usage, and can also improve the operation of the NLU framework and agent automation system by reducing resource usage when deriving meaning from natural language utterances. 
     Written-Modality Prosody Subsystem 
     Existing virtual agents applying NLU techniques may fail to properly derive meaning from complex natural language utterances. For example, present approaches may fail to comprehend complex language and/or relevant context in a request. Further, existing approaches may not be suitable for or capable of customization and may not be adaptable to various communication channels and styles. 
     With this in mind, present embodiments are directed to an agent automation framework that is capable of extracting meaning from user utterances, such as requests received by a virtual agent (e.g., a chat agent), and suitably responding to these user utterances. To do this, the agent automation framework includes a NLU framework and an intent/entity model having defined intents and entities that are associated with sample utterances. The NLU framework includes a meaning extraction subsystem that is designed to generate meaning representations for the sample utterances of the intent/entity model, as well as a meaning representation for a received user utterance. Additionally, the disclosed NLU framework includes a meaning search subsystem that is designed to search the meaning representations of the intent/entity model to locate matches for a meaning representation of a received user utterance. As such, present embodiments generally address the hard problem posed by NLU by transforming it into a manageable search problem. 
     More specifically, present embodiments are directed to a prosody subsystem of the NLU framework that is designed to analyze collections of written messages for various prosodic cues to break down the collection into a suitable level of granularity (e.g., into sessions, segments, utterances, and/or intent segments) for consumption by other components of the NLU framework, thereby enabling operation of the NLU framework. It should be noted that, while prosody and prosodic cues are generally associated with spoken language, it is presently recognized that certain prosodic cues can be identified in different written language communication channels (e.g., chat rooms, forums, email exchanges), and these prosodic cues provide insight into how the written conversation should be digested into useful inputs for the NLU framework. These prosodic cues may include, for example, source prosodic cues that are based on the author and the conversation channel associated with each message, temporal prosodic cues that are based on a respective time associated with each message, and/or written prosodic cues that are based on the content of each message. For example, intent segments extracted by the prosody subsystem may be consumed by a training process for a machine learning (ML)-based structure subsystem of the NLU framework. Contextually-relevant groups of utterances extracted by the prosody subsystem may be consumed by another training process that generates new word vector distribution models for a vocabulary subsystem of the NLU framework. Intent segments extracted by the prosody subsystem may be consumed by a semantic mining framework of the NLU framework to generate an intent/entity model that is used for intent extraction. Episodes extracted by the prosody subsystem may be consumed by a reasoning agent/behavior engine (RA/BE) to generate episodic context information. Additionally, to enable episodic context management within the NLU framework, the prosody subsystem may also analyze a user message for prosodic cues and provide an indication as to whether the user message corresponds to a prior episodes or corresponds to a new episode. 
     As used herein, the terms “dialog” and “conversation” refer to an exchange of utterances between a user and a virtual agent over a period of time (e.g., a day, a week, a month, a year, etc.). As used herein, an “episode” refers to distinct portions of dialog that may be delineated from one another based on a change in topic, a substantial delay between communications, or other factors. As used herein, “context” refers to information associated with an episode of a conversation that can be used by the RA/BE to determine suitable actions in response to extracted intents/entities of a user utterance. For embodiments discussed below, context information is stored as a hierarchical set of parameters (e.g., name/value pairs) that are associated with a frame of an episode of a dialog, wherein “hierarchical” means that a value of a parameter may itself be another set of parameters (e.g., a set of name/value pairs). As used herein, “domain specificity” refers to how attuned a system is to correctly extracting intents and entities expressed actual conversations in a given domain and/or conversational channel. 
     As mentioned, a computing platform may include a chat agent, or another similar virtual agent, that is designed to automatically respond to user requests to perform functions or address issues on the platform. There are two predominant technologies in NLU, namely traditional computational linguistics and newer machine learning (ML) methods. It is presently recognized that these two technologies demonstrate different strengths and weaknesses with respect to NLU. For example, traditional computational linguistic methods, also referred to herein as “rule-based” methods, include precision rule-sets and manually-crafted ontologies that enable precise adjudication of linguistic structure and semantic understanding to derive meaning representations. Traditional cognitive linguistic techniques also include the concept of construction grammars, in which an aspect of the meaning of a natural language utterance can be determined based on the form (e.g., syntactic structure) of the utterance. Therefore, rule-based methods offer results that are easily explainable and customizable. However, it is presently recognized that such rule-based methods are not particularly robust to natural language variation or adept at adapting to language evolution. As such, it is recognized that rule-based methods alone are unable to effectively react to (e.g., adjust to, learn from) data-driven trends, such as learning from chat logs and other data repositories. Furthermore, rule-based methods involve the creation of hand-crafted rules that can be cumbersome, wherein these rules usually are domain specific and are not easily transferable to other domains. 
     On the other hand, ML-based methods, perform well (e.g., better than rule-based methods) when a large corpus of natural language data is available for analysis and training. The ML-based methods have the ability to automatically “learn” from the data presented to recall over “similar” input. Unlike rule-based methods, ML-based methods do not involve cumbersome hand-crafted features-engineering, and ML-based methods can support continued learning (e.g., entrenchment). However, it is recognized that ML-based methods struggle to be effective when the size of the corpus is insufficient. Additionally, ML-based methods are opaque (e.g., not easily explained) and are subject to biases in source data. Furthermore, while an exceedingly large corpus may be beneficial for ML training, source data may be subject to privacy considerations that run counter to the desired data aggregation. 
     When attempting to derive user intent in a written modality or medium (e.g., in chat logs, dynamic conversations, user forums, or other databases where user communication is stored) it is presently recognized that collections of documents and utterances should be suitably segmented into pieces that are consumable by particular downstream NLP tasks. In order to do so, it is presently recognized that a NLU framework should include a prosody component that takes cues from these stored documents to decompose these documents into differing levels of granularity, in which the level of granularity is dictated by the task at hand. For instance, in certain embodiments, the prosody subsystem is capable of decomposing individual utterances into segments that express granular intents (e.g., intent segments). These intent segments may then be individually matched with an NLU framework&#39;s meaning representation model and sequentially consumed by a RA/BE to act upon. In certain embodiments, the prosody subsystem is capable of decomposing long-lived conversations into episodes in order to allow the RA/BE to determine the appropriate context information that should be applied when acting in response to a user utterance. For instance, a conversation that occurred yesterday will most likely have a completely different context than a conversation happening today, whereas a conversation that occurred five minutes ago will most likely have a bearing on a conversation happening now. It is presently recognized that the delineation of context applicability has a substantial impact on reference resolution during processing of user requests by the RABE. Furthermore, in certain embodiments, the prosody subsystem is capable of decomposing conversations into segments that are useful in training ML-based components of the NLU framework. For instance, the prosody subsystem may provide pieces of conversations (e.g., intent segments, utterances in context) that are useful for performing statistical analyses of word context for generation of semantic vectors, as well as for other learning/training endeavors within the NLU framework. 
     Accordingly, present embodiments are generally directed toward an agent automation framework capable of applying a combination rule-based and ML-based cognitive linguistic techniques to leverage the strengths of both techniques in extracting meaning from natural language utterances. More specifically, present embodiments are directed to a prosody subsystem of the NLU framework that is designed to analyze collections of written messages for various prosodic cues to break down the collection into a suitable level of granularity (e.g., into episodes, sessions, segments, utterances, and/or intent segments) for consumption by other components of the NLU framework, enabling operation of the NLU framework. These prosodic cues may include, for example, source prosodic cues that are based on the author and the conversation channel associated with each message, temporal prosodic cues that are based on a respective time associated with each message, and/or written prosodic cues that are based on the content of each message. For example, as discussed below, in certain embodiments, intent segments extracted by the prosody subsystem may be consumed by a training process for a ML-based structure subsystem of the NLU framework. In certain embodiments, contextually-relevant groups of utterances extracted by the prosody subsystem may be consumed by another training process that generates new word vector distribution models for a vocabulary subsystem of the NLU framework. In certain embodiments, intent segments extracted by the prosody subsystem may be consumed by a semantic mining framework of the NLU framework to generate an intent/entity model that is used for intent extraction. In certain embodiments, episodes extracted by the prosody subsystem may be consumed by a reasoning agent/behavior engine (RA/BE) to generate episodic context information. In certain embodiments, to enable episodic context management, the prosody subsystem is also designed analyze a user message and, based on the extracted episodes, provide an indication as to whether the user message corresponds to a previous episodes or corresponds to a new episode. 
     With the foregoing in mind,  FIG. 28  is a diagram illustrating an embodiment of the prosody subsystem  174  digesting conversation logs  820  (e.g., chat logs, email logs, forum logs, or a combination thereof) into a number of different outputs for consumption by various components of the NLU framework  104 , in accordance with aspects of the present technique. In certain embodiments, the conversation logs  820  may be stored in the database  106  as part of the corpus of utterances  112 , as illustrated in  FIG. 4A . The diagram of  FIG. 28  includes a conversation timeline  822 , which represents a collection of written messages of the conversation logs  820  over time. As such, it may be appreciated that the conversation logs  820  may include any number of communications between any number of users or agents that take place over any number of written conversation channels, such as email, chat, forums, and so forth. As mentioned, in addition to the messages themselves, the conversation logs  820  include metadata and/or annotations for each message that capture additional conversation information, such as a time that each message was sent and/or received, a size of each message, a source and recipient of each message, a conversational channel of each message, and so forth. 
     As illustrated in  FIG. 28 , particular conversation logs  820  may be selected from a particular time period in the conversation timeline  822 . For example, in certain embodiments, the prosody subsystem  174  may select all conversations across all conversational channels that involve a particular user and that occur within a particular time period. Once the conversation logs  820  have been selected, the prosody subsystem  174  may first divide the conversation logs  820  into different conversation channel groups  824  based on the conversation channel associated with each message in the conversation logs  820 . For example, the prosody subsystem  174  may split the selected conversation logs  820  into a first conversational channel group that includes conversations that occur via an email conversation channel, a second conversational channel group that includes conversations that occur via chat, and a third conversational channel group that includes conversations that occur via forum posts, based on the metadata prosodic cues associated with each message in the conversation logs  820 . 
     For the embodiment illustrated in  FIG. 28 , each conversational channel group  826  of the conversation channel groups  824  can be subsequently processed again by the prosody subsystem  174  to divide each conversational channel group  826  into a number of different sessions  828  (e.g., chat sessions, email sessions, forum sessions). For example, in certain embodiments, the prosody subsystem  174  may analyze the metadata associated with messages in the conversation channel group  826  and identify time gaps between each of the messages based on temporal prosodic cues. The prosody subsystem  174  may then determine that, when a time difference between two messages of the conversation channel group  826  is greater than a predefined threshold value (e.g., 5 hours), then this time gap signifies the end of one of the sessions  828  and a beginning of another one of the sessions  828 . Accordingly, for the illustrated embodiment, the prosody subsystem  174  splits the messages of the conversation channel group  826  into a suitable number of sessions  828  based on the metadata or temporal prosody cue associated with each message. As discussed below, the sessions  828  generated by the prosody subsystem  174  can be consumed by the RA/BE  102  to enable episodic context management within the agent automation framework  100 . 
     For the embodiment illustrated in  FIG. 28 , each of the sessions  828  can be subsequently processed again by the prosody subsystem  174  to divide each session  830  into a number of different segments  832  (e.g., chat segments). For clarity, segments  832  may also be referred to herein as “conversation segment” to differentiate from “intent segments” discussed below. For example, in certain embodiments, the prosody subsystem  174  may analyze the contents of each message in the session  830  for written prosodic cues to identify topic changes. By way of particular example, the prosody subsystem  174  may cooperate with the structure subsystem  172  to identify all nouns/subjects within each intent segment of the message, and identify a change in the nouns/subjects as an indication of topic change. In certain embodiments, the prosody subsystem  174  may additionally or alternatively include a collection of transition words and phrases or interrupts (e.g., “anyway”, “moving on”, “by the way”, “next”, etc.) that are indicative of a shift in the topic within the session  830 . The prosody subsystem  174  may then determine that, when a topic change is identified based on written prosodic cues, this signifies the end of one of the segments  832  and a beginning of another of the segments  832 . Accordingly, for the illustrated embodiment, the prosody subsystem  174  splits the messages of the session  830  into a suitable number of segments  832  based on topic changes. 
     For the embodiment illustrated in  FIG. 28 , each of the segments  832  can be subsequently processed again by the prosody subsystem  174  to divide each segment  834  into a number of utterances  836 . For example, in certain cases, the prosody subsystem  174  may first divide the segment  834  into individual messages, wherein each message represents a distinct utterance  836 . However, in certain embodiments, the prosody subsystem  174  may further analyze the contents and/or the metadata associated with each message of the segment  834 , and combine multiple messages that are determined to be part of a single utterance based on temporal and/or written prosodic cues. For example, the prosody subsystem  174  may determine that, when a user sends multiple messages before the other party to the conversation responds (e.g., a message burst group) and/or when the user sends multiple messages within a predefined time window (e.g., within 1 minute), then these multiple messages actually represent a single utterance. Accordingly, for the illustrated embodiment, the prosody subsystem  174  splits the messages of the segment  834  into a suitable number of utterances  836 . 
     For the embodiment illustrated in  FIG. 28 , each of the utterances  836  can be subsequently processed again by the prosody subsystem  174  to divide each utterance  838  into a number of different intent segments  840 . For example, in certain embodiments, the prosody subsystem  174  may analyze the punctuation of the utterance as a written prosody cue to identify intent segments  840  for the utterance  838 . By way of particular example, the prosody subsystem  174  may identify two sentences within the utterance  838  that are separated by a period and/or two phrases or sentence fragments separated by a comma or semicolon, as two separate intent segments  840  of the utterance  838 . 
     However, in certain embodiments, differentiation between intent segments  840  within each utterance  838  may be achieved using cognitive construction grammar (CCG) forms. In other words, specific CCG forms can be used as formulations for specific, and potentially nested, intents. For such embodiments, the prosody subsystem  174  may provide the utterance  838  to the structure subsystem  172 , and the structure subsystem  172  may parse the utterance  838  into an utterance tree. The CCG forms are then detected by traversing these utterance trees and matching predetermined CCG form tree patterns with the patterns found in the utterance trees. In certain embodiments, the predetermined CCG forms are utterance trees stored in a discourse-specific CCG forms database, which may be part of the database  106  of the client instance  42 . The CCG forms themselves can be derived or predetermined in a number of ways, such as via linguistic formulation, via general forms databases available for specific languages, via unsupervised learning, or combinations thereof. For example, the predetermined CCG forms may include forms of phrases that people typically use to, for instance, change a topic (e.g., “Now, with regards to . . . ”, “Speaking of which . . . ”, “Going back to . . . ”, and so forth). As such, by matching the predetermined CCG form tree patterns to portions of the utterance  838 , the prosody subsystem  174  can detect topic context changes within a portion of a written conversation. As noted, for the NLU framework  104  to be trained and operate in a precise and domain specific manner, it is important that the prosody subsystem  174  generates the various digested outputs (e.g., utterances  836  and intent segments  840 ) in proper context. 
     Accordingly, for the illustrated embodiment, the prosody subsystem  174  splits the utterance  838  into a suitable number of intent segments  840 . As discussed below, the intent segments  840  generated by the prosody subsystem  174  can be further processed and consumed to train one or more of the ML-based parsers  188  of the structure subsystem  172  of the agent automation framework  100 . It may further be appreciated that, after completely processing the conversation logs  820 , each of the intent segments  840  may be associated with a particular utterance  838 , a particular segment  834 , a particular session  830 , and a particular conversation channel group  826 . 
     Additionally, it may be appreciated that present embodiments enable entrenchment, which is a process whereby the agent automation system  100  can generally continue to learn or infer meaning of new syntactic structures in new natural language utterances based on previous examples of similar syntactic structures to improve the domain specificity of the NLU framework  104  and the agent automation system  100 . For example, in an embodiment, certain models (e.g., NN structure or prosody models, word vector distribution models) are initially trained or generated using generic domain data (e.g., such as a journal, news, or encyclopedic data source). Since this generic domain data may not be representative of actual conversations (e.g., actual grammatical structure, prosody, and vocabulary) of a particular domain or conversational channel, the disclosed NLU framework  104  is capable of analyzing conversations within a given domain and/or conversational channel, such that these models can be conditioned to be more accurate or appropriate for the given domain. With this in mind, it is presently recognized it is advantageous for the agent automation system  100  to have a continuously learning grammar structure model capable of accommodating changes in syntactic structure, such as new grammatical structures and changes in the use of existing grammatical structures. Additionally, as noted above, the prosody subsystem  174  can be used to generate training data (e.g., intent segments  840 ) that can be used to train a ML-based component of the structure subsystem  172 . 
     With the foregoing in mind,  FIG. 29  is a flow diagram illustrating an embodiment of a process  320  whereby the agent automation system  100  continuously improves a ML-based parser  188 , which may be plugged into the structure subsystem  172  of the NLU framework  104 , as discussed with respect to  FIG. 7 . For the example illustrated in  FIG. 29 , the ML-based parser  188  is specifically a recurrent neural network (RNN)-based parser that operates based on a RNN model  322 . As such, it is appreciated that, by adjusting signal weighting within the RNN model  322 , the ML-based parser  188  can continue to be trained throughout operation of the agent automation system  100  using training data generated by the prosody subsystem  174  from the conversation logs  820 . For the example illustrated in  FIG. 29 , the conversation logs  820  include a continually growing collection of stored user utterances  122  and agent utterances  124 , such as a chat log. As set forth above, the conversation logs  820  include metadata associated with each message that is exchanged between the agent and the user. 
     For the embodiment illustrated in  FIG. 29 , prior to operation of the agent automation system  100 , the RNN-based model  322  may initially have a set of weights (e.g., a matrix of values) that are set by an initial training. For this example, the ML-based parser  188  may be initially trained using a first corpus of utterances having a particular grammatical style, such as a set of books, newspapers, periodicals, and so forth, having a formal or proper grammatical structure. However, it is appreciated that many utterances exchanges in different conversational channels (e.g., chat rooms, forums, and emails) may demonstrate different grammatical structures, such as less formal or more relaxed grammatical structures. With this in mind, the continual learning loop illustrated in  FIG. 29  enables the RNN-model  322  associated with the ML-based parser  188  to be continually updated and adjusted, such that the ML-based parser  188  can become more adept at parsing different (e.g., less-formal or less-proper) grammatical structures in newly received user utterances  122 . 
     The continual leaning process  320  illustrated in  FIG. 29  includes receiving and responding to the user utterance  122 , as discussed above with respect to the process  145  of  FIG. 5 . As mentioned, in certain embodiments, the user utterances  122  and the agent utterances  124 , along with corresponding metadata, are collected as part of the conversation logs  820 . As some point, such as during regularly scheduled maintenance, the prosody subsystem  174  of the NLU framework  104  repeatedly segments (block  842 ) the conversation logs  820  into intent segments  840 , as discussed with respect to  FIG. 28 . Then, different rule-based parsers  186  and/or ML-based parsers  188  of the structure subsystem  172  of the NLU framework  104  parse (block  325 ) each of the intent segments  840  to generate multiple annotated utterance tree structures  326  for each of the intent segments  840 . The meaning extraction subsystem  150  then determines (in decision block  328 ) whether a quorum (e.g., a simple majority consensus) has been reached by the different parsers. 
     For the example illustrated in  FIG. 29 , when the NLU framework  104  determines in block  328  that a sufficient number (e.g., a majority, greater than a predetermined threshold value) of annotated utterance trees  326  for a particular intent segment are substantially the same for a quorum to be reached, then the meaning extraction subsystem  150  may use the quorum-based set of annotated utterance trees  330  to train and improve a ML-model  322  associated with the ML-based parser  188 , as indicated by the arrow  331 . For example, the weights within the ML-model  322  may be repeatedly adjusted until the ML-based parser  188  generates the appropriate structure from the quorum-based set of annotated utterance trees  330  for each of the intent segments  840 . After this training, upon receiving a new user utterance  122  having a grammatical structure similar to a structure from the quorum-based set of annotated utterance trees  330 , the operation of the ML-based parser  188 , the NLU framework  104 , and the agent automation system  100  is improved to parse the grammatical structure of the user utterance  122  and extract the intents/entities  140  therefrom with enhanced domain specificity. 
     Additionally, in certain embodiments, the agent automation system  100  can continue to learn or infer meaning of new words and phrases. It is presently recognized that this can enable the agent automation system  100  to have a continuously expanding/adapting vocabulary capable of accommodating the use of unfamiliar words, as well as changes to the meaning of familiar words. For example,  FIG. 30  is a flow diagram illustrating an embodiment of a process  340  whereby the agent automation system  100  continuously improves a word vector distribution model  342 , which may be plugged into the vocabulary subsystem  170  of the NLU framework  104 , such as the learned multimodal word vector distribution model  178  or the learned unimodal word vector distribution model  180  discussed above with respect to  FIG. 7 . As such, it is appreciated that, by expanding or modifying the word vector distribution model  342 , operation of the vocabulary subsystem  170 , the NLU framework  104 , and the agent automation system  100  can be improved to handle words with new or changing meanings using only training data that can be generated from a continually growing conversation logs  820 . For the example illustrated in  FIG. 30 , the corpus of utterances  112  may be, for example, a collection of chat logs storing utterances user utterances  122  and agent utterances  124  from various chat room exchanges, or other suitable source data. 
     For the embodiment illustrated in  FIG. 30 , prior to operation of the agent automation system  100 , the word vector distribution model  342  may initially be generated based on a first corpus of utterances that have a particular diction and vocabulary, such as a set of books, newspapers, periodicals, and so forth. However, it is appreciated that many utterances exchanges in different conversational channels (e.g., chat rooms, forums, emails) may demonstrate different diction, such as slang terms, abbreviated terms, acronyms, and so forth. With this in mind, the continual learning loop illustrated in  FIG. 30  enables the word vector distribution model  342  to be modified to include new word vectors, and to change values of existing word vectors, based on source data gleaned from the growing collections of user and agent utterances  122  and  124  in the conversation logs  820 , to improve the domain specificity of the NLU framework  104 . 
     It should be noted that word-vector learning is based on the premise that words are generally used in specific contexts, and this defines a probability that specific words will appear given a specific set of surrounding words or, conversely, a probability that surrounding words will appear given a certain word, or similar context-aware derivations. As such, it is presently recognized that word vectors should be learned using optimization functions related to context (e.g., where words are appropriately “couched” within the context of other words). As such, one important aspect of prosodic segmentation is determining when one context starts and when one context ends. That is, it may be appreciated that, in certain written language source data (e.g., online or news articles), context boundaries may be well-defined; however, in other types of written language source data (e.g., free-form chat), these context boundaries may not be as readily apparent. As such, the disclosed prosody subsystem  174  determines these context boundaries to suitably group utterances in a context-specific manner, such that word meanings are extracted with the correct context specified. 
     Like  FIG. 29 , the process  340  illustrated in  FIG. 30  includes receiving and responding to the user utterance  122 , as discussed above with respect to  FIG. 5 . As mentioned, the user utterances  122  and the agent utterances  124  can be collected to as part of the conversation logs  820 , which may form at least a portion of the corpus of utterance  112  stored in the database  106  illustrated in  FIG. 4A . As some point, such as during regularly scheduled maintenance, the prosody subsystem  174  of the NLU framework  104  segments (block  844 ) the conversation logs  820  into distinct utterances  836 , as discussed above with respect to  FIG. 28 . It should be noted that, because of the manner in which the conversation logs  820  are broken down into sessions  828  before being broken down into utterances  836 , as noted above with respect to  FIG. 28 , the utterances  836  segmented in block  343  are grouped with similar context. For example, in certain embodiments, the utterances  836  generated in block  343  may be grouped based on the session  828  from which each of the utterances  836  are derived, such that each group of utterances  836  are likely to be contextually relevant to one another. Then, in block  345 , the NLU framework  104  performs rule-augmented unsupervised learning to generate a refined word vector distribution model  346  containing new or different word vectors  348  generated from the segmented utterances  836 . 
     For example, as discussed above, the meaning extraction subsystem  150  may analyze the set of segmented utterances  836  and determine word vectors  348  for the words of these utterances based on how certain words tend to be used together. For such embodiments, two words that are frequently used in similar contexts within these utterances  836  are considered closely related and, therefore, are assigned a similar vector value (e.g., relatively closer in terms of Euclidean distance) in one or more dimensions of the word vectors  348 . In this manner, the meaning extraction subsystem  150  may adapt to changes in the meaning of a previously understood term based on new context in which the term is used. 
     As illustrated in  FIG. 30 , the refined word vector distribution model  346  is used to replace the existing word vector distribution model  342 , such that the vocabulary subsystem  170  can use this refined model to provide word vectors for the words and phrases of new user utterances  122  received by the agent automation system  100 . For example, an initial word vector distribution model  342  may have a word vector for the term “Everest” that is relatively close in one or more dimensions to other word vectors for terms such as, “mountain”, “Himalayas”, “peak”, and so forth. However, when a client creates a new conference room that is named “Everest,” the term begins to be used in a different context within user utterances  122 . As such, in block  345 , a new word vector would be generated for the term “Everest” that would be relatively close in one or more dimensions to word vectors for terms such as “conference”, “meeting”, “presentation”, and so forth. After updating the word vector distribution model  342 , upon receiving a user utterance  122  having the revised term “Everest,” the operation of the vocabulary subsystem  170 , the NLU framework  104 , and the agent automation system  100  is improved to more provide more accurate word vectors, annotated utterance trees, and meaning representations, which result in more accurately extracted intents/entities  140  for the given domain (e.g., enhanced domain specificity). 
     As mentioned, the disclosed agent automation framework  100  is capable of generating a number of outputs, including the intent/entity model  108 , based on the corpus of utterances  112  and the collection of rules  114  stored in the database  106 .  FIG. 31  is a block diagram depicting a high-level view of certain components of the agent automation framework  100 , in accordance with an embodiment of the present approach. In addition to the NLU framework  104  and the reasoning agent/behavior engine  102  discussed above, the embodiment of the agent automation framework  100  illustrated in  FIG. 31  includes a semantic mining framework  860  that is designed to process the conversation logs  820 , using various subsystems of the NLU framework  104 , to generate and improve the intent/entity model  108  and to improve the conversation model  110 . 
     More specifically, for the illustrated embodiment, the semantic mining framework  860  includes a number of components that cooperate with other components of the agent automation framework  100  (e.g., the NLU framework  104 , the vocabulary manager  118 ) to facilitate generation and improvement of the intent/entity model  108  based on the conversation logs  820 , which may form at least a part of the corpus of utterances  112  stored in the database  106 . That is, as discussed in greater detail below, the semantic mining framework  860  cooperates with the NLU framework  104  to decompose utterances  112  into intent segments (e.g., intents and entities), and to map these to intent vectors  862  within a vector space. In certain embodiments, certain entities (e.g., intent-specific or non-generic entities) are handled and stored as parameterizations of corresponding intents of the intent vectors within the vector space. For example, in the utterance, “I want to buy the red shirt,” the entity “the red shirt” is treated as a parameter of the intent “I want to buy,” and can be mapped into the vector space accordingly. The semantic mining framework  860  also groups the intent vectors based on meaning proximity (e.g., distance between intent vectors in the vector space) to generate meaning clusters  864 , as discussed in greater detail below with respect to  FIG. 32 , such that distances between various intent vectors  862  and/or various meaning clusters  864  within the vector space can be calculated by the NLU framework  104 , as discussed in greater detail below. 
     For the embodiment illustrated in  FIG. 31 , the semantic mining framework  860  begins with a semantic mining pipeline  866 , which is an application or engine that generates the aforementioned intent vectors  862 , as well as suitable meaning clusters  864 , to facilitate the generation of the intent/entity model  108  based on the conversation logs  820 . For example, in certain embodiments, the semantic mining pipeline  866  provides all levels of possible categorization of intents found in the conversation logs  820 . Additionally, the semantic mining pipeline  866  produces a navigable schema (e.g., cluster formation trees  868  and/or dendrograms) for intent and intent cluster exploration. As discussed below, the semantic mining pipeline  866  also produces sample utterances  870  that are associated with each meaning cluster, and which are useful to cluster for exploration and training of the reasoning agent/behavior engine  102  and/or the conversation model  110 . In certain embodiments, the outputs  872  of the semantic mining pipeline  866  (e.g., meaning clusters  864 , cluster formation trees  868 , sample utterances  870 , and others discussed below) may be stored within one or more tables of the database  106  in any suitable manner. 
     Once the outputs  872  have been generated by the semantic mining pipeline  866 , in certain embodiments, an intent augmentation and modeling module  874  may be executed to generate and improve the intent/entity model  108 . For example, the intent augmentation and modeling module  874  may work in conjunction with other portions of the NLU framework  104  to translate mined intents into the intent/entity model  108 . In particular, meaning clusters  864  may be used by the intent augmentation and modeling module  874  as a basis for intent definition. This follows naturally from the fact that meaning proximity is used as the basis for formation of the meaning clusters  864 . As such, related and/or synonymous intent expressions are grouped together and, therefore, can be used as primary or initial samples for intents/entities when creating the intent/entity model  108  of the agent automation framework  100 . Additionally, in certain embodiments, the intent augmentation and modeling module  874  utilizes a rules-based intent augmentation facility to augment sample coverage for discovered intents, which makes intent recognition by the NLU engine  116  more precise and generalizable. In certain embodiments, the intent augmentation and modeling module  874  may additionally or alternatively include one or more cluster cleaning steps and/or one or more cluster data augmentation steps that are performed based on the collection of rules  114  stored in the database  106 . This augmentation may include a rule-based re-expression of sample utterances included in the discovered intent models and removal of structurally similar re-expressions/samples within the discovered model data. For example, this augmentation can include an active-to-passive re-expression rule, wherein a sample utterance “I chopped this tree” may be converted to “this tree was chopped by me”. Additionally, since re-expressions (e.g., “buy this shoe” and “purchase this sneaker”) have the same parse structure and similarly labeled parse node words that are effectively synonyms, this augmentation can also include removing such structurally similar re-expressions. 
     For the embodiment illustrated in  FIG. 31 , the semantic mining framework  860  includes an intent analytics module  876  that enables visualization of conversation log statistics, including intent and entity prevalence, and so forth. The illustrated embodiment also includes a conversation optimization module  878  that works in conjunction with the intent analytics module  876  to identify blind spots or weak points in the conversation model  110 . For example, in an embodiment, the intent analytics module  876  may determine or infer intent prevalence values for certain intents based on cluster size (or another suitable parameter). Subsequently, intent prevalence values can be used by the conversation optimization module  878  as a measure of the popularity of queries that include particular intents. Additionally, when these intent prevalence values are compared to intents associated with particular responses in the conversation model  110 , the conversation optimization module  878  may identify portions of the conversation model  110  that provide insufficient coverage (e.g., blind-spot discovery). That is, when the conversation optimization module  878  determines that a particular intent has a particularly high prevalence value and is not associated with a particular response in the conversation model  110 , the conversation optimization module  878  may identify this deficiency (e.g., to a designer of the reasoning agent/behavior engine  102 ), such that suitable responses can be associated with these intents to improve the conversation model  110 . Additionally, in certain embodiments, the intent analytics module  876  may determine a number of natural clusters within the meaning clusters  864 , and the conversation optimization module  878  may compare this value to a number of breadth of intents associated with responses in the conversation model  110  to provide a measure of sufficiency of the conversation model  110  to address the intent vectors  862  generated by the semantic mining pipeline  866 . 
       FIG. 32  is a block diagram of an embodiment of the semantic mining pipeline  866  that includes a number of processing steps of a semantic mining process used to generate outputs  872  to facilitate the generation of the intent/entity model  108  from the conversation logs  820 . As such, the steps that are illustrated as part of the semantic mining pipeline  866  may be stored in suitable memory (e.g., memory  86 ) and executed by suitable a suitable processor (e.g., processor  82 ) associated with the client instance  42  (e.g., within the data center  22 ). 
     For the illustrated embodiment, the semantic mining pipeline  866  includes a cleansing and formatting step  900 . During the cleansing and formatting step  900 , the processor  82  analyzes the conversation logs  820  and removes or modifies any source data that may be problematic for intent mining, or to speed or facilitate intent mining. For example, the processor  82  may access rules  114  stored in the database  106  that define or specify particular features that should be modified within the corpus of utterances  112  before intent mining of the utterances  112  occurs. These features may include special characters (e.g., tabs), control characters (e.g., carriage return, line feed), punctuation, unsupported character types, uniform resource locator (URLs), internet protocol (IP) addresses, file locations, misspelled words and typographical errors, and so forth. In certain embodiments, the vocabulary manager  118  of the NLU framework  104  may perform at least portions of the cleansing and formatting step  900  to substitute out-of-vocabulary words based on synonyms and domain-specific meanings of words, acronyms, symbols, and so forth, defined with the rules  114  stored in the database  106 . 
     For the illustrated embodiment, after cleansing and formatting, the conversation logs  820  undergo an intent detection, segmentation, and vectorization step  902 . During this step, the processor  82  analyzes the conversation logs  820  using the NLU framework  104 , including the NLU engine  116  and the vocabulary manager  118 , to detect and segment the utterances into intents and entities based on the rules  114  stored in the database  106 . Within this step, the prosody subsystem  174  of the NLU framework  104  is particularly responsible for repeatedly digesting the conversation logs  820  into intent segments  840 , as discussed above with respect to  FIG. 28 . As mentioned above, the intent segments  840  may be generated from utterances  836  based on punctuation, based on CCG grammar form detection/recognition, or a combination thereof. Since these intent segments  840  form the basis for clustering intent vectors  862  during semantic mining, it is presently recognized that proper intent segmentation is important to the precise and domain specific operation of the NLU framework  104 . 
     As discussed, in certain embodiments, certain entities can be stored in the intent/entity model  108  as parameters of the intents. Additionally, these intents are vectorized, meaning that a respective intent vector is produced for each intent by the NLU framework  104 . It may be appreciated by those skilled in the art that these vectors may be generated by the NLU framework  104  in a number of ways. For example, in certain embodiments, the NLU framework  104  may algorithmically generate these vectors based on pre-built vectors in a database (e.g., a vector for an intent “buy a shoe” might include a pre-built vector for “buy” that is modified to account for the “shoe” parameter). In another embodiment, these vectors may be based on the output of an encoder portion of an encoder-decoder pair of a language translation system that consumes the intents as inputs. 
     For the illustrated embodiment, after intent detection, segmentation, and vectorization, a vector distance generation step  904  is performed. During this step, all of the intent vectors produced in block  902  are processed to calculate distances between all intent vectors (e.g., as a two-dimensional matrix). For example, the processor  82  executes a portion of the NLU framework  104  (e.g., the NLU engine  116 ) that calculates the relative distances (e.g., Euclidean distances, or another suitable measure of distance) between each intent vector in the vector space to generate this distance matrix, which is later used for cluster formation, as discussed below. 
     For the illustrated embodiment, after vector distance generation, a cluster discovery step  906  is performed. In certain embodiments, this may be a cross-radii cluster discovery process; however, in other embodiments, other cluster discovery processes can be used, including, but not limited to, agglomerative clustering techniques (e.g., Hierarchical Agglomerative Clustering (HAC)), density based clustering (e.g., Ordering Points To Identify the Clustering Structure (OPTICS)), and combinations thereof, to optimize for different goals. For example, discussion cluster discovery may benefit more from density-based approaches, such as OPTICS, while intent model discovery may benefit more from agglomerative techniques, such as HAC. 
     For example, in one embodiment involving a cross-radii cluster discovery process, the processor  82  attempts to identify a radius value that defines a particular cluster of intent vectors in the vector space based on the calculated vector distances. The processor  82  may determine a suitable radius value defining a sphere around each intent vector, wherein each sphere contains a cluster of intent vectors. For example, the processor  82  may begin at a minimal radius value (e.g., a radius value of 0), wherein each intent vector represents a distinct cluster (e.g., maximum granularity). The processor  82  may then repeatedly increment the radius (e.g., up to a maximum radium value), enlarging the spheres, while determining the size of (e.g., the number of intent vectors contained within) each cluster, until all of the intent vectors and meaning clusters merge into a single cluster at a particular maximum radius value. It may also be appreciated that the disclosed cross-radii cluster discovery process represents one example of a cluster discovery process, and in other embodiments, cluster discovery may additionally or alternatively incorporate measures and targets for cluster density, reachability, and so forth. 
     For the illustrated embodiment, after cluster discovery, a stable range detection step  908  is performed. For example, for embodiments that utilize the cross-radii cluster discovery process discussed above, the processor  82  analyzes the radius values relative to the cluster sizes determined during cluster discovery  906  to identify stable ranges  908  of radius values, indicating that natural clusters are being discovered within the vector space. Such natural intent clusters are commonly present within a corpus of utterances, and are generally particular to a language and/or a context/domain. 
     Additionally, the prosody subsystem  174  also supports and enables episodic context management within the agent automation system  100 . For example, in certain embodiments, the RA/BE  102  may include a number of different personas, each designed to address different aspects or facets of the behavior of the RABE  102 , such as a sales persona, a marketing persona, a support persona, a persona for addressing requests during business hours, a persona for addressing requests after business hours, and so forth. Additionally, these personas of the RA/BE  102  manage context information associated with each episode, wherein the context information may be stored as a hierarchical set of name/value pairs in the database  106 . As such, the RA/BE  102  ensures that appropriate episodic context information can be applied when responding to user messages. 
     For example, in certain embodiments, a persona of the RA/BE  102  may initially respond to a user message based on a current context that only includes context information from the current episode (e.g., today&#39;s episode context). However, when the prosody subsystem  174  determines that a user message includes prosodic cues indicating that the user message is associated with the context information of another episode (e.g., yesterday&#39;s context), the RA/BE  102  responds by retrieving and overlaying the context information of the current episode with the context of the referenced episode based on persona-specific overlay rule templates, which may be stored in the database  106 . As such, the persona of the RA/BE  102  can subsequently perform suitable actions in response to the user message, as well as subsequent user message, in a context-appropriate manner. Accordingly, the disclosed RA/BE  102  design provides a substantial improvement by enabling virtual agents having automatic context management. 
     For the illustrated example of  FIG. 33 , the RA/BE  102  manages information for a number of different episodes in one or more suitable tables of the database  106 . More specifically, a particular persona of the RA/BE  102  manages episode conversation information  930  (e.g., episode conversation information  930 A,  930 B,  930 C, and  930 D), which includes messages and corresponding metadata that are part of each episode of conversation between the user and the persona of the RA/BE  102 . Additionally, the persona of the RA/BE  102  manages episode context information  932  (e.g., episode context information  932 A,  932 B,  932 C, and  932 D), which includes name/value pairs storing details (e.g., user information, topic information, prices, stock identifiers, weather information, and so forth) relating to the episode of conversation. 
     For the illustrated example, the persona of the RA/BE  102  calls on the prosody subsystem  174  of the NLU framework  104  to determine how to segment a conversation into episodes  934 , including episodes  934 A,  934 B,  934 C, and  934 D, which represent discrete or disparate portions of the conversation logs  820  (indicated by the conversation timeline  822  in  FIG. 33 ) that are pertinent to a specific topic/set of topics during one-on-one or group interactions involving the RA/BE  102 . It may be noted that, in certain embodiments, these episodes  934  correspond to sessions  828  or segments  832  that are identified from the conversation logs  820  by the prosody subsystem  174 , as illustrated in  FIG. 28 , while in other embodiments, each of the episodes  934  can include messages from different sessions  828  and/or segments  832  that are topically related, based on written prosodic cues, and/or temporally related, based on temporal prosodic cues. For the illustrated example of  FIG. 33 , each of the episodes  934  corresponds to a particular session  830 , which includes respective boundaries to mark start and end times that are determined from the conversation log  820  as the conversation progresses. To identify these boundaries, the prosody subsystem  174  may apply rules and/or statistical learning (e.g., machine-learning) techniques to determine how to divide and group messages from the conversation log  820  to support episodic context management by the RA/BE  102 . 
     For example, in certain embodiments, the prosody subsystem  174  determines the start and end time associated with each of the episodes  934  based on written prosodic cues that indicate changes in topic, based on temporal prosodic cues that indicate a substantial delay between messages, or other suitable factors. Specifically, in certain embodiments, the prosody subsystem  174  may use heuristic rules to identify episode start and end times. Learning mechanisms, similar to human autonoetic introspection, can be used to determine approximations of attention span, identify what context information  932  needs to be propagated across episode boundaries, determine property-override-rules, determine derivative scoping rules, and so forth, and these, in turn, use features of the user (e.g., user demographic, user mood, and so forth) alongside current relevant context (e.g., current time-of-day, location, weather, and so forth). 
     For the illustrated embodiment, the prosody subsystem  174  may use the rules  114  stored in the database  106  to digest the conversation log  820  represented by the conversation timeline  822  into sessions  828 , wherein each session  830  corresponds to one of the episodes  934 . For example, in certain embodiments, the rules  114  may define a time gap  936  between episodes  934 , such that an amount of time between messages in the conversation log  820  that is greater than or equal to this stored threshold value indicates the end of a first episode (e.g., episode  934 A) and the beginning of the next episode (e.g., episode  934 B). In certain embodiments, the rules  114  may additionally or alternatively define an inter-episode cadence  938 , which defines a length of time or a number of messages along the conversation timeline  822  that generally corresponds to a single episode, such that a duration or a number of messages of a conversation can be used as an indication of demarcation between episodes  934 . In certain embodiments, the rules  114  may also define written prosodic cues  940  (also referred to as per-utterance cues), which are written prosodic cues within the messages of the conversation logs  820  that may signal the beginning or end of an episode. Additionally, in certain embodiments, the rules  114  may also define message burst grouping  942 , which indicates how certain distinct messages  944  within the conversation logs  820  may be combined by the prosody subsystem  174  to represent a single utterance  838  based on temporal and/or written prosodic cues. As noted above, in certain embodiments, this may involve the prosody subsystem  174  matching utterance trees of the messages  944  and/or utterances  838  to predetermined CCG forms representing phrases that are indicative of topic changes (e.g., “Now, with regards to . . . ”, “Speaking of which . . . ”, “Going back to . . . ”, and so forth) to determine which messages  944  should be treated as a single utterance  838 . 
     However, as mentioned, in certain embodiments, the prosody subsystem  174  may be or include a ML-based prosody system  196 , which learns how to digest the conversation log  820  into episodes  934  based on different prosodic cues. For example, in certain embodiments, the ML-based prosody system  196  may analyze the conversation log  820  to determine the inter-episode cadence  938  for a set of conversation logs  820 . Additionally, the ML-based prosody system  196  may analyze the conversation logs  820  to determine the typical time gap  936  between episodes  934  based on temporal prosodic cues. The ML-based prosody system  196  may also analyze written prosodic cues in the conversation logs  820  to determine written prosodic cues  940  that signal the beginning or end of an episode. The ML-based prosody system  196  may further analyze temporal prosodic cues in the conversation logs  820  to determine the message burst grouping  942 , which dictates how the ML-based prosody system  196  groups messages  944  within the episodes  934  as a distinct utterance  838 . In certain embodiments, the conversation logs  820  may be further annotated by a human to indicate when the human believes that the episodic boundaries should occur to enhance learning by the ML-based prosody system  196 . 
       FIG. 34  is a flow diagram depicting an example of a persona  970  of a RA/BE  102  using the prosody subsystem  174  of the NLU framework  104  to manage episodic conversation context, in accordance with an embodiment of the present approach. For the illustrated embodiment, the persona  970  is a script of the RA/BE  102  that is designed to address a particular aspect of conversations with a user, such as a sales persona, a marketing persona, a support persona. Additionally, the persona  970  of the RA/BE  102  stores and manages context information  932  (e.g., context information  932 A,  932 B, and  932 C) that is associated with each distinct chat episode  934  between the user and the persona  970 , wherein the context information  932  may be stored as a hierarchical set of name/value pairs in the database  106 . For example, stored context information  932  may include user information (e.g., role, gender, age), conversational topic information (e.g., items discussed, actions requested), and other conversational details (e.g., results/outcomes) for each episode of conversation between the user and the persona  970 . 
     When the persona  970  of the RA/BE  102  receives a new message  972  from the user, the persona  970  consults the prosody subsystem  174  (block  974 ) to determine (block  976 ) whether the new message should be treated as a continuation of a prior conversation episode (e.g., chat episode) or the beginning of a new episode. As set forth above with respect to  FIG. 33 , in certain embodiments, the prosody subsystem  174  may include a rules-based prosody system  194  that applies rules  114  stored in the database  106  to determine whether the new message  972  is a continuation of a prior chat episode. For example, the rules-based prosody system  194  may apply a rule that defines the typical time gap  936  between episodes, and when the time gap between the new message  972  and the previous message in the conversation logs  820  is less than the typical time gap  936 , the rules-based prosody system  194  may determine that the new message  972  is a continuation of the previous episode. In other embodiments, the rules-based prosody system  194  may apply a rule that defines written prosodic cues  940  that signal the start of a new episode or that signal that the new message  972  is a continuation of a prior episode. In still other embodiments, the prosody subsystem  174  may include a ML-based prosody system  196  that learns from the conversation logs  820  the typical time gap  936  between episodes and/or written prosodic cues  940  that are then applied to determine whether the new message  972  is a continuation of a prior episode. 
     When the persona  970  of the RA/BE  102  determines that the prosody subsystem  174  has provided an indication that the new message  972  is a continuation of a prior episode, the RA/BE  102  responds by resuming (block  978 ) the conversation using the context of the prior episode. To do this, as illustrated by the arrow  980 , the RA/BE  102  overlays the episode context information of the prior episode (e.g., episode context information  932 A) over the current context information (e.g., episode context information  932 B) in order to use at least a portion of the context information of the prior episode when responding to the new user message  972 . In certain embodiments, the prosody subsystem  174  may additionally provide the persona  970  with intent segments (e.g., intents/entities) that are identified within the new user message  972 , such that the persona  970  can identify context-overlay cues within these intent segments and use these context-overlay cues to identify which prior episode context information should be overlaid. One example of a context-overlay cue in a message might be, “Remember what we discussed on Wednesday?” Additionally, the persona  970  of the RA/BE  102  may be programmed to perform particular actions in response to particular intents/entities parsed from the new user message  972  by the prosody subsystem  174 . 
     In certain embodiments, overlaying may involve the persona  970  of the RA/BE  102  applying persona-specific overlay rule templates stored in the database  106  that define how the hierarchical set of name/value pairs of the context information associated with the prior episode augments or modifies the context information associated with the current conversation with the user. In certain embodiments, this may also involve the RA/BE  102  combining the context information of multiple episodes based on persona-specific multi-episode aggregation rules stored in the database  106 . However, when the prosody subsystem  174  signals to the RA/BE  102  that the new message  972  is not a continuation of a prior episode, the persona  970  of the RA/BE  102  starts (block  982 ) a new episode with fresh context (e.g., episode context information  932 C) without overlaying context information of another episode. As such, the persona  970  of the RA/BE  102  can subsequently perform suitable actions in response to the new user message  972 , as well as subsequent user messages of the current episode, in a context-appropriate manner. 
     Technical effects of this section of the present disclosure include providing an agent automation framework that is capable of extracting meaning from user utterances, such as requests received by a virtual agent (e.g., a chat agent), and suitably responding to these user utterances. Additionally, present embodiment include a prosody subsystem of the NLU framework that is designed to analyze collections of written messages for various prosodic cues to break down the collection into a suitable level of granularity (e.g., into episodes, sessions, segments, utterances, and/or intent segments) for consumption by other components of the NLU framework, enabling operation of the NLU framework. These prosodic cues may include, for example, source prosodic cues that are based on the author and the conversation channel associated with each message, temporal prosodic cues that are based on a respective time associated with each message, and/or written prosodic cues that are based on the content of each message. For example, to improve the domain specificity of the agent automation system, intent segments extracted by the prosody subsystem may be consumed by a training process for a ML-based structure subsystem of the NLU framework, and contextually-relevant groups of utterances extracted by the prosody subsystem may be consumed by another training process that generates new word vector distribution models for a vocabulary subsystem of the NLU framework. Additionally, intent segments extracted by the prosody subsystem may be consumed by a semantic mining framework of the NLU framework to generate an intent/entity model that is used for later intent extraction. Additionally, to enable episodic context management within the NLU framework, the prosody subsystem may also analyze a received user message and provide an indication as to whether the user message corresponds to a prior episodes or corresponds to a new episode. 
     Systematic Characterization and Optimization of NLU Framework 
     As discussed above, a NLU framework may include a number of complex subsystems, such as vocabulary management subsystems, data augmentation subsystems, written-modality prosody subsystems, and so forth, and each of these subsystems may include a number of components (e.g., plugins, rules, forms). It is presently recognized that it would be advantageous to be able to model the complex interactions of these subsystems and components in order to thoroughly understand the operation of the NLU framework and provide explainability of why the NLU framework inferenced a user utterance in a given way. This enables a more transparent understanding model, which aids users to better understand and use the NLU framework and aids designers to improve the configuration and operation of the NLU framework. Additionally, each of these subsystems and components may be configured in a number of different manners, and it can be challenging for a designer to determine the impact of adjusting the configuration of one or more of these components in terms of overall NLU performance (e.g., latency time, precision, recall) and computational performance (e.g., processing time, memory usage, storage usage) of the NLU framework. As such, it is presently recognized that it would be advantageous to have a facility within the NLU framework that enables the configurations of the components of the NLU framework to be optimized to provide a desired level of performance to a client. 
     With this in mind,  FIG. 35  is a block diagram illustrating an embodiment of a NLU framework  1000  that includes NLU system  1002  and a modeling and optimization system  1004 . The illustrated NLU system  1002  has a number of components  1006 , including various rules-based linguistic components and ML-based components discussed above. As used herein, a “component” refers to a portion or element of the NLU framework  1000  used in the processing of a user utterance during inference, such as a system, subsystem, model, rule, method, plugin, threshold value, weight value, or coefficient. As such, a component may be a granular component of the NLU framework (e.g., a particular rule or coefficient), or may be related to or include other components (e.g., a subsystem). It may be appreciated that, while the present discussion is directed to modeling and optimizing the NLU system  1002  of the NLU framework  1000 , in other embodiments, the NLU framework  1000  may include other systems that cooperate with the NLU system  1002  during inference of user utterances, and these systems may be similarly modeled and optimized by the modeling and optimization system  1004 , as discussed below. Additionally, while the modeling and optimization system  1004  is illustrated as being part of the NLU framework  1000 , in other embodiments, it may be separate from the NLU framework  1000 , and may be hosted by different hardware resources than the NLU framework  1000 . 
     In general, the modeling and optimization system  1004  leverages ML-based (e.g., statistical) techniques and linguistic theory to enhance the explainability of the NLU framework  1000 , which improves a user or designer understanding of the operation of the components  1006  at various levels of the NLU framework. The modeling and optimization system  1004  enables the modeling of the behavior of components  1006  of the NLU framework  1000  to enable deeper understanding of the inter-component relationships within the NLU framework  1000  during operation. The modeling and optimization system  1004  enables components  1006  that interact strongly and positively to be identified, which can be used to improve the efficiency of various processing pipelines of the NLU framework  1000 . The modeling and optimization system  1004  can also provide information regarding the various paths that an utterance can take through the NLU framework  1000  during inference, which can be used to generate diagrams explaining the operation of the NLU framework  1000 . Additionally, the modeling and optimization system  1004  enables configurations of the NLU framework  1000  to be learned or automatically determined, including which components should be enabled or disabled, as well as which numerical values (e.g., threshold values, coefficient values, weight values) certain components should use to improve the NLU performance (e.g., reduce latency, improve precision, improve recall) and/or the computational performance (e.g., reduce processing time, reduce memory usage, reduce storage usage) of the NLU framework  1000 . 
     For the embodiment illustrated in  FIG. 35 , the modeling and optimization system  1004  includes one or more configuration vectors  1008 , which individually or collectively store the current configuration of the NLU framework  1000  in vector form. In certain embodiments, the configuration vectors  1008  may include a configuration vector having a corresponding entry (e.g., a binary value) for each component of the NLU framework  1000  (e.g., 0 for disabled components, 1 for enabled components). In some embodiments, the configuration vectors  1008  may include a configuration vector storing numerical values (e.g., floating point values, integers) that correspond to a value that a particular component may apply during utterance inference (e.g., a value of a FAM coefficient, a threshold value, a weight value). In some embodiments, the configuration vectors  1008  may include a configuration vector having both binary and numerical values. For example, in some embodiments, within a configuration vector, a cognitive construction grammar (CCG) scoring coefficient component may include a corresponding binary value indicating whether or not it is disabled, as well as a corresponding numerical value indicating the value of the coefficient that is applied during inference. In certain embodiments, the modeling and optimization system  1004  is configured to analyze the NLU framework  1000  to determine one or more suitable configuration vectors that, individually or collectively, represent the configuration of the components  1006  of the NLU framework  1000 . 
     For the embodiment illustrated in  FIG. 35 , the modeling and optimization system  1004  also includes sets of test data  1010 . Each of these sets of test data  1010  includes a set of test utterances  1012  and corresponding desired artifacts  1014  (e.g., intents and/or entities) for each of these utterances (e.g., labeled data). As discussed below, the modeling and optimization system  1004  is configured to provide the test data  1010  to the NLU framework  1000 , wherein the NLU framework  1000  inferences the test utterances  1012  to generate and score artifacts based on a configuration represented by the configuration vectors  1008  of the modeling and optimization system  1004 . The modeling and optimization system  1004  determines performance metrics  1016  (e.g., precision, recall, latency, processing time, memory usage, storage usage, inference information per test utterance) for each configuration of the NLU framework  1000  during inference of the test utterances  1012 . For example, the modeling and optimization system  1004  may compare scored artifacts returned by the NLU framework  1000  to the desired artifacts  1014  of the test data  1010  to calculate the precision and/or recall of the NLU framework  1000 . The modeling and optimization system  1004  may also determine the usage of computing resources (e.g., processors, memory, storage) when processing the test data  1010  to determine certain performance metrics  1016  for each configuration of the NLU framework  1000 . The modeling and optimization system  1004  may subsequently modify (e.g., mutate) the one or more configuration vectors  1008 , submit the test data  1010  to the NLU framework again, and then collect the performance metrics  1016  for the modified configuration vectors  1008 . 
     As such, the modeling and optimization system  1004  enables the operation of components of the NLU framework  1000  to be seamlessly disabled or modified, and evaluates the impact of these changes to the overall performance of the NLU framework  1000  to provide enhanced understanding of the operation of the NLU framework  1000  during utterance inference. Using the performance metrics  1016  collected for a number of different configuration vectors  1008 , the modeling and optimization system  1004  may determine which components  1006  have a positive and important impact during inference of the test utterances  1012 , and generate an importance vector  1018  with values reflecting their relative importance. For example, the importance vector  1018  includes a corresponding entry (e.g., a floating point value) for each of the components  1006  of the NLU framework  1000 , wherein the magnitude of the value reflects the positive impact of the component during utterance inference. As discussed below, the importance vector  1018  may be used for mutating a configuration vector, selecting between configuration vectors that provide similar performance metrics, or combinations thereof. Additionally, the modeling and optimization system  1004  may use the performance metrics  1016  to determine, for each configuration vector, which paths each of test utterances  1012  traversed through the NLU framework  1000  during inference, and use this to construct an utterance flow model  1020 . The utterance flow model  1020  can be used to determine, for a given utterance, what potential paths through the NLU framework  1000  the utterance may take during inference. In certain embodiments, the modeling and optimization system  1004  may be configured to generate a diagram visually indicating these paths to a user or designer of the NLU framework  1000  to enhance explainability. 
     In addition to modeling, the modeling and optimization system  1004  can also enable the optimization of the one or more configuration vectors  1008  to provide a desired level of performance within the NLU framework  1000 . For the embodiment illustrated in  FIG. 35 , the modeling and optimization system  1004  also includes one or more optimization plugins  1022  and one or more optimization constraints  1024 . The optimization plugins  1022  may include any suitable optimization plugin, such as a stochastic gradient descent (SGD) plugin, a particle swarm plugin, or any other suitable optimization plugins. The optimization constraints  1024  generally include values indicating when an optimization process should terminate. For example, in certain embodiments, the optimization constraints  1024  may include the value of an objective function that should be reached or exceeded to indicate that the configuration vector  1008  has been sufficiently optimized. In certain embodiments, the optimization constraints  1024  may include a time limit, a memory size limit, a number of iterations limit, or any other suitable limit value for the optimization process discussed below. 
     To illustrate the operation of the modeling and optimization system  1004 ,  FIGS. 36, 37 and 38  are flow diagrams illustrating embodiments in which the modeling and optimization system  1004  provides test utterances  1012  to a NLU system  1002  of a NLU framework  1000  to process the test utterances  1012  based on a particular configuration vector  1008  (e.g., configuration vector  1008 A, configuration vector  1008 B, and configuration vector  1008 C) to determine respective performance metrics  1016  (e.g., performance metrics  1016 A, performance metrics  1016 B, performance metrics  1016 C) for each configuration. For these examples, the configuration vectors are binary configuration vectors, wherein each value indicates that a particular component of the NLU framework  1000  is either enabled or disabled. These figures are discussed with reference to elements illustrated in  FIGS. 5, 6, and 35 . Additionally, while  FIGS. 36-38  are described with respect to test utterances  1012 , it may be appreciated that the figures also represent the operation of the NLU framework to perform NLU inference of a received user utterance in accordance with the particular configuration vector  1008 . 
     For the embodiment illustrated in  FIG. 36 , when the configuration vector  1008 A is used, an example test utterance  1040  may take a number of different paths through the NLU system  1002  during inference. For example, the test utterance  1040  may be provided along arrow  1042  to suitable vocabulary subsystem  1044  (labeled as “Encoder”) that generates a semantic vector for the test utterance  1040 , and then a vector comparison component  1046  of the meaning search subsystem  152  may attempt to match the semantic vector of the test utterance  1040  to semantic vectors generated from entire sample utterances of the intent-entity model to extract and score artifacts  1047  (e.g., intents and/or entities of the test utterance  1040 , corresponding confidence scores). In certain embodiments, this type of meaning search may be referred to as an utterance meaning search operation, the semantic vector representing the test utterance  1040  may be referred to as a search key (e.g., an utterance search key), and the semantic vectors representing the sample utterances of the intent-entity model may be referred to as a search space (e.g., an utterance search space). In certain embodiments, the configuration vector  1008 A defines which components  1006  of the NLU framework  1000  are enabled and disabled, as well as numerical parameters used by these components  1006 , when generating the search key and search space for the utterance meaning search. 
     For the embodiment illustrated in  FIG. 36 , the test utterances  1012  may also be provided along arrow  1048  to suitable structure subsystem  1050  (e.g., Multi-fold Aggregation of Parses via Linguistic Expansion (MAPLE)) for parsing, autocorrection, and re-expression, as discussed above, during a NLU meaning search operation. For the embodiment illustrated in  FIG. 35 , parses  1 ,  2 , and  3  of the test utterance  1040  are generated, and the autocorrection rules  1052  only enable parse  2  to advance to re-expression along arrow  1054 . During re-expression, the re-expression rules  1056  (e.g., model generalizing rules, model refining rules) generate alternative forms of the test utterance  1040 , which are parsed to yield parses  2 A,  2 B, and  2 C. Then, the NLU system  1002  determines cognitive construction grammar (CCG) form identities for each of these parses. As such, CCG form 3 of parse  2 A, CCG form 2 of parse  2 B, and CCG form 1 of parse  2 C are advanced along arrows  1058  to the vocabulary subsystem  1044  for vector encoding and construction of the meaning representations  162  of the utterance meaning model  160 , as discussed above. After vector encoding, the meaning search subsystem  152  may perform a meaning search operation, during which compatible CCG forms of these meaning representations  162  are compared to meaning representations  158  of the understanding model  157  by the compatible form comparison component  1049 , as discussed above. In certain embodiments, the configuration vector  1008 A defines which components  1006  of the NLU framework  1000  are enabled and disabled, as well as numerical parameters used by these components  1006 , when generating the meaning representations  158  of the understanding model  157  in advance of this meaning search operation. 
     As illustrated in  FIG. 36 , a component of the NLU framework  1000  (e.g., an autocorrect rule, a model generalizing rule, a model refining rule, a CCG form, a FAM coefficient) that is applied while processing the utterance at an earlier stage can determine which components  1006  will be applied at a later stage when processing the utterances. Thus, for the same utterance, removing a single component of the NLU framework  1000 , such as a re-expression rule, may result in a different CCG form identity at a later stage of processing than without removal of the re-expression rule. As such, the explainability of such a NLU framework  1000  becomes increasingly complex with each additional component. 
     For example,  FIG. 37  illustrates an embodiment of the NLU framework  1000  in which a single component (e.g., CCG form 3) is disabled by the modeling and optimization system  1004  in the configuration vector  1008 B. For the illustrated example, disabling the CCG form effects several components downstream of the disabled component in the process. For example, by disabling CCG form 3, this CCG form can no longer be compared using CCG form algebra to any sample utterances represented within the search space. This may result, for example, in the NLU framework  1000  determining the scored artifacts  1047  of the test utterances based on the utterance meaning search of the entire user utterance, rather than the CCG form comparison during the NLU meaning search operation, which may include different intents and/or entities than before the CCG form was disabled. 
       FIG. 38  illustrates an embodiment of the NLU framework  1000  in which two components (e.g., CCG form 2 and a re-expression rule that previously generated parses  2 A and  2 C) are disabled by the modeling and optimization system  1004  in the configuration vector  1008 B to determine how strongly the components are related. For example, the modeling and optimization system  1004  may first determine performance metrics  1016  for the configuration vector  1008 A, in which no components of the NLU framework  1000  are disabled, as illustrated in  FIG. 36 . Then, the modeling and optimization system  1004  may determine performance metrics  1016  for configuration vector  1008 B, in which only one of the two components  1006  of the NLU framework  1000  are disabled. Finally, the modeling and optimization system  1004  may determine performance metrics  1016  for the configuration vector  1008 C, in which both of the components of the NLU framework  1000  are disabled, as illustrated in  FIG. 37 . The modeling and optimization system  1004  may determine, based on the collected performance metrics  1016 , that the NLU framework  1000 , without any components disabled, provides a certain level of precision. The modeling and optimization system  1004  may then determine that, when either of the two components is disabled, the precision of the NLU framework  1000  drops by a first amount (e.g., 5%). The modeling and optimization system  1004  may further determine that, when both of the components are disabled, the precision of the NLU framework drops by a substantially larger amount (e.g., 20%). In such a situation, the modeling and optimization system  1004  determines that these two components strongly interact during inference of the test utterances  1012 . 
     For the example illustrated in  FIG. 38 , when the modeling and optimization system  1004  disables CCG form 2 and the re-expression rule that previously generated parses  2 A and  2 C, the scored artifacts  1047  are based solely on the utterance meaning search operation (i.e., the semantic vector matching of the entire test utterance  1040 ). That is, without the re-expression rule, only CCG form 2 results in a valid parse tree, and by disabling the re-expression rules, the NLU framework  1000  has no way to generate meaning representations  162  or perform the NLU meaning search operation. It may be appreciated that the modeling and optimization system  1004  enables these relationships and patterns to be observed, improving understanding of the inter-component relationships and their impact to the overall operation of the NLU framework  1000 . 
     As such, by systematically disabling components and determining the effect on the performance metrics  1016  of the NLU framework  1000 , the modeling and optimization system  1004  can determine the relative importance of each of the components to the operation of the NLU framework  1000 , as well as how strongly the operation of one component affects the operation of another component of the framework. The modeling and optimization system  1004  may also determine all of the different paths that each of the test utterances  1012  take through the NLU framework  1000  to produce a particular set of scored artifacts  1047 . In certain embodiments, the modeling and optimization system  1004  may be configured to provide a diagrammatical output that visually depicts each potential path that the test utterances  1012  may take during inference, which enables the ability of users and designers of the NLU framework  1000  to understand, interpret, and explain the operation of the NLU framework  1000  for a given utterance. The modeling and optimization system  1004  can also be used to identify components of the NLU framework  1000  that should be improved (e.g., components that negatively impact the performance metrics  1016 ), as well as what impact that improvement might have to the overall performance of the framework. Furthermore, the modeling and optimization system  1004  may be used to evaluate new components that are being considered for inclusion in the NLU framework  1000  to determine how these components will interact with the other components of the NLU framework  1000 , as well as how they will improve or degrade the overall performance of the framework. 
     As noted above, in addition to enabling enhanced insight into the interoperation of the various components of the NLU framework  1000  and their effect on the overall performance of the NLU framework  1000 , the modeling and optimization system  1004  may also be used to optimize the configuration of the components of the NLU framework  1000 .  FIG. 39  is a flow diagram illustrating an embodiment of an optimization process  1060  whereby the modeling and optimization system  1004  applies ML-based techniques to “learn” or automatically determine optimized configurations for components of the NLU framework  1000 . More specifically, the optimization process  1060  is an implementation of a genetic algorithm or technique, which generates mutations of configuration vectors  1008  and compares the performance of each mutation to optimize a set of components of the NLU framework  1000  over a number of iterations.  FIG. 39  is discussed with reference to elements illustrated in  FIG. 35 . 
     It may be appreciated that the optimization process  1060  may be used to optimize the entire NLU framework  1000 , a subsystem or group of components  1006  of the NLU framework, a single component  1006  of the NLU framework, or any combination thereof. For example, in certain embodiments, the optimization process  1060  may be used to optimize a configuration vector having binary entries representing which components of the NLU framework are enabled or disabled to optimize the NLU framework, or representing which components of a subsystem are enabled or disabled to optimize the subsystem. In certain embodiments, the optimization process  1060  may be used to optimize a configuration vector having numerical entries representing threshold values, coefficient values, weight values, and so forth, for components  1006  of the NLU framework  1000 . However, regardless of which portion of the NLU framework  1000  is being optimized, it may be appreciated that the portion is optimized based on the operation of the entire NLU framework  1000 . For example, when the components  1006  of a subsystem of the NLU framework  1000  is being optimized, the configuration vector being optimized represents components of the subsystem; however, the performance metrics  1016  collected when using the configuration vector are based on the performance of the entire NLU framework  1000  when the components  1006  of the subsystem utilize the configuration represented within the configuration vector. 
     For the embodiment illustrated in  FIG. 39 , the optimization process  1060  begins with the modeling and optimization system  1004  creating (block  1062 ) and initializing the configuration vector  1008  (e.g., either with all null values or based on the current configuration of the NLU framework  1000 ), and creating an empty set of candidate configuration vectors. The modeling and optimization system  1004  determines (decision block  1064 ) whether an optimization constraint  1024  (e.g., an objective function value, a time limit, an iteration limit) of the modeling and optimization system  1004  has been reached during execution of the optimization process  1060 . 
     For the embodiment of the optimization process  1060  illustrated in  FIG. 39 , in response to the modeling and optimization system  1004  determining that one or more of the optimization constraints  1024  have not yet been reached, the modeling and optimization system  1004  evaluates (block  1066 ) the performance of the NLU framework by inferencing multiple sets of test data  1010  using the current configuration vector  1008  and collecting the resulting performance metrics  1016 , as discussed above. The modeling and optimization system  1004  then calculates (block  1068 ) a difference between the current configuration vector and the previous configuration vector and stores the difference for later use. For example, the modeling and optimization system  1004  may compute a difference vector, in which each entry stores a calculated difference between a corresponding entry in the current configuration vector and the previous configuration vector. Since there is not a previous configuration vector for the first iteration through the optimization process  1060 , the difference vector may initially be filled with entries having a value of zero. 
     For the embodiment of the optimization process  1060  illustrated in  FIG. 39 , the modeling and optimization system  1004  compares (block  1070 ) the performance metrics of the current configuration vector  1008  to previous performance metrics determined during previous iterations of the optimization process  1060 . Based on this comparison, the modeling and optimization system  1004  updates the values of the corresponding entries in the importance vector  1018 . For example, when disabling a component results in a substantial reduction of a performance metric, the modeling and optimization system  1004  may calculate and include an entry in the importance vector  1018  for the component that is calculated based on this reduction of performance. The importance vector  1018  may be used when determining which configuration vectors are better or worse than others, when mutating configuration vectors, or any combination thereof, as discussed below. In certain cases, when the configuration vector lacks numerical values, the actions of blocks  1068  and/or  1070  may be skipped. 
     For the embodiment of the optimization process  1060  illustrated in  FIG. 39 , the modeling and optimization system  1004  determines (decision block  1072 ) whether the current configuration vector performed better that the candidate configuration vectors. For example, the modeling and optimization system  1004  may determine whether the performance metrics  1016  determined in block  1066  for the current configuration vector are better than the performance metrics determined for other candidate configuration vectors during other iterations of the optimization process  1060 . When the performance metrics of the current configuration vector are worse than the performance metrics of at least one of the candidate configuration vectors, then the modeling and optimization system  1004  discards (block  1074 ) the current configuration vector. When the performance metrics of the current configuration vector are better than the performance metrics all of the candidate configuration vectors, then the modeling and optimization system  1004  adds (block  1076 ) the current configuration vector to the set of candidate configuration vectors and discards any candidate configuration vector having performance metrics that are worse than the performance metrics of the current configuration vector  1008 . Since the set of candidate configurations is initially empty for the first iteration through the optimization process  1060 , the current configuration is saved to the set of candidate configurations. 
     When determining which performance metrics  1016  represent a better or worse performance of a configuration vector, the modeling and optimization system  1004  may compare various values of the performance metrics  1016  provided by each configuration vector. For certain metrics (e.g., precision, recall) higher values may correspond to better performance, while lower values may correspond to better performance for other metrics (e.g., latency, processing time, memory usage). In certain embodiments, the comparison may be customized for a particular client, for example, based on the NLU performance and/or computation performance needs and limitations of the client. In certain embodiments, the modeling and optimization system  1004  may determine that the performance metrics of two configuration vectors are tied, meaning that the values of corresponding performance metrics are within a threshold value of each other. To determine which is the better configuration vector, in certain embodiments, the modeling and optimization system  1004  may use the importance vector  1018  to resolve the tie, for example, by comparing the dot product of the importance vector  1018  and the first configuration vector to the dot product of the importance vector  1018  and the second configuration vector, and then selecting the configuration vector with the higher value as the better configuration vector. 
     For the embodiment of the optimization process  1060  illustrated in  FIG. 39 , the modeling and optimization system  1004  selects (block  1078 ) a random configuration vector from the set of candidate configuration vectors and mutates (block  1080 ) the selected configuration vector. It may be appreciated that, depending on the nature of the configuration vector, the mutation can occur in different manners. For example, when the configuration vector includes binary values representing components of the NLU framework  1000  that are activated or deactivated, the modeling and optimization system  1004  may iterate through each binary entry in the selected configuration vector, and may probabilistically flip bits to the opposite value based on a particular probability (e.g., the inverse of the number of binary entries in the configuration vector). In certain embodiments, the probability may be weighted based on the values in the importance vector  1018 , resulting in components having higher values in the importance vector  1018  being less likely to be mutated (e.g., flipped). For example, in certain embodiments, the modeling and optimization system  1004  may probabilistically flip bits to the opposite value based on a probability calculated using the importance vector value (e.g., the inverse of the number of binary entries in the configuration vector multiplied by the inverse of the corresponding importance vector value). 
     In certain embodiments, when the configuration vector includes numerical entries representing thresholds, coefficients, weight values, and so forth, the modeling and optimization system  1004  mutates the selected configuration vector in different manners. For example, in certain embodiments, the modeling and optimization system  1004  mutates each of the numerical entries in the configuration vector based, at least in part, on the corresponding entries in the importance vector  1018 , wherein higher values in the importance vector  1018  result in a reduced probability of mutation and/or a reduced change in value during mutation. In certain embodiments, the modeling and optimization system  1004  mutates each of the numerical entries in the configuration vector based, at least in part, on the corresponding entries in the difference vector calculated in block  1068 , which can guide the mutation of numerical values configuration vector relative to past mutations. 
     In certain embodiments, the modeling and optimization system  1004  mutates the selected configuration vector utilizing an optimization plugin  1022  (e.g., particle swarm optimization plugin, a differentiable or non-differentiable optimization plugin, a neural network model plugin). For example, the modeling and optimization system  1004  may provide one or more inputs to the optimization plugin  1022 , such as the importance vector  1018 , configuration vectors of the current iteration or previous iterations of the optimization process  1060 , performance metrics  1016  determined during the current iteration or previous iterations of the process  1060 , and/or difference vectors calculated for the current iteration or previous iterations of the process  1060 . Based on these inputs, the optimization plugin  1022  (e.g., a particle swarm plugin) may gradually increase or decrease the numerical entries of the current configuration vector across iterations of the optimization process  1060 , seeking to maximize an objective function. The objective function may be determined from the performance metrics determined in block  1066 . For example, in certain embodiments, the objective function may be calculated as a number of artifacts correctly extracted by the NLU framework  1000  using the current configuration vector divided by the total number of artifacts extracted by the NLU framework using the current configuration, as determined when processing the test data  1010  in block  1066 . In certain embodiments, the optimization plugin  1022  may execute in a separate process or thread from other portions of the optimization process  1060  for enhanced efficiency and/or reduced runtime. 
     For the embodiment of the optimization process  1060  illustrated in  FIG. 39 , once the current configuration vector has been mutated in block  1080 , the modeling and optimization system  1004  returns to decision block  1064 , and determines whether at least one of the optimization constraints  1024  have been reached. In response to determining that none of the optimization constraints  1024  have been reached or satisfied, the modeling and optimization system  1004  returns to step  1066 , repeating the portion of the process  1060  discussed above. In response to determining at least one of the optimization constraints  1024  have been reached or satisfied, then the optimization process  1060  concludes with the modeling and optimization system  1004  saving (block  1082 ) the top performing configuration vector from the set of candidate configurations vectors as the optimized configuration of the NLU framework  1000  (or the optimized configuration of one or more components  1106  of the NLU framework  1000 ). Additionally, any relevant information collected during the optimization process  1060  (e.g., number of iterations, time per iteration, total runtime, the importance vector  1018 , the performance metrics  1016 , the utterance flow model  1020 ) may be saved for later review. 
       FIG. 40  is a flow diagram illustrating an embodiment of a distributed optimization process  1100  whereby the modeling and optimization system  1004  optimizes subsets of a configuration vector in parallel for reduced runtime. While the illustrated embodiment includes three servers  1102  (e.g., server  1102 A, server  1102 B, server  1102 C), which may be virtual servers or hardware servers, in other embodiments any suitable number of servers may be used. The illustrated process  1100  begins with the modeling and optimization system  1004  randomly splitting (block  1104 ) the configuration vector to be optimized into three subarrays, each representing a random subset of components of the NLU framework  1000  to be optimized. Each of these configuration vector subarrays are provided to a respective server, and each of the servers  1102  independently performs (blocks  1106 A,  1106 B, and  1106 C) the optimization process  1100  of  FIG. 39  on its respective configuration vector subarray. 
     For the embodiment of the distributed optimization process  1100  illustrated in  FIG. 40 , once each of the servers  1102  has completed their respective optimization process, the modeling and optimization system  1004  receives the optimized configuration vector subarrays from each of the servers  1102 , and combines these together to form a complete configuration vector (block  1108 ). The modeling and optimization system  1004  then performs (block  1110 ) additional iterations of the optimization process  1060  of  FIG. 39 , using the combined configuration vector as a starting point. The distributed optimization process  1100  concludes with the modeling and optimization system  1004  storing (block  1112 ) the optimized configuration for the NLU framework  1000  (e.g., for one or more components  1006  of the NLU framework  1000 ). Additionally, any relevant information collected during the distributed optimization process  1100  (e.g., number of iterations, time per iteration, total runtime, the importance vector  1018 , the performance metrics  1016 , the utterance flow model  1020 ) may be saved for later review. 
     In certain embodiments, the distributed optimization process  1100  can be used to drill down into the structure of the NLU framework  1000 . For example, the modeling and optimization system  1004  may begin by performing the optimization process  1060  of  FIG. 38  on the entire NLU framework  1000  using a configuration vector with binary entries representing components of the NLU framework  1000  that are enabled and disabled. Once the modeling and optimization system  1004  determines that a particular component is important (e.g., the component has a corresponding value in the importance vector  1018  that reaches a predefined importance threshold value), and determines that this component includes other components (e.g., numerical threshold values, coefficients, weight values), then the modeling and optimization system  1004  may begin a separate optimization process  1060  on a separate server to optimize these numerical values, and subsequently merge the configuration vectors and perform iterations of the optimization process  1060  on the combined configuration vectors, as discussed above. 
     Technical effects of the present disclosure include providing an agent automation framework that is capable of extracting meaning from user utterances, such as requests received by a virtual agent (e.g., a chat agent), and suitably responding to these user utterances. A NLU framework includes a modeling and optimization system that enables enhanced understanding and explainability to the operation of the NLU framework during utterance inference. The modeling and optimization system learns or automatically determines optimized configurations for the NLU framework, such as which components of the NLU framework should be activated or deactivated, as well as which numerical values (e.g., threshold values, coefficients, weight values) are used by these components during utterance inference. 
     The specific embodiments described above have been shown by way of example, and it should be understood that these embodiments may be susceptible to various modifications and alternative forms. It should be further understood that the claims are not intended to be limited to the particular forms disclosed, but rather to cover all modifications, equivalents, and alternatives falling within the spirit and scope of this disclosure. 
     The techniques presented and claimed herein are referenced and applied to material objects and concrete examples of a practical nature that demonstrably improve the present technical field and, as such, are not abstract, intangible or purely theoretical. Further, if any claims appended to the end of this specification contain one or more elements designated as “means for [perform]ing [a function] . . . ” or “step for [perform]ing [a function] . . . ”, it is intended that such elements are to be interpreted under 35 U.S.C. 112(f). However, for any claims containing elements designated in any other manner, it is intended that such elements are not to be interpreted under 35 U.S.C. 112(f).