Incremental Learning for Knowledge-Grounded Dialogue Systems in Industrial Scenarios (2024)

Abstract: In today’s industrial landscape, seamless collaboration between humans and machines is essential and requires a shared knowledge of the operational domain. In this framework, the technical knowledge for operator assistance has traditionally been derived from static sources such as technical documents. However, experienced operators hold invaluable know-how that can significantly contribute to support other operators. This work focuses on enhancing the operator assistance tasks in the manufacturing industry by leveraging spoken natural language interaction. More specifically, a Human-in-the-Loop (HIL) incremental learning approach is proposed to integrate this expertise into a domain knowledge graph (KG) dynamically, along with the use of in-context learning for Large Language Models (LLMs) to benefit other capabilities of the system. Preliminary results of the experimentation carried out in an industrial scenario, where the graph size was increased in a 25%, demonstrate that the incremental enhancing of the KG benefits the dialogue system’s performance.

## introduction human-machine interaction (hmi) is revolutionizing traditional industrial processes. smart manufacturing relies on the collaboration between highly advanced machinery and the knowledge and decisionmaking abilities of human operators. the industry of the near future requires qualified personnel specialized in technologies such as robotics and artificial intelligence (ai), capable of making informed decisions based on these factors. in this context, a human-centered approach positions operators as a crucial element in new industrial plants. thanks to the latest technological advances, voice interaction between operators and industrial manufacturing systems or machines is now feasible. moreover, these technologies are hands-free and eyes-free, enabling operators to perform physical tasks, support natural language communication that requires minimal training, and are highly flexible, allowing communication at various levels of detail. consequently, there has been an increase in the number of prototypes and systems exploring the use of voice as a natural interaction interface between operators and machines in industrial environments in recent years. additionally, dialogue modeling and management have drastically changed due to the recent success of large language models (llms). however, any application based on llms needs reliable and up-to-date knowledge sources. in particular, industrial scenarios require robust models capable of handling very technical and precise knowledge, which is necessary for tasks shared by humans and machines. traditionally, hmi has relied on rule-based systems to represent knowledge and actions, ensuring everything remains under control. as a result, these interaction systems are static, failing to capture the expert human knowledge of the factory that is not documented or included in the system's knowledge base. this limitation can be addressed through the concept of human in the loop (hil), also known as operator in the loop (oil) in industrial contexts. these ai systems facilitate collaboration between humans and machines to enhance results and accelerate the learning process. the hil paradigm involves continuous interaction throughout all postdeployment stages of ai models. as illustrated in figure 1 , in the industrial sector, the oil paradigm enables the integration of expert knowledge into hmi interfaces by providing feedback using natural language. this approach allows voice interaction systems to evolve over time, adapting to the unique dynamics of each factory and incorporating the expertise that operators develop. in this work, an oil incremental learning approach to manage knowledge-grounded, taskoriented dialogue (tod) systems in industrial settings is proposed, being its main contributions (1) extending a previously defined ontology to support the management and storage of new knowledge provided by experts; (2) developing online learning capabilities to collect user feedback, thereby updating and expanding a knowledge graph and (3) developing an llm-based natural language understanding (nlu) system that queries a kg to constrain it within the task. preliminary evaluations show promising results in nlu performance and kg grounding. the rest of the paper is structured as follows: section 2 presents the related work. the proposed knowledge graph-based incremental dialogue system with the detailed description of each of its modules is described in section 3, and the initial validation results in section 4. finally, conclusions of this work are shown in section 5. ## related work the current state of the art considers knowledge graphs as a useful asset in industrial settings and, more specifically in human-centric approaches @xcite , such as robot interaction and collaborative manufacturing @xcite . in this line, approaches such as the one proposed by @xcite are observed, in which knowledge graphs are used to model factors related to the operator and their conditions, such as movements or collaboration with machines. moreover, knowledge graphs have been used in this scenario for task-oriented dialogue, which enable operators to communicate to industrial systems in a more natural way. in this context, knowledge graphs have been traditionally used to model the domain of the use case, providing a detailed representation of the scenario and reducing ambiguity between the agents involved @xcite . however, more modern approaches also make use of knowledge graphs for dialogue management @xcite of course, this process also has an impact on dialogue management, since one of the most widespread techniques is to obtain this information from users. to do this, dialogues are generated dynamically to be able to obtain the necessary information for the system to learn, as well as the appropriate moment for it, based on a strategy @xcite . some approaches also base these interactions on the feedback obtained from the user taking into account, for example, evaluations such as "it's not what i wanted" or "you didn't understand me well" @xcite . in the field of natural language processing (nlp), there's a clear surge in leveraging stateof-the-art strategies across multiple applications, particularly through the deployment of pre-trained large language models (llms) in dialogue systems. @xcite describes these models as ai models that often, though not exclusively, stem from the transformer architecture. they are crafted to understand and generate human language, code, and beyond. also, they are trained on immense troves of text, and they can tackle a vast array of language-related tasks, from simple text classification to elaborate text generation. as highlighted by this author, the llms available in the market (like various versions of gpt, gemini, llama, among others) have been pre-trained on extensive datasets from diverse sources using distinct methodologies. thus, not all llms perform equally, and their training processes significantly influence their performance in specific applications. therefore, to optimize pre-trained language models for task-oriented dialogue systems, different works employ models like alpaca, gpt-neo, bart, t5, llama 2 and gpt-3.5 @xcite @xcite ; among others. also, different authors adopt various approaches for constructing these dialogue systems. prominent among these is fine-tuning pre-trained language models using methods like lora (low rank adaptation) @xcite , prompt tuning @xcite and reinforcement learning from human feedback (rlhf) @xcite , among others. this indicates a clear trend in using llms in tod sys-tems and various optimization strategies. however, many of these methods demand substantial specific data for training as they are data-driven, which may not be available for certain industrial use cases. to address this issue and avoid the need for handcrafted rules, in-context learning approaches are becoming increasingly popular. these approaches involve designing prompts using snippets of example dialogues, the user's goal, and the dialogue history @xcite . this optimization method, known as prompt tuning, allows adapting the model to task requirements without requiring a corpus or extensive training, just relying on natural language instructions to guide the model's behaviour. ## knowledge-grounded incremental task-oriented dialogue system two of the most common applications of tod systems in industrial scenarios are to provide assistance through processes and to deliver tasks to a certain industrial intelligent system. therefore, the expected interactions from the user can mainly be classified as navigation instructions through processes and action requests to industrial systems, respectively. the tod system's responses, on the other side, must be in the form of steps of the processes on which the user will request assistance for the former, and the corresponding machinereadable action for the latter. so, in this type of scenarios, towards an incremental approach, feedback may be useful in these two situations, mainly: (1) the content presented does not meet the needs of the user or (2) the interpretation of the interaction indicates that what the user wants to do next or deliver to the system is not appropriate. this work presents the extension and adaptation of kide4i, presented in @xcite and based on the todo ontology @xcite , to provide it with feedback-capturing and management capabilities. the aim of such task is to achieve a system that is capable of learning from interactions with users over time and, thus, improve its interpretation and dialogue capacities, as well as adapting to the users' needs. to this end, the following aspects have been addressed: likewise, and towards assessing the benefits when updating kide4i with the most recent technologies, in-context learning (icl) of llms through prompt-tuning has been explored and implemented in the natural language understanding (nlu) module, as detailed in section 3.3. this task has allowed to compare more traditional strategies, such as rule-based ones, with the most disruptive one nowadays: the use of llms in scenarios with limited resources (in terms of training corpus), such as industrial ones. ## initial results the extended tod system with feedback capabilities has been tested and validated in an experimentation task. in this particular case, the system's function was to provide support through the different phases for manufacturing a piece using cnc programming on a milling machine. when given instructions, the users that were not satisfied with the answer given by the assistant would mark the response as inadequate. some initial results related to this experimentation, mainly regarding the impact of those iterations in the knowledge graph, are presented in section 4.1. in terms of incrementing knowledge through feedback, the assistance scenario is suitable for evaluation. however, the variability in the interactions (e.g. "show me the next step", "i need more information") is limited. due to this, and in order to provide more insights, the llm-based and rule-based nlu components have been tested in a collaborative bin-picking scenario, which is richer in terms of references to key elements in user commands. the obtained results for both have been compared in order to determine if the llmbased approach, which makes the task of adapting the dialogue system easier, is able to maintain or improve the accuracy of the rule-based approach. the experimental setup, as well as the benchmark results, are presented in section 4.2. ## conclusions this work introduces a knowledge graph-based method for managing the knowledge base of a task-oriented dialogue system for industrial settings, in which the knowledge graph is in charge of storing both domain and dialogue-managementrelated knowledge. this dialogue system features incremental learning capabilities that, by using the hil/oil paradigms, allows, on the one hand, for users to give feedback regarding the output of the system and, on the other hand, for experts to improve the knowledge included in the knowledge graph according to operators' feedback. for this, the ontology used in the knowledge graph, which originates from an existing ontology for taskoriented dialogue systems, has been extended to cover the addition of knowledge and the generation of additional dialogues for that end. furthermore, for the natural language understanding (nlu) module, which originally was de-signed by following a rule-based approach, has been implemented by using large language models (llms) to improve both the system's maintenance and the quality of the interpretations obtained by it. the system has been evaluated in two real-world industrial settings: a bin-picking scenario, in which the nlu component was implemented by using llms, and a manufacturing scenario, in which the incremental learning capabilities of the system have been tested. for the first scenario, the results show that the performance of llm-based nlu is higher than the rule-based approach by 16%, which is a significant improvement, especially for the fact that llms are easier to adapt to other scenarios than rules. for the second scenario, the addition of feedback interfaces has allowed to improve the existing knowledge graph of the system. the result of this is the addition of more explanations to existing procedures and even two new procedures; all in all, this translates into 25% more knowledge than at the time of the experimentation. this is expected to impact positively in the system's performance from now on, which will be evaluated in a new experimentation task as part of future work. these results show that the use of knowledge graphs for managing the knowledge base of taskoriented dialogue systems in industrial settings is a promising approach, especially when combined with incremental learning capabilities, and that the use of llms for other modules of the system leads to systems that are easy to maintain over time and to adapt to new scenarios.