T rust B oost: Balancing flexibility and compliance in conversational AI systems (2025)

Abstract: Conversational AI (ConvAI) systems are gaining growing importance as an alternative for more natural interaction with digital services. In this context, Large Language Models (LLMs) have opened new possibilities for less restricted interaction and richer natural language understanding. However, despite their advanced capabilities, LLMs can pose accuracy and reliability problems, as they sometimes generate factually incorrect or contextually inappropriate content that does not fulfill the regulations or business rules of a specific application domain. In addition, they still do not possess the capability to adjust to users’ needs and preferences, showing emotional awareness, while concurrently adhering to the regulations and limitations of their designated domain. In this paper we present the TrustBoost project, which addresses the challenge of improving trustworthiness of ConvAI from two dimensions: cognition (adaptability, flexibility, compliance, and performance) and affectivity (familiarity, emotional dimension, and perception). The duration of the project is from September 2024 to December 2027.

## introduction the term "conversational ai" (convai) has gained prominence in recent academic research, encompassing several nlp technologies like dialog systems, chatbots, and intelligent virtual assistants. these systems leverage artificial intelligence extensively to facilitate natural language conversations, offering users a seamless and intuitive way to access information, interact with services, and navigate data on the internet, as well as their surrounding environment @xcite @xcite in recent years, the development of llms has significantly transformed the landscape of conversational ai, offering unparalleled interaction flexibility. unlike traditional rule-based or intent-based dialogue systems, llms can understand and generate human-like language that is less restricted because they are not as tied to specific training phrases. they have the ability to grasp context, adapt to diverse user inputs, and generate coherent responses in multiple conversational scenarios. as for dialogue/interaction management, their pretraining on vast and diverse datasets enables them to generalize well to various domains, eliminating the need for explicit rule programming. although rule-based systems often struggle to accommodate dynamic and evolving language patterns, llms contribute to a more natural and engaging user experience. on the opposite side of the spectrum, intent-based dialogue systems have the advantage of being completely compliant with business rules and domain restrictions due to their inherent structure and rule-based logic. in intent-based systems, user interactions are categorized into predefined intents, each associated with a specific action or task. this structured approach provides a level of control and predictability that is advantageous to maintaining compliance. regarding language understanding, intent-based systems are often specialized in specific domains or applications, enabling a focused understanding of user queries within a defined domain. in terms of dialogue management, this specialization contributes to a better alignment with domain-specific rules and restrictions, offering explainability. conversational llms are considered the future. however, they are still not widely adopted due to trustworthiness issues @xcite . trust is crucial for user acceptance and engagement. trust in ai systems is intricately linked to users' expectations of consistency, reliability, and adherence to established norms. business rules serve as a set of guidelines that dictate the permissible behavior of the ai system within a given context. when a conversational system lacks compliance with domain restrictions, it introduces an element of unpredictability and inconsistency in its responses. this deviation from expected behavior can undermine users' trust. moreover, trust in convai systems is intricately connected to interpretability: if users cannot understand the reasoning behind the system's decisions, it creates a perceived lack of transparency. from the perspective of the entity/company offering the conversational system to users, when a conversational system fails to align with business rules, the provider faces the risk of diminished user trust. this can lead to a decline in user engagement, increased user dissatisfaction, and potential reputational damage. in more extreme cases, they could even face legal and regulatory consequences. ## description in trustboost, we assume that for convai systems to be truly usable and reliable, they must adhere to the rules and restrictions of their designated domain as well as to adapt to their users' needs and preferences (cognition/performance branch of trustworthiness). with this aim, we will address the balance between flexibility and compliance by endowing convai systems powered by llms the ability to comply with business/domain rules while simultaneously adapting to the needs of their users. one of the primary challenges in incorporating rule-based constraints into the training and finetuning processes of llms is the discrepancy between the vast and diverse data sources typically used to train llms and the specific guidelines and regulations defined by organizations in the form of business rules. training data may not adequately capture these intricacies, as llms lack the context of organizational policies and domain idiosyncrasies. moreover, fine-tuning llms to adhere perfectly to business rules can be challenging due to the limited nature of fine-tuning datasets, which may not fully represent the complexity of these rules. additionally, llms often generate outputs that are not easily interpretable, making it difficult to ensure that the generated content aligns with spe-cific business rules and compliance requirements. due to the computational cost and resource constraints associated with training llms, their accessibility has remained limited to a select group of organizations with significant computational resources. to address this challenge, we propose to take advantage of the capabilities of 'big" pretrained llms to generate new resources, such as synthetic dialogues, that can be used to develop or fine-tune smaller and more efficient llm models tailored to specific tasks and domains. these compact models would be able to run on average gpus, expanding the reach of llm technology to a wider range of organizations and applications. this approach would effectively break down the existing barrier to entry for llms, democratizing their use and fostering innovation across industries. concurrently, we will devise automated evaluation procedures for the resources generated, minimizing manual review while ensuring the efficiency and reliability of the resources produced. within the framework of the trustboost project, we also aim to protect data governance and sovereignty by promoting the use of open source llms (to avoid sending data to third parties, unlike e.g. chatgpt alternatives), together with techniques that make it possible to combine smaller models fine-tuned to specific tasks, ensuring the protection of personal data, lower hardware requirements for learning and deployment, and more efficient energy use. in addition, in trustboost a truly flexible con-vai system should dynamically tailor its responses based on the user's emotional cues (affective branch of trustworthiness). this adaptability involves adjusting the tone, language, content, and type of responses to align with the detected emotional state, thereby creating a more personalized and empathetic interaction. by doing so, the ai system can dynamically adapt the dialog flow to the identified emotional states, enhancing the overall user experience and building trust. the challenge of recognizing the user's emotional state in convai systems involves addressing multiple research goals. the first goal is the development of sophisticated natural language processing (nlp) techniques to extract and interpret emotional cues from various sources such as facial expressions, tone of voice, and word choice. multimodal data integration, including spoken language, facial expressions, and body language, is crucial to enhance emotional recognition. recognizing the dynamic nature of emotions during interactions is also important. contextualizing emotional cues within the conversation and the user's overall situation is vital for providing personalized and empathetic responses. however, current conversational ai struggles to grasp the broader emotional landscape, facing limitations in understanding evolving user sentiments due to context window constraints. overcoming these challenges requires advances in emotional state recognition and improving the contextual understanding capabilities of language models to enhance emotional intelligence in conversational interactions. ## main objectives the main objective of trustboost is to find new methods for trustful conversational ai balancing performance and affectiveness. in order to achieve this aim, we will address several research lines. first, enhancing user-awareness for trustworthy convai. this research line aims to enhance userawareness capabilities through the integration of techniques and algorithms from three perspectives: 1) integrating multimodal emotion recognition; 2) developing advanced nlp techniques to extract diverse emotional cues from user utterances; 3) focusing on memorability, enabling systems to identify and retain pivotal elements in dialogues. second, modeling trustworthiness in convai. our aim is to develop novel computational models that can predict and evaluate the trustworthiness of convai systems based on performance, transparency and emotional intelligence. third, transitioning from intent-based dialogue systems to deep learning convai. the most widely adopted technology for dialogue systems in industry is intent-based. we aim to transition from such technology to llm-based convai to achieve a more flexible interaction. fourth, generating conversational models compliant with business rules and domain restrictions. we will investigate how to generate the minimal model capable of adapting to well-defined business rules and domain restrictions, exploring ways of defining such rules, make them queryable through rule engines and coupling the engine with llms. in relation to this line, we plan to develop innovative approaches for generating new resources through prompt-based or instruction-based learning using llms, while simultaneously creating automated evaluation procedures for these resources. finally, trustboost advocates for the innovative exploration of strategies commonly used to detect hallucinations in conversational models, with the aim of evaluating whether individuals may encounter similar challenges. we will address this objective not only by using high-quality dialogue datasets avoiding biases, but also by assessing different techniques for hallucination mitigation: prompt engineering, self-refinement through feedback and reasoning, prompt tuning to adjust the instructions provided to a pre-trained llm, decoding strategies, knowledge graphs, faithfulness based loss functions, or supervised fine-tuning. ## scientific and technical impact trustboost foreseen advances affect the architecture of convai systems by proposing the integration of llms and additional components to improve the interaction context, compliance with associated business rules, multimodal interaction, and user adaptation. the project will provide scientific impact related but not limited to: transitioning from intent-based dialogue systems to llm-based conversational ai provided added flexibility while maintaining compliance; generating quality openaccess dialogue resources for the training of these models in spanish and english for multiple domains; new tools and platforms to develop and evaluate convai systems; developing new methods for the integration of smaller language models that meet the requirements associated to data protection, provide accuracy results comparable to larger models, and allow reducing hardware and energy requirements for their deployment; generating new methods for mitigating hallucinations and explainability, fostering trust; reducing the number of responses factually incorrect or contextually inappropriate that do not fulfill the regulations or business rules of a specific application domain; new methods for understanding and responding to user emotional cues; new techniques for integrating the emotional cues into llm-based conversational ai; and novel and more trustworthy convai models that are user-aware, and emotionally interactive. by enhancing user experiences, building trust, and promoting explainable and adaptable interactions with convai systems, trustboost can contribute to a more positive and supportive technological environment, ultimately benefiting the wellbeing of the population.