Converso: Improving LLM Chatbot Interfaces and Task Execution via Conversational Form (2024)

Abstract: Recent advancements in large language models (LLMs) have enabled more autonomous conversational AI agents. However, challenges remain in developing effective chatbots, particularly in addressing LLMs’ lack of “statefulness”. This paper presents Converso, a novel chatbot framework that introduces a new conversation flow based on stateful conversational forms designed for natural data acquisition through dialogue. Converso leverages LLMs, LangChain, and a containerized architecture to provide an end-to-end chatbot system with Telegram as the user interface. The key innovation in Converso is its implementation of conversational forms, which guide users through form completion via a structured dialogue flow. Converso’s chatbots can be linked with multiple forms that are automatically triggered based on the user’s intent. Our forms are fully integrated into the LangChain ecosystem, allowing the LLM to use tools for form completion and dynamic validation. Evaluations show that this approach significantly improves task completion rates compared to LLMs alone. Converso demonstrates how specifically designed conversational flows can enhance the capabilities of LLM-based chatbots for practical data collection applications. Our implementation is available at: https://github.com/gianfrancodemarco/converso-chatbot.

## introduction chatbots have emerged as one of the most widely adopted applications of artificial intelligence (ai) in consumer products and services. these conversational agents directly interact with end users through natural language interfaces, serving various domains such as entertainment, education, information retrieval, e-commerce, and more @xcite . since the early conceptualization of chatbots in the 1960s, numerous approaches have been explored to enhance their capabilities, transitioning from basic pattern-matching techniques to leveraging advanced machine learning algorithms and language models. recent advancements in large language models (llms) have led to a significant shift, advancing chatbots to new levels of independence and conversational ability @xcite . llms are complex neural network architectures with billions of parameters, trained on extensive text corpora. the extensive data on which these models are trained, along with additional techniques like reinforcement learning from human feedback (rlhf) @xcite , enables them to generate natural language responses that closely resemble human communication. however, llms still face limitations, including a lack of access to up-todate knowledge, an inability to perform complex reasoning, and difficulties interacting with external environments @xcite . researchers have developed techniques such as retrieval-augmented generation (rag) @xcite and model-calling capabilities @xcite to address these challenges, allowing llms to leverage external data sources and tools during inference. furthermore, open-source frameworks like langchain @xcite have emerged to streamline the development of llm-based applications, including chatbots. langchain provides a comprehensive suite of libraries, tools, and templates that facilitate the integration of llms, rag techniques, and external tools, enabling the creation of context-aware and reasoning-capable chatbot systems. this paper explores the design and implementation of modern chatbot systems leveraging llms, the langchain ecosystem, and related techniques. it introduces converso, a novel chatbot framework that incorporates a conversation flow based on conversational forms, enhancing traditional web forms for data acquisition through natural language interactions. the paper discusses converso's system architecture, conversational flow, and use cases, highlighting the benefits of integrating llms and the langchain ecosystem. additionally, an evaluation protocol is presented to assess the effectiveness of conversational forms in improving task completion rates. the rest of this paper is structured as follows: section 2 discusses related work, section 3 presents the converso framework, section 4 describes our experimental use cases and presents the results obtained, and section 5 concludes the paper. ## related work chatbots have been an active area of research for a long time, with early systems like eliza @xcite and parry @xcite employing patternmatching techniques with pre-defined rules and responses. subsequent work incorporated machine learning algorithms for intent classification and entity extraction @xcite , leading to more advanced chatbots capable of understanding user intents and relevant context. with recent developments in llms like gpt-3 @xcite and palm @xcite , there has been renewed interest in leveraging these powerful models for building conversational ai systems. llms have demonstrated impressive emergent abilities in few-shot prompting settings @xcite and can engage in substantive multi-turn dialogues by conditioning on previous conversation history @xcite . however, llms still face limitations such as hallucinating incorrect facts @xcite , being confined to their training data distributions, and lacking mechanisms to interact with external tools or information sources. several techniques have been proposed to overcome these limitations. for instance, retrieval-augmented generation (rag) @xcite integrates external data retrieval into the language model's generation process to enhance factual accuracy and access to up-to-date information. tool calling @xcite allows llms to call and receive results from external tools/apis, enabling capabilities beyond pure text generation, like mathematical reasoning @xcite and real-world interactions @xcite . such advancements, combined with the ability to influence the behavior of llms through prompt engineering @xcite , have opened a wide range of applications for chatbot systems, such as data acquisition. for example, hakimov et al. @xcite proposed a modular system based on llms for form filling, creating a method to evaluate dialogues using user simulation with an llm. additionally, some simple (usually closed-source) implementations of forms for data acquisition with llms have emerged online. in contrast to these approaches, we propose a framework for creating chatbots that is fully integrated with the langchain ecosystem. our innovative approach does not tie the llm to a specific form, allowing chatbot developers to specify an arbitrary number of forms that can be invoked based on user intents. this flexibility enables users to interact with the chatbot for various goals beyond executing actions requiring form completion. furthermore, with converso, we propose a fully containerized architecture streamlining the chatbot creation process. ## our framework: converso in this section, we present converso, a framework developed as an extension of langchain that enables building conversational forms for goal-oriented interactions. converso introduces stateful conversational forms that guide users through structured data collection processes, reducing reliance on lengthy conversation histories and mitigating hallucinations or deviations from the intended goals. converso facilitates the creation of fully containerized chatbot applications, leveraging kubernetes for container orchestration, an event manager for asynchronous communication, and a telegram bot interface as the front end. this enables a production-ready, scalable implementation of conversational chatbots that can be seamlessly integrated into existing systems. ## conclusion our work demonstrates the significant advancements in chatbot systems by integrating large language models and modern frameworks like langchain. the proposed converso framework, which incorporates conversational forms, showcases the potential to enhance user interactions by transforming traditional data acquisition methods into dynamic, interactive conversations. the evaluation results indicate a marked improvement in task success rates when using conversational forms, particularly with complex tools requiring detailed input. by addressing the limitations of basic conversation flows-such as the hallucination of data and lack of confirmation-converso improves accuracy and ensures a safer and more reliable user experience. this is especially crucial in real-world applications where incorrect data could lead to unintended and potentially harmful actions. furthermore, the converso framework's robustness, demonstrated by its ability to handle diverse and complex use cases in the evaluation experiments, underscores its versatility. the consistent performance improvements, nearly doubling success rates in some cases, highlight the effectiveness of integrating stateful interactions and form-based data gathering. our work contributes to conversational ai by providing a practical and scalable solution for developing sophisticated chatbot applications. the insights gained from this research pave the way for future innovations in chatbot design, aiming to bridge the gap between human-like interactions and automated systems.