Low-code LLM : Graphical User Interface over Large Language Models (2024)

Abstract: Utilizing Large Language Models (LLMs) for complex tasks is challenging, often involving a time-consuming and uncontrollable prompt engineering process. This paper introduces a novel human-LLM interaction framework, Low-code LLM. It incorporates six types of simple low-code visual programming interactions to achieve more controllable and stable responses. Through visual interaction with a graphical user interface, users can incorporate their ideas into the process without writing trivial prompts. The proposed Low-code LLM framework consists of a Planning LLM that designs a structured planning workflow for complex tasks, which can be correspondingly edited and confirmed by users through low-code visual programming operations, and an Executing LLM that generates responses following the user-confirmed workflow. We highlight three advantages of the low-code LLM: user-friendly interaction, controllable generation, and wide applicability. We demonstrate its benefits using four typical applications. By introducing this framework, we aim to bridge the gap between humans and LLMs, enabling more effective and efficient utilization of LLMs for complex tasks. The code, prompts, and experimental details are available at https://github.com/moymix/TaskMatrix/tree/main/LowCodeLLM. A system demonstration video can be found at https://www.youtube.com/watch?v=jb2C1vaeO3E.

## introduction large language models (llms), such as chat-gpt @xcite and @xcite , have garnered significant interest from both academia and industry, as they demonstrate impressive capability across a range of tasks @xcite , and are increasingly utilized in a variety of other fields as well @xcite et al., 2023; baidoo-anu and owusu ansah, 2023). however, it is not yet perfect in handling complex tasks. for example, when generating a long paper, the presented arguments, supporting evidence, and overall structure may not always meet expectations in diverse user scenarios. or, when serving as a task completion virtual assistant, chatgpt may not always interact with users in the intended manner and may even display inappropriate behavior in various business environments. effective utilization of llms like chatgpt requires careful prompt engineering @xcite . however, prompt engineering can be particularly challenging when instructing llms to perform complex tasks, as reflected in more uncontrollable responses and more time-consuming prompt refining @xcite . there exists a gap between providing prompts and receiving responses, and the process of generating responses is not accessible to humans. to reduce this gap, this paper proposes a new human-llm interaction pattern low-code llm, which refers to the concept of low-code visual programming @xcite , like visual ba-sic @xcite or scratch @xcite . users can confirm the complex execution processes through six predefined simple operations on an automatically generated workflow, such as adding or deleting, graphical dragging, and text editing. as shown in figure 1 , human-llm interaction can be completed through the following steps: (1) a planning llm generates a highly structured workflow for complex tasks. (2) users edit the workflow using predefined low-code operations, which are all supported by clicking, dragging, or text editing. (3) an executing llm generates responses based on the reviewed workflow. (4) users continue to refine the workflow until satisfactory results are obtained. compared with the conventional human-llm interaction pattern, low-code llm has the following advantages: ## related work large language models large language models (llms) have emerged as a prominent area of research in recent years. recent llms, such as gpt-4 and chatgpt, have made impressive strides in generating more coherent and contextually relevant responses. they have been applied in various industries and fields, including content creation, code development @xcite , customer support(george and george, 2023), and more. however, while llms have demonstrated promising potential, they still face limitations @xcite @xcite . in particular, controlling the behavior and output of llms for complex tasks remains a challenge, which has led to the development of new techniques, such as prompt engineering, and methods to improve results @xcite @xcite @xcite . prompt engineering prompt engineering has emerged as an essential technique for interacting with llms to achieve desired outcomes. the success of large language models relies heavily on their ability to produce answers to various queries @xcite . however, providing effective prompts that convey the exact intent of humans is a non-trivial task, especially when it comes to complex tasks and requirements. the challenge in prompt engineering lies in crafting prompts that can manipulate the llm into generating specific outcomes. researchers have explored various techniques to simplify prompt engineering, ranging from giving explicit instructions to providing context for llms to understand the desired output better @xcite . some recent advancements in prompt engineering include techniques such as few-shot learning @xcite @xcite , reinforcement-learning @xcite . however, these techniques often demand substantial expertise and time, making it difficult for end-users to leverage the full potential of these llms. the low-code llm framework proposed in our paper provides an innovative solution by involving the users in the process of designing workflows, which ultimately controls the llm's response generation. ## limitations while the low-code llm framework promises a more controllable and user-friendly interaction with llms, there are some limitations. one such limitation is the increase in the cognitive load for users, who now need to understand and modify the generated workflows. furthermore, accurate and effective structured planning within the planning llm may be challenging, and bad structured planning poses a heavy user editing burden. but we believe with the evolution of llms and research on task automation, the planning ability will be getting satisfactory. lastly, the current design assumes that users have sufficient domain knowledge and skills to modify the generated workflows effectively. ## conclusion we proposed a novel human-llm interaction framework, which aims to improve the control and efficiency of utilizing large language models for complex tasks. low-code llm allows users to better understand and modify the logic and workflow underlying the llms' execution of instructions. compared with traditional prompt engineering, the proposed low-code llm framework advances the state-of-the-art in human-llm interactions by bridging the gap of communication and collaboration between humans and llms. we believe the low-code llm framework presents a promising solution to many of the challenges faced by llm users today and has the potential to greatly impact a wide range of industries and applications.