FlowMind: Automatic Workflow Generation with LLMs
Authors: name: Zhen Zeng; name: William Watson; name: Nicole Cho; name: Saba Rahimi; name: Shayleen Reynolds; name: Tucker Balch; name: Manuela Veloso
Published: 2024-03-17 00:36:37

Abstract: The rapidly evolving field of Robotic Process Automation (RPA) has made
significant strides in automating repetitive processes, yet its effectiveness
diminishes in scenarios requiring spontaneous or unpredictable tasks demanded
by users. This paper introduces a novel approach, FlowMind, leveraging the
capabilities of Large Language Models (LLMs) such as Generative Pretrained
Transformer (GPT), to address this limitation and create an automatic workflow
generation system. In FlowMind, we propose a generic prompt recipe for a
lecture that helps ground LLM reasoning with reliable Application Programming
Interfaces (APIs). With this, FlowMind not only mitigates the common issue of
hallucinations in LLMs, but also eliminates direct interaction between LLMs and
proprietary data or code, thus ensuring the integrity and confidentiality of
information - a cornerstone in financial services. FlowMind further simplifies
user interaction by presenting high-level descriptions of auto-generated
workflows, enabling users to inspect and provide feedback effectively. We also
introduce NCEN-QA, a new dataset in finance for benchmarking question-answering
tasks from N-CEN reports on funds. We used NCEN-QA to evaluate the performance
of workflows generated by FlowMind against baseline and ablation variants of
FlowMind. We demonstrate the success of FlowMind, the importance of each
component in the proposed lecture recipe, and the effectiveness of user
interaction and feedback in FlowMind.