LawFlow : Collecting and Simulating Lawyers' Thought Processes
Authors: name: Debarati Das; name: Khanh Chi Le; name: Ritik Sachin Parkar; name: Karin De Langis; name: Brendan Madson; name: Chad M. Berryman; name: Robin M. Willis; name: Daniel H. Moses; name: Brett McDonnell; name: Daniel Schwarcz; name: Dongyeop Kang
Published: 2025-04-26 15:01:55

Abstract: Legal practitioners, particularly those early in their careers, face complex,
high-stakes tasks that require adaptive, context-sensitive reasoning. While AI
holds promise in supporting legal work, current datasets and models are
narrowly focused on isolated subtasks and fail to capture the end-to-end
decision-making required in real-world practice. To address this gap, we
introduce LawFlow, a dataset of complete end-to-end legal workflows collected
from trained law students, grounded in real-world business entity formation
scenarios. Unlike prior datasets focused on input-output pairs or linear chains
of thought, LawFlow captures dynamic, modular, and iterative reasoning
processes that reflect the ambiguity, revision, and client-adaptive strategies
of legal practice. Using LawFlow, we compare human and LLM-generated workflows,
revealing systematic differences in structure, reasoning flexibility, and plan
execution. Human workflows tend to be modular and adaptive, while LLM workflows
are more sequential, exhaustive, and less sensitive to downstream implications.
Our findings also suggest that legal professionals prefer AI to carry out
supportive roles, such as brainstorming, identifying blind spots, and surfacing
alternatives, rather than executing complex workflows end-to-end. Building on
these findings, we propose a set of design suggestions, rooted in empirical
observations, that align AI assistance with human goals of clarity,
completeness, creativity, and efficiency, through hybrid planning, adaptive
execution, and decision-point support. Our results highlight both the current
limitations of LLMs in supporting complex legal workflows and opportunities for
developing more collaborative, reasoning-aware legal AI systems. All data and
code are available on our project page
(https://minnesotanlp.github.io/LawFlow-website/).