R ecipe2 P lan: Evaluating Planning Abilities of LLM s for Efficient and Feasible Multitasking with Time Constraints Between Actions (2025)

Abstract: While Large Language Model-based agents have demonstrated substantial progress in task completion, existing evaluation benchmarks tend to overemphasize single-task performance, with insufficient attention given to the crucial aspects of multitask planning and execution efficiency required in real-world scenarios. To bridge this gap, we present Recipe2Plan, a novel benchmark framework based on real-world cooking scenarios. Unlike conventional benchmarks, Recipe2Plan challenges agents to optimize cooking time through parallel task execution while respecting temporal constraints i.e. specific actions need to be performed within a particular timeintervals following the preceding steps.Overly aggressive local parallelization may disrupt this constraint, potentially compromising the entire cooking process.This strict time constraint between actions raises a unique challenge for agents to balance between maximizing concurrent operations and adhering to critical timing constraints. Extensive experiments with state-of-the-art models reveal challenges in maintaining this balance between efficiency and feasibility. The results highlight the need for improved temporal awareness and global multitasking capabilities in large language models. We open-source our benchmark and code at https://github.com/WilliamZR/Recipe2Plan.

## introduction large language models (llms) @xcite @xcite have demonstrated the ability to plan and reason step by step @xcite . leveraging this ability, llm-based agents can automate complex real-world tasks @xcite @xcite . the effectiveness of llm-based agents is primarily evaluated based on the feasibility of their * corresponding author. however, the ability to manage concurrent objectives remains an often overlooked yet crucial requirement in real-world applications, as exemplified by everyday scenarios where humans prepare multiple dishes simultaneously for a meal or conduct parallel laboratory experiments @xcite @xcite . current planning benchmarks assume that models execute tasks by decomposing the overall goal into steps and achieving these subgoals sequentially, one at a time @xcite . consequently, these datasets fail to account for the duration of an action and the potential for multitasking. the multitasking scenario proposes a different objective in addition to feasibility. it challenges the model to optimize the efficiency to reach multiple goals simultaneously. time constraints between actions are often imposed in the recipe of dishes or experiments, indicating specific actions must be performed within a particular time interval after the preceding step is completed. for instance, the pouring and dripping actions for pour-over coffee must be carried out in sequence without any delay, as recommended by professionals @xcite . this property introduces a unique challenge for multitask planning apart from conventional benchmarks. the first plan in figure 1 illustrates that if the agent prioritizes maximizing efficiency by rushing to multitask whenever it is idle, it may inadvertently violate future time constraints. consequently, the agent must balance the need for efficiency with adherence to time constraints to achieve feasible multitask planning as shown in the second plan in figure 1 . we propose a new benchmark recipe2plan based on real-world recipes and constraints to eval-figure 1 : a simplified demonstration of our benchmark. actions will either occupy the agent or leave it idle. the four steps of brewing must be executed sequentially as time constraints between actions. the goal for the agent is to plan multitasking to complete the recipes in the shortest time possible without violating any constraints. the first plan illustrates a scenario where the agent attempts always to keep the agent occupied for higher efficiency, resulting in violations of time constraints. the second plan maintains the balance between the efficiency and feasibility of the plan by leaving the agent idle on purpose to maintain the time constraints for all actions. our benchmark provides a testbed for the efficiency of llm-based agents, as they are approaching the upper limits of feasibility in current textbased agent benchmarks @xcite and multitasking scenarios without time constraints between actions ( in this study, we experiment with various sizes of open-source models, such as qwen2.5 @xcite and llama3.1 @xcite , as well as closed-source models, including gemini-1.5-pro @xcite and @xcite . our experiments reveal that gpt-4o achieves the highest success rate of only 21.5% and the main failure source is the violation of time constraints between actions. it suggests that current llms fail to deliver feasible plans while attempting efficient multitasking. we show that llms can deliver feasible plans if time constraints between actions are absent. however, their efficiency still lags significantly behind a simple heuristic method ( §4). we also indicate that gpt-4o can trade efficiency for success rate if focusing solely on feasibility ( §5.1). overall, we demonstrate that current llms struggle to balance efficiency and feasibility when multitasking with time constraints. we further analyze the commonsense reasoning, local planning, and global planning capabilities of llms. by isolating each ability, we identify global planning as the primary source of task failure and inefficient multitasking ( §5.3). reasoning models excel under time constraints compared with non-reasoning models but struggle with action concurrency ( § 6). our contributions are as follows: ## related work planning benchmarks. to evaluate the planning abilities of llm-based agents, researchers have proposed benchmarks across various domains such as web browsing @xcite @xcite , tool usage @xcite , and computer manipulation @xcite . these benchmarks assess an agent's ability to execute a sequence of actions to achieve a general goal in a partially observable environment @xcite . however, these environments do not account for the duration of each action. additionally, they evaluate planning abilities based solely on feasibility, without comparing the efficiency of task completion between different agents. ## recipe2plan recipe2plan evaluates the planning ability of llms for efficient and feasible multitasking under constraints. specifically, we provide the model with multiple goals that can be achieved by following recipes. each recipe a is represented as a linear sequence of actions @xmath0, a 1 , ..., a n ), with each action assigned a specific execution time t n . the task is to plan the action sequence to complete all goals in the shortest time possible, adhering to the properties and constraints detailed in §3.1 and §3.2. recipe2plan challenges the model to apply commonsense reasoning to infer any unwritten constraints from the recipe, including action concurrency, action dependencies, and resource limitations while planning the action sequence to minimize overall execution time. ## conclusions our paper introduces the recipe2plan benchmark, which evaluates the feasible and efficient multitasking abilities of existing llms. this benchmark pushes the limits of current agent planning capabilities beyond mere task completion to include the optimization of time and resource management. our experiments reveal that while strong models like gpt-4o can generate feasible plans without time constraints, their performance decreases sharply when time constraints are imposed. this highlights a significant gap between current capabilities and the requirements for feasible and efficient multitasking. our analysis identifies global planning as the primary area needing improvement, paving the way for future work to focus on enhancing temporal reasoning and strategic planning.