# Balancing Project Schedule, Cost, and Value under Uncertainty: A Reinforcement Learning Approach
## 1 Introduction  
Uncertainty is a common challenge in project management and scheduling, which causes many projects to go over budget and miss their deadlines [1]. According to a report that analyzed over 50,000 projects from 1000 organizations, more than half of the projects (5%) had cost overruns, and 60% of them were delayed [2]. According to a 2021 publication by the Project Management Institute, projects worldwide exceed their budget and schedule by 38% and 45%, respectively [3]. The same report also reveals that the products or services delivered by the projects failed to meet the expectations of 44% of customers, implying that the projects did not provide the value that customers anticipated. This situation has prompted researchers in recent years to develop new frameworks for project management that can better deliver value and handle uncertainty. This paper addresses three new problems in project management and scheduling. The problems account for uncertainty by using stochastic activity durations and are formulated as chance-constrained mixed integer programs (MIPs). The problems also use a multimode setting, where each activity has one or more modes or options from which to choose. Therefore, the solution to the problem involves selecting one mode for each activity. The first challenge that this study tackles is a new application of Lean Project Management (LPM), which is a widely used framework to deal with the issue of schedule and cost overruns. The aim of LPM is to maximize value (also referred to as benefit in the literature) and minimize waste in the shortest possible time. The value of a project is determined by a set of attributes that vary according to stakeholders' preferences. These attributes may include aspects such as design aesthetics, features, functions, reliability, size, speed, availability, and so on [4]. We follow the approach proposed in [5], which defines an objective function that captures the value from the perspective of the customers and stakeholders. In Section 6, we offer an example of how to compute the value of a project. We present a new LPM approach that can handle uncertain activity durations. Unlike previous studies on project scheduling that did not consider project value, we aim to maximize the value of the project while avoiding cost and schedule overruns. We use reinforcement learning (RL) algorithms to generate a stable project plan that meets the desired threshold for schedule and budget violation probabilities. We present an MIP model that adopts a multimode approach. Each mode has data on the project scope, such as fixed and resource costs and stochastic duration parameters, and on the product scope, i.e., value parameters. The mode selection affects not only the project cost, duration, and probabilities of meeting the schedule and budget (through the stochastic duration parameters), but also the project value. Hence, the optimal activity modes both stabilize the project plan and maximize the project value. In Section 6, we illustrate this with a small project example where each activity mode has value and duration parameters. We explain how to compute the project value using these parameters and how to plot an efficient frontier to balance value, on-time, and on-budget probabilities. We use a novel approach to solve the problem by applying a heuristic based on RL, which we explain in Section 5. RL-based heuristics are known for finding fast solutions in various applications with uncertain environments. The type of heuristic we propose is uncommon in the project scheduling domain in general and has never been used for our type of problem. We use this approach because solving chance-constrained MIPs is often impractical and time-consuming during the project planning stage. For example, when planning a new product development project, multiple project tradespace alternatives are usually created and solved, and the solution time of these alternatives is crucial.