# Balancing Project Schedule, Cost, and Value under Uncertainty: A Reinforcement Learning Approach
## 4 Quantitative Models
### Lpm  
Our LPM deterministic model is formulated as a MIP, aiming to maximize the project value while considering duration and cost constraints. We now provide an overview of our notation, model, and explanations of its objectives and constraints. We consider a project consisting of \(J\) activities, where each activity \(j\) can be executed in one of \(M_{j}\) modes and is preceded by a set of immediate predecessors \(\mathcal{P}(j)\). Executing activity \(j\) in mode \(m\) requires a duration \(d_{jm}\) and incurs a fixed cost \(c_{jm}\). Additionally, there are \(K\) renewable resources available, each with a unit cost \(c_{k}\) per period. Resource \(k\) is consumed by activity \(j\) in mode \(m\) at a rate of \(r_{jmk}\) units. The project is constrained by a due date \(D\) and a budget \(C\). We assume that if the project adheres to the budget constraint, the required resources can be readily acquired. Similar assumptions regarding resource availability can be found in studies on time-cost tradeoff problems and recent project scheduling research [73; 74; 75; 76; 77]. The project encompasses \(V\) different value attributes, denoted by the parameter \(V_{jm}\), which represents the value of attribute \(v\) for activity \(j\) executed in mode \(m\). Decision variable \(V^{\prime}_{jv}\) corresponds to the value of attribute \(v\) for activity \(j\) when executed in its selected mode. To determine the project value for each attribute \(v\), we define the function  
Figure 1: Method flowchart. \(F_{v}\left(V^{\prime}_{1v},\ldots,V^{\prime}_{lv}\right)\) which takes into account the individual attribute values \(V^{\prime}_{jv}\), and the function \(V^{\prime\prime}\left(F_{1},\ldots,F_{V}\right)\), which calculates the project value based on the attribute values. Within our model, the binary decision variable \(\delta_{jm}\) indicates whether activity \(j\) is performed in mode \(m\). Additionally, decision variable \(t_{j}\) denotes the starting time of activity \(j\), where \(j\) ranges from 0 to \(J+1\). In this context, \(j=0\) denotes a project milestone that has only one mode and does not have any duration, cost, resources, or value. It functions as the starting point of the project. Conversely, \(j=J+1\) represents another milestone signifying the project's end. The model itself is:  
\[\text{Maximize }V^{\prime\prime}\left(F_{1}\left(V^{\prime}{}_{11},\ldots,V^{ \prime}_{J1}\right),\ldots,F_{V}\left(V^{\prime}_{1v},\ldots,V^{\prime}_{lv} \right)\right), \tag{1}\]  
subject to:  
\[V^{\prime}_{jv}=\sum_{m=1}^{M_{j}}\delta_{jm}V_{jmv},\;\forall j=1,\ldots,J,\; \forall v=1,\ldots,V, \tag{2}\]  
\[\sum_{m=1}^{M_{j}}\delta_{jm}=1,\;\forall j=1,\ldots,J, \tag{3}\]  
\[t_{0}=0, \tag{4}\]  
\[t_{J+1}\leq D, \tag{5}\]  
\[t_{j}\geq t_{i}+\sum_{m=1}^{M_{j}}d_{im}\delta_{im},\;\forall i\in\mathcal{P}( j),\;\forall j=1,\ldots,J+1, \tag{6}\]  
\[\sum_{j=1}^{J}\sum_{m=1}^{M_{j}}\left(c_{jm}+\sum_{k=1}^{K}c_{k}r_{jmk}d_{jm} \right)\;\delta_{jm}\leq C, \tag{7}\]  
\[\delta_{jm}\in\{0,1\},\;\forall j=1,\ldots,J,\;\forall m=1,\ldots,M_{j}, \tag{8}\]  
\[t_{j}\geq 0,\;\forall j=0,\ldots,J+1. \tag{9}\]  
Objective (1) aims to maximize the project value, which is a specific function of the chosen modes. Constraints (2) are responsible for determining the value attributes based on the selected modes. Constraints (3) introduce the binary decision variable that indicates the chosen mode for each activity. These constraints ensure that exactly one mode is selected for each activity. Constraint (4) sets the beginning of the project as the starting time for milestone 0. Constraint (5) ensures that the project is completed within the specified due date. Constraints (6) ensure that an activity cannot start before its immediate predecessor is finished. Constraint (7) restricts the fixed and resource costs to be within the project budget. Lastly, constraints (8) and (9) are integrality and nonnegativity constraints. In the case of a model with stochastic activity durations, constraints (5) and (7) cannot be guaranteed with certainty.