## 3 Proposed Methods
### Priority Rules  
In each state \(S_{v}\) of ADP, we use MC and simulate \(|\Theta|\) schedules. These schedules are usually forgotten when we try to approximate policy functions for the subsequent states. Such a technique can be called random prioritization. In contrast, we can also learn from the schedules generated by MC simulations to promote exploitation in the subsequent MC simulations. In this study, we explore two techniques based on metaheuristics: EDA and ACO. EDA [29] involves the estimation of the distribution model based on selected elite ancestral generations. The distribution model can be represented in a matrix form. An element \(a_{ij}\in P_{\gamma\times J}\) denotes the probability of activity \(i\) being scheduled at position \(j\). We initialized this matrix as \(P_{l\times J}(0)\) such that \(a_{ij}(0)=\frac{1}{J},\forall i,j\in[1,J]\) represents a uniform probability for the execution of each activity at each position. Given a schedule \(s\) in state \(S_{v}\), we update the elements of \(P\) as:  
\[a_{ij}:=(1-\gamma)a_{ij}+\gamma[I_{i}-C_{i}][s[i]=j] \tag{9}\]where \(\gamma\in(0,1]\), \(I_{i}\) and \(C_{i}\) are defined as in Equation (3). \([\mathbb{S}[i]=j]\) is in Iverson bracket notation, which returns 1 whenever the job at position \(i\) in schedule \(s\) equals \(j\) and 0 otherwise. Once the elements are updated, a column normalization is performed. Given \(|\Theta|\) schedules, only the top \(|\,0.25\times|\Theta|\,|\) solutions are selected to update \(P\) because our distribution must represent the best values of the NPVs. In this study, we set \(\gamma=0.3\) to give more importance to the historical information. To use \(P\) in ADP, we substitute the \"priority\" by \(P\) as an input to the \"PSGS algorithm with ADP\", as shown in Figure 1. The algorithm then schedules an activity \(i\) at position \(j\), which has the highest value of \(a_{ij}\) in \(P\) in an Epsilon-Greedy Algorithm [67; 68] fashion with \((1-\epsilon=0.90)\) when it has multiple options. Alternatively, we can use the information generated by ACO [42] as the priority rule. ACO depicts the foraging phenomenon of ants. In this study, the pheromone information of the ants is stored in a matrix \(\phi(S_{v}):=\phi(S_{v})_{I\times J}\). The elements of \(\phi(S_{0})\) are set to:  
\[\tau_{ij}=a_{ij}(s_{0}):=|S_{i}|^{-1}\big{[}i\in P_{j}\big{]} \tag{10}\]  
where \(\big{[}i\in P_{j}\big{]}\) is in the Iverson bracket notation checking the existence of job \(i\) in the precedence of \(j\). \(S_{i}\) is the set of successors of an activity \(i\). Given a schedule \(s\), the pheromone is updated as:  
\[\tau_{ij}:=(1-\mu)\tau_{ij}+\mu\frac{Z_{s_{i}}}{(\vartheta\times Z_{b}-Z_{s})} [\text{s}\rightarrow(i,j)] \tag{11}\]  
where \(\mu\in[0,1],Z_{s{i}}\) is the cumulative objective value of schedule \(s\) up to activity \(i\) calculated using Equation (3) are the final objective values of the historical best schedule and schedule \(s\), respectively. In this study, we set \(\vartheta=2\), and \(s\rightarrow(i,j)\) denotes the job \(i\) schedule immediately before activity \(j\) in schedule \(s\). To use this priority rule in our ADP, we substitute \(\phi(S_{v})\) for \"priority\" as an input to the \"PSGS algorithm with the ADP\" shown in Figure 1. In the MC simulations, an activity with the minimum value of \(u_{ij}\) given by Equation (12) is selected from the eligible set with a certain probability \((1-\epsilon=0.90)\)[67; 68]. \[u_{ij}:=\frac{\tau_{ij}^{\omega}h_{ij}^{1-\omega}}{\sum_{l\in S_{i}}\tau_{il} ^{\omega}h_{ij}^{1-\omega}}\big{[}i\in P_{j}\big{]} \tag{12}\]  
For each simulation in \(\Theta\), we first choose the parameter control \((\omega)\) of the pheromone and then a heuristic. We randomly set \(\omega\) to any value from \(\{0,0.2,0.4,0.6,0.8,1\}\), and the heuristic is randomly selected from Section 3.3. We also set \(\mu\) in Equation (11) to 0.3 to give more weight to the historical information.