## 4 Policy Optimization
### Crane Dispatching Policy
\(C_{2}\) gives precedence to more urgent jobs. While the method of measuring the urgency depends on the job type, the general idea is to measure the time remaining to the due time of the job. The smaller this margin, the more urgent the job is. Note that the margins for delayed jobs are negative. For a seaside job, the due time is its deadline set up by the corresponding discharging or loading plan. For a preparatory (rehandling or a repositioning) job of a seaside job, the due time is the time its corresponding main job should be started to meet its deadline. The margins of the landslide jobs are all negative, as they are for the ETs that have already arrived at the landslide HP waiting to be serviced. The due times for those jobs are the arrival times of the corresponding ETs. The due times for the preparatory jobs for the landslide jobs are calculated similarly to those of the seaside preparatory jobs. Since the remarshaling jobs are not given any due time, they are ranked by the preferences based on the estimated time taken to have each of them finished. A remarshaling job is preferred to another if the former is estimated to take less time (to have the required rearrangement work finished) than the latter. The reason for this preference is that a remarshaling job that can be finished in a shorter time is less likely to delay the main jobs. After ranking the remarshaling jobs according to this preference, the rank value of each job is deemed as its margin. One reason for assigning different weights to different job types for criterion \(C_{2}\) is that the margins of different job types are of a different nature. Another reason is that the seaside jobs are the most important and the remarshaling jobs the least important for the productivity of the terminal in the short term. If, for instance, the seaside jobs and the landslide jobs are carelessly assigned the same weight, the landslide jobs tend to have better (lower) scores, as their margins are always negative. \(C_{3}\) prefers the jobs that are less likely to involve the ASCs in an interference. Interference can occur during empty travel, as well as loaded travel. Among the probabilities of these two interferences, the higher one is taken as the evaluation value of this criterion. The probability of interference can be estimated based on the ASC's target bay (for pickup or drop-off) in the block by using the empirical probability distribution suggested in [27], which is obtained through simulation. \(C_{4}\) estimates the total time taken to process the candidate job (including the expected delay by interference), calculated as the product of the probability of interference and the average waiting time due to interference (assumed \begin{table}
\begin{tabular}{c c c} \hline
**Description** & **Criterion** & **Relevant Job Type** \\ \hline Empty travel distance & \(C_{1}\) & All types \\  & & Seaside job \\  & \(C_{2,a}\) & Seaside rehandling job \\  & & Seaside repositioning job \\ Margin to due time & & Landside job \\  & \(C_{2,b}\) & Landside rehandling job \\  & & Landside repositioning job \\  & \(C_{2,c}\) & Remarshaling job \\ Probability of ASC interference & \(C_{3}\) & All types \\ Estimated processing time & \(C_{4}\) & All types \\  & \(C_{5,d}\) & Seaside job \\  & \(C_{5,e}\) & Seaside rehandling job \\  & \(C_{5,f}\) & Seaside repositioning job \\ Gain by doing the current job & \(C_{5,g}\) & Landside job \\  & \(C_{5,h}\) & Landside rehandling job \\  & \(C_{5,j}\) & Landside repositioning job \\  & \(C_{5,j}\) & Remarshaling job \\ Advantage if done by the other ASC & \(C_{6}\) & All types \\ Waiting time by ASC interference & \(C_{7}\) & All types \\ Job priority & \(C_{8}\) & All types \\ Time available for remarshaling & \(C_{9}\) & All types \\ \hline \end{tabular}
\end{table} 
Table 1: Evaluation criteria for different job types for the crane dispatching policy. to be 100 s in this paper). The job with the shortest processing time is given the highest priority by this criterion.