## 4 Result and Discussion
### Three-Round Delphi Study Result  
In the first round of the Delphi survey, the respondents were asked to give their perceptions of the importance of the metrics in Table 4 on a 5-point Likert scale. In the second and third rounds, respondents were provided with the summary results from the previous rounds and were asked to reconsider their opinion and adjust their original options if they wished. The first-round Delphi questionnaire was sent to 14 panel members by email. After two weeks we collected and analyzed the results of the first round, and the results are described in Table 6.  
To analyze the data, we employed the content validity ratio method (CVR-Lawshe's method) [54]. The CVR is a validity test to evaluate the level of agreement among several experts by considering the essential level based on an expert agreement where the minimum CVR value is 50% [55]. The CVR value was calculated using Excel. The results of the CVR method analysis included three metrics where the CVR value was below 50%, namely, hauling distance with a CVR value of 43%, truck capacity with a CVR value of 43%, and mean time to repair (MTTR) with a CVR of 36%. These three metrics were excluded in the next Delphi round. The reliability test was conducted to evaluate the consistency and reliability of the selected potential metrics using [PERSON]'s alpha test. An internal consistency test was used to determine whether these metrics could be used as a measurement tool based on a minimum value of 0.7 [51]. [PERSON]'s alpha (\(\alpha\)) was calculated using IBM SPSS version 26 software, and the \(\alpha\) value result was 0.721. With a [PERSON]'s alpha of 0.721, the potential metrics could be used for the next questionnaire. There were six potential metrics from the first Delphi round: physical availability, use of availability, productivity, fuel efficiency, coal losses, and quality of service. Second- and third-round questionnaires were prepared based on input from the first round, including a newly proposed metric; these enabled respondents to give their opinion about the newly proposed metric. This new metric was constructed based on the existing OEE approach. The basis for this grouping was the well-known, existing OEE metric; a researcher proposed the new metric MTOVE. MTOVE was constructed based on combining the elements of physical availability, use of availability, performance, and quality (see Equation (1)). The grouping of the proposed MTOVE metrics is described in Figure 3. As in the first round, the second- and third-round questionnaires were sent to the group of panel members by email. In the second and third rounds of Delphi, respondents were \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline  & **Physical** & **Use of** & **Productivity** & **Coal** & **Fuel** & **Quality** & **Hauling** & **Truck** & **Capacity** & **MTTR** \\  & **Availability** & **Availability** & **Productivity** & **Losses** & **Performance** & **Rate** & **Distance** & **Capacity** & **MTTR** \\ \hline CVR & 100\% & 100\% & 100\% & 86\% & 86\% & 93\% & 43\% & 43\% & 36\% \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Result of Delphi’s first round. \begin{table}
\begin{tabular}{c c} \hline \hline
**No** & **Potential Metric** \\ \hline
1 & Availability \\
2 & Utilization \\
3 & Productivity \\
4 & Fuel consumption \\
5 & Hauling distance \\
6 & Quality Rate \\
7 & Truck capacity \\
8 & Coal losses \\
9 & Mean time to repair (MTTR) \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Potential metrics from the literature review and in-depth interview result. asked to give their opinion about the importance of the six selected potential metrics in the previous round. Respondents were asked to give scores based on a 5-point Likert scale. In addition, there were questions to determine respondents' opinions on the new overall metric to measure overall transportation effectiveness by combining all metrics (MTOVE) based on a 5-point Likert scale. This purpose was to validate the proposed MTOVE.