## 3 Results
### Localization Performance  
In addition to the higher training efficiency, the proposed method also significantly outperformed the standard RL-based localization approach (\(p\)-value \(<0.03\)). The individual agents in the usual RL method [11] could give an average localization error of \(2.64\pm 1.38\) mm, yielding improvement over the end-to-end deep learning methods [7; 13] (see Table 2). The proposed mirror environment based RL could further lower the error to \(2.33\pm 1.17\) mm with its single agent utilizing the inter-fold similarity. Table 2 also presents the corresponding localization accuracy for each method. The proposed agent showed similar performance for localizing the RVF and LVF, showing an error of approximately \(2.36\pm 1.19\) and \(2.30\pm 1.16\) mm, respectively (\(p\)-value \(\approx 0.693\)). The corresponding localization accuracies were 96.67% and 95.00%. \begin{table}
\begin{tabular}{l c c} \hline \hline
**Methods** & **Required Episodes** & **Final Reward** \\ \hline Standard RL (RVF) & \(110\)k \(\pm\) 5k & \(126.495\pm 14.257\) \\ Standard RL (LVF) & \(96\)k \(\pm\) 4k & \(137.152\pm 17.309\) \\ Standard RL (RVF and LVF) & \(206\)k \(\pm\) 8k & \(131.823\pm 15.783\) \\ Mirror Environment & \(122\)k \(\pm\) 5k & \(147.296\pm 13.679\) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Number of required episodes and final reward. Figure 4: Reward plot over the training episodes. Proposed agent can learn to localize both folds in a similar time the usual agent learns a single fold. In Figure 5, we present two examples of improvements by the proposed method, where the localization trajectories of the standard RL agents and the mirror environment agent are plotted. Being ignorant of the inter-fold symmetry and similarity, the standard RL agents gave localization failure (Figure 5, left) or higher error (Figure 5, right) for one fold despite correctly localizing the other. In contrast, the proposed agent could improve on both cases by enforcing a similar and more general policy for both folds, resulting from the implicit sharing of RVF and LVF features during the simultaneous training. To further illustrate the improved generalization and feature-similarity utilization of the proposed method, we compared the learned feature representation of different agents (Figure 6). We collected approximately 6000 states from multiple episodes employing the standard RL agents to localize the right and left folds on different volumes. For these states, we then extracted the corresponding features (outputs of the final convolution layer) of the policy networks of the standard and proposed agents. Manifold representation [14] is used to effectively plot these high dimensional features into 2D space. State similarity between right and left folds can clearly be observed in Figure 6 (left), where the raw voxel array of the state is represented. However, the standard RL agents learned different representations for the two folds despite their general similarity. On the \begin{table}
\begin{tabular}{l c c} \hline \hline
**Methods** & **Loc. Error (mm)** & **Loc. Accuracy** \\ \hline Deep Heatmap Regression [7] & \(2.92\pm 1.88\) & 82.50\% \\ Deep Coordinate Regression [13] & \(3.70\pm 2.02\) & 77.08\% \\ Standard RL [11] & \(2.64\pm 1.38\) & 91.25\% \\ Mirror Environment & \(2.33\pm 1.17\) & 95.83\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Localization error and accuracy. Figure 5: Agent-trajectories for localizing the vocal folds in two CT volumes (**(left)**: axial view, **(right)**: coronal view). Localization failure (**left**) or higher localization error (**right**) for one fold despite correct localization of the other fold by the standard RL. The mirror environment showed improvement by employing similar policies for both. Red and green plus (+) signs indicate the expert-annotated RVF and LVF locations, respectively. The cross (\(\times\)) signs indicate the agent-positions at the end of different trajectories. Figure 6: Manifold representation of the learned features by different agents.