# Machine Learning Driven Contouring of High-Frequency Four-Dimensional Cardiac Ultrasound Data
## 3 Results and Discussion  
### Model Fitting Results  
A visual summary of the test set prediction results for models 1, 2, and 3 are demonstrated in Figure 2A-C, respectively. Qualitatively, we can see improvements in R\({}^{2}\) and mean squared error (MSE) for each successive model. Of note, we observed higher MSE for all three models around the posterior-septum in the basal slice. We believe this may be because the basal septum commonly lies posterior to the sternum and is thus affected by shadowing artifacts. Not only does that make the basal septum harder to annotate, which lowers precision in the ground-truth data, but also the lack of border contrast means it may be incorrectly accounted for in the PCA-based image compression. Table 1 provides a numerical summary of the three models' performance. We notice that with regard to every metric in Table 1, model 2 outperforms model 1, and model 3 outperforms model 2. Taken together, these results suggest that simultaneously incorporating all four slices in the prediction model (model 2) yields modest but noticeable improvement over using individual slices (model 1). Furthermore, annotation of a single point at t = 0 (model 3) can significantly improve model predictive accuracy above the previous versions not incorporating user annotations. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline  & \multicolumn{3}{c}{**Endocardial**} & \multicolumn{3}{c}{**Epicardial**} \\ \cline{2-9}  & **MSE (mm2)** & **R2** & **vs. M1** & **vs. M2** & **MSE (mm2)** & **R2** & **vs. M1** & **vs. M2** \\ Model 1 & 0.069 \(\pm\) 0.054 & 0.41 & — & — & 0.068 \(\pm\) 0.044 & 0.51 & — & — \\ Model 2 & 0.060 \(\pm\) 0.049 & 0.49 & 48.7 & — & 0.058 \(\pm\) 0.039 & 0.59 & 54.4 & — \\ Model 3 & 0.030 \(\pm\) 0.021 & 0.71 & 88.0 & 83.5 & 0.037 \(\pm\) 0.020 & 0.71 & 81.9 & 71.3 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Numerical comparison of model performance for both endocardial and epicardial boundaries. With each model, mean test-set mean squared error (MSE) with associated standard deviations and R2 values are displayed. Additionally, the percentage of significant \(t\)-tests when the given row’s model’s test set MSE is smaller than that of the model indicated in the column is also provided. Figure 2: Heatmap representations of median mean-squared-error (left) and associated R values (right) following 100 Monte Carlo simulations of the 6:1:1 testing paradigm, stratified by theta, b, and z. Predictions are based on models (**A**) 1 (i.e., individual z slices), (**B**) 2 (i.e., combined z slices), and (**C**) 3 (i.e., assisted). Specifically here, model 3 was assisted by incorporating t = 0 radius values taken from the endocardial anterior mid-ventric position (i.e., z = 0.5, theta = 90 deg). While Figure 2 and Table 1 summarize aggregated model performance across all 100 test-set permutations, it also is instructive to see a particular example of the models' predictions vs. the ground truth. Figure 3A shows the predicted vs. actual radii for the 30\({}^{\circ}\) endocardial anchor point at the base of the heart plotted for a single test set using model 2. Qualitatively, model 2\({}^{\prime}\)s predictions overall appear relatively close to the ground truth. Errors in predictions seem to be mainly due to offsets in the size of the heart, rather than the wall kinematics (i.e., curve shape is correct but placed off from border). Figure 3B,E shows the accuracy gained using model 3 by incorporating user-assistance on an endocardial point. The heart size is more accurately inferred in this case, yielding better predictions overall. These results illustrate that qualitatively, both the unassisted and assisted models show reasonable performance for both the endocardial and epicardial boundaries on unseen data, with slightly higher accuracy for the endocardial boundary (Figure 3A,B vs. Figure 3D,E).