# Verification of Convolutional Neural Network Cephalometric Landmark Identification
## 4 Discussion  
The use of AI as a management tool has already found an application in orthodontic patient selection/referral within public health care systems [61]. The use of deep learning in AI as a diagnostic tool to perform cephalometric landmark identification can potentially eliminate intra/inter-observer variation as well as vastly reduce the time invested and decreased efficiency in performing this task manually [62]. In order to do so, there have been several methods of application of deep learning in landmark detection [63]. The method used in the present study is regression-based deep learning as described by [PERSON] et al. [64].  
[PERSON] et.al. proposed as correct/acceptable a \(\pm\)2 mm differential in human versus computerized lateral cephalometric landmark identification, where if more than 20% of the total localizations were unacceptable equated as a failed comparison [43]. Based on this definition of \"correct\", the purpose of the present study was to compare the identification of lateral cephalometric landmark accuracy by the latest generation of artificial intelligence to that of human observers with varied amounts of aggregated clinical experience. The amount of error in landmark identification was the difference between that produced by experienced observers and by AI.  
Twenty-one cranial landmarks (Table 1) were registered according to automatic point detection (Algocoph\({}^{\circledright)}\). Human observer identification was plotted as an envelope surrounding the former which was taken as an origin (0,0), with each such point delineated into its \(x\),\(y\) coordinates. It was found that in 36 out of 42 (85.7%) coordinates no statistically significant differences were found between AI and all human observers (Table 2 and Figure 6a,b).  
It was found that coordinates of six anatomic landmarks showed statistically significant differences between human and automatic identification; SoftpogY, whose measurement error was clinically insignificant (2.67 mm \(\pm\) 2.55), UpperlipY which was clinically acceptable (1.11 mm \(\pm\) 1.16), as were OrbitaleX (1.07 mm \(\pm\) 1.29), PTMX (0.99 mm \(\pm\) 0.98), PorionY (1.14 mm \(\pm\) 1.41), and BasaleX (1.03 mm \(\pm\) 0.90). These landmarks were also previously described as highly prone to erroneous identification by [PERSON] [19,20], however, the differences found in the present study were small enough to deem them diagnostically relevant (Table 2).  
Adoption of digital technologies in orthodontics were initially applied to data storage and automated clear aligner production. Current efforts are to \"teach\" these tools to perform diagnostics and treatment planning [65]. Doing so stipulates the understanding that the reliability of any measurement derived from radiographic analysis depends on the reproducibility in the identification of defined landmarks. Factors such as the quality of the radiographs (contrast, scaling, etc.), and operator reliability have been shown to influence the magnitude of identification error. Earlier studies of the performance of AI to identify (fewer) cephalometric landmarks reported far lower levels of accuracy than those found in the present study using the CephX Algo method (Table 3) [36,37,44,45,46]. A more recent report by [PERSON] et al. described findings in agreement with those of the present study [66]. Taken together; these suggest that the CephX Algo method has reduced the above sources of measurement error to the extent that its output can be accepted as diagnostically accurate.