# Verification of Convolutional Neural Network Cephalometric Landmark Identification
## 2 Material and Methods  
Ten digital lateral cephalometric radiographs of subjects submitting for treatment in a university post-graduate orthodontic clinic, without congenital craniofacial/dental anomalies, history of facial trauma and a fully erupted permanent dentition, excluding third molars, were randomly selected. The digital radiograph picture size was of a magnification of 1.08-1.13, resolution (digital) max 5.7 lp/mm, image field (digital) 24/27 \(\times\) 18/30 cm and image pixel size 48 \(\upmu\)m.  
All digital radiographs were reviewed and data points selected manually by 30 operators (7 experienced orthodontic faculty members, 9 third year and 10 first year orthodontic residents, and 4 imaging center technicians), and automatically using Algoceph(r) point detection AI (Orca Dental, Hertzliya, Israel). A total of 21 commonly referenced soft and hard tissue lateral cephalometric points were selected (Figure 1), as defined by [PERSON] and [PERSON] (Table 1) [57]. Figure 1: Localization of the 21 lateral cephalometric landmarks used in the present study as defined by [PERSON] and [PERSON] (see Table 1). \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Landmark** & **Definition** \\ \hline
1 & Sella & Midpoint ofella Turica \\
2 & Nasion & Most anterior point on frontonasal suture \\
3 & Upper incisor tip (UI) & Tip of most prominent upper central incisor \\
4 & Lower incisor tip (LI) & Tip of most prominent lower central incisor \\
5 & B point & Deepest bony point on mandibular symbiosis between \\  & & pogonion and infradentale \\
6 & Pogonion (Pog) & Most anterior point of mandibular symbiosis \\
7 & Menton & Lowest point on mandibular symbiosis \\  & & Junction between inferior surface of the cranial base and \\
8 & Articulate & the posterior border of the ascending ramus of the \\  & & mandible \\
9 & A point & deepest point of premaxilla concavity bellow ANS \\
10 & ANS & Tip of anterior nasal spine \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Description of hard and soft tissue cranial landmarks used for comparative evaluation of human and AI detection. A training algorithm for AI recognition of these points was performed to imprint the CNN according to [PERSON] and [PERSON] (Figure 2) [58]. Values underwent forward and backward passes according to [PERSON] and [PERSON] until target accuracy is equivalent to or decreasing slowly from ground truth [59]. Subsequent to training, algorithm detection is performed and mapped point coordinates (x, y) are superimposed onto the original image with x and y-coordinates for each point being recorded given a defined 2 mm circumscribed precision range, according to Wang CW [60]. Each point was plotted 5 times using the AI method for which no differences in point detection were found, therefore, each point was noted as a single location. Manual landmark plotting was undertaken once by each observer. Statistical analyses were carried out using SPSS version 23 (IBM Corp, Armonk, New York, NY, USA). Non-parametric tests were required since descriptive statistics and the Kolmogorov-Smirnov tests for normality revealed that the data was not normally distributed at a level of significance where \(p<0.05\). Associations between AI and manual landmark detection were assessed using Wilcoxon Rank Sum Test (to compare AI to operator group), Spearman's Correlation (to compare findings at each delineated landmark), and Bartlett's Test (to test differences between variances of both AI and operators). Signed informed consent for use of medical records for teaching and research purposes was obtained for each subject prior to inclusion and as a standard procedure in agreeing to undergo treatment in the university postgraduate orthodontic clinic. Ethics committee approval was sought but deemed unnecessary.