# Automated Bone Age Assessment with Image Registration Using Hand X-ray Images
## 4 Image Registration  
Image registration is the transformation process to geometrically align all the images into a standardized form [38], where the segmented hand region will be positioned in the middle of the image with an upright position, where the angle between the middle finger and the horizontal axis is 90\({}^{\circ}\). It is an important step so that a regressor can be trained using a standardized pose which will put more emphasis on the age differences, rather than focusing on the variations of the X-ray images. In this work, three operations are performed in order to register the image into a standardized representation, which are segmentation, rotation, and alignment, as shown in Figure 3. Let define a set of input images, \(\mathbf{X}\) to be \(\{x_{1},\dots,x_{|\mathbf{X}|}\}\in\mathbb{R}^{4}\), where an \(i\) image of \(X_{i}\in\mathbb{R}^{3}\) is a vector comprises of its width (\(w_{i}\)), height (\(h_{i}\)), and channel (\(c_{i}\)) information. One of the novelties of this work is the application of separable convolutional neural networks in performing the image segmentation and rotation. It is a factorized form of the standard convolution into two operators, which are depthwise  
Figure 2: The first-row images depict some hand X-ray images taken with various contrast values. The second-row images depict some hand X-ray images taken from various tilting pose angles. and pointwise convolutions. A standard two-dimensional convolutional neural network operation with a kernel \(\mathbf{K}\in\mathbb{R}^{3}\) with a size of \(k_{w}\times k_{h}\times k_{s}\) can be written as  
\[\mathbf{V}(m,n,o_{out})=\sum_{\forall m,n,o_{in}}\mathbf{X}(m+j,n+k,o_{in}). \mathbf{K}(m,n,o_{in},o_{out}) \tag{1}\]  
where \(k_{w}=k_{h}\) is used throughout this work and \((m,n)=\in\{0,\dots,k_{w}\}x\{0,\dots,k_{w}\}\). \(o\) represents the number of channel either for input or output layer with \(o_{in}\in\{0,\dots,k_{s}\}\). Subsequently, a two-dimensional separable convolution can be formulated, as follows  
\[\mathbf{P}(m,n,o_{in}) =\sum_{\forall m,n}\mathbf{X}(m+j,n+k,o_{in}).\mathbf{K}(m,n,o_{ in}) \tag{2}\] \[\mathbf{Q}(m,n,o_{out}) =\sum_{\forall m,n,o_{in}}\mathbf{X}(m,n,o_{in}).\mathbf{K}(0_{ in},o_{out}) \tag{3}\]  
where \(\mathbf{P}\) represents the separable convolution operation and \(\mathbf{Q}\) represents the pointwise convolution operator, which can be linked to the standard convolution in the form of  
\[\hat{\mathbf{V}}(m,n,o_{out})=\mathbf{P}(m,n,o_{in})\mathbf{Q}(m,n,o_{out}) \tag{4}\]  
Hence, the registered image, \(\hat{\mathbf{X}}\in\mathbb{R}^{4}\) is a composite function of three operations, which are masked segmentation (\(I_{segment}\)), rotation alignment through keypoints regression (\(I_{rotate}\)), and positional alignment (\(I_{align}\)). \[\hat{\mathbf{X}}=I_{align}(I_{rotate}(I_{segment}(\mathbf{X}))) \tag{5}\]  
The first step goal, which is a masked segmentation, is to extract the hand region from the background. The output of \(I_{segment}\) function \(\mathbf{X_{1}}\) is an element-wise multiplication between the input image \(\mathbf{X}\) and the masked label \(\mathbf{X}_{mask}\) that was generated from the segmentation network. \[\mathbf{X_{1}}=\mathbf{X}\odot\mathbf{X}_{mask} \tag{6}\]  
This operation will remove all background objects that include identification plates, medical tubes and equipment, noises, supporting tools, etc. from affecting the bone age prediction accuracy. DeepLab V3 plus [16] is used as the segmentation network, which uses Xception-65 [18] as its backbone. The full segmentation architecture follows the encoder-decoder format, where the encoder network comprises of Xception-65 and atrous spatial pyramid pooling (ASPP) module. On the other hand, the decoder network utilizes two up-sampling steps through a composite operation of separable convolutional layer and bilinear interpolation. The ASPP module consists of three parallel down-pooling branches with three different dilation rates, \(\mathcal{D}=6,12,18\) and a residual network as used in [39].