# Estimation of the Prostate Volume from Abdominal Ultrasound Images by Image-Patch Voting
## 3 Proposed Method
### Quadruplet Network  
The quadruplet patches described in the previous section vote to estimate diameter endpoints through a network we refer to as Quadruplet Deep Convolutional Neural Network (QDCNN), whose structure is shown in Figure 5. QDCNN comprises four ResNet-18 DCNNs with a joint classification layer and a joint loss. Other types of quadruplet networks are very popular in re-identification or similarity learning studies [43], which are trained with four images where two of them are from the same class, and the others are from different classes. Pairs are learned as positive or negative depending on whether they are in the same or different classes. A quadruplet loss is calculated in this way to achieve greater inter-class and less intra-class variation while identifying images. Differently, in our study, each quadruplet patch obtained at different scales is the input of each of the quadruplet networks with a joint classification layer and a joint loss. The first 16 shared layers of these networks were taken as pretrained from the PyTorch/vision library [44] and frozen, and only the last two layers were fine-tuned during the training process. Thanks to this design, QDCNN can retrieve scale-specific information from each scale. Figure 4: Patch centers are represented on AUS images. Expert marks are shown in green, while patch centers are shown in red. **(a)** Random sample training patch centers on a transverse image. **(b)** Random sample training patch centers on a sagittal image. **(c)** Sample test patch centers on a transverse image. **(d)** Sample test patch centers on a sagittal image. Figure 5: The QDCNN structure is composed of 4 DCNNs with a joint classification layer and a joint loss. The number of outputs k is 8 for transverse and 4 for sagittal images. Our quadruplet network is actually a multi-task classifier that learns to predict the distance and orientation classes relative to each diameter endpoint for a given quadruplet patch of a location. Figure 6 shows the calculation of distance and orientation class values. The distance values between 0 and 1000 pixels are quantized into 10 classes, and the 11 th class is for values greater than 1000. The intervals for distance classes are smaller for small distance values, and they get larger for larger distances. Orientation values \([0,2\Pi]\) were quantized into eight equal classes. Consider the \(j^{th}\) patient with a transverse image \(T^{j}\) with diameter endpoints \(e_{1},e_{2},e_{3},e_{4}\) and a sagittal image \(S^{j}\) with diameter endpoints \(e_{5}\) and \(e_{6}\), where \(e_{i}\in R^{2}\). For a given point \(T^{j}(x,y)\) on the transverse image of the patient \(j\), four equal-size patches \(P_{1}(T^{j}(x,y)),P_{2}(T^{j}(x,y)),\)\(P_{3}(T^{j}(x,y)),\) and \(P_{4}(T^{j}(x,y))\) of different scales were created that composed a quadruplet patch \(PQ(T^{j}(x,y))\) of the transverse image. Quadruplet patch \(PQ(S^{j}(x,y))\) was created similarly for the sagittal image. Due to the different structures of transverse and sagittal planes, our system trains two different QDCNN classifiers with a different number of outputs. We defined two functions \(c_{d}\) and \(c_{o}\) to obtain distance (\(cd\)) and orientation (\(co\)) classes, respectively. For a given point \(T^{j}(x,y)\) on a given transverse image \(T^{j}\), \(cd_{i}^{j}(x,y)=c_{d}(T^{j}(x,y),e_{i})\), and \(co_{i}^{j}(x,y)=c_{o}(T^{j}(x,y),e_{i})\) where \(i=1,\dots,4\). Similarly, for a given point \(S^{j}(x,y)\) on a given sagittal image \(S^{j}\), \(cd_{i}^{j}(x,y)=c_{d}(S^{j}(x,y),e_{i})\), and \(co_{i}^{j}(x,y)=c_{o}(S^{j}(x,y),e_{i})\) where \(i=5,6\). In other words, the QDCNN classifier for the transverse plane (\(QDCNN_{T}\)) has eight classification tasks, and the QDCNN classifier for the sagittal plane (\(QDCNN_{S}\)) has four classification tasks. In the training phase, for each AUS image from transverse or sagittal planes, quadruplet patches are extracted from normally distributed random locations around diameter endpoints.