## 4 Results  
Our proposed 3D object detection methods based on established U-Net architecture were evaluated by using the testing set of the USOVA3D database. This testing set contains 19 ovarian ultrasound volumes, whereas manual annotations from two raters for follicles and ovaries are part of a precisely specified validation protocol (see Section 2.1). The USOVA3D database is supplemented by the 3D DWT and 2D UNET baseline functions [7] which set the baseline statistical metrics of follicle and ovary detection effectiveness. For that reason, these baseline metrics, as well as the inter-rater variabilities, were incorporated into the results. Algorithms were ranked using an overall (algorithm) score \(\xi_{alg}\), whereat a higher value indicates a more effective detection algorithm. Table 1 contains the results of ovarian follicle detection for the original 3D U-Net (3D UNET) and by Deep Supervision (DS) upgraded (+) 3D U-Net method (3D UNET + DS). Each method was trained and evaluated 50 times on the USOVA3D database. Minimum, maximum, mean and median are presented of the algorithm's effectiveness over 50 runs. The 'best' designates that the CNN model with the lowest validation loss was selected, while the 'last' means that the model was picked in a particular run after the final training step. Although both proposed U-Net extensions, i.e., EXT 1 and EXT 2, were not developed primarily for follicle detection, we trained them to detect follicles according to the above described procedure (see also the previous section) and in an 'end-to-end' manner. These results were added to the Table 1 as well. Afterwards, a comparison was made with both USOVA3D baseline functions, with the state-of-the-art SNET method [11], and with the variability of both raters by follicle annotating. Inter-rater variability represents the upper limit of performance to which we aspire. This comparison can be seen in Table 2. We entered the results in this table only for the more effective CNN model, which we got among 50 runs for our individual method (i.e., a run where the max from Table 1 was obtained). In one row, there are aggregated results over all 19 test volumes for a particular method: Min and max denote the effectiveness (i.e., the final score), on the worst or best detected volume respectively, followed by median statistics over 19 test volumes, and, finally, the overall algorithm score is given. The implementation of the SNET [11] is not available publicly, so we implemented this method by ourselves based on published information. We applied the specified hyperparameters from [11]. The same training protocol (i.e., 50 runs) was utilised as by all our methods, and only the max obtained result was entered in Table 2.  
The evaluation protocol used in this study (see Section 2.1) was published recently in [7], and is, therefore, not yet used regularly by publishing results. For this reason, we have evaluated the effectiveness of selected follicle detection methods further using Sensitivity or Recall (S), Precision (P), Dice Similarity Coefficient (DSC), Jaccard Index (JCI) and the F1 score (F1), which are established metrics, but each of them covers only one aspect of the algorithm's detection performance (On the other hand, our used evaluation protocol combines several aspects over several raters into a common assessment or overall score, respectively!). Data in the USOVA3D database were annotated by two raters, therefore, the algorithm's segmentation result on each volume was compared with each of the annotations, and, finally, the average and Standard Deviation of the selected metric were calculated over all 19 testing volumes and both raters. The metrics calculated in this way are gathered in Table 3. Only the more successful variants of the proposed and compared methods have been added to this table (see also Table 2).