# Visual Saliency Detection Using a Rule-Based Aggregation Approach
## 3 Experimental Results
### Evaluation
#### 3.3.3 Saliency Detection Comparison with All Thresholds  
As mentioned above, in our second experiment, to evaluate all the methods under their best specific conditions, we obtained the binary result of the saliency maps with all possible thresholds in the range of \([0,255]\). Results of the second experiment are shown in Figure 7. In this figure, the horizontal-axis corresponds to the threshold value and the vertical-axis corresponds to the F-measure resulting from the use of the given threshold. Since our method RSD does not need a post-processing/thresholding step, the F-measure is the same for all possible thresholds. As shown in this figure, other methods reached maximum performance in a bounded range of the threshold values. In contrast, the solution of the proposed approach showed a constant performance due to thresholding independence.  
In the MSRA1K dataset (Figure 7a), our RSD achieved a maximum and steady F-measure value of 0.896 while the PW model reached a maximum value of 0.910 in a bounded range. The RC and PW models slightly overcame the proposed model in ECSSD and iCoseg datasets, as presented in Figure 7b,c, respectively. These models obtained higher F-measure values in a bounded range. In the ECSSD dataset the RC and PW obtained 0.726 and 0.747, respectively, and our proposal achieved 0.679. On the other hand, in the iCoseg dataset, our model attained 0.735 while the RC and PW achieved 0.765 and 0.775, respectively.  
In Table 4, we present the average F-measure from all the computed values on each threshold. In this table, we can observe that the average result of our proposed RSD performed higher than the rest of the models on the three datasets used. The model with the highest performance is highlighted in bold.  
The maximum F-measure rate reached at the curve by individual and aggregation models occured in a bounded threshold range. This aspect of the behavior of the models produced a peak when plotting the F-measure curve. An ascending and descending slope is the typical shape observed of the F-measure curve, which implies that there exist threshold values where the performance of the model is minimum. In contrast, the binary outcome produced by the RSD model maintained the same performance across the whole threshold range. The results obtained point out that the outcome from  
\begin{table}
\begin{tabular}{l c c c} \hline \hline \multirow{2}{*}{**Method (year)**} & **MSRA1K** & **ECSSD** & **iCoseg** \\ \cline{2-4}  & **F-Measure** & **F-Measure** & **F-Measure** \\  & **Average** & **Average** & **Average** \\ \hline FT (2009) & 0.481 & 0.280 & 0.434 \\ CA (2010) & 0.439 & 0.374 & 0.407 \\ RC (2011) & 0.761 & 0.628 & 0.675 \\ HC (2011) & 0.652 & 0.388 & 0.553 \\ SF (2012) & 0.733 & 0.489 & 0.611 \\ PW (2013) & 0.816 & 0.637 & 0.677 \\ MBS (2015) & 0.596 & 0.415 & 0.516 \\ MDC (2017) & 0.584 & 0.447 & 0.515 \\ RSD (ours) & **0.896** & **0.679** & **0.735** \\ \hline \hline \end{tabular}
\end{table}
Table 4: The averaged F-measure of the results obtained on each fixed threshold. The model with the highest performance is highlighted in bold.  
our RSD model performed steadily, being insensitive to the threshold. In contrast, the individual and aggregation models achieved their best performance at a certain range of threshold values, which was a bounded and ambiguous range. The output obtained from our RSD model gave us the certainty of constant performance.  
Figure 7: Comparison of the F-measure curve obtained with different thresholds for eight state-of-the-art algorithms and our approach RSD on three representative benchmark datasets: (**a**) MSRA1K dataset; (**b**) ECSSD dataset; and (**c**) iCoseg dataset.