# WIRE: A Weighted Item Removal Method for Unsupervised Rank Aggregation
## 5 Experiments
### Results and Discussion
\begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline \multirow{2}{*}{**TFVLO**} & \multirow{2}{*}{**MAP**} & \multicolumn{8}{c}{**Precision**} & \multicolumn{8}{c}{**NDCG**} \\ \cline{2-13}  & **P@1** & **P@2** & **P@3** & **P@4** & **P@5** & **N@1** & **N@2** & **N@3** & **N@4** & **N@5** \\ \hline Borda Count & 0.785 & 0.700 & 0.800 & 0.767 & 0.787 & 0.780 & 0.700 & 0.777 & 0.759 & 0.774 & 0.771 \\ CombMNZ & 0.785 & 0.700 & 0.775 & 0.767 & 0.787 & 0.780 & 0.700 & 0.758 & 0.756 & 0.772 & 0.769 \\ Condorcet & 0.785 & 0.650 & 0.750 & 0.800 & 0.800 & 0.800 & 0.650 & 0.727 & 0.768 & 0.773 & 0.777 \\ Copeland & 0.786 & 0.700 & 0.800 & 0.800 & 0.800 & 0.700 & 0.777 & 0.783 & 0.786 & 0.787 \\ Outranking & 0.785 & 0.700 & 0.800 & 0.767 & 0.787 & 0.780 & 0.700 & 0.777 & 0.759 & 0.774 & 0.771 \\ MC1 & 0.785 & 0.800 & 0.850 & 0.817 & 0.787 & 0.780 & 0.800 & 0.839 & 0.818 & 0.798 & 0.792 \\ MC4 & 0.786 Now, let us discuss these numbers. At first, the results indicate that DIBRA was the most effective weighted rank aggregation method, achieving the highest MAP scores among the other two methods, PRG and AAM. Interestingly, in all eight cases, our proposed WIRE method managed to further improve the performance of DIBRA by a margin between 1% and 20% (in the FESO dataset). In fact, in most of the examined benchmark datasets, the combination of DIBRA and WIRE outperformed all the competitive aggregation methods, weighted or not. In the MOLO dataset with the long lists of 100 items, the MAP gain of DIBRA-WIRE compared to DIBRA was roughly 2% (0.250 vs. 0.244). In contrast, the simple list pruning method of [13] offered only infinitesimal improvements in terms of MAP scores. In addition, its effect on the quality of the top-5 elements of the aggregate list was negative in terms of both Precision and nDCG. MC1 and MC4 were also quite effective in this dataset. The next three datasets, MASO, FESO, and MOSO, were of particular interest, because they involved short lists, and the simple pruning method of [13] is known to perform poorly in such cases. In this experiment, we examined three different scenarios, where variable populations of rankers (i.e., 100, 10, and 50, respectively) submitted short preference lists comprising 30, 10, and 30 items, respectively. The results were particularly satisfactory in all these scenarios. Compared to DIBRA, our proposed item selection method achieved superior performance in terms of MAP, Precision, and nDCG. For example, the MAP improvements over DIBRA were roughly 1% for MASO and MOSO, and 20% for FESO. Improvements of similar magnitudes were also observed for Precision and nDCG, especially for P@1 and N@1. As it was expected, the Mean Average Precision of DIBRA-P was slightly worse compared to that of DIBRA for MASO (0.323 vs. 0.324) and significantly worse in FESO (0.272 vs. 0.332). DIBRA and DIBRA-WIRE were also superior to all the other aggregation methods in all three datasets. The only exception to this observation was MC1 in the FESO dataset, which was superior to DIBRA but inferior to the combination of DIBRA-WIRE. The fifth case study, abbreviated MAVSO, introduced a scenario that resembles that of a recommender system, namely, numerous rankers submitting very short preference lists of 5 elements. The Outranking Approach of [24] exhibited the best performance in this test, achieving the highest MAP (0.416), P@1 = 0.400, and N@1 = 0.400. Our proposed DIBRA-WIRE method was the second best method, achieving a slightly worse MAP (0.414), P@1 = 0.350, and N@1 = 0.350. The original DIBRA without list pruning outperformed the other two weighted methods, PRG and AAM, but the application of the simple list pruning method of [13] had a strong negative impact on its performance. The sixth test resembled that of a metasearch engine, with few rankers submitting very long ranked lists of 200 items. Once again, DIBRA-WIRE achieved top performance in terms of MAP (0.789), P@1 = 0.800, and N@1 = 0.800.