# WIRE: A Weighted Item Removal Method for Unsupervised Rank Aggregation
## 4 WIRE: Algorithm Design and Analysis
Thus, if the item was preferred by numerous experts, then its preservation score will be high. In contrast, the elements submitted by a few non-experts will be assigned lower preservation scores. Moreover, notice the independence of \(P_{\mathbf{e}}\) from other parameters, e.g., the individual rankings, or the number of pairwise wins and losses, etc. This design choice renders the preservation scores robust to manipulation from spammers or low-quality preferences originating from non-expert users. The left diagram in Figure 2 illustrates an example of how confidence and preservation scores are computed for \(n=6\) rankers and \(B=3\) buckets. The preservation score is a measure of the importance of an element \(\mathbf{e}\) and determines whether it will be included in the aggregate list or whether it will be discarded. Formally, the higher the preservation score of \(\mathbf{e}\), the higher the probability that \(\mathbf{e}\) will be preserved in \(\mathcal{L}^{\prime}\). For this reason, the _protection score_ is introduced to determine the number of elements  
Figure 2: Indicative examples of the execution of WIRE. The left part demonstrates how the rankers are grouped into buckets and the computation of the confidence and preservation scores (\(\mathbf{a}\)). The right part illustrates the removal of the weakest elements from an input preference list (\(\mathbf{b}\)). that will be taken into account during the aggregation. Similarly to the confidence score, the protection score \(R_{b}^{(u)}\) is defined for each ranker \(u\) according to the following equation:  
\[R_{b}^{(u)}=\left\lceil|I^{(u)}|C_{b^{(u)}}\right\rceil. \tag{8}\]  
Equation (8) defines the protection score as an integer threshold that denotes the number of elements of \(I^{(u)}\) that will contribute during the aggregation process. In other words, it tells us that the \(|I^{(u)}|-R_{b}^{(u)}\) weakest elements must be removed from \(I^{(u)}\). The item removal process is shown in steps 20-28 of Algorithm 1. For each input list, we build a min-heap structure \(H\) to efficiently identify the items to be removed without having to sort \(I^{(u)}\). \(H\) is built by inserting all elements of \(I^{(u)}\) and keeps on its head the element with the minimum preservation score. Then, the \(|I^{(u)}|-R_{b}^{(u)}\) items with the lowest preservation scores are indicated by performing an equal number of pop operations on \(H\) (steps 24-28). The diagram on the right in Figure 2 shows the removal of the two weakest elements from an exemplary input list based on the preservation scores of its elements. After the removal of the weakest elements from each input list, the aggregator \(\mathcal{A}_{w}\) is executed once more on the new lists to obtain the enhanced aggregate list \(\mathcal{L}^{\prime}\) (step 29). **Illustrative Examples. ** Figure 2 illustrates two examples of how the stages of Algorithm 1 operate in practice. The left part (a) shows the distribution of \(n=6\) rankers in \(B=3\) equally sized buckets. Initially, each ranker \(u\) submits a preference list \(I^{(u)}\) and a weighted aggregator \(\mathcal{A}_{w}\) constructs a consensus ranking \(\mathcal{L}\) by simultaneously assigning weights \(w^{(u_{1})},\ldots,w^{(u_{6})}\) to all rankers. Setting \(\delta_{1}=0.5\), the confidence scores for each bucket \(b=1,2,3\) are computed using Equation (6). Subsequently, the preservation scores of the elements of the aggregate list are computed with Equation (7). For example, the element \(\mathbf{e}_{1}\in\mathcal{L}\) appeared in the input lists of \(u_{1}\), \(u_{2}\), and \(u_{3}\), which have been previously grouped into buckets 1, 1, and 2, respectively. Therefore, its preservation score will be computed by summing the confidence scores of the following corresponding buckets: \(P_{\mathbf{e}_{1}}=C_{b^{(u_{1})}}+C_{b^{(u_{2})}}+C_{b^{(u_{3})}}=1+1+0.803=2.803\). Having the preservation scores computed, we can now go back to the input lists and remove the weakest elements. First, we need to compute the protection scores for each ranker.