# Detection of Decision-Making Manipulation in the Pairwise Comparison Method
## 6 Experiments and Results
\begin{table}
\begin{tabular}{c c c c} \hline \hline \(n\) & **Naive Algorithm** & **Basic Algorithm** & **Advanced Algorithm** \\ \hline
3 & 64 & 99 & 88 \\ \hline
4 & 78 & 99 & 92 \\ \hline
5 & 89 & 99 & 93 \\ \hline
6 & 96 & 99 & 93 \\ \hline
7 & 98 & \(\sim\)100 & 92 \\ \hline
8 & 99 & \(\sim\)100 & 92 \\ \hline
9 & \(\sim\)100 & 99 & 90 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Attack detection rate. \begin{table}
\begin{tabular}{c c c} \hline \hline \(n\) & **Naive Algorithm** & **Basic Algorithm** & **Advanced Algorithm** \\ \hline
5 & FP: 86, FN: 354 & FP: 0, FN: 40 & FP: 235, FN: 48 \\ \hline
6 & FP: 166, FN: 21 & FP: 1, FN: 16 & FP: 179, FN: 51 \\ \hline
7 & FP: 72, FN: 5 & FP: 0, FN: 9 & FP: 148, FN: 76 \\ \hline
8 & FP: 29, FN: 5 & FP: 0, FN: 2 & FP: 146, FN: 57 \\ \hline
9 & FP: 11, FN: 2 & FP: 0, FN: 8 & FP: 138, FN: 53 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: False positive and false negative numbers. \begin{table}
\begin{tabular}{c c} \hline \hline
**Upper Threshold Value** & **Attack Detection Rate [\%]** \\ \hline
9 & 70 \\ \hline
20 & 79 \\ \hline
50 & 86 \\ \hline
100 & 93 \\ \hline
200 & 97 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Upper threshold impact on the attack detection rate. To evaluate the model, we used preprocessed sample matrices. Our initial attempt to train a neural network on bare matrices \(\mathbf{C}\) and \(\mathbf{C}^{\prime}\) yielded unacceptable results. Next, we attempted to use the error matrix (13) in the preprocessing step to achieve better results. However, the detection rate for the _basic_ algorithm oscillated around 65%, which is also unacceptable. Finally, in the preprocessing step, we map a square matrix \(M\subset\mathbb{R}^{2}\) to a cube \(D\subset\mathbb{R}^{3}\) using the transformation  
\[\mathbf{D}(\mathbf{C})= \begin{bmatrix}d_{i,j,k}\end{bmatrix} ,\quad d_{i,j,k}=\det \begin{bmatrix}c_{i,i}&c_{i,j}&c_{i,k}\\ c_{j,i}&c_{j,j}&c_{j,k}\\ c_{k,j}&c_{k,j}&c_{k,k}\end{bmatrix} ,\quad\forall\,i,j,k\in\{1,\dots,n\} \tag{17}\]  
This transformation converts the original matrix \(C\) into its three-dimensional map of local inconsistencies calculated in the same way as proposed by [PERSON] and [PERSON] (12) [81]. Thus, in this approach, the object of neural network identification is not the PC matrix itself but its representation, which is a three-dimensional distribution of local inconsistencies. This approach comes across the heuristic according to which dishonest experts make changes \"on the fly\" that are often not locally consistent [11]. Since our matrix representation is of dimension \(n\times n\times n\), we used 3-dimensional convolutional layers (Section 3.5). The concept of the architecture is a classical approach used for CNNs, with a block of convolutional layers followed by a densely connected section. In subsequent layers of the convolutional block, the size of the feature maps decreases. At the same time, the number of the filters grows, allowing further layers to become more specialized in detecting more complex phenomena in the matrices. The 3-dimensional structures are then flattened and fed to a sequence of dense layers which, in the end, yield the manipulation flag. To tune the network, we implemented an automated process using the KerasTuner module to tune the hyperparameters ([[https://keras.io/keras_tuner/](https://keras.io/keras_tuner/)]([https://keras.io/keras_tuner/](https://keras.io/keras_tuner/)), (accessed on 1 December 2023)). The following parameters were subjected to the tuning process:  
* the number of hidden (convolutional) layers;
* the number of feature maps in each layer, parametrized by the index of the layer;
* the size of filters;  
Figure 5: Weak positive correlation between inconsistency of manipulated matrix and distance indicators comparing original and manipulated matrices measured for 9-by-9 matrices. The ”advanced” algorithm was used for manipulation.