# Optimized Weighted Nearest Neighbours Matching Algorithm for Control Group Selection
## 5 Results
### Results of the Scenario II  
The dataset presented in Scenario I was based on a widely used, general benchmark dataset. As the proposed WNNSA method aims to improve the efficiency of the WNNEM method in a conflicted environment, such a kind of dataset was generated (see Section 4.2) for the second scenario. This dataset contains fewer descriptive features to better illustrate the problem arising from conflicting candidates. Additionally, it contains covariates with negative and positive associations and also nominal, ordinal and continuous variables. As  
Figure 3: Results of the Hansen and Bowers test in Scenario I. Comparison of the \(p\)-values of the Hansen and Bowers test in case of the nearest neighbour matching (NNM), Mahalanobis metric matching (MMM), two types of PSM methods (PSM_02, PSM_DYN), the WNNEM method and the WNNSA method for the 100 datasets in Scenario I.  
this dataset contains negatively associated variables as well, the WNNEM method would not be able to run on it in its basic form. To perform a full evaluation, the WNNEM method was modified for this scenario to handle negative covariates. The dimension-wise weight calculation was extended to include \(OR_{i}<1\) according to the Equation (2). Figure 4 shows the individual balance values for ordinal and nominal variables. For the attributes \(x_{1}\) and \(x_{2}\), the NNM method produced the most similar distributions. As these attributes had only minor influences on the value of the output variable, the WNNSA method placed less emphasis on their fitting. However, the second-best fit for these attributes was achieved by the the WNNSA method. Comparing the WNNSA method to the extended WNNEM method, the distribution of the balance in the case of \(x_{1}\) and \(x_{2}\) variables is better in the case of the WNNSA method. For attributes \(x_{3},x_{4},x_{5},x_{6}\), the WNNEM and WNNSA methods chose the same values for all paired individuals. This is partly because these are binary attributes at which the value-mismatch would result in a significant discrepancy; furthermore, some of them already had a medium impact on the outcome variable. That is, they had more effect on the distance calculations. The NNM method, which performed well on attributes \(x_{1}\) and \(x_{2}\), did not always achieve an equivalent pairing for these attributes. In the case of continuous variables (Figure 5), the extended WNNEM method gave the second worse results. The disadvantage of the WNNEM method in balancing continuous attributes is due to the local optimization and normalized distance computation. In the normalized distance calculation, the difference in the values of features with fewer values resulting in more significant differences in distance; however, a small difference between the values of the continuous attributes results in a more negligible difference between the two individuals. As the WNNEM method only selects from the two nearest neighbours, the WNNEM algorithm involuntarily favours value matching on variables with lower cardinality. The WNNSA method reduces this bias by working in a larger environment of the individuals and, thus, better balances the distribution of continuous feature values. In this way, the WNNSA method was able to improve the fit of the WNNEM method significantly, and on attribute \(x_{7}\) it achieved the best fit. It also achieved a good result on attribute \(x_{8}\). Considering the results of Figures 4 and 5 together, the WNNSA method achieved the best result for five out of eight attributes, the second-best match for two attributes, and the third best match for one attribute. Table 2 shows the values of the group-level similarity and distance measures for Scenario II. The quality measures were calculated based on the result of the control group selections performed on the 100 datasets. It can be seen that the SM method yielded the worst results in most cases, analogously to Scenario I.