# Effects of the Hybrid CRITIC-VIKOR Method on Product Aspect Ranking in Customer Reviews
## 4 Results and Discussion  
The performance evaluation of the proposed hybrid CRITIC-VIKOR technique placed a strong emphasis on evaluating the retrieved product attributes gleaned from Internet reviews. Based on the suggested strategy, this study sought to rank the most crucial product features. The performance of the suggested method was compared with the subjective VIKOR method in order to offer a thorough analysis. This made it possible to compare and contrast the results of the ranking for the product attributes that were extracted. For evaluation, the first benchmark dataset introduced by [PERSON] [5], consisting of consumer reviews of four electrical devices, was used to conduct the tests. Based on actual consumer input, these datasets were used to assess the effectiveness and performance of the suggested method. The research aimed to confirm the superiority and efficacy of the suggested hybrid  
Figure 1: Product aspect ranking in the hybrid CRITIC–VIKOR. Figure 2: Structure of the decision matrix. CRITIC-VIKOR technique in producing precise and significant rankings of product aspects, allowing potential buyers to make informed selections. The details of the datasets are shown in Table 1.  
The second benchmark dataset that was used in the evaluation is the SemEval 2016 Task 5 dataset of laptop reviews [47]. The original laptop reviews contain about 2375 sentences. Among these sentences, we focused on the sentences that have opinions, as subjective information is the focus of sentiment analysis. Accordingly, the total number of opinionated sentences in laptop reviews is 2039. The normalized discounted cumulative gain at top k (NDCG@k) measure was used in the evaluation procedure because it is well known to be a superior tool for evaluating ranking quality [48; 49]. By combining graded relevance judgments, NDCG@k provides various advantages over other ranking metrics. The capability of NDCG@k to handle many levels of relevance judgments is one of its main advantages. NDCG@k takes graded relevance into account as a measure of utility, in contrast to other ranking metrics like mean average precision (MAP), which solely deal with binary relevance (either \"relevant\" or \"irrelevant\"). Because of its versatility, the ranking quality may be evaluated in a more nuanced manner, offering a truer representation of the preferences and pleasure of the user. This study intends to measure the genuine efficacy of the suggested hybrid CRITIC-VIKOR approach in rating product characteristics by using NDCG@k as the assessment metric. The usage of NDCG@k improves the accuracy and thoroughness of the evaluation, allowing for a more thorough examination of the performance of the suggested method in contrast to other ranking strategies. The measure of NDCG accumulated at a particular rank \(k\) is defined in Equation (10):  
\[NDCG@k=\frac{1}{Z}\sum_{i=1}^{k}\frac{2^{t(i)}-1}{log(1+i)}, \tag{10}\]  
where \(t(i)\) refers to the relative importance of the candidate product aspect at position i, and Z is a normalization term derived from the perfect ranking at the top-k aspects. The evaluation approach introduced by [50] was used to determine the importance of each aspect. Typically, the evaluation approach for aspect importance is based on human judgments. Thus, three annotators were invited to evaluate the importance of each aspect according to three levels of importance, namely, \"unimportant\", \"ordinary\", and \"important\" represented numerically by \"1\", \"2\", and \"3\", respectively. One of the main tasks of the CRITIC method is to compute the correlation coefficient of the evaluation criteria to discover the relative importance of a corresponding criterion based on the degree of its conflict with the other criteria. Based on Equation (4) mentioned previously, Tables 2-5 present the linear correlation coefficients for each pair of criteria used in this research.