Incorporating User-Item Similarity in Hybrid Neighborhood-based Recommendation System

Abstract: Modern hybrid recommendation systems require a sufficient amount of data. However, several internet privacy issues make users skeptical about sharing their personal information with online service providers. This work introduces various novel methods utilizing the baseline estimate to learn user interests from their interactions. Subsequently, extracted user feature vectors are implemented to estimate the user-item correlations, providing an additional fine-tuning factor for neighborhood-based collaborative filtering systems. Comprehensive experiments show that utilizing the user-item similarity can boost the accuracy of hybrid neighborhood-based systems by at least $2.11\%$ while minimizing the need for tracking users' digital footprints.

Reviews and author responses:

This paper proposed several ideas to improve neighborhood-based recommendation systems by characterizing user preferences using both rating data and item content information. The 2 major proposed ideas are 1) weight item feature by feature score, 2) incorporate user-item similarity score in item score prediction. Experiments over the MovieLens dataset show that the proposed approach can outperform the baselines. Strength:
This paper is well written and easy to read. What's really great is that the authors use examples to motivate and explain their ideas.

Weakness:
1. No literature review of related work. the authors didn't provide an overview of the related research direction, especially the latest SOTA solution of the problem studied in this paper.
2. The proposed ideas are simple and lack technical complexity and novelty. Idea 1 of using feature score to weight feature makes sense to me. Idea 2 doesn't seem to be a new idea as modeling user-item score as u+b_i+b_u+s_ui is a standard approach existed for quite a long time.
3. The baseline methods used in experiments are too simple. The authors didn't use any advanced algorithms proposed in recent years. It's hard to convince the readers without comparing with such baselines. This paper lacks novelty and has some flaws in the content. I recommend rejection.

In this paper, the authors study the recommendation systems. They first propose three new methods to characterize user preferences utilizing both rating data and item content information. The first method takes the weighted average of the feature vectors of items, the second one incorporates the effect of biases in the first one, and the third method puts weight on the biased feature vector of users using the item feature vector.  Next, they revise the baseline estimate by integrating the user-item similarity. They further integrate these methods into two baseline models, namely kNNBaseline and kNNContent. Experimental results suggest that these improvements lead to an increase in accuracy (2%-6%) with the expense of the additional complexity.  Pros:
The idea of using both users’ rating data and items’ content information is very intuitive in RecSys. 
The proposed methods are well presented with giving the related work and motivation. 
The experimental results show that the predicting accuracy increases when the proposed user feature vector is utilized in baseline models.  
Cons:
The proposed user feature vectors are very incremental, just slightly modifying the current methods. For example, in the first method they propose (normalized feature vector), they dot product users’ rating info with the average of the items feature vectors to calculate the users’ interest in the items. But, as we see in Equation (1), the current neighborhood-based models do the same thing but just multiply users’ rating info with the similarity degree of items, whereas the similarity between items is calculated using the items’ feature vectors. In the second method they propose, they again slightly modify Koren’s formula in Equation (2). We observe a similar method in the baseline estimate. They again just slightly modify the current baseline estimate by including the user-item similarity score. These are very small changes in the current methods. 

The paper lacks enough computational considerations. For example, as we see in Table 2, the proposed user feature vectors have about 3 times more running time than the current method, but the time complexity of the proposed methods is not provided. 
 
The trade-off between efficiency and effectiveness is not well justified. The increase in the accuracy (i.e., the decrease in the errors) is between 2% and 6%. At the same time, the proposed methods are computationally more complex based on the running times. As the authors mention, in the recommendations systems, the data that needs to be analyzed is huge. Hence, the complexity is a big deal. The authors claim that their method can be performed in parallel with a low computational cost without any justification and backing up this reasoning. Hence, this is not enough reason to neglect the complexity.
 The paper proposes recommendation system methods with better performance and additional complexity. However, this paper lacks novelty and the improvements in the predicting accuracy are not enough when we consider the increase in running times. 


This paper incorporates user-item similarity score to item-based CF recommender systems.

The user-item similarity score is calculated by Pearson Correlation Coefficient or Cosine Similarity between user features and item features, where item features are tags, and user features are a weighted sum of (user's rated) items' features (tags). The calculated score are used as part of the baseline estimate of the item-based CF method. Strength
- The proposed method is clear and simple.


Weakness
- The novelty of this work is very limited. The overall framework of this paper can actually be seen as an ensemble of an item-based CF, and a simple content-based recommender system. Both of them are well studied in the literature.
- Lack of comparison with state-of-the-arts. [1], [2], [3], [4], ...

[1] Cheng, Heng-Tze, et al. "Wide & deep learning for recommender systems." Proceedings of the 1st workshop on deep learning for recommender systems. 2016.

[2] He, Xiangnan, et al. "Neural collaborative filtering." Proceedings of the 26th international conference on world wide web. 2017.

[3] Xiao, Jun, et al. "Attentional factorization machines: Learning the weight of feature interactions via attention networks." arXiv preprint arXiv:1708.04617 (2017).

[4] He, Xiangnan, et al. "Nais: Neural attentive item similarity model for recommendation." IEEE Transactions on Knowledge and Data Engineering 30.12 (2018): 2354-2366.
 Overall the paper is clearly with limited novelty, so I am recommending with strong rejection.

This paper presents how to add user-item simiarity into Neighoborhood-based recommender system. Expriements are conducted on a MovieLens 20M dataset to show its effectiveness. (1)The depth of technique is not impressed. The idea of considering user-item simiarity is widely studied in RS ;
(2)The study on Neighoborhood-based recommender system is also not so interesting. Most STOA studies have not been introduced. Therefore, the motivation of the article is not clear enough and needs to be discussed well. 
(3) The selected baselines are not strong to let readers confirm that the significance of the proposed system. For example, the most recent baseline is published in 2019, titled" Utilizing an autoencoder-generated item representation in hybrid recommendation system. IEEE Access, PP:1–1, 04 2020. doi: 10.1109/ACCESS.2020.2989408.
(4) A thorough survey of RS should be done before writing a paper. Based on my review comments, I recommend to reject this submission. The authors should get to know the related work of RS in recent three years before starting a paper draft.

This paper studies hybrid recommendation methods that specifically combines neighborhood-based techniques with content-based user/item feature techniques. It proposes several ways to estimate a user-item correlation matrix and integrates it into neighborhood-based methods as an additional fine-tuning factor. The paper compares proposed methods with several baselines and evaluates RMSE and MAE on MovieLens20M. $\textbf{strengths}$
- The paper integrates content-based techniques to classic neighborhood-based methods, resulting in a hybrid recommendation method. The methodology is well explained and easy to understand.
- The paper proposes several variants of the method with justifications.

$\textbf{weaknesses}$
- The method proposed by the paper seems to be a relatively incremental modification on neighborhood-based methods with an additional tuning factor.
- The paper evaluates its method on only one dataset, MovieLens20M. Ideally, the paper should evaluate on several datasets with diverse settings.
- The evaluation metrics used in the paper are RMSE and MAE. To my knowledge, recommender system community is more interested in ranking based metric such as mAP and NDCG. The paper can include the performance on these metrics.
- It seems that the paper does not compare its method with other hybrid recommendation methods. The proposed approach seems to be incremental and a more thorough evaluation is needed.

Meta-review: This paper proposed to improve hybrid neighborhood-based recommender systems by incorporating learned user-item similarity. Overall, the scores are towards negative. The reviewers did acknowledge that the paper proposed a simple-to-implement method and reads well. However, the negatives are plenty: the lacking of a comprehensive literature review as well as more relevant state-of-the-art baselines in the experiments is a common concern among most reviewers. The novelty of the proposed approach is also rather limited as incorporating user-item similarity from user rating and item contents is a well-explore topic within the literature. Finally, using rating prediction as evaluation method ignores the missing-not-at-random nature of a recommender system. The authors didn't provide any response. Therefore, I vote for reject.

Decision: Reject