A Game Theoretic Evaluation Framework of Recommendation Algorithms

While several approaches to evaluate Recommendation algorithms do exist, there is little research available to find best suitable approach for recommendation to a particular user. We consider using a novel game theoretic approach to find a way to evaluate the best suitable recommendation algorithm for a user over a given span of time. We formulate payoff values for the game players to find best algorithm for the user over a number of sequential, repeated games. Introduction Recommendation systems are subclass of information filtering system that seeks to predict preference that a user would give to an item. Recommendation system changed the way websites communicate with their users. In the current age of the internet and big data, recommender systems have become more and more prevalent. Amazon and Netflix are not the only websites that use recommendation systems, but most medium and large scale commercial and social websites have recommendation engines in place for them. Many different types of recommendation algorithms have been developed and thus a key research area currently in recommender systems is on how to evaluate them, or how to measure the ‘goodness’ of a given recommendation algorithm. In this project, we compare recommendation based algorithms using a game theoretic approach. Research has been done for the evaluation of various algorithms but often times it considers a customized form of data set and user group. Also, there exists a gap between linking algorithm evaluation to user feedback evaluation of recommendations being provided to them by those algorithms. In other studies, the most common method is to allow the users themselves to judge the recommendations by giving them a survey or questionnaire. The questionnaire might ask a series of questions to evaluate different aspects of the recommendations such as accuracy, novelty, diversity, satisfaction, etc. The only issue with this approach is that the users, more often than not, will not opt to take part in the survey. Indeed, in the aforementioned paper, it was found that less than 1/3rd of the participants actually completed the survey. This makes things difficult to truly assess the recommender system with complete accuracy. Our approach makes use of game theory concepts to formulate a novel evaluation framework for different recommendation algorithms. This method can be used to analyze the ‘goodness’ of each algorithm without requiring any explicit participation from the user. We do this by devising a ‘game’ played by the recommender system and a unique user. The recommender system’s strategies will be the various algorithms that we are to judge. The user’s strategies are simple: given the recommendations pertaining to the chosen algorithm, he can choose to Accept or Ignore. We consider this as the ‘stage game’. Since the user will be recommended items throughout his usage of the application or website, we can repeat this stage game multiple times to improve the accuracy of the evaluation. This is known as a ‘repeated game’. Also, this game will be ‘sequential’ and will be a game with ‘incomplete information’. The recommender system will employ a form of ‘trigger strategy’. We will explain these terms in the next section.