DISSERTATION ABSTRACT

Metric Optimization and Mainstream Bias
Mitigation in Recommender Systems
Roger Zhe Li
Delft University of Technology
The Netherlands
zhe.roger.li@gmail.com

Abstract

Recommender Systems have drawn extensive attention in recent decades. The most importantfactor to make a recommender succeed is user satisfaction, which is largely reflected by the recommendation accuracy. In accordance, this thesis dives into recommendation accuracy from two
different perspectives that, unfortunately, are at tension with each other: achieving the maximum
overall recommendation accuracy, and balancing that accuracy among all users.
The first part of this thesis focuses on maximizing the overall recommendation accuracy Li
et al. [2021a]. This accuracy is usually evaluated with some user-oriented metrics tailored to therecommendation scenario. However, recommendation models could be trained to maximize someother generic criteria that do not necessarily align with the criteria ultimately captured by themetric above. Recent research usually assumes that the metric used for evaluation is also themetric used for training. We challenge this assumption, mainly because some metrics are more
informative than others. Indeed, we show that models trained via the optimization of a loss
inspired by Rank-Biased Precision (RBP) tend to yield higher accuracy, even when accuracy ismeasured with metrics other than RBP. However, the superiority of this RBP-inspired loss stems
from further benefiting users who are already well-served, rather than helping those who are not.
This observation inspires the second part of this thesis, where our focus turns to helping nonmainstream users, who are difficult to recommend to either because of the lack of data for effectivemodeling or a niche taste that is hard to match similar users.
Our first effort consists in using side data, beyond the user-item interaction matrix, so thatusers and items are better represented. This will be of benefit especially for the non-mainstream
users, for which the user-item matrix alone is ineffective. We propose Neural AutoEncoder Collaborative Filtering (NAECF) Li et al. [2021b], an adversarial learning architecture that, in addition
to maximizing the recommendation accuracy, leverages side data to preserve the intrinsic properties of users and items and show that NAECF leads to better recommendations specially fornon-mainstream users, while at the same time there is a marginal loss for the mainstream ones.
Our second effort Li et al. [2023] consists in explicitly focusing more on non-mainstream users
in a recommendation model. In particular, we propose a mechanism based on cost-sensitive
learning that weighs users according to their mainstreamness, so that they get more attention
during training. The result is a recommendation model tailored to non-mainstream users, thatnarrows the accuracy gap, and again at negligible cost to the mainstream users.
ACM SIGIR Forum

1Vol. 57 No. 2 – December 2023

Awarded by: Delft University of Technology, Delft, The Netherlands on 14 November 2023.
Supervised by: Alan Hanjalic and Julián Urbano.
Available at: https://doi.org/10.4233/uuid:235a5f5c-5e18-4e67-a8e6-84a7e310db12.

Selected Publications
Roger Zhe Li, Julián Urbano, and Alan Hanjalic. Leave no user behind: Towards improving theutility of recommender systems for non-mainstream users. In WSDM, pages 103–111. ACM,2021a.
Roger Zhe Li, Julián Urbano, and Alan Hanjalic. New insights into metric optimization for
ranking-based recommendation. In SIGIR, pages 932–941. ACM, 2021b.
Roger Zhe Li, Julián Urbano, and Alan Hanjalic. Mitigating mainstream bias in recommendation
via cost-sensitive learning. In ICTIR, pages 135–142. ACM, 2023.

ACM SIGIR Forum

2Vol. 57 No. 2 – December 2023