CM^3: Calibrating Multimodal Recommendation
Authors: name: Xin Zhou, name: Yongjie Wang, name: Zhiqi Shen

Alignment and uniformity are fundamental principles within the domain of
contrastive learning. In recommender systems, prior work has established that
optimizing the Bayesian Personalized Ranking (BPR) loss contributes to the
objectives of alignment and uniformity. Specifically, alignment aims to draw
together the representations of interacting users and items, while uniformity
mandates a uniform distribution of user and item embeddings across a unit
hypersphere. This study revisits the alignment and uniformity properties within
the context of multimodal recommender systems, revealing a proclivity among
extant models to prioritize uniformity to the detriment of alignment. Our
hypothesis challenges the conventional assumption of equitable item treatment
through a uniformity loss, proposing a more nuanced approach wherein items with
similar multimodal attributes converge toward proximal representations within
the hyperspheric manifold. Specifically, we leverage the inherent similarity
between items' multimodal data to calibrate their uniformity distribution,
thereby inducing a more pronounced repulsive force between dissimilar entities
within the embedding space. A theoretical analysis elucidates the relationship
between this calibrated uniformity loss and the conventional uniformity
function. Moreover, to enhance the fusion of multimodal features, we introduce
a Spherical B\'ezier method designed to integrate an arbitrary number of
modalities while ensuring that the resulting fused features are constrained to
the same hyperspherical manifold. Empirical evaluations conducted on five
real-world datasets substantiate the superiority of our approach over competing
baselines. We also shown that the proposed methods can achieve up to a 5.4%
increase in NDCG@20 performance via the integration of MLLM-extracted features.
Source code is available at: https://github.com/enoche/CM3.