# Diversified recommendations of cultural activities with personalized determinantal point processes

Carole Ibrahim, Hiba Bederina, Cuesta Daniel, Montier Laurent, Delabre

Cyrille, Jill-Jenn Vie

► To cite this version:

Carole Ibrahim, Hiba Bederina, Cuesta Daniel, Montier Laurent, Delabre Cyrille, et al. Diversified recommendations of cultural activities with personalized determinantal point processes. RecSoGood 2025 - Second International Workshop on Recommender Systems for Sustainability and Social Good, Sep 2025, Prague, Czech Republic. hal-05252385

HAL Id: hal-05252385

https://inria.hal.science/hal-05252385v1

Submitted on 12 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Diversified recommendations of cultural activities with personalized determinantal point processes

Carole Ibrahim¹[0009-0006-2148-5600], Hiba Bederina²[0009-0003-9926-4099], Daniel Cuesta¹[0009-0000-3445-990X], Laurent Montier¹[0009-0006-7294-8441], Cyrille Delabre¹[0009-0000-9739-5739], and Jill-Jenn Vie²[0000-0002-9304-2220]

¹ pass Culture

² Soda Team, Inria Saclay

**Abstract.** While optimizing recommendation systems for user engagement is a well-established practice, effectively diversifying recommendations without negatively impacting core business metrics remains a significant industry challenge. In line with our initiative to broaden our audience's cultural practices, this study investigates using personalized Determinantal Point Processes (DPPs) to sample diverse *and* relevant recommendations. We rely on a well-known quality-diversity decomposition of the similarity kernel to give more weight to user preferences. In this paper, we present our implementations of the personalized DPP sampling, evaluate the trade-offs between relevance and diversity through both offline and online metrics, and give insights for practitioners on their use in a production environment. For the sake of reproducibility, we release the full code for our platform and experiments on GitHub.

**Keywords:** Recommender systems · diversity · determinantal point processes.

## 1 Introduction

The *pass Culture* is a French government initiative launched in 2019 to promote cultural engagement among 3 million young individuals aged 15 to 20. Each user is awarded a fixed credit to spend on the application and access to a diverse array of millions of possible cultural activities (books, cinema, opera, etc.). A central objective of the program is to increase youth participation in cultural activities and to broaden their cultural horizons. While the Pass Culture has been widely adopted (84% of 18-year-olds registered by August 2024), a significant portion of the reservations are concentrated on books (especially manga initially, though its share has decreased, with literature now at 40% of book purchases) and cinema. Other forms of live performance (theater, dance, circus arts) and museums struggle to attract new audiences, accounting for a marginal share of reservations.

A study [7] suggests that the Pass Culture has only partially succeeded in diversifying cultural practices. While some users report discovering new activities,
a significant portion of reservations are for cultural activities they were already familiar with. There's a risk of the Pass primarily intensifying existing cultural practices rather than fostering new ones, particularly among young people who already have higher cultural capital. There is a recognized need to improve the "discoverability" of offers beyond books and recorded music, potentially through enhanced recommendation systems within the application.

To achieve this goal, we investigated the use of Determinantal Point Processes (DPPs) in the recommendation system to enhance diversity in users' recommendations while preserving their engagement. Our results show that DPPs can achieve diversity-utility trade-offs, as described in [8].

In this paper, we describe our approach to experimenting with a DPP-based diversity filter in a production-scale recommender system. We compare various configurations for incorporating user preferences into diversity sampling. In particular, our approach is stochastic: if the user refreshes the page, they get different recommendations. We evaluate the proposed methods in both offline and online settings.

The whole *pass Culture* app code is available under Mozilla Public License on GitHub<fnref n="3" />. Due to the complexity of our production environment, we provide a simplified demonstration of our implementation, available in this GitHub repository<fnref n="4" />.

## 2 Key performance indicators

The universe of all items in the catalog is denoted by $\Omega = \{1, \dots, N\}$. In our database, $N$ is in the order of millions.

For each item $i$, we denote its semantic embedding $\phi_i \in \mathbb{R}^d$, and we assume that we can compute a quality score $q_i \in [0, 1]$ corresponding to the probability of a certain user interacting with this item. This score may rely on a machine learning model trained on historical user interactions.

*Relevance* The foremost goal of a recommender system is to provide relevant recommendations. The relevance for a set $S \subset \Omega$ of recommended items is $\frac{\sum_{i \in S} q_i}{|S|}$. In online evaluation, it can correspond to a click-through rate. A model can be trained to predict $q_i$, to rank items by decreasing predicted relevance.

*Quality-diversity trade-off* We now describe how we model the quality diversity of a set $S$. We note $K$ the similarity kernel between items. In this paper, we focus on the linear kernel, as it simplifies the computation of decomposition: $K_{ij} \propto L_{ij} = q_i q_j \phi_i^T \phi_j$. The volume of a set of vectors $(\phi_i)_{i \in S}$ is $Vol(S) = \det((\phi_i^T \phi_j)_{i,j \in S})^{1/2}$. If we consider the kernel above, then

$$ \log \det K(S, S) = \sum_{i \in S} \log q_i + 2 \log Vol(S). \quad (1) $$

<footnote n="3">³ https://github.com/pass-culture</footnote>

<footnote n="4">⁴ https://github.com/pass-culture/dpp-paper-demo</footnote>
This decomposition illustrates the relevance-diversity trade-off of a set *S* of recommendations, which is of key importance in this paper.

*Diversification* Moreover, we introduce a business-oriented diversity metric, crucial for communicating usage statistics to our stakeholders, such as the French Ministry of Culture. The metric is defined as follows: each recommendation is assigned a score based on its novelty relative to the user's historical interactions, with a maximum score of 6.5 (2.5 points for a new category, 2 points for a new venue type, 1 point for a new subcategory, and 0.5 points for a new venue or genre). It quantifies the user's increase in coverage of the catalog [2] and is similar to the notion of serendipity encountered in some works [11].

# 3 Related work

Determinantal point processes are repulsive point processes that allow sampling points in vector spaces proportionally to the square of the volume formed by their embeddings [6], i.e., the determinant of their similarity kernel. They have successfully been used in retrieval and recommendations [8]. The original methods for sampling a subset among *N* items may have a complexity of *O*(N³) due to a kernel decomposition, but it can be reduced to *O*(Nd²) in the case of a linear kernel and embeddings of size *d*, and recent techniques have been proposed to cut the complexity to *O*(log *N*) after a preprocessing step of *O*(Nd²) [4] or *O*(α*N*) where α < 1 [1], which means, without looking at all items.

The closest setting to our approach is [8]. They claim they sample according to a DPP, but in practice, they take a fully *deterministic* approach that consists of computing the greedy max determinant by adding one item at a time. In particular, it means that for a new user, the first recommended set will always be the same, which can harm the diversity of collected data. We decided to adopt the original implementation of DPP, which is *stochastic*, as we assume this may have a positive impact on exploration. [5] also suggests that, thanks to its randomization, DPP sampling may achieve lower regret than the greedy max determinant because it allows more exploration.

# 4 Recommendation System Overview

We operate a real-time personalized recommendation engine for the pass Culture application. The system follows a multi-stage architecture consisting of three main stages as illustrated in Figure 1.

*Two-Tower Retrieval model* The retriever is a two-tower model [10, 9] from TensorFlow Recommenders<fnref n="5" /> trained on historical user–item interactions, optimizing recall of clicks. The model learns user and item embeddings using neural networks and stores them in a vector database. The probability of observing a click depends on the similarity between user and item embeddings. It retrieves 1,000 items.

<footnote n="5">⁵ https://www.tensorflow.org/recommenders</footnote>
<figure><img src="image_2.png" /><figcaption>**Fig. 1.** Overview of the recommender architecture with DPP integration.</figcaption></figure>

**DPP Diversity Filter** We implemented a Determinantal Point Process (DPP) module as a post-retrieval step, illustrated in orange in Figure 1 defined as in Equation 1. We used DPPy, an open-source Python library designed to facilitate the experimental use of DPPs in real-world applications [3]. Among the top 1,000 items retrieved using the two-tower model embeddings, the DPP module selects a subset of 60 diverse items using the `sample_exact_k_dpp` class from the `FiniteDPP` class of DPPy [3].

We compute item semantic embeddings $\phi_i' \in \mathbb{R}^{384}$ using the all-MiniLM-L6-v2 model$^6$, to embed item title and description. These embeddings are later reduced to $d = 64$ dimensions for latency constraints$^7$.

**Compliance Filtering & Popularity Ranking** The list of recommendations is filtered to remove any items that do not comply with business rules or user constraints. The resulting items are finally ranked by decreasing popularity.

# 5 Evaluation

We run offline and online evaluations on the three following recommender pipeline versions:

A recommender without the DPP filter (baseline);

B recommender with a DPP filter using personalized quality scores $q_i$ based on the two-tower model: $q_i$ represents the cosine similarity between user and item two-tower embeddings to encourage user preferences;

C recommender with a DPP filter using a constant value of $q_i = 1$ for all items, relying solely on diversity in semantic space to sample items among the top 1,000 retrieved ones.

We refer to those versions as models in the offline metrics and groups in the online metrics. We present up to 60 items to each user on their homepage. We evaluate the diversity of recommendations $S$ using the volume and the business diversity metric described in Section 2. To compute the volumes, we use the

<footnote>$^6$ https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2</footnote>

<footnote>$^7$ We observed empirically that embeddings $\phi_i$ of size 64 retain sufficient semantic variation to sample diverse items.</footnote>
non-reduced 384-dimensional item semantic embeddings. We evaluate relevance using the mean cosine similarity between recommended items and users in an offline setting and the click rate in an online setting.

## 5.1 Offline evaluation

For the offline evaluation, we sample 10,000 random users from our database, and we predict their recommendations using the three models, without the compliance filtering. The results are presented in Table 1.

<table><caption>**Table 1.** Offline results comparing the baseline (A) with DPP-based recommenders (B and C). Volume ratios are relative to Model A.</caption><thead><tr><th></th><th>Cosine similarity</th><th>Volume ratio</th><th>Business diversity metric</th></tr></thead><tbody><tr><td>Model A</td><td><strong>0.525</strong></td><td>1</td><td>2.759</td></tr><tr><td>Model B</td><td>0.399</td><td>×24.7</td><td><strong>3.404</strong></td></tr><tr><td>Model C</td><td>0.381</td><td>×28.8</td><td><strong>3.482</strong></td></tr></tbody></table>

We observe a notable drop in relevance of approximately 24% and 27% for versions B and C, respectively, compared to the baseline. However, this decline is accompanied by a substantial gain in diversity. Specifically, the business diversity metric increases by 23% and 26%, while the volume is multiplied by at least 24.7 when filtering with DPP. Although the reduction in relevance is notable, the substantial improvement in diversity offers compelling justification for deploying the DPP-based approach in a production setting, particularly given this paper's primary objective of enhancing diversity.

## 5.2 Online evaluation

We conducted a 10-day large-scale online A/B/C test comparing the baseline recommender (Group A with 137,188 users) with two DPP-based variants: personalized quality scores (Group B with 132,063 users) and constant quality scores (Group C with 131,458 users). The experiment was submitted for ethical approval to our institutional review board. Although explicit consent forms were impractical for this scale, all users had previously agreed to data usage for system improvement during account registration. The A/B/C test results are summarized in Table 2.

As anticipated from the offline results, the adoption of DPP resulted in a decrease in relevance: the click-through rate (CTR) dropped by approximately 37% from the baseline to Group B, and by about 46% from the baseline to Group C. Several factors may contribute to this discrepancy. First, CTR measures user behavior and differs fundamentally from mean cosine similarity, which is used as a proxy for relevance in offline evaluations. Secondly, the offline evaluation was conducted on a subset of 10,000 predominantly active users with multiple
<table><caption>**Table 2.** Online A/B/C test results. Values with * denote statistical significance ($p < 0.001$) versus Group A with the Mann-Whitney test.</caption><thead><tr><th></th><th>Click rate</th><th>Volume ratio</th><th>Business diversity metric</th></tr></thead><tbody><tr><td>Group A</td><td><strong>0.54%</strong></td><td>1</td><td>3.132</td></tr><tr><td>Group B</td><td>0.34%*</td><td>×12</td><td><strong>3.512*</strong></td></tr><tr><td>Group C</td><td>0.29%*</td><td>×15.8</td><td><strong>3.590*</strong></td></tr></tbody></table>

interactions in the logs, whereas the online evaluation includes the full user base, encompassing both active and less active users.

In contrast, diversity metrics improved significantly. The volume of distinct items recommended increased by a factor of 12 in Group B and 15.8 in Group C. Business diversity scores also saw meaningful gains, rising by +12.1% ($p < 0.001$) for Group B and +14.6% ($p < 0.001$) for Group C. These results highlight a trade-off between relevance and diversity, with the latter showing promising improvements under the DPP framework.

A particularly noteworthy finding is the disparity in click-through rate between the two DPP variants. Specifically, the CTR drops by an additional 14% when removing the personalized relevance scores ($q_i$). In contrast, the business diversity metric increases by only 2%.

## 6 Conclusion

Our findings highlight the potential of DPP sampling to effectively promote diversity in recommendations. Based on the A/B/C test results, this work positions DPP as a promising mechanism to foster the discovery of new cultural activities, albeit sometimes at the expense of short-term user engagement. Despite these promising results, the challenge of maintaining relevance alongside diversity remains an open issue. To address this, we plan to adjust the weight of the quality component in Equation 1, in order to mitigate relevance loss while still enhancing diversity. This could help identify a Pareto frontier that balances relevance and diversity, providing stakeholders with clear trade-off options to support decision-making.

**Acknowledgments.** This work was accomplished under a research agreement between Inria, the French Ministry of Culture, and the pass Culture SAS. We thank Clémence Réda for the interesting discussions and remarks on the paper. The authors have no competing interests to declare that are relevant to the content of this article.

## References

[1] Daniele Calandriello, Michal Derezinski, and Michal Valko. “Sampling from a k-DPP without looking at all items”. In: *Advances in Neural Information Processing Systems* 33 (2020), pp. 6889–6899.
[2] Erica Coppolillo, Giuseppe Manco, and Aristides Gionis. "Relevance meets Diversity: A User-Centric Framework for Knowledge Exploration through Recommendations". In: *Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining*. 2024, pp. 1-12.

[3] Guillaume Gautier et al. "DPPy: DPP Sampling with Python". In: *Journal of Machine Learning Research* 20.180 (2019), pp. 1-7. URL: http://jmlr.org/papers/v20/19-179.html.

[4] Jennifer Gillenwater et al. "A tree-based method for fast repeated sampling of determinantal point processes". In: *International Conference on Machine Learning*. PMLR. 2019, pp. 2260-2268.

[5] Tarun Kathuria, Amit Deshpande, and Pushmeet Kohli. "Batched Gaussian process bandit optimization via determinantal point processes". In: *Proceedings of the 30th International Conference on Neural Information Processing Systems*. 2016, pp. 4213-4221.

[6] Alex Kulesza, Ben Taskar, et al. "Determinantal point processes for machine learning". In: *Foundations and Trends in Machine Learning* 5.2-3 (2012), pp. 123-286. URL: https://arxiv.org/abs/1207.6083.

[7] Mandy Llamas. Mesurer et définir la diversification des pratiques culturelles : analyse des réservations des utilisateurs du pass Culture. URL: https://medium.com/passcultureofficiel/mesurer-et-d%C3%A9finir-la-diversification-des-pratiques-culturelles-analyse-des-r%C3%A9servations-des-69e6f87643f3.

[8] Mark Wilhelm et al. "Practical diversified recommendations on youtube with determinantal point processes". In: *Proceedings of the 27th ACM International Conference on Information and Knowledge Management*. 2018, pp. 2165-2173.

[9] Ji Yang et al. "Mixed negative sampling for learning two-tower neural networks in recommendations". In: *Companion proceedings of the web conference 2020*. 2020, pp. 441-447.

[10] Xinyang Yi et al. "Sampling-bias-corrected neural modeling for large corpus item recommendations". In: *Proceedings of the 13th ACM conference on recommender systems*. 2019, pp. 269-277.

[11] Shuo Zhang et al. "Auralist: Introducing serendipity into music recommendation". In: *Proceedings of the Fifth ACM International Conference on Web Search and Data Mining (WSDM)*. ACM. 2012, pp. 13-22. DOI: 10.1145/2124295.2124300.