# Tracing Desired, Exposed and Adopted Algorithmic Novelty on Music Streaming Platforms

Dougal Shakespeare, Camille Roth

<figure><img src="image_1.png" /></figure>

## To cite this version:

Dougal Shakespeare, Camille Roth. Tracing Desired, Exposed and Adopted Algorithmic Novelty on Music Streaming Platforms. 2026. hal-05461786

HAL Id: hal-05461786

https://hal.science/hal-05461786v1

Preprint submitted on 16 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Tracing Desired, Exposed and Adopted Algorithmic Novelty on Music Streaming Platforms

Dougal Shakespeare

Camille Roth

CAMS (Centre d'analyse et de mathématique sociales), CNRS & EHESS

Paris, France

dougal.shakespeare@ehess.fr, roth@ehess.fr

## Abstract

Almost a decade after the Filter Bubble's instantiation, concerns that algorithmic guidance may limit diversity remain as widespread as ever. Nonetheless, supporting empirical evidence has, over the years, weakened. Across some of the most widely adopted search engines and social media platforms, several works have found that users' own selections of algorithmic recommendations may principally be responsible for constraining the diversity of media which they end up consuming. Phrased differently, users may be in what literature has coined, "self-imposed filter bubbles". On music streaming platforms, similar concerns have frequently been raised over the role which algorithmic guidance plays in influencing users' consumption behaviours particularly in terms of novelty and diversity. Nonetheless, existing empirical literature geared at appraising these dynamics generally leverage solely consumption logs and thus fail to disentangle the impact of user choices from algorithmic decisions. Drawing on both behavioural and survey data from users of the French music streaming platform Deezer, we examine the relationship between the levels of algorithmic novelty users desire, the levels exposed to them and the levels ultimately selected to be consumed. In turn, our work demonstrates that despite commonly expressing a desire for balanced novelty, Deezer users oppose algorithmic novelty, selecting less novelty to consume than what is exposed to them algorithmically. At the same time, our work underscores the complexities of designing platform strategies geared at nudging users towards diversity.

## CCS Concepts

* Human-centered computing → Empirical studies in HCI; 
* Social and professional topics → User characteristics; 
* Information systems → Multimedia streaming.

## Keywords

musical recommendation systems, exposure and selection, filter bubble, consumption diversity, user behaviour

## 1 Introduction

To disentangle the role of algorithmic guidance in shaping users' online media consumption, it is essential to study not only what users select, but also what they are exposed to and thus, choose not to consume. In Political Science, research across multiple platforms has shown that while algorithms often propose diverse, cross-cutting media, users tend to select in ways that reduce that diversity [Bakshy et al. 2015; Robertson et al. 2023]. This has led to a more nuanced view of filter bubble dynamics [Borgesius et al. 2016;

Pariser 2011] - one that emphasises the agency of users within the human-machine feedback loops ubiquitous across modern day platforms [Brinkmann et al. 2023]. In recent years, some scholars have even begun to refer to such dynamics as *self-imposed filter bubble dynamics* [Ekström et al. 2022]. Nevertheless, this strand of literature commonly speaks of average effects and thus, it remains largely unclear how this behaviour can vary for different types of users or even under different algorithmic behaviours (e.g. when recommending different levels or types of diversity).

In the case of music streaming platforms, similar concerns have also arisen around how platforms and the algorithmic recommendation devices they contain influence music listening, particularly in terms of diversity, novelty, and the volume of music consumed [Hesmondhalgh et al. 2023]. Yet, quantitative studies in this space typically rely on selection logs alone. For example, algorithmic concentration is often inferred by comparing diversity in consumption data from algorithmic affordances (like Spotify's Discover Weekly or Deezer's Flow) with that from less algorithmic ones (such as personal playlists or editorial curations) [Anderson et al. 2020; Mok et al. 2022; Shakespeare et al. 2025; Villermet et al. 2021]. While such research reveals how algorithmic devices are used in practice, it says little about the extent to which the observed diversity levels stem from algorithmic or, rather, user decisions. Users rarely consume recommendations blindly; they actively appreciate and choose what to consume based on a range of factors such as their openness [Anderson et al. 2021], goals [Garcia-Gathright et al. 2018], and algorithmic literacy [Gran et al. 2021]. Without access to exposure or non-selection logs, such user-driven patterns remain obscured, limiting our understanding of how algorithmic guidance functions and influences user behaviour in-practice.

The contribution of this work addresses this limitation by examining empirically the degree to which users of the French music streaming platform Deezer select different levels of novelty to that which algorithmic devices expose. Developing upon the methodology of existing literature from Political Science, our work provides a nuanced account of how this dynamic varies depending on both the type of users considered and the behaviour of algorithmic recommendation devices. Focusing on one of the most widely adopted algorithmic devices on-platform, Flow, our work traces the relationship between the different levels of algorithmically recommended novelty desired by users, exposed to them and then finally, selected to be consumed by them. In turn, our work reveals a striking paradox: users select less novelty than what is recommended to them algorithmically even when they express desiring more.
# 2 Related Work

In the following section, we review the literature most relevant to this work. We begin by discussing empirical studies on self-imposed filter bubbles, which have inspired the present research. We then situate our work within the broader literature on filter bubbles and music streaming. Finally, we review research on user rejection behaviours (i.e., skipping) on music streaming platforms - one of the main signals we use to disentangle differences between the kinds of information exposed by algorithms and selected by users.

## 2.1 Self-imposed Filter Bubbles

Empirical works which provide evidence for self-imposed filter bubbles tend to stem from the field of Political Science and are generally conducted by either platform employees or researchers working closely with platforms. Aside from some recent efforts more geared towards capturing users' attention (e.g. via eye tracking) rather than selection behaviours [Michiels et al. 2022], the majority of these works rely on empirically comparing the recommendations which algorithmic guidance *exposes* to users with that which they *select* to uncover differences in diversity [Bakshy et al. 2015; Robertson et al. 2023]. This is also the approach taken in this work. Existing empirical research suggests that perhaps surprisingly, on both search engines [Robertson et al. 2023], social media platforms [Aiello and Barbieri 2017; Bakshy et al. 2015] and digital news outlets [Haim et al. 2018] confinement in users' consumptions may emerge mainly from users' self-imposed so-called *organic* [Roth 2019] rather than *algorithmic* personalisation processes. These results are commonly interpreted as providing empirical support for the selective exposure hypothesis, a core concept in communication and media studies which describes the propensity for individuals to consume media that reinforces their prior beliefs [Katz et al. 2017].

## 2.2 Filter Bubbles & Music Streaming

Driven by growing concerns about artist fairness [Ferraro et al. 2021] and how new technologies may reinforce or disrupt social structures and positions [Airoldi 2021], an expanding body of interdisciplinary research has sought to understand the role which streaming platforms and algorithmic devices play in shaping and supporting diverse music listening practices [Hesmondhalgh et al. 2023]. In recent years, a novel strand of this literature has emerged, which aims to understand the outcome of algorithmic adoption on listening diversity by benchmarking it against listening diversity under so-called *organic* affordances which rely less on *programmed* guidance (e.g. personal playlists, the search bar, ect.) [Anderson et al. 2020; Mok et al. 2022; Shakespeare et al. 2025; Villermet et al. 2021]. Findings in this area suggest that, across several of the most widely adopted music streaming platforms, users' adoptions of algorithmic devices may bring about less semantic diversity (understood as spread in a dense, high dimensional geometric item embedding space) [Anderson et al. 2020; Shakespeare et al. 2025] but more novelty [Shakespeare et al. 2025; Villermet et al. 2021] than what they attain organically. These results complicate and nuance the conventional filter bubble narrative [Borgesius et al. 2016; Pariser 2011], indicating that in practice, the adoption of algorithmic guidance both broadens and narrows a user's listening experiences depending on the item representation method chosen [Mok et al.

2022; Shakespeare et al. 2025]. Nonetheless, a key limitation of this branch of literature is that the sets of recommendations algorithmically exposed to users are not usually analysed in parallel. As such, it remains unclear if these differences in diversity stem principally from algorithmic or user decisions.

## 2.3 Skipping Behaviours in Music Streaming

Whilst algorithmic exposure signals have rarely been leveraged to assess diversity-constraining dynamics on music streaming platforms, one particular subset has received significant attention within the Music Information Retrieval community, skips [Lamere 2014]. Research on skipping can be broadly conceptualised under two fractions depending on whether it tries to understand skipping behaviours in terms of variations of song features (e.g. audio fluctuations, changes in loudness etc.) [Donier 2020; Lamere 2014; Montecchio et al. 2020] or user features (e.g. platform behaviours, demographics, etc.) [Anderson et al. 2021; Deshmane et al. 2022; Meggetto et al. 2021, 2023].

Regarding the former, this type of research attempts to shed light on the typical locations within a song where skips occur and to identify any internal features of the song that may be related to skips. As early as 2014, a preliminary analysis of skipping behaviours on the Spotify platform [Lamere 2014] demonstrated that the majority of skips occur within the first 20 seconds of listening to a song and furthermore, that song skips follow a so-called U-shaped distribution which describes the propensity for songs to be skipped at the beginning and to a lesser extent, end of a track. Similarly, Sguerra et al. [2022] show (although do not explicitly mention) that on the Deezer platform, this U-shape distribution relates not just to the stream time of the song but also to the number of times in which the song is repeatedly exposed to the user. Montecchio et al. [2020] show how the skipping distribution of a song closely relates to its musical structure which leads the authors to suggest that it may also be seen as an intrinsic property of the song itself. In a later work, Donier [2020] explore this concept more formally with a kernel-based approach to model skip intensity. In turn, they demonstrate that so-called "salient musical events" mostly corresponding to within-song transitions are good indicators of skipping spikes. Furthermore, their model tends to be generally stable across songs with the exception of those tagged as classical or jazz which are noted to attract markedly lower skip rates in comparison to songs belonging to other genres. Similarly, Ng and Mehrotra [2020] show how audio state fluctuations at the session level are also associated with an increased probability of users skipping.

Research on skipping which places the user rather than item at the forefront - the approach also undertaken in our work - is, by contrast to the previously detailed research strand, less common. Meggetto et al. [2021] focus on characterising the sequential skip patterns of first sessions and then later, users. Their work shows how a user's skip typology varies alongside their session length, time of day of platform usage and platform affordance (e.g., algorithmic recommendation, personal playlists etc.) adopted to reach content. In a more recent follow-up work Meggetto et al. [2023] focus on further understanding *why* users reject songs on streaming platforms. By applying a deep reinforcement learning model to predict users' song skips, their work finds that the selection/rejection
<figure><img src="image_3.png" /><figcaption>**Figure 1:** Distribution of desired (a) vs actual exposed (b) A-novelty in Flow's recommendations. Red lines on Figure denote tercile bounds which separate users into r, m and v classes from left to right.</figcaption></figure>

status of previously consumed item is the feature which is most strongly correlated with users' skipping behaviours. Literature has also indicated that skipping may also be strongly associated with users' personality types. Anderson et al. [2021] find that on the Spotify platform, emotionally stable users tend to have lower skip rates.

## 3 Dataset

We work with the rich combination of survey data and granular platform interaction logs from a cohort of users from the French music streaming platform, Deezer. On the platform interaction side, we focus primarily on analysing interactions with one of Deezer's most popular algorithmic devices, Flow. This device follows what has been coined a 'lean back formulation' [Meggetto et al. 2023] and is designed to act as an endless radio in which users can neither view nor select in advance future recommendations. As such, each next track is a surprise for users and the main way of progressing from one recommendation to the next is either by letting the current track finish or alternatively, to 'skip'. Flow offers a unique opportunity for studying differences between the media selected by and exposed to users since by design, it ensures that the union of streams and skips constitutes all algorithmic exposures. As such, this simplifies our method and ensures we do not have to approximate exposures from recommended item lists using more complex methods like eye tracking or position bias correction. We study 3 months of platform interaction logs (02/22, 03/22, 04/22) and only consider users who are active in the first and last week of our observation, have minimal drop-off (max 1 week) and have at least one monadic (i.e containing only Flow) session in 03/22 or 04/22. This yields a total of $n = 5250$ users. We process listening histories by splitting them into sessions of continuous engagement whereby a new session is defined when $≥ 20$ minutes passes between engagements. Interactions with tracks are labelled as skips if the stream duration is < 30 seconds and otherwise as streams. Each listening history is assigned an affordance, indicating how the track was accessed: algorithmically via one of the platform's recommendation device such as Flow (A), editorially through a Deezer curator's playlist (E), or organically by the user for example, through the search bar or their own personal playlists (O). Alongside our survey respondents' platform interaction logs, we also work with a random control dataset which is processed using the same data processing pipeline ($n = 4802$). We use this dataset to assert the generalisability of the conclusions we derive from processing respondents listening

histories. On average, survey respondents are slightly older (39.0 vs. 37.5 years) and heavier users of algorithmic devices (38.5% vs. 33.9% of streams) but otherwise are broadly representative of the Deezer population. For further details of the dataset's representativeness we refer readers to the Supplementary Materials.

On the survey side, we make use of a subset of the survey responses collected through the RECORDS research project <fnref n="1" /> between the period of February and June 2023 and previously utilised by Marey et al. [2024]. Whilst the full dataset has not yet been publicly released at the time of publication it is anticipated that this will be made open access as the project reaches a conclusion in 2025. We focus primarily on the questions concerning users' preferences for novelty in Flows recommendations, what we coin A-novelty from hereon out.

## 4 Desired A-novelty

We first focus on measuring how much novelty Deezer users would like Flow to expose to them algorithmically. To this end, we analyse users' responses to the following survey question: Q. 'What percentage of discoveries would you like to see in Flow?'. In the survey, users were presented with a slider movable in intervals of 10 from [0..100] to indicate what percentage of novelty they desire Flow to recommend. As depicted in Figure 1, users tend to be relatively consistent in the amount of A-novelty they prefer. Letting $p$ denote the proportion rather than percentage of desired novelty, we observe that on-average users prefer a balanced mix of novel and redundant recommendations ($\langle p \rangle = 0.523$, $\sigma = 0.219$). Nonetheless, whilst less common, there also exist a marked proportion of users who want Flow to recommend almost exclusively novelty or redundancy. Segmenting users into terciles based on their responses yields 3 types of users which we label based on their desire for redundant ($r, n = 1943$), mixed ($m, n = 1897$) or novel ($v, n = 1410$) recommendations. Examining what, aside from their expressed desires for A-novelty, makes each of these groups distinct we observe a significant difference in terms of how they use the platform.<fnref n="2" /> In particular, $m$ users rely less on A adoption ($\langle p_A \rangle = 0.363$) and lean more towards $O$ affordances ($\langle p_O \rangle = 0.554$). In contrast, $r$ users rely more heavily on A guidance ($\langle p_A \rangle = 0.404$) and less on $O$ ($\langle p_O \rangle = 0.515$) relative to other user groups. Interestingly, we observe no significant differences between the groups in terms of E adoption.

## 5 Exposed A-Novelty

We next examine the extent to which Flow offers users opportunities for novel consumptions in its recommendations and how this aligns with users' expressed desire of A-novelty. At this stage of our analysis, we focus exclusively on sessions that only make use of the Flow affordance in the period of 03/22 and 04/22. Whilst this might seem like a strict requirement, these kinds of sessions are actually surprisingly common. In our dataset, 66.17% of Flow streams and skips are enacted within sessions that only make use of the Flow affordance.

<footnote n="1">¹https://records.huma-num.fr/</footnote>

<footnote n="2">²Significance computed via two-way ANOVA with $\alpha = 0.05$. See Supplementary Material.</footnote>
<figure><img src="image_4.png" /><figcaption>Figure 2: Relation between Flow adoption and exposed Novelty. Figure (a) - shows within session adoption and (b) shows the relation to between-session adoption adoption.</figcaption></figure>

<figure><img src="image_5.png" /><figcaption>Figure 3: Coefficient values associated with increasing % A-novelty exposed by Flow on users’ exploitation behaviours. β₀ is extracted from fixed effect model, * marks significance at α = 0.05 and bars show 95% CI.</figcaption></figure>

We define a recommendation in a session, S as novel if it has not previously been streamed one month prior to the start of S. For each Flow session we compute the proportion of recommendations which were novel and then per user, average over all of their monadic Flow sessions to form one data point per user which we define as pnov. As shown in Figure 1, where we plot the distribution of these scores over all users, at an aggregate level, the amount of novelty users desire roughly aligns with what they receive (⟨pnov⟩ = 0.516). Nevertheless, users who desire the most or least A-novelty are not the ones who receive this in practice. Running an ANOVA we find no significant differences between r, m and v users in terms of the levels of A-novelty exposed to them (p = 0.923, see Supplementary Materials). As such, typically, v users tend to receive less novelty than they desire whilst r users tend to receive more.

Whilst the amount of exposed A-novelty does not vary alongside a user's desired A-novelty, it does however, vary alongside algorithmic engagement. As shown in Figure 2, exposed A-novelty increases alongside the length of sessions (r = 0.106***<fnref n="3" />). <fnref n="4" /> and exposed A-novelty decreases alongside the number of monadic Flow sessions a user has (r = -0.308***). This creates a polar opposite dynamic whereby increased Flow engagement within sessions tends to be associated with greater levels of exposed novelty whereas more frequent, perhaps more regular, enactments of Flow sessions are associated with reduced A-exposed novelty. These findings are inline with the work of Lathia et al. [2010] who show that activity and inter-event time are negatively related to temporal next-session A-novelty.

## 6 Adopted A-novelty

Having established that Deezer users are typically exposed to a balanced mix of novelty and redundancy when using Flow (even when it is not desired) we now turn our attention towards understanding the tendency for users to adopt A-novelty. To this end, we

measure the extent to which users bias their streams towards exploiting familiar recommendations over exploring novel ones, how this preference differs across r, m, and v user groups, and how it is influenced by both user and algorithmic behaviour. We understand the behaviour of biasing streams towards exploration as being the scenario in which, by skipping recommendations, users end up consuming more novelty than they otherwise would have. Likewise, the inverse holds regarding biasing streams towards exploitation.

**Method:** We segment each Flow session into 2 sub-lists, redundant and novel, and compute on each sub-list, the proportion of recommendations which are played (i.e. P) rather than skipped. Thus, each session has two data points, pP nov and pP red. These are combined over all users to form our dependent variable Y. For each data point we assign a binary identifier R which is set to 1 when the redundant part of the session is considered and 0 otherwise. We assign L to be the session length and pnov to be the proportion of novel recommendations exposed in the session. In addition, let pY define the proportion of recommendations (either relating to those that are redundant or novel) that are semantically similar to a user's recent consumptions. As detailed in the Supplementary Materials, this is computed as the proportion of recommendations which fall within the hypersphere constructed by a user's recent consumptions (i.e, in the past month) positioned in high-dimensional geometric item embedding space. The motivation for including this variable as a control is to understand if Deezer users' tendency to bias their streams towards exploration or exploitation can be explained more generally, by their preference for engaging within semantic similarity or dis-similarity. Using each of these variables, we then apply the following fixed effect model.

$$Y = \beta_0 + \beta_1 R + \beta_2 L + \beta_3 p_{nov} + \beta_4 p_{Y} + \beta_5 LR + \beta_6 p_{nov}R + \beta_7 p_{Y}R + \alpha + \epsilon \quad (1)$$

$\beta_0$ represents the intercept. $\beta_1$ captures a user's tendency to bias their selections (i.e. streams) more towards exploration ($\beta_1 < 0$) or exploitation ($\beta_1 > 0$) when using Flow which more precisely, is characterised by the change in streaming probability when redundant as opposed to novel tracks are recommended. $\beta_2$ and $\beta_3$

<footnote n="3">³*, ** and *** used throughout to denote significance at p < 0.05, 0.01 and 0.001 respectively.</footnote>

<footnote n="4">⁴Interestingly, it appears that perhaps by design, first recommendations are more redundant than others.</footnote>
capture the effects of user and algorithmic behaviour in particu-
lar, the change in streaming probability associated with a 1 unit
increases in session length and exposed novelty respectively. The
former variable and coefficient ($\beta_2$) is incorporated into our model
based on an understanding that music streaming users streams
lean more into exploitation with increased algorithmic engagement
[Villermet et al. 2021] whilst the latter ($\beta_3$) is included based on
literature that has shown both online and offline, consumers' af-
fect for [Chmiel and Schubert 2017] and likelihood of streaming
[Sguerra et al. 2022] music commonly varies as a function of the
number of exposures (following an inverted U-shaped curve). $\beta_4$ is
associated with $p_{\gamma}$ and captures the effect of a 1 unit increase in
the proportion of recommendations which are within the seman-
tic hypersphere (which defines a user's recent consumptions) on
stream rate. $\beta_5$, $\beta_6$ and $\beta_7$ are associated with the interaction terms
which capture the effect which changes in $L$, $p_{nov}$ and $p_{\gamma}$ can have
on users biasing their streams towards exploration or exploitation.
$\alpha$ captures the user level fixed effects. Standard errors are clustered.
Full results tables are detailed in the Supplementary Materials. We
apply this model on each of user classes ($r$, $m$, $v$) to study variations
in each of these dynamics which occur alongside users' declared
preference for exposed A-novelty.

**Results:** Even after controlling for the semantic similarity of recommendations we observe that, much like on social media platforms where users have been shown to select less diversity than they are exposed to [Bakshy et al. 2015], likewise music streaming users select less novelty than is exposed to them algorithmically. More precisely, we find that users tend to prefer biasing their streams towards exploiting rather than exploring when using Flow. Put simply, redundant recommendations are significantly more likely to be streamed than novel ones ($\beta_1 = 0.0970^{***}$). Examining variations between $r$, $m$ and $v$ users we find that surprisingly, this behaviour is largely independent of a user's desires for exposed A-novelty. Even when users declare wanting to see more A-novelty than redundancy, they still bias their streams towards recommendations that are redundant.

Regarding how this behaviour varies alongside the levels of
and types of diversity recommended to users, we observe some
highly non-trivial dynamics. When Deezer users are recommended
a greater proportion of novelty in Flow sessions, they tend to lean
more into exploitation ($\beta_6 = 0.0573^{***}$) and furthermore, this effect
varies strongly alongside a user's openness to exposed novelty. As
depicted in Figure 3, when *r* users are exposed to higher levels of
novelty within sessions, they tend to lean further into exploita-
tion ($\beta_{6r} = 0.0763^{***}$), increasingly favouring familiar tracks over
novel ones whereas this dynamic is markedly less for *m* users
($\beta_{6m} = 0.0601^{***}$) and even less for *v* users ($\beta_{6v} = 0.0261$). Fur-
thermore, whilst Deezer users preference for exploitation can not
solely be explained by them favouring recommendations that are
semantically similar over those that are not, the semantic simi-
larity of algorithmic recommendations can affect how the degree
to which users lean towards exploration or exploitation. Deezer
users tend to be more open to using Flow for exploration when
recommendations are less diverse and thus, align more closely with
a user's past consumptions ($\beta_7 = -0.0351^{***}$). Regarding varia-
tions across user groups, we observe similar relative difference

to before (when $p_{nov}$ was considered). The less users desire ex-
posed A-novelty the more susceptible they are to this dynamic
($\beta_{7v} = -0.0235^{**}$, $\beta_{7m} = -0.0332^{***}$, $\beta_{7d} = -0.0441^{***}$). Put differ-
ently, as both the dense (semantic) and discrete (novel) diversity
of recommendations increases, users lean more into biasing their
streams towards exploitation.

Regarding algorithmic engagement, we observe that longer ses-
sions tend to attract significantly lower stream rates than shorter
ones ($\beta_2 = -0.0018^{***}$) even after controlling for exposed $p_{nov}$
which as we previously demonstrated, also increases alongside ses-
sion length. Several short, bursty sessions likely create more novelty
for users than prolonged sessions which amount to the same overall
length. Nevertheless, that does not mean that users become more
exploitative with longer sessions - we observe no significant effect
associated with the LR interaction term ($p = 0.1191$).

Overall, these findings clearly highlight Deezer users' tendency
to favour familiarity over novelty when using algorithmic recom-
mendation devices which allow for both. Users generally choose
to consume less novelty than they are exposed to irrespective of
how much novelty they express wanting. At the same time, these
findings also underscore the complexity of designing platform-side
interventions geared at fostering more diverse consumptions prac-
tices. As we show, increasing the levels of both dense (geometric,
semantic) and discrete (novelty-based) diversity in algorithmic rec-
ommendations can cause users to rather surprisingly, bias their
streams even further towards exploitation. These results align with
existing literature which has demonstrated that increasing the di-
versity of algorithmic music recommendations does not necessarily
cause users to consume more diversely [Chen et al. 2013] and lit-
erature that suggest that in context of online music consumption
behaviours, preference for dense and discrete consumption diver-
sity may be orthogonal concepts [Mok et al. 2022; Shakespeare et al.
2025].

7 A-novelty → O-novelty

When engaging with algorithmic guidance that permits varied us-
age styles, Deezer users tend to lean towards exploitation. But what
happens when users *do* adopt A-novelty? What knock-on effects
can this have on their subsequent organic platform interactions?
For example, a user might discover a new artist or genre through
a Flow recommendation and then organically explore the artist's
catalogue within the same session, leading to a sudden increase
in novelty within their organic streams. In the following section,
we focus on understanding what are the short term within-session
effects of streaming A-novelty upon the novelty of users' future
organic interactions? To this end, we employ a fuzzy difference-in
differences (DID) model with propensity score matching.

**Data Processing:** To generate a dataset which is suitable for this form of analysis we filter skips and collect all sessions in 03-04/22 that contained a sequence of consecutive organic streams, followed by Flow streams followed again by organic streams. This results in 3776 sessions which were enacted by a total of 960 users. We split each session into sub-sessions of consistent affordance usage, $O_1$, $A$, $O_2$. $A$ acts as the treatment and the period of $O_1$ and $O_2$ captures pre and post treatment time blocks respectively. For each sub-session we compute the proportion of streams that were
<figure><img src="image_6.png" /><figcaption>Figure 4: Estimated treatment effects computed from DID applied to each user group. * marks significance ($\alpha$ < 0.05).</figcaption></figure>

novel, stack these per user and then take column-wise means to produce three data point per user, ($p_{nov|O_1}$, $p_{nov|A}$, $p_{nov|O_2}$). We consider a session as treated when $p_{nov|A} > \langle p_{nov|A} \rangle$. Since we deal with thresholds rather than a control which never received any treatment, our DID is fuzzy and the effect we identifying is for users whose adoption behaviour changes around the threshold (i.e. the average $p_{nov|A}$).

DID estimation relies on the parallel trends assumption, which assumes that in the absence of treatment, the treatment and control groups would have exhibited similar trends over time. While our data suggest that this assumption holds in the pre-treatment period (see Supplementary Materials), it remains fundamentally untestable, as it concerns an unobservable counterfactual scenario. To enhance the credibility of our analysis, we additionally implement propensity score matching (PSM) prior to applying the DID estimator. By matching treated and control observations on relevant covariates, PSM increases the balance between groups, thereby improving the plausibility of the parallel trends assumption and reducing potential bias in the estimated treatment effects.

**Propensity Score Matching:** We match treated and control units on both user-level and session-level features to improve covariate balance prior to applying DID. At the user level, we include algorithmic adoption rates ($p_A$) and also their registration year which captures how users make use of the platform and how long they have been on the platform respectively. At the session level, we match on the average session length (i.e. per user) and the average organic novelty at session start ($p_{nov|O_1}$). These variables collectively form our covariate matrix $X$, which we use to estimate propensity scores via the following logistic regression:

$$G \sim \operatorname{logit}(X) \qquad (2)$$

where $G$ is a binary indicator equal to 1 if the treatment is applied, and 0 otherwise. We then perform nearest neighbour matching with a caliper of 0.1 to pair treated units with similar control units.

Post-matching diagnostics (see Supplementary Materials) indicate that covariate balance between the groups is substantially improved and all standardized mean differences are < 0.25, the threshold commonly applied for two groups to be considered balanced [Stuart 2010].

**DID:** After matching, we then apply our DID model which can be expressed as:

$$Y = \beta_0 + \beta_1 T + \beta_2 G + \beta_3 GT + \epsilon \qquad (3)$$

$Y$ represents the outcome ($p_{nov|O_1}$), $T$ is binary indicator variable which is 0 before intervention and 1 after. $\beta_3$ which is associated with the interaction term captures the LATE. We apply PSM and DID at both an aggregate level and also split per $r$, $m$, $v$ user groups.

As depicted in Figure 4, we observe that consuming A-diversity has a significant positive effect on users' organic diversity ($\beta = 0.122^{***}$) indicating that organic knock-on effects can and do happen. Nonetheless, the effects are not equal for all user types. Examining differences in effect sizes between $r$, $m$ and $v$ users we observe that surprisingly, the users who benefit the most from A-novelty are $r$ users who in principle, oppose A-novelty the most ($\beta = 0.148^{**}$). At the other end of the spectrum, for $v$ users, it appears that on the whole, their organic novelty would have been similar even in the absence of consuming algorithmic novelty ($\beta = 0.051$). This appears to indicate that although $r$ users oppose A-novelty, it can, when it aligns with their taste (i.e. since it was streamed), have a greater impact on keeping their organic consumptions novel.

## 8 Discussion

Familiarity plays a crucial role in moulding our relationship with music, influencing both our preferences and subsequent listening behaviours. Repeated exposure to certain songs or genres can deepen our appreciation [Chmiel and Schubert 2017], as familiarity often brings comfort, predictability, or even nostalgia, which can be pleasant for a listener. However, excessive repetition of favourite songs can limit opportunities for novel music discoveries that, though uncertain, might ultimately be more rewarding. Such forms the classical exploitation, exploration trade-off common across both algorithmic optimisation problems and also ubiquitous in user decision making processes. Striking the right balance between listening to familiar favourites or rather, taking gambles on new discoveries is thus essential for a dynamic and enriching listening experience.

Modern-day so-called lean-back algorithmic devices, such as Deezer's Flow are designed to respect this balance, promoting a mix of novelty and redundancy in their song recommendations. The contribution of this work explores whether this balance is likewise respected by users in their engagement with Flow's recommendations. Comparing declared survey response data with behavioural platform interaction data, our work traces the alignment between the levels of A-novelty users desire, algorithms expose and then finally, users choose to consume. Our work demonstrates that in-practice, Deezer users generally favour redundancy over novelty in algorithmic recommendations. In line with recent empirical findings from Political Science [Aiello and Barbieri 2017; Bakshy et al. 2015; Haim et al. 2018; Robertson et al. 2023], we find that users select less novelty than they are recommended, even amongst users who express wanting more A-novelty. At the same time, our work underscores the complexity of designing platform intervention strategies geared at nudging users towards more diverse consumption practices. Rather conversely, Deezer users react to more novelty by leaning more into exploitation and furthermore, tend to be more explorative when recommendations are less semantically diverse and to use Pariser's metaphor, within their bubble [Pariser 2011].
Put differently, users react to increasing dense and discrete diversity in algorithmic recommendations by biasing their streams more towards exploitation. Nevertheless, when A-novelty aligns with users' tastes (i.e. when it is streamed) our work finds that it has a positive effect on users' organically driven listening novelty even at the within-session scale and surprisingly, for users who oppose it in principal. Thus, users who oppose A-novelty the most may in fact be the ones who also benefit from it the most.

Taken together, these findings not only offer a comprehensive account of how human selections of algorithmic recommendations can limit diversity (specifically in terms of discrete novelty) thereby challenging dominant techno-deterministic narratives, but also underscore the nuanced challenges of increasing novelty in recommendations across different user groups, offering valuable insights for recommender system design moving forward.

## References

Luca Maria Aiello and Nicola Barbieri. 2017. Evolution of Ego-networks in Social Media with Link Recommendations. CoRR abs/1702.01398 (2017). arXiv:1702.01398 http://arxiv.org/abs/1702.01398

Massimo Airoldi. 2021. Machine habitus: Toward a sociology of algorithms. John Wiley & Sons.

Ashton Anderson, Lucas Maystre, Ian Anderson, Rishabh Mehrotra, and Mounia Lalmas. 2020. Algorithmic effects on the diversity of consumption on Spotify. In Proceedings of the web conference 2020. 2155–2165.

Ian Anderson, Santiago Gil, Clay Gibson, Scott Wolf, Will Shapiro, Oguz Semerci, and David M Greenberg. 2021. “Just the way you are”: Linking music listening on Spotify and personality. Social Psychological and Personality Science 12, 4 (2021), 561–572.

Eytan Bakshy, Solomon Messing, and Lada A. Adamic. 2015. Exposure to ideologically diverse news and opinion on Facebook. Science 348, 6239 (2015), 1130–1132. https://doi.org/10.1126/science.aaa1160 arXiv:https://www.science.org/doi/pdf/10.1126/science.aaa1160

Frederik Borgesius, Damian Trilling, Judith Moeller, Balazs Bodo, Claes de Vreese, Natali Helberger, and Internet Review. 2016. Should we worry about filter bubbles? Internet Policy Review Volume 5 (03 2016). https://doi.org/10.14763/2016.1.401

Levin Brinkmann, Fabian Baumann, Jean-François Bonnefon, Maxime Derex, Thomas F Müller, Anne-Marie Nussberger, Agnieszka Czaplicka, Alberto Acerbi, Thomas L Griffiths, Joseph Henrich, et al. 2023. Machine culture. Nature Human Behaviour 7, 11 (2023), 1855–1868.

Li Chen, Wen Wu, and Liang He. 2013. How personality influences users' needs for recommendation diversity? In CHI '13 Extended Abstracts on Human Factors in Computing Systems (Paris, France) (CHI EA '13). Association for Computing Machinery, New York, NY, USA, 829–834. https://doi.org/10.1145/2468356.2468505

Anthony Chmiel and Emery Schubert. 2017. Back to the inverted-U for music preference: A review of the literature. Psychology of Music 45, 6 (2017), 886–909.

Abhishek Deshmane, Noah Askin, and Khwan Kim. 2022. Keep It or Skip It? Sequential Consumption of Music with Reference Effects. (2022).

Jonathan Donier. 2020. The universality of skipping behaviours on music streaming platforms. arXiv preprint arXiv:2005.06987 (2020).

Axel G Ekström, Diederick C Niehorster, and Erik J Olsson. 2022. Self-imposed filter bubbles: Selective attention and exposure in online search. Computers in Human Behavior Reports 7 (2022), 100226.

Andres Ferraro, Xavier Serra, and Christine Bauer. 2021. What is fair? Exploring the artists' perspective on the fairness of music streaming platforms. In IFIP conference on human-computer interaction. Springer, 562–584.

Jean Garcia-Gathright, Brian St. Thomas, Christine Hosey, Zahra Nazari, and Fernando Diaz. 2018. Understanding and evaluating user satisfaction with music discovery. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 55–64.

Anne-Britt Gran, Peter Booth, and Taina Bucher. 2021. To be or not to be algorithm aware: a question of a new digital divide? Information, communication & society 24, 12 (2021), 1779–1796.

Mario Haim, Andreas Graefe, and Hans-Bernd Brosius. 2018. Burst of the filter bubble? Effects of personalization on the diversity of Google News. Digital journalism 6, 3 (2018), 330–343.

David Hesmondhalgh, Raquel Campos Valverde, D Kaye, and Zhongwei Li. 2023. The impact of algorithmically driven recommendation systems on music consumption and production: a literature review. UK Centre for Data Ethics and Innovation Reports (2023).

Elihu Katz, Paul F Lazarsfeld, and Elmo Roper. 2017. Personal influence: The part played by people in the flow of mass communications. Routledge.

Paul Lamere. 2014. The Skip. https://musicmachinery.com/2014/05/02/the-skip. Accessed: 2024-07-02.

Neal Lathia, Stephen Hailes, Licia Capra, and Xavier Amatriain. 2010. Temporal diversity in recommender systems. In Proceedings of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval (Geneva, Switzerland) (SIGIR '10). Association for Computing Machinery, New York, NY, USA, 210–217. https://doi.org/10.1145/1835449.1835486

Lilian Marey, Bruno Sguerra, and Manuel Moussallam. 2024. Modeling Activity-Driven Music Listening with PACE. In Proceedings of the 2024 Conference on Human Information Interaction and Retrieval. 346–351.

Francesco Meggetto, Crawford Revie, John Levine, and Yashar Moshfeghi. 2021. On skipping behaviour types in music streaming sessions. In Proceedings of the 30th ACM international conference on information & knowledge management. 3333–3337.

Francesco Meggetto, Crawford Revie, John Levine, and Yashar Moshfeghi. 2023. Why people skip music? On predicting music skips using deep reinforcement learning. In Proceedings of the 2023 Conference on Human Information Interaction and Retrieval. 95–106.

Lien Michiels, Jens Leysen, Annelien Smets, and Bart Goethals. 2022. What are filter bubbles really? A review of the conceptual and empirical work. In Adjunct proceedings of the 30th ACM conference on user modeling, adaptation and personalization. 274–279.

Lillio Mok, Samuel F Way, Lucas Maystre, and Ashton Anderson. 2022. The dynamics of exploration on Spotify. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 16. 663–674.

Nicola Montecchio, Pierre Roy, and François Pachet. 2020. The skipping behavior of users of music streaming services and its relation to musical structure. Plos one 15, 9 (2020), e0239418.

Aaron Ng and Rishabh Mehrotra. 2020. Investigating the impact of audio states & transitions for track sequencing in music streaming sessions. In Proceedings of the 14th ACM conference on recommender systems. 697–702.

Eli. Pariser. 2011. The filter bubble: what the Internet is hiding from you. Penguin Press, New York. - pages. http://www.worldcat.org/search?qt=worldcat_org_all&q=1594203008

Ronald E Robertson, Jon Green, Damian J Ruck, Katherine Ognyanova, Christo Wilson, and David Lazer. 2023. Users choose to engage with more partisan news than they are exposed to on Google Search. Nature (2023), 1–7.

Camille Roth. 2019. Algorithmic Distortion of Informational Landscapes. Intellectualia 70, 1 (2019), 97–118.

Bruno Sguerra, Viet-Anh Tran, and Romain Hennequin. 2022. Discovery dynamics: Leveraging repeated exposure for user and music characterization. In Proceedings of the 16th ACM Conference on Recommender Systems. 556–561.

Dougal Shakespeare, Victor Chareyron, and Camille Roth. 2025. Reframing the filter bubble through diverse scale effects in online music consumption. Scientific Reports 15, 1 (2025), 4071.

Elizabeth A Stuart. 2010. Matching methods for causal inference: A review and a look forward. Statistical science: a review journal of the Institute of Mathematical Statistics 25, 1 (2010), 1.

Quentin Villermet, Jérémie Poiroux, Manuel Moussallam, Thomas Louail, and Camille Roth. 2021. Follow the guides: disentangling human and algorithmic curation in online music consumption. In Proceedings of the 15th ACM Conference on Recommender Systems. 380–389.
# 9 Supplementary Material

## 9.1 Dataset Representativeness

<table><caption>Table 1: Descriptive statistics for respondents and control group. Mean shown with standard deviation in parentheses.</caption><thead><tr><th>Variable</th><th>Respondents</th><th>Control</th></tr></thead><tbody><tr><td>Age</td><td>38.96 (12.12)</td><td>37.50 (10.84)</td></tr><tr><td>p<sub>A</sub></td><td>0.38 (0.25)</td><td>0.34 (0.26)</td></tr><tr><td>p<sub>O</sub></td><td>0.53 (0.26)</td><td>0.56 (0.27)</td></tr><tr><td>p<sub>E</sub></td><td>0.08 (0.11)</td><td>0.10 (0.14)</td></tr><tr><td>registration year</td><td>2013.85 (4.29)</td><td>2014.39 (4.13)</td></tr></tbody></table>

## 9.2 Exposed A-novelty

2-way ANOVA to check for statistical differences in exposed novelty between *r*, *m* and *v* users.

<table><caption>Table 2: ANOVA results</caption><thead><tr><th></th><th>sum_sq</th><th>df</th><th>F</th><th>PR(&gt;F)</th></tr></thead><tbody><tr><td>C.cluster)</td><td>0.004639</td><td>2.0</td><td>0.079613</td><td>0.923475</td></tr><tr><td>Residual</td><td>152.873058</td><td>5247.0</td><td>-</td><td>-</td></tr></tbody></table>

## 9.3 Embedding Space Usage

<figure><img src="image_7.png" /><figcaption>Figure 5: t-SNE projection of the artist embedding space derived from Deezer users' playlists. For clarity, the visualization includes only artists belonging to the top 10 most popular genres in the dataset.</figcaption></figure>

### 9.3.1 Embedding Construction
As per. existing literature [Anderson et al. 2020; Mok et al. 2022], we generate a 128-dimensional artist embedding space by applying SVD to a PPMI normalised artist-artist co-occurrence matrix constructed from a random sample of Deezer users personal playlists (n = 100,000). This forms our embedding matrix, *E*. As depicted in Figure 5, semantically similar artists that belong to the same genre tend to be positioned closely in vector space. Since even before embedding, some artists may be too *niche* to gather enough co-occurrence information to accurately represent their true structure we choose to filter these artists prior to applying SVD. We define an artist as *niche* if it is featured in less than 10 unique playlists. When computing dense measurements of concentration we map recommended tracks to their associated first assigned artists. This decision was made to both limit the necessary compute power as well as ensure that we minimise the number of cases for which we have to filter tracks due to their *niceness*.
### 9.3.2 Computing the dense concentration of Flow recommendations:
We use this embedding to compute the dense concentration of Flow's recommendations. This is then used as a control within our fixed effect model. To this end, let the weighted centroid of a user's recent streams (i.e. in the past month) be defined as $\mu$. We define a Flow recommendation as densely concentrated when it exists within the hypersphere defined by a user's past recent consumption history, $H$, which are spread from $\mu$ in vector space. Put differently, a Flow recommendation, $r$ is deemed to be densely concentrated ($\gamma$) if $cos(E_r, \mu) \le GS(\mu, H)$ where GS represents the generalist-specialist score as defined by [Anderson et al. 2020; Mok et al. 2022] and cos represents cosine distance. Within our fixed effect model we compute the proportion of items which are concentrated under this dense formulation which we define as $p_Y$. This is computed for the sub-list of recommendations that are both redundant and novel within a given Flow session.

## 9.4 Adopted A-novelty

Results tables from fixed effect models applied to each user group.

### (a) User group: Control

<table><thead><tr><th></th><th>β</th><th>Std. Err.</th><th>T-stat</th><th>P-value</th><th>Lower CI</th><th>Upper CI</th></tr></thead><tbody><tr><td>R</td><td>0.1037</td><td>0.0055</td><td>18.965</td><td>0.0000</td><td>0.0930</td><td>0.1144</td></tr><tr><td>L</td><td>-0.0019</td><td>0.0001</td><td>-18.299</td><td>0.0000</td><td>-0.0021</td><td>-0.0017</td></tr><tr><td>p<sub>nov</sub></td><td>-0.0514</td><td>0.0068</td><td>-7.6011</td><td>0.0000</td><td>-0.0646</td><td>-0.0381</td></tr><tr><td>p<sub>Y</sub></td><td>0.0528</td><td>0.0038</td><td>13.843</td><td>0.0000</td><td>0.0453</td><td>0.0603</td></tr><tr><td>R: L</td><td>7.374e-05</td><td>3.943e-05</td><td>1.8699</td><td>0.0615</td><td>-3.553e-06</td><td>0.0002</td></tr><tr><td>R: p<sub>nov</sub></td><td>0.0485</td><td>0.0073</td><td>6.6918</td><td>0.0000</td><td>0.0343</td><td>0.0628</td></tr><tr><td>R: p<sub>Y</sub></td><td>-0.0258</td><td>0.0048</td><td>-5.3745</td><td>0.0000</td><td>-0.0352</td><td>-0.0164</td></tr></tbody></table>

### (c) User group: r

<table><thead><tr><th></th><th>β</th><th>Std. Err.</th><th>T-stat</th><th>P-value</th><th>Lower CI</th><th>Upper CI</th></tr></thead><tbody><tr><td>R</td><td>0.1016</td><td>0.0084</td><td>12.126</td><td>0.0000</td><td>0.0852</td><td>0.1180</td></tr><tr><td>L</td><td>-0.0016</td><td>0.0001</td><td>-11.637</td><td>0.0000</td><td>-0.0018</td><td>-0.0013</td></tr><tr><td>p<sub>nov</sub></td><td>-0.0690</td><td>0.0096</td><td>-7.1825</td><td>0.0000</td><td>-0.0879</td><td>-0.0502</td></tr><tr><td>p<sub>Y</sub></td><td>0.0584</td><td>0.0055</td><td>10.702</td><td>0.0000</td><td>0.0477</td><td>0.0691</td></tr><tr><td>R: L</td><td>-3.335e-05</td><td>5.804e-05</td><td>-0.5746</td><td>0.5655</td><td>-0.0001</td><td>8.041e-05</td></tr><tr><td>R: p<sub>nov</sub></td><td>0.0763</td><td>0.0107</td><td>7.1089</td><td>0.0000</td><td>0.0552</td><td>0.0973</td></tr><tr><td>R: p<sub>Y</sub></td><td>-0.0441</td><td>0.0070</td><td>-6.2745</td><td>0.0000</td><td>-0.0579</td><td>-0.0304</td></tr></tbody></table>

### (b) User group: Respondents

<table><thead><tr><th></th><th>β</th><th>Std. Err.</th><th>T-stat</th><th>P-value</th><th>Lower CI</th><th>Upper CI</th></tr></thead><tbody><tr><td>R</td><td>0.0970</td><td>0.0052</td><td>18.629</td><td>0.0000</td><td>0.0868</td><td>0.1073</td></tr><tr><td>L</td><td>-0.0018</td><td>7.854e-05</td><td>-22.429</td><td>0.0000</td><td>-0.0019</td><td>-0.0016</td></tr><tr><td>p<sub>nov</sub></td><td>-0.0562</td><td>0.0058</td><td>-9.7102</td><td>0.0000</td><td>-0.0676</td><td>-0.0449</td></tr><tr><td>p<sub>Y</sub></td><td>0.0571</td><td>0.0034</td><td>16.767</td><td>0.0000</td><td>0.0504</td><td>0.0637</td></tr><tr><td>R: L</td><td>6.124e-05</td><td>3.929e-05</td><td>1.5586</td><td>0.1191</td><td>-1.577e-05</td><td>0.0001</td></tr><tr><td>R: p<sub>nov</sub></td><td>0.0573</td><td>0.0067</td><td>8.5649</td><td>0.0000</td><td>0.0442</td><td>0.0705</td></tr><tr><td>R: p<sub>Y</sub></td><td>-0.0351</td><td>0.0044</td><td>-7.9589</td><td>0.0000</td><td>-0.0437</td><td>-0.0264</td></tr></tbody></table>

### (d) User group: m

<table><thead><tr><th></th><th>β</th><th>Std. Err.</th><th>T-stat</th><th>P-value</th><th>Lower CI</th><th>Upper CI</th></tr></thead><tbody><tr><td>R</td><td>0.0966</td><td>0.0087</td><td>11.050</td><td>0.0000</td><td>0.0795</td><td>0.1138</td></tr><tr><td>L</td><td>-0.0018</td><td>0.0001</td><td>-15.046</td><td>0.0000</td><td>-0.0020</td><td>-0.0015</td></tr><tr><td>p<sub>nov</sub></td><td>-0.0539</td><td>0.0093</td><td>-5.8192</td><td>0.0000</td><td>-0.0721</td><td>-0.0358</td></tr><tr><td>p<sub>Y</sub></td><td>0.0534</td><td>0.0055</td><td>9.7610</td><td>0.0000</td><td>0.0427</td><td>0.0641</td></tr><tr><td>R: L</td><td>3.658e-05</td><td>6.820e-05</td><td>0.5364</td><td>0.5917</td><td>-9.709e-05</td><td>0.0002</td></tr><tr><td>R: p<sub>nov</sub></td><td>0.0601</td><td>0.0108</td><td>5.5560</td><td>0.0000</td><td>0.0389</td><td>0.0813</td></tr><tr><td>R: p<sub>Y</sub></td><td>-0.0332</td><td>0.0074</td><td>-4.4692</td><td>0.0000</td><td>-0.0477</td><td>-0.0186</td></tr></tbody></table>

### (e) User group: v

<table><caption>Table 3: Results from fixed effects models across user groups. 95% confidence intervals shown.</caption><thead><tr><th></th><th>β</th><th>Std. Err.</th><th>T-stat</th><th>P-value</th><th>Lower CI</th><th>Upper CI</th></tr></thead><tbody><tr><td>R</td><td>0.0899</td><td>0.0100</td><td>8.9733</td><td>0.0000</td><td>0.0702</td><td>0.1095</td></tr><tr><td>L</td><td>-0.0021</td><td>0.0001</td><td>-15.047</td><td>0.0000</td><td>-0.0023</td><td>-0.0018</td></tr><tr><td>p<sub>nov</sub></td><td>-0.0405</td><td>0.0114</td><td>-3.5473</td><td>0.0004</td><td>-0.0629</td><td>-0.0181</td></tr><tr><td>p<sub>Y</sub></td><td>0.0595</td><td>0.0071</td><td>8.4087</td><td>0.0000</td><td>0.0457</td><td>0.0734</td></tr><tr><td>R: L</td><td>0.0003</td><td>8.087e-05</td><td>3.1306</td><td>0.0017</td><td>9.467e-05</td><td>0.0004</td></tr><tr><td>R: p<sub>nov</sub></td><td>0.0261</td><td>0.0138</td><td>1.8960</td><td>0.0580</td><td>-0.0009</td><td>0.0531</td></tr><tr><td>R: p<sub>Y</sub></td><td>-0.0235</td><td>0.0085</td><td>-2.7461</td><td>0.0060</td><td>-0.0402</td><td>-0.0067</td></tr></tbody></table>

## 9.5 DID Application

### 9.5.1 Propensity Score Matching.
To assert the alignment of our matching we compute standardised mean differences for each user groups and variable we consider. As depicted in Table 4, these are < 0.25 which is usually the threshold applied for two groups to be considered balanced [Stuart 2010].

<table><caption>Table 4: Standardized mean differences between control and treatment groups after matching.</caption><thead><tr><th>Group</th><th>p<sub>nov</sub>|O<sub>1</sub></th><th>p<sub>A</sub></th><th>registration year</th><th>⟨L⟩</th></tr></thead><tbody><tr><td>all</td><td>-0.0163</td><td>-0.1306</td><td>-0.0451</td><td>-0.1134</td></tr><tr><td>r</td><td>0.0127</td><td>-0.2112</td><td>-0.0827</td><td>-0.0242</td></tr><tr><td>m</td><td>0.0803</td><td>-0.0349</td><td>0.0051</td><td>0.1136</td></tr><tr><td>v</td><td>0.1618</td><td>0.1395</td><td>0.0559</td><td>0.0413</td></tr></tbody></table>
9.5.2 *Parallel Trends.* To check for parallel trends we examine trends between treatment and control groups' organic novelty pre-treatment. To this end we fit the following model.

$$Y = \beta_0 + \beta_1 Gt + \beta_2 G + \beta_4 t + \epsilon \quad (4)$$

Where Y represents the pre-treatment novelty, G the indicator variable which is 1 when we consider the treatment and 0 otherwise and t represents the time step. Examining the significance of the coefficient associated with the interaction term (i.e. β₂) enables us to assert if we have reason to reject the parallel trends assumption pre-treatment. Since before aggregating, sessions have a variable number of O streams which lead up to Flow ones, we run our model several times on sessions of different lengths. As depicted in Figure 6, we filter to only consider sessions which have a pre-treatment number of streams then aggregate as before at the user scale before subsequently running our model. As depicted in Figure 6 where p-values are shown, we observe no significant effect associated with β₂ and thus, no reason to reject the parallel trends assumption pre-treatment.

<figure><img src="image_8.png" /><figcaption>Figure 6: Pre-treatment stream rates on different session lengths. Black lines display treatment group and grey lines the control. p-values on each subplot are extracted from the interaction term run on the linear model to check for parallel trends pre-treatment.</figcaption></figure>