# Privacy Settings and Ad Perception: The Shift from Third-Party Cookies to the Privacy Sandbox

Abir Benzaamia, Oana Goga

<figure><img src="image_1.png" /></figure>

## To cite this version:

Abir Benzaamia, Oana Goga. Privacy Settings and Ad Perception: The Shift from Third-Party Cookies to the Privacy Sandbox. CHI 2026 - Conference on Human Factors in Computing Systems, Apr 2026, Barcelone, Spain. 10.1145/3772318.3790444. hal-05508996

HAL Id: hal-05508996

https://hal.science/hal-05508996v1

Submitted on 13 Feb 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Privacy Settings and Ad Perception: The Shift from Third-Party Cookies to the Privacy Sandbox

Abir Benzaamia

Oana Goga

LIX, CNRS, Inria, École Polytechnique

LIX, CNRS, Inria, École Polytechnique

IP Paris

IP Paris

Palaiseau, France

Palaiseau, France

abir.benzaamia@inria.fr

oana.goga@inria.fr

## Abstract

Online behavioral advertising, heavily reliant on privacy-invasive third-party cookie tracking, faces a significant shift as browsers like Safari, Brave, and Firefox have already deprecated them. Google Chrome announced its parallel move with the “Privacy Sandbox Initiative” in 2019, proposing privacy-preserving advertising mechanisms. The extent to which Privacy Sandbox can deliver comparable ad effectiveness to the established third-party cookies system will likely determine its adoption as a widespread alternative. This paper presents the first user study evaluating the impact of Privacy Sandbox APIs on ad perception. Our findings show that users perceive Privacy Sandbox ads as less relevant and exhibit lower purchase intent compared to third-party cookie-based ads, without a corresponding increase in perceived privacy protection. These results contribute to the ongoing assessment of Privacy Sandbox as an alternative to third-party cookies.

## CCS Concepts

* Information systems → Online advertising; * Security and privacy → Usability in security and privacy.

## Keywords

Privacy Sandbox, Advertising, Third-party cookies, Tracking

## ACM Reference Format:

Abir Benzaamia and Oana Goga. 2026. Privacy Settings and Ad Perception: The Shift from Third-Party Cookies to the Privacy Sandbox. In *Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (CHI '26)*, April 13–17, 2026, Barcelona, Spain. ACM, New York, NY, USA, 17 pages. https://doi.org/10.1145/3772318.3790444

## 1 Introduction

Online behavioral advertising delivers targeted ads by tracking users' browsing behavior across multiple websites to infer their interests and preferences. This practice typically relies on third-party cookies, which collect data on users' online activities to build detailed behavioral profiles. However, the pervasive data collection enabled by third-party cookies raises significant privacy concerns—not only due to the personal data directly collected but also because of the sensitive inferences that can be drawn from it [15, 30]. As a result, third-party cookies have been deprecated

by different web browsers (Tor Browser [47], Safari WebKit [12], Brave Browser [69], and Mozilla Firefox [48]). Over the past five years, Google Chrome has shown its intention to gradually phase out third-party cookies through a plan called “Privacy Sandbox Initiative” that provides privacy-preserving alternatives to existing ad targeting and measurement mechanisms, thus allowing the advertising ecosystem to continue to operate effectively without relying on third-party cookies [11, 65].

Driven by the need to adapt to a future with restricted cross-site tracking, many advertising platforms—such as Google Ad Manager [61], Criteo [14, 66], and Taboola [31]—have begun implementing and testing the Privacy Sandbox APIs. First, market evaluations, such as Criteo's analysis [14, 66], suggest a potential 60% decrease in publishers' revenue in Chrome if third-party cookies are replaced solely by Privacy Sandbox APIs, and an even more significant drop of 95% in the absence of both third-party cookies and the Privacy Sandbox. Second, a body of work examines the theoretical and empirical privacy guarantees of the Privacy Sandbox, identifying potential risks like re-identification [7–9, 13, 39, 45], cross-site tracking [7, 13], and violations of privacy regulations [77]. Despite these explorations, uncertainty remains regarding the adoption and real-world impact of the Privacy Sandbox as a widespread alternative.

While market and technical evaluations of the Privacy Sandbox are vital, understanding the user perspective is equally crucial for a comprehensive assessment of its viability. To address this gap, we present the first user study specifically focused on Privacy Sandbox ads, in which we study how individuals perceive online advertising delivered through these novel privacy-preserving mechanisms. Our research employs a user study with *n* = 449 participants who shared their attitudes towards ads on Chrome under three distinct privacy configurations (Section 3): (1) *Privacy Sandbox only*, (2) *third-party cookies enabled* (representing the current Status quo), and (3) *first-party cookies only* (representing a scenario where third-party cookies are blocked without any alternative). This latter group (first-party cookies only) holds particular significance considering recent legal developments that challenge the legal basis of third-party cookies for advertising [54], coupled with Google's multiple delays [17, 18] and recent announcement to discontinue with the Privacy Sandbox [19].

Our findings highlight that, despite its promises in providing effective and personalized advertising while protecting users' privacy, users are more likely to perceive Privacy Sandbox ads as less relevant and less persuasive. In particular, Privacy Sandbox proposals did not reach the same levels of perceived relevance or purchase intent as ads delivered with third-party cookies, nor did users report stronger feelings of privacy protection. This suggests

<figure><img src="image_3.png" /></figure>

This work is licensed under a Creative Commons Attribution 4.0 International License. CHI '26, Barcelona, Spain

© 2026 Copyright held by the owner/author(s).

ACM ISBN 979-8-4007-2278-3/26/04

https://doi.org/10.1145/3772318.3790444
a potential gap between the Privacy Sandbox's privacy and utility claims from a user perspective. Nevertheless, the Privacy Sandbox mechanisms still provide incremental gains in ad effectiveness compared to purely cookieless ad targeting available today.

While our findings suggest that the Privacy Sandbox, as implemented in Chrome, is not yet a perfect replacement for third-party cookie-based advertising, it represents a significant improvement and a transitional step toward a post-cookie era where third-party cookies are deprecated without an alternative due to legal restrictions and users' increased awareness. Our findings offer a crucial user-centric perspective that informs the evaluation of Privacy Sandbox, which was a potential successor to third-party cookies, and for future privacy-preserving proposals. Google's recent decision to deprecate most of the Privacy Sandbox APIs raises questions about the future of online advertising in the post-cookie era.

# 2 Background and Related Work

We discuss mechanisms of online targeting and tracking, privacy protection, user perceptions of targeted ads and tracking, and Google's Privacy Sandbox Initiative.

## 2.1 Online Targeting and Tracking

2.1.1 *Online Behavioral Advertising.* Online behavioral advertising (OBA), also known as interest-based advertising, is a pervasive technology designed to target users with ads aligned with their interests and preferences based on data collected from their online browsing activities [6, 43, 75]. OBA operates within a complex system of advertisers, publishers, and ad networks. Advertisers seek to maximize their return on investment by targeting consumers with relevant ads, while publishers aim to gain profit by showing personalized ads to their users [20, 26, 29]. Ad networks such as Google, Amazon, and Criteo, and many others track users' online activity using a variety of tracking mechanisms, including third-party cookies, browser fingerprinting, server-side tracking, and persistent identifiers that assign unique identifiers to users and collect their data, such as web searches, visited websites, offline behaviors (e.g., in-store purchases), and user-provided personal information to build detailed user profiles [10, 24, 44]. Ad networks implement Real Time Bidding (RTB) auctions to sell impressions to advertisers [5, 24, 82], and determine the delivery of their ads to select which ads to show to which users and when.

A large body of measurement studies has investigated how OBA and tracking operate in practice. Prior work measures the prevalence of tracking on the web, the flow of personal data, and the extent to which ad platforms profile and target users [5, 34, 43, 79, 82, 83]. While OBA enables effective ad targeting, the extensive tracking and profiling it relies on raise serious privacy and security concerns. Ad platforms collect, infer, share, and exploit highly detailed and privacy-sensitive user information, often without user consent [15, 20, 30, 46, 76], and uniquely identify users [8, 10, 37]. This access to user data allows advertisers to microtarget individuals based on sensitive attributes such as political beliefs, religion, ethnicity, or health conditions [53, 62, 83]. Targeted ads can also discriminate against, manipulate, or exploit vulnerable populations, with algorithms disproportionately exposing certain groups to sensitive, problematic, or deceptive content [2, 62, 70, 83]. These risks

are amplified by the lack of transparency and limited user control over how personal data is collected, shared, and used in ad delivery [3, 20, 24, 43, 79].

2.1.2 *Privacy protection.* Most efforts to protect user privacy on the web focus on limiting or eliminating the use of third-party cookies. Web browsers like Safari [12] (via Safari's Intelligent Tracking Prevention [41]), and Firefox [48] have already blocked third-party cookies for their users and effectively shut down behavioral targeting by blocking the execution of scripts embedded by third-party servers. Moreover, Privacy regulations like the CCPA require websites to offer a “Do Not Sell My Personal Information” option, preventing publishers from sharing users' personal data, such as cookies or geolocation data, with third parties. However, when cookies are no longer employed, advertisers have significantly less information about users and cannot engage in effective behavioral ad targeting. Facing this, privacy-first alternative models such as contextual targeting[33] are gaining popularity as they are safe and less intrusive, but less relevant ads [51].

2.1.3 *User perceptions.* Prior work has examined users' perceptions of online advertising and tracking, showing that reactions to targeted ads vary widely. While some users appreciate relevant ads and are willing to share data when they feel in control of what is being collected [16, 27, 75] and find targeted ads are less relevant [64], others dislike the merely presence of ads, often due to certain ad content they perceive as problematic, deceptive, or intrusive [1, 81]. Users are also concerned about the privacy concerns of targeted ads and tracking. Although users may be largely aware of the data collected, they are unaware or underestimate the amount of tracking and inferences made about them [34, 75, 79], and may have negative attitudes towards tracking [25, 52]. Also, they often find the ads to be creepy and privacy-invasive [32, 75, 78, 79]. Specifically, users show a persistent tension between the benefits of personalized and useful ads and concerns about the invasiveness of tracking [16, 75]. Moreover, even when users adopt privacy-protection tools, their use may be based on misconceptions or a placebo effect rather than a clear understanding of how these tools work [80].

## 2.2 Google's Privacy Sandbox

In response to growing privacy concerns surrounding third-party cookies, Google announced in 2019 that Chrome would improve cookie controls and move toward new, more privacy-preserving models of tracking [11, 65]. Instead of phasing out third-party cookies outright, Google introduced a set of proposals or APIs, known as the **Privacy Sandbox Initiative<fnref n="1" />**, that provide privacy-preserving solutions to the invasive third-party cookies and reduce passive and covert tracking on both web and mobile. While third-party cookies allow cross-site trackers and ad techs to collect, store, and link a user's browsing behavior across the web, the Privacy Sandbox provides browser-side storage and processing, i.e., the user data is never shared with third parties directly, but only aggregated privacy-preserving data is. So, advertisers will still be able to target their audiences without having their detailed profiles. This represents a structural shift in how data is collected and how ads are targeted and delivered. The Privacy Sandbox includes proposals for

<footnote n="1">¹https://privacysandbox.com</footnote>
private advertising and measurement, notably the Topics API [59], Protected Audience API [55], and Attribution Reporting API [60]. The APIs were gradually enabled in Chrome 115 beginning Q3 of 2023. In our study, we focus on how ads are delivered through:

Topics API. Instead of serving interest-based ads using browsing and behavioral data collected using third-party cookies, Topics API allows advertisers to access only users' interest segments (e.g., sports, pets) inferred from their recent browsing activity without revealing a specific activity, ensuring that users cannot be uniquely re-identified [59]. Technically, when a user browses the web, Chrome classifies each visited website into high-level topics of interest using an internal classifier based on its eTLD+1 (e.g., www.football.com is categorized under “Sports”). The user’s interests are stored locally in his browser and can be viewed and managed via chrome://settings/adPrivacy/interests, and are updated weekly. More details are also provided on the Topics internal Chrome link chrome://topics-interals/. When an advertiser wants to target a user with an ad, Topics API exposes at most three topics of interest per request per user.

Protected Audience API. To allow advertisers to retarget their customers and build custom audiences without tracking, Protected Audience API provides mechanisms and infrastructure to store user behavioral data and run real-time ad auction directly within the user's browser rather than remote advertisers' servers [55]. It introduces a new browser storage mechanism called **Interest Groups**, which keeps users' behavioral data and bidding information on-device. When a user visits an advertiser's website, a script adds the user to an **Interest Group**, which will participate in on-device auctions later when the user visits a publisher. The winning ad is rendered within a *fenced frame*<fnref n="2" /> [68], and auction outcomes along with ad measurements are managed by the *Attribution reporting API*. Users can view and manage the list of sites that have added them to an **Interest Group** through chrome://settings/adPrivacy/sites.

Research on the Privacy Sandbox has largely followed two directions: evaluating its privacy and utility guarantees, and analyzing its adoption in the advertising ecosystem. Studies of the Topics API show that although it reduces the disclosure of detailed browsing data, it does not fully prevent re-identification, as repeated topic access can reveal users' interests over time [8, 9, 39]. Researchers further report that assigned topics often misrepresent users' actual interests, with at least one true topic in 60% of cases [8]. Early empirical analysis of the Protected Audience API highlights continued risks such as cross-site leakage, tracking, service disruption, and pollution attacks [13], and studies of auction reporting and measurement show that adversaries could still link requests across sites [45]. In parallel, several studies investigated the real-world deployment of the APIs on popular websites [40, 50, 77], finding limited and uneven adoption because of the uncertainty about its promises. Industry evaluations have also examined the economic impact of the Privacy Sandbox on publishers and advertisers and report revenue decreases [66, 67]. Our study extends this line of work by shifting focus to users: we examine how the browser privacy settings affect their perception of ads.

# 3 Methodology

In this section, we describe the methodology of our user study, in which we examine how different Chrome privacy settings—Privacy Sandbox, third-party cookies, and first-party cookies—shape users' perceptions of ad relevance, purchase intent, and perceived privacy violations when exposed to online ads.

## 3.1 Overview

We designed a user study in which we recruited participants and asked them to follow a structured set of instructions to configure their browser privacy settings. Participants were then asked to view a set of ads and complete a survey about their perceptions. First, participants were redirected to our study website, where they provided consent to our privacy notice before taking part in the study. We then applied our selection criteria through automated screening checks to identify eligible participants and invite them to continue. Participants who did not meet the criteria were excluded and received a bonus payment for their willingness to participate.

Following the selection process, we randomly assigned each participant to one of three testing groups: the Privacy Sandbox group, the Status Quo group (third-party cookies enabled), and the Cookieless group (third-party cookies blocked), which we describe in the following sections. Next, participants were directed to a specific page where we provided instructions to configure their browser settings using Chrome feature flags according to their assigned testing group. Before starting the task, participants answered two questions about their use of privacy protection tools and their likelihood of accepting cookies while browsing.

The task consisted of participants visiting three publishers, randomly selected from our predefined list, using their browsers to see ads. For each publisher visit, participants were asked to locate an ad and then return to the study website to submit the ad URL and complete the perception survey. We asked participants four questions about their perceptions of the targeted ad: (1) perceived ad relevance, (2) purchase intent, and (3) whether they felt a privacy violation. On average, participants took 10 minutes to complete the study, and we compensated each participant with £1.50 for their participation. Figure 1 illustrates the workflow of our experiment. We describe each phase of the study in detail in the following sections.

## 3.2 Recruitment

We recruited $n = 449$ participants through Prolific<fnref n="3" />, a research-oriented online crowdsourcing platform, and redirected them to our experiment website between January and April 2025. We recruited only native or primary English speakers to ensure a clear understanding of the study instructions and surveys. Participants were 18 years of age or older ($M = 35.9, SD = 11.9$). Of the sample, 53% identified as male and 42% as female, and most participants (52%) were employed full-time. We provide additional demographic details in Table 4.

### 3.2.1 Selection Criteria.
Before starting the study, we selected eligible participants according to our conditions and collected additional data that could potentially affect the results. We implemented the following automatic checks on our experiment website:

<footnote n="2">²A secure frame isolating embedded content from the embedding context.</footnote>

<footnote n="3">³https://www.prolific.com/</footnote>
<figure><img src="image_4.png" /><figcaption>**Figure 1: Workflow of the experiment.**</figcaption></figure>

(1) **Browser and Device:** Our study focused on Chrome desktop users, as Chrome is currently the only browser implementing Privacy Sandbox APIs, and we limited our scope to only the web, as mobile and web implementations differ. This was stated in the Prolific study as a condition, and we implemented additional checks on our website by parsing the user agent string to check both the browser (Chrome) and the device type (desktop).

(2) **Google login:** The implementation of the Privacy Sandbox is at the browser level and is independent of being logged in to Google or not. While prior research shows that logged-in users have additional data signals [4], Google and many ad networks continue to track users even when they are logged out, using diverse tracking mechanisms. Our study was limited in addressing this concern as we did not ask participants to log into their accounts. We assume that our random selection and assignment of participants could mitigate any variation in login status and any bias.

(3) **Third-party cookies:** The past use of third-party cookies may influence participants' advertising profiles and, consequently, the ads they receive and their perceptions. To avoid any related bias and ensure that all experimental conditions operated as intended, we excluded participants having third-party cookies blocked by default. This exclusion doesn't necessarily imply that the remaining participants were less privacy-conscious, as they may rely on other tracking-protection tools. It has been shown that Prolific samples exhibit relatively high external validity for studies of privacy beliefs and perceptions [72]. Only 8.4% of Prolific users who accessed our study had third-party cookies blocked. To detect third-party cookies status, we implemented a third-party service that writes and reads a cookie, and we embedded it in our website. When the script fails to set the cookie, this indicates that third-party cookies are blocked, so we then screened out the participant of the study.

(4) **Ad blockers:** We asked participants to disable any ad blockers before starting the study, as it would otherwise prevent them from seeing ads. We detected ad blockers as used in [29] by triggering a JS request to a bait URL<fnref n="4" /> commonly used by trackers. If this request is blocked, the participant uses an ad blocker, and they are prompted to disable it temporarily for the study. Because prior ad blocker usage may still influence ad delivery even after being paused, we recorded whether participants had been using one and included this variable as a control in our analysis.

### 3.2.2 Testing Groups Assignment.
Regulators, industry groups, and technology companies are participating in testing and evaluating the Privacy Sandbox as a promising alternative to third-party cookies and its potential impact on web actors. For instance, the UK's Competition and Markets Authority (CMA) is monitoring the Sandbox to ensure these changes promote both privacy and fair competition [21, 38]. The CMA has been involved in developing experimental and testing designs for ad networks, publishers, and advertisers, to help them evaluate the impact of third-party cookie deprecation and the effectiveness of the Privacy Sandbox's APIs. The experimental design consists of comparing advertising performance in a **Treatment Group** using the new Privacy Sandbox APIs against a **Control Group** that continues using third-party cookies, to determine if the new tools maintain ad effectiveness. However, this design is designed for big ad tech companies that have wide access and traffic on the web (for more context, see Appendix A.1). In our study, we adopt the CMA's definition of testing groups, but we change the way it creates and add browsers to the groups. Inspired by the browser-level testing approach in the **CMA's Design 2**, we propose a new experimental setting that leverages Chrome's facilitated testing capabilities with Chrome feature flags [56] to add participants to testing groups, i.e., to set up privacy settings. Our design offers a new setup for future user studies on the Privacy Sandbox technologies.

**Testing Groups.** We compare ad perception over the following testing groups as defined by the CMA: (1) **The Privacy Sandbox group:** (CMA's Treatment) ads are served using only the Privacy Sandbox APIs, while third-party cookies are blocked, (2) **The Status Quo group:** (CMA's Control 1) ads are served with data related to third-party cookies without the new Privacy Sandbox APIs (business as usual), (3) **The Cookieless group:** (CMA's Control 2) ads are served only with first-party data and other stateless tracking mechanisms; neither Privacy Sandbox tools nor third-party cookies data are used.

**Privacy Sandbox Chrome feature flags.** The time we ran the study, Privacy Sandbox APIs were still evolving, and ad networks were still adapting their advertising ecosystem, while Chrome enabled the APIs and blocked cookies for only 1% of all users. This means the remaining users don't have Privacy Sandbox ad delivery APIs and cookies blocked by default. Chrome enables developer to test

<footnote n="4">⁴https://connect.facebook.net/en_US/fbevents.js</footnote>
features before their general availability via Chrome feature flags<fnref n="5" /> via a Chrome internal link chrome://flags/. To test the Privacy Sandbox, Chrome offers Chrome-facilitated testing flags [56, 57] to set up labels and join one of the CMA's testing groups. We randomly assigned each participant to one testing group and showed them how to set up the following feature flags:

**Third-party Cookie Phase Out Facilitated Testing:** allows the browser to join a testing group via a drop-down menu, as illustrated in Figure 2. We focused on the options marked **Force** to ensure that the intended experimental behavior is activated regardless of other browser settings [56, 57] (e.g., the label **Enabled Force Treatment** joins the browser to the Privacy Sandbox group).

**Test Third Party Cookie Phaseout:** To ensure robust cookie blocking in the Privacy Sandbox and the *Cookieless* groups, we asked participants to set this flag to **Enabled**. As we observed a discrepancy, despite the documentation suggesting cookie blocking when the user joined these groups, cookies were not blocked from the browser.

Table 1 summarizes the flags to set up for each testing group, their internal Chrome link, and their labels. Figure 4 illustrates how we show participants instructions to set up the flags on our study website.

*Verifying Ad Delivery.* Google's documentation [56–58] suggests that this configuration ensures ads are delivered based on the testing group the browser is in. For instance, when the flag was set to **Enabled Force Treatment**, ads are delivered solely through the Privacy Sandbox APIs. To verify the validity of this, we performed multiple tests where we manually checked via the *Chrome Developer Tools* the **Interest Groups** status (the new browser storage for *Protected Audience API*) on publishers' websites, i.e., when ads are displayed. The **Interest Groups** status includes loads, bids, and auction wins of the stored **Interest Groups** in the browser. We additionally checked the status of *Topics API* via the Chrome internal link chrome://topics-interals/. When the user is in the *Privacy Sandbox* group, while cookies is completely blocked, **Interest Groups** are always loaded and invited to bid and **Topics** are observed by context domains (ad networks), which is not the case when the user is in the *Status Quo* or the *Cookieless* groups.

*Verifying Participant's Configuration.* To apply configurations, participants reload their browser, and then we verified that flags were correctly set. To do so, we used Chrome's internal function `navigator.cookiesDeprecationLabel.getValue()` to check the browser-level third-party cookie deprecation label associated with the **Third-party Cookie Phase-Out Facilitated Testing** flag. This function returned `fake_control_1.1` for participants in the *Status Quo* group, while it returned no value for the *Privacy Sandbox* and *Cookieless* groups (it was deprecated during our study). To address this, we additionally verified third-party cookie blocking using our third-party cookie test service (same as in part 3.2.1).

*Pretesting.* Before running the final study, we conducted several pilot tests. Initially, many Prolific participants returned to the study, reporting that they did not understand the purpose of the privacy settings they were asked to configure. To address this, we added

a “Learn More” button on the configuration page (Figure 4) that redirects them to brief, neutral explanations (Figure 3). This reduced return rates while providing necessary transparency without revealing hypotheses or introducing bias.

Our approach does not require a warm-up phase; participants were not asked to browse or wait for sufficient topics of interest or Interest Groups to accumulate in their browsers. In the Privacy Sandbox, ad delivery is independent of local data collection—users may already have stored topics or Interest Groups without necessarily receiving Privacy Sandbox ads or being in the *Privacy Sandbox* group. Instead, setting a feature flag to a testing group label forces Chrome to use the corresponding ad delivery mechanism. We assume that participants already had relevant browsing histories from advertisers implementing the *Topics* and/or *Protected Audience* APIs.

As our study modified browser settings, we instructed participants on how to restore their default settings at the end of the study. All participants were asked to reset the *Third-party Cookie Phase Out Facilitated Testing* flag to **Default** to revert to their default group. Those in the *Privacy Sandbox* and *Cookieless* groups also reset the **Test Third Party Cookie Phaseout** flag to **Disabled** to re-enable third-party cookies.

## 3.3 Procedure

3.3.1 *Publishers selection.* To expose participants to targeted ads, we ask them to visit three publishers that serve ads. Precisely, we are interested in publishers that actively support Privacy Sandbox APIs for a controlled and valid evaluation of ad delivery across the three privacy settings. We went through a selection process for a final list of publishers for our study:

(1) We identified websites implementing Privacy Sandbox APIs in their ad delivery using the Chrome's public monthly statistics<fnref n="6" /> on feature usage<fnref n="7" />. We identified and selected the Privacy Sandbox JS features for ad delivery (see Appendix A.3), and downloaded a list of 831 website URLs for January 2025.

(2) We filtered this list and kept only publishers' URLs (serving ads) by checking the presence of the `ads.txt` file through a simple HTTP call code<fnref n="8" />. This filtering step reduced the list of websites to 590 publishers.

(3) We limited our selection to English-language publishers operating in the Americas and Europe. Accordingly, we excluded websites using top-level domains from other regions (e.g., .ru, .cn) and any sites not published in English. This process yielded 173 URLs.

(4) To help participants in finding ads during the experiment, we retained only websites that (i) displayed ads prominently on the main page, (ii) provided textual rather than audio or video content, and (iii) did not require a login to access at least some content as highlighted in [64]. We implemented a crawler that scrapes the content of the main page of each publisher and identifies ads appearing in `frame` or `div` elements by matching their content or attributes to known ad patterns such as keywords like “ad” “banner” or “adsystem” in element IDs or classes, as well as URLs from known

<footnote n="5">⁵By configuring a specific feature flag (enabled, disabled, or a specific value), the browser creates an experimental setup to test the feature.</footnote>

<footnote n="6">⁶https://chromestatus.com/metrics/feature/popularity</footnote>

<footnote n="7">⁷These statistics report, based on the HTTP Archive (https://httparchive.org/), the share of page loads of HTML and JavaScript features usage by crawling the Chrome (CrUX) Top Million Websites list [28].</footnote>

<footnote n="8">⁸This file is required by the Interactive Advertising Bureau (IAB) transparency standards for a publisher to publicly list its authorized ad sellers [42].</footnote>
<table><caption>**Table 1:** Chrome feature flags and their internal link for our experiment. For each testing group, we provide the label to set for the flag.</caption><thead><tr><th>Flag</th><th>Third-party Cookie Phase Out Facilitated Testing</th><th>Test Third Party Cookie Phaseout</th></tr></thead><tbody><tr><td>Chrome internal link</td><td>chrome://flags/#3PC-phase-out-facilitated-testing</td><td>chrome://flags/#test-third-party cookie-phaseout</td></tr><tr><td>Privacy Sandbox</td><td>Enabled Force Treatment</td><td>Enabled</td></tr><tr><td>Status Quo</td><td>Enabled Force Control 1</td><td>–</td></tr><tr><td>Cookieless</td><td>Enabled Force Control 2</td><td>Enabled</td></tr></tbody></table>

ad networks (e.g., "googleadservices.com" or "criteo.com"). We then manually visited each publisher to check the ad placement.

(5) Although we selected websites implementing Privacy Sandbox APIs for advertising, this does not guarantee that ads are actually served through these APIs. We verified the status of **Interest Groups** and **Topics** via Chrome Developer Tools (as we described before in Verifying Ad Delivery). We retained only websites where **Topics** were observed and **Interest Groups** were invited to bid in ad auctions. The final list includes 14 publishers (see Appendix A.4).

As the study settings via Chrome flags can be time-consuming and potentially confusing for simple users, we limited the task to three randomly selected publishers from the pool of 14 websites, i.e., each participant was shown only three ads. This restriction reduced the risk that participants would become fatigued or disoriented, which could lead them to respond merely to obtain the reward. While this provides a limited snapshot of ad relevance and may not capture the full variability in participants' preferences across different products or contexts, randomizing ad selection across participants and ensuring diversity in the ad pool helps mitigate this variability without impacting the results.

### 3.3.2 Survey.
After participants configured their browsers with success, they completed two surveys: a baseline survey about their privacy habits before publisher visits, and another survey after each publisher visit (three times) to assess their perceptions of ads.

**before ad exposure.** To assess how participants' online privacy behaviors influenced their ad perception, we collected data on their use of protection tools and their interactions with cookie consent banners, both of which are known to affect ad targeting quality and user attitudes [74]. These measures were included as control variables in our analysis. Participants answered two questions (Q1 and Q2 in Table 2): Q1 captured their likelihood of accepting cookies while browsing, and Q2 captured their prior use of privacy protection tools such as ad blockers, tracking-blocking extensions, and VPNs. Although participants were instructed to disable ad blockers during the task, prior use could still influence ad targeting. The self-reports from Q1 served as a complement to our automated detections presented in section 3.2.1.

**After ad exposure.** After a publisher visit, we instructed participants to locate an ad, copy its URL (Right-Click and Copy Link), and submit it to the study website. We collected the ad URLs to verify that the submitted URL corresponds to an ad link by looking for advertising parameters (e.g., "clid" and "utm"). Participants then answered four ad perception questions (Q3–Q6 in Table 2): (1) Ad

relevance, (2) Privacy violation, (3) Purchase intent, and (4) Prior visit to the advertiser (retargeting). Such measures are commonly used in advertising research as independent and dependent variables [36]. Q3–Q5 were rated on a 5-point Likert scale (**Strongly Disagree** to **Strongly Agree**), while Q6 required participants to choose **Yes**, **No**, or **Do Not Remember**. To ensure data quality, we included an attention check and excluded participants who failed it. Figure 5 illustrates how the ad perception survey was presented to users.

## 3.4 Ethics

This research investigates user perceptions of online advertising under different tracking mechanisms, focusing on Google Chrome's Privacy Sandbox APIs in comparison to third-party and first-party cookie-based approaches. All procedures adhered to established ethical research principles to protect participant rights and data.

(1) *Informed Consent:* Participants were fully informed about the study's objectives, the data being collected (including survey responses and ad URLs), and the browser configurations required, and consent to that through the privacy notice before being recruited.

(2) *Minimal Risk:* We collected only non-sensitive data, including user agent strings, ad metadata, and survey responses, and did not gather any personally identifiable information. All data were securely stored, accessible only to the research team, and used exclusively for analysis. To minimize potential risks, participants were instructed at the end of the study on how to revert their browser configuration to its default state, as detailed in our measurement methodology.

(3) *Ethical Approval:* This study received legal approval from our institution's DPD. We also have the approval (autorisation n° 2025-48) of our Institutional Review Board (IRB) that the study design complies with GDPR standards, including clear consent mechanisms and data minimization practices.

## 4 Results

Using G\*Power [35], we estimated that a sample size of a minimum of *n* = 472 participants would be required to detect a medium effect size in a one-way ANOVA with three independent groups. We initially launched the Prolific study with a target of 500 participants, but we excluded 51 participants who either blocked third-party cookies or returned an incorrect Prolific completion code.

A total of *n* = 449 participants completed the study. Each participant visited three publishers and completed a survey after each visit, resulting in 1,332 survey responses. Participants were randomly assigned to one of three groups: 138 in Privacy Sandbox, 162
<table><caption>Table 2: Survey questions used to assess participants' ad perception. Q1 and Q2 were asked before ad exposure, while Q3–Q6 were administered after participants viewed the advertisements.</caption><thead><tr><th>Phase</th><th>ID</th><th>Questions</th></tr></thead><tbody><tr><td rowspan="2">Before ad exposure</td><td>Q1</td><td>Do you use any kind of privacy protection tools (e.g., ad blockers, tracking restriction extensions, VPNs)?</td></tr><tr><td>Q2</td><td>How likely are you to accept cookies when browsing online?</td></tr><tr><td rowspan="4">After ad exposure</td><td>Q3</td><td>How relevant is this Ad to your interests?</td></tr><tr><td>Q4</td><td>How much do you feel that your privacy is violated when seeing this Ad?</td></tr><tr><td>Q5</td><td>How likely are you to purchase this product / use this service or similar ones from this Advertiser?</td></tr><tr><td>Q6</td><td>Did you visit the Advertiser's website before?</td></tr></tbody></table>

in *Status Quo*, and 149 in *Cookieless*. Prior to the study, 14.3% of participants had an active ad blocker (detected automatically). In self-reports, 23.8% indicated using privacy-protection tools, while 70.8% reported not using them. Additionally, 63.5% of participants stated that they were likely or very likely to accept cookies while browsing. Regarding prior exposure to advertisers, participants in the *Status Quo* group reported that 25.5% of ads were from advertisers they had already visited, compared to 18.6% in the *Privacy Sandbox* group and 14.4% in the *Cookieless* group. We excluded 36 responses that failed attention checks and 246 that did not include a valid ad URL (e.g., section links or unrelated websites). After this data cleaning process, we retained 1,050 valid responses for analysis.

To compare user perception between the three privacy settings (i.e. *Privacy Sandbox*, *Status Quo*, and *Cookieless*), we used a One-Way ANOVA test to assess the main effects of the groups for the dependent variables Ad relevance (Q3), Privacy violation (Q4), and Purchase intent (Q5). The significance level for all tests was set at $\alpha = .05$. We evaluated the magnitude of any statistically significant effects using the Eta squared effect size $\eta^2$ (with thresholds of < .01 for a small effect, < .06 for a medium effect, and < .14 for a large effect). In case of a significant effect, we conducted post hoc analyses using pairwise ANOVA tests to identify specific group differences. For each pairwise comparison, we report the p-values, along with the test statistics t and the effect size r (small effect < .01, medium effect < .06, large effect < .14). Table 3 summarizes the statistical results.

To examine the impact of user privacy behaviors on ad perception, we conducted logistic regression analyses to test the impact of the independent variables tracking restriction tool (Q1) and cookie acceptance (Q2) on the dependent variables ad relevance (Q3), perceived privacy violation (Q4), and purchase intent (Q5). We also incorporate the past usage of ad blockers detected with our automatic checks. In case of a significant effect $p < .05$, we report the Odds ratio OR (Odds ratio > 1 positive effect, and Odds ratio < 1 negative effect). Similarly, we examine the impact of retargeting (Q6) on ad perception for Q3, Q4, and Q5.

## 4.1 Ad Effectiveness Perceptions

We tested for statistical differences in ad relevance (Q3) and purchase intent (Q5) across the three experimental groups using One-Way ANOVA. For Q3, the analysis revealed a significant effect of group, $F(2, n = 449) = 8.60, p < .0002, \eta^2 = .0162$. Post hoc

comparisons showed significant differences between the *Privacy Sandbox* and *Status Quo* groups, $t = 2.38, p < .017, r = .008$, and between the *Status Quo* and *Cookieless* groups, $t = 4.07, p < .00005, r = .023$. The difference between the *Privacy Sandbox* and *Cookieless* groups was marginal, $t = 1.68, p < .092, r = .004$. For Q5, ANOVA also indicated a significant group effect, $F(2, n = 449) = 4.80, p < .0083, \eta^2 = .0091$. Post hoc tests revealed significant differences between the *Privacy Sandbox* and *Status Quo* groups, $t = 2.22, p < .027, r = .007$, and between the *Status Quo* and *Cookieless* groups, $t = 2.90, p < .004, r = .012$, but no significant difference between the *Privacy Sandbox* and *Cookieless* groups, $t = 0.69, p < 0.48, r = .001$.

We examined the impact of past use of privacy-protection tools (Q1) and cookie acceptance (Q2) on ad relevance (Q3) and purchase intent (Q5). Logistic regression analyses revealed a significant positive effect of cookie acceptance on both ad relevance, OR = 1.26, $p < .001$, and purchase intent, OR = 1.25, $p < .001$. In contrast, past use of privacy-protection tools did not have a significant effect on either outcome ($p > .05$). Finally, for detected ad blockers, we found significant negative effects on ad relevance, OR = 0.52, $p < .001$, and purchase intent OR = 0.47, $p < .001$.

We tested the impact of retargeting (Q6) on ad relevance (Q3) and purchase intent (Q5). Logistic regression tests showed a significant positive effect when it is question of retargeting on both Q3, OR = 2.13, $p < .001$, and Q5, OR = 2.62, $p < .001$, and a negative effect when participants did not visit the advertiser on both Q3, OR = 0.28, $p < .001$, and Q5, OR = 0.26, $p < .001$.

## 4.2 Privacy Perceptions

We tested for statistical significance in perceived privacy violation (Q4). The ANOVA did not reveal a significant effect across the three groups, $F(2, n = 449) = 2.21, p < 0.1097, \eta^2 = .0042$. We then examined the impact of past privacy behaviors on Q4. Logistic regression indicated a significant negative effect of cookie acceptance on perceived privacy violation, OR = 0.80, $p < .001$. In contrast, the use of privacy-protection tools doesn't have a significant effect ($p = .68$). Finally, we tested whether the presence of an ad blocker influenced Q4, but found no significant effect ($p = .215$). We tested the impact of retargeting (Q6) on perceived privacy violation (Q4). Logistic regression analysis showed no significant effect ($p = 0.7$).
<table><caption>Table 3: Summary of statistics of self-reported variables (Ad Relevance, Privacy Violation and Purchase Intent) by testing group. The last three columns show One-Way ANOVA for differences across groups and differences in means for the pairwise ANOVA. *p < .05 (statistical significance).</caption><thead><tr><th rowspan="2">Variables</th><th>Privacy Sandbox (1)</th><th>Status Quo (2)</th><th>Cookieless (3)</th><th>ANOVA</th><th colspan="2">Pairwise</th></tr><tr><th>Mean ± SD</th><th>Mean ± SD</th><th>Mean ± SD</th><th>F-Stat</th><th>(1) vs. (2)</th><th>(1) vs. (3)</th></tr></thead><tbody><tr><td>Relevance</td><td>2.14 ± 1.21</td><td>2.37 ± 1.32</td><td>1.99 ± 1.20</td><td>8.60*</td><td>-0.23*</td><td>0.16</td></tr><tr><td>Privacy Violation</td><td>1.92 ± 1.07</td><td>1.98 ± 1.14</td><td>1.81 ± 0.99</td><td>2.21</td><td>-0.06</td><td>0.11</td></tr><tr><td>Purchase Intent</td><td>2.24 ± 1.26</td><td>2.46 ± 1.39</td><td>2.17 ± 1.27</td><td>4.80*</td><td>-0.22*</td><td>0.07</td></tr></tbody></table>

# 5 Discussion

Participants reported their perceptions of ads served in three different privacy environments: *Privacy Sandbox*, *Status Quo*, and *Cookieless*. Our results indicate that ads delivered via Privacy Sandbox APIs were perceived as less relevant and generated lower purchase intent from participants compared to the *Status Quo*, while users did not report an increase in perceived privacy protection. We discuss the implications of these findings below.

*Ad Effectiveness Declines with Privacy Sandbox.* We found that participants perceived ads delivered through the Privacy Sandbox APIs as less effective than ads delivered with third-party cookies. Notably, there were significant differences between the *Privacy Sandbox* and *Status Quo* groups, as well as between the *Status Quo* and *Cookieless* groups, indicating that Privacy Sandbox ads were rated as less relevant and elicited lower purchase intent from participants. A marginal difference was observed between the *Privacy Sandbox* and *Cookieless* groups for ad relevance, with an overall difference in the mean ratings. These findings suggest that while the Privacy Sandbox improves perceived ad relevance compared to a first-party cookies environment with no privacy-preserving alternative, its actual implementation does not provide the same perceived ad effectiveness as traditional third-party cookies. Moreover, the absence of a significant difference in purchase intent between the *Privacy Sandbox* and *Cookieless* groups suggests that Privacy Sandbox ads are not more persuasive than non-personalized ads to users, even if they are seen as more relevant.

Furthermore, the significant impact of cookie acceptance on ad effectiveness suggests that participants who are more likely to accept cookies rated ads as both more relevant and more persuasive. This was expected; users who allow cookies enable more extensive tracking and profiling, and they typically exhibit lower privacy concern, which has been shown to increase positive perceptions of personalized advertising [27, 75]. In contrast, self-reported use of privacy-protection tools showed no measurable effect on perceived ad relevance or persuasiveness. This may suggest that such tools are adopted passively, without a clear understanding of how tracking shapes advertising, or that users do not connect ad relevance to underlying data collection. It has been shown in prior research that users commonly underestimate or misunderstand how behavioral targeting operates [34, 75, 79]. Moreover, privacy protection extensions can enable the fingerprinting [71] and create security breaches [49]. The impact of ad blockers we detected indicates that their use can diminish perceived ad effectiveness. This may indicate that ad blocker users tend to hold more negative attitudes toward

advertising, or it might be because they were deactivated just before the study. The positive effect of retargeting and ad effectiveness suggests that participants are more likely to perceive retargeted ads as more relevant and persuasive.

Our results show that without Privacy Sandbox and third-party cookies, users perceive ads as significantly less effective. This highlights the potential risks for digital advertising if third-party cookies are being deprecated in response to evolving privacy and data protection regulations, and privacy-respecting alternatives like the Privacy Sandbox are not widely adopted in the post cookie world.

*Privacy-preserving Alternatives Do Not Impact Users' Perceived Privacy.* The lack of significant differences between testing groups for perceived privacy violation suggests that implementing Privacy Sandbox does not substantially alter users' sense of privacy intrusion compared to the current use of third-party cookies. Furthermore, our analysis reveals a significant relationship between users' cookie acceptance and their perception of intrusiveness. Users who are more inclined to accept cookies report a lower sense of privacy violation when exposed to ads. Conversely, the absence of a significant effect from the use of tracking restriction tools and ad blockers implies that simply employing these mechanisms does not necessarily lead to increased privacy protection. This was also the case for the effect of retargeting, where users did not feel invasive when they received ads from previously visited websites. This might be interpreted with previous research where participants underestimate tracking and use privacy protection extension with uncertainty about how they work and what they protect [63, 80].

Despite its aim to better protect users' privacy compared to invasive third-party cookies, the Privacy Sandbox participants could not provide participants with a stronger sense of privacy. Our results suggest that users' privacy perceptions might be more strongly influenced by other disruptive elements such as cookie disclaimers, privacy notices, and intrusive ad experiences, such as the ad content, rather than the underlying mechanism that delivers ads. This is because users' reactions to ads are complex and differ widely from positive to negative [27, 75, 79]. We suggest that user perception of privacy may be improved by providing notices or pop-ups about the Privacy Sandbox to users.

*Google is abandoning the Privacy Sandbox.* In late October 2025, Google officially announced that it will phase out most of the Privacy Sandbox APIs, including *Topics*, *Protected Audience*, and *Attribution Reporting* after nearly six years of development and delays. Google was following Safari and Firefox, both of which block third-party cookies by default, and wanted to have an alternative in place
before deprecating cookies, and invited feedback from the industry. The feedback may have inadvertently laid the groundwork for the Sandbox's eventual demise. Our evaluation supports and provides evidence from a sample of real users that the proposed APIs failed to balance the dual objectives of effective advertising while enhancing user privacy. Our study also encourages future work to include users in privacy and utility studies besides empirical evaluations. The proposals can help to learn from for future development of privacy-preserving technologies, where cookie deprecation is inevitable.

# 6 Limitations

Although our study aimed to expose participants to ads in a realistic scenario as they are browsing the web under different privacy configurations, we observed a potential shortcoming in our methodological choices that may have influenced the results.

We instructed participants to set up via *Chrome feature flags*, which changes how ads are delivered, and just after that visiting publishers to see displayed ads. However, these changes may not have propagated immediately due to the dynamic nature of the ad bidding and auction, mainly for the *Privacy Sandbox* group. This latter group knows many technical challenging issues, including that the widespread adoption of the technology was limited according to measurement studies [40] and on-device data storage on ad auctions. We assume in this case that participants have a browsing history and have previously visited websites implementing Privacy Sandbox data collection APIs. Moreover, our verification of settings for participants in the Privacy Sandbox group of the *Enabled Force Treatment flag* was technically not possible as Chrome deprecated such functionality during our experiment. This means that some participants may have configured this flag and only block third-party cookies. This was also the case for our approach in verifying Privacy Sandbox ads, as we could only verify it through calls to the APIs, but this does not fully guarantee that the displayed ads are actually served by Privacy Sandbox and not first-party data solely, as it's related to the fact that advertisers would not bid for this type of ads and also the dynamic nature of ad rendering, despite the documentation saying the selection of the "Force" flag will do it [56–58].

Some additional measures may have influenced ad targeting and delivery. First, we did not check whether participants were logged into their Google accounts or other platforms. Research has shown that online platforms can aggregate data from different devices (e.g., tablets, laptops, smartphones) when a user is logged in [4], providing more effective ads. Because the data used for ad targeting in the Privacy Sandbox is contained within the user's browser, it is unclear how these interact with logged-in users. Second, our selection of 14 publishers that rely on Privacy Sandbox APIs to deliver their ads might not be representative of the regular habits of the users, possibly influencing the ads they receive and the ad exchanges they were delivered from. However, we verified the platforms that served ads to our participants during the experiment and provided the summary in table 5. We found that Google is, by far, the largest ad platform with over 60% of the ads served in each group, and we did not find any important discrepancies between the platforms for any of the groups. To reduce bias regarding the website's contents, as

in other work in the field [64], we randomly select 3 publishers for each user, avoiding user fatigue due to the experimental setup. However, this led to participants being exposed to only 3 ads, whereas in real browsing environments, users typically face hundreds of ads, often within complex A/B testing pipelines with continuously optimized targeting strategies to drive their attention. Future work may mitigate this by building larger datasets, by monitoring users' browsing activities, and by splitting the study's configuration setup from the user's tasks.

Our participants faced several challenges when answering the survey's questions. First, we included self-reported measures of past privacy behaviors at the beginning of the study, which are susceptible to recall and social desirability biases and may have heightened participants' privacy concerns when evaluating ads. Second, asking participants to assess relevance after exposure to a single ad within a dynamic publisher environment may have biased them towards selecting less privacy-intrusive or less interest-aligned ads. Prior work also shows that users are sometimes unable to accurately recognize personalized ads and may hesitate to express strong attitudes towards them [16, 27, 36, 64, 75]. Moreover, a participant's stated willingness to buy a product after exposure to the ad may not reflect real engagement, which is typically measured through click-through rates (CTRs). While differences between experimental groups help mitigate some biases, users generally struggle to evaluate privacy practices, often lacking awareness of data collection and relying on protection tools passively, which can lead to placebo effects [80]. We recommend that future user studies on OBA and privacy carefully address these challenges and adopt robust designs.

Overall, we've tried to avoid many of these limitations through the randomization of users, of publishers, and of testing groups, as well as verifying browser configurations and assisting users through orientations. However, there are still external factors that we are unable to control. Our study focused on short-term user perceptions of ads in a controlled experiment, and our results are limited when attempting to assess long-term exposure to Privacy Sandbox ads.

# 7 Conclusion

This study examined the transition from third-party cookies to the Privacy Sandbox and its impact on user perceptions of online advertising. Through a user study, we compared ad relevance, purchase intent, and perceived privacy violation across three privacy settings: the Status Quo with third-party cookies, Privacy Sandbox as a privacy-preserving alternative, and a Cookieless setting with only first-party data. Our results show that ads delivered through Privacy Sandbox are perceived as less relevant and lead to lower purchase intent than third-party cookie-based ads, without improving users' sense of privacy. Nevertheless, Privacy Sandbox brings an improvement over purely contextual targeting, suggesting that while it cannot yet match the effectiveness of third-party cookies, it offers a partial path forward in a post cookie world. These findings align with prior industry reports highlighting potential revenue declines under Privacy Sandbox [66, 67]. Although the Privacy Sandbox is designed to enhance user privacy while enabling advertising without relying on invasive third-party cookies, our findings
show that it does not fully replicate the effectiveness of traditional
cookies. Despite these limitations, we see potential for the Privacy
Sandbox to evolve through industry collaboration and technical
improvements. Balancing user privacy with the needs of the open
web ecosystem remains crucial, particularly under increasing legal
and regulatory pressure on third-party cookies.

Acknowledgments

We thank the anonymous reviewers for their helpful and construc-
tive feedback. This research was supported in part by the French
National Research Agency (ANR) ANR-21-CE23-0031-02, and by
EU Grant No. 101041223.

The authors used AI-based tools, including OpenAI ChatGPT,
Gemini, and Grammarly, to assist with correcting typographical
errors, grammatical issues, and awkward phrasing throughout the
manuscript.

References

[1] Muhammad Ali, Angelica Goetzen, Alan Mislove, Elissa M. Redmiles, and Piotr Sapiezynski. 2023. Problematic Advertising and its Disparate Exposure on Facebook. doi:10.48550/arXiv.2306.06052 arXiv:2306.06052 [cs].

[2] Muhammad Ali, Piotr Sapiezynski, Miranda Bogen, Aleksandra Korolova, Alan Mislove, and Aaron Rieke. 2019. Discrimination through optimization: How Facebook's Ad delivery can lead to biased outcomes. *Proceedings of the ACM on human-computer interaction* 3, CSCW (2019), 1–30.

[3] Athanasios Andreou, Giridhari Venkatadri, Oana Goga, Krishna P Gummadi, Patrick Loiseau, and Alan Mislove. 2018. Investigating Ad Transparency Mechanisms in Social Media: A Case Study of Facebook's Explanations. In *NDSS 2018 - Network and Distributed System Security Symposium*. San Diego, United States, 1–15. doi:10.14722/ndss.2018.23204

[4] Muhammad Ahmad Bashir, Umar Farooq, Maryam Shahid, Muhammad Fareed Zaffar, and Christo Wilson. 2019. Quantity vs. Quality: Evaluating User Interest Profiles Using Ad Preference Managers. In *Proceedings 2019 Network and Distributed System Security Symposium*. Internet Society, San Diego, CA. doi:10.14722/ndss.2019.23392

[5] Muhammad Ahmad Bashir and Christo Wilson. 2018. Diffusion of User Tracking Data in the Online Advertising Ecosystem. *Proceedings on Privacy Enhancing Technologies* 2018, 4 (Oct. 2018), 85–103. doi:10.1515/popets-2018-0033

[6] Steven C Bennett. 2010. Regulating online behavioral advertising. *J. Marshall L. Rev.* 44 (2010), 899.

[7] Alex Berke and Dan Calacci. 2022. Privacy Limitations of Interest-based Advertising on The Web: A Post-mortem Empirical Analysis of Google's FLOC. In *Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security* (Los Angeles, CA, USA) (CCS '22). Association for Computing Machinery, New York, NY, USA, 337–349. doi:10.1145/3548606.3560626

[8] Yohan Beugin and Patrick McDaniel. 2024. Interest-disclosing Mechanisms for Advertising are Privacy-Exposing (not Preserving). *Proceedings on Privacy Enhancing Technologies* 2024, 1 (Jan. 2024), 41–57. doi:10.56553/popets-2024-0004

[9] Yohan Beugin and Patrick McDaniel. 2024. A Public and Reproducible Assessment of the Topics API on Real Data. In *2024 IEEE Security and Privacy Workshops (SPW)*. 1–8. doi:10.1109/SPW63631.2024.00005

[10] Sarah Bird, Ilana Segall, and Martin Lopatka. 2020. Replication: Why we still can't browse in peace: On the uniqueness and reidentifiability of web browsing histories. In *Sixteenth Symposium on Usable Privacy and Security (SOUPS 2020)*. 489–503.

[11] Chromium Blog. 2020. Chromium Blog: Building a more private web: A path towards making third party cookies obsolete. https://blog.chromium.org/2020/01/building-more-private-web-path-towards.html

[12] by John Wilander. 2020. Full Third-Party Cookie Blocking and More | WebKit. https://webkit.org/blog/10218/full-third-party-cache-blocking-and-more/

[13] Giuseppe Calderonio, Mir Masood Ali, and Jason Polakis. 2024. Fledging Will Continue Until Privacy Improves: Empirical Analysis of Google's Privacy-Preserving Targeted Advertising. In *33rd USENIX Security Symposium (USENIX Security 24)*. USENIX Association, Philadelphia, PA, 4121–4138. https://www.usenix.org/conference/usenixsecurity24/presentation/calderonio

[14] Caio Camilli. 2023. Privacy Sandbox Market Testing | by Caio Camilli | Criteo Tech Blog. https://techblog.criteo.com/privacy-sandbox-market-testing-a-statistical-model-for-measuring-performance-impacts-2e99ff402898

[15] Claude Castelluccia, Mohamed-Ali Kaafar, and Minh-Dung Tran. 2012. Betrayed by your ads! reconstructing user profiles from targeted ads. In *Proceedings of the 12th International Conference on Privacy Enhancing Technologies* (Vigo, Spain) (PETS'12). Springer-Verlag, Berlin, Heidelberg, 1–17. doi:10.1007/978-3-642-31680-7_1

[16] Farah Chanchary and Sonia Chiasson. 2015. User perceptions of sharing, advertising, and tracking. In *Proceedings of the Eleventh USENIX Conference on Usable Privacy and Security* (Ottawa, Canada) (SOUPS '15). USENIX Association, USA, 53–67.

[17] Anthony Chavez. 2024. A new path for Privacy Sandbox on the web. https://privacysandbox.com/news/privacy-sandbox-update/

[18] Anthony Chavez. 2025. Next steps for Privacy Sandbox and tracking protections in Chrome. https://privacysandbox.com/news/privacy-sandbox-next-steps/

[19] Anthony Chavez. 2025. Update on Plans for Privacy Sandbox Technologies. https://privacysandbox.google.com/blog/update-on-plans-for-privacy-sandbox-technologies

[20] Salim Chouaki, Islem Bouzenia, Oana Goga, and Beatrice Roussillon. 2022. Exploring the Online Micro-targeting Practices of Small, Medium, and Large Businesses. *Proceedings of the ACM on Human-Computer Interaction* 6, CSCW2 (Nov. 2022), 1–23. doi:10.1145/3555103 Publisher: Association for Computing Machinery (ACM).

[21] Competition and Markets Authority. 2022. CMA to keep 'close eye' on Google as it secures final Privacy Sandbox commitments - GOV.UK. https://www.gov.uk/government/news/cma-to-keep-close-eye-on-google-as-it-secures-final-privacy-sandbox-commitments

[22] CMA Competition and Markets Authority. 2023. Quantitative testing of Google's Privacy Sandbox technologies - Additional CMA guidance to third parties on testing. https://assets.publishing.service.gov.uk/media/653a4bebe6c968000daa9b7b/Additional_guidance_to_third_parties_on_testing_.pdf

[23] CMA Competition and Markets Authority. 2023. Quantitative testing of Google's Privacy Sandbox technologies - CMA guidance to third parties on testing. https://assets.publishing.service.gov.uk/media/649d6a5f45b6a2000c3d455f/20230629_CMA_industry_testing_update_B.pdf

[24] John Cook, Rishab Nithyanand, and Zubair Shafiq. 2020. Inferring Tracker-Advertiser Relationships in the Online Advertising Ecosystem using Header Bidding. *Proceedings on Privacy Enhancing Technologies* 2020, 1 (Jan. 2020), 65–82. doi:10.2478/popets-2020-0005

[25] Kovila PL Coopamootoo, Maryam Mehrnezhad, and Ehsan Toreini. 2022. "I feel invaded, annoyed, anxious and I may protect myself": Individuals' Feelings about Online Tracking and their Protective Behaviour across Gender and Country. (2022), 287–304.

[26] Tobias Dehling, Yuchen Zhang, and Ali Sunyaev. 2019. Consumer Perceptions of Online Behavioral Advertising. In *2019 IEEE 21st Conference on Business Informatics (CBI)*. IEEE, Moscow, Russia, 345–354. doi:10.1109/CBI.2019.00046

[27] Claire Dolin, Ben Weinshel, Shawn Shan, Chang Min Hahn, Euirim Choi, Michelle L. Mazurek, and Blase Ur. 2018. Unpacking Perceptions of Data-Driven Inferences Underlying Online Targeting and Personalization. In *Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems*. ACM, Montreal QC Canada, 1–12. doi:10.1145/3173574.3174067

[28] Zakir Durumeric. 2023. zakird/crux-top-lists: Downloadable snapshots of the Chrome Top Million Websites pulled from public CrUX data in Google BigQuery. https://github.com/zakird/crux-top-lists

[29] Asmaa El Fraihi, Nardjes Amieur, Walter Rudametkin, and Oana Goga. 2024. Client-side and Server-side Tracking on Meta: Effectiveness and Accuracy. *Proceedings on Privacy Enhancing Technologies* 2024, 3 (July 2024), 431–445. doi:10.56553/popets-2024-0086

[30] Steven Englehardt, Dillon Reisman, Christian Eubank, Peter Zimmerman, Jonathan Mayer, Arvind Narayanan, and Edward W. Felten. 2015. Cookies That Give You Away: The Surveillance Implications of Web Tracking. In *Proceedings of the 24th International Conference on World Wide Web*. International World Wide Web Conferences Steering Committee, Florence Italy, 289–299. doi:10.1145/2736277.2741679

[31] Yoni Epstein. 2024. Taboola Tech is Built for A Cookieless World | Taboola.com - EN. https://www.taboola.com/taboola-updates/taboola-tech-is-built-for-a cookieless-world/

[32] Motahhare Eslami, Sneha R Krishna Kumaran, Christian Sandvig, and Karrie Karahalios. 2018. Communicating algorithmic process in online behavioral advertising. In *Proceedings of the 2018 CHI conference on human factors in computing systems*. 1–13.

[33] IAB Europe. 2021. THE IAB EUROPE GUIDE TO CONTEXTUAL ADVERTISING. https://iabeurope.eu/wp-content/uploads/2021/07/IAB-Europe-Guide-to-Contextual-Advertising-July-2021.pdf

[34] Florian M Farke, David G Balash, Maximilian Golla, Markus Dürmuth, and Adam J Aviv. 2021. Are privacy dashboards good for end users? Evaluating user perceptions and reactions to Google's My Activity. In *30th USENIX Security Symposium (USENIX Security 21)*. 483–500.

[35] Franz Faul, Edgar Erdfelder, Axel Buchner, and Albert-Georg Lang. 2009. Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. *Behavior research methods* 41, 4 (2009), 1149–1160.

[36] John Ford, Varsha Jain, Ketan Wadhwani, and Damini Goyal Gupta. 2023. AI advertising: An overview and guidelines. *Journal of Business Research* 166 (Nov. 2023).
2023), 114124. doi:10.1016/j.jbusres.2023.114124

[37] José González-Cabañas, Ángel Cuevas, Rubén Cuevas, Juan López-Fernández, and David García. 2021. Unique on Facebook: formulation and evidence of (nano) targeting individual users with non-PII data. In *Proceedings of the 21st ACM Internet Measurement Conference*. 464–479.

[38] Chris Jenkins. 2023. Google's Privacy Sandbox commitments: Implementation and what comes next – Competition and Markets Authority. https://competitionandmarkets.blog.gov.uk/2023/04/28/googles-privacy-sandbox-commitments-implementation-and-what-comes-next/

[39] Nikhil Jha, Martino Trevisan, Emilio Leonardi, and Marco Mellia. 2023. On the Robustness of Topics API to a Re-Identification Attack. http://arxiv.org/abs/2306.05094 arXiv:2306.05094 [cs].

[40] Garrett Johnson. 2025. Unearthing privacy-enhancing ad technologies (peat): The adoption of google's privacy sandbox. *Available at SSRN* 4983927 (2025).

[41] Konrad Kollnig, Anastasia Shuba, Max Van Kleek, Reuben Binns, and Nigel Shadbolt. 2022. Goodbye tracking? Impact of iOS app tracking transparency and privacy labels. In *Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency*. 508–520.

[42] IAB Tech Lab. 2017. ads.txt - Authorized Digital Sellers. https://iabtechlab.com/ads-txt/

[43] Bin Liu, Anmol Sheth, Udi Weinsberg, Jaideep Chandrashekar, and Ramesh Govindan. 2013. Adreval: Improving transparency into online targeted advertising. In *Proceedings of the Twelfth ACM Workshop on Hot Topics in Networks*. 1–7.

[44] Zengrui Liu, Jimmy Dani, Shujiang Wu, Yinzhi Cao, and Nitesh Saxena. 2024. Identified-and-Targeted: The First Early Evidence of the Privacy-Invasive Use of Browser Fingerprinting for Online Tracking. http://arxiv.org/abs/2409.15656 arXiv:2409.15656.

[45] Minjun Long and David Evans. 2024. Evaluating Google's Protected Audience Protocol. *Proceedings on Privacy Enhancing Technologies* 2024, 4 (Oct. 2024), 892–906. doi:10.56553/popets-2024-0147

[46] Célestin Matte, Natalia Bielova, and Cristiana Santos. 2020. Do cookie banners respect my choice?: Measuring legal compliance of banners from iab europe's transparency and consent framework. In *2020 IEEE Symposium on Security and Privacy (SP)*. IEEE, 791–809.

[47] Perry Mike, Clark Erinn, Murdoch Steven, and Koppen Georg. 2019. The Design and Implementation of the Tor Browser [DRAFT]. https://2019.www.torproject.org/projects/torbrowser/design/#identifier-linkability

[48] Mozilla. 2022. Firefox Rolls Out Total Cookie Protection By Default. https://blog.mozilla.org/en/mozilla-rolls-out-total cookie-protection-by-default-to-all-users-worldwide/

[49] Emmanouil Papadogiannakis, Panagiotis Papadopoulos, Nicolas Kourtellis, and Evangelos P. Markatos. 2021. User Tracking in the Post cookie Era: How Websites Bypass GDPR Consent to Track Users. In *Proceedings of the Web Conference 2021 (WWW '21)*. Association for Computing Machinery, New York, NY, USA, 2130–2141. doi:10.1145/3442381.3450056

[50] Michiel Philipse. 2024. Post-Third-Party Cookies: Analyzing Google's Protected Audience API. https://www.cs.ru.nl/masters-theses/2024/M Philipse__Post-Third-Party_Cookies_Analyzing_Google's_Protected_Audience_API.pdf

[51] Omid Rafieian and Hema Yoganarasimhan. 2021. Targeting and privacy in mobile advertising. *Marketing Science* 40, 2 (2021), 193–218.

[52] Nathan Reitinger, Bruce Wen, Michelle Mazurek, and Blase Ur. 2024. What Does It Mean to Be Creepy? Responses to Visualizations of Personal Broadcasting Activity, Online Tracking, and Targeted Ads. *Proceedings on Privacy Enhancing Technologies* 2024, 3 (July 2024), 715–743. doi:10.56553/popets-2024-0101

[53] Filipe N Ribeiro, Koustuv Saha, Mahmoudreza Babaei, Lucas Henrique, Johnnatan Messias, Fabricio Benevenuto, Oana Goga, Krishna P Gummadi, and Elissa M Redmiles. 2019. On microtargeting socially divisive ads: A case study of russia-linked ad campaigns on facebook. In *Proceedings of the conference on fairness, accountability, and transparency*. 140–149.

[54] Johnny Ryan. 2025. EU ruling: tracking-based advertising by Google, Microsoft, Amazon, X, across Europe has no legal basis - Irish Council for Civil Liberties. https://www.iccl.ie/digital-data/eu-ruling-tracking-based-advertising-by-google-microsoft-amazon-x-across-europe-has-no-legal-basis/

[55] Privacy Sandbox. 2022. Protected Audience API overview | Privacy Sandbox. https://privacysandbox.google.com/private-advertising/protected-audience

[56] Privacy Sandbox. 2024. Chrome-facilitated testing | Privacy Sandbox | Google for Developers. https://privacysandbox.google.com/private-advertising/setup/web/chrome-facilitated-testing

[57] Privacy Sandbox. 2024. Third-party cookie restrictions | Privacy Sandbox. https://privacysandbox.google.com/cookies/prepare/overview

[58] Privacy Sandbox. 2024. Third-party cookie restrictions | Privacy Sandbox - Test with temporary mitigations blocked. https://privacysandbox.google.com/cookies/prepare/test-for-breakage#test_with_temporary_mitigations_blocks

[59] Privacy Sandbox. 2024. Topics API for Web | Privacy Sandbox. https://privacysandbox.google.com/private-advertising/topics/web

[60] Privacy Sandbox. 2025. Overview of Attribution Reporting API | Privacy Sandbox. https://privacysandbox.google.com/private-advertising/attribution-reporting

[61] Privacy Sandbox. 2025. Privacy Sandbox initiative and Ad Manager - Google Ad Manager Help - support.google.com. https://support.google.com/admanager/answer/12270545?hl=en&ref_topic=12264880&dark=0&sjid=4025224594427094127-EU.

[62] Piotr Sapiecznyski, Levi Kaplan, Alan Mislove, and Aleksandra Korolova. 2024. On the Use of Proxies in Political Ad Targeting. *Proceedings of the ACM on Human-Computer Interaction* 8, CSCW2 (2024), 1–31.

[63] Florian Schaub, Aditya Marella, Pranshu Kalvani, Blase Ur, Chao Pan, Emily Forney, and Lorrie Faith Cranor. 2016. Watching Them Watching Me: Browser Extensions Impact on User Privacy Awareness and Concern. In *Proceedings 2016 Workshop on Usable Security*, Vol. 10. Internet Society, San Diego, CA. doi:10.14722/usec.2016.23017

[64] Eduardo Schnadower Mustri, Idris Adjerid, and Alessandro Acquisti. 2023. Behavioral Advertising and Consumer Welfare: An Empirical Investigation. *SSRN Electronic Journal* (2023). doi:10.2139/ssrn.4398428

[65] Justin Schuh. 2019. Building a more private web. https://blog.google/products/chrome/building-a-more-private-web/

[66] ELIAS SELMAN. 2024. Criteo's Privacy Sandbox Market Testing | by ELIAS SELMAN | Criteo Tech Blog. https://techblog.criteo.com/criteos-privacy-sandbox-market-testing-2ec711fd4797

[67] ELIAS SELMAN. 2024. Criteo's Privacy Sandbox Market Testing | by ELIAS SELMAN | Criteo Tech Blog. https://techblog.criteo.com/criteos-privacy-sandbox-market-testing-2ec711fd4797

[68] Dominic Farolino Shivani Sharma. 2022. WICG/fenced-frame: Proposal for a strong boundary between a page and its embedded content. https://github.com/WICG/fenced-frame

[69] Brave Software. 2020. OK Google, don't delay real browser privacy until 2022 | Brave. https://brave.com/blog/ok-google/

[70] Till Speicher, Muhammad Ali, Giridhari Venkatadri, Filipe Nunes Ribeiro, George Arvanitakis, Fabricio Benevenuto, Krishna P Gummadi, Patrick Loiseau, and Alan Mislove. 2018. Potential for discrimination in online targeted advertising. In *Conference on fairness, accountability and transparency*. PMLR, 5–19.

[71] Oleksii Starov and Nick Nikiforakis. 2017. Extended Tracking Powers: Measuring the Privacy Diffusion Enabled by Browser Extensions. In *Proceedings of the 26th International Conference on World Wide Web*. International World Wide Web Conferences Steering Committee, Perth Australia, 1481–1490. doi:10.1145/3038912.3052596

[72] Jenny Tang, Eleanor Birrell, and Ada Lerner. 2022. Replication: How well do my results generalize now? The external validity of online privacy and security surveys. In *Eighteenth symposium on usable privacy and security (SOUPS 2022)*. 367–385.

[73] Dan Taylor. 2023. Results from Google Ads' interest-based advertising testing. https://blog.google/products/ads-commerce/iba-test-results/#:~:text=The%20experiment%20showed%20that%20when,party%2Dcookie%2Dbased%20results

[74] Vilma Todri. 2022. Frontiers: The impact of ad-blockers on online consumer behavior. *Marketing Science* 41, 1 (2022), 7–18. https://pubsonline.informs.org/doi/epdf/10.1287/mksc.2021.1309

[75] Blase Ur, Pedro Giovanni Leon, Lorrie Faith Cranor, Richard Shay, and Yang Wang. 2012. Smart, useful, scary, creepy: perceptions of online behavioral advertising. In *Proceedings of the Eighth Symposium on Usable Privacy and Security*. ACM, Washington, D.C., 1–15. doi:10.1145/2335356.2335362

[76] Yash Vekaria, Yohan Beugin, Shaoor Munir, Gunes Acar, Natalia Bielova, Steven Englehardt, Umar Iqbal, Alexandros Kapravelos, Pierre Laperdrix, Nick Nikiforakis, et al. 2025. SoK: Advances and Open Problems in Web Tracking. *arXiv preprint arXiv:2506.14057* (2025).

[77] Alberto Verna, Nikhil Jha, Martino Trevisan, and Marco Mellia. 2024. A First View of Topics API Usage in the Wild. In *Proceedings of the 20th International Conference on emerging Networking Experiments and Technologies*. ACM, Los Angeles CA USA, 48–54. doi:10.1145/3680121.3697810

[78] Miranda Wei, Madison Stamos, Sophie Veys, Nathan Reitinger, Justin Goodman, Margot Herman, Dorota Filipczuk, Ben Weinshel, Michelle L Mazurek, and Blase Ur. 2020. What Twitter knows: Characterizing ad targeting practices, user perceptions, and ad explanations through users' own Twitter data. In *29th USENIX Security Symposium (USENIX Security 20)*. 145–162.

[79] Ben Weinshel, Miranda Wei, Mainack Mondal, Euirim Choi, Shawn Shan, Claire Dolin, Michelle L. Mazurek, and Blase Ur. 2019. Oh, the Places You've Been! User Reactions to Longitudinal Transparency About Third-Party Web Tracking and Inferencing. In *Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security*. ACM, London United Kingdom, 149–166. doi:10.1145/3319535.3363200

[80] Maximiliane Windl, Roman Amberg, and Thomas Kosch. 2025. The Illusion of Privacy: Investigating User Misperceptions in Browser Tracking Protection. In *Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems*. ACM, Yokohama Japan, 1–10. doi:10.1145/3706598.3713912

[81] Eric Zeng, Tadayoshi Kohno, and Franziska Roesner. 2021. What Makes a “Bad” Ad? User Perceptions of Problematic Online Advertising. In *Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems*. ACM, Yokohama
Japan, 1–24. doi:10.1145/3411764.3445459

# A Methodology materials

## A.1 CMA's Evaluation Framework

The CMA's proposed experimental designs aim to compare selected markets, such as outcomes for subsamples of impressions served using Privacy Sandbox technologies ('treatment impressions') against a subsample of ads served under counterfactual scenarios ('counterfactual impressions' or 'control impressions') [22, 23]. More precisely, to make such comparisons, the CMA defines the following testing groups: (1) *Treatment group*. ads served using only the Privacy Sandbox APIs, without any third-party cookie. (2) *Control 1 group*. (the 'Status quo'). ads served with data related to third-party cookies and removing data related to new Privacy Sandbox APIs (business as usual). (3) *Control 2 group*. ads served based on only first-party data, i.e. neither the Privacy Sandbox tools nor third-party cookies-related data are used. Comparing ads served with and without the Privacy Sandbox requires a mechanism for allocating

ads to Treatment and Control groups. To do so, the CMA proposed two experimental designs. *Design 1* allows participating ad tech companies and publishers to independently segment their own treatment and control groups by randomly enabling or disabling third-party cookies or Privacy Sandbox technologies in subsets of ad requests. This approach was taken by Google Ads in its Topics API experiment [73]. However, this design does not guarantee that only data from the Privacy Sandbox APIs is used in auctions for the treatment group, as third-party cookies' data may still be available unless blocked at the browser level(e.g., a DSP could bid on the treatment group ads using cookies obtained from outside the auctions). Alternatively, in *Design 2*, the Chrome browser creates Treatment and Control groups by randomly enabling or disabling third-party cookies and Privacy Sandbox technologies at the browser level, i.e., third-party cookies are completely blocked. Starting 2024, Google deprecated third-party cookies for 1% of randomly selected traffic. In both designs, SSPs and DSPs rely on Chrome's experimental labels to get labels within the ad request to identify the experimental group of the user [21, 38]. However, these evaluation setups are designed for entities like DSPs and SSPs (or advertisers and publishers, respectively) that directly access ad requests, bidding, and auction signals across their ad network, and cannot be used by third-party observers. Our initial plan to implement this design by registering as a publisher and advertiser on Google Ad Manager was unsuccessful, as our requests were rejected due to Google's strict quality standards.

<footnote>[82] Eric Zeng, Rachel McAmis, Tadayoshi Kohno, and Franziska Roesner. 2022. What factors affect targeting and bids in online advertising?: a field measurement study. In *Proceedings of the 22nd ACM Internet Measurement Conference*. ACM, Nice France, 210–229. doi:10.1145/3517745.3561460</footnote>

<footnote>[83] Eric Zeng, Xiaoyuan Wu, Emily N Ertmann, Lily Huang, Danielle F Johnson, Anusha T Mehendale, Brandon T Tang, Karolina Zhukoff, Michael Adjei-Poku, Lujo Bauer, et al. 2025. Measuring Risks to Users' Health Privacy Posed by Third-Party Web Tracking and Targeted Advertising. In *Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems*. 1–26.</footnote>
## A.2 Study website screenshots

<figure><img src="image_5.png" /><figcaption>Figure 2: A screenshot to set the flag “Test Third Party Cookie Phaseout” from the Chrome-facilitated testing to join a testing group.</figcaption></figure>

<figure><img src="image_6.png" /><figcaption>Figure 3: A screenshot of the explanation provided to participants when they click on the “Learn More” button before setting Chrome feature flags. This is an example of the *Privacy Sandbox* group setup.</figcaption></figure>
<figure><img src="image_7.png" /><figcaption>Figure 4: A screenshot of the configuration page of the Chrome feature flags for the *Privacy Sandbox* testing group.</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>Figure 5: A screenshot of the ad perception survey as presented to participants.</figcaption></figure>
## A.3 Privacy Sandbox JS Attributes for ad targeting

For Protected Audience API:
* `V8Navigator_RunAdAuction_Method`
* `V8Navigator_CreateAuctionNonce_Method`
* `FledgeAuctionReportBuyers`
* `FledgeAuctionReportBuyerDebugModeConfig`, and for Topics API:
* `TopicsAPIAll`
* `TopicsAPI_BrowsingTopics_Method`
* `TopicsAPIFetch`
* `IframeBrowsingTopicsAttribute`,

## A.4 Publishers List

* https://weather.com/
* https://apnews.com/
* https://blackfriday.com/
* https://letterboxd.com/
* https://www.businessinsider.com/
* https://edition.cnn.com
* https://www.forbes.com/
* https://best-hashtags.com/
* https://www clickthecity.com/
* https://www.gmanetwork.com/
* https://www.harpersbazaar.com/
* https://www.livemint.com/
* https://www.ndtv.com/
* https://www.thesaurus.com/
# B Results

## B.1 Recruited participants

<table><caption>Table 4: Participant demographics for our study. The 'Ad Blocker Usage' columns show the percentage of participants within each demographic group for whom our automated script detected an ad blocker.</caption><thead><tr><th>Demographic Categories</th><th>participants</th><th>Ad Blocker Usage</th></tr></thead><tbody><tr><td><strong>Gender</strong></td><td></td><td></td></tr><tr><td>Male</td><td>53%</td><td>19.32%</td></tr><tr><td>Female</td><td>42.55%</td><td>7.85%</td></tr><tr><td>Prefer not to say</td><td>0.45%</td><td>-</td></tr><tr><td>No Data</td><td>4%</td><td>16.66%</td></tr><tr><td><strong>Age</strong></td><td></td><td></td></tr><tr><td>18-24</td><td>15.6%</td><td>18.57%</td></tr><tr><td>25-34</td><td>35.6%</td><td>15.62%</td></tr><tr><td>35-44</td><td>23.4%</td><td>13.33%</td></tr><tr><td>45-54</td><td>12.9%</td><td>6.89%</td></tr><tr><td>55+</td><td>8.5%</td><td>13.15%</td></tr><tr><td>No Data</td><td>4%</td><td>16.66%</td></tr><tr><td><strong>Employment Status</strong></td><td></td><td></td></tr><tr><td>Full-Time</td><td>52.33%</td><td>14.04%</td></tr><tr><td>Part-Time</td><td>16.92%</td><td>9.21%</td></tr><tr><td>Not in paid work</td><td>5.79%</td><td>11.53%</td></tr><tr><td>Unemployed</td><td>9.79%</td><td>20.45%</td></tr><tr><td>Other</td><td>1.78%</td><td>25%</td></tr><tr><td>No Data</td><td>4%</td><td>16.66%</td></tr><tr><td>Data Expired</td><td>9.13%</td><td>14.63%</td></tr><tr><td><strong>Student Status</strong></td><td></td><td></td></tr><tr><td>Yes</td><td>20.71%</td><td>16.12%</td></tr><tr><td>No</td><td>65.92%</td><td>13.51%</td></tr><tr><td>Data Expired</td><td>9.35%</td><td>14.28%</td></tr><tr><td>No Data</td><td>4%</td><td>16.66%</td></tr><tr><td><strong>Nationality</strong></td><td></td><td></td></tr><tr><td>United States</td><td>25.2%</td><td>8.9%</td></tr><tr><td>United Kingdom</td><td>39.2%</td><td>16.5%</td></tr><tr><td>Canada</td><td>8.24%</td><td>27.0%</td></tr><tr><td>Other</td><td>23.3%</td><td>-</td></tr><tr><td>No Data</td><td>4%</td><td>16.66%</td></tr></tbody></table>

## B.2 Ad exchanges in ads

<table><caption>Table 5: Distribution of ad exchanges serving ads during our experiment extracted from participants' submitted ads.</caption><thead><tr><th>Testing Group</th><th>Ad Exchange</th><th>domains</th><th>Frequency</th></tr></thead><tbody><tr><td rowspan="6">Privacy Sandbox</td><td>Google</td><td>doubleclick.net</td><td>63.25%</td></tr><tr><td>Taoola</td><td>taboola.com</td><td>3.61%</td></tr><tr><td>Google</td><td>googlesyndication.com</td><td>3.31%</td></tr><tr><td>Amazon</td><td>amazon.co.uk</td><td>1.81%</td></tr><tr><td>Others</td><td>-</td><td>28.01%</td></tr><tr><td>Google</td><td>doubleclick.net</td><td>37.14%</td></tr><tr><td rowspan="5">Status Quo</td><td>Google</td><td>googleadservices.com</td><td>23.87%</td></tr><tr><td>Google</td><td>googlesyndication.com</td><td>5.04%</td></tr><tr><td>Taboola</td><td>taboola.com</td><td>4.51%</td></tr><tr><td>Others</td><td>-</td><td>29.44%</td></tr><tr><td>Google</td><td>doubleclick.net</td><td>66.86%</td></tr><tr><td rowspan="4">Cookieless</td><td>Google</td><td>googlesyndication.com</td><td>2.35%</td></tr><tr><td>Taboola</td><td>taboola.com</td><td>2.35%</td></tr><tr><td>Google</td><td>2mdn.net</td><td>1.17%</td></tr><tr><td>Others</td><td>-</td><td>27.27%</td></tr></tbody></table>