Privacy Rarely Considered: Exploring Considerations in the
Adoption of Third-Party Services by Websites
Christine Utz

Sabrina Amft

Martin Degeling

CISPA Helmholtz Center for
Information Security
Saarbrücken, Germany
christine.utz@cispa.de

CISPA Helmholtz Center for
Information Security
Hannover, Germany
sabrina.amft@cispa.de

Ruhr University Bochum
Bochum, Germany
martin.degeling@rub.de

Thorsten Holz

Sascha Fahl

Florian Schaub

CISPA Helmholtz Center for
Information Security
Saarbrücken, Germany
holz@cispa.de

CISPA Helmholtz Center for
Information Security
Hannover, Germany
fahl@cispa.de

University of Michigan
School of Information
Ann Arbor, Michigan, USA
fschaub@umich.edu

ABSTRACT

online marketing campaigns. User expectations regarding the lookand functionality of websites, paired with time and resource constraints in web development, were also found to drive the adoptionof third-party resources [19], such as design frameworks, contact
forms, and external media hosting. This reliance on third parties
can come at the cost of website visitors’ privacy. By embedding
external resources, websites provide third-party vendors with theopportunity to collect personal data about the website’s visitors,
such as their IP address, visited pages, and access to long-term identifiers the third party may have stored in visitors’ browsers [50].
This data collection potentially allows them to track people acrossthe Web, learn large shares of their browsing histories, and use thisinformation to infer interests or demographics.
Considering that third-party resources are often automatically
retrieved in the background without visible indication, this may beat odds with privacy legislation. For example, the European Union’s
General Data Protection Regulation (GDPR) [20], in effect since
2018, demands that processing of personal data is grounded on oneof six legal bases, including user consent, is transparently communicated, and a “privacy by design and by default” approach is followed.
Privacy risks of third-party website resources have been pointed
out by courts and technical guides, noting, for example, that use of
the most prevalent third-party service [16, 34, 39], Google Analytics,
is only compliant with privacy law with IP anonymization [80]. Recent years have also seen the introduction of more privacy-friendly
ways to embed externally hosted media or social media functionality [30, 31]. Still, post-GDPR measurements have shown littlechange in the prevalence of third-party web tracking [14, 76, 88],
and practices that are already “quite pervasive” [19] may be hard tochange. In early 2022, several European courts and data protection
authorities have directed attention towards the privacy implications of third-party use through decisions that declared the use ofcertain services a GDPR violation: the Austrian and French dataprotection boards for Google Analytics [59, 67], the Belgian one for
IAB Europe’s Transparency and Consent Framework (TCF), the basis for many third-party consent providers [7], and a German court
for Google Fonts [43], with more decisions expected to follow [8].
Website creators are a crucial part of the third-party tracking
ecosystem, as it is them who integrate third parties into websites
and enable them to track visitors’ behavior across the Web. Thus,

Modern websites frequently use and embed third-party services tofacilitate web development, connect to social media, or for monetization. This often introduces privacy issues as the inclusion of
third-party services on a website can allow the third party to collectpersonal data about the website’s visitors. While the prevalence andmechanisms of third-party web tracking have been widely studied,little is known about the decision processes that lead to websites
using third-party functionality and whether efforts are being madeto protect their visitors’ privacy.
We report results from an online survey with 395 participants
involved in the creation and maintenance of websites. For ten common website functionalities we investigated if privacy has played
a role in decisions about how the functionality is integrated, if
specific efforts for privacy protection have been made during integration, and to what degree people are aware of data collection
through third parties. We find that ease of integration drives thirdparty adoption but visitor privacy is considered if there are legal
requirements or respective guidelines. Awareness of data collection
and privacy risks is higher if the collection is directly associatedwith the purpose for which the third-party service is used.

KEYWORDS
Web privacy, web tracking, third parties, survey.

1

INTRODUCTION

Contemporary websites often use third-party services for certain
functionality, design, or media resources. The underlying reasonsare as multifaceted as the purposes for which external resources areused in web development. Web content is often monetized via online advertising and marketing [50], which frequently involves theinclusion of advertising networks to target ads to website visitors’
presumed interests and web analytics to measure the success ofThis work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license
visit https://creativecommons.org/licenses/by/4.0/ or send a
letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.
Proceedings on Privacy Enhancing Technologies YYYY(X), 1–25
© YYYY Copyright held by the owner/author(s).
https://doi.org/XXXXXXX.XXXXXXX
1

Proceedings on Privacy Enhancing Technologies YYYY(X)

Christine Utz, Sabrina Amft, Martin Degeling, Thorsten Holz, Sascha Fahl, and Florian Schaub• We find higher awareness of data collection pertaining toa third-party service’s core functionality, such as financial
information for payment or behavioral data for analytics,
whereas awareness is lacking for data collected in less prominent contexts, particularly the transmission of IP addresses
and device information.
• From a methodological perspective we contribute to theongoing discussion about ethics in security and privacy research by discussing implications and lessons learned fromusing public GitHub data to recruit people involved with
web development, a method previously used by developercentered research [1, 2, 26, 56, 71, 73, 74, 81, 84, 92].
Our findings show the need for researchers and the web development community to raise awareness of the privacy risks associatedwith third-party use on websites, as well as the need for clearer regulatory guidance and requirements for privacy-friendly defaults.

the lack of change in third-party use on websites under the GDPR
raises the question to what extent people tasked with the creationand maintenance of websites are aware of the privacy risks ofthird-party use and if visitors’ privacy is considered both in thedecision that leads to the selection of third-party services and inintegration itself. Though prior work has studied the history [45, 91]
and prevalence [16] of third-party web tracking and its underlying
mechanisms, little is known about the decision processes behindthe use of third-party services on websites and if website visitors’
privacy is considered in the process.
Previous work that has studied developer behavior in adopting [65] and updating [71, 72] third-party libraries focused on smartphone apps, e. g., investigating developers’ privacy considerationsin their use of mobile advertising networks [55, 82], their awarenessof data collection through third-party tools for unspecified types
of functionality including ads and analytics [4], and their adoptionof alternative APIs that preserve location privacy [37]. Third-party
services and libraries for websites differ from those for the mobile
ecosystem in their availability for a greater variety of purposes, thepotential for higher technical complexity, and higher sophisticationof advertising ecosystems [36, 46, 87]. Websites also lack apps’ distribution through a centralized platform, whose requirements may
shape developers’ understanding of privacy aspects, including what
data is considered sensitive [85]. On the Web, the omnipresence ofconsent notices that implement IAB Europe’s TCF [32] and often
list a site’s third-party vendors could have led to higher awarenessof data collection through third parties on websites compared to themobile space, where consent prompts are much less prevalent [41].
In this work, we address this research gap with findings from amixed-methods online study with 395 participants involved in thedesign, development, deployment, maintenance, or management of
websites. We combine survey answers with web privacy measurements and investigate how ten website functionalities frequently
associated with use of third-party services have been integrated into
websites and how visitors’ privacy was considered in the process.
We go beyond prior work by exploring privacy considerations between different types of functionality that may not be equally prone
to third-party use [50], as well as factors that influence the adoptionof first- vs. third-party solutions to integrate a functionality. Morespecifically, we make the following contributions:• We extend web privacy research on the prevalence of thirdparty services by contrasting their use with first-party integrations for different purposes, regarding their prevalence,factors that drive use of first vs. third-party solutions, andconsideration of alternatives. We find that the decision infavor of third-party services, as in the mobile domain [71],is driven by ease of integration, features, cost, and familiarity with a service, while privacy rarely is a decisive factor.
However, we find use of privacy-friendly integration forweb analytics and programming/design resources, and selfhosting tends to be the primarily considered alternative tothird-party solutions, rather than another third party.
• Like work on cryptographic APIs [1] and mobile ad networks [55], we find that changes to a service’s default configuration are rarely reported. However, participants who did
adjust defaults often did so in response to privacy-related
court rulings or guidelines by data protection authorities.

2

THIRD-PARTY SERVICES IN WEB
DEVELOPMENT

Advantages of third-party use in web development differ by actor:
Web developers benefit from ease of integration as often all that isrequired is to copy and paste HTML or JavaScript snippets from thevendor’s website [69]; potentially faster website load times throughuse of content delivery networks (CDNs) or caching in visitors’browsers if widely used [66, 69]; and the fact that many popular
third-party services are available free of charge. The latter often
comes at the cost of the third-party vendor collecting data about
the website’s visitors for monetization through advertising [19, 50].
Independent of the functionality a third-party service provides tothe website, requesting a remotely hosted resource via HTTP inherently involves the transmission of the website visitors’ IP address,
which some jurisdictions consider personal information [17], to theremote server, along with device information in the browser’s useragent and the currently viewed page. The third party can use theseto infer additional information about individuals, such as otherwebsites they visit that also include the third-party service [49]. A
mitigation is to host the remote resource locally, if possible [19, 50].
Other privacy risks and mitigations depend on the type of functionality provided. As our study is centered around common use
cases for third-party services in web development, we started by
identifying these through review and comparison of existing categorizations in the literature and by web tracking projects. We foundsuch classifications in the works of Sørensen and Kosta [76], Libert
and Nielsen [50], by WhoTracks.me [38], Third Party Web [34, 35],
and DuckDuckGo’s Tracker Radar [15]. While categorizations differ in granularity and focus, we identified large overlap from theperspective of website owners. We did not consider categories that
apply only in a first-party context (e. g., hosting, distribution) or
only make sense combined with other categories (e. g., tag management). We ended up with ten common website functionalities,
shown in Table 1 with associated privacy risks and possible mitigations. The latter are generally possible on two levels: selection how
to integrate the desired functionality (self-implemented, locally or
remotely hosted third-party service) and efforts in integration of
the selected solution to configure it in a more privacy-friendly way.
2

Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websites

Proceedings on Privacy Enhancing Technologies YYYY(X)

Table 1: Categories of website functionalities included in this study for which use of third-party (3P) services is common.
Specific privacy risks1

Possible alternatives2

Functionality Definition

Popular 3P solution(s)Advertising

Advertising for third-party goods or services
to generate revenue for the website.

Google AdSense, Amazon Advertis- Targeting and profiling based on browsing be- Staticorcontext-based
ing, Criteo, Taboola, Outbrain
havior and device info; data sharing w. large ads [19], affiliate links,
advertising ecosystems
sponsored content

Analytics

Measurement of visitors’ behavior to evaluate
website performance and marketing success.

Google Analytics, Scorecard Re- Extensive data collection; data sharing withsearch, New Relic, Yandex
others (e. g., ad networks); tracking of browsing behavior across the Web due to widespread use [16, 34, 39]

Embedded me- Non-text content (e. g., videos, audio files,diaslideshows, interactive maps) embedded into
web pages.

Hosting: YouTube (videos), Google
Maps (maps); embedding code by
hosting 3PConfig. to collect less data [25];services that collect less dataor can be self-hosted (e. g.,
Matomo) [19]Data transmitted upon page load, not only Self-hosting, twoupon interaction with remotely hosted em- click solutions [31],bedded content
YouTube-nocookie [19]Customerinteraction

Mechanisms that enable specific website– Google Forms, Facebook Comments, Various personal data transmitted; leakage Plugins for content managevisitor interactions (e. g., contact forms, com- Disqusof this data to third parties, including ad net- ment system (CMS)
ments, chat).
works, even before submission [75, 79]; Disqus: data sharing with ad networks by default
without notice [6, 28]

User login /authentication

Allows users to create accounts on the web- Single-sign on with credentials
site and log in.
from popular services (e. g., Apple,
Google, Twitter, Facebook)

Payment

Allows visitors to pay for services and goods
offered on the website.

Varies between regions [9], e. g., Pay- Sharing of sensitive personal and financial inPal, Venmo, Alipay
formation with payment provider [68] andpossibly other 3Ps uninvolved in transaction [62]Limited by prevalence andpracticality; pure 1P: cash, giftcards; only banks: direct banktransfer

Privacy
notices /formsInterface elements that help fulfill transparency, consent, and opt-out requirementsfrom privacy laws (e. g., GDPR / ePrivacy Directive in EU, CCPA in US).

Consent Management Providers Not always correctly implemented, so visitor(CMPs) implementing compli- data is collected without prior consent [52];ance frameworks by the Internet frequent use of dark patterns [58, 89]
Advertising Bureau (IAB)

Self-implementation [14]; ensuring proper integration withcritical website features

Programming/ design

Programming frameworks and design re- Google Fonts, jQuery, Bootstrap
sources (e. g., web fonts, CSS / JS libraries).

Only general risks1Socialmedia
integration

Interface elements that connect a website
with social media (SM) services (e. g., link to
the website’s SM profile, SM share buttons,
embedded SM feeds).

Data transmission upon page load; in EU, li- Limited (3P by definition reability of site owners for data processing by quired) – two-click mechaSM companies through buttons/widgets [5] nisms [30, 31, 61], static profile
links

Websiteprotection

Mechanisms to protect against (distributed) Google reCAPTCHA, services
denial-of-service attacks, spam, or data scrap- based on text / behavioral analying.
sis, security proxies (Cloudflare)

Code provided by SM service (e. g.,
Facebook, Twitter, Instagram)

Providers can learn on which other sites peo- CMS-provided
integration,ple use their credentials and when [40]privacy-friendly
identity
providersWide range of behavioral data collected to
distinguish humans from bots [13, 19, 60]

Self-hosting [19, 66, 69]

Against non-targeted spam:honeypots, easy math or language questions [13]1 General risks are (i) transmission of visitors’ IP address and user agent to the third-party service, which can allow the latter to track people across the Web, especially if the service

is widely used [49]; and (ii) the third party potentially requiring visitors to accept extensive privacy policies [13, 19].
2 Always viable are self-implementation (except for payment and some social media integration) and using a third-party service that collects less personal information.

3

RELATED WORK

companies collect less data from the latter [11], previous researchoverall did not find significant positive changes due to the GDPR.

Previous work has studied the prevalence and evolution of thirdparty web tracking and developers’ privacy behaviors in third-party
use in the mobile app ecosystem.

Developers’ Privacy Considerations. Developers’ considerationsof users’ privacy have been studied in different contexts, but there
are few insights into why specific third-party services are usedin web development. Previous work found that developers of mobile apps are often unaware of third-party data collection [4], andtherefore tend to collect more data than necessary. Furthermore,developers showed a limited perception of privacy threats, often
based on their organization’s guidelines [29]. Mhaidli et al. investigated how and why mobile app developers use and choose ad
networks and whether they consider associated risks for users [55].
They found that developers see advertisements as the only viable
way to monetize their apps and consider ad networks to be responsible for protecting app users’ privacy, not themselves. Tahaei etal. confirmed this and showed that app developers find existing
privacy information and controls confusing and hard to use [82].
Other studies investigated public forums to see how developers deal
with privacy regulations and changes to them, finding that they

Evolution of Third-Party Web Tracking. Web tracking has beenstudied extensively, including the prevalence of third-party tracking
services on websites. Tracking has been identified since 1996, andsince then increased in prevalence and complexity [45], with themost popular services covering up to 75 % of websites in 2015 [91]and hundreds of different known tracking services [70] whose
use increases with website popularity, and visible differences between regions and website types [33]. Large-scale investigations
confirmed that more than half of websites leak user data or load
third-party scripts [49]. The GDPR going into effect in May 2018
increased the prevalence of cookie consent notices, while actual
tracking practices did not change much [14] or could not be directly
attributed to the GDPR [76]. While there were clear differences between website visits from US or European users, implying that3

Proceedings on Privacy Enhancing Technologies YYYY(X)

Christine Utz, Sabrina Amft, Martin Degeling, Thorsten Holz, Sascha Fahl, and Florian Schaub

mostly try to uphold standards defined by large companies [85]or are focused on recent changes or events [48] when discussing
privacy. When asked to solve privacy-focused tasks, developers
tend to use better-documented alternatives and copy examples,
which could be adopted by privacy-friendly services [37]. They
often struggle with embedding privacy into their application due
to a lack of knowledge, privacy contradicting app requirements, ortask complexity [63, 74]. Another problem are third-party vendors’
competing business interests, leading them to employ dark patterns
that steer developers towards privacy-unfriendly defaults [83].

4sources, and the people involved. Part 4 explored participants’ understanding of the data collected through third-party services andefforts made to protect visitors’ privacy in the integration process.
Finally, Part 5 asked demographic questions and if participants
had received training or educated themselves on data protection or
privacy. At the end, participants were debriefed about the study’s
privacy focus and given the option to either withdraw from thestudy or to submit their answers. Six participants withdrew here.
To assure survey quality, we first conducted “think-aloud” cognitive interviews with seven web developers and two content creators,
recruited via convenience sampling. After each interview, we addressed identified issues and repeated this process until no further
issues emerged. A pilot launch of the survey with 101 participants
recruited from GitHub (see Section 4.2) did not yield evidence ofany remaining issues, so we proceeded with data collection.

METHODTo investigate the privacy practices and decision processes behind
third-party use on websites, we conducted a mixed-methods study
consisting of an online survey with 395 people involved in thecreation and administration of websites, paired with an analysis ofparticipants’ websites, if provided in the survey.

4.1

4.2

Recruitment

Our recruitment approach was guided by the goal to obtain different perspectives on website functionality integration. We leveraged
two recruitment channels to reach a diverse sample: websites’ contact information to reach individuals in a range of website-relatedroles, and GitHub to reach web developers. People were eligible toparticipate if they were at least 18 years old, worked on websites
in some capacity (e. g., website design, development, deployment,
maintenance, management), and were comfortable taking the survey in English. Participation was voluntary and uncompensated.
To cover a diverse range of websites in recruitment, we searched
the top 100,000 popular website domains on the Tranco list1 [44]for email addresses related to a website’s technical administration.
We visited each domain on the Tranco 100K in October 2020 using
OpenWPM 0.13 [16] and searched the homepage for links assumed
to lead to subpages containing privacy policies, terms of service,
and contact information. We identified these using a list of key
phrases compiled through manual inspection of 10 websites randomly sampled for each of the top 20 website languages in the
Tranco list. We downloaded the corresponding subpages and thehomepage and searched them for email addresses with a regularexpression. Since websites often list contacts responsible for thecontent (e. g., editors on news pages, politicians on government
sites) rather than administration, we excluded subpages with morethan four email addresses. After removing duplicates, invalid email
addresses, and subpages with more than 4 addresses, we were left
with 109,862 unique email addresses for 53,496 websites.
Previous work studying web developers’ security and privacy
practices has used public GitHub repositories to recruit developers
on a large scale [1, 2, 26, 56, 71, 73, 74, 81, 84, 92]. We also used thisapproach because it allowed us to recruit people likely involved
with web development without hand-picking them, as would havebeen the case for one-by-one contact on platforms such as LinkedIn.
Though prior work is not always clear on where exactly on GitHub
users’ email addresses were collected (options include commit email
addresses and users’ profile pages), from discussions with authorsof some previous studies we know that the use of commit email
addresses is common. Following this previously used method, weanalyzed commits made into public GitHub repositories in August

Survey Design

Our survey was inspired by the work of Mhaidli et al. [55] and consisted of five parts. It was conducted in English and implemented
on a self-hosted LimeSurvey instance. To prevent early priming
about privacy, we framed the survey as exploring practices in theselection and use of web technologies on websites and only introduced questions about privacy and data collection practices inPart 4. Appendix A contains the full survey.
Part 1 assessed participants’ background regarding their work on
websites, including experience with the functionalities in Table 1.
To provide context for the rest of the survey, Part 2 asked participants to think of one specific website they had recently worked
on and to only keep this website in mind for subsequent questions.
Participants could optionally provide the website’s URL (Q2-0). The
survey consent form explained that this information would be used
to check which web technologies were present on the website. At
this point we investigated the methodological question if requiring
participants to provide a website had an effect on dropout rates: We
made Q2-0 mandatory for half of GitHub-recruited participants (seeSection 4.2) but could not find evidence that this had an impact on
dropout rates or willingness to provide a website. Part 2 proceeded
to ask about website metadata, including the country it was basedin, the participant’s role with regard to the website, and which of
the ten functionalities in Table 1 were present on the site (Q2-6).
To balance level of detail and survey length, we chose to display
more detailed questions only for up to three functionalities. For this,Q2-7 asked, for each functionality indicated to be present in Q2-6,
to what degree the participant had been involved in the decisionof how this functionality should be integrated (selection), in the
integration process itself, and in maintenance or management of
the integrated solution. From the functionalities for which any kindof involvement had been indicated, three were randomly selected,
for which Parts 3 and 4 would be shown.
Part 3 investigated how a functionality was integrated in terms offirst- vs. third-party solutions and, if applicable, embedding mechanism. It also asked about the underlying decision process including reasons for selection and considered alternatives, information

1 List from September 1, 2020 (https://tranco-list.eu/list/64WX).

4

Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websites

2020 to identify e-mail addresses of people working on websites, as
indicated by the respective commit including file extensions related
to web development (.js, .php, .css, .html, .htm). Anticipating a low
response rate, we sent invitations to 37,000 email addresses, inaddition to 12,000 contacted during pilot testing.

4.3

Proceedings on Privacy Enhancing Technologies YYYY(X)The method was established in the community [1, 2, 26, 56, 73,
74, 92], even post-GDPR [71, 81, 84], and had passed ethical or
IRB review at different universities in the US, Europe, Australia,
and at the NIST Human Subjects Protection Office. As such wefollowed established research practice at the time, as well as sought
consultation/approval regarding GDPR from two data protection
officers from different institutions, who independently concluded
the recruitment method to be covered by the GDPR’s researchprivilege. In hindsight, we agree with participants’ and the DPA’s
concerns regarding GitHub recruitment, which is why we decided
to fully discuss our experience in this paper. We consider this aspectof our work a valuable lesson learned for the community in how
legal or ethical assessment of established study methods can – andshould – evolve. Section 7.4 discusses implications for future work.
We want to stress that all participants whose data is reported
in this paper provided their information with informed consent,
obtained both at the beginning of the survey and at the end after
debriefing about the study’s privacy focus. The issue pointed out
by the DPA lies with the recruitment method, not with the data we
received from the willing and consenting survey participants.

Research Ethics

Prior to conducting the study we looked into opportunities for ethical and data protection review at our institutions. At the time thisstudy was designed, conducted, and evaluated, the authors wereaffiliated with Leibniz University Hannover (LUH) and Ruhr University Bochum (RUB), both located in Germany, and the Universityof Michigan (U-M) in the US. RUB only had an IRB for research inpsychology, which was not meant to be mandatorily consulted by
security and privacy researchers. LUH’s IRB only targeted project
proposals, not individual research papers. The co-author from U-M
did not directly work with raw response data or interact with participants and confirmed with U-M’s IRB that their oversight andapproval was therefore not required. Nevertheless, we followed
best practices for research conduct and transparency. To ensure
GDPR compliance of our study, we consulted RUB’s and LUH’s
data protection officers. They both independently considered ourstudy design and specifically the approach for GitHub recruitment
to be covered by the GDPR’s research privilege.
In Q2-2 we required some participants to provide the URL of
a website they had worked on, following Mhaidli et al.’s study
design [55]. We explained in the initial consent form that this data
would only be used to check the website for the presence of thirdparty services. Participants required to fill this field were able todrop out or proceed without penalty by entering arbitrary input.
Regarding recruitment, we carefully considered the implicationsof sending email invitations to website contacts and GitHub developers at a large scale. As mentioned above, the two consulted
DPOs considered this recruitment approach to be GDPR-compliant.
We contacted each email address only once (i. e., we did not send
any confirmations or reminders) and gave email recipients a oneclick option to opt-out of further contact. Still, we received a small
number of emails with negative sentiments from people who were
not aware that their public GitHub commits contained their email
address. Upon this feedback we put up a page on our institution’s
website that explained our study, why the GitHub-recruited recipient’s email address was visible in commits into public repositories,
and what steps could be taken to hide it. Despite these efforts, one
recipient filed a complaint with our state’s data protection authority, upon which we immediately stopped recruitment via GitHub,
rather than waiting for the outcome. Three months later the DPA
informed us that they did not consider the GDPR’s research privilege to apply, because GitHub users, who are often unaware of theircommit email addresses being publicly available, do not expect tobe contacted via these addresses for the purpose of scientific research. We discuss the concrete problem with GitHub’s mechanics
for email addresses in more detail in Section 7.4. The DPA advised
us to refrain from future recruitment via public GitHub commits
but did not take formal action.
When we designed and launched the study, ethical concernswith recruitment via public GitHub commits were not obvious:

4.4

Data Cleaning

Across all recruitment phases, 2,177 people opened the survey link,
667 proceeded past the welcome page, and 452 completed the survey. Out of these, we removed 41 that had not seen Parts 3 and 4
due to a lack of reported involvement, nine who selected contradictory levels of involvement, and seven who provided multiple
websites. To increase data quality, we examined response times.
Average completion time was 20:42 minutes. We did not observe
any suspicious patterns and thus did not remove any answers. This
left us with a total of 395 valid responses. Two authors inspected all
open-response “Other” answers and re-coded answers that matchedexisting closed-ended options after discussion and mutual agreement. For website analysis, one author inspected all provided URLs
(Q2-0) and removed all answers that were not URLs (e. g., “client
confidential”) or could not be resolved to a website.

4.5

Data Analysis

Two of the authors applied thematic analysis [10] to the answers to
open-ended questions. First they independently reviewed the data
to identify recurring themes and created individual codebook drafts
for each question. Next, they discussed these drafts and merged
them into a first joint codebook. All data was then jointly coded byboth researchers, who discussed problematic cases until an agreement was reached, which at times required refining codes’ definitions and scopes and, thus, revisiting previously coded answers. We
did not compute inter-rater reliability, as the number of responses
was small enough to not require splitting up between multiple researchers [54]. Each open-ended response could be assigned one ormore codes, as participants often mentioned more than one relevant
talking point. Appendix B contains the final codebooks.
To assess to which extent participants’ responses about websites’
integrated functionalities matched actual practice, we checked theprovided websites with OpenWPM [16]. For each provided URL,
we accessed the front page, searched it for links to subpages, andvisited up to 100 unique pages randomly selected from these to5

Proceedings on Privacy Enhancing Technologies YYYY(X)

Christine Utz, Sabrina Amft, Martin Degeling, Thorsten Holz, Sascha Fahl, and Florian Schaub

ensure we gained a complete picture [87]. We performed crawls
from Germany, California, and India to cover possible differences
between jurisdictions [11, 32, 90]. For each page, we collected all
HTTP(S) requests and compared the list of found third-party services with those mentioned in the respective survey response, using
the WhoTracks.me [39] categorization as a basis. Finally, we compiled metadata on the provided websites: top-level domains (TLDs),
website topics based on the McAfee Real-Time Database [53], andpopularity based on the same Tranco list we used for recruitment.
For data analysis we mainly rely on descriptive statistics because
the variance in response counts per website functionality would
cause statistical tests to often be underpowered. Where statisticaltests are appropriate and possible we used Fisher’s exact tests tocheck if differences between categories were significant and corrected for multiple tests with the Benjamini-Hochberg procedure.

5

employer training, courses at a university or school, and other nononline courses, including certifications such as CISSP. Table 6 inAppendix D has detailed data about participants’ demographics
and background in their work with websites.
5.1.2 Websites Provided by Participants. In Q2-0, we asked participants to provide a website they had recently worked on thatwould serve as a reference for Parts 3 and 4 of the survey. Datacleaning left us with 361 unique valid websites, for which we compiled descriptive statistics. The most frequently occurring TLDs
were .com, .org, and .de, followed by domains associated with web
development, such as .github.io or .dev. Thematic classifications by
McAfee were available for 264 (83.8 %) domains, the most common
being Business, Internet Services, and Education/Reference. 141 registered domains (44.8 %) appeared on the Tranco top 1-million list,
with a mean ranking of 104,767 (min 5, max 958,899, std 168,620.3,
median 46,695). Overall we find that participants mainly reported
international sites aimed at providing services or information, butalso a significant amount of smaller and/or personal sites hosted
on popular platforms and a multitude of other thematic categories,
creating a diverse sample of websites.
Participants named 72 different countries as the seat of the company behind the website (Q2-3). Coding of the open-ended answers
to Q2-4 revealed that the websites were mostly targeted at a global
or multi-regional audience; Table 7 in Appendix D also lists themost popular individual target regions. Almost half of the websites(44.8 %) were reported not to have a website-specific revenue model(Q2-5). On average they relied on 0.91 sources of revenue (std 1.03,
min 0, max 5, median 1). Most common were products/services
sold on websites (20.5 %), subscriptions/membership (17.5 %), andrevenue streams not explicitly listed in Q2-5 (14.4 %).
Table 7 in Appendix D contains the full website statistics.

RESULTS

Our results show that, as in other domains, user privacy is rarely
considered in web development. Yet, we do find influence of regulators’ guidelines for some types of functionality, and self-hostingis a prominently considered alternative to third-party use. We alsofind a widespread lack of awareness that third-party use implies
transmission of IP addresses and device metrics to the third party.

5.1

SampleWe first describe the sample of 395 participants and 361 websites
they provided to support the main part of the survey.
5.1.1 Participant Demographics and Background. Participants predominantly identified as men (85.1 %; Q5-2), are most frequently inthe 18–24 (33.4 %) or 25–34 (30.6 %) age ranges (Q5-1), and the majority holds a bachelor’s degree (35.2 %; Q5-3). Most reported degrees(Q5-4) were in technical fields, with the most common non-technicaldegree being in business/economics (10.4 %). This is consistent withdemographics surveys of people working with web technologies,
whose large majority are men, typically in the 24–34 age range,holding a bachelor’s degree in technical fields [12, 27, 78, 94].
Participants’ work with websites (Q1-2) was most frequently
in a full-time position (41.8 %), though freelancing and part-time
employment were also common, as was non-paid work (hobbyist
31.4 %). In the last three years, participants had mostly worked on
2–5 websites (43.8 %; Q1-1). As for previous experience with theten website functionalities (Q1-3), all but one participant reported
at least one functionality, with a mean of 5.28 (sd 2.37, median5). Experience with front-end programming or design libraries(83.0 %) and user login or authentication (80.5 %) was most common,
while the fewest participants had worked with privacy plugins(29.9 %) and advertising (23.0 %). Participants held on average 3.4
different website-specific roles (std 2.58, min 1, max 13, median 3;Q2-1) and most often worked as (web) developer, programmer, orsoftware engineer (85.3 %). Other frequently reported roles include
administrator/web operator, user experience design, content creatoror contributor, and product or project manager. Most participants
worked alone (35.7 %) or in teams of sizes 2–5 (35.7 %) (Q2-2). 42.0 %had received prior privacy training. The most common resources ofsuch training were self-study (38.6 % of participants with training),

5.2

Privacy Considerations in Selection

To find out if privacy played a role in the decision how to integrate
a desired functionality, we investigated what functionalities were
present on participants’ websites, whether they were integrated via
first- or third-party solutions, and the underlying decision process,including considered alternatives, consulted information sources,
and the people involved.
5.2.1 Integrated Functionalities. In Q2-6 we asked participants
which of the ten functionalities in Table 1 were present on theirwebsite. Participants’ websites used on average 5.2 of them (sd2.3, min. 1, max. 10, median 5). In its “present” column, Table 2
lists how often each functionality was mentioned. The numbers
show that the reported prevalence of functionalities differs greatly.
Most commonly used were programming or design resources (355
/ 89.9 % of websites), customer interaction tools (268 / 67.8 %), andweb analytics (251 / 63.5 %).
To assess the number of third parties the websites actually use,
we combined the data collected from three server locations to ensurethat no configurations dependent on visitors’ IP or region biased
our results. Out of 361 unique websites provided we were not able
to access 10. On average, each website contacted 6.2 third-party
domains (min 0, max 144, std 6.95, median 3) and 80 sites made norequests to third parties at all.
6

Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websites

Table 2: Reported functionalities on websites (Q2-6; 𝑛 =395), participants’ involvement with them (Q2-7; relative
to “present”), and, based on that, how often they were randomly assigned survey parts 3 and 4.

Advertising
Analytics
Customer Interaction
Embedded Media
Login/Auth.
Payment
Programming/Design
PrivacySocial Media
Website Protection

presentnParticipants’ involvement
sel.
int. maint. none
%
%
%
%

67
251
268
248
265
101
355
136
186
187

44.8
47.4
53.0
55.6
48.7
43.6
61.7
40.4
53.8
51.3

46.3
40.6
46.6
48.0
41.5
40.6
57.7
36.0
44.1
39.0

32.8
46.2
45.1
45.2
40.8
29.7
46.2
33.8
40.3
39.0

Last, we compared the hosting strategies against privacy-friendly
recommendations [19]. Table 3 lists results for selected services.
We found that for many common third-party services like analytics,
videos, and maps the main strategy was to embed the well-known
services. For example, 158 websites made use of Google Analytics,
while only 15 used the privacy-friendly alternative Matomo. Out of
those 15 another 4 were found to be using both, e. g., on subsites.
For more technical functionalities like programming and design resources we observed more variation in first- vs. third-party hosting.
While we found only six websites that used privacy-friendly fonthosting sites (such as Fork Awesome or Fontello [19]), 86 hosted
additional fonts on their own server. For the widely used web programming library jQuery the results were reversed: The majority(138) self-hosted the script, while 72 used CDNs to serve the files.
Again there were sites using both strategies, for example, when alibrary was used multiple times by different components or plugins.

assigned
n

26.9
17.1
10.8
9.7
17.4
26.7
8.7
30.9
16.7
24.6

25
126
138
141
137
37
235
57
101
70

n

%

Analytics

Google Analytics
Google Analytics w/ IP anonymization
Privacy-friendly (Matomo/Piwik)
Only privacy-friendly

158
24
15
11

45.0
6.8
4.3
3.1

Video

YouTubeVimeo
Privacy-friendly (YouTube-nocookie)
Only privacy-friendly

74
12
16
6

21.1
3.4
4.5
1.7

Maps

Google Maps
Privacy-friendly (OpenStreetMap)
Only privacy-friendly

38
3
2

10.8
0.9
0.6

DesignGoogle Fonts / Font Awesome
Privacy-friendly (3P-hosted)
Privacy-friendly (self-hosted)
Only self-hosted fonts

244
6
86
22

69.5
1.7
24.5
6.3

Progr.

Table 3: Prevalence of common third-party services used on
351 websites compared to privacy-friendly alternatives.
Integration Solution

jQuery from CDN
Privacy-friendly (self-hosted)
Only privacy-friendly

72
138
101

20.5
39.3
28.8

Proceedings on Privacy Enhancing Technologies YYYY(X)

5.2.2 Prevalence of First-Party vs. Third-Party Solutions. Q3-2 investigated how the different functionalities were integrated into
websites. We focused on the hosting location (first-party solution,
third-party software installed locally on the own system, or thirdparty service remotely included from vendor’s server). For embedded media and social media, we also investigated (Q3-2c/2d) how
remote resources were embedded into the website: via self-written
code, code provided by the third party, or an embedding method
provided by another third party (such as social media plugins that
support multiple social media sites). Figure 1 shows the prevalenceof each hosting and embedding type. We observe that websites
predominantly self-host solutions for customer interaction (usercomments, contact forms, chat, etc.), privacy popups and forms,
and embedded audio. Remotely hosted third-party solutions are
dominant for analytics, payment, and hosting of embedded video
and map content, while prevalence of the different hosting types
was more varied in the other categories.
As shown in Figure 1(b), remotely hosted media are typically embedded using the code provided by the hosting service. Social media
share buttons and embedded feeds, whose functionality implies therequirement to access an API provided by the social network, moreor equally often use one of the two third-party embedding variants.
By contrast, buttons or links to the website’s social media profiles,
which do not trigger an action specific to the social network, aremore frequently integrated via first-party solutions.
Q3-2 also asked participants to specify which concrete service thewebsite used. Coding revealed the following categories of functionalities to have a clear market leader: advertising (Google Ads / AdSense / DoubleClick for Publishers [63.6 % of participants who useda third party and provided an answer]), analytics (Google Analytics,
65.7 %, followed by Matomo, 10.3 %), embedded videos (YouTube,
90 %), embedded maps (Google Maps, 62.5 %). We observed a morevaried use of third-party services for programming and design resources (top 3: Bootstrap (18.2 %), React (17.5 %), jQuery (14.7 %)).
For website protection, participants equally often mentioned web
security libraries, which they considered self-hosted third-party
services, and Google’s reCAPTCHA as the most popular remote
third-party service (12.1 % for both).
Overall, our findings match expectations: Third-party use seems
more prevalent for website functionalities that (mostly) require

For 76 sites we found mismatches between Q2-6 responses andthird parties observed on the website. The most common observation was a request to Google’s advertising domain doubleclick.com(42 cases), followed by site analytics (14), CDNs (12), customer interaction (6), and embedded media (5). The rest belonged to otherfunctionalities not covered by the survey. The high prevalence ofrequests to advertising domains despite the fact that developers
had not reported the use of advertising – confirmed by manual
inspection – can be explained by third parties loading additional
services [87]. The majority of requests went to doubleclick.com,
contacted by locally hosted Google Analytics scripts. Other cases involved social media bookmarking services like AddThis or ShareThisthat contact various advertising domains.
In the other direction, 136 responses reported functionalities
for which website analysis did not find obvious requests to matching third parties. The majority of these cases concern scripts forcustomer interaction (64), embedded media (70), or social media
integration (46). Besides methodological limitations outlined in Section 6, the explanation was often that the functionality was hosted
locally, e. g., via CMS plugins, as reported in Section 5.2.2.
7

Proceedings on Privacy Enhancing Technologies YYYY(X)

Christine Utz, Sabrina Amft, Martin Degeling, Thorsten Holz, Sascha Fahl, and Florian Schaub

)
(2
0

)
SM

-F

ee

d

)

e(
58

ro

fil

re
(3
8

(5
)

ha

(4
9
eo
Em

b.
Em

Em

b.

M

Vi
d

ap

s(
33

)
ap

b.
Em

b.
Em

M

Vi
d

di
Au

s(
39

(7
4

)
o

eo

(2
2

7)
(1
3
h.
Em
b.

gi

n

/A

ut

en
t(
37

6)
Pa

tic
ly

An
a

ym

s(
12

(7
0

)

n
io
ct

Pr
o

te

ac

y

35
Pr
iv

Don't know
NaN

Lo

Pr

og

ra
m

m

in

g

(2

(1
n
io
ct

ra

(5
7

)

)
38

25
s(
Ad
te
In
C.

-S

0.0

SM

0.0

Other 3P
Other

io

0.2

Au
d

0.2

b.

0.4

)

0.4

)

0.6

)

0.6

)

0.8

)

1P
Hosting 3P-PDon't know
NaN

0.8

)

frequency(b) Integration type (embedding)

3P, 3P-hosted
Other

SM

(a) Integration type (hosting)
1P, self-hosted
3P, self-hosted / only banks

functionalities

functionalities

Figure 1: Integration type (Q3-2) for different website functionalities. Left: use of first- vs. third-party hosting; right: source ofembedding code for embedded media and social media integration. Numbers are relative to how often the respective question
had been displayed (see the survey logic in Appendix A; n values in x-axis labels).
third parties to be involved, such as payment services or social
media integration, or that were deemed to be complex to self-hostor implement, such as analytics or video and map resources [50].
As for the concrete third-party services used, web tracking research
has repeatedly identified Google’s services to be the most prevalent
third-party services on the Web [16, 38, 88]. Still, we measured
some efforts at privacy-friendly configuration of Google services.

policies were rarely consulted, except for payment, privacy plugins, and advertising (16.7 % for each). Figure 7 in Appendix C has
detailed numbers. This suggests that not even functionality where
people directly enter sensitive information, such as customer interaction, prompts developers to look up a third-party service’s
data processing practices. This could be due to the complexity andlength of these documents, which reinforces the need that thirdparty services present their key privacy practices in a condensed,
easy to understand, and accessible form [4].

5.2.3 Decision Process. Next, we investigated how people had arrived at these solutions to integrate different website functionalities.

Reasons for the Selection of Existing Solutions. Coding of the openended answers to Q3-3 identified reasons why the respective integration solutions had been selected for each functionality. Figure 2
investigates the reported reasons for two mutually exclusive groups:
purely self-hosted solutions, whether first-party or via a locally
hosted third party, where collected data is expected to stay on thewebsite’s host system, vs. solutions that only rely on remote thirdparty hosting and thus can involve information being sent to athird-party server. Figure 2(a) shows the prevalence of each codefor each of these integration types, aggregated across all functionalities. We find that the most prevalent decision factors for either
integration type are ease of integration and features, though theseplay a bigger role in the adoption of pure third-party solutions.
The “Other” category mainly comprises generic answers such as“I just like it” (P323-Social) or “it’s the best” (P188-Login), which
explains its relatively high prevalence. Beyond these general factors
for adoption, we observed that some mainly occurred for certain
functionalities, such as revenue for advertising, legal considerations
for privacy plugins, security for login/authentication, familiarity
for programming/design and analytics, and popularity for payment.
Privacy aspects were rarely mentioned, except for analytics (“Iwanted something very minimalistic, non-intrusive” [P353], “I careabout users privacy” [P83]). These observations confirm findingsin the mobile space that third-party adoption is driven by the goal

People Involved in the Selection Process. We learned about who
was involved in the selection process in two ways. For participants
involved in the selection of how to integrate a functionality (Q2-7),
we evaluated their roles with regard to the website (Q2-1). Acrossall categories, people involved in selection predominantly had technical roles. For given roles we also observed higher involvement in
the selection of functionalities that closely relate to that role, suchas customer support for customer interaction or sales for advertising. Q3-8 asked participants not involved in selection who had
made that decision. Here participants most frequently referred todevelopers, with the notable exception of privacy popups or forms,
for which the decision often lay with the legal team, data protection officers, or management. This is also the functionality whereparticipants reported the lowest involvement rates (see Table 2).
Figures 5 and 6 in Appendix C have details for both questions.
Resources Used for Selection. Across all categories, participants
mainly relied on official websites and documentation to select how
to integrate a given functionality (Q3-6); also frequently named
were the website’s team, online articles, and forums. The same
information sources were reported as most commonly consulted in
the selection of ad networks for mobile apps [55]. Also confirming
the findings of previous work [4, 55], terms of service or privacy
8

Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websites

(7
9

on

0.01

0.02

0.00

0.00

0.04

0.06

0.05

0.03

0.01

0.22

0.30

0.12

0.20

0.17

0.14

0.09

0.15

0.04

0.08

0.06

0.09

0.05

0.01

0.05

0.03

0.06

0.00

0.15

0.21

0.08

0.10

0.09

0.08

0.07

0.11

0.11

0.07

0.02

0.08

0.00

0.01

0.08

0.15

0.01

0.06

0.07

0.11

0.03

0.03

0.04

0.08

0.00

0.00

0.04

0.00

0.03

0.04

0.09

0.08

0.03

0.05

0.21

0.16

0.02

0.06

0.04

0.00

0.02

0.03

0.02

0.09

0.01

0.02

0.02

0.05

0.02

0.00

0.02

0.00

0.05

0.00

0.03

0.00

0.01

0.00

0.03

0.00

0.14

0.11

0.19

0.08

0.16

0.20

0.28

0.16

0.30

0.27

0.21

0.19

(c) Reasons not
to consider
alternatives

0.5

0.4

0.3

0.2

frequency of codes

0.01

0.06

(b) Reasons
to consider
alternatives

ly

on
ly
P

P

e3

Re
m

ot

lo
or
1P

0.01

(a) Reasons
to select
given solution

)

09
)
)
ca
l3

P
e3

Re
m

ot

(1

59
)
(6
0

on

P
ca
l3
lo

1P

or

ly

on
ly

(1

9)

on
P

P
ot

e3

ca
l3
lo

Re
m

or
1P
codes

Revenue
Performance
Ease of Integration
Ease of use
Customization
Features
Cost
Resources
Popularity
Availability
Familiarity
Privacy
Security
Dependence
Legal
Other
No answer

ly

on
ly

(1
0

(2

82
)

integration types

0.1

0.0

Figure 2: Reasons why a given website functionality was integrated in a certain way (a) and why alternatives were considered (b) or not (c), aggregated across functionalities.

Proceedings on Privacy Enhancing Technologies YYYY(X)from other sources. Of the 62 people who had been asked this question for social media integration, 12 (19.4 %) had considered usingcode provided by the social networks and 4 (6.5 %) had considered
code by another third party. The embedded media category was
shown to 86 participants, 9 of whom (10.5 %) had considered selfwritten embedding code, 3 (3.5 %) code provided by the resourcehosting third party, and 4 (4.7 %) code by another third party.
As for the reasons why alternatives were considered or not (Q35), Figure 2 in (b) and (c) investigates this for self-hosting vs. pure
remote third-party use. We observe that, like for the selection of
the current solution (a), ease of integration is a prominent factor
to both consider and not to consider alternatives. Somewhat unexpectedly, for pure use of third parties this reason and resources
appear to be factors to research rather than to not consider possible
alternatives. This could hint towards users of third-party services
not always being content with what those offer and decision processes to be complex. However, the most important factor not toconsider alternatives appears to be familiarity with the selected solution, for self-hosted solutions even more so than for use of remote
third-party services. The “Other” responses to this question mainlycomprised satisfaction with the current solution, low priority of therespective functionality, or mere statements that it was unnecessary
to look for alternatives (“It wasn’t required” [P316-Privacy]; “Thefirst way worked” [P241-Analytics]).

5.3

Privacy Considerations in Integration

Beyond the selection phase, we investigated participants’ privacy
practices in the stage of integrating the selected solution.

to save time and effort through code reuse [71] and additionallyfinds that these factors can fuel the reasoning both for or against
third-party use and there are differences between functionalities.

5.3.1 Resources Used for Integration. For integration itself, the answers to Q3-7 paint a similar picture as the resources for selection(Q3-6). Again, the main sources of information were official websites/documentation and the website’s team. Online articles andforums are less often used for actual integration compared to theselection phase. Terms of service and privacy policies again wererarely consulted. Though not directly comparable in answer space,the 20 % of privacy plugin users who consulted terms of service ora privacy policy are in the same dimension as the legal information
sources used to integrate consent forms for advertising in mobile
apps [81] (14.1 % for “Legal policies (e. g., GDPR)” and 9.9 % for“legal teams”). Figure 8 in Appendix C shows detailed data for Q3-7.

Consideration of Alternatives. Participants involved in the selection of a functionality were asked in Q3-4 whether they had
considered alternatives to their chosen integration solution. Figure 3 shows that across all categories, this was answered negatively
by a large share of participants, from 16.7 (advertising) to 50.7 %(analytics). A similarly low rate was reported in the work of Mhaidli
et al., who found only two out of nine interview participants to havemade some effort in considering and comparing different mobile
ad networks before settling on one [55]. Rather, participants were
found to select a network based on some “vague awareness” ofwhat was popular and commonly used with good experience. We
found similar sentiments in our data for functionality with a clear
market leader, notably the prevalent use of analytics, for which theoutstanding popularity of Google Analytics was confirmed by our
measurements (Table 3). The answers to Q3-2 suggest that people
consider it the “default” solution and do not even think about possible alternatives. Except for payment, which is only practical with
the involvement of third parties, most considered alternatives werefirst-party solutions, even for functionalities considered difficult toself-host such as video content or (targeted) advertising [50]. This
could again hint at people rarely choosing between different thirdparty services but rather deciding between either self-implementing
a functionality or using a specific third-party service.
For embedded and social media, participants also had the option
to indicate whether they had considered embedding mechanisms

5.3.2 Privacy Protection Efforts. When asked in Q4-2 if they had
employed specific measures to protect website visitors’ privacy
when configuring their solution to implement a functionality, participants’ answers did not vary significantly (𝑝 > 0.05, Fisher’s
exact test) across functionalities. For all of them, about a quarter ofparticipants reported to have employed privacy protection mechanisms, another quarter stated to not have used them, about one
third did not know, and the rest did not provide an answer.
Table 4 shows what privacy protection efforts participants reported to have made in the configuration of their solution. Participants frequently referred to data minimization (“I don’t reallycollect user information, and when I do, I keep it to a minimum to
get the job done” [P361-Programming]) and secure transfer (“encryption and [TLS]” [P84-Inter]). Another prominent theme in theanswers was first- vs. third-party selection, including self-hosting as
9Proceedings on Privacy Enhancing Technologies YYYY(X)

Christine Utz, Sabrina Amft, Martin Degeling, Thorsten Holz, Sascha Fahl, and Florian SchaubAdver
Cu tisist. ngEm Inte (10)be rac
Pr dde tionog d M (46r
Pr amm edi )aiv
ac ing (72)
Pr y ((
ot 12 128ec ))An tioal n (3ySo tics 3)
cia (96l
Pa Me )
ym di
a(
e
n
3
Lo
gi t (26 3)
n
/A )
ut
h
(6
0)

functionalities

Considered alternatives (hosting)
1P, self-hosted
3P, self-hosted / only banks

3P, 3P-hosted
No

Other
Don't know

NaN

Financial 0.00 0.04 0.03 0.03 0.00 0.03 0.02 0.06 0.62 0.00
Contact info 0.10 0.46 0.19 0.20 0.17 0.12 0.16 0.21 0.62 0.58
Location 0.50 0.22 0.38 0.11 0.17 0.15 0.59 0.24 0.42 0.08

0.2
data types

frequency

0.75

Health 0.10 0.02 0.00 0.01 0.00 0.03 0.00 0.00 0.00 0.00

0.4

ut
h.
(

t(

83

24

)

)

)

/A
gi
n
Lo

a(
ed
i

m
en
Pa
y

69
s(
So

cia
lM

yt
ic
al
An

te
ct
Pr
o

62

)

1)
io
n

y
ac

(1

(3

55
Pr
iv

g
in

(5

6)

)

)
86
m
m
og
ra
Pr

ed

M

ed
ia
(

n
io
Em
be

dd

ra
ct
In

te

Ad

s(

12

)

(8
2)

0.0

Demographic 0.70 0.15 0.15 0.11 0.08 0.09 0.25 0.18 0.08 0.15
Personal IDs 0.10 0.04 0.04 0.05 0.08 0.03 0.04 0.03 0.12 0.03

0.45

Online activity 0.80 0.30 0.35 0.17 0.33 0.33 0.82 0.24 0.08 0.13
User profile 0.10 0.11 0.18 0.12 0.17 0.09 0.15 0.18 0.23 0.40

0.30

Social media data 0.30 0.13 0.18 0.06 0.00 0.09 0.16 0.27 0.04 0.10
IP addr./device ID 0.70 0.33 0.46 0.16 0.25 0.42 0.54 0.30 0.42 0.28

0.15

Cookies/tracking 0.70 0.33 0.36 0.17 0.83 0.39 0.55 0.36 0.42 0.38
Device info 0.80 0.35 0.47 0.26 0.42 0.39 0.81 0.45 0.27 0.32

functionalities

0.60

0.00

Figure 3: Alternatives considered (Q3-4) for the hosting of website func- Figure 4: Percentage of 3P-using participantstionalities. Numbers are relative to how often the question was displayed who thought what types of personal data the ser(see survey logic in Appendix A; n values in x-axis labels).
vice collected (Q4-1; n values next to func. labels).
Table 4: Privacy protection efforts (Q4-2) reported by participants involved in integration or maintenance of a functionality,
across all integrations (n = 224). For code definitions see Appendix B.

.

Code

Examplesn%

No personal data
Data minimization
Self-hosting
3P selection
3P setting
User consent
Transparency
Data access
Anonymization
Security
Other
No answer“No user data is logged” (P337-Analy), “No personal data is stored” (P46-Inter)“limited data retention” (P130-Prote), “only what we need” (P1178-Analy)“No external service used” (P190-Priva), “Coded [it] myself safely” (P221-Socia)“Remove GA :)” (P30-Analy), “non-Google CDNs” (P855-Progr)“use the no-cookie option” (P212-Embed), “anonymize IP on [GA]” (P1256-Analy)“I put them in containers [...] only executed after consent” (P214-Ads)“privacy policy” (P535-Ads), “we follow our privacy policy” (P66-Inter)“access to specific users” (P955-Progr), “don’t pass any user data” (P191-Embed)“anonymus [sic] identifiers” (P288-Analy), “obfuscate user ids” (P917-Inter)“HTTPS” (P855-Login), “password hash” (P619-Login), “encryption” (P1091-Inter)“too many to list” (P163-Login), “look through the [...] source code” (P695-Embed)
Nothing entered, “1.?????????” (P352-Login), “Don’t know specifics” (P53-Socia)

9
38
19
17
26
14
4
18
12
34
46
29

4.0
17.0
8.5
7.6
11.6
6.3
1.8
8.0
5.4
15.2
20.5
12.9

a means to protect visitors’ privacy (“Remove tracking from social
media buttons by replacing them with a similar button” [P385Social]), careful selection of the third party with privacy in mind(“I chose a font service that I believed would respect user privacy”[P136-Progr]), and using settings offered by the third-party service
to collect less data. Prominent themes in individual categories are
security for login/authentication (32.5 %) and customer interaction(28.1 %); anonymization, data minimization (22.2 % for both), andthird-party settings (30.6 %) for analytics. The explanation for the repeated occurrence of security mechanisms, including access control,
is that developers often conflate privacy with security [29, 85].
Across all categories, only 24 answers to Q4-3a explained the motivation behind the measures to protect visitors’ privacy. 20 named
regulatory requirements mostly from privacy law but, in the caseof payment providers, also industry regulations. Two participants
mentioned an unspecified “requirement” for analytics and anothertwo a self-commitment to privacy (for analytics and social media).
Table 5 shows the reported reasons not to make privacy-protecting
configurations. Most frequently, the solution was perceived not
to collect any personal data, which was especially prevalent for

programming/design (39.1 %; “because the third party does not collect anything” [P109-Progr]), embedded media (18.6 %), and social
media (34.8 %); in the latter case, the responses often referred to
first-party integrations of profile buttons or links (“they’re justlinks” [P98-Socia], “simply images, wrapped in anchor tags” [P289Socia]). Other prominent themes were trust in the third party toadequately protect users’ privacy (“I thought the default setup already protects the visitors’ privacy enough” [P243-Analy], “I trusted[Cloudflare] to not collect excessive information” [P321-Prote]) andthe perception that it was impossible to do anything about collected
data (“there is nothing I can do in GA to change the data Google
collects” [P396-Analy]), particularly for analytics (27.6 %). Trust in
third-party vendors and the perceived inability to do something
about the data collection were also recurring sentiments in why
developers of mobile apps stick to a service’s default configuration [55]. Finally, some answers simply deemed privacy protection
unnecessary (“I don’t care about privacy because ‘data is king’”[P295-Payment]), prominently for programming/design (39.1 %)and embedded media (18.6 %).

10

Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websites

Proceedings on Privacy Enhancing Technologies YYYY(X)

Table 5: Reasons not to make any specific effort to protect visitors’ privacy when integrating or maintaining a functionality,
across all integrations (n = 263). For code definitions, see Appendix B.

.

5.4

Code

Examplesn%

No data collected
Data minimization
Self-hosting
Trust in 3P
Impossible
Website purpose
Priorities
PayoffUnnecessary
Lack of knowledge
Other
No answer“no tracking involved” (P213-Prote), “nothing is saved” (P247-Progr)“We don’t ask for anything beyond email address and name” (P44-Inter)“system is on-premise” (P201-Inter), “own code without tracking” (P264-Socia)“The service I used [...] handles security” (P380-Pay)“no configuration options” (P11-Ads), “we are not developing it” (P32-Prote)“Internal use only” (P95-Login), “page is not ready yet” (Progr-361)“We [use] analytics to track users. That’s the opposite of privacy” (P290-Analy)“It’s more work” (P132-Analy), “won’t pay back” (P324-Priva)“Why should I” [P439-Social], “no need” (P63-Inter), “Didn’t have to” (P353-Progr)“I can’t understand whole of what [GA] collect[s]” (P382-Analy)“Its just a frontend library” (P338-Progr), “Existing solutions satisfies” (P282-Progr)
Nothing entered, “prefer not to say” (P91-Ads)

58
8
15
30
23
26
6
5
38
2
16
48

22.1
3.0
5.7
11.4
8.7
9.9
2.3
1.9
14.4
0.8
6.1
18.3

Awareness of Third-Party Data Collection

site. Analyzing self-reported information is common in research
involving developers [1, 55, 63, 74] and manual inspection of survey
responses suggests that participants answered consistently. Our
survey was voluntary and uncompensated, which might have introduced bias, especially since experts tend to be well-paid and hard toreach. However, a lack of compensation was found to yield higher
motivation or engagement in developer studies [1, 2, 26, 56].
Further limitations apply to our website analysis. As data collection took multiple weeks, it is possible that in some instances
websites changed between participants’ responses and website analysis. Additional discrepancies might have been introduced due toour categorization differing slightly from WhoTracks.me, third parties using the same domain for multiple purposes, or participants
not knowing or naming the functionalities on their website.

Q4-1 more closely investigated the assumed lack of awarenessof third-party data collection. For the third-party users of eachfunctionality (as by Q3-2), Figure 4 shows the percentages who
thought that the service collected specific types of data.
We observe that participants had a solid understanding of datacollection implied by a service’s core functionality. For example,
a majority of participants reported that third-party privacy popups/forms collect cookies, that payment services require contact
and financial information, or that advertising and analytics collectdevice information and user online activities. However, beyond this,participants’ understanding of data collection was limited. This isespecially evident in the case of IP addresses and device information:
As HTTP(S) requests to a remote resource involve transmissionof a user’s IP address and user agent, this information is always
available to the third party. More indirect is the opportunity forthe third party to derive additional information via these technicalparameters, such as tracking users across sites that use that service
and learning their browsing behavior. It appears that many participants embed third-party software and either do not know or are
uncertain of the true extent of data collection by the third party.
This is supported by the responses to Q3-9 that let participants
rate the integrated solution with regard to different metrics. Between 48 % (advertising) and 75.71 % (website protection) of participants reported to be Satisfied or Very Satisfied with the privacy
offered by their integrated solution, while only up to 8.73 % (analytics) expressed some degree of dissatisfaction. This suggests thatdata collection by third parties is often either accepted or unknown.

6

7

DISCUSSION

Our findings provide insights into how web developers and people in similar roles select how to integrate a desired functionality,
configure the selected solution, and if they are aware of the privacy
risks associated with third-party services. For selection, we find theprevalence of third-party use to vary by functionality. In configuration, specific efforts to protect website visitors’ privacy mostly
appear to be made if mandated by technical guidelines on privacy
law. Based on these findings, we discuss the need to raise awarenessof the privacy risks of third-party use on websites and to promote
adoption of privacy-friendlier alternatives. On the methodologicallevel, our work is a case study for how the perception of researchmethods previously deemed acceptable can change over time.

LIMITATIONS

7.1

Our study has some limitations. First, we aimed to recruit a diverse
sample and we are confident that it provides a wide range of perspectives on third-party adoption but may not include every typeof website or third-party user. Websites and third-party services
are not easy to categorize, and therefore participants might haveinterpreted our categories differently (see Table 1). However, we
provided examples and aimed for a sensible compromise between
lengthy explanations and too much room for interpretation.
Second, a limitation of any survey is self-reported data. We cannot verify to what degree participants were actually involved withthe provided website or if they consistently answered for the same

Lack of Awareness of 3P Data Collection

Our research confirms the previously suggested lack of awareness [19] to what extent the use of third-party functionality on
websites can pose risks to visitors’ privacy. While developers appear to be aware of data collection closely tied to the main purposeof a third-party service, they often seem to not know or ignore
the possibility that their visitors’ personal data could be collected
for other purposes, or simply trust the third-party service to notcollect data or to employ adequate privacy protection. For analytics,
our results hint at a somewhat higher privacy awareness than forother functionalities. This could be due to data collection simply
being the main objective of web analytics, or due to prominent and11Proceedings on Privacy Enhancing Technologies YYYY(X)

Christine Utz, Sabrina Amft, Martin Degeling, Thorsten Holz, Sascha Fahl, and Florian Schaub

recent guidelines on GDPR-compliant use of web analytics [42, 80].
Similarly, concrete legal requirements have led to the adoption ofprivacy notices and forms, while developers appear to find it difficult to implement the more generic “privacy by design” approachpromoted by the the GDPR or the NIST Privacy Framework [57].
Public discussion and additional guidelines could help raise awareness for the privacy risks of other types of third-party services on
websites, and on operationalizing “privacy by design” for website
development and integration, ideally addressing a wide range ofwebsite-related roles. Measures to raise awareness would also needto communicate risks beyond the immediate control of developers,
as third-party services often connect and share data with each otherwithout users’ knowledge [88], and different understandings of thesensitivity of the collected data, such as IP addresses.
Referring developers to a service’s privacy policy is insufficient
to communicate its privacy risks. While privacy policies can be
expected to contain information about the data collected by a thirdparty service, our results show that they are rarely used whenselecting or configuring services. This is unsurprising given thatprivacy policies are notoriously hard to understand, and the GDPR,
a law pursuing greater transparency, has even led to an increasein the length of online privacy policies [14]. As an additional aid,privacy labels for mobile apps have recently been introduced into
Apple’s and Google’s app stores [3, 21]. With web development not
taking place inside such a closed ecosystem, there are no centralized
platforms developers could turn to for advice and comparison ofdifferent services that integrate a given functionality. For those
who use common CMSes, their plugin repositories could introduce
similar labels, placing privacy information more prominently thanin a legal document. Alternatively, IDEs [47] and CMS editors could
help assess the number of third-party requests in website code or
problematic configurations for popular services and display advice.

7.2

from the beginning of the web development process, desired website functionality works as expected, and the burden is not placedon website visitors. A website that practices data minimizationand privacy by design could even render annoying consent notices
unnecessary for the benefit of both websites and visitors.
We found notable involvement of DPOs or legal experts only
for privacy popups or forms, i. e., functionality added for the administration of a website’s existing data processing practices. This
could be an indicator that privacy is still regarded as something to
be “added later” instead of being considered throughout the development process. Moreover, web development is often done in small
teams or by single persons without a privacy professional at hand.
When the decision is in the hands of developers and made in earlystages of the development process, our results show that ease of
integration and familiarity with solutions are the driving factors
for adoption. This does not necessarily mean that developers do
not care about privacy, but it is simply not an important concern
given deadlines and limited resources in small teams [51]. Whileat the beginning of development it is often unclear what user data
the final (web) application will need [4], this does not preclude theinvolvement of privacy considerations from the beginning. Iterative
privacy impact and risk assessment processes that continuously
evaluate functional requirements against privacy implications could
help ensure that the desired functionality is implemented using theleast amount of personal data, thus complying with frameworks
that follow a data minimization or privacy-by-default approach.

7.3

Promoting Privacy-Friendlier Alternatives

While advice to self-host [66, 69] or use privacy-friendly alternatives to popular third-party services [19] has increased in recent
years, we found that only few participants heeded such advice.
Others reported not knowing alternatives to the solution they usedor did not have the time or resources to look for them. This should
be interpreted as a challenge to better promote privacy-friendly
alternatives for both the developers of these services and the privacy and security research community at large. We found ease of
integration, features, and cost to be among the most frequently
reported factors that cause developers to adopt a certain solution –requirements currently easiest to satisfy by a service available freeof charge that instead monetizes visitor data. It remains a major
challenge to reconcile the demand for usability, features, and lowest
possible cost if monetization of visitor data is not an option.
On the configuration level, privacy-friendlier options do exist
but are often hidden or obscured by dark patterns [82]. For example,
YouTube’s setting for “privacy-enhanced mode” is only revealed
when one scrolls down in the “Embed” dialog while the standard
embed code is directly visible. Vendors could encourage use of
the privacy-friendly configuration by making it more prominent oreven the default, though there is no incentive for this if the service’s
business model is based on monetization of personal data, as is often
the case with third-party services offered free of monetary cost.
Privacy laws and court rulings were identified as drivers of privacyrelated settings in ad networks [83], analytics services [59, 67],
and cookie consent notices [18]. Thus, public policy measures andregulatory guidance could go one step further and require vendors
to make the privacy-friendly option the default.

Promoting Privacy Engineering

Our work confirms earlier findings from the mobile ad ecosystemthat developers often feel resigned and unable to effect change
in a third-party ecosystem governed by the exchange of revenue
or functionality for access to website visitor’s personal information: Previous work found sentiments that users’ personal data
would be collected by platforms and vendors, irrespective of thedeveloper’s decisions [55], and both developers [55] and third-party
vendors [83] deem the respective other party responsible for theprotection of users’ data. One option to break this cycle of blame
and instigate change would be to encourage developers that they
can indeed make a difference through privacy-conscious integration of functionality [55]; after all, it is developers and end usersthat made these vendors that prevalent and powerful through use
and promotion of their services. While in the past it was often
browser vendors and developers of privacy-enhancing extensions
who fueled advancements in website visitors’ privacy, such as theoption to block third-party cookies, relegating privacy protection
to the browser comes at the risk of breaking websites and could
overwhelm users with configuration options and prompts. Thus,
promoting privacy-by-design with website creators would be amore holistic approach that can ensure that privacy is considered12

Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websites

7.4

Methodological Implications

Proceedings on Privacy Enhancing Technologies YYYY(X)the US, do not have such a review board, or review is not always
mandatory, as was the case for our study. But even if prior IRB review had been available, it remains doubtful whether it could have
prevented the complaint to the DPA. The main goal of IRB review
is to ensure that a study complies with human subjects regulations,
not to provide a comprehensive ethics and legal assessment. In fact,
we took additional steps to get GDPR assessments from our institutions’ DPOs before running the study. The challenge is that inprivacy and security research, a deep ethics and legal review would
often require specific technical domain knowledge (e. g., GitHub’s
handling of commit email addresses), associated risks, and theirlegal evaluation. These are aspects that are often not covered by
IRB guidelines or board members’ background due to their differingfunction. Legal assessment in particular can be subject to rapid evolution through new laws and court rulings, requiring involvementof legal experts who keep up with this constant change.
Recently the privacy and security research community has identified this need for thorough ethical review and multiple venues have
set up ethics committees that can be involved in the review process
if a submission raises ethical concerns with reviewers. This work
went through this very process, and we highly value the thoroughethics review we received, which concluded that we adequately
addressed our study’s ethical implications. While ethical review
after submission is an important step in ensuring that publishedprivacy and security research did not cause undue harm to the people whose behavior and systems were studied, it effectively comes
too late, at a time any potential harm would have already beencaused. Hence, the community needs to consider how to provide
ethical guidance before potentially harmful research is carried out,for example, by means of a “standing ethics review board” of expert
volunteers that can complement institutional review in the study
design phase. Such a priori ethics review would (1) help prevent
unethical privacy and security research before it occurs, (2) provide researchers with experience and confidence in how to address
ethical implications, and (3) minimize the sometimes arbitrary andad-hoc assessments of a study’s ethical implications by reviewers.
An existing example is the Tor Research Safety Board [86]; providing committees of domain experts that cover the whole privacy andsecurity field would pose a major challenge. Hence, such a priori
review would not have to be mandatory for all submissions but
could become a valued community resource.

Section 4.3 described how recruitment via email addresses in public
GitHub commit metadata came under the scrutiny of our state’s
data protection authority. We now discuss what part of the process
had raised concerns with the DPA, what this means for future recruitment in privacy and security research, and what could be donein advance to decrease the likelihood of facing similar problems.
7.4.1 Recruiting Developers on GitHub. Email recipients who asked
how we found their email address on GitHub often pointed out thatthey had set their email address to “private” on their GitHub user
profiles. While this setting hides the address from the public profile,
it does not affect the visibility of the email address in commits topublic GitHub repositories. Any given commit into a public repository has a corresponding *.patch file, available at https://github.
com/<user>/<repository>/commit/<commit_hash>.patch. The second line in this file shows the author of the commit, along with theiremail address. This is due to the core concept behind GitHub’s public repositories, where all commits, including metadata, are public.
The documentation [24] describes how users can configure Git(Hub)to use their GitHub-provided “noreply” email address, which willremove their real email address from the commit metadata but still
associate their contributions with their GitHub account.
Email feedback showed that many GitHub users are not awareof these mechanics and settings. This was also the issue at the coreof the DPA’s assessment, which argued that GitHub users pushingcommits into public repositories did not expect to be contacted via
their commit email addresses for the purpose of scientific research,and this lack of awareness constituted a legitimate interest of theuser that outweighed public interest in scientific research. In addition, users of GitHub’s API are bound by GitHub’s terms of service
and privacy statement [23]. GitHub’s privacy policy considers a user
email address public information (unless made private as describedabove) but proceeds to limit its use “for the purpose for which [the]user authorized it” [22]. Following the DPA’s argument, this likely
does not include being contacted for the purpose of participationin scientific research. It remains for the community to decide what
influence such company policies should have on the question ofwhat is considered ethical in privacy and security research, and,looking further ahead, how to handle company policies on data usethat contradict what is permissible under applicable law.
For future recruitment of study participants we recommend, asalso suggested by the DPA, to only use contact information that has
visibly been made public by the individuals themselves with theintention of allowing the general public to contact them. GitHub’s
email address mechanics and users’ lack of knowledge about themhad neither been mentioned nor addressed by previous work thatused public GitHub repositories for recruitment. We hope that ourexperience can inform the ongoing debate about ethics in privacy
and security research and the search for alternatives to reach diverse
sets of developers in a reliable, ethical, and affordable way.

8

CONCLUSION

We report findings from an online survey with 395 people working
with websites on how common website functionalities are implemented, in particular if third-party services are used and whether
and how respective privacy implications have been considered.
While we observe that the selection process is influenced by avariety of factors, we find that often factors such as a third-party service’s popularity and ease of integration fuel adoption decisions. By
contrast, website visitors’ privacy only plays a notable role in web
analytics, a functional category which has been explicitly addressedby data protection authorities. Except for privacy popups and forms,
data protection officers and legal counsels are rarely involved in
the decision processes that lead to the integration of third-party
services into websites despite potential privacy implications.

7.4.2 The Need for A Priori Community-Based Ethics Review. It haslong been best practice in human subjects research to obtain priorreview via an institutional review board (IRB) or a similar entity toensure that participation in the study does not cause undue harm tohumans. However, in practice, many institutions, especially outside13Proceedings on Privacy Enhancing Technologies YYYY(X)

Christine Utz, Sabrina Amft, Martin Degeling, Thorsten Holz, Sascha Fahl, and Florian Schaub

ACKNOWLEDGMENTS[17] European Commission. 2022. What is personal data? Retrieved September 15,
2022 from https://ec.europa.eu/info/law/law-topic/data-protection/reform/whatpersonal-data_en (cited on p. 2).
[18] European Court of Justice. 2019. Judgment of the Court of 1 October 2019 inCase C-673/17 – Planet49. Retrieved September 15, 2022 from https://eurlex.europa.eu/legal-content/EN/TXT/HTML/?uri=CELEX:62017CJ0673 (cited on
p. 12).
[19] European Digital Rights. 2020. Ethical Web Dev – Guide for Ethical Website
Development and Maintenance. Retrieved September 15, 2022 from https://edri.
org/files/ethical_web_dev_web.pdf (cited on p. 1, 2, 3, 7, 11, 12).
[20] The European Parliament and the Council of the European Union. 2016. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016
on the protection of natural persons with regard to the processing of personal dataand on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Official Journal of the European Union, L 119/1.
Retrieved September 15, 2022 from https://eur-lex.europa.eu/eli/reg/2016/679/oj(cited on p. 1).
[21] Suzanne Frey. 2021. New safety section in Google Play will give transparency into
how apps use data. Retrieved September 15, 2022 from https://android-developers.
googleblog.com/2021/05/new-safety-section-in-google-play-will.html (cited on
p. 12).
Retrieved September[22] GitHub, Inc. 2022. GitHub Privacy Statement.
15, 2022 from https://docs.github.com/en/site-policy/privacy-policies/githubprivacy-statement (cited on p. 13).
[23] GitHub, Inc. 2022. GitHub Terms of Service.
Retrieved September 15,
2022 from https://docs.github.com/en/site-policy/github-terms/github-terms-ofservice (cited on p. 13).
[24] GitHub, Inc. 2022. Setting your commit email address. Retrieved September 15, 2022 from https://docs.github.com/en/account-and-profile/settingup-and-managing-your-personal-account-on-github/managing-emailpreferences/setting-your-commit-email-address (cited on p. 13).
[25] Google LLC. 2021. Privacy controls in Google Analytics. Retrieved September
15, 2022 from https://support.google.com/analytics/answer/9019185 (cited on p.
3).
[26] Peter Leo Gorski, Luigi Lo Iacono, Dominik Wermke, Christian Stansky, SebastianMöller, Yasemin Acar, and Sascha Fahl. 2018. Developers Deserve Security
Warnings, Too: On the Effect of Integrated Security Advice on Cryptographic
API Misuse. In Fourteenth Symposium on Usable Privacy and Security (SOUPS2018). USENIX Association, Berkeley, CA, USA, 265–280. https://www.usenix.
org/conference/soups2018/presentation/gorski (cited on p. 2, 4, 5, 11).
[27] Sacha Greif. 2022. The State of JS 2021. Retrieved September 15, 2022 fromhttps://2021.stateofjs.com/en-US/demographics (cited on p. 6).
[28] Martin Gundersen. 2019. Uncovering the Disqus Data Machine. Retrieved September 15, 2022 from https://twitter.com/martingund/status/1207327648093003777(cited on p. 3).
[29] Irit Hadar, Tomer Hasson, Oshrat Ayalon, Eran Toch, Michael Birnhack, Sofia
Sherman, and Arod Balissa. 2018. Privacy by designers: software developers’
privacy mindset. Empirical Software Engineering 23, 1 (Feb. 2018), 259–289.
https://doi.org/10.1007/s10664-017-9517-1 (cited on p. 3, 10).
[30] heise online. 2019. Shariff – Give Social Media Buttons Some Privacy. Retrieved
September 15, 2022 from https://github.com/heiseonline/shariff (cited on p. 1, 3).
[31] heise online. 2020. embetty. Retrieved September 15, 2022 from https://github.
com/heiseonline/embetty (cited on p. 1, 3).
[32] Maximilian Hils, Daniel W. Woods, and Rainer Böhme. 2020. Measuring theEmergence of Consent Management on the Web. In Proceedings of the ACM
Internet Measurement Conference (IMC ’20). ACM, New York, NY, USA, 317–332.
https://doi.org/10.1145/3419394.3423647 (cited on p. 2, 6).
[33] Xuehui Hu, Guillermo Suarez de Tangil, and Nishanth Sastry. 2020. Multi-country
Study of Third Party Trackers from Real Browser Histories. In Proceedings of the2020 IEEE European Symposium on Security and Privacy (EuroS&P 2020). InternetSociety, Reston, VA, USA, 70–86. https://doi.org/10.1109/EuroSP48549.2020.00013(cited on p. 3).
[34] Patrick Hulce. 2019. The Web Almanac – Third Parties. Retrieved September 15,
2022 from https://almanac.httparchive.org/en/2019/third-parties (cited on p. 1, 2,
3).
[35] Patrick Hulce. 2021. Third Party Web. Retrieved September 15, 2022 fromhttps://github.com/patrickhulce/third-party-web/ (cited on p. 2).
[36] Muhammad Ikram, Rahat Masood, Gareth Tyson, Mohamed Ali Kaafar, Noha
Loizon, and Roya Ensafi. 2019. The Chain of Implicit Trust: An Analysis of the
Web Third-party Resources Loading. In Proceedings of the Web Conference 2019
(WWW ’19). ACM, New York, NY, USA, 2851–2857. https://doi.org/10.1145/
3308558.3313521 (cited on p. 2).
[37] Shubham Jain and Janne Lindqvist. 2014. Should I Protect You? Understanding
Developers’ Behavior to Privacy-Preserving APIs. In Workshop on Usable Security(USEC 2014). Internet Society, Reston, VA, USA. https://www.ndss-symposium.
org/wp-content/uploads/2017/09/01_1-paper.pdf (cited on p. 2, 4).
[38] Arjaldo Karaj, Sam Macbeth, Rémi Berson, and Josep M. Pujol. 2018. WhoTracks.Me: Monitoring the online tracking landscape at scale. arXiv:1804.08959v1.

We would like to thank the anonymous reviewers for the detailed
feedback and the comprehensive ethics assessment, and our shepherd for their guidance to further improve this work. This research
was funded by the MKW-NRW Research Training Groups SecHuman and NERD.NRW.

REFERENCES[1] Yasemin Acar, Michael Backes, Sascha Fahl, Simson Garfinkel, Doowon Kim,
Michelle L. Mazurek, and Christian Stransky. 2017. Comparing the Usability of
Cryptographic APIs. In Proceedings of the 2017 IEEE Symposium on Security
and Privacy (S&P ’17). IEEE Computer Society, Washington, DC, USA, 154–
171. https://doi.org/10.1109/SP.2017.52 https://www.usenix.org/conference/
soups2017/technical-sessions/presentation/acar. (cited on p. 2, 4, 5, 11).
[2] Yasemin Acar, Christian Stransky, Dominik Wermke, Michelle L. Mazurek,
and Sascha Fahl. 2017. Security Developer Studies with GitHub Users: Exploring a Convenience Sample. In Proceedings of the Thirteenth Symposium
on Usable Privacy and Security (SOUPS 2017). USENIX Association, Berkeley,
CA, USA, 81–95. https://www.usenix.org/conference/soups2017/technicalsessions/presentation/acar (cited on p. 2, 4, 5, 11).
[3] Apple Inc. 2021. App privacy questions requirement starts December 8. Retrieved
September 15, 2022 from https://developer.apple.com/news/?id=em8fm29e (cited
on p. 12).
[4] Rebecca Balebako, Abigail Marsh, Jialiu Lin, Jason Hong, and Lorrie Faith Cranor.
2014. The Privacy and Security Behaviors of Smartphone App Developers. In
Workshop on Usable Security (USEC 2014). Internet Society, Reston, VA, USA. https://www.ndss-symposium.org/wp-content/uploads/2017/09/01_2-paper.pdf (cited
on p. 2, 3, 8, 12).
[5] Stephanie Bodoni. 2019.
Facebook’s Like Button Makes Websites Liable, Top EU Court Rules.
Retrieved September 15, 2022 fromhttps://www.bloomberg.com/news/articles/2019-07-29/facebook-s-likebutton-makes-websites-liable-top-eu-court-rules (cited on p. 3).
[6] Danny Brown. 2017. Disqus: Is Your Data Worth Trading for Convenience? Retrieved September 15, 2022 from https://replyable.com/2017/03/disqus-is-yourdata-worth-trading-for-convenience/ (cited on p. 3).
Belgian DPA fines IAB Europe 250K euros over[7] Jennifer Bryant. 2022.
consent framework GDPR violations.
Retrieved September 15, 2022
from https://iapp.org/news/a/belgian-dpa-fines-iab-europe-250k-euros-overconsent-framework-gdpr-violations/ (cited on p. 1).
[8] Matt Burgess. 2022. Europe’s Move Against Google Analytics Is Just the Beginning.
Retrieved February 17, 2022 from https://www.wired.com/story/google-analyticseurope-austria-privacy-shield/ (cited on p. 1).
[9] Karoline Busse, Mohammad Tahaei, Katharina Krombholz, Emanuel vonZezschwitz, Matthew Smith, Jing Tian, and Wenyuan Xu. 2020. Cash, Cards
or Cryptocurrencies? A Study of Payment Culture in Four Countries. In Proceedings of the 2020 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW). IEEE, Piscataway, NJ, USA, 200–209. https://doi.org/10.1109/
EuroSPW51379.2020.00035 (cited on p. 3).
[10] Victoria Clarke and Virginia Braun. 2014. Thematic Analysis. In Encyclopedia of
Critical Psychology, Thomas Teo (Ed.). Springer New York, New York, NY, USA,
1947–1952. https://doi.org/10.1007/978-1-4614-5583-7_311 (cited on p. 5).
[11] Adrian Dabrowski, Georg Merzdovnik, Johanna Ullrich, Gerald Sendera, andEdgar Weippl. 2019. Measuring Cookies and Privacy in a Post-GDPR World. In
Proceedings of the 20th International Conference on Passive and Active Measurement(PAM 2019). Springer Nature Switzerland AG, Cham, Switzerland, 258–270. https://doi.org/10.1007/978-3-030-15986-3_17 (cited on p. 3, 6).
[12] Data USA. 2022. Web Developers. Retrieved September 15, 2022 from https://datausa.io/profile/soc/web-developers (cited on p. 6).
[13] Kevin Davis. 2019. You (probably) don’t need ReCAPTCHA. Retrieved February
17, 2022 from https://kevv.net/you-probably-dont-need-recaptcha/ (cited on p.
3).
[14] Martin Degeling, Christine Utz, Christopher Lentzsch, Henry Hosseini, FlorianSchaub, and Thorsten Holz. 2019. We Value Your Privacy ... Now Take Some
Cookies: Measuring the GDPR’s Impact on Web Privacy. In Proceedings of the 26th
Annual Network and Distributed System Security Symposium (NDSS ’19). InternetSociety, Reston, VA, USA. https://doi.org/10.14722/ndss.2019.23378 (cited on p.
1, 3, 12).
[15] DuckDuckGo. 2021. DuckDuckGo Tracker Radar. Retrieved September 15, 2022
from https://github.com/duckduckgo/tracker-radar/ (cited on p. 2).
[16] Steven Englehardt and Arvind Narayanan. 2016. Online Tracking: A 1-millionsite Measurement and Analysis. In Proceedings of the 26th ACM Conference on
Computer and Communications Security (CCS ’16). ACM, New York, NY, USA,1388–1401. https://doi.org/10.1145/2976749.2978313 (cited on p. 1, 2, 3, 4, 5, 8).
14

Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websites

https://arxiv.org/abs/1804.08959v1 (cited on p. 2, 8).
[39] Arjaldo Karaj, Sam Macbeth, Rémi Berson, and Josep M. Pujol. 2019.
WhoTracks.Me: Shedding light on the opaque world of online tracking.
arXiv:1804.08959v2. https://arxiv.org/abs/1804.08959v2 (cited on p. 1, 3, 6).
[40] Farzaneh Karegar, Nina Gerber, Melanie Volkamer, and Simone Fischer-Hübner.
2018. Helping John to Make Informed Decisions on Using Social Login. In
Proceedings of the 33rd Annual ACM Symposium on Applied Computing (SAC ’18).
ACM, New York, NY, USA, 1165–1174. https://doi.org/10.1145/3167132.3167259(cited on p. 3).
[41] Konrad Kollnig, Reuben Binns, Pierre Dewitte, Max Van Kleek, Ge Wang, Daniel
Omeiza, Helena Webb, and Nigel Shadbolt. 2021. A Fait Accompli? An Empirical
Study into the Absence of Consent to Third-Party Tracking in Android Apps. In
Proceedings of the Seventeenth Symposium on Usable Privacy and Security (SOUPS
2021). USENIX Association, Berkeley, CA, USA, 181–195. https://www.usenix.
org/system/files/soups2021-kollnig.pdf (cited on p. 2).
[42] Konferenz der unabhängigen Datenschutzaufsichtsbehörden des Bundes
und der Länder. 2020.
Hinweise zum Einsatz von Google Analytics
im nicht-öffentlichen Bereich (in German; Notes concerning the use of
Google Analytics in the non-public sector).
Retrieved September 15,
2022 from https://www.datenschutzkonferenz-online.de/media/dskb/20200526_
beschluss_hinweise_zum_einsatz_von_google_analytics.pdf (cited on p. 12).
[43] Ravie Lakshmanan. 2022. German Court Rules Websites Embedding Google Fonts
Violates GDPR. Retrieved September 15, 2022 from https://thehackernews.com/
2022/01/german-court-rules-websites-embedding.html (cited on p. 1).
[44] Victor Le Pochat, Tom Van Goethem, Samaneh Talajizadehkhoob, Maciej Korczyński, and Wouter Joosen. 2019. TRANCO: A Research-Oriented Top Sites
Ranking Hardened Against Manipulation. In Proceedings of the 26th Annual Network and Distributed System Security Symposium (NDSS ’19). Internet Society,
Reston, VA, USA. https://doi.org/10.14722/ndss.2019.23386 (cited on p. 4).
[45] Ada Lerner, Anna Kornfeld Simpson, Tadayoshi Kohno, and Franziska Roesner.
2016. Internet Jones and the Raiders of the Lost Trackers: An ArchaeologicalStudy of Web Tracking from 1996 to 2016. In Proceedings of the 25th USENIX
Security Symposium (USENIX Security ’16). USENIX Association, Berkeley, CA,
USA, 997–1013. https://www.usenix.org/conference/usenixsecurity16/technicalsessions/presentation/lerner (cited on p. 2, 3).
[46] Christophe Leung, Jingjing Ren, David Choffnes, and Christo Wilson. 2016.
Should You Use the App for That? Comparing the Privacy Implications ofApp- and Web-based Online Services. In Proceedings of the 2016 Internet Measurement Conference (IMC ’16). ACM, New York, NY, USA, 365–372. https://doi.org/10.1145/2987443.2987456 (cited on p. 2).
[47] Tianshi Li, Yuvraj Agarwal, and Jason I. Hong. 2018. Coconut: An IDE Plugin for
Developing Privacy-Friendly Apps. Proceedings of the ACM on Interactive, Mobile,
Wearable and Ubiquitous Technologies 2, 4, Article 178 (Dec. 2018), 35 pages.
https://doi.org/10.1145/3287056 (cited on p. 12).
[48] Tianshi Li, Elizabeth Louie, Laura Dabbish, and Jason I. Hong. 2020. How
Developers Talk About Personal Data and What It Means for User Privacy:
A Case Study of a Developer Forum on Reddit. Proceedings of the ACM
on Human-Computer Interaction 4, CSCW3, Article 220 (Dec. 2020), 28 pages.
https://doi.org/10.1145/3432919 (cited on p. 4).
[49] Timothy Libert. 2015. Exposing the Hidden Web: An Analysis of Third-Party
HTTP Requests on 1 Million Websites. International Journal of Communication 9(2015), 3544–3561. https://ijoc.org/index.php/ijoc/article/view/3646 (cited on p.
2, 3).
[50] Timothy Libert and Rasmus Kleis Nielsen. 2018. Third-Party Web Content on EU News Sites: Potential Challenges and Paths to Privacy Improvement. https://timlibert.me/pdf/Libert_Nielsen-2018-Third_Party_Content_EU_
News_GDPR.pdf (cited on p. 1, 2, 8, 9).
[51] Kai-Uwe Loser and Martin Degeling. 2014. Security and Privacy as Hygiene
Factors of Developer Behavior in Small and Agile Teams. In Proceedings of the
11th IFIP TC 9 International Conference on Human Choice and Computers (HCC2014). Springer, Berlin, Heidelberg, 255–265. https://doi.org/10.1007/978-3-66244208-1_21 (cited on p. 12).
[52] Célestin Matte, Nataliia Bielova, and Cristiana Santos. 2020. Do Cookie Banners
Respect my Choice? Measuring Legal Compliance of Banners from IAB Europe’s
Transparency and Consent Framework. In Proceedings of the 2020 IEEE Symposium
on Security and Privacy (SP ’20). IEEE, Piscataway, NJ, USA, 791–809. https://doi.org/10.1109/SP40000.2020.00076 (cited on p. 3).
[53] McAfee, LLC. 2022. Customer URL Ticketing System. Retrieved February 16,
2022 from https://www.trustedsource.org/ (cited on p. 6).
[54] Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019. Reliability andInter-rater Reliability in Qualitative Research: Norms and Guidelines for CSCW
and HCI Practice. Proceedings of the ACM on Human-Computer Interaction 3,
CSCW, Article 72 (Nov. 2019), 23 pages. https://doi.org/10.1145/3359174 (cited
on p. 5).
[55] Abraham H. Mhaidli, Yixin Zou, and Florian Schaub. 2019. “We Can’t Live Without Them!” App Developers’ Adoption of Ad Networks and Their Considerationsof Consumer Risks. In Proceedings of the Fifteenth Symposium on Usable Privacy
and Security (SOUPS 2019). USENIX Association, Berkeley, CA, USA, 225–244.

Proceedings on Privacy Enhancing Technologies YYYY(X)

https://www.usenix.org/conference/soups2019/presentation/mhaidli (cited on p.
2, 3, 4, 5, 8, 9, 10, 11, 12).
[56] Sarah Nadi, Stefan Krüger, Mira Mezini, and Eric Bodden. 2016. Jumping throughhoops: why do Java developers struggle with cryptography APIs?. In Proceedingsof the 38th International Conference on Software Engineering (ICSE ’16). ACM, New
York, NY, USA, 935–946. https://doi.org/10.1145/2884781.2884790 (cited on p. 2,
4, 5, 11).
[57] National Institute of Standards and Technology. 2020. NIST Privacy Framework.
Retrieved September 15, 2022 from https://www.nist.gov/privacy-framework(cited on p. 12).
[58] Midas Nouwens, Ilaria Liccardi, Michael Veale, David Karger, and Lalana Kagal. 2020. Dark Patterns after the GDPR: Scraping Consent Pop-ups andDemonstrating their Influence. In Proceedings of the ACM CHI Conference on
Human Factors in Computing Systems (CHI ’20). ACM, New York, NY, USA.
https://doi.org/10.1145/3313831.3376321 (cited on p. 3).
[59] noyb. 2022. Austrian DSB: Use of Google Analytics violates “Schrems II” decision
by CJEU. Retrieved September 15, 2022 from https://noyb.eu/en/austrian-dsbeu-us-data-transfers-google-analytics-illegal (cited on p. 1, 12).
[60] Lara O’Reilly. 2015. Google’s new CAPTCHA security login raises ‘legitimate privacy
concerns’. Retrieved September 15, 2022 from https://www.businessinsider.com.
au/google-no-captcha-adtruth-privacy-research-2015-2 (cited on p. 3).
[61] panzi. 2012. Social Share Privacy. Retrieved September 15, 2022 from https://panzi.github.io/SocialSharePrivacy (cited on p. 3).
[62] PayPal. 2022. List of Third Parties (other than PayPal Customers) with Whom
Personal Information May be Shared. Retrieved September 15, 2022 fromhttps://www.paypal.com/ie/webapps/mpp/ua/third-parties-list (cited on p. 3).
[63] Mariana Peixoto, Dayse Ferreira, Mateus Cavalcanti, Carla Silva, Jéssyka Vilela,João Araújo, and Tony Gorschek. 2020. On Understanding How Developers
Perceive and Interpret Privacy Requirements Research Preview. In Proceedingsof the 26th International Working Conference on Requirements Engineering: Foundation for Software Quality (REFSQ 2020). Springer Nature Switzerland, Cham,
Switzerland, 116–123. https://doi.org/10.1007/978-3-030-44429-7_8 (cited on p.
4, 11).
[64] Pew Research Center. 2013. October 2013 Higher Education and Gender Survey. Retrieved September 15, 2022 from https://www.pewsocialtrends.org/wpcontent/uploads/sites/3/2014/02/higher-ed_topline.pdf (cited on p. 20).
[65] Aidan Polese, Safwat Hassan, and Yuan Tian. 2022. Adoption of Third-party
Libraries in Mobile Apps: A Case Study on Open-source Android Applications.
In Proceedings of the 8th IEEE/ACM International Conference on Mobile Software
Engineering and Systems 2022 (MOBILESoft 2022). ACM, New York, NY, USA.
https://doi.org/10.1145/3524613.3527810 (cited on p. 2).
[66] Barry Pollard. 2020. Should you self-host Google Fonts? Retrieved September
15, 2022 from https://www.tunetheweb.com/blog/should-you-self-host-googlefonts/ (cited on p. 2, 3, 12).
[67] Mathieu Pollet. 2022. France joins Austria in finding Google Analytics illegal. Retrieved September 15, 2022 from https://www.euractiv.com/section/dataprotection/news/france-joins-austria-says-google-analytics-data-notprotected-in-us/ (cited on p. 1, 12).
[68] Sören Preibusch, Thomas Peetz, Gunes Acar, and Bettina Behrendt. 2016. Shopping for privacy: Purchase details leaked to PayPal. Electronic Commerce Researchand Applications 15 (Jan. 2016), 52–64. https://doi.org/10.1016/j.elerap.2015.11.004(cited on p. 3).
[69] Harry Roberts. 2019. Self-Host Your Static Assets. Retrieved September 15, 2022
from https://csswizardry.com/2019/05/self-host-your-static-assets/ (cited on p.
2, 3, 12).
[70] Franziska Roesner, Tadayoshi Kohno, and David Wetherall. 2012. Detecting andDefending Against Third-Party Tracking on the Web. In Proceedings of the 9th
USENIX Symposium on Networked Systems Design and Implementation (NDSI
2012). USENIX Association, Berkeley, CA, USA. https://dl.acm.org/doi/10.5555/
2228298.2228315 (cited on p. 3).
[71] Pasquale Salza, Fabio Palomba, Dario Di Nucci, Andrea De Lucia, and Filomena
Ferrucci. 2020. Third-party libraries in mobile apps – When, how, and why developers update them. Empirical Software Engineering 25, 3 (May 2020), 2341–2377.
https://doi.org/10.1007/s10664-019-09754-1 (cited on p. 2, 4, 5, 9).
[72] Pasquale Salza, Fabio Palomba, Dario Di Nucci, Cosmo D’Uva, Andrea De Lucia,
and Filomena Ferrucci. 2018. Do Developers Update Third-Party Libraries in
Mobile Apps?. In Proceedings of the 2018 IEEE/ACM 26th International Conference
on Program Comprehension (ICPC 2018). ACM, New York, NY, USA, 255–265.
https://doi.org/10.1145/3196321.3196341 (cited on p. 2).
[73] Awanthika Senarath and Nalin A. G. Aarachchilage. 2018. Understanding Software Developers’ Approach towards Implementing Data Minimization. In 4thWorkshop on Security Information Workers (WSIW 2018). USENIX Association,
Berkeley, CA, USA. https://wsiw2018.l3s.uni-hannover.de/papers/wsiw2018Senarath.pdf (cited on p. 2, 4, 5).
[74] Awanthika Senarath and Nalin A. G. Arachchilage. 2018. Why developers cannot
embed privacy into software systems? An empirical investigation. In Proceedingsof the 22nd International Conference on Evaluation and Assessment in Software
Engineering 2018 (EASE’18). ACM, New York, NY, USA, 211–216. https://doi.org/
15

Proceedings on Privacy Enhancing Technologies YYYY(X)

Christine Utz, Sabrina Amft, Martin Degeling, Thorsten Holz, Sascha Fahl, and Florian Schaub

10.1145/3210459.3210484 (cited on p. 2, 4, 5, 11).
[75] Asuman Senol, Gunes Acar, Mathias Humbert, and Frederik Zuiderveen Borgesius. 2022. Leaky Forms: A Study of Email and Password Exfiltration Before
Form Submission. In Proceedings of the 31st USENIX Security Symposium (USENIX
Security 2022). USENIX Association, Berkeley, CA, USA. https://www.usenix.
org/system/files/sec22fall_senol.pdf (cited on p. 3).
[76] Jannick Sørensen and Sokol Kosta. 2019. Before and After GDPR: The Changes
in Third Party Presence at Public and Private European Websites. In Proceedingsof The World Wide Web Conference (WWW 2019). ACM, New York, NY, USA,
1590–1600. https://doi.org/10.1145/3308558.3313524 (cited on p. 1, 2, 3).
[77] Katta Spiel, Oliver Haimson, and Danielle Lottridge. 2019.
How
to do better with gender on surveys: A guide for HCI researchers.
https://interactions.acm.org/archive/view/july-august-2019/how-to-dobetter-with-gender-on-surveys (cited on p. 20).
[78] Stack Overflow. 2021. 2021 Developer Survey. Retrieved May 31, 2022 fromhttps://insights.stackoverflow.com/survey/2021 (cited on p. 6).
[79] Oleksii Starov, Phillipa Gill, and Nick Nikiforakis. 2016. Are You Sure You
Want to Contact Us? Quantifying the Leakage of PII via Website Contact Forms.
Proceedings on Privacy Enhancing Technologies 2016, 1 (Jan. 2016), 20–33. https://doi.org/10.1515/popets-2015-0028 (cited on p. 3).
[80] Tim Strack. 2020.
Use of Google Analytics without anonymizeIP isa violation of data protection law.
Retrieved February 18, 2022
fromhttps://web.archive.org/web/20210420080029/https://www.lhrlaw.de/en/magazine/use-of-google-analytics-without-anonymizeip-is-aviolation-of-data-protection-law/ (cited on p. 1, 12).
[81] Mohammad Tahaei, Alisa Frik, and Kami Vaniea. 2021. Deciding on Personalized
Ads: Nudging Developers About User Privacy. In Proceedings of the SeventeenthSymposium on Usable Privacy and Security (SOUPS 2021). USENIX Association,
Berkeley, CA, USA, 573–595. https://www.usenix.org/system/files/soups2021tahaei.pdf (cited on p. 2, 4, 5, 9).
[82] Mohammad Tahaei, Kopo M. Ramokapane, Tianshi Li, Jason I. Hong, and Awais
Rashid. 2022. Charting App Developers’ Journey Through Privacy Regulation
Features in Ad Networks. Proceedings on Privacy Enhancing Technologies 2022, 3(2022), 33–56. https://doi.org/10.56553/popets-2022-0061 (cited on p. 2, 3, 12).
[83] Mohammad Tahaei and Kami Vaniea. 2021. “Developers Are Responsible”: What
Ad Networks Tell Developers About Privacy. In Extended Abstracts of the 2021
CHI Conference on Human Factors in Computing Systems (CHI EA ’21). ACM, New
York, NY, USA, Article 253, 11 pages. https://doi.org/10.1145/3411763.3451805(cited on p. 4, 12).
[84] Mohammad Tahaei, Kami Vaniea, Konstantin Beznosov, and Maria K. Wolters.
2021. Security Notifications in Static Analysis Tools: Developers’ Attitudes,
Comprehension, and Ability to Act on Them. In Proceedings of the 2021 CHI
Conference on Human Factors in Computing Systems (CHI ’21). ACM, New York,NY, USA, Article 691, 17 pages. https://doi.org/10.1145/3411764.3445616 (cited
on p. 2, 4, 5).
[85] Mohammad Tahaei, Kami Vaniea, and Naomi Saphra. 2020. Understanding
Privacy-Related Questions on Stack Overflow. In Proceedings of the 2020 CHI
Conference on Human Factors in Computing Systems (CHI ’20). ACM, New York,NY, USA, 1–14. https://doi.org/10.1145/3313831.3376768 (cited on p. 2, 4, 10).
[86] The Tor Project. 2022. Research Safety Board. Retrieved September 13, 2022
from https://research.torproject.org/safetyboard/ (cited on p. 13).
[87] Tobias Urban, Martin Degeling, Thorsten Holz, and Norbert Pohlmann. 2020.
Beyond the Front Page: Measuring Third Party Dynamics in the Field. In Proceedings of The Web Conference 2020 (WWW ’20). ACM, New York, NY, USA,
1275–1286. https://doi.org/10.1145/3366423.3380203 (cited on p. 2, 6, 7).
[88] Tobias Urban, Dennis Tatang, Martin Degeling, Thorsten Holz, and Norbert
Pohlmann. 2020. Measuring the Impact of the GDPR on Data Sharing in Ad
Networks. In Proceedings of the 15th ACM Asia Conference on Computer andCommunications Security (ASIA CCS ’20). ACM, New York, NY, USA, 222–235.
https://doi.org/10.1145/3320269.3372194 (cited on p. 1, 8, 12).
[89] Christine Utz, Martin Degeling, Sascha Fahl, Florian Schaub, and Thorsten Holz.
2019. (Un)informed Consent: Studying GDPR Consent Notices in the Field. In
Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications
Security (CCS ’19). ACM, New York, NY, USA, 973–990. https://doi.org/10.1145/
3319535.3354212 (cited on p. 3).
[90] Rob van Eijk, Hadi Asghari, Philipp Winter, and Arvind Narayanan. 2019. The
Impact of User Location on Cookie Notices (Inside and Outside of the European
Union). In Workshop on Technology and Consumer Protection (ConPro ’19). IEEE,
Piscataway, NJ, USA. https://www.ieee-security.org/TC/SPW2019/ConPro/
papers/vaneijk-conpro19.pdf (cited on p. 6).
[91] Tim Wambach and Katharina Bräunlich. 2016. The Evolution of Third-Party Web
Tracking. In Proceedings of the Second International Conference on Information
Systems Security and Privacy (ICISSP 2016). Springer Nature Switzerland, Cham,
Switzerland, 130–147. https://doi.org/10.1007/978-3-319-54433-5_8 (cited on p.
2, 3).
[92] Chamila Wijayarathna and Nalin A. G. Arachchilage. 2018. Why Johnny Can’t
Store Passwords Securely? A Usability Evaluation of Bouncycastle Password
Hashing. In Proceedings of the 22nd International Conference on Evaluation andAssessment in Software Engineering 2018 (EASE’18). ACM, New York, NY, USA,
205–210. https://doi.org/10.1145/3210459.3210483 (cited on p. 2, 4, 5).
[93] Shomir Wilson, Florian Schaub, Aswarth Abhilash Dara, Frederick Liu, Sushain
Cherivirala, Pedro Giovanni Leon, Mads Schaarup Andersen, Sebastian Zimmeck,
Kanthashree Mysore Sathyendra, N. Cameron Russell, Thomas B. Norton, Eduard
Hovy, Joel Reidenberg, and Norman Sadeh. 2016. The Creation and Analysis of
a Website Privacy Policy Corpus. In Proceedings of the 54th Annual Meeting of
the Association for Computational Linguistics (Volume 1: Long Papers) (ACL 2016).
Association for Computational Linguistics, Stroudsburg, PA, USA, 1330–1340.
https://doi.org/10.18653/v1/P16-1126 (cited on p. 19).
[94] Zippia. 2022. Web Developer Demographics and Statistics in the US. Retrieved
May 31, 2022 from https://www.zippia.com/web-developer-jobs/demographics/(cited on p. 6).

16

Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websites

A

SURVEY

Proceedings on Privacy Enhancing Technologies YYYY(X)Please enter the website’s web address below, including the top-leveldomain (e. g., youtube.com, guardian.co.uk).
For the remainder of this survey, all questions are going to refer tothis website as “the website.” [free text](In the GitHub–Mandatory condition, we required participants to entersomething but did not check if it was a valid URL.)

This appendix presents the main part of the survey, i. e., without the intro
text, privacy policy, debriefing, and end message. Except for Q2-0 in the
GitHub–Mandatory condition, all questions were non-mandatory.

Survey Title

2b. Website Info. In Part 2 of the survey, we would like to learn some more
information about the website you just named.
2-1 What is / are your role(s) with regard to the website? [multiple
choice]• Product or project manager• Content creator or contributor• Social media manager• Marketing• Sales• Quality assurance• User experience• (Web) developer, programmer, or software engineer• Administrator or (web) operator• Legal counsel• Data protection officer• Customer service / customer support / customer relations• Other: [free text]
2-2 What is roughly the size of the team working on the website, i. e.,
how many people have been involved in the website’s design, development, deployment, maintenance, and management? [single choice,answer options: I am the only team member, 2–5, 6–10, 11–25, 26–50,
51–100, > 100, Don’t know]
2-3 Please select which country the company or organization operating
the website is based in. If the company or organization has sites in
multiple countries, please select the country in which the company
or organization’s headquarters are located. [single choice, answer
options: dropdown list with names of all countries]
2-4 What regions or countries is the website targeting or being used in?
[free text]
2-5 What is the website’s revenue model? [multiple choice]• Targeted advertising (e. g., ad networks)• Non-targeted advertising (e. g., contextual or static ads)
• Affiliate marketing / affiliate links• Donations• Subscriptions / membership• Sponsored posts / articles• Products / services sold on the website• Supported by other revenue streams (i.e., goods or services notdirectly sold on the website)• Other: [free text]• Not applicable (website does not have a revenue model)• Don’t know
2-6 Which of the following features or functionalities are used on thewebsite? [single choice for each, answer options: Yes / No / Not
sure]• Advertising (e. g., banner ads, video ads, content recommendation,affiliate links)• Customer / user interaction (e. g., user comments, contact forms,
chat, mailing lists)• Embedded media (e. g., video, audio, maps, slideshows)• Front-end libraries or design resources (e. g., non-standard fonts,
CSS frameworks, JavaScript libraries)• User login / authentication• Payment systems• Privacy popups / privacy forms (e. g., cookie consent notices, CCPA
“Do not sell”)

Web Technologies: Selection, Integration, and Configuration

1. Your Background
First we would like to learn about your background and your work on
websites. Throughout this survey, by “work on websites” we mean yourinvolvement to some degree in the design, development, deployment, maintenance, and/or management of a website.
1-1 How many websites have you worked on in the last 3 years? [single
choice, answer options: 0, 1, 2–5, 6–10, 11–25, 26–50, 51–100, > 100]
1-2 What is your current employment status with regard to your workon websites? [multiple choice]• Full-time employment• Part-time employment• Self-employed / freelancer• Intern• Hobbyist• Unemployed• Retired• Unable to work
• Other: [free text]• Prefer not to say
1-3 Below is a list of functionalities often found on websites. Which ofthese functionalities have you previously worked with on websites?
[multiple choice; order of answers randomized]• Advertising (e. g., banner ads, video ads, content recommendation,affiliate links)• Customer / user interaction (e. g., user comments, contact forms,
chat, mailing lists)• Embedded media (e. g., video, audio, maps, slideshows)• Front-end libraries or design resources (e. g., non-standard fonts,
CSS frameworks, JavaScript libraries)• User login / authentication• Payment systems• Privacy popups / privacy forms (e. g., cookie consent notices, CCPA
“Do not sell”)• Website protection (e. g., anti-spam, bot mitigation techniques)• Social media integration (e.g., social media buttons, widgets, embedded feeds)• Web analytics (e. g., page visits, heatmaps, session replay)
2a. Website. To learn more about your experience with different web technologies, the rest of the survey will ask you about a specific website you
have recently worked on.
2-0 Please name one website you recently worked on, i. e., you wereinvolved in the design, development, deployment, maintenance, ormanagement of that website, and that you remember well.
(If recruited via website: Ideally, this is the website through which
we contacted you, which is mentioned in the email invitation tothis survey. If you were not in any way involved in the design,development, deployment, maintenance, or management of thatwebsite, you are welcome to provide another website you recently
worked on.)We will keep this website – and any other information that could
identify you – confidential and only share it with involved researchers.
17

Proceedings on Privacy Enhancing Technologies YYYY(X)

Christine Utz, Sabrina Amft, Martin Degeling, Thorsten Holz, Sascha Fahl, and Florian Schaub· The (map | video | audio) resources are hosted on thewebsite’s host system· The (map | video | audio) resources are hosted with athird-party service (please name service:) [free text]· Other (please specify:) [free text]· Don’t know(b) (If map, audio, or video and third-party hosting:) How
are these externally hosted (map | video | audio resources)
embedded into the website? If the website uses multiple
technologies for this, please consider all of them combined (your “solution”) when answering the following
questions. [multiple choice]· Embedding code provided by the third party that hosts
the resources· Embedding code provided by another third-party service(please specify service:) [free text]· Embedding code we have written ourselves· Other (please specify:) [free text]· Don’t know(2) (If “Other”:) You indicated that the website uses someother kind of embedded content. How is this content integrated into the website? If the website uses multiple technologies for this, please consider all of them combined (your“solution”) when answering the following questions. [freetext]d. (Social Media:)i. What type of social media integration does the website use?
[multiple choice]∗ Profile buttons or links∗ Share buttons or widgets∗ Embedded posts or feeds∗ Other: [free text]∗ Don’t knowii. (1) (If profile / share buttons or embedded:) You indicatedthat the website uses (buttons or links to social media profiles | social media share buttons or widgets | embedded
social media posts or feeds). Which technology has beenused to integrate them into the website? If the website uses
multiple technologies for this, please consider all of them
combined (your “solution”) when answering the following
questions. [multiple choice]· Code we have written ourselves· Code provided by social media site(s)· Code or plugin provided by another third-party service(please specify service:) [free text]· Other (please specify:) [free text]· Don’t know(2) (If “Other”:) You indicated that the website uses someother kind of social media integration. How is it integrated
into the website? If the website uses multiple technologies
for this, please consider all of them combined (your “solution”) when answering the following questions. [free text]
3-3 (If involved in selection:) You indicated that you were involved indeciding how [FUNCTIONALITY] was integrated into the website.
Please describe why this specific type of integration or this particularservice was selected. [free text]
3-4 (If involved in selection:)a. (Generic:) When making this decision, were other ways for integrating [FUNCTIONALITY] into the website considered? [multiple choice]– We considered a solution we have developed (or were going todevelop) ourselves• Website protection (e. g., anti-spam, bot mitigation techniques)• Social media integration (e. g., social media buttons, widgets, embedded feeds)• Web analytics (e. g., page visits, heatmaps, session replay)
2-7 For each of the following functionalities present on the website,
how involved have you been regarding their integration into thewebsite? [list of all functionalities tagged with “Yes” in previous
question, single choice for each, answer options:]• I decided how to integrate this functionality• I integrated / implemented this functionality• I maintain or manage the integration of this functionality• I have not been involved in the integration of this functionality

3. Integration of Website Functionalities(category-specific)
In Part 3 we would like to ask you a few questions about the integration ofsome of the functionalities you indicated to have worked with on the website.
You will be shown these questions for at most three different functionalities,
regardless of how many you have selected in the previous question.
(For up to three categories randomly selected from those the participant has
indicated involvement in the previous question, they are asked the following
questions.)
You indicated that you have been involved to some degree in the integrationof [FUNCTIONALITY (examples)] on the website. Now we would like to ask
you a few more questions about how this functionality has been integrated.
3-1 For which purposes or use cases is [FUNCTIONALITY] technology
used on the website? [free text]
3-2a. (Generic:) Which technology has been used to integrate [FUNCTIONALITY] into the website? If the website uses multiple technologies for this, please consider all of them combined (your“solution”) when answering the following questions. [multiple
choice + free text]– We developed it ourselves– We installed a third-party software on the website’s host system(please name software:)– We integrated an external third-party service (please name service:)– Other (please specify):– Don’t knowb. (Payment:) What kind of payment service(s) does the website use?
[multiple choice + free text]– Payment method(s) that do not require other parties for processing (e. g., cash, gift cards) (please name method(s):)– Service(s) that only involve banks on either side (e. g., bank transfer, Lastschrift) (please name service(s):)– Service(s) that involve third parties (e. g., credit card, PayPal)(please name service(s):)– Other (please specify:)– Don’t knowc. (Embedded Media:)i. What type of embedded media does the website use? [multiple
choice]∗ Embedded maps
∗ Embedded videos∗ Embedded audio∗ Other (please specify:) [free text]∗ Don’t knowii. (1) (If map, audio, or video:) You indicated that the website
uses embedded (maps | videos | audio).
(a) Where are these (map | video | audio) resources hosted?
[multiple choice]18

Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websites– We considered (another) third-party software installed on thewebsite’s host system (please name software:) [free text]– We considered a(nother) service hosted with a third party (pleasename service(s):) [free text]– We directly decided to use the current solution– Other (please specify:) [free text]– Don’t knowb. (Payment:) When making this decision, were other ways for integrating payment systems into the website considered? [multiple
choice]– We considered (other) methods that do not include any otherparty (e. g., cash, gift cards) (please name method(s):) [freetext]– We considered service(s) that only involve banks on either side(please name service(s):) [free text]– We considered (other) service(s) that involve third parties (pleasename service(s):) [free text]– We directly decided to use the current solution– Other (please specify:) [free text]– Don’t knowc. (Embedded Media:) When making this decision were other ways
for integrating embedded media into the website considered?
[multiple choice]– We considered self-hosting the embedded media resources– We considered hosting the embedded media resources with a(nother)third party (please specify service:) [free text]– We considered embedding code provided by the third-party service that hosts the resources (please specify service:) [free text]– We considered embedding code provided by a different thirdparty service (please specify service:) [free text]– We considered embedding code we have written (or were goingto write) ourselves– We directly decided to use the current solution– Other (please specify:) [free text]– Don’t knowd. (Social Media:) When making this decision, were other ways for
integrating social media into the website considered? [multiple
choice]– We considered a solution we have developed (or were going todevelop) ourselves– We considered code provided by the social media site(s)– We considered a solution provided by a different third-party
service (please specify service:) [free text]– We directly decided to use the current solution– Other (please specify:) [free text]– Don’t know
3-5 (If involved in selection:) Why were other ways to integrate[FUNCTIONALITY] into the website (not) considered? [free text]
3-6 (If involved in selection:) Which sources of information did you
use to select a solution to integrate [FUNCTIONALITY] into thewebsite? [multiple choice]• The website’s team
• Professional network (people external to the website team)• Private network (e. g., friends)• Sales representative of third-party software / service• Official website(s) / documentation of third-party software / service• Legal documents by third-party software / service (e. g., terms ofservice, privacy policy)• Online blogs / magazine articles• Online discussion forums (e. g., Reddit, StackOverflow)• Other: [free text]

Proceedings on Privacy Enhancing Technologies YYYY(X)

3-7 (If involved in implementation or maintenance:) Which sourcesof information did you use to configure the [FUNCTIONALITY]
solution on the website? [multiple choice, same answer options as
in Q3-6]
3-8 (If not involved in selection:) You indicated that you were not involved in the decision how to integrate [FUNCTIONALITY] into thewebsite. Who decided how [FUNCTIONALITY] should be integrated
into the website? [multiple choice]• Product or project manager(s)• Content creator(s) or contributor(s)• Social media manager(s)• Marketing• Sales• Quality assurance• User experience• (Web) developer(s), programmer(s), or software engineer(s)• Administrator(s) or (web) operator(s)• Legal counsel(s)• Data protection officer(s)• Customer service / customer support / customer relations• CEO and/or other upper level management• Investor(s)• Other: [free text]• Don’t know
3-9 Overall, how satisfied are you with the [FUNCTIONALITY] integration solution on the website, with regard to the following criteria?
[single choice for each of the following, answer options: Very satisfied, Satisfied, Neither satisfied nor dissatisfied, Dissatisfied, Very
dissatisfied, Don’t know]• Visitors’ privacy• Ease of integration• Ease of use for visitors• Performance (e. g., page speed)• Features meet requirements

4. Data Practices of Website Functionalities(category-specific)
In Part 4 of the survey, we would like to learn more about your experience
with the data practices of the technologies we just asked you about in Part
3.
(The following questions are asked for each functionality for which the participant has also seen Part 3.)
4-1 (If third-party service is used to implement [FUNCTIONALITY]:) Sometimes third-party services, when integrated into a website, collect information about the website’s visitors, either to provide the service or for their own/other purposes. To the best of yourknowledge, what information about the website’s visitors does thethird-party solution used for [FUNCTIONALITY] collect?
[Items taken from the “Information Type” section of the annotation
scheme for the OPP-115 corpus of privacy policies [93]; single choicefor each, answer options: Yes, No, Unsure]• Financial information (e. g., credit or debit card data, credit scores)• Health, genetic, or biometric data
• Contact information (e. g., name, email address, phone number)• Location (e. g., GPS location, postal code)• Demographic data (e. g., gender, age, education)• Personal identifiers (e. g., social security, ID card or driver’s license
number)• User online activities (e. g., pages visited, time spent on pages)• User profile on the website (e. g., profile settings, data the user has
uploaded to the website)• Social media data19

Proceedings on Privacy Enhancing Technologies YYYY(X)

Christine Utz, Sabrina Amft, Martin Degeling, Thorsten Holz, Sascha Fahl, and Florian Schaub• Prefer not to disclose
5-5 Have you ever received any kind of training or educated yourself
on data protection or privacy? [single choice]• Yes (please specify:) [free text]• No• Prefer not to disclose• IP address or device IDs• Cookies or other tracking elements• Device information (e. g., browser or operating system used by website visitors)
4-2 (If involved in implementation or maintenance:) Did you makeany specific effort(s) to protect the website’s visitors’ privacy when
configuring the [FUNCTIONALITY] solution on the website? [single
choice]• Yes• No• Don’t know
4-3a. (If yes:) Please describe which efforts you have made and why. [freetext]b. (If no:) Please describe why you did not make any specific efforts.
[free text]

5. Demographics
Finally, we would like to ask you some basic demographic questions tobetter understand who participated in our study.
5-1 What is your age (in years)? [single choice] [18–24, 25–34, 35–44,
45–54, 55–64, 65–74, 75+, Prefer not to disclose]
5-2 What is your gender?2 [multiple choice]• Woman• Man• Nonbinary• Prefer to self-describe: [free text]• Prefer not to disclose
5-3 What is the highest educational degree you have completed? [single
choice]• No schooling completed• Some high school, no diploma• High school graduate, diploma, or equivalent (e. g., GED, Abitur,baccalauréat)• Some college credit, no degree• Trade / technical / vocational training• Associate degree• Bachelor’s degree• Master’s degree or equivalent (e. g., German Diplom)• Professional degree (e. g., JD, MD, German Staatsexamen)• Doctoral degree (e. g., PhD)
• Other: [free text]• Prefer not to disclose
5-4 In what field(s) did you receive your degree or vocational training?3[multiple choice]• Computer and information sciences• Mathematics• Engineering• Life sciences (e. g., biology, health sciences, medicine)• Social sciences / social work / human services• Education• Law• Psychology / behavioral science• Business / economics• Liberal arts / humanities• Art / music• Journalism• Vocational• Other: [free text]• Not applicable2 As recommended by Spiel et al. [77].
3 Adapted from a Pew Research survey [64], using the subcategories for some fields.

20

Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websites

B CODEBOOKS
B.1 Reasons For/Against Certain Solutions to
Integrate a Functionality (Q3-3/Q3-5)

Security Security practices to avoid known attacks or vulnerabilities (e. g.,
to avoid XSS) are in place, that increase privacy by decreasing theprobability of data leaks.
Other Other concrete reasons not covered by the codes above.
No answer The participant did not provide an answer to the question, either
by filling in nothing, something incomprehensible, or not providing
an answer to the question.

Revenue (Not) using this solution affects revenue and conversion, andtherefore income.
Performance (Not) using this service affects site performance, e. g., loading
times or server computation load.
Ease of Integration It is very easy/hard to implement or integrate the solution.
Ease of Use It is very easy/hard to use the solution (once it has been integrated).
Customization The solution can(not) be easily customized to the participant’s needs.
Features The solution (does not) offer(s) specific features that the participant
deems important for their use case.
Cost It would be cheap/expensive to use the solution.
Resources The solution was cheap in non-monetary resources, such as time
or workforce.
Popularity The solution is very popular, widespread, or even a market
leader.
Availability The solution is easily accessible, e. g., because it is already inuse.
Familiarity Friends, colleagues, or the participant themselves know or use
the service, allowing the participant to benefit from this experience.
Privacy Privacy was a relevant reason; the service was used because it, e. g.,allowed privacy-increasing configurations.
Security Security was a relevant reason; the service was used because it,
e. g., allowed security-increasing configurations.
Dependence (In)dependence on/from libraries or services that, e. g., might
suffer from outages or be abandoned by their developers in the
future.
Legal The service was used due to legal requirements to, e. g., add a privacy
policy or cookie banner.
Other Other concrete reasons not covered by the codes above.
No answer The participant did not provide an answer to the question, either
by filling in nothing, something incomprehensible, or not providing
an answer to the question (e. g., instead repeating what they did, not
why).

B.2

Proceedings on Privacy Enhancing Technologies YYYY(X)

B.3

Reasons to Protect Website Visitors’
Privacy (Q4-3a)

Regulatory Some regulatory framework, e. g., law or industry standards,
mandate privacy protection measures.
Requirement An unspecified requirement, e. g., by the customer, mandates
privacy protection measures.
Self-Commitment The participant applied privacy protection measuresout of intrinsic motivation, without external influence.

B.4

Reasons Not to Protect Website Visitors’
Privacy (Q4-3b)

No Data Collected The solution does not collect any personal data, so there
is no need for privacy protection.
Data Minimization Only strictly necessary data is collected, so there was/isno need for privacy protection.
Self-Hosting The service is self-hosted, and there is no need for additional
measures as access is limited and no external services are involved.
Trust in 3P Trust in the third party to employ adequate measures to protect
visitors’ privacy.
Impossible Data collection cannot be controlled or limited, it is impossible
to increase privacy.
Website Purpose The website’s purpose makes privacy protection unnecessary, e. g., because its main content is only accessible in a logged-instate.
Priorities Functionality (by adding third party services) has a higher priority than increasing privacy by avoiding these services.
Payoff Privacy measures include too much effort in terms of e. g., workload,cost, time.
Unnecessary It is not necessary to increase privacy. Answers with thiscode include no explanation, but often indicate a lack of awareness,care or external requirements.
Lack of Knowledge Participants are not able to adjust settings due to e. g.,a lack of knowledge or skill with the service.
Other Other concrete reasons not covered by the codes above.
No answer The participant did not provide an answer to the question, either
by filling in nothing, something incomprehensible, or not providing
an answer to the question.

Type of Effort Made to Protect Website
Visitors’ Privacy (Q4-3a)

No Personal Data No personal data is collected.
Data Minimization Only the necessary personal data is collected; datacollection is as minimal as possible.
Self-Hosting Services are self-hosted; all data stays within the respective
organization.
3P Selection Third-party services are carefully selected; there was a conscious decision for/against certain third parties.
3P Setting Third-party services are configured in ways that increase privacy,e. g., by limiting the amount of collected data, encrypting data etc.
User Consent Users were informed that their data would be available tothird parties and gave their consented to this data processing before
the functionality was loaded.
Transparency Privacy Policies or similar information on data practices isavailable to users.
Data Access The access to the data/server is limited; access is controlled.
Anonymization Data is anonymized and cannot be used to identify certain
individuals.
21

Proceedings on Privacy Enhancing Technologies YYYY(X)C

Christine Utz, Sabrina Amft, Martin Degeling, Thorsten Holz, Sascha Fahl, and Florian Schaub

PEOPLE AND RESOURCES INVOLVED IN SELECTION AND INTEGRATION
functionalities

Adve
In rtiste in
ra g
Em cti (13b. on ( )Pr Me 56)og dir aPr am (55iv m )
ac in
Pr y ( g (ot 21 80)e)
An ctio
al n (
yt 19
So ic
cia s ( )5
Pa l M 7)
ym edi
Lo ent a (3gi (1 9)n
/ A 3)
ut
h.
(5
4)

Ad
ve
In rtiste in
ra g
Em cti (12b. on ( )Pr Me 82)og dir a
Pr am (86iv m )
ac in
Pr y ( g (ot 36 155)e)
An ctio
al n (
yt 51
So ic
cia s ( )6
Pa l M 9)ym edi
Lo ent a (6gi (2 2)n
/ A 4)uth.
(83)

functionalities

1.0

Product/project manager 0.58 0.52 0.38 0.42 0.50 0.47 0.43 0.47 0.46 0.43

Product/project manager
Content creator
Social media manager
Marketing
Sales
Quality assurance
User experience
Developer/programmer
Admin/operator
Legal
Data protection officer
Customer support
CEO / management
Investor
Other
Don't know

Content creator 0.42 0.48 0.42 0.39 0.39 0.39 0.39 0.52 0.38 0.31
Social media manager 0.33 0.27 0.12 0.15 0.14 0.10 0.13 0.26 0.25 0.10

0.8

Marketing 0.25 0.24 0.15 0.17 0.25 0.16 0.20 0.27 0.33 0.10
Sales 0.25 0.07 0.07 0.03 0.08 0.10 0.09 0.11 0.17 0.04
0.6

roles

roles

Quality assurance 0.33 0.32 0.30 0.26 0.28 0.27 0.32 0.27 0.29 0.23
User experience 0.33 0.60 0.51 0.51 0.44 0.43 0.55 0.56 0.54 0.37
Developer/programmer 0.92 0.90 0.91 0.94 0.83 0.92 0.87 0.89 0.88 0.93

0.4

Admin/operator 0.50 0.71 0.51 0.54 0.61 0.63 0.64 0.56 0.54 0.54
Legal 0.08 0.06 0.06 0.04 0.06 0.06 0.07 0.08 0.04 0.02
0.2

Data protection officer 0.25 0.15 0.13 0.14 0.19 0.29 0.16 0.13 0.17 0.18
Customer support 0.17 0.34 0.23 0.21 0.28 0.25 0.25 0.21 0.25 0.24
Other 0.00 0.04 0.03 0.05 0.06 0.08 0.07 0.08 0.12 0.08

0.0

0.15 0.25 0.16 0.19 0.24 0.16 0.23 0.15 0.31 0.19

0.5

0.08 0.04 0.09 0.03 0.05 0.00 0.02 0.03 0.00 0.04
0.08 0.02 0.07 0.01 0.00 0.00 0.02 0.10 0.00 0.00
0.15 0.09 0.02 0.03 0.05 0.00 0.14 0.10 0.08 0.00

0.4

0.00 0.02 0.02 0.01 0.00 0.00 0.07 0.00 0.00 0.00
0.00 0.04 0.02 0.03 0.00 0.05 0.00 0.03 0.00 0.02
0.08 0.16 0.11 0.07 0.10 0.05 0.04 0.15 0.00 0.04

0.3

0.00 0.48 0.42 0.44 0.29 0.53 0.35 0.23 0.38 0.50
0.08 0.16 0.11 0.05 0.10 0.26 0.14 0.10 0.00 0.22
0.08 0.07 0.02 0.01 0.38 0.00 0.05 0.05 0.00 0.04

0.2

0.08 0.05 0.00 0.01 0.29 0.00 0.07 0.05 0.00 0.06
0.00 0.05 0.00 0.03 0.00 0.00 0.05 0.00 0.00 0.02
0.15 0.16 0.11 0.11 0.29 0.11 0.21 0.15 0.23 0.11

0.1

0.00 0.04 0.04 0.04 0.05 0.05 0.02 0.00 0.00 0.00
0.08 0.04 0.09 0.09 0.05 0.16 0.09 0.10 0.15 0.06
0.31 0.09 0.05 0.09 0.00 0.11 0.09 0.13 0.00 0.06

0.0

Figure 5: Responsibility for selection: for participants in- Figure 6: Responsibility for selection: for participants not involved in the selection of a given functionality (Q2-7), their volved in selection, who was responsible (Q3-8).
roles in relation to the website (Q2-1).

Resources used for selection
Website's team
Professional network

Private network
Sales representative

Official website(s) / docs
ToS / privacy policy

Online articles
Online forums

Other
NaN

0.6

frequency

0.5
0.4
0.3
0.2
0.1

n
gi
Lo

Pa

ym

/A
ut

en

h.

t(

(8

24

3)

)

2)
cia

So

An

al

lM

yt

ed

ics

ia

(6

(6

9)

1)
tio
ec
ot
Pr

Pr

iv

ac

y

n

(3

(5

6)

)
55
g
m
m
ra

og
Pr

Em

be

dd

In
t

ed

er

M

in

ed

ac
tio

n

ia

(1

(8

2)
(8

2)
(1
Ad
s

6)

0.0

functionalities

Figure 7: Resources used to select how to integrate a website functionality (Q3-6). Numbers are relative to the people involved
in selection of the respective functionality, shown in the x-axis labels.

22

Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websites

Proceedings on Privacy Enhancing Technologies YYYY(X)Resources used for integration
Website's team
Professional network

Private network
Sales representative

Official website(s) / docs
ToS / privacy policy

Online articles
Online forums

Other
NaN

frequency

0.6

0.4

0.2

/A
n
gi
Lo

Pa

ym

ut
h

en

.(

t(

10

29

2)

)

)
ed
ia
lM

So
c

An

al
y

tic

ia

s(
99

(4
n
ct
io
te
Pr
o

(7
1

)

8)

)
ac
Pr
iv

(1
g
m
in
ra
m
Pr

og

y

85

(4
2

)

)
09
(1
ia
M
ed

dd
ed
Em
be

In

te

ra
ct

io

n

Ad

(1

s(

02

18

)

)

0.0

functionalities

Figure 8: Resources used in the integration of a website functionality (Q3-7). Numbers are relative to the people involved in
integration or maintenance of the respective functionality, shown in the x-axis labels.

23

Proceedings on Privacy Enhancing Technologies YYYY(X)D

Christine Utz, Sabrina Amft, Martin Degeling, Thorsten Holz, Sascha Fahl, and Florian Schaub

PARTICIPANT & WEBSITE STATISTICS

Table 6: Participants’ demographics (Part 5 of the survey) and background (Part 1 of the survey, Q2-1, and Q2-2). C indicatescoded open-ended answers, M indicates multiple-choice questions or multiply assigned codes for which (response) counts can
sum up to more than 100 %. Percentage values are relative to the total number of survey responses (𝑛 = 395). For the coded
open-ended answers to the type of privacy training received (Q5-5; bottom left, indented list), percentage values are relative
to to the number of participants who indicated to have received prior privacy training (𝑛 = 166).

40
336
4
3
13

10.1
85.1
1.0
0.8
3.3

No schooling completed
Some high school, no diploma
High school graduate
Some college credit, no degree
Trade / technical / vocat. training
Associate degree
Bachelor’s degree
Master’s degree
Professional degree
Doctoral degree
Other
N/A

5
14
57
39
13
5
139
77
9
21
4
2

1.3
3.5
14.4
9.9
3.3
1.3
35.2
19.5
2.3
5.3
1.0
0.5

Computer & information sciences
Mathematics
Engineering
Life sciences
Physical sciences
Social sciences
Education
Law
Psychology
Business / economics
Liberal arts / humanities
Art / music
Journalism
Vocational
Not applicable
Other
N/A

222
53
89
19
26
23
19
2
5
41
23
10
7
3
24
9
12

56.2
13.4
22.5
4.8
6.6
5.8
4.8
0.5
1.3
10.4
5.8
2.5
1.8
0.8
6.1
2.3
3.0

Yes

166

42.0

Self-taught
Employer training‘Learning by doing’
University / school
Online courses
Other courses
Professional network
Other
N/A
No
N/A

64
39
10
18
11
25
7
5
15
189
40

38.6
23.5
6.0
10.8
6.6
15.1
4.2
3.0
9.0
47.8
10.1

24

%

# Websites

GenderM

Woman
Man
NonbinarySelf-described
N/A

n

1
2–5
6–10
11–25
26–50
51–100
> 100
N/A

18
173
107
47
29
10
10
1

4.6
43.8
27.1
11.9
7.3
2.5
2.5
0.3

Employm. TypeM

33.4
30.6
19.2
7.6
5.1
1.3
0.3
2.6

Full-time employment
Part-time employment
Self-employment / freelancer
InternStudent
Hobbyist
Unemployed
Retired
Other

165
49
130
30
15
124
39
3
6

41.8
12.4
32.9
7.6
3.8
31.4
9.9
0.8
1.5

Exp. w. FunctionalityM

132
121
76
30
20
5
1
10

Advertising
Analytics
Customer interaction
Embedded media
User login / authentication
Payment
Programming / design
Privacy popups / forms
Social media integration
Website protection
N/A

91
215
293
258
318
129
328
118
204
130
1

23.0
54.4
74.2
65.3
80.5
32.7
83.0
29.9
51.6
32.9
0.3

Role(s) with WebsiteM

Age

18–24
25–34
35–44
45–54
55–64
65–74
75+
N/A

Privacy TrainingC, M%

Education

BackgroundnField of DegreeM

Demographics

Product / project manager
Content creator / contributor
Social media manager
Marketing
Sales
Quality assurance
User experience(Web) developer etc.
Administrator / (web) operator
Legal counsel
Data protection officer
Customer support / relations
Other

136
142
51
63
19
93
162
337
194
13
43
71
19

34.4
35.9
12.9
15.9
4.8
23.5
41.0
85.3
49.1
3.3
10.9
18.0
4.8

Privacy Rarely Considered: Exploring Considerations in the Adoption of Third-Party Services by Websites

Proceedings on Privacy Enhancing Technologies YYYY(X)

Table 7: Statistics about the self-selected websites participants considered while answering the survey. C indicates coded openended answers, M indicates multiple-choice questions or multiply assigned codes or tags for which (response) counts can sum
up to more than 100 %. Statistics in the left column are from Part 2 of the survey and percentage values are relative to the total
number of survey responses (𝑛 = 395). Statistics in the right column result from the analysis of the website URLs provided
by participants in Q2-0 and percentage values are relative to the number of unique entered domains (𝑛 = 361). For the mostfrequently occurring TLDs, subdomains on popular hosting platforms such as github.io or herokuapp.com, used by 57 sites,
were considered distinct TLDs. .

Team SizeI am the only team member
2–5
6–10
11–25
26–50
51–100> 100
Don’t know

145
141
50
36
5
5
10
3

36.7
35.7
12.7
9.1
1.3
1.3
2.5
0.8

Country of Website HQ

United States of America
Germany
United Kingdom
RussiaBrazilIndia
China
Switzerland
CanadaThe Netherlands
Other
N/A

70
46
21
20
18
15
13
12
11
11
154
4

17.7
11.6
5.3
5.1
4.6
3.8
3.3
3.0
2.8
2.8
39.0
1.0

Target Region/AudienceC

GlobalEuropeMultiple regions
United States of America
East Asia
BrazilSoutheast Asia
AfricaRussia / CIS
North America
Other
N/A

128
56
30
26
17
15
15
12
12
11
20
53

32.4
14.2
7.6
6.6
4.3
3.8
3.8
3.0
3.0
2.8
5.1
13.4

Targeted advertising
Non-targeted advertising
Affiliate marketing / links
Donations
Subscriptions / membership
Sponsored posts / articles
Products / services sold on website
Other revenue streams
Not applicable / no revenue modelDon’t know
Other
N/A

32
22
21
37
69
22
81
57
177
5
17
2

8.1
5.6
5.3
9.4
17.5
5.6
20.5
14.4
44.8
1.3
4.3
0.5

Site Analysis

25

n

%

Top-Level Domains

%
.com
.org
.de
.github.io.herokuapp.com
.dev
.net
.com.br
.ru
.io
Other

107
30
24
19
17
12
11
10
10
7
115

29.6
8.3
6.6
5.3
4.7
3.3
3.0
2.7
2.7
1.9
31.9

Website CategoriesMn

Revenue modelM

Survey Responses

Business
Internet Services
Education / Reference
Personal Pages
Software / Hardware
Interactive Web Applications
Blogs / Wiki
Marketing / Merchandising
Finance / Banking
Online Shopping
Other
Uncategorized

65
54
38
21
19
18
15
11
10
10
129
48

18.0
15.0
10.5
5.8
5.3
5.0
4.2
3.0
2.8
2.8
35.7
13.3