Deceptive, Disruptive, No Big Deal: Japanese People React toSimulated Dark Commercial Patterns
Katie Seaborn

seaborn.k.aa@m.titech.ac.jp
Tokyo Institute of Technology
Tokyo, Japan

Yijia Wangwang.y.cf@m.titech.ac.jp
Tokyo Institute of Technology
Tokyo, Japan

Yuto Mandai

mandai.y.aa@m.titech.ac.jp
Tokyo Institute of Technology
Tokyo, Japan

Tatsuya Itagaki

Tokyo Institute of Technology
Tokyo, Japan
itagaki.t.ad@m.titech.ac.jp

Ping Geng

geng.p.aa@m.titech.ac.jp
Tokyo Institute of Technology
Tokyo, Japan

Miu Kojima

kojima.m.ap@m.titech.ac.jp
Tokyo Institute of Technology
Tokyo, Japan

Mizuki Watanabe

Tokyo Institute of Technology
Tokyo, Japanwatanabe.m.ca@m.titech.ac.jp

Takao Fujii

fujii.t.av@m.titech.ac.jp
Tokyo Institute of Technology
Tokyo, Japan

Suzuka Yoshidayoshida.s.av@m.titech.ac.jp
Tokyo Institute of Technology
Tokyo, Japan

Figure 1: User flow of the website representing a typical online shopping experience involving all DPs. Illustration © shigureni.

ABSTRACT
Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
CHI EA ’24, May 11–16, 2024, Honolulu, HI, USA© 2024 Copyright held by the owner/author(s).
ACM ISBN 979-8-4007-0331-7/24/05
https://doi.org/10.1145/3613905.3651099

Dark patterns and deceptive designs (DPs) are user interface elements that trick people into taking actions that benefit the purveyor.
Such designs are widely deployed, with special varieties found incertain nations like Japan that can be traced to global power hierarchies and the local socio-linguistic context of use. In this breaking
work, we report on the first user study involving Japanese people

CHI EA ’24, May 11–16, 2024, Honolulu, HI, USA(n=30) experiencing a mock shopping website injected with simulated DPs. We found that Alphabet Soup and Misleading Reference
Pricing were the most deceptive and least noticeable. Social Proofs,
Sneaking in Items, and Untranslation were the least deceptive but
Untranslation prevented most from cancelling their account. Moodsignificantly worsened after experiencing the website. We contribute the first empirical findings on a Japanese consumer basealongside a scalable approach to evaluating user attitudes, perceptions, and behaviours towards DPs in an interactive context. We
urge for more human participant research and ideally collaborations
with industry to assess real designs in the wild.

CCS CONCEPTS• Human-centered computing → Empirical studies in HCI;
User studies; User models; Web-based interaction; User interface
design; • Information systems → Web applications; Web interfaces; • Applied computing → Psychology.

KEYWORDS
Dark Patterns, Deceptive Design, Deceptive Design Pattern, Manipulative Design, Persuasive Design, User Interface Design, User
Study, JapanACM Reference Format:
Katie Seaborn, Tatsuya Itagaki, Mizuki Watanabe, Yijia Wang, Ping Geng,
Takao Fujii, Yuto Mandai, Miu Kojima, and Suzuka Yoshida. 2024. Deceptive,
Disruptive, No Big Deal: Japanese People React to Simulated Dark Commercial Patterns. In Extended Abstracts of the CHI Conference on Human Factors
in Computing Systems (CHI EA ’24), May 11–16, 2024, Honolulu, HI, USA.
ACM, New York, NY, USA, 8 pages. https://doi.org/10.1145/3613905.3651099

1

INTRODUCTION AND BACKGROUND

Dark patterns and deceptive designs (DPs) are aspects of user interfaces (UI) that have been specifically designed to manipulate
user actions and choices [22, 26]. Much work has gone into describing [21, 23], finding [16, 24, 29, 31, 35, 44, 52], and regulating [41, 42] DPs. However, relatively less work has explored theuser side of deceptive UIs: how everyday consumers, end-users,
and netizens feel about the DPs that they encounter. Notably, few
have explored real digital offerings using participant research methods [8, 12, 20, 34, 40]. This gap can be traced to the sensitive and evencontroversial nature of DPs. When portrayed as “deceptive,” theimplication for companies and other purveyors who employ DPs isnegative. Also, whether a particular interface pattern is “deceptive”
and for whom remains a matter of debate [17, 18, 21, 23, 31, 36].
While ideal, the sensitivity of the topic, ethical restrictions, andpractical issues have hampered work with real users on real products. Some researchers have not even named the systems evaluated,e.g., [8, 29, 40]. One alternative is the use of realistic simulations,which include simulated DPs injected into real systems [3, 4, 39, 47],
imagined designs of complete apps [7, 43], and fully-fledged systems created from scratch [15, 19, 25, 49, 50]. Still, much of thiswork focuses on specific contexts or DPs in isolation, e.g., app permission requests [7], cookie consent banners [3, 4, 15, 25], the Sneak
into Basket/Toying with Emotion DP classes [49]. Moreover, thesimulations are often non-interactive—screen shots, mockups, andvideos, e.g., [7, 16, 43]—which limits ecological validity in terms of

Seaborn et al.

use case realism and assessment of DPs that require interaction [11].
Much more work is needed—with a larger volume of potential andreal end-users across an array of digital offerings and varieties of
DPs—before consensus can be reached.
One emerging user base to study is Japanese consumers. Deception in commercial UIs has become a key topic in Japan, especiallyas the number of online shoppers continues to grow. Nearly 53%
of households went online to shop in 2022, a sharp incline from
34.3% in 20181 . Last CHI, Hidaka et al. [29] reported on a heuristic
analysis of DPs in the Japanese mobile app market. They found that
93.5% of the top 200 Google Play Store apps contained an average of
3.9 DPs [29]. Moreover, they also discovered the presence of special
Japanese DPs linked to global power structures, i.e., the influenceof foreign corporations and their local presence in Japan, and theJapanese socio-linguistic context [29]. Even so, to date, no researchin Japan has involved human participants directly experiencing
DPs. Japan has also started to take legal action on DPs [41]. The
Consumer Affairs Agency of the Government of Japan also led the2023 OECD follow-up report on vulnerability and consumer policy [42]. Even so, the degree to which Japanese netizens are awareof DPs, how they feel about their “deceptive” properties, and theimplications in typical use cases remain unknown.
To this end, we conducted the first human participants user
study on DPs with a diversity of Japanese people. We asked two
entwined research questions (RQs): RQ1: How deceptive are DPs
to the average Japanese person? and RQ2: How does the average
Japanese person feel about each form of DP? Due to ethics
restrictions, we developed a simulation (refer to 2.3.1) containing
a range of DPs based on those typically found in real e-commerce
websites (refer to Figure 1 and 2.3 for details). We found that the
deceptibility of people and interfaces varied significantly, and moodwas negatively affected by the experience. We contribute:• the first empirical findings on how Japanese consumers perceive a variety of DPs, including the Japanese varieties; and• a methodology for conducting interactive human participant research on DPs using a simulated website that can betranslated to in-situ studies.
We hope to inspire further interactive, participant-based research on
DPs in Japan and elsewhere. We also aim to create a sensitivity-freeartefact and training tool for engaging consumers and purveyors
alike in discussions on deception in design.

2

METHODS

We conducted a human factors user study [45] featuring a concurrent think-aloud protocol [1, 48], questionnaires, and a semistructured interview [5]. We received ethics approval (#2023265)
and conducted the study in December 2023.

2.1

Participants

We recruited 30 Japanese people with the aid of NHK, a publicbroadcaster and third-party recruiter, aiming to capture a diverse
sample from the larger population. Fourteen women and sixteenmen (none of another gender identity) participated. Two were aged
18–24, three 25–34, two 35–44, eight 45–54, nine 55–64, four 65–74,
1 https://www.statista.com/statistics/1182675/japan-online-shopping-penetration-

households/

Deceptive, Disruptive, No Big Dealand two 75+. Most had gone to university but not completed studies(n=11), while eight had a bachelor’s degree, two a graduate degree,
one a degree from a preparatory school, and seven the equivalentof a high school diploma; one had not completed high school yet.
Most did not know about DPs (only four of 28; for two, this data
was missed). The recruiter used several different participant pools,
each with a different rate of compensation; hence, participants
were compensated in accordance with each pool at ∼5000-8000 yen(USD ∼$30-50) each. Participants were aware and gave consent inadvance for their anonymous data and screen footage to be used
for media broadcast as well as academic research.

2.2

Procedure

Participants were asked to evaluate the interactive features of a
new e-commerce website. As in prior work, e.g., [4], the true focus on DPs was hidden to avoid priming [27]. Participants were
informed about the procedure (without mentioning DPs) and provided consent. They then sat before a laptop and filled out the
pre-questionnaire (refer to 2.4.1).
Next, participants watched a training video about the concurrent think aloud protocol procedure. A staple of HCI research, thisprotocol involves participants speaking their thoughts out loudwhile performing tasks with a system [1, 48]. In the video, a researcher performed the protocol while reading a cook book. Then,
participants practiced with the same cook book for ∼2 minutes.
Next, they experienced the shopping website. They were guided
by a researcher through a typical shopping user flow (refer toFigure 1 and 2.3.3). They were not forced to carry out the tasks in acertain way or order. This meant that not all participants viewed all
pages on the website or experienced all DPs. An observer watchedand filled in the observation checklist (refer to 2.4.2). After the task,
participants filled out the post-questionnaire (refer to 2.4.1).
Participants were then interviewed about their experience (refer
to 2.4.3). They were told about DPs and shown a PDF containing
screen shots of each page in the website. They were asked to pointout and describe any notable parts. The observer noted whether
they noticed any DPs this second time (refer to 2.4.2).
Finally, the host researcher revealed the true nature of the study:
to evaluate DPs. Participants were asked to keep the true focus of
the study on DPs a secret. Each session took ∼50-60 minutes.

2.3

Materials and Activity

2.3.1 Website Simulation. A small team of developers created amock online shopping website for a fake electronics retailer called“CyberStore,” (Figure 2), which was modelled on e-commerce websites used by Japanese people. A WordPress WooCommerce theme2
was modified to embed DPs within the shopping experience.
2.3.2 DPs. We used the seven categories of DPs compiled in 2022
by the Organisation for Economic Co-operation and Development(OECD) [41], which was based on the taxonomical work of Contiand Sobiesk [13], Bösch et al. [9], Gray et al. [21], Mathur et al. [35],and Luguri and Strahilevitz [33], plus the Japanese-only DPs foundin 2023 by Hidaka et al. [29]: Forced Action, Interface Interference,
2 Powered by the WooCommerce WordPress plugin with a customized WP Cirqle

theme as the base.

CHI EA ’24, May 11–16, 2024, Honolulu, HI, USA

Nagging, Obstruction, Sneaking, Social Proof, Urgency, and Linguistic Dead-Ends. For each, we created DP classes—subtypes of
DPs within these categories—and DP cases—a point of interaction
wherein one or more DPs can be experienced as a gestalt [16]. These
were based on typical examples found in e-commerce websites andsourced from Brignull’s “Hall of Shame.”3 Refer to the visual guideon OSF4 or the Supplementary Materials for screen shots (AppendixA) and a full list (Appendix C) of each DP case.
2.3.3 Tasks. We developed a user flow (Figure 1) representing typical tasks in an online shopping environment that would also leadparticipants to experience all DPs within the website (Figure 3).
Specifically, the tasks were: (i) Browse the homepage; (ii) Browse
the shop for a set of items: a computer, a bag, and cleaning supplies;(iii) Check-out and sign up; and (iv) Cancel your membership. Participants were asked to carry out the tasks in this order. However,they were not forced to complete each task nor prevented from
exploring the website. In doing so, they were additionally able toexperience other DPs placed throughout the website. The DPs that
could be experienced per task are outlined in Figure 1. The tasks
were rigorously developed to ensure realism and encounters withall DPs, as well as timed in pilot tests with Japanese lab members.
Participants were given a 3-minute limit for the last task.

2.4

Data Collection

2.4.1 Questionnaire Instruments and Measures. We used pre- andpost-task questionnaires. We used valence and arousal to assess
affective state before and after the experience. This duo is among
the most common subjective measures of emotion, with arousal
in particular found to thoroughly explain affective states [37]. For
self-reporting, we used the valence and arousal dimensions fromthe face-based tactile version [30] of the Self-Assessment Manikin(SAM) [10]. This language-free self-report instrument uses a 9-point
semantic differential scale, where 1 is low valence/arousal and 9
is high valence/arousal. In the post-task questionnaire, we used
the Japanese translation [53] of the 10-item System Usability Scale(SUS) [32] to assess acceptability [2] with a 5-point Likert scale.
2.4.2 Observation Checklist and Deceptibility Metric. We created achecklist for observers to record whether each participant noticed
each DP and their reaction to it. To account for priming [27], thechecklist distinguished between the participant first experiencing
the website, after being prompted to comment on the page (as partof the concurrent think-aloud protocol), and the semi-structured
interview, which involved revealing the focus on DPs and the user
flow (refer to 2.4.3). The checklist also accounted for the chance thatparticipants were not tricked by certain DPs or found alternative
paths through the website, thus missing certain DPs. The noticed
options were: on their own; after prompting; during the reveal;
missed chance; and did not notice. The reaction options were: felt
manipulated or without a choice, and felt negative (“deceptive”); disturbed or interrupted and wanted to avoid it (“disruptive”); acceptedit (“no big deal”); and no reaction, i.e., tricked.
We used the noticed data to create a deceptibility metric basedon the lead researcher’s expertise in DPs and as a designer and
3 https://www.deceptive.design/hall-of-shame

4 https://osf.io/65wzr

CHI EA ’24, May 11–16, 2024, Honolulu, HI, USA

Seaborn et al.

Figure 2: Screen shots of the website: the homepage (left), check-out page (middle), and acceptable use policy page (right).

Figure 3: A participant experiences an unintentional premium membership sign-up after making a purchase (left) and an
unnecessary location disclosure request pop-up (right).
scholar with over 15 years experience in UX. We added the ratios ofeach option with weights: .7 for noticing on their own; .25 for after
prompted, and .05 for after learning about DPs. We multiplied thissum by the quotient of 10/7 and inverted the product for a measure
between 0 and 1, where 0 is undeceived and 1 is fully deceived.

(parametric and/or non-parametric, depending on the data types
and normality results via Shapiro-Wilk tests). One participant did
not supply “before” affect scores and another did not fill out the
SUS; their data was excluded from analyses using these measures.
Quotes were sourced from the interviews.

2.4.3 Semi-Structured Interview. We used semi-structured interviews [5] with a funnel approach [51] to avoid priming responses
while also capturing prior familiarity with and understanding ofDPs. The questions were: Have you heard of “dark patterns” before?
Have you experienced dark patterns before? Were there any aspectsof the design that you felt were deceptive during operation? Whatother feelings or impressions do you have about the design? Refer toAppendix B for the Japanese versions. The participant was showna PDF (refer Appendix A) of the user flow and asked to point out
specific aspects of the design that aligned with their perceptions
and feelings. The observer recorded their responses in the checklist.

3 RESULTS
3.1 Deception: Ability, Noticeability, Trickery,and Deceptibility (RQ1)

2.5

Data Analysis

We generated descriptive statistics by participant, DP class, DP
case, and DP category. We used the acceptability interpretationof the SUS from Bangor et al. [2]: 51.6 as unacceptable, 51.7–71 as
marginal, and 71.1–100 as acceptable. We ran exploratory inferential
statistics to evaluate affect before and after the experience (paired
t-test) and relationships between deceptibility and other variables

3.1.1 Task Completion. Two DP cases affected the path throughthe website: (i) a Hidden Subscription to the “CyberSelect” premium membership through Trick Wording and Pre“Un”Selectionof a checkbox at checkout and (ii) Untranslation of the accountcancellation page (Hard to Cancel). For (i), half unintentionally
signed up for “CyberSelect.” Of the 42% that noticed, 20% accepted
it, 6% felt it was deception, and 75% felt it was disruptive. Most
(n=27, 90%) were unable to complete (ii). Of the 35% that noticed,
deceptive (72%) or disruptive (15%), but some accepted it (13%).
3.1.2 Noticeability and Reactions to Trickery. Descriptive statistics
on the noticeability and reactions to all 25 DP cases are in Supplementary Tables 2 and 3 (refer to Appendix D). Most (58%) DP
cases were unnoticeable (M=17.3, SD=7, MD=19.3, IQR=12). When
noticed, 47% cases were deemed “no big deal” (M=6.4, SD=6, MD=5,
IQR=3.5), 29% were deceptive (M=3.7, SD=4.9, MD=2, IQR=2), andDeceptive, Disruptive, No Big Deal

24% were disruptive (M=2.6, SD=2.9, MD=2, IQR=2). The most noticeable were the homepage call-outs (n=25, 83%), the low stock
indicators (n=24, 80%), the testimonials on the product pages (n=23,77%), and Sneaking an Item (n=23, 77%). The least noticeable were
the CyberSelect membership fee featuring Alphabet Soup and theSocial Proof on the search dropdown (only three each).
Descriptive statistics for DP category and class are in Supplementary Table 4 (Appendix D). The most noticeable category was
Obstruction (80%) and the least was Forced Action (73%). Obstruction was most deceptive (53%), Interface Interference was most disruptive (68%), and Urgency was most acceptable (41%). For classes,
Sneaking an Item (Sneaking) and Testimonial (Social Proof) weremost noticeable (23%), while Alphabet Soup (Linguistic Dead-End)and Misleading Reference Pricing (Interface Interference) were least
noticeable (90%). Sneaking an Item (Sneaking) (58%), Hard to Cancel(Obstruction) (53%), and Untranslation (Linguistic Dead-End) (48%)
were most deceptive. Trick Questions (Interface Interference) weremost disruptive (53%). Low Stock, High Demand (Urgency) (67%)
and Testimonials (Social Proof) (63%) were most acceptable.
3.1.3 Deceptibility Metric. We calculated deceptibility metrics forparticipants and DPs (refer to Appendix D).
Participant deceptibility ran the gamut: a Shapiro-Wilk test did
not find a statistically significant departure from normality, W (30) =.98, p = .783, indicating a bell curve. The highest score was 0.92 andthe lowest was 0.40 (M=0.7, SD=0.1, MD=0.7, IQR=0.1). There wereno statistically significant correlations to age, gender, or education.
This indicates that everyone was susceptible to DPs, erring on theside of being more deceived overall (skewed -0.4).
The deceptibility of DP cases (Figure 4, right) varied (M=0.7,
SD=0.2, MD=0.7, IQR=0.3) but were skewed high (-0.6). A ShapiroWilk test indicated a statistically significant departure from normality, W (25) = .92, p = .047. The most deceptive were the CyberSelectfee with Alphabet Soup (0.96), search dropdown with Social Proofs
and False Hierarchies (0.93), and membership price comparison
chart (0.92). The least were sneaking in an item (0.25), the productpage testimonials (0.26), and the low stock indicators (0.28).
At a higher level, the deceptibility of DP categories and classes(Figure 4, left) also varied (M=0.6, SD=0.2, MD=0.7, IQR=0.2). A
Shapiro-Wilk test found a statistically significant departure fromnormality, W (28) = .92, p = .033, skewed high (-0.7). The most deceptive categories were Forced Action (0.76), Interface Interference(0.75) and Linguistic Dead-Ends (0.75). The least was Obstruction(0.40). The most deceptive classes were Alphabet Soup (0.96), Misleading Reference Pricing (0.90), Hidden Information (0.86) andConfirmShaming (0.86). The least were Sneaking an Item (0.25),
Testimonials (0.26), Trick Questions (0.35), and Hard to cancel (0.40).
In short, some DPs were more deceptive than others, thoughmost were deceptive. Later, we will discuss the nuances of theseresults given the noticeability results and reactions, as well as the
feelings of participants, up next.

3.2

Feelings: Mood and Judgments (RQ2)

The SUS scores indicated that most participants did not accept thewebsite (M=35.5, SD=32.5, MD=32.5, IQR=32.5, hi=85, lo=0). 69%(n=20) deemed the website unacceptable, with three accepting it
and six with marginal scores.

CHI EA ’24, May 11–16, 2024, Honolulu, HI, USA

A paired-t test showed a statistically significant large difference
between valence before (M=6.9, SD=1.4) and after (M=4.6, SD=2.5),t(28) = 5, p < .001. Since the data was non-normal, we confirmed
this result with a Wilcoxon Signed-Rank test, which indicated thesame for before (M=7, n=29) and after (MD=5, n=29), Z = -3.7, p< .001, r = -0.8. In short, valence decreased significantly for mostpeople right after experiencing the website. Results of a paired-t test
for arousal also indicated a statistically significant large difference
before (M=2.9, SD=2) and after (M=5.5, SD=2.4), t(28) = 5.5, p < .001.
Arousal greatly rose for most people after the experience. Taken
together, the valence and arousal results suggest that people became
significantly more agitated after the experience.
Quotes from participants help understand these results. As A2
explained, “I was concerned about being solicited for information.
The location request pop-up worried me, now that I think aboutit, because I pressed it without paying attention to anything.” Theparticularly problematic Untranslation DP was summed up by A11:
“I couldn’t delete my account. The page was written in English.”A20 provided further context: “I feel that the content of the English
text may be important, or that I am being deceived.” B6 helped
make sense of why Social Proofs—fake reviews and testimonials
by fake people—were generally noticeable and also not considered
deceptive: “The reviews were mostly good ... it’s word-of-mouth.”
Sneaking in an Item was among the most noticeable and least
deceptive according to the deceptibility matrix, but among those
who were not tricked, most (58%) considered it deceptive. A27
explained: “That cloth comes with the cleaner. They force you to
buy it. They don’t mention it in the description.” We will discuss
this variability and the nuanced relationship between noticeability
and trickery as represented in the deceptibility metric next.

4

DISCUSSION

Dark patterns and deceptive designs are multifarious UI phenomena.
In the case of our online shopping simulation, nearly two-thirds
went unnoticed (RQ1). For the rest, noticeability and deceptibility
varied (RQ1), as did people’s reactions and feelings towards eachpattern (RQ2). We now attempt to bring a depth of understanding
to these findings by considering particular examples.
Almost no one was able to delete their account. This shows
the power of the newly found Untranslation DP as a mode of Obstruction. Untranslation is tied to knowledge of the local linguistic
context and relies on assumptions of language ability. We used
English in our simulation, given its global reach and power, as well
as statistics on Japan’s low English proficiency 5 . The statistics boreout in our case. Other nations with low English proficiency may
also be at risk of Untranslation, but this has yet to be explored.
A gestalt of DPs led half of participants to unknowingly sign up
for the premium member service. Yet, rather than deceptive, thosethat realized found it disruptive. This may point to attribution
biases [28] that may be self-serving [38]. Whether people blamed
themselves or attributed the problem to the system will need tobe researched. At least, our participants assumed good intentions,which raises the issue of user vulnerability to such DPs.
Reactions to the Social Proofs raise the issue of subversive influences. Repeated exposure to media stimuli can lead to unconscious
5 https://www.nippon.com/en/japan-data/h01843/

CHI EA ’24, May 11–16, 2024, Honolulu, HI, USA

Seaborn et al.

Figure 4: Deceptibility metric scores for classes (left) and cases (right).
shifts in preference by way of familiarity [14]. Social Proofs, among
the most and least noticeable, may be implicated, given the social media age we live within. Most participants deemed theseDPs acceptable. This may be explained by social mobilization effects, where even small profile pictures can have massive effects
on behaviour, perhaps especially when representing “friends” andinfluencers [6]. The interviews also suggest a perceived benefit of“word-of-mouth” information. Japan has been characterized as acollectivist market culture, where social bonds are prioritized andgroup acceptance of commercial offerings dictate market value [46].
We must be wary of taking advantage of people’s implicit trustin social indicators. Similarly, Urgency DPs were also considered
noticeable and acceptable, perhaps because of the store context andrepeated exposure to these rather widespread DPs in the wild.
Perhaps the most subversive DP was Alphabet Soup, found in two
membership fee displays. This DP was noticed only by a handful ofparticipants and was a top-scorer on the deceptibility metric (0.96
and 0.92). This study is the first to empirically explore this novel DP
and its sibling Untranslation, so it is difficult to contextualize theseresults within previous research. However, the National Consumer
Affairs Center of Japan has recently drawn attention to this DP
in the form of the ¥ symbol, which is used for both Chinese andJapanese currencies6 . Consumers and companies must be cautiousof conflating these symbols, as their emergence represents a sublime
form of deception for the Asian market.
Misleading Reference Pricing, a form of Interface Interference,
was also highly unnoticeable and deceptive, according to our metric.
Indeed, virtually no one noticed it, even when shown the user flow
and told about DPs. This is a common DP in shopping websites
and may escape the notice of the average consumer, allowing it tolive on even when laws change in response to advocacy initiatives.
We will need to raise attention to this DP and critically engage
designers, purveyors, and stakeholders [23] in light of regulatory
shifts around less deceptive forms of DPs.
Despite its name, “Sneaking” DPs were the least deceptive andeasily noticed. Obstruction, too. These may be more “blunt” formsof DPs, irritants rather than deceptive. Controlled experimental
6 https://www.kokusen.go.jp/news/data/n-20230419_2.html

work with and without these DPs may confirm and provide further
context on these findings from a consumer perspective [50].

4.1

Limitations

We used a simulation that was developed to the level of commercial
viability, containing DPs based on real cases from live websites.
Nevertheless, we recognize that this limits the ecological validityof our results. Our simulation also contained significantly more
DPs than the average 3.9 previously found for apps in the Japanese Google Play Store [29]. This may have lessened the realism of
the experience. At the same time, our results confirm that mostDPs went unnoticed. Still, future work should study UIs with more
realistic levels of DPs. Importantly, other kinds of UI, such as smartphone apps and video games, as well as other kinds of online stores,
such as clothing stores and grocery markets, should be explored,as the level and range of DPs may vary by platform.
We acknowledge limitations related to recruitment. We relied
on a third-party recruiter publicly known to participants (NHK).
This allowed us to recruit a diverse and random selection of participants from the Japanese populace. However, participants may
have self-selected in or out of the study based on the recruiter. We
also recruited a relatively small sample (N=30) and did not have anexperimental design with a control setup, limiting the generalizability of our findings. Finally, we did not control or capture relevant
metrics, such as daily Internet usage, type of degree or career, or
knowledge of English (for the Untranslation DP).
Finally, this was a pilot study. Our protocol was not preregistered,and we adapted our procedure and methods based on the pilot
tests, notably switching from retrospective to concurrent protocol
due to timing issues and adding options to the observer checklist.
A preregistered study will be conducted next. We also created anovel metric of deceptibility that could be applied to individuals
as well as DPs. This weighted measure was useful for determining
relative deceptibility, especially by taking into account priming andexposure to DP stimuli. Still, it rests on our operationalizations of
performance and arbitrary weights. This metric should be vetted
and discussed with experts in design and DPs, modified, if needed,
and then tested empirically. We also did not measure affective
reactions to specific DPs; targeted measurement can do so in future.

Deceptive, Disruptive, No Big Deal

5

CONCLUSION

We have taken a first step towards better understanding the deceptibility and UX of DPs commonly deployed in online stores for theJapanese context. We have demonstrated variability in perceptions
and deceptions while highlighting an overall trend of deceptibility
for most Japanese participants and DPs. The next step is experimental work with larger sample sizes and real commercial UI.

ACKNOWLEDGMENTS
Our sincere gratitude to NHK, especially Tatsuro Imono and Yuya
Higashi, for funding part of this research (development and participant fees) and handling the recruitment of participants. We
sincerely thank Émilie Fabre for development assistance. We thankTokyo Tech Innovation and Dai Senoo for supporting this researchin various ways. We thank Peter Pennefather for pre-reviewing thismanuscript. Katie Seaborn and Takao Fujii conscientiously dissent
to in-person participation at CHI this year; read their positionality
statement here: https://bit.ly/chi24statement

REFERENCES[1] Thamer Alshammari, Obead Alhadreti, and Pamela Mayhew. 2015. When to ask
participants to think aloud: A comparative study of concurrent and retrospective
think-aloud methods. International Journal of Human Computer Interaction 6, 3(2015), 48–64.
[2] Aaron Bangor, Philip T. Kortum, and James T. Miller. 2008. An empirical evaluation of the System Usability Scale. International Journal of Human–Computer
Interaction 24, 6 (2008), 574–594. https://doi.org/10.1080/10447310802205776[3] Benjamin Maximilian Berens, Heike Dietmann, Chiara Krisam, Oksana Kulyk,
and Melanie Volkamer. 2022. Cookie Disclaimers: Impact of Design and Users’
Attitude. In Proceedings of the 17th International Conference on Availability, Reliability and Security (Vienna, Austria) (ARES ’22). ACM, New York, NY, USA,
Article 12, 20 pages. https://doi.org/10.1145/3538969.3539008[4] Carlos Bermejo Fernandez, Dimitris Chatzopoulos, Dimitrios Papadopoulos, andPan Hui. 2021. This Website Uses Nudging: MTurk Workers’ Behaviour on
Cookie Consent Notices. Proceedings of the ACM on Human-Computer Interaction
5, CSCW2, Article 346 (oct 2021), 22 pages. https://doi.org/10.1145/3476087[5] Ann Blandford, Dominic Furniss, and Stephann Makri. 2016. Qualitative HCI
Research: Going Behind the Scenes. Morgan & Claypool Publishers, Kentfield, CA,
USA.
[6] Robert M. Bond, Christopher J. Fariss, Jason J. Jones, Adam D.I. Kramer, Cameron
Marlow, Jaime E. Settle, and James H. Fowler. 2012. A 61-million-person experiment in social influence and political mobilization. Nature 489, 7415 (2012),
295–298. https://www.nature.com/articles/nature11421[7] Kerstin Bongard-Blanchy, Jean-Louis Sterckx, Arianna Rossi, Anastasia Sergeeva,
Vincent Koenig, Salvador Rivas, and Verena Distler. 2023. Analysing the Influenceof Loss-Gain Framing on Data Disclosure Behaviour: A Study on the Use Caseof App Permission Requests. In Proceedings of the 2023 European Symposium on
Usable Security (Copenhagen, Denmark) (EuroUSEC ’23). ACM, New York, NY,
USA, 112–125. https://doi.org/10.1145/3617072.3617108[8] Ida Borberg, Rene Hougaard, Willard Rafnsson, and Oksana Kulyk. 2022. “So I
sold my soul”: Effects of dark patterns in cookie notices on end-user behavior
and perceptions. In Workshop on Usable Security and Privacy (USEC) (San Diego,
CA, USA), Vol. 3. HATS, Bloomintgton, IN, USA, 11 pages. https://doi.org/10.
14722/usec.2022.23026[9] Christoph Bösch, Benjamin Erb, Frank Kargl, Henning Kopp, and Stefan Pfattheicher. 2016. Tales from the dark side: Privacy dark strategies and privacy dark
patterns. Proceedings on Privacy Enhancing Technologies 2016, 4 (2016), 237–254.
https://doi.org/10.1515/popets-2016-0038[10] Margaret M. Bradley and Peter J. Lang. 1994. Measuring emotion: The SelfAssessment Manikin and the semantic differential. Journal of Behavior Therapy
and Experimental Psychiatry 25, 1 (1994), 49–59. https://www.sciencedirect.com/
science/article/abs/pii/0005791694900639[11] Scott Carter, Jennifer Mankoff, Scott R. Klemmer, and Tara Matthews. 2008.
Exiting the cleanroom: On ecological validity and ubiquitous computing.
Human–Computer Interaction 23, 1 (2008), 47–99. https://doi.org/10.1080/
07370020701851086[12] Akash Chaudhary, Jaivrat Saroha, Kyzyl Monteiro, Angus G. Forbes, and Aman
Parnami. 2022. “Are You Still Watching?”: Exploring Unintended User Behaviors
and Dark Patterns on Video Streaming Platforms. In Proceedings of the 2022 ACM

CHI EA ’24, May 11–16, 2024, Honolulu, HI, USA

Designing Interactive Systems Conference (Virtual Event, Australia) (DIS ’22). ACM,
New York, NY, USA, 776–791. https://doi.org/10.1145/3532106.3533562[13] Gregory Conti and Edward Sobiesk. 2010. Malicious Interface Design: Exploiting
the User. In Proceedings of the 19th International Conference on World Wide Web(Raleigh, North Carolina, USA) (WWW ’10). ACM, New York, NY, USA, 271–280.
https://doi.org/10.1145/1772690.1772719[14] Dena Cox and Anthony D. Cox. 2002. Beyond first impressions: The effects ofrepeated exposure on consumer liking of visually complex and simple productdesigns. Journal of the Academy of Marketing Science 30, 2 (2002), 119–130.
https://journals.sagepub.com/doi/abs/10.1177/03079459994371[15] Lorrie Faith Cranor. 2022. Cookie Monster. Commun. ACM 65, 7 (June 2022),
30–32. https://doi.org/10.1145/3538639[16] Linda Di Geronimo, Larissa Braz, Enrico Fregnan, Fabio Palomba, and Alberto
Bacchelli. 2020. UI dark datterns and where to find them: A study on mobile
applications and user perception. In Proceedings of the 2020 CHI Conference on
Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). ACM, New
York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376600[17] Madison Fansher, Shruthi Sai Chivukula, and Colin M. Gray. 2018. #darkpatterns:
UX Practitioner Conversations About Ethical Design. In Extended Abstracts ofthe 2018 CHI Conference on Human Factors in Computing Systems (Montreal, QC,
Canada) (CHI EA ’18). ACM, New York, NY, USA, 1–6. https://doi.org/10.1145/
3170427.3188553[18] Jiaying Feng, Fan Mo, Yuki Yada, Tsuneo Matsumoto, Nao Fukushima, Fukuyo
Kido, and Hayato Yamana. 2023. Analysis of Dark Pattern-related Tweets from2010. In 2023 IEEE 8th International Conference on Big Data Analytics (ICBDA).
IEEE, Piscataway, NJ, USA, 100–106. https://doi.org/10.1109/ICBDA57405.2023.
10104855[19] P.A.J. Graßl, H.K. Schraffenberger, F.J. Zuiderveen Borgesius, and M.A. Buijzen.
2021. Dark and bright patterns in cookie consent requests. Journal of Digital
Social Research 3, 1 (2021), 1–38. https://doi.org/10.33621/jdsr.v3i1.54[20] Colin M. Gray, Shruthi Sai Chivukula, Kassandra Melkey, and Rhea Manocha.
2021. Understanding “Dark” Design Roles in Computing Education. In Proceedings of the 17th ACM Conference on International Computing Education Research (Virtual Event, USA) (ICER 2021). ACM, New York, NY, USA, 225–238.
https://doi.org/10.1145/3446871.3469754[21] Colin M. Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L. Toombs.
2018. The Dark (Patterns) Side of UX Design. In Proceedings of the 2018 CHI
Conference on Human Factors in Computing Systems (Montreal, QC, Canada) (CHI’18). ACM, New York, NY, USA, 1–14. https://doi.org/10.1145/3173574.3174108[22] Colin M. Gray, Lorena Sanchez Chamorro, Ike Obi, and Ja-Nae Duane. 2023.
Mapping the Landscape of Dark Patterns Scholarship: A Systematic Literature
Review. In Companion Publication of the 2023 ACM Designing Interactive Systems
Conference (Pittsburgh, PA, USA) (DIS ’23 Companion). ACM, New York, NY, USA,
188–193. https://doi.org/10.1145/3563703.3596635[23] Colin M. Gray, Cristiana Santos, Nataliia Bielova, Michael Toth, and Damian
Clifford. 2021. Dark Patterns and the Legal Requirements of Consent Banners:
An Interaction Criticism Perspective. In Proceedings of the 2021 CHI Conferenceon Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). ACM, New
York, NY, USA, Article 172, 18 pages. https://doi.org/10.1145/3411764.3445779[24] Johanna Gunawan, Amogh Pradeep, David Choffnes, Woodrow Hartzog, andChristo Wilson. 2021. A Comparative Study of Dark Patterns Across Web andMobile Modalities. Proceedings of the ACM on Human-Computer Interaction 5,
CSCW2, Article 377 (oct 2021), 29 pages. https://doi.org/10.1145/3479521[25] Hana Habib, Megan Li, Ellie Young, and Lorrie Cranor. 2022. “Okay, whatever”’: An evaluation of cookie consent interfaces. In Proceedings of the 2022
CHI Conference on Human Factors in Computing Systems (New Orleans, LA,
USA) (CHI ’22). ACM, New York, NY, USA, Article 621, 27 pages. https://doi.org/10.1145/3491102.3501985[26] Brignull Harry. 2023. Deceptive Patterns: Exposing the Tricks Tech Companies Use
to Control You. Testimonium Ltd, London, UK.
[27] Thomas C. Head, Ricky W. Griffin, Thomas S. Bateman, Lynn Lohman, andValerie L Yates. 1988. The priming effect in task design research. Journal ofManagement 14, 1 (1988), 33–39. https://doi.org/10.1177/014920638801400104[28] Fritz Heider. 2013. The Psychology of Interpersonal Relations. Psychology Press,
London, UK.
[29] Shun Hidaka, Sota Kobuki, Mizuki Watanabe, and Katie Seaborn. 2023. Linguistic
Dead-Ends and Alphabet Soup: Finding Dark Patterns in Japanese Apps. In
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems(Hamburg, Germany) (CHI ’23). ACM, New York, NY, USA, Article 3, 13 pages.
https://doi.org/10.1145/3544548.3580942[30] Gonzalo Iturregui-Gallardo and Jorge Luis Méndez-Ulrich. 2020. Towards thecreation of a tactile version of the Self-Assessment Manikin (T-SAM) for theemotional assessment of visually impaired people. International Journal of Disability, Development and Education 67, 6 (2020), 657–674. https://doi.org/10.1080/
1034912X.2019.1626007[31] Lin Kyi, Sushil Ammanaghatta Shivakumar, Cristiana Teixeira Santos, Franziska
Roesner, Frederike Zufall, and Asia J. Biega. 2023. Investigating Deceptive Design
in GDPR’s Legitimate Interest. In Proceedings of the 2023 CHI Conference on

CHI EA ’24, May 11–16, 2024, Honolulu, HI, USA

Human Factors in Computing Systems (Hamburg, Germany) (CHI ’23). ACM, New
York, NY, USA, Article 583, 16 pages. https://doi.org/10.1145/3544548.3580637[32] James R Lewis. 2018. The System Usability Scale: Past, present, and future.
International Journal of Human–Computer Interaction 34, 7 (2018), 577–590. https://doi.org/10.1080/10447318.2018.1455307[33] Jamie Luguri and Lior Jacob Strahilevitz. 2021. Shining a light on dark patterns.
Journal of Legal Analysis 13, 1 (2021), 43–109. https://doi.org/10.1093/jla/laaa006[34] Aditi M. Bhoot, Mayuri A. Shinde, and Wricha P. Mishra. 2021. Towards the
Identification of Dark Patterns: An Analysis Based on End-User Reactions. In
Proceedings of the 11th Indian Conference on Human-Computer Interaction (Online,
India) (IndiaHCI ’20). ACM, New York, NY, USA, 24–33. https://doi.org/10.1145/
3429290.3429293[35] Arunesh Mathur, Gunes Acar, Michael J. Friedman, Eli Lucherini, Jonathan Mayer,
Marshini Chetty, and Arvind Narayanan. 2019. Dark Patterns at Scale: Findings
from a Crawl of 11K Shopping Websites. Proceedings of the ACM on Human–
Computer Interaction 3, CSCW, Article 81 (nov 2019), 32 pages. https://doi.org/
10.1145/3359183[36] Arunesh Mathur, Mihir Kshirsagar, and Jonathan Mayer. 2021. What Makes aDark Pattern... Dark? Design Attributes, Normative Considerations, and Measurement Methods. In Proceedings of the 2021 CHI Conference on Human Factors
in Computing Systems (Yokohama, Japan) (CHI ’21). ACM, New York, NY, USA,
Article 360, 18 pages. https://doi.org/10.1145/3411764.3445610[37] Iris B. Mauss and Michael D. Robinson. 2009. Measures of emotion: A review. Cognition & Emotion 23, 2 (Feb. 2009), 209–237. https://doi.org/10.1080/
02699930802204677[38] Dale T. Miller and Michael Ross. 1975. Self-serving biases in the attribution of
causality: Fact or fiction? Psychological Bulletin 82, 2 (1975), 213.
[39] Alberto Monge Roffarello and Luigi De Russis. 2022. Towards Understanding
the Dark Patterns That Steal Our Attention. In Extended Abstracts of the 2022
CHI Conference on Human Factors in Computing Systems (New Orleans, LA,
USA) (CHI EA ’22). ACM, New York, NY, USA, Article 274, 7 pages. https://doi.org/10.1145/3491101.3519829[40] Dmitry Nazarov and Yerkebulan Baimukhambetov. 2022. Clustering of Dark Patterns in the User Interfaces of Websites and Online Trading Portals (E-Commerce).
Mathematics 10, 18 (2022), 3219. https://doi.org/10.3390/math10183219[41] OECD. 2022. Dark commercial patterns. Number 336 in OECD Digital Economy Papers. OECD Publishing, Paris, France. 96 pages. https://doi.org/10.1787/44f5e846en[42] OECD. 2023. Consumer Vulnerability in the Digital Age. Number 355 in OECD
Digital Economy Papers. OECD Publishing, Paris, France. 80 pages. https://doi.org/10.1787/4d013cc5-en

Seaborn et al.

[43] René Schäfer, Paul Miles Preuschoff, and Jan Borchers. 2023. Investigating Visual
Countermeasures Against Dark Patterns in User Interfaces. In Proceedings ofMensch Und Computer 2023 (Rapperswil, Switzerland) (MuC ’23). ACM, New York,NY, USA, 161–172. https://doi.org/10.1145/3603555.3603563[44] Than Htut Soe, Oda Elise Nordberg, Frode Guribye, and Marija Slavkovik. 2020.
Circumvention by design - Dark patterns in cookie consent for online news outlets. In Proceedings of the 11th Nordic Conference on Human-Computer Interaction:
Shaping Experiences, Shaping Society (Tallinn, Estonia) (NordiCHI ‘20). ACM, New
York, NY, USA, Article 19, 12 pages. https://doi.org/10.1145/3419249.3420132[45] Neville A. Stanton, Paul M. Salmon, Laura A. Rafferty, Guy H. Walker, Chris
Baber, and Daniel P. Jenkins. 2017. Human Factors Methods: A Practical Guide forEngineering and Design. CRC Press, Boca Raton, Florida, USA.
[46] Nicolaos E. Synodinos. 2001. Understanding Japanese consumers: Some importantunderlying factors. Japanese Psychological Research 43, 4 (Nov. 2001), 235–248.
https://doi.org/10.1111/1468-5884.00181[47] Christine Utz, Martin Degeling, Sascha Fahl, Florian Schaub, and Thorsten Holz.
2019. (Un)Informed Consent: Studying GDPR Consent Notices in the Field. In
Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications
Security (London, United Kingdom) (CCS ’19). ACM, New York, NY, USA, 973–990.
https://doi.org/10.1145/3319535.3354212[48] Maaike van den Haak, Menno de Jong, and Peter Jan Schellens. 2003. Retrospective vs. concurrent think-aloud protocols: Testing the usability of an online
library catalogue. Behaviour & Information Technology 22, 5 (2003), 339–351.
https://doi.org/10.1080/0044929031000[49] Christof van Nimwegen and Jesse de Wit. 2022. Shopping in the dark: Effectsof platform choice on dark pattern recognition. In International Conference on
Human-Computer Interaction. Springer, London, UK, 462–475. https://doi.org/10.
1007/978-3-031-05412-9_32[50] Christian Voigt, Stephan Schlögl, and Aleksander Groth. 2021. Dark patterns in online shopping: Of sneaky tricks, perceived annoyance and respective brand trust.
In International Conference on Human-Computer Interaction. Springer, London,
UK, 143–155. https://link.springer.com/chapter/10.1007/978-3-030-77750-0_10[51] Chauncey Wilson. 2014. Interview Techniques for UX Practitioners: A User-Centered
Design Method. Morgan Kaufmann, Waltham, MA, USA.
[52] Yuki Yada, Jiaying Feng, Tsuneo Matsumoto, Nao Fukushima, Fuyuko Kido, andHayato Yamana. 2022. Dark patterns in e-commerce: A dataset and its baseline
evaluations. In 2022 IEEE International Conference on Big Data (Big Data). IEEE,
Piscataway, NJ, USA, 3015–3022. https://ieeexplore.ieee.org/document/10020800[53] Shigeru Yamanouchi. 2015. エンジニアのための人を対象とする研究計画入門 (Introduction to Human Subjects Research Planning for Engineers). Maruzen,
Tokyo, Japan, Chapter 科学的合理性と倫理的妥当性 (Scientific rationality
and Ethical Relevance), 74–118.