“We’ve Disabled MFA for You”: An Evaluation of the Security andUsability of Multi-Factor Authentication Recovery Deployments
Sabrina Amft

Sandra Höltervennhoff

Nicolas Huaman

CISPA Helmholtz Center for
Information Security
Hanover, Germany

Leibniz University Hannover
Hanover, Germany

Leibniz University Hannover
Hanover, Germany

Alexander Krause

Lucy Simko

Yasemin Acar

CISPA Helmholtz Center for
Information Security
Hanover, Germany

The George Washington University
Washington, DC, USA

Paderborn University &The George Washington University
Paderborn, Germany

Sascha Fahl
CISPA Helmholtz Center for
Information Security
Hanover, Germany

ABSTRACTCCS CONCEPTS

Multi-Factor Authentication is intended to strengthen the securityof password-based authentication by adding another factor, suchas hardware tokens or one-time passwords using mobile apps.
However, this increased authentication security comes with potential drawbacks that can lead to account and asset loss. If users
lose access to their additional authentication factors for any reason,
they will be locked out of their accounts. Consequently, services
that provide Multi-Factor Authentication should deploy procedures
to allow their users to recover from losing access to their additional
factor that are both secure and easy-to-use.
In this work, we investigate the security and user experienceof Multi-Factor Authentication recovery procedures, and compare
their deployment to descriptions on help and support pages.
We first evaluate the official help and support pages of 1,303
websites that provide Multi-Factor Authentication and collect documented information about their recovery procedures. Second, weselect a subset of 71 websites, create accounts, set up Multi-Factor
Authentication, and perform an in-depth investigation of their recovery procedure security and user experience.
We find that many websites deploy insecure Multi-Factor Authentication recovery procedures and allowed us to circumventand disable Multi-Factor Authentication when having access to theaccounts’ associated email addresses. Furthermore, we commonly
observed discrepancies between our in-depth analysis and the official help and support pages, implying that information meant toaid users is often either incorrect or outdated.
Based on our findings, we provide recommendations for bestpractices regarding Multi-Factor Authentication recovery.

• Security and privacy → Usability in security and privacy;
Multi-factor authentication.

CCS ’23, November 26–30, 2023, Copenhagen, Denmark© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
This is the author’s version of the work. It is posted here for your personal use. Not
for redistribution. The definitive Version of Record was published in Proceedings of the2023 ACM SIGSAC Conference on Computer and Communications Security (CCS ’23),
November 26–30, 2023, Copenhagen, Denmark, https://doi.org/10.1145/3576915.3623180.

KEYWORDS
authentication, multi-factor authentication, usable security
ACM Reference Format:
Sabrina Amft, Sandra Höltervennhoff, Nicolas Huaman, Alexander Krause,
Lucy Simko, Yasemin Acar, and Sascha Fahl. 2023. “We’ve Disabled MFA forYou”: An Evaluation of the Security and Usability of Multi-Factor Authentication Recovery Deployments. In Proceedings of the 2023 ACM SIGSAC
Conference on Computer and Communications Security (CCS ’23), November
26–30, 2023, Copenhagen, Denmark. ACM, New York, NY, USA, 24 pages.
https://doi.org/10.1145/3576915.3623180

1

INTRODUCTION

Username and password are the de facto standard for user authentication on the web and beyond. However, they come with many
security problems: Users tend to choose easily guessable passwords
and re-use them for multiple accounts [18, 69, 87, 89, 90], or sufferfrom insecure or hard-to-use password policies [35, 52, 84]. Additionally, service providers may implement insecure and inadequate
password storage, leaving millions of passwords unprotected due
to data breaches [17, 29, 32, 70, 72, 95]. Multi-Factor Authentication(MFA) adds an extra factor and additional security to passwordbased authentication schemes, and has become important in manyauthentication deployments on the web. With MFA, users authenticate themselves using additional factors, e. g., biometric features,
hardware tokens, smartphone applications, or secret information
only they know [67].
Prior work has found tensions between security and usability inmany MFA implementations, revealing that the complexity of MFAsetup is a barrier to widespread use [74, 75, 76], and user acceptance of long-term MFA use can suffer due to the increased log-in
and setup time [6, 31]. While most prior work focused on MFA
methods and deployment, recovery procedures for loss of MFA are

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

less well understood. Hence, in this paper, we focus on this criticalaspect of MFA security and usability: We investigate the security
and usability of MFA recovery procedure deployments on the web.
Recovery procedures are crucial to regaining access to accounts incase MFA factors are lost, stolen, or broken. The deployment of arecovery procedure by service providers has a significant impact
on authentication security and usability. While locking users outof an account after losing access to MFA factors keeps up security
assumptions of MFA, it might contribute to users’ frustration [61],users leaving a service, or even avoiding MFA in the future. However, allowing users to access their accounts after losing their MFA
factors by returning to password-based authentication significantly
decreases security, allowing attackers to more easily gain access touser accounts than with MFA. While security and usability seemto be directly at odds in many current implementations of MFA
recovery, it is essential for service providers to relieve this tension
through better communication, and policy and technical changes
in order to provide both security and usability to support users.
We aim to better understand the experience of users who havelost access to their MFA, and systematically evaluate MFA deployment and recovery, asking the following research questions:
RQ1: “What MFA recovery procedures are commonly deployed on
the web?” Due to their crucial impact on authentication security and usability, the variety of deployed MFA recovery
procedures need to be better understood in the research community. We investigate the deployment and documentationof common MFA recovery procedures on the web.
RQ2: “How are MFA recovery procedures on the web implemented
and what is their impact on authentication security?” MFA
recovery procedures aim to support users getting back into
their accounts in case of losing access to MFA factors. We
investigate how service providers balance the usability andsecurity of MFA recovery procedures.
RQ3: “How can the security and usability of MFA recovery procedures be improved?” Standards or established best practices
for MFA recovery procedures are missing. Based on our results, we make recommendations to balance the security
and usability of MFA recovery procedures for future deployments.
First, we created a list of 1,303 websites offering MFA. We systematically evaluated MFA recovery procedures deployed on thesewebsites using their official help and support documentation. In afollow-up ERB-approved study, we performed an in-depth analysisof the security and usability of deployed MFA recovery procedures
for a subset of 71 websites. We created accounts, enabled and configured MFA, and went through the entire recovery process.
In this paper, we make the following contributions:• MFA Recovery Procedure Deployment: By analyzing
official help and support pages, and FAQs, we find that mostwebsites only offer one recovery procedure, which is mostcommonly a backup code or direct support contact. A quarterof pages do not mention recovery in their help.
• User Experience of MFA Recovery Procedures: We perform an in-depth investigation of the user experience of MFA
recovery procedures on 71 websites. We could recover 52.11%

Amft et al.

of the accounts. In most cases, access to the associated email
inbox was sufficient.
• Recommendations: Based on our results, we provide recommendations for future research on secure and usable MFA
recovery procedures. We also provide guidance for web developers to help deploy easier-to-use and secure MFA recovery procedures, spanning all areas of recovery such as setup,
communication, and recovery itself.
• Replication: We provide all material including communication templates, the study protocols and our codebooks
to support methodological transparency, replicability, andvalidity, and to help future research on this topic as part of a
replication package1 .
This work is structured as follows. In Section 2, we describe
previous work on the subject. We present the methodology andresults of our deployment evaluation in Section 3.1, followed byour practical recovery test in Section 3.2. In Section 4, we elaborateon the ethics and limitations of our work. We discuss our findings
in Section 5, and conclude in Section 6.

2

RELATED WORKIn this section, we present previous work on related areas anddiscuss our novelty and contributions compared to them. We first
showcase related papers regarding MFA security and usability, thendiscuss research on the recovery of account access regarding both
single-factor authentication and recovery of additional factors.

2.1

Multi-Factor Authentication Usability

Several previous works have investigated the usability of different
popular MFA methods.
In 2018, Colnago et al. observed the enrollment of Duo twofactor authentication at their university. While adopters found it
annoying, they also perceived it as easy to use and secure [25].
In the same year, Reynolds et al. conducted two studies with Yubikeys, asking participants to set them up or to include them into
their daily lives. Despite participants embracing hardware keys,
the study uncovered severe usability problems, especially during
setup [76]. In 2019, Reese et al. conducted a usability study with
72 participants comparing five different 2FA methods, confirming
findings by Reynolds et al. [74].
Ciolino et al. also conducted both a lab and longitudinal study
comparing SMS OTP and hardware keys, and similar to previous
works reported that SMS is both faster to use and set up, and that
hardware keys are perceived as less usable [24]. In 2020, Reynolds
et al. evaluated authentication data sets from two universities that
introduced 2FA, finding that more than five percent of authentication attempts failed, mostly due to technical difficulties such as
timeouts, or users canceling or mistyping the codes [75]. Similarly,
Abbott and Patil examined authentication logs and surveyed university users after 2FA became mandatory for them. They foundthat user acceptance was not influenced when 2FA was required
for some sensitive authentication purposes, but decreased when
it was enforced for every login [6]. Also in 2020, Farke et al. accompanied the introduction of FIDO2 keys as a single-factor (i. e., a
1 We provide this material on our accompanying website [13] and in our replication

package: https://doi.org/10.25835/9v3k2sx0.

An Evaluation of the Security and Usability of Multi-Factor Authentication Recovery Deployments

password-less login, not a MFA method) in a small company, findingthat while participants considered them usable, they stopped using
them due to concerns such as login efficiency or missing browser
support [31].
A more recent work by Lyastani et al. evaluated 85 websites
regarding their MFA usability by creating accounts and evaluating their MFA communication and settings. They found that MFA
implementations are largely inconsistent between different websites, and that many designs have previously been identified as
obstructive or problematic [36].
Overall, previous work shows that the usability depends on the
MFA method used, e. g., SMS OTPs are often faster and more usable
than hardware keys, although they are known to be less secure.
Additionally, users commonly struggle with setup and continued
usage, obstructing widespread voluntary adoption of MFA. While
our work does not investigate user sentiments, we discuss the user
experience of MFA methods and recovery procedures, and how
website and service providers communicate MFA and its security
benefits for online security.

2.2

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

1. MFA and Recovery Procedure Categorization(1) Compile initial list of 1,303 websites with MFA support.
(2) Identify MFA and recovery procedures based on support and help
pages.
(3) Select a sample of 71 websites for in-depth analysis.

2. In-Depth MFA Recovery
User Experience(1) Account creation and MFA
configuration on selected
sample of 71 websites.
(2) Wait for one week.
(3) Go through MFA recovery
procedure.
(4) Debrief websites.

3. Detailed Analysis ofDocumented MFA
Recovery Procedures(1) Collect MFA documentation
for selected sample of 71
websites.
(2) Perform in-depth analysis of
documented MFA recovery
procedures and extend
evaluation in step 1.

Compare documented MFA recovery procedures with our experiences in
the in-depth recovery procedure evaluation in step 2.

Figure 1: Summary of our methodology, consisting of the
MFA and recovery procedure categorization (cf. Section 3.1)
and a follow-up in-depth evaluation of deployed MFA recovery procedures for 71 websites (cf. Section 3.2)

Account Recovery

While not many previous works have discussed the recovery of
inaccessible MFAs, several have worked on account recovery ingeneral, and its potential obstacles.
One of the currently still popular forms of account recovery,
security questions, was criticized as early as 2008 when Rabkinaddressed the issue of using security questions to regain accountaccess. The work highlighted several issues, including how questions ask for easily obtainable knowledge, especially with the riseof social media [71]. A similar work by Schechter et al. in 2009
confirmed this, finding that in 17% of attempts, user acquaintanceswere able to guess the answer to security questions, 13% were easily
guessable within the first five attempts in general. Further issues
included usability problems, as 20% of users forgot the answers
themselves six months after setting them up [77]. The usability of
security questions was further researched by Bonneau et al. in 2015
in a survey. They found that 37% of participants lied when answering security questions, and 40% did not remember their answers
when requiring them to recover their account [19].
In 2006, Brainard et al. proposed a novel approach for accountand MFA recovery, in which a trusted person can vouch for account owners and help them regain access. They further discussed
limitations and variants, like additional measures to prevent users
from abusing this recovery method as its own MFA [21]. A similar
approach was investigated in 2009 through a lab study by Schechteret al. [78]. In their study, 17 of 19 participants successfully usedtrusted individuals, however, similar to the answers to security
questions, some users forgot who their trusted individual was.
In 2015, Hang et al. researched fallback authentication methods
for smartphones, finding that current methods were largely sufficient, but that a minority of users struggled with it due to a lack ofknowledge or alternative authentication options [42]. While mostof these studies examined specific account recovery procedures,
Neil et al. evaluated real-world account recovery advice on 57 popular websites. Their results indicated that help sections were often

incomplete and that 39% of websites were failing to address the
topic at all [65].
In 2021, Kunke et al. evaluated twelve account recovery procedures for passwordless authentication with FIDO2 regarding
different criteria within the areas of usability, deployability, andsecurity [48]. They found the recoveries to be lacking and, in somecases, such as security questions and backup passwords, jeopardize
the idea of passwordless authentication. In 2023, Gilsenan et al.
performed a usability analysis of 22 popular TOTP 2FA mobile apps
regarding their backup implementations, finding that most rely on
less secure methods such as SMS or email for backups [37].
Furthermore, a recent study by Gerlitz et al. has also investigated
how websites act when additional factors are lost. They were able
to recover about half of their created accounts, and find that there
are no best practices for recovery behavior or communication [34].
To summarize, previous work has evaluated the security, usability, and communication of fallback authentication methods. However, this typically happened in the context of account recovery,
and with no regard for MFA loss. In contrast, we evaluate existing
MFA recovery procedures by experiencing them first-hand, withspecial regard to its usability and (loss of) security benefits.

3

MFA RECOVERY PROCEDURE ANALYSISThis section presents the method and findings of our analysis of deployed MFA recovery procedures. Our analysis spans two parts; wefirst investigated deployed MFA recovery procedures for 1,303 websites using their publicly available help and support pages. Second,
we performed an in-depth analysis of MFA recovery procedures
for a subset of 71 websites. We simulated a user’s experience afterlosing access to their MFA factor. See Figure 1 for a summary of all
steps in our methodology.

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

3.1

Deployment of MFA Recovery ProceduresFirst, we conducted a systematic evaluation of MFA recovery procedures deployed on 1,303 websites that use MFA, based on theinformation provided on help and support pages. Overall, we investigate the impact of MFA recovery procedures on authentication
security and usability.
3.1.1 Methodology. We aimed to simulate users who lost access totheir MFA factor and tried to find information on how to recoveraccess to their accounts. While our overall focus was on the users’
perspective on MFA recovery procedures instead of a measurementof deployed MFA methods on the web, we provide some context
on the spread of MFA methods and recovery procedures in thefollowing.
Our dataset included all 1,303 websites with MFA deploymentsfrom 2f a.directory [2], a high-quality crowdsourced collectionof websites with information about MFA deployment on the web.
Most websites included on 2f a.directory offer services to users,
and typically collect user-connected information, e. g., addresses
for online shopping, or financial information for online banking.
The 2fa.directory excludes self-hosted websites and websites thatare not within the top 200,000 according to SimilarWeb [5]. We
verified the rank requirements by cross-checking our sample with
the Tranco list [50]2 , and found 1,173 (90.02%) of the websites inour dataset within the top 200,000 websites, of which 692 (53.11%)
were even within the top 20,000 (cf. Table 3in Appendix B.2)
Overall, we found 2,021 websites on 2fa.directory. From these,
we excluded 44 duplicates that were included in multiple categories,
7 unreachable websites. We further only examined websites thatwere either in English or German and excluded 155 websites inother languages. Finally, 512 pages did not offer MFA and were
therefore not relevant for our study. This gave us a total of 1,303
websites. We focused on German and English websites, since all
authors were either proficient in English or German. We excluded
other languages since in some cases automated translations of help
and support pages in other languages were ambiguous in the MFA
methods or recovery used, too vague, or inaccurate. While somewebsites provided official translations, they were not always wellmaintained or even outdated. We further removed 15 websites we
could not reliably reach due to, e. g., geo-restrictions, or unavailable
services.
To evaluate help and support pages, we used the support links
provided in the 2f a.directory database, and the websites’ help
and support sections that we could either find by manually going
through the websites, or using Google with the following searchterm "<website> two-factor authentication"3 . Within our
Google search, we followed all links that were officially affiliatedwith the website in question (i. e., no third-party news websites)
within the first page of Google results [22, 43, 44]. We visited all ofthem and examined them for input regarding MFA methods andrecovery procedures. We followed all links and relevant prompts,
including "Trouble logging in?" or "Forgot your MFA?" prompts.

2 Based on https://tranco-list.eu/list/82Q5V.
3 We found this search term to work well, as Google also detected various synonyms(such as multi-step verification) or alternative spellings (2-factor instead of two-factor).
We complemented the Google search with our manual search.

Amft et al.

This did not include interactive processes to reset passwords, as we
focused on lost MFA.
Two researchers analyzed the help and support pages for documented MFA recovery procedures using an inductive categorization
approach [59]. We used a semi-open coding approach and an initialcodebook including the five MFA methods given on 2fa.directoryand several initial recovery procedures based on personal knowledge and previous work [48, 71]. We extended or updated the codebook accordingly whenever we identified a novel MFA recovery
procedure, and revisited previous codings. The full codebook and amore detailed description of MFA methods and recovery procedures
can be found in the replication package in Table 4, and Section A
in the Appendix respectively
We resolved all coding conflicts as they came up, leading to ahypothetical agreement of 100%. In line with common practice andprevious high-quality qualitative research, we, therefore, chose toomit the calculation of an inter-rater reliability score [20, 60, 62,92].
For the initial categorization, we coded for documented MFA
and recovery procedures. We used counts, means, and other aggregation measures to report our findings and generate descriptive
statistics. During our evaluation, we made notes of every recovery
procedure that stood out, either positively or negatively. This includes constraints on MFA, e. g., if it was only provided for certain
user groups or customers, interesting security-related statements,
such as stressing that lost MFA could not be recovered in any way,or interesting methods, such as receiving recovery codes via mail.
We used these notes as a basis for the detailed coding described in
Section 3.2 and our best practice recommendations in Section 5.

3.1.2 Results. Overall, mobile applications were the most popular
MFA method, deployed on 1,036 pages (79.51%), mostly as TOTP
generator apps. Second most popular were SMS (559, 42.90%). Allother methods, i. e., phone calls, email, hardware tokens, and various other methods were noticeably less widespread and present in
between 10.67%–15.73% (139–205) of our sample. On average, websites offered 1.75 (median: 1.0) MFA methods. We provide details
for MFA methods and their recovery procedures in Table 1.
Websites offered an average of 1.28 (median: 1.0) different recovery procedures. Most websites used direct support contacts (R01)(491; 37.68%) and backup codes (R02) (451; 34.61%). However, 321(24.64%) websites that deployed MFA did not provide publicly accessible information about MFA recovery procedures at all (R17).
While some websites might provide MFA recovery information
behind a login, we argue that users locked out due to MFA loss
would benefit greatly from public recovery information.
While 179 (13.74%) websites offered MFA using hardware tokens,no website in our sample offered them as a recovery procedure.
In most cases, users could only use hardware tokens with another
MFA method – a requirement we did not find for other MFA options.
While websites did not justify this decision, we suspect known usability issues of hardware tokens and a perceived higher likelihoodof becoming inaccessible [24, 31, 57, 76].
We compared MFA recovery procedures between website categories provided by the 2f a.directory categories, e. g., Banking,An Evaluation of the Security and Usability of Multi-Factor Authentication Recovery Deployments

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

Deployed Recovery Procedures

Table 1: Summary of deployed MFA methods and recovery procedures in our sample of 1,303 websites. For example, we findthat of 559 websites using SMS (42.90% of all websites), 199 (35.6% of websites that offer SMS as main MFA) offer support contact
as a recovery procedure. Because some websites offered multiple MFA and recovery procedures, not all percentages add to 100.
We provide a more detailed description of these methods in our replication packageand in Section A in the Appendix

R01
R02
R03
R04
R05
R06
R07
R08
R09
R10
R11
R12
R13
R14
R15
R16
R17

All Recovery
Contact Website Support
Backup Codes
Contact Local Admin
Additional MFA Method
Backup SMS/Phone Call
Backup Email
Dedicated Account Recovery System
TOTP Seed
Trusted Device
Photo/Official ID Proof
MFA Not Needed for Login
Security Questions
Password Reset
Other Recovery
Help Page Not Accessible
No MFA Recovery Available
No MFA Recovery Documented

All MFA

Deployed MFA Methods
SMS
Phone Call Email

HardwareMobile App

Other MFA

1,303 (100%)
491 (37.68%)
451 (34.61%)
159 (12.20%)
150 (11.51%)
78 (5.99%)
49 (3.76%)
45 (3.45%)
44 (3.38%)
37 (2.84%)
30 (2.30%)
25 (1.92%)
16 (1.23%)
13 (1.00%)
77 (5.91%)
9 (0.69%)
1 (0.08%)
321 (24.64%)

559 (42.90%)
199 (35.6%)
155 (27.73%)
79 (14.13%)
119 (21.29%)
35 (6.26%)
23 (4.11%)
34 (6.08%)
6 (1.07%)26 (4.65%)
10 (1.79%)
14 (2.5%)
9 (1.61%)
2 (0.36%)38 (6.8%)
2 (0.36%)0 (0.0%)153 (27.37%)

179 (13.74%)
57 (31.84%)
71 (39.66%)
60 (33.52%)
70 (39.11%)
19 (10.61%)
5 (2.79%)
19 (10.61%)
6 (3.35%)
18 (10.06%)
3 (1.68%)
9 (5.03%)0 (0.0%)2 (1.12%)27 (15.08%)
0 (0.0%)0 (0.0%)24 (13.41%)

1,036 (79.51%)
413 (39.86%)
435 (41.99%)
152 (14.67%)
134 (12.93%)
74 (7.14%)
41 (3.96%)
39 (3.76%)
44 (4.25%)
33 (3.19%)
30 (2.9%)23 (2.22%)
10 (0.97%)
11 (1.06%)
62 (5.98%)
6 (0.58%)
1 (0.1%)180 (17.37%)

139 (10.67%)
35 (25.18%)
17 (12.23%)
12 (8.63%)
15 (10.79%)
3 (2.16%)4 (2.88%)
3 (2.16%)1 (0.72%)
4 (2.88%)
1 (0.72%)
3 (2.16%)1 (0.72%)
1 (0.72%)
16 (11.51%)
3 (2.16%)0 (0.0%)65 (46.76%)

or University websites. While most recovery procedures were uniformly distributed across website categories, some recovery procedures were more common in certain categories. Providing selfies
with handwritten notes and governmental IDs (R10) almost exclusively appeared on Cryptocurrency websites. Universities were more
likely to rely on local administrators (R03) for MFA recovery. Figure 6 in the Appendix illustrates the distribution of MFA recovery
procedures across website categories in our dataset.
Key Points | MFA Recovery Procedure Analysis More than 80%of the websites we analyzed deployed mobile apps and SMS for
MFA. Most offered one MFA option, and one recovery procedure;
Most websites used backup codes or direct support contact astheir recovery procedure. A quarter of websites did not provide
public MFA recovery information.

3.2

Recovery User Experience

The previous experiment sheds light on the deployment frequencies
of different MFA recovery procedures, including recovery instructions for 1,303 websites supporting MFA.
In this section, we investigate the user experience of deployed
MFA recovery procedures.
Therefore, we selected a subset of 71 websites, created accounts,
and performed the entire MFA recovery procedure process. We
focused on authentication security and usability and how well theimplemented MFA recovery procedures were documented.
3.2.1 Methodology. Since we were further interested in detailed
but also diverse insights into MFA recovery procedures that cannot be achieved from collecting help and support pages alone, we
selected a subset of 71 websites for an in-depth analysis of theuser experience. Similar to previous work [37, 52, 65, 68, 81], wechose to comprise our sample from different sources, including 28of the highest ranking, 27 random, and 16 hand-picked websites.
We chose the 16 handpicked websites based on interesting edge

156 (11.97%)
32 (20.51%)
32 (20.51%)
41 (26.28%)
59 (37.82%)
14 (8.97%)
4 (2.56%)
20 (12.82%)
0 (0.0%)16 (10.26%)
3 (1.92%)
1 (0.64%)
3 (1.92%)
0 (0.0%)15 (9.62%)
1 (0.64%)0 (0.0%)53 (33.97%)

205 (15.73%)
72 (35.12%)
34 (16.59%)
13 (6.34%)
31 (15.12%)
7 (3.41%)
7 (3.41%)
3 (1.46%)
2 (0.98%)
2 (0.98%)
2 (0.98%)
3 (1.46%)
3 (1.46%)
3 (1.46%)
7 (3.41%)1 (0.49%)
0 (0.0%)79 (38.54%)

cases in our larger dataset. This included 6 websites that offered security products such as SSL certificates or antivirus programs, and6 websites that mentioned unusual recovery procedures. Examples
of unusual recovery procedures were preventive measures suchas enforcing downloads of recovery codes, offering users to store
various types of personal data which would be later used to recover
the account, or multiple tests of the availability of their chosen recovery. Similarly, we chose some websites based on their recovery
procedures, including a website that involved team members as
witnesses to vouch for or against a recovery decision or enforcing
fixed waiting times before MFA would be reset. Finally, we chose 2
websites that explicitly mentioned that the account would be lost if
users lose their MFA, including advice to buy multiple YubiKeys,
and 2 websites with unclear or no instructions to see what would
happen in these cases.
From an initial qualitative sample of several websites, sampled
after our three criteria above, we excluded websites that required
pre-existing contract or group memberships (25), or data we could
not or did not want to provide to protect the privacy of the researchers conducting this study, e. g., credit or ID card data (23). We
further excluded websites with terms of service that prohibited ouruse case (20). Other reasons included that the website shared its
account with another website already in our sample (12), that wewere blocked on due to unspecified reasons (11), or websites whose
free versions were only time-limited trials (17). Finally, 3 domains
forwarded us to other websites or were offline since the deployment evaluation. An anonymized overview of excluded websites isgiven in our replication package and in Table 7 in the Appendix B.2
Whenever we could not create accounts, we replaced a website
with the next best alternative where possible– e. g., if a top-ranked
website required us to have a contract with them to acquire an
account such as a bank, we replaced it with the next most popular
website not yet included. We ended up with a sample of 71 websites,on which we tested MFA recovery. Despite the exclusion of websites, we feel confident that our in-depth analysis of the actual MFA

recovery process on 71 websites provides deep and diverse insights
valuable to the research community. Additionally, the distributionof offered MFA methods in our final sample is comparable to theoverall distribution on all 1,303 websites.
We created accounts on all 71 websites using free account plans,
enabled MFA, and triggered the respective MFA recovery procedure. Using screencasts, we made video recordings of the entire
procedure to compare it with the official documentation duringlater evaluation. To receive email and text messages for accountcreation, MFA setup, or MFA recovery, we used a Google email
address created for the study and an eSIM with a US-based phone
number.
During the MFA setup, we simulated a user who follows instructions but only fulfilled the minimum requirements, following
findings from previous work [45, 90, 96]. During the process, wenoted all steps, whether we regained access, and how long recovery took. We also captured security and usability details. For eachwebsite, we followed the protocol below:(1) We enabled the first MFA method provided by a website. We
set specific or additional MFA and recovery procedures if thewebsite gave explicit instructions to do so. This decision was
further driven by the assumption that the first listed MFA
method and recovery procedure is preferred by a website
and likely best supported.
(2) After one week had passed, we revisited the websites. Thisdecreased suspicion because we did not immediately lose
access after creating the accounts.
(3) We tried to log in without our supposedly lost MFA and recovery procedures. If the respective website allowed accountrecovery for lost or inaccessible MFA devices, we followed
it without using our configured recovery procedure (i. e., we
did not use our backup codes to regain access), to simulate
the experience of a user who has lost MFA completely.
(4) If a website did not provide an automated emergency solution for login without MFA or the recovery procedure we
configured, we contacted the website’s support directly. In
our initial request(cf. Appendix B.1) we said that we recently
lost a backpack containing our phone and wallet, and foundourselves without access to the configured MFA factor.
(5) If there was no way to contact the website (e. g., because support channels were only available for paying account plans),
we stopped, marked the account as lost, and continued with
the next website.
For an in-depth analysis of MFA recovery procedure experienceand comparison with their documentation, we qualitatively assessed the help and support pages of the 71 websites in our data set.
We collected all relevant help and support pages for all websites:• We performed a Google search for "<website> two-factor
authentication"3 similar to the one for our deployment
evaluation.
• We collected the help and support sections linked in the
2fa.directory database.
• We manually navigated websites, starting with their main
pages and using their search functions if available.
Following the steps above, we saved screenshots of help pages
and expanded our codebook from the initial categorization (seeAmft et al.

Recovery Procedures

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

Available, but
Unrelated (n=5)
Backup Code (n=39)
No Recovery (n=21)
Other (n=5)
TOTP (n=5)
020Recovered

40

60
Lost

80

100
No Answer

Figure 2: Overview of the recovery success rates per recovery
procedure in percent per method. Other encompasses twophone-based recoveries and contact support. TOTP also includes the TOTP secrets. Numbers do not add up to 71 due to
websites offering multiple recovery procedures.

Table 4) We added details such as whether a website explicitly
warned their users of account loss in case of losing access to their
MFA, if MFA was required in general, and how often it needed to beentered, or whether users were required to set a recovery procedure(see Table 5 for the codebook expansion) We coded both the MFA
account setup processes and the information provided on the help
pages. We compared them to identify potential shortcomings orcontradictions in how providers communicate MFA to their users.
Our institution’s ethical review board (ERB) approved the study, andwe took measures to protect the websites’ identities, their resources,
and the time of their staff (cf. Section 4.1).
3.2.2 Results. Below, we provide details for our MFA recovery
procedures evaluation. Furthermore, we compare our user experience of MFA recovery procedures with their official MFA recovery
documentation.
Account Recovery Success Rates. Overall, we created 71 accounts in August and September 2022. We regained access to 37(52.11%), and lost access to 30 (42.25%). As of the time of submission,
we did not receive an answer from 4 (5.63%) service providers. Figure 5 in the Appendix connects our website sample to MFA methods
and recovery success. We configured mobile apps (49, 69.01%), SMS(15, 21.13%), email (6, 8.45%) and finally hardware token (1, 1.41%)
as primary MFA methods. We found that websites that deployed
SMS-based MFA had slightly worse recovery success rates than
TOTP or other app-based approaches. As we retained access to ouremail inbox during our study, i. e., we never lost access to the main
MFA method on the respective website, email had a perfect successrate. We decided not to further bias our results due to a change
in methodology by treating websites with email as the main MFA
differently from those that deployed other MFA methods.
Overall, we set up at least one MFA recovery procedure on 50(70.42%) websites. However, 21 (29.58%) either did not offer any
MFA recovery option or did not prompt us to configure a recovery
option during MFA setup. The recovery rates for all MFA recovery
procedures are illustrated in Figure 2.
We configured and stored backup codes (R02) for 39 websites.
Only 6 (15%) of those websites tried to verify backup code availability during setup, e. g., by asking for the code or confirming thatAn Evaluation of the Security and Usability of Multi-Factor Authentication Recovery Deployments

Figure 3: Summary of account recovery and loss split byroot cause. We distinguished between recovery due to email
as the main MFA method, and email as the main recovery
communication channel due to which access is regained.

we stored the code. We simulated having lost access to our backup
codes for the recovery requests. We could regain access to only
slightly more than half (22, 56.41%), and lost 16 accounts (41.03%).
The remaining 1 website did not reply to our recovery request.
We were unable to regain access to any account that used TOTP
apps or their seeds for MFA recovery (5, 100.0%). On 5 websites, we
tried to recover our account using available, but unrelated procedures, i. e., alternative contact methods such as backup email (R06)or phone numbers (R05) we set up during account creation thatwere never officially dedicated as recovery or mentioned during
MFA setup. Despite the vague communication, this led to indeed 4(80.0%) accounts being recovered from MFA loss. For other recovery
procedures, mostly consisting of phone-based (R05) recovery or
the direct mention of support channels (R01) as a backup solution,
we regained access to 2 (40.0%) accounts. Finally, we did not setup (R16) MFA recovery procedures on 21 websites, as this was notpart of the MFA setup process. Despite the lack of a recovery procedure, we could regain 10 (47.62%) accounts. Overall, we configured
MFA recovery for 54 accounts. We successfully regained access to27 (50%), and lost access to 23 (42.59%) accounts. We received noanswer from 4 (7.41%).
Account Recovery User Experience. During our user experience, we found MFA recovery implementations to differ vastly,e. g., websites offering different MFA or recovery procedures, orsome including the recovery procedure configuration in MFA setup
while others do not. We provide details on the reasons for accepting
or denying manual account recovery, finding similarly inconsistent and contradictory recovery philosophies, e. g., some websites
argued that because our account was barely used, there was noharm in allowing us back in, while others argued that this meant
there was no harm in us losing access and creating a new account.
Our findings hereof are described in the following paragraphs andillustrated in Figure 3.

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

Among the 30 accounts to which we did not regain access, 6(8.45%) were because we could not contact the website’s administrator or customer support team. Reasons included that on somewebsites, our free account plan was not eligible to receive any support, leaving us, and any user with a free account, no way to reachout and ask for help. Additionally, some websites required us to
use contact forms for which we did not have all the necessary information, e. g., credit card information or banking transactions,
IMEI phone identifiers, or information about the specific programfrom the vendor’s offer we were using, the information we were notasked to provide or set up during account creation. Another website
required us to provide our location and an order number, as thecontact form assumed that only paying customers with questions
or problems regarding their orders would reach out. Finally, one
website had removed their contact information from their website,arguing that due to a high request volume, only specific user groups
were allowed to contact them. We refrained from reaching out witha dishonest request.
Of the websites we were able to contact, 10 (14.08%) had strict
policies regarding recovery and would not allow access without theconfigured MFA or recovery procedure. Finally, 13 (18.31%) websites
required us to provide various sensitive and identifying documents,
such as governmental ID cards or bank transaction receipts; when
we refused, we lost access to these accounts. However, we successfully recovered 6 (8.45%) accounts due to being able to access ourprimary email MFA method and 10 (14.08%) due to being able toprovide specific knowledge about the account during the recovery
procedure. This encompassed metadata or contextual information,such as our organization name, our address, the account creation
date, typical login locations as well as our ISP or IP. One website
asked us a series of questions about our typical behavior and settings, many of which were potentially public knowledge, such as
the date of our last broadcast or other connected platforms. We recovered one account by answering security questions that had beenpart of the account creation process. Three websites asked about potentially sensitive, personal, or identifying information such as, e. g.,recent orders or payment information—which we could not provide
as we never completed a transaction on the website—or use of an
SSH shell or GitHub recovery—which we had never connected—but
several websites allowed us to regain access despite failing theserequirements.
For 4 (5.63%) accounts, we regained access immediately after our
initial request with no further questions asked and no requirements
or needing to go through any recovery process. In 3 (4.23%) cases,
the recovery was time based. Here, MFA was disabled after a waiting
period of 3–14 days. In these cases, our primary email accountreceived multiple reminders and warnings, designed to give thelegitimate account owners time to interrupt the recovery in case amalicious or unknown third party made the recovery request.
Overall, we found that most of our requests to regain access were
fulfilled after just a few emails from our primary email accountand that we rarely required more than contextual knowledge about
either the account or us as the main user. Our findings suggest thatthe security of many deployed MFA recovery procedures is similar
to that of the email account connected to the MFA we wished torecover, or that of security questions, as many requested data points
were often easily guessable or public knowledge such as connectedCCS ’23, November 26–30, 2023, Copenhagen, Denmark

platforms or our companies address, or accessible via our email
inbox such as account creation dates or ISPs.
Correspondence with Website Support Teams. The majorityof accounts, to which we regained access, it was sufficient to have
email access (26, 70.27%); this encompasses accounts that used email
as their main MFA or recovery procedure, as well as accounts that
asked us to confirm our request by email to disable MFA and regain
access. While only confirmed by one website explicitly, we assume
this was required as a form of phishing protection.
In two cases, websites typically used for commercial purposes
tried to ask our account owner to allow them to disable MFA for us.
As they assumed a hierarchy within the account – e. g., a dedicatedmain user, admin or owner, and several team members–but foundus to be the only user, we received messages to the same email
both as a user who applied for a MFA reset, and as account ownerthat can allow or deny this request. Hence, we were able to allow
ourselves back into the account. While this might be an edge caseof their recovery procedure, it might lead to a decreased securityof important administrator roles in comparison to normal users, as
this cancels the extra obstacle of requiring admin approval.
Overall, we set up a recovery procedure during the account and
MFA setup for 50 (70.42%) websites. When we initially applied
for account recovery (cf. Appendix B.1 for the message) we did
not mention these recovery procedures, but only stressed that welost our wallet and phone. While this would render most pre-set
recoveries, such as backup codes, still usable (47, 87.04%), only aminority of 10 (14.08%) websites mentioned using the pre-set recovery procedure, with one of them disabling our MFA without further
requirements after we responded that we lost the backup codes aswell. 10 (14.08%) websites insisted on only allowing us access if weused either the original MFA or the recovery procedure we set up.
Only a few websites enforced and verified the configuration of a
MFA recovery procedure. During our experiment, 10 (14.08%) websites required us to set up and test the recovery procedure before
being allowed to complete the MFA setup or required a manual
confirmation that backup codes were stored before finalizing the
MFA configuration.
Documented MFA (Recovery) Procedures. In this paragraph,
we discuss our evaluation of how websites communicated the detailsof MFA and recovery procedures with users, encompassing both
our user experience during the account and MFA setup process, aswell as the publicly accessible help and support pages.
Building on our evaluation of help pages in our initial categorization (Section 3.1.1), for our user experience, we revisited each of
the 71 websites’ help and support pages in a more in-depth investigation of their MFA and recovery procedures and formed a basis
for comparison with our user experience. We collected additional
data about, e. g., whether the recovery setup was part of enabling
MFA, if users were warned about account loss in case of MFA loss,
or whether account restrictions or forced setup of recoveries werementioned. We next discuss interesting findings and differences between the documentation and our own experience. Figure 4 displays
the frequency with which the topics in this section are present ineither our experience or the official help and documentation pages.
Overall, our recovery experience never matched the documented

Amft et al.

Mention of
MFA Trigger
MFA Enforces
Recovery Setup
Account Usage
Mandates MFA
Warning about
SMS Usage
Vague Warning
About Account Loss
Explicit Warning
About Account Loss
Recovery is PartOf MFA Setup
020

Present in Experience

40

60

Percentage

80

100

Present in Documentation

Figure 4: Percentage of occurrence of certain topics or hints
within either our own experience when setting up accounts
and MFA, or the respective help and support pages. Notethat not all services offer SMS, i. e., related topics are notnecessarily applicable for all tested websites.

recovery procedure perfectly. Even if only the offered MFA methods are considered, we still only find 9 (12.68%) of all help andsupport pages to match our experience regarding available MFA
methods and recovery procedures. This difference suggests thatrecovery procedures are commonly not communicated properly
during setup, or that documentations are outdated or incomplete.
Our detailed analysis also reveals that at most two-thirds ofwebsites discussed recovery procedures during MFA setup, slightly
more frequently present in documentations (66.2%) than in (our)user experience (59.15%).
Although a perfectly secure MFA implementation would include
account loss for users unable to prove their identity, we foundonly 16 (22.54%) websites to explicitly warn us about the potential
danger of irrevocably losing access. This was even rarer within
official documentation (13, 18.31%). However, 35 websites (49.3%)
included more vague warning phrasings that mentioned accountloss but refrained from portraying it as a guaranteed consequenceof losing MFA and its recovery. Overall, we find that most of these
do not convey the urgency of keeping the MFA secured, or do noteven clearly state the connection between losing one’s MFA andtherefore losing access to the account. In the following, we examine
aspects from Figure 4 in tandem with our recovery rates.
We further find that the warnings did not always portray thetruth: 9 (12.68%) websites allowed us back into our accounts despite
having warned us explicitly that they would not. On the other hand,
we found that we lost 6 (8.45%) accounts without receiving any priorwarning. As such, we emphasize that when warning users aboutaccount loss in case of MFA loss, websites should also introduce
users to the recovery process, particularly during MFA setup. We
find that of the websites that do warn about account loss, only 14(19.72%) adhere to this, suggesting that users are often left alone
with the task of researching and enabling MFA recovery procedures,
despite the danger of account loss.
In our initial evaluation of deployed MFA and recovery procedures, we found SMS to be a popular MFA solution despite its
known insecurities [15, 46, 51, 80]. We were therefore interested

An Evaluation of the Security and Usability of Multi-Factor Authentication Recovery Deploymentsin the number of websites that acknowledged these insecurities
and warned about SMS and found that while SMS was available forroughly half of all pages, websites rarely (0% during user experience;9.86% of documentations) admitted its insecurity.
Further, our analysis indicates that websites were often inconsistent regarding their communication of mandatory MFA usage or
the enforcement of recovery procedures. We find usage restrictions
mentioned more commonly within official documentation (30.99%)
than we encountered (4.23%). This is likely because websites mostcommonly mentioned requiring account administrators to enable
MFA, and the accounts we created might not have had this status. Regarding enforcing recovery setup, only a small portion of
websites includes this (14.08% during user experience; 7.04% of documentations). As websites cannot verify that users did, e. g., store
their backup codes, we considered every mechanism that tried toonly allow users to continue the process after the recovery was set,e. g., by using checkboxes in which users confirm that they stored
the backup, asking users to use the backup as a proof, or setting up
recovery before MFA itself is set.
Websites often mention in which situations MFA is triggered,e. g., any login, only logins from new locations or devices, or specific functionalities such as accessing security-relevant settings orwithdrawing funds. However, we find the MFA trigger more commonly mentioned on help and support pages (92.96%) than as a partof the MFA setup (70.42%), although, e. g., how commonly users are
required to provide the information can be an important reason foror against enabling MFA.
Finally, we examined the number of pages within the documentation, help sites, and FAQs that discussed MFA and its recovery.
In cases where a website offered multiple services, we included
only those related to the specific account we set up, and argue that
ideally, all information should be present on a few pages to help
users find information quickly without getting lost within the documentation and to help website providers keeping all information
current as there are fewer pages requiring updates. We find that 18(25.35%) websites within our sample used more than three different pages to discuss all details of MFA and its recovery, therefore
potentially obstructing a user’s search for help, and involuntarily
burying information within too many pages.
Sample Characteristics. This study explored different MFA recovery implementations that services deployed in an in-depth studyof recovery procedures. While our sample is not generalizable,
we provide an overview of the websites within our sample andthe respective recovery results to contextualize our findings. An
anonymized overview of websites in our sample can be found in
our replication packageand in Table 6 in the Appendix. In general,
it is difficult to quantify the value of accounts, as this is not only
dependent on financial information linked to them, but also on
social, physical, or sentimental value, and therefore subjective foreach user. For example, while losing a social media account can bemeaningless for some, it might be the base of income for others.
Likewise, the precise threat model is dependent on how valuable
users estimate their accounts to be, which results in a similar subjective outcome. However, we deem the value of an account to behighly relevant to users, as this has an impact on their security
decisions and how they protect accounts.

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

To estimate the risk and damage for users when they lose accessto an account, or when a malicious third party is wrongfully given
access, we took notes on which types of data a website requires
to function, including financial data (e. g., credit card or banking
information), addresses (typically in the form of delivery or invoice addresses), or others such as telephone numbers. We find the
majority (42) of websites to require financial data to utilize the website, typically because its main purpose is related to investments,
shopping, or money pooling, and while accounts can be created
without providing payment data, they are unusable with regard
to the website’s purpose. Additionally, 25 websites do not require
storing payment data, but offer e. g., voluntary subscriptions. In
our recovery experiment, we were able to recover half of thesewebsites (50% for required data; 56% for voluntary data), showingthat the improper MFA implementations we encountered indeed
endangered valuable user data. Similarly, a large portion requires(30) or optionally collects (20) an address, and again was able to regain access to a significant number of websites (33.33% for required
data; 60.00% for voluntary data).
Overall, most websites in our sample offer paid services, therefore collecting personal data. While financial data is typically notdisplayed in plain text (i. e., attackers cannot gain the credit card
number as only the last digits are shown), it might still be abused by
an attacker if malicious orders are placed using stored payment information. Other information, such as personal addresses or phone
numbers, could be leaked and abused. In cases where the accountis lost without a malicious party gaining access, users might lose
access to products they legitimately bought, or lose content withsentimental value, such as social media they maintained for years,
or photos and memories they shared. However, we found somewebsites to re-allow access in case an invoice or bank statement
was provided, as this was accepted to identify the user.
When regarding the different sample sources, i. e., whether theaccount was added to our sample because it was interesting, topranked, or randomly selected, we see almost no differences regarding the success of our recovery study. Overall, we successfully
regained half of all samples, with the largest difference being that
we received any kind of answer or reaction from all top-ranked accounts, therefore losing more accounts than in other sample types.
However, the non-answer of 4 handpicked and random accounts
can also be considered as a form of account loss. Similarly, we found
top-ranked accounts to more commonly offer contact forms rather
than email addresses, which might be due to them being more professionally structured due to their popularity and large user bases.
While the recovery results were similar for all samples, we foundhandpicked accounts to diverge from the other two samples in somecases, e. g., by less commonly offering SMS or software tokens andoverall having less variety in their offered MFA methods, as well as
offering no contact forms, but usually, email addresses to reach out
to support staff.
While we excluded websites that required the provision of, e. g.,governmental IDs at start-up to protect the sensitive documents of
the researchers conducting this study, we found that 13 websites
later required them for recovery purposes, leading to account loss
on our side. This includes websites that mainly deal with, e. g.,
cryptocurrencies, payments, or IT-focused websites that offered,e. g., domain registrations, and website hosting. However, we also

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

encountered similar websites within the sample that allowed us
access without requiring a governmental ID, i. e., we regained accessto potentially valuable accounts with typically only access to therespective email inbox.

Key Points | Recovery User Experience We created 71 accounts,
lost access to our MFA, and recovered access to half of them.
Three quarters of the websites prompted us to set up any kindof recovery during MFA setup, typically backup codes. Reasonsfor success included email access and contextual knowledge;
reasons for account loss were strict policies, lack of ID, andlack of contact options. User experience and documentations
rarely matched, with documentations usually providing additional information.

4

ETHICS AND LIMITATIONSBelow, we discuss ethical concerns and the limitations of our work.

4.1

EthicsThis work was approved by our institution’s ethical review board.
In the first part of our evaluation, we manually examined nonpersonal, publicly available data. We further took care to not cause
unusual resource burdens, and refrain from naming any website
in the paper. In the second part of this work we contacted human
support staff who were not initially aware that our support requests
were part of an academic research project. However, we carefully
designed our study according to the Menlo Report [28] and its
recommendations for a deception study. Overall, we deem the harmcaused by our study to be justified by the benefits to society offeredby the evaluation of current MFA setups and recovery procedures
and the resulting recommendations for developers and website
operators. During our work, we caused an additional workloadwith our dishonest support requests. To minimize the burden on
individuals, we took care to exclude websites working solely withvolunteers. Furthermore, we limited ourselves to creating only one
account per service and sent only one support request. We kept
all communication as concise as possible and did not send out
reminders. We also refrained from collecting sensitive data, and did
not evaluate the support staff members themselves.
For each website, we studied the terms of service and refrained
from registering with 20 services for which we would have violated
them. As we were unable to ask for prior consent due to the nature of
the study, we followed best practices and sent a post hoc notificationand debriefing email to all services we evaluated (see Appendix B.1)
We informed them about the fully anonymous use of their data andallowed them to drop out of the study, in which case we deleted
all collected data for the respective website, and did not use it in
our work. A total of 4 websites declined participation and were
subsequently removed from our data set. Overall, we deem thedeception in this work to be necessary for our goal of evaluating
unbiased MFA recovery procedures, as otherwise support staff
members might have adapted their behavior. All details of our
methodology, including all texts we used to communicate withhuman support workers, were part of our approved ERB application.

Amft et al.

4.2

Limitations

In observational studies like ours, there are multiple potential
sources of bias or error. First, regarding our data source, we obtained
a list of websites offering MFA using the 2fa.directory database. As
we illustrate (c.f. Section 3.1) this database offers a public repository,
where volunteers can contribute data according to certain quality
criteria [3, 4, 5]. The quality criteria nicely align with our researchgoals, since they require high popularity ranks. We, therefore, decided to use 2fa.directory instead of manually going through top
websites. During our evaluation, we noticed that the list is slightly
biased towards including more technical websites (e. g., 43.59% of
all 1,303 websites belong to technical or related categories). However, these websites are likely more tech-affine and therefore may
be more likely to offer MFA. This suggests that our measurement
constitutes an upper bound to MFA recovery procedure quantity. To
categorize the websites, we restricted ourselves to publicly available
information, and might therefore have missed information only
accessible to authorized users. However, this reflects the perspective of users who lost access, search information on MFA recovery
procedures, and can also only rely on publicly available data.
The user experience study had additional limitations. We could
not create accounts on 111 websites, including websites that required us to provide a government-issued ID, sign a (paid) contract
for, e. g., a bank or investment service and utility providers for, e. g.,gas or electricity, be part of a certain group such as enrolled students, or own certain devices such as IoT devices. This decision wasmade to protect the privacy of the Ph.D. students conducting thisresearch. Additionally, our sample of service providers is globally
distributed, and some services abroad did not allow us to create
accounts located in Germany. We further focused on free accountplans and did not pay for any service. While premium account tiers
could have led to different results (e. g., because we would havereceived premium support), we think that our results still provide
valuable insights for the community, and reflect the experience ofusers who are unable to pay for such services. Furthermore, we excluded 155 websites that were not primarily in German or English.
While we aimed to sample high-ranked, handpicked, and random
websites to increase diversity, and resampled whenever we needed
to skip a website (cf. Section 3.1.1), the number of handpicked websites was limited and could therefore not be arbitrarily extended,resulting in uneven group sizes. Our study is further limited byour choice to not utilize the recovery codes we received on somewebsites and to retain our email inbox. However, our goal was totreat all websites equally, and we argue that losing our recovery
procedure along with the main MFA sufficiently reflects reality,
especially since the configuration is rarely enforced.
Finally, during our detailed analysis of our experience and official
help and documentation pages, we might have missed information
due to not following every path during the account and MFA setup.
We especially did not follow links to the documentation given
during MFA setup, but only the texts and prompts were shown
during the process.

5

DISCUSSION

Below, we discuss our results and address our research questions.

An Evaluation of the Security and Usability of Multi-Factor Authentication Recovery Deployments

5.1

RQ1: Diverse Landscape of Recovery
Procedures

We identified 17 different recovery procedures based on the publichelp and support pages of 1,303 websites (cf. Section 3.1.2). Most
providers offered support telephone hotlines, email addresses, or
local IT staff, followed by backup codes. All procedures offer limited
usability and security: Users may reach out to support channels
and IT staff over insecure channels, such as unencrypted email [53,82]. While backup codes are easy to distribute, they have similar
issues as passwords: users can lose them or store them insecurely.
The high diversity of MFA recovery procedures we found isin line with work by Ghorbani Lyastani et al. who investigated
MFA deployments without evaluating recovery. Implementations
between websites and service types differed vastly, and we could
not identify best practices regarding MFA recovery procedures. Dueto the lack of standards or best practices, most website providers
deployed custom procedures, often leading to contradictions between websites and making the lives of their users unnecessarily
complicated. For example, we both encountered websites suggesting setting up multiple backup methods, while others encouraged
their users to only configure one recovery procedure to reduce
their attack surface. While we agree that too many backup options
decrease security, it is sensible to not rely on a single method and toclearly communicate the offered methods and the significance of aworking recovery. Similarly, some websites warned us about using
SMS-based services due to various possible attacks on mobile networks, while others praised SMS for its availability and usability. In
general, SMS should be avoided due to several vulnerabilities [15, 46,
51, 80] – however, due to its advantages, it can be a valid method
for low-risk services if there is an alternative for users that areuncomfortable with sharing their private phone numbers. Afterreaching out to their support, some providers recommended wecreate new accounts instead of recovering the existing ones. They
made this recommendation after manually checking our accounts
and seeing that they were not frequently used.
Another provider re-allowed us access without identification
because the account was empty and not much used. In this specificcase, we deem both methods valid for empty accounts but argue
that account deletion is the safer option to give users full control
over the stored data in case recovery procedures are abused. In all ofthese cases, the deviating arguments have a well-meaning, true core,and in many cases, compromises based on the nature of the service
are necessary. We discuss shortcomings and recommendations in
Section 5.3.

5.2

RQ2: Inconsistencies Compromise Security& Usability

Overall, we found examples of both websites that prioritized security, and did not allow us access without the proper MFA or
recovery procedure we configured, and websites that helped us toregain access without having access to our second factor, lowering
authentication security (cf. Section 3.2.2). However, from a user’s
point of view, this means either sacrificing security or usability,
which can be especially critical for accounts that manage importantresources such as monetary funds or medical data.

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

We identified inconsistencies between the authentication security associated with the use of MFA, and the recovery procedures implemented on many websites. Using MFA is expected to strengthen
the security of authentication by limiting account access to users
who know the account password and are in possession of the configured additional authentication factor. However, having access tothe email address we used for account creation was sufficient formost of the accounts we regained access to. This effectively reduces
the security of MFA to that of email security, which has been criticized in prior work about account recovery [8, 53], and would allow
an attacker with email access to circumvent MFA. Whenever wewere required to provide contextual evidence such as our addressor the nature of connected platforms, the security decreased to thelevel of security questions, which are also known to be insecureand guessable [19, 71].
Although we never said anything about having lost backup codes
as well, services typically did not request them to recover our MFA.
This might be based on experiences with prior users that typicallylose both MFA and recovery, or that many users did not configure arecovery procedure. However, it also means that services refrained
from utilizing their own suggested recovery. Websites that distribute backup codes should adhere to these security decisions and not
accept other, even less secure alternative identifications.
We ascribed many of the rejected recovery requests to usability
issues. This includes not being able to contact the support, e. g.,
because contact is only available to logged-in or paying users. Thiscan be especially frustrating when users are left without any otherrecovery option.
In other cases, providers requested any form of ID or data without previously informing us that they would be part of the MFA
recovery procedure. While governmental IDs are a valid way toverify identities in some situations, they are not suitable to do sofor accounts that often know little more about their users than fullnames, birthdates, or email addresses. Furthermore, even videobased identification processes have been successfully circumvented,
allowing attackers to falsify documents and successfully impersonate their victims [30].
We find that websites rarely prepare users to provide ID identification, as they are usually not prompted to provide it duringaccount setup, or informed that it will become relevant during recovery. However, the availability of these documents can otherwise
be an issue, and lead to higher obstacles during recovery later on.
For example, not all countries, including the USA and Switzerland,
require their citizens to own governmental IDs. Further, the causesof MFA loss might also affect the availability of recovery: a lost or
stolen purse might mean that not only the TOTP app on a user’s
phone, but also their wallet and ID might be gone, or an emergency,such as a house fire, could destroy both mobile phones and identifying documents. Finally, users who are not aware of the relevanceof identifying information for MFA recovery might provide false
information to protect their privacy.
Some websites had exceptional expectations. For example, somehelp and support pages suggested setting up multiple hardware keys
in case one was lost or broken. While having a backup configured
is, in general, sensible, hardware keys are expensive, and having
to set up multiple keys can impose a financial burden excluding

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

users from setting up MFA or its recovery. However, this is also
encouraged by official FIDO guidelines regarding security keys [10].
Other websites downplayed the process of regaining access totheir users’ phone numbers, by suggesting users get a replacement
SIM card with which they can access their original MFA again.
However, this replacement can require significant time to arrive,
especially if the user needs replacement during, e. g., a vacation.
We find the documentation to largely match our experience, but
none of them matches perfectly. While we expect help and support
pages to be more detailed, we argue that some details, such as warnings about account loss, or the intended recovery procedure should
also be communicated during MFA setup. However, they are often
not found at all in either, therefore not sufficiently preparing users
for a potential loss of MFA. We further find that websites often do
not match their documentation regarding the most basic elements,
such as the offered MFA and setup methods, which suggests that
help and support pages might be outdated and not reflect the current recovery procedures. This could lead to unpleasant surprises
as users are not properly prepared for MFA recovery.

5.3

RQ3: Improving MFA Recovery Procedures

The three most pressing problems we identified related to unreachable support teams, insufficient MFA recovery procedure documentation, and information that was needed for recovery but not configured during setup. Based on our findings, we make the following
recommendations:Prepare. Our findings illustrate that recovery procedures requirea sufficient setup that should be part of the MFA setup. We suggestthat all information service providers requests during recovery
are collected at account setup. Users need to be warned about
everything that is strictly necessary, i. e., they should be told about
required documents and how to avoid recovery failures due tosecurity reasons. However, asking for too much or too sensitive
information can be risky in regard to, e. g., information abuse anddata breaches [47, 58] and might deter users. Future research should
work on identifying the right balance between the interests ofboth user privacy and MFA recovery. While we agree that limiting
the number of MFA recovery procedures is useful to not open up
multiple attack vectors, having fallbacks available is in line withgeneral security recommendations depending on the respective usecase and threat model [27, 56]. As some procedures, such as SMS
or security questions, are known to be insecure or impractical, wesuggest offering backup codes, additional unique communication
channels, e. g., bank statements where possible to identify the user
as account owners, or additional (desktop) devices with TOTP.
Communicate. The process of setting up, disabling, or recovering
MFA and what to expect in the worst case should be clearly anddirectly communicated during the setup process and be part ofpublicly available help and support pages. We argue that having too
many help pages can be confusing and lead to both users not finding
the information they need [73], and website maintainers struggling
to keep them all up-to-date. We, therefore, suggest keeping thenumber of different help and support pages as low as possible andusing structural elements like HTML tabs to help users more easilyfind the required information for their platform or MFA method.

Amft et al.

Maintain. Recovery procedures should be tested regularly. Websites could prompt users to, e. g., enter one of their backup codes or
a TOTP to make sure that the recovery procedure is still available
and working. Previous work has shown that pop-ups and security warnings can be perceived as obtrusive or might be ignoredby users [7, 14, 83, 91]. Hence, future work is required to find agood trade-off between verifying that recovery procedures are stillavailable, and obstructing user workflows too much.
Recover. For MFA recovery, we urge websites to adhere to their
documented recovery procedures. When websites deviate fromtheir documented recovery procedures, e. g., by restoring accessdespite warning about definite account loss, or by allowing additional recovery measures, they might create opportunities for
attackers that users are not aware of, or create (false) expectations
for other websites. Additionally, honest and clear communication
about the consequences of MFA loss can help users make informed
and empowered decisions about their authentication security. We
advise against using sensitive documents, such as governmental
IDs or utility bills, as identity proof, as they often cannot reliably
identify authorized users. However, if this kind of identification issensible and necessary, websites should take precautions to collectit early on and make sure that users are informed of this part oftheir recovery procedure.

5.4

Putting our Work into Context

Previous work often focused on investigating the theoretical usability of recovery procedures in different areas [36, 37, 42, 48, 65, 71,75]. In contrast, our work studies the first-hand user experience of
MFA recovery procedures on websites: We went through the actual
MFA recovery procedure processes implemented by 71 websites
and reported lessons learned. We could regain access to almost
half of the websites we tested based on having access to our email
inbox. Hence, our study illustrates that MFA recovery procedures’security on websites is affected by similar security risks as TOTP
apps or smartphone security implementing insecure methods as
fallback [37, 48, 75, 78].
Previous work evaluated security documentation for setting up
MFA [36, 76], and encountered issues around unclear instructions.
Our work shows many websites did not sufficiently document
their MFA recovery processes, e. g. instructions described recovery
processes that did not match the implemented deployments. Somewebsites did not provide documentation at all.

6

CONCLUSION

MFA recovery procedures need to balance security and usability
by allowing authorized users to access accounts without locking
them out due to MFA loss. We are the first to analyze deployed MFA
recovery procedures and related documentations on the web. In
this work, we first categorized the recovery procedures deployedon 1,303 websites, then conducted an in-depth analysis in which wecreated 71 accounts, configured MFA and recovery procedures, andfinally tried to recover them from supposed MFA loss. We foundthat most websites only offered a limited selection of recovery
procedures and that these were not necessarily part of the MFA
setup process. In the recovery process, websites often did not adhere
to their documentation and allowed us access without requiring

An Evaluation of the Security and Usability of Multi-Factor Authentication Recovery Deployments

the configured MFA or recovery. In most cases, having access tothe account’s associated email address was sufficient for accountrecovery. Overall, we recommend websites clearly and correctly
document their MFA recovery procedures. Measures to detect lossof MFA or recovery procedures before users are locked out should
be in place.

[18]

[19]

[20]

ACKNOWLEDGMENTS
We thank the included websites for allowing us to use their dataand our insights into their processes for this work. Furthermore, we
thank our reviewers for their valuable feedback. Finally, we thankPhilip Klostermeyer and Juliane Schmüser for proofreading ourwork. Funded by the and by the VolkswagenStiftung Niedersächsisches Vorab – ZN3695 and by the Deutsche Forschungsgemeinschaft(DFG, German Research Foundation) under Germany´s Excellence
Strategy - EXC 2092 CASA – 390781972.

REFERENCES
[1]
[2]
[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]
[11]
[12]

[13]

[14]

[15]

[16]

[17]

1Password. 2023. The World’s Most-Loved Password Manager. https://1passw
ord.com/ (visited on 09/06/2023). (2023).
2factorauth. 2023. 2FA Directory. https : / / 2fa . directory / int/ (visited on
09/06/2023). (2023).
2factorauth. 2023. 2fa.directory Excluded Categories and Websites. https://gith
ub.com/2factorauth/twofactorauth/blob/master/EXCLUSION.md (visited on
09/06/2023). (2023).
2factorauth. 2023. Contributing to 2fa.directory. https://github.com/2factorau
th/twofactorauth/blob/master/CONTRIBUTING.md (visited on 09/06/2023).
(2023).
2factorauth. 2023. Issue: Add Site with 2FA. https://github.com/2factorauth/tw
ofactorauth/issues/new?assignees=&labels=add+site&template=01-add-sitewith-2fa.yml&title=Add+%5Bsite+name%5D (visited on 09/06/2023). (2023).
Jacob Abbott and Sameer Patil. 2020. How Mandatory Second Factor Affects
the Authentication User Experience. In Proceedings of the 2020 CHI Conferenceon Human Factors in Computing Systems, 1–13.
Devdatta Akhawe and Adrienne Porter Felt. 2013. Alice in Warningland: A
Large-Scale Field Study of Browser Security Warning Effectiveness. In Proc.
22nd Usenix Security Symposium (SEC’13). USENIX Association.
Fatma Al Maqbali and Chris J Mitchell. 2018. Email-Based Password Recovery–
Risking or Rescuing Users? In 2018 International Carnahan Conference on Security Technology (ICCST). IEEE, 1–5.
FIDO Alliance. 2016. FIDO 2.0: Client To Authenticator Protocol. https://fidoal
liance.org/specs/fido-v2.0-rd-20161004/fido-client-to-authenticator-protoco
l-v2.0-rd-20161004.html (visited on 09/06/2023). (2016).
FIDO Alliance. 2023. FIDO Security Key UX Guidelines. https://fidoalliance.or
g/ux-guidelines/security-key-ux-guidelines/ (visited on 09/06/2023). (2023).
FIDO Alliance. 2021. FIDO1.0 U2F Specification Archive. https://fidoalliance.o
rg/specs/fido-u2f-v1.0-ps-20141009/ (visited on 09/06/2023). (2021).
Ron Amadeo. 2019. The Pixel 4’s Face Unlock Works on Sleeping, Unconscious
People. https://arstechnica.com/gadgets/2019/10/the-pixel-4s-face-unlockworks-on-sleeping-unconscious-people/ (visited on 09/06/2023). (2019).
Sabrina Amft, Sandra Höltervennhoff, Nicolas Huaman, Alexander Krause,
Lucy Simko, Yasemin Acar, and Sascha Fahl. 2023. Website: “We’ve Disabled
MFA for You”: An Evaluation of the Security and Usability of Multi-Factor
Authentication Recovery Deployments. https://publications.teamusec.de/2023ccs-multi-factor-recovery/ (visited on 09/06/2023). (2023).
Daniel V. Bailey, Philipp Markert, and Adam J. Aviv. 2021. "I Have No Idea
What They’re Trying to Accomplish:" Enthusiastic and Casual Signal Users’
Understanding of Signal PINs. In Seventeenth Symposium on Usable Privacy
and Security (SOUPS 2021). USENIX Association, (Aug. 2021), 417–436. isbn:978-1-939133-25-0. https://www.usenix.org/conference/soups2021/presentati
on/bailey.
Brian Barrett. 2019. Wired: How Twitter CEO Jack Dorsey’s Account Was
Hacked. https://www.wired.com/story/jack-dorsey-twitter-hacked/ (visited
on 09/06/2023). (2019).
David Basin and Cas Cremers. 2010. Degrees of Security: Protocol Guarantees in
the Face of Compromising Adversaries. In International Workshop on Computer
Science Logic. Springer, 1–18.
Erick Bauman, Yafeng Lu, and Zhiqiang Lin. 2015. Half a Century of Practice: Who is Still Storing Plaintext Passwords? In International Conference on
Information Security Practice and Experience. Springer, 253–267.

[21]

[22]

[23]
[24]

[25]

[26]

[27]
[28]

[29]

[30]

[31]

[32]

[33]

[34]

[35]

[36]

[37]

[38]
[39]

[40]

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

Joseph Bonneau. 2012. The Science of Guessing: Analyzing an Anonymized
Corpus of 70 Million Passwords. In 2012 IEEE Symposium on Security andPrivacy. IEEE, 538–552.
Joseph Bonneau, Elie Bursztein, Ilan Caron, Rob Jackson, and Mike Williamson.
2015. Secrets, Lies, and Account Recovery: Lessons from the Use of Personal
Knowledge Questions at Google. In Proceedings of the 24th International Conference on World Wide Web, 141–150.
Xander Bouwman, Harm Griffioen, Jelle Egbers, Christian Doerr, Bram Klievink,
and Michel van Eeten. 2020. A different Cup of TI? The Added Value of Commercial Threat Intelligence. In 29th USENIX Security Symposium (USENIX Security
20). USENIX Association, (Aug. 2020), 433–450. isbn: 978-1-939133-17-5. https://www.usenix.org/conference/usenixsecurity20/presentation/bouwman.
John Brainard, Ari Juels, Ronald L Rivest, Michael Szydlo, and Moti Yung. 2006.
Fourth-Factor Authentication: Somebody You Know. In Proceedings of the 13th
ACM Conference on Computer and Communications Security, 168–178.
Brian Dean. 2019. We Analyzed 5 Million Google Search Results - Here’s What
We Learned About Organic Click Through Rate. https://backlinko.com/googlectr-stats (visited on 01/04/2022). (2019).
Pew Research Center. 2021. Mobile Fact Sheet. https://www.pewresearch.org/i
nternet/fact-sheet/mobile/ (visited on 09/06/2023). (2021).
Stéphane Ciolino, Simon Parkin, and Paul Dunphy. 2019. Of Two Minds about
Two-Factor: Understanding Everyday FIDO U2F Usability through Device Comparison and Experience Sampling. In Fifteenth Symposium on Usable Privacy
and Security (SOUPS 2019).
Jessica Colnago, Summer Devlin, Maggie Oates, Chelse Swoopes, Lujo Bauer,
Lorrie Cranor, and Nicolas Christin. 2018. “It’s Not Actually that Horrible” Exploring Adoption of Two-Factor Authentication at a University. In Proceedingsof the 2018 CHI Conference on Human Factors in Computing Systems, 1–11.
Sanchari Das, Andrew Dingman, and L Jean Camp. 2018. Why Johnny Doesn’t
Use Two Factor a Two-Phase Usability Study of the FIDO U2F Security Key. In
International Conference on Financial Cryptography and Data Security. Springer,
160–179.
Roger Dingledine, Nick Mathewson, and Paul Syverson. 2004. Tor: The SecondGeneration Onion Router. Tech. rep. Naval Research Lab Washington DC.
D Dittrich and E Kenneally. 2012. The Menlo Report: Ethical Principles Guiding
Information and Communication Technology Research. https://catalog.caida.org/paper/2012_menlo_report_actual_formatted (visited on 09/06/2023). (2012).
Paul Ducklin. 2022. Slack Admits to Leaking Hashed Passwords for Five Years.
https://nakedsecurity.sophos.com/2022/08/08/slack-admits-to-leaking-hashed-passwords-for-three-months/ (visited on 09/06/2023). (2022).
erdgeist. 2022. Chaos Computer Club Hacks Video-Ident. https://www.ccc.de/en/updates/2022/chaos- computer- club- hackt- video- ident (visited on
09/06/2023). (2022).
Florian M Farke, Lennart Lorenz, Theodor Schnitzler, Philipp Markert, andMarkus Dürmuth. 2020. “You Still Use the Password After All”–Exploring
FIDO2 Security Keys in a Small Company. In Sixteenth Symposium on Usable
Privacy and Security (SOUPS 2020), 19–35.
SelfKey Foundation. 2022. All Data Breaches in 2019 – 2022 – An Alarming
Timeline. https://selfkey.org/data-breaches-in-2019/ (visited on 09/06/2023).
(2022).
Sergiu Gatlan. 2022. Lastpass Says Hackers Accessed Customer Data in New
Breach. https://www.bleepingcomputer.com/news/security/lastpass-says-ha
ckers-accessed-customer-data-in-new-breach/ (visited on 09/06/2023). (2022).
Eva Gerlitz, Maximilian Häring, Charlotte Theresa Mädler, Matthew Smith, andChristian Tiefenau. 2023. Adventures in Recovery Land: Testing the Account
Recovery of Popular Websites When the Second Factor is Lost. In NineteenthSymposium on Usable Privacy and Security (SOUPS 2023). USENIX Association,
227–243.
Eva Gerlitz, Maximilian Häring, and Matthew Smith. 2021. Please do not use!?
_ or your License Plate Number: Analyzing Password Policies in German
Companies. In Seventeenth Symposium on Usable Privacy and Security (SOUPS
2021), 17–36.
Sanam Ghorbani Lyastani, Michael Backes, and Sven Bugiel. 2023. A Systematic
Study of the Consistency of Two-Factor Authentication User Journeys on
Top-Ranked Websites. In 30th Annual Network & Distributed System Security
Symposium (NDSS’23). The Internet Society.
Conor Gilsenan, Fuzail Shakir, Noura Alomar, and Serge Egelman. 2023. Security and Privacy Failures in Popular 2FA Apps. In 32nd USENIX Security
Symposium (USENIX Security 23).
Google. 2023. Google. https://google.com (visited on 09/06/2023). (2023).
Andy Greenberg. 2013. Forbes: German Hacker Group Says It’s Broken The
iPhone’s TouchID Fingerprint Reader. https://www.forbes.com/sites/andygree
nberg/2013/09/22/german-hackers-say-theyve-broken-the-iphones-touchidfingerprint-reader/ (visited on 09/06/2023). (2013).
Andy Greenberg. 2017. Wired: Watch a 10-Year-Old’s Face Unlock His Mom’s
iPhone X. https://www.wired.com/story/10-year-old-face-id-unlocks-mothers-iphone-x/ (visited on 09/06/2023). (2017).

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

[41]
[42]

[43]
[44]

[45]

[46]
[47]

[48]

[49]
[50]

[51]

[52]

[53]

[54]

[55]

[56]

[57]

[58]

[59]

[60]

[61]

[62]

[63]

Ameya Hanamsagar, S Woo, Christopher Kanich, and Jelena Mirkovic. 2016.
How Users Choose and Reuse Passwords. Information Sciences Institute.
Alina Hang, Alexander De Luca, Emanuel Von Zezschwitz, Manuel Demmler,and Heinrich Hussmann. 2015. Locked Your Phone? Buy a New One? From
Tales of Fallback Authentication on Smartphones to Actual Concepts. In Proceedings of the 17th International Conference on Human-Computer Interaction
with Mobile Devices and Services, 295–305.
Nadjla Hariri. 2011. Relevance Ranking on Google: Are Top Ranked Results
Really Considered more Relevant by the Users? Online Information Review.
Helene Hembrooke, Bing Pan, Thorsten Joachims, Geri Gay, and Laura Granka.
2005. In Google we Trust: Users Decisions on Rank, Position and Relevancy.
Journal of Computer-Mediated Communication, Special Issue on Search Engines.
Cormac Herley. 2009. So long, and No Thanks for the Externalities: The Rational
Rejection of Security Advice by Users. In Proceedings of the 2009 Workshop on
New Security Paradigms Workshop, 133–144.
Roger Piqueras Jover. 2020. Security Analysis of SMS as a Second Factor of
Authentication. Communications of the ACM, 63, 12, 46–52.
Sowmya Karunakaran, Kurt Thomas, Elie Bursztein, and Oxana Comanescu.
2018. Data Breaches: User Comprehension, Expectations, and Concerns withHandling Exposed Data. In Fourteenth Symposium on Usable Privacy and Security (SOUPS 2018), 217–234.
Johannes Kunke, Stephan Wiefling, Markus Ullmann, and Luigi Lo Iacono. 2021.
Evaluation of Account Recovery Strategies with FIDO2-Based Passwordless
Authentication. In Roßnagel, Schunck et al.(Eds.): Open Identity Summit 2021
(OID’21), Lyngby, Denmark, June 1st and 2nd, 2021. Gesellschaft für InformatikeV, 59–70.
LastPass. 2023. LastPass | Password Manager. https : / / www . lastpass . com/(visited on 09/06/2023). (2023).
Victor Le Pochat, Tom Van Goethem, Samaneh Tajalizadehkhoob, Maciej Korczyński, and Wouter Joosen. 2019. Tranco: A Research-Oriented Top Sites
Ranking Hardened Against Manipulation. In Proceedings of the 26th Annual
Network and Distributed System Security Symposium (NDSS 2019). (Feb. 2019).
doi: 10.14722/ndss.2019.23386.
Kevin Lee, Benjamin Kaiser, Jonathan Mayer, and Arvind Narayanan. 2020. An
Empirical Study of Wireless Carrier Authentication for SIM Swaps. In SixteenthSymposium on Usable Privacy and Security (SOUPS 2020). USENIX Association,(Aug. 2020), 61–79. isbn: 978-1-939133-16-8. https://www.usenix.org/conferen
ce/soups2020/presentation/lee.
Kevin Lee, Sten Sjöberg, and Arvind Narayanan. 2022. Password Policies of
Most Top Websites Fail to Follow Best Practices. In Eighteenth Symposium on
Usable Privacy and Security (SOUPS 2022), 561–580.
Yue Li, Haining Wang, and Kun Sun. 2018. Email as a Master Key: Analyzing
Account Recovery in the Wild. In IEEE INFOCOM 2018-IEEE Conference on
Computer Communications. IEEE, 1646–1654.
Google LLC. 2023. Google Authenticator. https://play.google.com/store/apps/details?id=com.google.android.apps.authenticator2 (visited on 09/06/2023).
(2023).
D. M’Raihi, S. Machani, M. Pei, and J. Rydell. 2011. TOTP: Time-Based One-Time
Password Algorithm. RFC 6238. http://www.rfc- editor.org/rfc/rfc6238.txt.
RFC Editor, (May 2011). http://www.rfc-editor.org/rfc/rfc6238.txt.
Pratyusa K. Manadhata and Jeannette M. Wing. 2011. An Attack Surface Metric.
IEEE Transactions on Software Engineering, 37, 3, 371–386. doi: 10.1109/TSE.20
10.60.
Karola Marky, Kirill Ragozin, George Chernyshov, Andrii Matviienko, Martin
Schmitz, Max Mühlhäuser, Chloe Eghtebas, and Kai Kunze. 2022. “Nah, it’s justannoying!” A Deep Dive into User Perceptions of Two-Factor Authentication.
ACM Transactions on Computer-Human Interaction, 29, 5, 1–32.
Peter Mayer, Yixin Zou, Florian Schaub, and Adam J Aviv. 2021. "Now I’m a bit{ angry: } " Individuals’ Awareness, Perception, and Responses to Data Breaches
that Affected Them. In 30th USENIX Security Symposium (USENIX Security 21),393–410.
Philipp Mayring. 2014. Qualitative Content Analysis: Theoretical Foundation,
Basic Procedures and Software Solution. Social Science Open Access Repository(SSOAR), Klagenfurt, 143.
Allison McDonald, Catherine Barwulor, Michelle L Mazurek, Florian Schaub,and Elissa M Redmiles. 2021. "It’s stressful having all these phones": Investigating Sex Workers’ Safety Goals, Risks, and Practices Online. In 30th USENIX
Security Symposium. USENIX, 375–392.
Allison McDonald, Carlo Sugatan, Tamy Guberek, and Florian Schaub. 2021.
The Annoying, the Disturbing, and the Weird: Challenges with Phone Numbers
as Identifiers and Phone Number Recycling. In Proceedings of the 2021 CHI
Conference on Human Factors in Computing Systems, 1–14.
Nora McDonald, Sarita Schoenebeck, and Andrea Forte. 2019. Reliability andInter-Rater Reliability in Qualitative Research: Norms and Guidelines for CSCW
and HCI Practice. ACM on Human-Computer Interaction, 3, CSCW, Article 72,
23 pages.
Inc. Meta Platforms. 2023. Facebook. https://facebook.com (visited on 09/06/2023).
(2023).

Amft et al.

[64]

[65]

[66]
[67]

[68]

[69]

[70]

[71]

[72]

[73]

[74]

[75]

[76]

[77]

[78]

[79]

[80]

[81]

[82]

[83]

[84]

[85]
[86]
[87]

Patrick Mutchler, Adam Doupé, John Mitchell, Christopher Kruegel, and Giovanni Vigna. 2015. A Large-Scale Study of Mobile Web App Security. In Proc.
2015 Mobile Security Technologies Workshop (MoST’15). IEEE.
Lorenzo Neil, Elijah Bouma-Sims, Evan Lafontaine, Yasemin Acar, and Bradley
Reaves. 2021. Investigating Web Service Account Remediation Advice. In Seventeenth Symposium on Usable Privacy and Security (SOUPS 2021), 359–376.
Netflix. 2023. Netflix. https://netflix.com (visited on 09/06/2023). (2023).
William Newhouse, Brian Johnson, Sarah Kinling, Jason Kuruvilla, Blaine
Mulugeta, and Kenneth Sandlin. 2019. NIST SPECIAL PUBLICATION 180017 Multifactor Authentication for E-Commerce Risk-Based, FIDO Universal
Second Factor Implementations for Purchasers. https : / / doi . org / 10 . 6028
/NIST.SP.1800-17. (July 2019).
Marten Oltrogge, Nicolas Huaman, Sabrina Amft, Yasemin Acar, Michael
Backes, and Sascha Fahl. 2021. Why Eve and Mallory Still Love Android: Revisiting TLS (In)Security in Android Applications. In Proc. 30th Usenix Security
Symposium (SEC’21). USENIX Association.
Sarah Pearman, Jeremy Thomas, Pardis Emami Naeini, Hana Habib, Lujo
Bauer, Nicolas Christin, Lorrie Faith Cranor, Serge Egelman, and Alain Forget.
2017. Let’s Go in for a Closer Look: Observing Passwords in Their Natural
Habitat. In Proceedings of the 2017 ACM SIGSAC Conference on Computer andCommunications Security, 295–310.
Vilius Petkauskas. 2022. Thomson Reuters Collected and Leaked at Least 3TB
of Sensitive Data. https://cybernews.com/security/thomson-reuters-leaked-ter
abytes-sensitive-data/ (visited on 09/06/2023). (2022).
Ariel Rabkin. 2008. Personal Knowledge Questions for Fallback Authentication:
Security Questions in the Era of Facebook. In Proceedings of the 4th Symposium
on Usable Privacy and Security, 13–23.
Simone Raponi and Roberto Di Pietro. 2020. A Longitudinal Study on Websites
Password Management (in)Security: Evidence and Remedies. IEEE Access, 8,52075–52090.
Elissa M Redmiles, Noel Warford, Amritha Jayanti, Aravind Koneru, Sean Kross,
Miraida Morales, Rock Stevens, and Michelle L Mazurek. 2020. A Comprehensive Quality Evaluation of Security and Privacy Advice on the Web. In 29th
USENIX Security Symposium (USENIX Security 20), 89–108.
Ken Reese, Trevor Smith, Jonathan Dutson, Jonathan Armknecht, Jacob Cameron,and Kent Seamons. 2019. A Usability Study of Five Two-Factor Authentication
Methods. In Fifteenth Symposium on Usable Privacy and Security (SOUPS 2019).
Joshua Reynolds, Nikita Samarin, Joseph Barnes, Taylor Judd, Joshua Mason,
Michael Bailey, and Serge Egelman. 2020. Empirical Measurement of Systemic
2FA Usability. In 29th USENIX Security Symposium (USENIX Security 20), 127–
143.
Joshua Reynolds, Trevor Smith, Ken Reese, Luke Dickinson, Scott Ruoti, and
Kent Seamons. 2018. A Tale of Two Studies: The Best and Worst of Yubikey
Usability. In 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 872–888.
Stuart Schechter, AJ Bernheim Brush, and Serge Egelman. 2009. It’s No Secret.
Measuring the Security and Reliability of Authentication via “Secret” Questions.
In 2009 30th IEEE Symposium on Security and Privacy. IEEE, 375–390.
Stuart Schechter, Serge Egelman, and Robert W Reeder. 2009. It’s Not What You
Know, But Who You Know: A Social Approach to Last-Resort Authentication.
In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems,
1983–1992.
Zoë Schlanger. 2017. A Man was Able to Use Face ID to Unlock His Brother’s
New iPhone X. https://qz.com/1120545/a-man-was-able-to-use-face-id-to-un
lock-his-brothers-apple-aapl-iphone-x (visited on 09/06/2023). (2017).
Hossein Siadati, Toan Nguyen, Payas Gupta, Markus Jakobsson, and Nasir
Memon. 2017. Mind Your SMSes: Mitigating Social Engineering in Second
Factor Authentication. Computers & Security, 65, 14–28.
Peter Snyder, Cynthia Taylor, and Chris Kanich. 2017. Most Websites Don’t
Need to Vibrate: A Cost-Benefit Approach to Improving Browser Security. In
Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 179–194.
Christian Stransky, Oliver Wiese, Volker Roth, Yasemin Acar, and Sascha Fahl.
2022. 27 Years and 81 Million Opportunities Later: Investigating the Use of
Email Encryption for an Entire University. In 43rd IEEE Symposium on Security
and Privacy, IEEE S&P 2022, May 22-26, 2022. IEEE Computer Society, (May2022).
Joshua Sunshine, Serge Egelman, Hazim Almuhimedi, Neha Atri, and Lorrie
Faith Cranor. 2009. Crying Wolf: An Empirical Study of SSL Warning Effectiveness. In Proc. 18th Usenix Security Symposium (SEC’09). USENIX Association.
Joshua Tan, Lujo Bauer, Nicolas Christin, and Lorrie Faith Cranor. 2020. Practical Recommendations for Stronger, More Usable Passwords Combining Minimumstrength, Minimum-length, and Blocklist Requirements. In Proceedings of the2020 ACM SIGSAC Conference on Computer and Communications Security, 1407–
1426.
Twilio. 2023. Authy. https://authy.com/ (visited on 09/06/2023). (2023).
Inc. Twitter. 2023. X. https://twitter.com (visited on 09/06/2023). (2023).
Blase Ur, Fumiko Noma, Jonathan Bees, Sean M Segreti, Richard Shay, Lujo
Bauer, Nicolas Christin, and Lorrie Faith Cranor. 2015. “I Added ‘!’at the End

An Evaluation of the Security and Usability of Multi-Factor Authentication Recovery Deployments

[88]

[89]

[90]

[91]

[92]

[93]
[94]

[95]

[96]

A

to Make It Secure”: Observing Password Creation in the Lab. In Symposium on
Usable Privacy and Security (SOUPS).
Chun Wang, Steve TK Jan, Hang Hu, Douglas Bossart, and Gang Wang. 2018.
The Next Domino to Fall: Empirical Analysis of User Passwords Across Online
Services. In Proceedings of the Eighth ACM Conference on Data and Application
Security and Privacy, 196–203.
Ke Coby Wang and Michael K. Reiter. 2019. How to End Password Reuse on theWeb. In 26th Annual Network and Distributed System Security Symposium, NDSS
2019, San Diego, California, USA, February 24-27, 2019. The Internet Society.
https://www.ndss-symposium.org/ndss-paper/how-to-end-password-reuseon-the-web/.
Rick Wash, Emilee Rader, Ruthie Berman, and Zac Wellmer. 2016. Understanding Password Choices: How Frequently Entered Passwords are Re-used AcrossWebsites. In Twelfth Symposium on Usable Privacy and Security (SOUPS 2016),
175–188.
Joel Weinberger and Adrienne Porter Felt. 2016. A Week to Remember: The
Impact of Browser Warning Storage Policies. In Proc. 12th Symposium on Usable
Privacy and Security (SOUPS’16). USENIX Association.
Dominik Wermke, Noah Wöhler, Jan H Klemmer, Marcel Fourné, Yasemin
Acar, and Sascha Fahl. 2022. Committed to trust: a qualitative study on security& trust in open source software projects. In 2022 IEEE Symposium on Security
and Privacy (SP). IEEE, 1880–1896.
Inc. Wikimedia Foundation. 2023. Wikipedia. https://en.wikipedia.org (visited
on 09/06/2023). (2023).
Davey Winder. 2019. Forbes: Apple’s iPhone FaceID Hacked In Less Than 120
Seconds. https://www.forbes.com/sites/daveywinder/2019/08/10/apples-iphone-faceid-hacked-in-less-than-120-seconds/ (visited on 09/06/2023). (2019).
Davey Winder. 2019. Forbes: Collection 1: More Than 770M People Pwned In
Biggest Stolen Data Dump Yet. https://www.forbes.com/sites/daveywinder/20
19/01/17/collection-1-more-than-770m-people-pwned-in-biggest-stolen-dat
a-dump-yet/ (visited on 09/06/2023). (2019).
Shundan Xiao, Jim Witschey, and Emerson Murphy-Hill. 2014. Social Influences on Secure Development Tool Adoption: Why Security Tools Spread. In
Proceedings of the 17th ACM Conference on Computer Supported Cooperative
Work & Social Computing, 1095–1106.

MFA METHODS AND KNOWN RECOVERYPROCEDURES

Below, we provide background information for MFA approaches
and recovery procedures we encountered. We also discuss theirimpact on authentication security and usability.

A.1

Email

MFA based on email, commonly uses pre-generated one-time passcodes (OTP) or a verification link sent to the user via email. Consequently, users retain account access as long as they can access theiremail account. However, email security is lacking when comparedto other mechanisms, as emails are usually sent in plaintext [82],and email account security often depends on password-based authentication [41, 88]. Password resets also often occur via email [38,63, 66, 86, 93], effectively downgrading security to the email accounts’ password. It is further possible to use email OTPs as arecovery procedure for MFA4 .

A.2

SMS & Phone Call OTP

A common alternative to email are OTPs via SMS. Apart from
that, users can receive automated phone calls to provide OTPs.
Most people own a phone [23], making SMS or phone calls popular
additional authentication factors. The recovery of this method ispotentially expensive and time-consuming because it requires users
to obtain a new device and regain access to their phone number.
While popular, telephone-based MFA has proven to be insecure due
to its susceptibility to various severe attacks [15, 46, 51, 80], that
4 In our evaluation, we only considered this as recovery if it was explicitly a secondary

backup, and otherwise considered it as an additional MFA method.

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

enable third parties to intercept traffic or change the user’s phone
number to the attacker’s device via social engineering. Similar toemail, a secondary phone number can be used to offer SMS or phone
calls as MFA recovery4 .

A.3

Mobile Applications

Mobile applications, such as time-based one-time password (TOTP)
apps like Google Authenticator [54], are a common MFA method.
They generate one-time passwords [55] (OTPs) that users can provide to prove their identity4 . The app is set up by scanning a confidential QR code or entering a secret key provided by the respective
website, which serves as a seed for the TOTP algorithm. As longas both server and client have the same current time and share thesame secret, they can independently compute the same TOTP values. Besides this, some apps like Authy [85], or password manager
apps such as LastPass [49] or 1Password [1] require users to registeran account and offer a cloud backup of the supported MFA as atrade-off. While this eases recovery, it also means that users are notfully in control over their data, as service providers also have accessto the backups. Related to this, LastPass leaked user data during
two hacks in 2022 [33]. Lastly, some websites provide custom apps
for their respective services, and include MFA in the forms of, e. g.,
TOTP, usage of biometric sensors embedded in smartphones, orpush notifications. The provided security and usability in thesecases is limited by the app vendor and its implementation [64].

A.4

Hardware Tokens

Hardware tokens (e. g., smart-cards), code generators (e. g., Transaction Authentication Number (TAN) generators), or Universal Second Factor (U2F) [11] hardware keys, are established MFA methods.
WebAuthn supports the use of trusted platform modules, including
biometric identity checks [9]. The adoption of U2F and WebAuthn
has been limited [26], often due to their complex initial setups anddevice requirements. However, recent improvements provide better
usability [74]. The protocol offers verified security guarantees [16]and the main assumption is the secrecy of the chosen hardware
token, resulting in the credential proof being something you have.

A.5

User-Dependent Methods

Below, we illustrate approaches that are dependent on the user, their
input, or inherent features. First, biometry-based MFA commonly
relies on apps that use the internal biometry security options of,e. g., mobile phones to authenticate. The main advantage comparedto mobile applications is that authentication uses biometric features,
requiring no additional device or memorization. While technicallysecure, there have been reports of malfunctioning biometrics [12,39, 40, 79, 94], and they further require very sensitive personal information for authentication. Another approach are secret questions
that supposedly only the authorized user can answer, and backup
codes that users are given and supposed to store securely or print,
or the storage of the TOTP secret, i. e., the secret key or QR code
shown during MFA setup. While these methods are commonly
deployed as recovery or alternative to MFA, the security is often
lacking: security questions can easily be guessed, especially since
they tend to require information that is often known to acquaintances or social media contacts [19, 71, 77], and user-set passwords

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

are often weak or re-used [18, 69, 87, 89, 90]. Methods such as
backup codes or stored secrets are easily lost or allow attackers to
bypass MFA completely if they gain access.

A.6

Website-Dependent Methods

We also distinguish methods that are dependent on the websites.
In case access to MFA is lost, users can often contact the website
support. In other cases, especially for team-accounts or self-hosted
tools, the account is managed by local administrators that can becontacted. Both cases can include fixed waiting times or identity
verification for account recovery. In some cases, this process is morestraightforward, as users are asked to provide a photo proof ofthem holding, e. g., handwritten notes or governmental IDs. Other
websites utilize automated dedicated recovery systems, or disable
MFA automatically when the password is reset. The security ofthese methods depends on the respective communication channel,
such as email or HTTPS. While easy to use, the usability can suffer
from, e. g., long response times or queries for additional information,
as methods involving any form of communication are often dynamic
and resolved on a case-by-case basis.

A.7

Additional Methods

In some cases, MFA is not needed for login, but required forcertain service features such as withdrawing funds. Some websites
allow to omit MFA for trusted devices. While used as recovery,
those methods are often bound to a time-limit of, e. g., 30–90 days
after which MFA is required again, leading to a decreased backup
usability if MFA is lost outside this time frame.
Other recovery advice might combine some of the previously
mentioned factors, or give additional advice to, e. g., contact mobile
providers to replace inaccessible SIM cards. Finally, the recovery
can simply consist of setting up additional MFA methods, i. e.,
extra methods that can all act as a backup for whichever method
was lost by the user. While this has been an official FIDO recommendation [10], it also increases the attack vectors for malicious
third parties.
While neither a MFA nor recovery procedure, we encounteredadditional edge cases in which we were unable to determine themethods. This includes inaccessible help pages we were unable
to visit due to, e. g., geo-blocking or deleted pages, and websites
either mentioning explicitly that they had no recovery methods,
or websites that did not mention recovery at all.

B GENERAL APPENDIX
B.1 Recovery Test Templates
Recovery Request Message. Subject: Lost Second Factor
Hello,I recently registered an account on your website and enabled 2FA.
Yesterday, my backpack with my wallet and phone was stolen, so
now I cannot access anything, and am unable to get past the 2FA.
What do I need to do to regain access? My username/email/accountnumber is data.
Best regards,
First Author

Amft et al.

Debriefing Message. Dear website support,
We are a research team at the Anonymized For Review Institution,
Country, and study the usability of multi factor authentication(MFA). We are currently investigating the trade-off between usability and security of multi factor authentication recovery. As apart of this research, we created accounts on several websites, enabled MFA, and tried to recover our accounts after a while due tosupposedly inaccessible MFA. These websites were chosen because
they were listed on 2fa.directory [1], ranked highly on Tranco [2],or because their documentation implied interesting MFA methods.
As you might already guess, we write to you today because yourwebsite was among the ones we chose. We wanted to inform you
about your involuntary participation, as well as give you the time
to decline and have your data removed. On date, we send you anemail/chat request (request ID if possible), which was part of ourresearch, i.e., it was not a real users’ request to recover their MFA.
While we are not content about deceiving you and sending a false
request, we wanted to assure you that it was necessary for the goalof our research. We wanted to truly gauge the experience of a typical user, and to assess the usability and security of MFA procedureson the web. Due to this goal, we were sadly unable to communicate
the true purpose of our request upfront. Please be assured that weonly sent one line of request, and that we kept it as short as possible
to decrease the load on your staff caused by our inquiry.
As mentioned, we would like to give you the opportunity to haveyour data removed and drop out of our data set before we de-identify
it and use it as part of a scientific article. If you let us know until
October 10th5 , we will irrevocably delete all communication withor data about your website. Please be assured that we currently
only store the data on encrypted, self-hosted servers, and that only
involved researchers have access to our results. Furthermore, wewill never publish the name of your website or any involved staff
member. We will de-identify your website in our work, and wewill never mention your service’s or website name. We hope that
our study did not cause too much harm, and that you do not have
objections against us using the data. In any case, we thank you forbeing able to create an account and collect valuable experiences,
for the helpful support you gave us, and want to apologize for any
inconvenience caused by our request.
If you are interested in more details of our research, please visit our
project website: https://anonymizedforreview.tld/projects/multifactorrecovery/.
In case you are interested in the results of our research, we would
be happy to share the paper once it is published. Finally, we arealso interested in deepening our understanding of MFA and its
recovery by also including your perspective in future research (e.g.
an interview). If you would be available for a follow-up study, we
would be delighted to hear from you!
Best regards,
First Author (PhD Candidate)
Anonymized For Review Institution[1] https://2fa.directory/int/[2] https://tranco-list.eu/

5 This initially encompassed two weeks, but if any further opt-outs would have reached

us until submission, we would have removed them as well.

An Evaluation of the Security and Usability of Multi-Factor Authentication Recovery Deployments

B.2

Additional Tables

Table 2: Distribution of the initial 1,303 websites over various
TLDs.
Top-Level Domain
ca
ch
co.uk
com
com.au
de
edu
gov
io
net
org
other

CountsRelative

15
13
21
944
22
48
32
10
52
32
41
73

1.15%
1.00%
1.61%
72.45%
1.69%
3.68%
2.46%
0.77%
3.99%
2.46%
3.15%
5.60%

Table 3: Distribution of the initial 1,303 websites over thelisted bins of Tranco ranks.
Tranco Ranking
1 - 20000
20001 - 40000
40001 - 60000
60001 - 80000
80001 - 100000
100001 - 120000
120001 - 140000
140001 - 160000
160001 - 180000
180001 - 200000≥ 200001

CountsRelative

692
165
95
55
42
34
37
23
11
19
130

53.11%
12.66%
7.29%
4.22%
3.22%
2.61%
2.84%
1.77%
0.84%
1.46%
9.98%

We were further interested in the top-level domains (TLDs) of
websites in our sample. Due to our removal of 155 websites inlanguages we were not fluent in, we found the vast majority ofremaining websites (944, 72.45%) to be .com websites, with theremaining being other English (e. g., .ca, .com.au, .co.uk) or German(.de, .ch) TLDs. Additionally, we find some neutral ones (.io, .net,.org), and .edu and .gov, which we almost exclusively found in therespective categories of Education, Universities and Government(see Table 2). We find no significant differences between recovery
methods offered on different TLDs.

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

Amft et al.

Table 4: Codes used in our research. MFA Methods and Recovery Methods coded within our procedure categorization describedin Section 3.1.1
Code
MFA Methods
SMS
Phone Call
Email
Hardware Token
Mobile App
Other MFANo MFA
Recovery Methods
Contact Website Support
Backup Codes
Contact Local Admin
Additional MFA Method
Backup SMS/Phone Call
Backup Email
Dedicated Account Recovery System
TOTP Seed
Trusted Device
Photo/Official ID Proof
MFA Not Needed for Login
Security Question
Password Reset
Other Recovery
Help Page Not Accessible
No MFA Recovery Available
No MFA Recovery Documented

Frequency Description
559 (42.90%) SMS (usually with OTP) sent to users’ phone number.
156 (11.97%) Call used to convey information (e. g., spoken code, code hidden in caller ID).
205 (15.73%) Email (usually with OTP) sent to users’ associated email address.
179 (13.74%) Hardware device, e. g., U2F security key, or physical OTP generator.
1,036 (79.51%) Mobile app to verify user. Usually TOTP apps, but also proprietary apps using, e. g., push notifications.
139 (10.67%) Other methods such as biometry, local files, printed codes via mail and others.
512 (28.21%) No MFA present, i. e., the website never offered MFA, removed it, or never mentions it on public help pages.
491 (37.68%) Contact the service over varying mediums. Can include fixed wait or ID checks.
451 (34.61%) Code provided by service. It can be both an OTP or secret knowledge, distributed number varies.
159 (12.20%) Contact local admin or designated superuser to, e. g., reset MFA.
150 (11.51%) Explicit mention of using multiple MFA methods to have a backup.
78 (5.99%) Send SMS or call user. Only when SMS/Call is not a MFA method,or if it is explicitly a separate backup number.
49 (3.76%) Send email. Only when email is not a method, or if it is explicitly a separate backup address.
45 (3.45%) Service offers a designated form to initiate recovery for lost MFA. If this was just a standard contact form,
we chose Contact Service/Website instead.
44 (3.38%) Store the initial MFA secret to be able to set it up again. Usually TOTP seeds.
37 (2.84%) MFA not required after first authentication. Excludes cases in which this is limited to a certain time frame,as it is only a backup during this time.
30 (2.30%) Requires upload of a selfie of user holding, e. g., ID cards and written notes as identity proof.
25 (1.92%) MFA not required for login, only for certain functions. Users can simply log in and update MFA.
16 (1.23%) Answer a (set of) security questions to regain access.
13 (1.00%) Users can use the password reset function to deactivate MFA as well, therefore avoiding real recovery.
77 (5.91%) Other methods or combinations of methods, e. g., Contact the phone provider for new SIM, letters withrecovery codes, or using verification links via email and SMS and a photo proof.
9 (0.69%) Details cannot be verified due to unavailable help pages (e. g., geo-blocking, deleted pages)
1 (0.08%) The help and documentation explicitly stresses that there is no recovery procedure.
321 (24.64%) MFA is available, but no recovery is mentioned.

Table 5: Codes used in our research. Detailed Information describes the codes used within our recovery test (cf. Section 3.2.1) toaccount for additional interesting details not covered in the previous categorization.
Code

Frequency Frequency
Description
Experiment Documentation

Recovery is Part Of MFA Setup
Explicit Warning About Account Loss

42 (59.15%)
16 (22.54%)

Vague Warning About Account Loss

32 (45.07%)

Warning about SMS Usage
Account Usage Mandates MFA
MFA Enforces Recovery Setup
Mention of MFA Trigger
Number of Help Pages

0 (0%)
3 (4.23%)
10 (14.08%)
50 (70.42%)-

47 (66.2%) Whether recovery is discussed during/alongside MFA setup (yes, no)13 (18.31%) Whether website warns users about losing access if MFA is lostand no(t enough) recovery methods were set. Only includes if the warning was explicit
and stresses that account loss is final. (emphasized, yes, no)35 (49.3%) Similar to Warning, but also includes soft warnings in which alternative recoveries
are mentioned, or the account loss is only portrait as a vague possibility. (yes, no)7 (9.86%) Whether website warns about using SMS OTPs due to their insecurity (yes, no).
22 (30.99%) Whether users are hindered to freely enable and disable MFA (yes, no).
5 (7.04%) Whether users are forced to set up a backup (yes, no).
66 (92.96%) List of triggers that cause the users MFA to be requested (free text)
18 (25.35%) Whether a service has more than three relevant help pages regarding MFA
and its recovery (yes, no).

An Evaluation of the Security and Usability of Multi-Factor Authentication Recovery Deployments

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

Table 6: List of websites examined during our recovery experience described in Section 3.2, including their approximatedTranco rank, category or sample group. We chose to omit details on present MFA and recovery methods, as well as the recovery
result, within the public version of this work to ensure the anonymity of included websites.

ID

TrancoCategory

Sample

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71

10,001-50,000
10,001-50,000
50,001-200,000
50,001-200,000
50,001-200,000<1,00010,001-50,000
50,001-200,000<1,000<1,0005,001-10,000
1,001-5,000<1,0005,001-10,000
1,001-5,000
1,001-5,000
1,001-5,000<1,000<1,000<1,000<1,000<1,000<1,000<1,000<1,0001,001-5,000
5,001-10,000>200,000<1,000<1,000<1,000<1,0005,001-10,000
1,001-5,000
5,001-10,000>200,000
10,001-50,000
10,001-50,000
10,001-50,000<1,00050,001-200,000<1,000<1,0005,001-10,000
1,001-5,000<1,000<1,000<1,000<1,000<1,000<1,000<1,000<1,00010,001-50,000<1,000<1,00050,001-200,000
10,001-50,000<1,000<1,000<1,000<1,000<1,000<1,000<1,000>200,000
10,001-50,000
10,001-50,000
50,001-200,000
5,001-10,000
10,001-50,000

Cryptocurrencies
Hosting & VPS
Developers
Communication
Email
Security
Domains
Identity Management
SocialGamingCommunication
Marketing & Analytics
SocialDevelopers
Other
Crowdfunding
Security
Domains
Other
Entertainment
Hotels & Accommodations
Other
Education
Crowdfunding
Domains
Other
VPN Providers
Cryptocurrencies
Communication
RetailOther
Crowdfunding
Domains
Cryptocurrencies
GamingCloud Computing
Identity Management
SocialCryptocurrencies
SocialBetting
Marketing & Analytics
Developers
Hosting & VPS
Domains
SocialSocialBackup & Sync
Developers
Communication
Domains
Communication
Remote Access
Finance
Other
RetailFood
Communication
Other
RetailDomains
Investing
RetailSecurity
Hosting & VPS
Cryptocurrencies
Domains
Cryptocurrencies
Payment
Cryptocurrencies
Domains

Handpicked
RandomRandomRandom
Handpicked
Handpicked
Handpicked
Handpicked
Top-Ranked
Top-Ranked
Random
Handpicked
Top-Ranked
RandomRandom
Handpicked
Handpicked
Handpicked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
RandomRandomRandomRandomRandomTop-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
RandomRandomRandomRandomRandomRandom
Handpicked
Top-Ranked
RandomTop-Ranked
Handpicked
Handpicked
Handpicked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Handpicked
Handpicked
Top-Ranked
Top-Ranked
RandomRandom
Handpicked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
RandomRandomRandomRandomRandomRandomRandom

∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗

∗
∗

∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗

∗

∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗

Other PII

used

FinancialofferedAddressData∗ Specific purpose ∗ Always required

∗
∗
∗
∗
∗

∗

∗
∗

∗
∗
∗
∗
∗
∗

∗
∗

∗

∗
∗

∗
∗
∗
∗
∗

∗
∗

∗

∗

∗

∗
∗

∗
∗
∗

∗
∗

∗
∗

∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗
∗

∗
∗

∗
∗
∗

∗
∗
∗

∗
∗
∗
∗
∗

Contact
Email
Email
Email
Email
Email
Email
Email
Email
Email
Contact Form
Email
Email
Contact Form
Reset FormContact Form
Email
Email
Reset Form
Email
Email
Email
Email
Email
Email
Email
Email
Reset Form
EmailContact FormContact Form
EmailContact Form
Live Chat
Contact Form
Email
Email
Live Chat
Live Chat
Reset Form
Reset Form
Live Chat
Reset Form
Email
Email
Email
Contact Form
Not Possible
Contact FormContact Form
Not Possible
Email
Email
Not Possible
Not Possible
Not Possible
Live Chat
Contact Form
Not Possible
EmailContact FormContact Form
Reset Form
Reset Form
Reset Form
Reset Form
EmailReset FormContact Form
EmailReset Form
Email

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

Amft et al.

Table 7: List of websites skipped during our recovery experience described in Section 3.2, including their approximated Tranco
rank, category, sample group and reason for skipping it.
ID

Tranco Range

Category

SampleSkip Stage

Reason

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72

1,001-5,000
1,001-5,000
1,001-5,000
50,001-200,000
1,001-5,000
10,001-50,000
50,001-200,000<1,000<1,000>200,000
10,001-50,000<1,000<1,000<1,000<1,000<1,000<1,000<1,000<1,000<1,000<1,000<1,00050,001-200,000
1,001-5,000
1,001-5,000
1,001-5,000
10,001-50,000
10,001-50,000<1,000<1,000<1,000<1,000<1,000<1,000<1,000<1,000<1,000<1,000<1,000<1,000>200,000>200,000>200,000
10,001-50,000
10,001-50,000
10,001-50,000
10,001-50,000
10,001-50,000<1,000<1,000<1,000<1,000<1,000<1,000<1,00010,001-50,000
5,001-10,000
50,001-200,000
50,001-200,000
50,001-200,000
50,001-200,000
50,001-200,000
50,001-200,000
50,001-200,000<1,000<1,000>200,000
10,001-50,000
5,001-10,000<1,000<1,000<1,000Developers
Investing
Banking
Betting
Security
GamingDevelopers
SocialSocialCryptocurrencies
Developers
Utilities
Entertainment
Backup & Sync
Communication
Email
Payment
Payment
Backup & Sync
Backup & Sync
Task Management
IoT
GamingRetailGamingPayment
Cryptocurrencies
Other
Hosting & VPS
IoT
SocialSocialDevelopers
RetailEmailEntertainment
GamingGaming
Domains
Developers
Cloud Computing
GamingCryptocurrencies
Developers
GamingMarketing & Analytics
Payment
Developers
Communication
SocialCreativity
Other
RetailCloud Computing
Communication
InvestingUtilities
Marketing & Analytics
Hosting & VPS
Banking
Communication
Email
Developers
Banking
Cloud Computing
Other
Utilities
Remote Access
IoT
Remote AccessSocialCommunication

RandomRandomRandomRandomRandomRandomRandom
Handpicked
Top-Ranked
Random
Handpicked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
RandomRandomTop-Ranked
RandomRandomRandomRandomRandomRandomRandomRandomRandomTop-Ranked
Handpicked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
RandomRandomRandom
Handpicked
Handpicked
RandomRandomRandomTop-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
RandomRandomRandomRandomRandomRandomRandomRandomRandomRandomRandomRandomRandom
Handpicked
Top-Ranked
RandomTop-Ranked

Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
MFA Setup
MFA Setup
Registration
Registration
Registration
Registration
Registration
MFA Setup
MFA Setup
MFA Setup
MFA Setup
MFA Setup
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration

Blocked (eSIM usage)
Blocked (Location)
Blocked (Location)
Blocked (Location)
Blocked (unknown reason)
Blocked (unknown reason)
Blocked (unknown reason)
Blocked (unknown reason)
Blocked (unknown reason)
Blocked (unknown reason)
Domain already within sample.
Domain already within sample.
Domain already within sample.
Domain already within sample.
Domain already within sample.
Domain already within sample.
Domain already within sample.
Domain already within sample.
Domain already within sample.
Domain already within sample.
Domain already within sample.
Domain already within sample.
Maintained by volunteers
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
Method prohibited by ToS.
No MFA on base website.
No MFA on base website.
No MFA on base website.
No MFA on base website.
No MFA on base website.
No MFA on base website.
No MFA on base website.
No MFA on base website.
No MFA on base website.
No MFA on base website.
No MFA on base website.
No MFA on base website.
Requirements (contract)
Requirements (contract)
Requirements (contract)
Requirements (contract)
Requirements (contract)
Requirements (contract)
Requirements (contract)
Requirements (contract)
Requirements (contract)
Requirements (contract)
Requirements (contract)
Requirements (contract)
Requirements (device ownership)
Requirements (device ownership)
Requirements (device ownership)
Requirements (device ownership)
Requirements (device ownership)

An Evaluation of the Security and Usability of Multi-Factor Authentication Recovery Deployments

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

Table 7: List of websites skipped during our recovery experience continued.
ID

Tranco Range

Category

SampleSkip Stage

Reason

73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124

10,001-50,000
50,001-200,000<1,0001,001-5,000
10,001-50,000
10,001-50,000
10,001-50,000
10,001-50,000
10,001-50,000
5,001-10,000
50,001-200,000
50,001-200,000
50,001-200,000>200,000
50,001-200,000<1,0001,001-5,000<1,000<1,000<1,000<1,00010,001-50,000
50,001-200,000<1,000<1,000<1,000<1,000>200,000
1,001-5,000
1,001-5,000
10,001-50,000
10,001-50,000
10,001-50,000
10,001-50,000
10,001-50,000
10,001-50,000
5,001-10,000
50,001-200,000
50,001-200,000
50,001-200,000
50,001-200,000<1,000<1,000<1,000<1,000<1,000>200,000>200,000>200,000
50,001-200,000
50,001-200,000
50,001-200,000

Finance
IoT
Payment
Banking
Banking
Banking
Finance
Investing
Banking
Banking
Investing
Banking
Investing
Cryptocurrencies
Banking
Banking
Universities
Universities
Universities
Universities
Universities
Finance
VPN Providers
Cloud Computing
Cloud Computing
Cloud Computing
Hosting & VPS
Investing
Hosting & VPS
Other
Task Management
Finance
Other
Other
Security
Developers
HealthHosting & VPS
Other
Cloud Computing
Security
FinanceCommunication
Domains
Other
Other
Hosting & VPS
Backup & Sync
Other
Cloud Computing
Backup & Sync
HealthRandomRandomTop-Ranked
RandomRandomRandomRandomRandomRandomRandom
Handpicked
RandomRandomRandomRandomTop-Ranked
RandomRandomTop-Ranked
Top-Ranked
Top-Ranked
Handpicked
Random
Handpicked
Top-Ranked
Top-Ranked
Top-Ranked
Random
Handpicked
RandomRandom
Handpicked
Handpicked
RandomRandomRandomRandomRandomRandomRandomRandomTop-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
Top-Ranked
RandomRandomRandomRandomRandomRandomRegistration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Registration
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice
Website Choice

Requirements (Foreign Address)
Requirements (Foreign Address)
Requirements (Foreign Phone)
Requirements (ID)
Requirements (ID)
Requirements (ID)
Requirements (ID)
Requirements (ID)
Requirements (ID)
Requirements (ID)
Requirements (ID)
Requirements (ID)
Requirements (ID)
Requirements (ID)
Requirements (ID/payment)
Requirements (ID/payment)
Requirements (membership)
Requirements (membership)
Requirements (membership)
Requirements (membership)
Requirements (membership)
Requirements (payment)
Requirements (payment)
Requirements (payment)
Requirements (payment)
Requirements (payment)
Requirements (payment)
Requirements (payment)
Requirements (payments)
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Time-limited free trial.
Website not available.
Website not available.
Website not available.

CCS ’23, November 26–30, 2023, Copenhagen, DenmarkC

ADDITIONAL FIGURES

Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009

Amft et al.

An Evaluation of the Security and Usability of Multi-Factor Authentication Recovery Deployments

CCS ’23, November 26–30, 2023, Copenhagen, Denmark

Websites

MFA Methods

RecoveryHandpicked(N=16)

Hardware Token
(N=1)

Lost
(N=30)

1
2
12
1

Random(N=27)

Email
(N=6)

6
Mobile App(N=49)

2

21

20

3

5

Top-Ranked(N=28)

1

2

17

25

SMS
(N=15)

91
5

9Figure 5: Illustration of our website sample, configured MFA method and recovery result.

No Answer(N=4)

Recovered(N=37)CCS ’23, November 26–30, 2023, Copenhagen, Denmark

Recovery Method Present per Category
Backup & Sync 53.3 20.0 23.3 10.0 16.7 0.0 10.0 6.7 0.0 0.0 20.0 0.0 3.3 0.0 3.3 10.0 0.0

Amft et al.

100

Banking 2.2 34.6 0.0 6.6 1.5 0.7 1.5 0.0 5.1 0.0 51.5 0.0 0.0 0.7 0.0 11.0 0.0
Cloud Computing 51.4 62.9 22.9 17.1 2.9 8.6 5.7 2.9 5.7 0.0 8.6 0.0 0.0 8.6 0.0 2.9 0.0
Communication 50.0 32.0 32.0 12.0 6.0 14.0 6.0 6.0 6.0 0.0 10.0 0.0 0.0 2.0 2.0 2.0 0.0
Cryptocurrencies 24.5 44.5 0.0 4.5 0.0 2.7 1.8 10.0 1.8 20.0 12.7 4.5 22.7 0.0 0.9 6.4 0.0
Developers 61.4 25.7 10.9 8.9 3.0 3.0 1.0 0.0 0.0 0.0 27.7 0.0 0.0 1.0 0.0 3.0 0.080

Domains 44.1 54.2 5.1 6.8 1.7 8.5 0.0 5.1 1.7 0.0 13.6 0.0 10.2 0.0 0.0 6.8 0.0
Education 8.3 50.0 0.0 0.0 0.0 0.0 8.3 0.0 0.0 0.0 41.7 0.0 0.0 0.0 0.0 0.0 0.0
Email 60.7 35.7 17.9 14.3 3.6 17.9 3.6 7.1 0.0 0.0 7.1 0.0 3.6 3.6 0.0 10.7 0.0
Finance 18.5 33.3 11.1 13.0 0.0 1.9 7.4 5.6 1.9 7.4 31.5 1.9 0.0 0.0 1.9 3.7 0.0
Gaming 37.7 52.8 1.9 5.7 0.0 11.3 3.8 1.9 0.0 0.0 18.9 0.0 1.9 1.9 0.0 7.5 1.9

60

Government 11.8 35.3 0.0 17.6 0.0 0.0 0.0 0.0 5.9 0.0 41.2 0.0 0.0 0.0 0.0 0.0 0.0
Health 19.0 42.9 14.3 9.5 4.8 0.0 19.0 4.8 0.0 0.0 23.8 0.0 0.0 0.0 0.0 9.5 0.0
Hosting & VPS 43.2 50.0 2.7 9.5 0.0 13.5 2.7 1.4 1.4 0.0 21.6 0.0 6.8 0.0 4.1 6.8 0.0
Identity Management 45.0 15.0 20.0 40.0 10.0 15.0 0.0 0.0 5.0 0.0 20.0 0.0 0.0 0.0 0.0 5.0 0.0
Investing 25.5 51.0 0.0 9.8 0.0 3.9 2.0 0.0 2.0 3.9 31.4 0.0 0.0 3.9 2.0 2.0 0.0
IoT 47.1 41.2 5.9 5.9 0.0 5.9 5.9 0.0 0.0 0.0 17.6 0.0 0.0 0.0 0.0 5.9 0.0

40Marketing & Analytics 42.4 47.5 25.4 8.5 0.0 6.8 5.1 0.0 0.0 0.0 22.0 0.0 0.0 0.0 0.0 3.4 0.0
Other 39.0 42.7 14.6 13.4 2.4 9.8 4.9 3.7 1.2 1.2 17.1 0.0 2.4 2.4 2.4 1.2 0.0
Payment 26.5 44.1 17.6 17.6 11.8 5.9 2.9 11.8 5.9 0.0 32.4 0.0 0.0 0.0 0.0 14.7 0.0
Remote Access 56.2 18.8 12.5 6.2 0.0 6.2 6.2 0.0 0.0 0.0 31.2 0.0 0.0 0.0 0.0 6.2 0.0
Retail 40.0 44.0 0.0 20.0 8.0 4.0 0.0 8.0 0.0 4.0 28.0 0.0 0.0 0.0 0.0 4.0 0.0

20

Security 39.5 28.9 23.7 7.9 2.6 5.3 5.3 0.0 0.0 0.0 15.8 0.0 5.3 0.0 5.3 2.6 0.0
Social 53.6 28.6 3.6 10.7 10.7 3.6 7.1 3.6 0.0 0.0 25.0 0.0 0.0 0.0 0.0 7.1 0.0
Task Management 45.0 10.0 20.0 0.0 0.0 10.0 0.0 0.0 0.0 0.0 25.0 5.0 0.0 0.0 0.0 0.0 0.0
Universities 14.6 0.0 82.9 56.1 0.0 4.9 2.4 4.9 0.0 0.0 2.4 2.4 0.0 0.0 2.4 22.0 0.00Co
nta Bact ckuS
Co ervi p Co
Ad ntac ce/W desdit t L eb
Ba iona ocal site
ck l M Ad
up F mSM A M in
S/P ethTr hone od
us
Ac
te Ca
co
un Ba d De llt R ck vic
ec up eoE
Co A very mai
nta cc Se l
ct ess rvi
wi N cet
No h Ph ot Lo
thi oto st
He ng M Pro
lp en ofNo tiot A nec d
Ini cesi
Pa tial ble
Se sswo Secr
cu rd etrit R
y e
Ot Que set
he stir R one s
No cov
Re erycovery

Utilities 15.4 23.1 11.5 7.7 7.7 11.5 7.7 7.7 0.0 0.0 53.8 3.8 3.8 0.0 0.0 0.0 0.0

Figure 6: Presence of MFA recovery methods within the initial 1,303 websites over different categories on 2fa.directory in %. As
websites can offer more than one recovery option, the numbers may not add up to 100%.