27 Years and 81 Million Opportunities Later:
Investigating the Use of Email Encryption for an
Entire University
Christian Stransky∗ , Oliver Wiese† , Volker Roth† , Yasemin Acar

‡ , and Sascha Fahl∗§

∗ Leibniz University Hannover, stransky@sec.uni-hannover.de† Freie Universität Berlin, {oliver.wiese,volker.roth}@fu-berlin.de‡ Max Planck Institute for Security and Privacy, yasemin.acar@mpi-sp.org§ CISPA Helmholtz-Center for Information Security, sascha.fahl@cispa.de

Abstract—Email is one of the main communication tools andhas seen significant adoption in the past decades. However, emails
are sent in plain text by default and allow attackers easy access.
Users can protect their emails by end-to-end encrypting themusing tools such as S/MIME or PGP.
Although PGP had already been introduced in 1991, it is
a commonly held belief that email encryption is a niche toolthat has not seen widespread adoption to date. Previous user
studies identified ample usability issues with email encryption
such as key management and user interface challenges, which
likely contribute to the limited success of email encryption.
However, so far ground truth based on longitudinal field datais missing in the literature. Towards filling this gap, we measure
the use of email encryption based on 27 years of data for 37,089
users at a large university. While attending to ethical and data
privacy concerns, we were able to analyze the use of S/MIME
and PGP in 81,612,595 emails.
We found that only 5.46% of all users ever used S/MIME or
PGP. This led to 0.06% encrypted and 2.8% signed emails. Users
were more likely to use S/MIME than PGP by a factor of six.
We saw that using multiple email clients had a negative impact
on signing as well as encrypting emails and that only 3.36% of
all emails between S/MIME users who had previously exchanged
certificates were encrypted on average.
Our results imply that the adoption of email encryption isindeed very low and that key management challenges negatively
impact even users who have set up S/MIME or PGP previously.

I. I NTRODUCTION
Email is one of the major online communication tools. As
of February 2021, there are more than 4 billion email users
worldwide sending and receiving over 300 billion emails per
day [41]. While email is used for all kinds of information including the most sensitive kinds such as trade secrets, accountcredentials, and health data, regular email is not encrypted andallows network attackers and service providers unauthorizedaccess. This is not for a lack of tools. Both S/MIME [14]and PGP [46] were introduced almost 30 years ago with thegoal to provide end-to-end encryption for email. However,in contrast to modern messaging tools such as Signal [37]or WhatsApp [43] that implement end-to-end encryption bydefault, S/MIME and PGP require a complex manual setup
by users. Consequently, previous work has shown that using

email encryption correctly and securely is challenging formany users [20], [32], [34]–[36], [45]. They struggle withsetting up and configuring encryption keys, distributing them,
managing keys on multiple devices, and revoking them. Thesefindings, already anticipated by Davis [12], are corroborated
by public reports of failed PGP use. For example, it took
Edward Snowden and the journalist Glenn Greenwald a few
months and serious effort to set up PGP for email in order tocommunicate securely [28]. Hence, it is commonly believedin the security community that end-to-end encrypted email
is not widely used, mostly because of lacking usability andawareness issues identified in a multitude of user studies in
the past 22 years (cf. [11], [20]–[22], [30], [34], [36]). To thebest of our knowledge, our work is the first scientific collectionand evaluation of ground truth on the adoption of end-to-end
email encryption. Our work is mainly motivated as follows:
Ground Truth. We aim to confirm the security community’s
anecdotal knowledge about the low adoption of end-to-end
email encryption and provide ground truth based on field data.
Our longitudinal field data can help motivate future work to
improve the adoption of end-to-end encryption for email.
Method Extension. We extend the toolbox of the past 22
years of email encryption research that was initiated with theseminal paper “Why Johnny Can’t Encrypt” [45] at USENIX
Security’99 that is mostly based on laboratory experiments
and self-reporting studies: In this work, we investigate a large
dataset including millions of data points of thousands of users
and years of their email data.
Validate Results from Previous Work and Investigate Underexplored Challenges. We confirm findings from previous
work (e.g. [1], [26], [27], [30]) obtained by other methods
including smaller-scale interviews, surveys, and controlled experiments. Additionally, we also investigate further challenges
that require large scale field data.
Motivated by the above, we make the following contributions in the course of this work:
Data Collection Pipeline. In collaboration with our dataprotection officer, university staff council, and the technicalstaff of the university IT department, we developed and tested

a reproducible and privacy friendly data collection pipelinethat allows to analyze large amounts of email data with afocus on S/MIME and PGP usage (cf. Section IV-A). The
data collection pipeline is part of our replication package.
We aim to encourage other institutions to investigate theiradoption of S/MIME and PGP to contribute to an even better
understanding of the email encryption ecosystem.
Adoption of Email Encryption at a Large University.
We provide a detailed evaluation of the adoption of email
encryption at our university in the past 27 years. In ourevaluation, we focus on the use of S/MIME and PGP for
37,089 total email accounts. Our investigation of 81,612,595
emails found that 2.8% of them were digitally signed and0.06% were encrypted. We found that only 5.46% of our users
ever used S/MIME or PGP and that S/MIME was more widelyused than PGP. However, PGP was the more popular email
encryption tool amongst researchers.
Use of S/MIME and PGP. We provide a detailed overview of
S/MIME certificates and PGP keys in our dataset and find that
RSA is the most widely used key algorithm, employing 2048
bits keys most often for S/MIME. PGP keys used 4096 bits
most often, although newer PGP keys used less secure 2048
bits. We find that more than one third of all PGP keys did nothave an expiration date set making revocation unnecessarily
complicated and Deutsche Telekom to be the root CA for
64.95% of all S/MIME certificates.
User Interaction Challenges including Key Management.
We report on an investigation of user interaction challenges
that previous work identified in user studies. Most interestingly, we focus on key management issues during key
distribution, multi device use, and key rollover. We find thateven after exchanging public keys, only 3.36% of all emails
between S/MIME users were encrypted on average. The useof multiple email clients had a negative impact on the amountof signed and encrypted emails. While most S/MIME and
PGP users renewed their keys in time, 11.5% of S/MIME key
rollovers occurred after the keys’ expiration.
Overall, our results confirm the pessimistic assessment of
the security community: Although our university provides all
researchers, staff, and students with free access to S/MIME
certificates, only very few make use of them and only a negligible amount of emails was encrypted or signed. Our findings
also support results from previous user studies and illustrate
additional challenges. Management of email encryption keys
is hard and distributing keys, using multiple email clients, or
having to renew keys complicates matters.
The rest of the paper1 is organized as follows: In Section II
we provide information on S/MIME, PGP, and our university’s
S/MIME certificate authority. We provide an overview ofrelated work and contextualize our contributions in Section III.
In Section IV, we describe our methodology by providing
details for our data collection pipeline, discussing ethical and
1 Findour companion website at: https://publications.teamusec.de/2022oakland-email/data privacy implications of our work, illustrating limitations,and summarizing the replication package. In Section V, we
provide detailed results of our evaluation, discuss their implications in Section VI, and conclude the paper in Section VII.
II. BACKGROUND
In this section, we provide background information on
OpenPGP, S/MIME, and the email ecosystem of our university
including its S/MIME certificate authority.
A. OpenPGP
OpenPGP2 is an encryption standard (cf. [10], [15]) which
is used for email encryption and digital signatures. PGP is
an open source project and was first standardized in 1996.
In the first standardization, PGP messages were added to thetext body (named: PGP Inline) of an email. Later versions
introduced a separate MIME type for PGP messages (named:
PGP MIME). Over time, new algorithms have been added,
including the Camellia and ECDSA cryptography algorithms.
PGP supports the use of key servers for public key exchange. Users can search these servers for keys for given
email addresses. However, keys may also be exchanged byattaching public keys to emails. Additionally, several email
clients, like K9 on Android or Thunderbird using the Enigma
plugin, support hidden key exchange by adding public keys toemail headers. This feature has been standardized and further
developed by the open source project Autocrypt3 since 2016.
In contrast to centralised trust infrastructures known fromthe web PKI or S/MIME, PGP relies on the Web of Trustto verify identities. In the web of trust approach, users sign
each other’s key when meeting other PGP users in person.
Therefore, users can trust a new key if another trusted key
previously signed the new key, relying on a decentralized trust
chain.
B. S/MIME
Secure/Multipurpose Internet Mail Extensions (S/MIME) isa standard to encrypt and sign emails. It was first introduced in1998 (RFC2311 [15]) and has constantly been improved sincethen. S/MIME utilizes a Public Key Infrastructure to verify
certificates and as such has mostly been used in corporate
environments, where a certificate authority (CA) is deployed or
third party CAs are utilized to issue certificates to employees.
It has been widely supported out-of-the box without the needfor third party plugins in commercially used email clients like
Outlook 98 and higher or Thunderbird.
C. Email Ecosystem at our University
Email at our university is a centralized service. The university’s computing center provides email accounts for all
administrative staff members, for all students as well as faculties, departments, and research groups. Overall, our university
offers 90 different study subjects reaching from engineering
to humanities, and has about 30,000 students and 5,000
2 Abbreviated as PGP in the paper
3 cf. https://autocrypt.org

staff members. Faculties, departments and research groups areorganized in decentralized units, i.e. each faculty, and mostdepartments and research groups have their own subdomain(e.g., sec.uni-hannover.de for the information security researchgroup) for their email. Users can access their email accounts
either through a web interface or dedicated email clients using
the university’s POP3, or IMAP and SMTP servers.
D. University Certificate Authority / Registration Authority
Our university is part of the public key infrastructure of thecommunications network for science and research in Germany(DFN).4 The university’s computing center provides a registration authority for the DFN CA to issue certificates for email
end-to-end encryption and signing, server authentication for
TLS, and document signing for its scientific and administrative
staff and all students (DFN-PKI).5 Certificate signing andrevocation is processed through the DFN CA.
Certificate Policies. All university employees and students
are eligible to obtain S/MIME certificates. However, certificate
use is neither officially endorsed, nor are issuances automatically triggered. Individual work groups may informallyencourage certificate use. While the CA also provides server
certificates (e.g., for TLS), our work focuses on user certificates for email encryption/signing.
User Certificates. To apply for a certificate, eligible staff andstudents can apply online, receive a certificate signing request,
make an appointment with the registration authority, show
up in person, present a proof of identity, and then receivea valid certificate. This process is comparably complicated. In
contrast, creation of a student ID that can be used to accessfree transport and student discounts, and is used as proof ofidentity in exams, does not require in-person interaction. The
process is also not embedded in any other existing onboarding
process at the university. New certificates for the same user canbe issued without another identification if the last identification
is not older than 39 months.
Certificate Signing Request Process. Users can generate acertificate signing request (CSR) for email certificates using
a web application entering their personal details and a revocation pin. The web application generates a CSR, saves itin the browser certificate store, and asks the user to enter apassphrase which protects the CSR’s private key.
Certificate Revocation Process. The CA provides a web
interface for certificate revocation. To revoke certificates, users
have to enter their certificate’s serial number and a revocation
PIN.
Certificate Expiration. Once 30 days and a second time 15days before the user certificate expires, the DFN sends users
an expiration warning via email and encourages users to renew
the certificate.

History and Milestones. Our university began to issue
S/MIME certificates in 2004. The first root certificate used tosign the original CA (G1) through the DFN expired in 2019.
Starting from 2017, certificates were issued using a new root
CA (G2). Additionally, 479 certificates issued using the G1
CA had to be replaced with new certificates issued by theG2 root CA. Figure 10 in Appendix C illustrates the issued
certificates per year. In the first years, the amount of server
and user certificates is roughly equal. However, the number of
S/MIME certificates is only slowly increasing compared to thenumber of server certificates. We can see a drop in requested
certificates in 2020, probably due to the COVID-19 pandemic.
III. R ELATED W ORK
We discuss related work in two key areas and put our work
into context: User studies for end-to-end encryption with end
users and email field studies.
A. End-to-End Encryption Studies for End Users
The usability of end-to-end encryption has been a research
focus for decades. With their seminal work, Whitten andTygar [45] set this line of research off in 1999 when theyevaluated the usability of PGP and identified challenges for
end-users in a lab study. In their qualitative user study,
participants had trouble with key management, specifically key
exchange with other participants. One participant forgot the
password for their key pair and thus had to generate a newone. Another participant was unable to encrypt an email andothers were unable to decrypt emails. Following Whitten andTygar, Garfinkel and Miller [20] and Roth [29] hypothesizedthat some of the challenges could be overcome by simplifying
the key verification process. Their solutions were based onomitting the verification of keys by third parties and instead
using a trust on first use (TOFU) approach. Garfinkel andMiller tested their approach on lay users and found that colorcoding messages depending on their signature status makesusers significantly less susceptible to social engineering attacks
overall. Garfinkel et al. surveyed 470 merchants who received
digitally-signed VAT invoices from Amazon and found thatmerchants should send signed emails by default as the passive
exposure seems to increase acceptance and trust [19], [21].
Ruoti et al. studied different self-designed and publicly available encryption tools [31]–[33], [35] to improve the usabilityof email encryption in their laboratory. Their research interests
were key management, key distribution, and automatically vs.
manually enabled encryption. In several laboratory studies by
Ruoti et al. [31], [32], [35], participants rated at least one tool
as usable and indicated interest in secure email, but did notknow when or how they wanted to use it. In another recent
journal article, Ruoti et al. came to the conclusion that secureand usable systems so far had only been tested in short-term
studies and future research should investigate long term usability and adoptability of secure email systems [30]. Atwater
et al. and Lerner et el. proposed clients for PGP similar to
Keybase6 to study how to simplify key distribution [5], [25].

4 https://www.dfn.de/en/
5 https://www.pki.dfn.de/ueberblick-dfn-pki/ (german)

6 https://keybase.io/

They proposed to upload users’ public keys to a website andconfirm ownership such that they can retrieve emails fromthe corresponding email address. In a lab study, Atwater etal. found that such a key distribution mechanism enabled
participants to send more encrypted emails and had improved
usability for (webmail) users. Lerner et al. compared their tool
called Confidante with Mailvelope7 and showed that their tool
reduces the error potential [25]. In a lab study, all participantsin the Mailvelope group struggled to import public keys or toshare their own public key. In the Confidante group, three outof nine participants struggled to import public keys. However,
all of them managed to share their public keys successfully.
Bai et al. proposed encryption prototypes to study user flows
of different key management approaches [8], [9]. In an interview study, participants preferred to register their publickeys on a webpage and automatically retrieve public keys ofcommunication partners from the webpage over manual key
management. Fahl et al. examined different usability aspects
for Facebook message encryption mechanisms and found that
automatic key management and key recovery capabilities areimportant for adoption [16]. McGregor et al. reported thatcooperating journalists used PGP to encrypt their emails wheninvestigating the Panama Papers [27]. However, journalists
also identified obstacles when using encryption with multiple
devices. Consequently, they used secure messengers such as
Signal instead of PGP on their smartphones. Gaw et al.
interviewed nine employees from the same company and foundthat users flagged encrypted mails as urgent and found thoseto be annoying when used for all messages [22]. They argued
that understanding of social factors is important for adoption.
In a combined lab and field study, Mauriés et al. participants
struggled with Enigmail for Thunderbird and the Mailvelopebrowser plugin [26]. Enigmail users needed help to setup thetool on their computer. The setup process in Mailevelope was
unclear. One participant struggled to import a public key andsend an encrypted email.
In addition to email, mobile messaging apps including
Signal, Threema and WhatsApp, made end-to-end encryption
available for the masses. WhatsApp, for example, introduced
end-to-end encrypted messages for all its users by default in2016 [43]. In an interview study, Abu-Salma et al. identified
blockers and barriers for the adoption of end-to-end encryption
including incompatible tooling and misconceptions of end-toend encryption features [2]. They argue that usability is not
the primary obstacle and that fragmented userbases or a lackof multi-device support significantly contribute to the nonadoption of end-to-end encryption. In a different study, AbuSalma et al. further explored users’ mental models and foundmisconceptions about security properties of messengers [1].
They argue that adoption is no longer the main challenge
for end-to-end encryption tools but that people instead switch
to non secure communication tools and need assistance inchoosing the right one for sensitive information. Stransky et
al. confirmed these findings in an online study with WhatsApp
7 Mailvelope is a browser plugin for PGP: https://www.mailvelope.com/

users. They found that security perceptions of end-to-end
encryption in mobile messaging apps heavily depended on
the reputation and expectations of an app provider, whilevisualizing encryption has only limited impact on perceived
security [38]. Similarily, Akgul et al. found that participants
noticed educational messages and that they improved understanding of security concepts when they are used in isolation.
However, when those messages were implemented in a realistic environment, they could not find significant improvementsin the mental models of end-to-end encryption of users [3].
Overall, previous work primarily focused on identifying
blockers and barriers to adopting end-to-end encryption foremail or mobile messaging apps and studied alternatives toor extensions of existing approaches using laboratory andinterview studies. In contrast, we aim to extend the toolboxof end-to-end encryption research and provide ground truth
based on longitudinal field data by evaluating a large dataset
including millions of data points of thousands of users andyears of their email data.
B. Email Field Studies
In addition to the user studies discussed above, researchers
performed multiple field studies on the use of email. In 1996,
Whittaker and Sidner analyzed the mailboxes of 18 NotesMailusers containing 2,482 emails. They postulated email overload
and studied how users handle a mass of emails [44]. In 2006,
Fisher et al. revisited this analysis with a sample of 600
mailboxes containing 28,660 emails [17]. They analyzed users’
email sorting strategies, especially with respect to dealing with
the increased volume of emails, and postulated that large folders would make email retrieval hard. Alrashed et al. studied asample of anonymized email logs from Outlook Web Accessover a four month period, containing about 800 million actions[4]. They aimed to understand how users handle incoming
emails. They find that most emails have a short lifetime andthat deleting email is the most common action on messages
users interacted with once. Avigdor-Elgrabli et al. analyzed
a sample of donated mailboxes of a major email service
provider, containing about 5 million emails [7]. They usedmachine learning techniques to identify relationships between
emails. Roth et al. performed a study with anonymized mailboxes from 17 voluntary users that contained approximately
139,000 mails to investigate which security mechanisms would
be most appropriate for their communication patterns [29].
They argue that for individual non-commercial users, out-ofband verification of keys would be more feasible than relying
on public key certificates issued by third parties.
In addition to the above field studies that focused on email
usability more broadly, researchers investigated the adoptionof security protocols for email. Foster et al. scanned the
Simple Mail Transfer Protocol (SMTP) configurations of about
300,000 major email providers and email generators in March2014 and February 2015, and investigated the behavior of
known email providers [18]. They found that TLS is widelyused and discovered a dramatically low adoption of effective
TLS certificate validation. Durumeric et al. examined log datafor SMTP handshakes of Google’s Gmail service from January
2014 to April 2015 and compared it with a snapshot of SMTP
configs of Alexa top million domains as of April 2015 [13].
The authors examined the distribution and use of TLS andother server-side security mechanisms. They found that the top
mail providers proactively encrypted and authenticated messages. However, these practices had yet to reach widespread
adoption in a long tail of over 700,000 SMTP servers withless secure configurations. Ulrich et al. evaluated the PGP key
database (SKS-Keyserver) in December 2009, examining 2.7
million keys of which 400,000 were expired and 100,000 wererevoked [42].
Overall, the above field studies investigate the adoption andusability of email in a broader context, measure email servers’
security configurations, and conduct small-scale security analyses of email encryption. We extend previous field studies byfocusing on adopting email encryption using a longitudinallarge field dataset.
IV. A NALYZING 27 Y EARS OF E MAIL DATA
Below, we provide detailed information on the data collection and analysis process in our work.
We performed our longitudinal analysis of a large email
dataset in coordination with the technical staff of the university’s IT department, the data protection officer, and the university’s staff council (see Section IV-C for more details). We implemented a data collection pipeline to collect pseudonymized8
metadata for all email accounts, including the use of S/MIME
and PGP (cf. Figure 1) at our institution in the last 27 years. At
no point did we collect email subject or body information to
avoid the disclosure of personally identifiable information (PII)
to the researchers. We also ensured that metadata included
neither email account names nor the departments’ names or
subdomains. We aimed to keep the number of processing
errors low and consistently tested the pipeline with our own
mailboxes until no further processing errors occurred.
The IT department’s technical staff reviewed the pipeline forfunctionality and data protection aspects, and then executed iton the university’s standby backup email server. The backup
server is a hot-standby of the primary mail server, and automatically takes over in case of a failure. Data between bothservers is constantly synchronized and as a result, the backup
is identical to the live data. The backup server retains all email
data, dating back to early 1994.
Figure 1 provides an overview of the nine-step processing
pipeline: We initially started with a local testing environment
on a small sample emailbox created specifically for ourstudy (1); The technical staff reviewed the initial pipeline
and iteratively tested it against the full set of emailboxes of
the researchers and their own emailboxes (2); We exported
the emailboxes to JSON-formatted files (3); We parsed andpseudonymized all emails (4); We performed assertion checks
on every email to ensure that neither the emailaddress nor
the domain was present in any result fields to account for
8 The pseudonymization process is described in Sections IV-A and IV-C.

emailclients writing private data to unexpected places (5);In the case of a succeeding assertion check, we stored theresulting email metadata for further analysis on a secure serverin the university’s computing center (6a); In the event of a
failure, we dropped all email metadata to avoid the leakage ofprivate information (6b); The IT department’s technical staff
transferred pseudonymized results to the authors’ secure cloudstorage (7); We analyzed the pseudonymized results (8).
A. Data Collection Pipeline
The university uses Dovecot9 for their email servers. Dovecot offers an export feature to extract all emails as a JSONformatted file. We implemented our processing pipeline usinga large JSON file per mailbox containing all exported emails as
input. For parallel processing, we used the GNU Parallel [40]tool. On behalf of the researchers, the university’s IT staff
executed the pipeline on the backup email server to makesure raw emails were not exposed to the researchers. Below,
we describe the metadata we collected. In the cases where
we applied pseudonymization, we provide a description of thepseudonymization procedure. Table IV in Appendix B gives
an overview of both general and S/MIME and PGP specificinformation.
General Information. For each email we collected the local
user account, message ID, the sender, and the list of receiver
email addresses. If present, we also collected the lists of carbon
copy and blind carbon copy addresses for outgoing emails.
For pseudonymization, we hashed all of these values using
a secret salt10 and the SHA-256 hash function. We groupedemail users into: Student, Staff, Faculty, NX Unknown11 andExternal. For data protection reasons, we did not collect theexact send and receive dates as well as times of an email but
only the corresponding week. If set, we collected the raw useragent string to identify email client software, operating system, and compute platform if possible. We grouped mailbox
folders into: Inbox, Subfolder of Inbox, Outbox, Subfolderof Outbox, Junk, Trash, and Spam. For further cryptographic
metadata analysis, we stored whether an email was signed
and encrypted or contained Autocrypt headers. Below, we use
the term cryptographic emails for all emails that contained
cryptographic metadata.
Cryptographic Metadata. For all cryptographic emails, wecollected attached cryptographic metadata for S/MIME, PGP
and Autocrypt. Table IV gives an overview of the collected
information and storage formats including pseudonymization.
For S/MIME, we collected the pseudonymized serial number of the leaf certificate, validity start and end date (granularity by week), the signing hash algorithm, the key size
and key type (e.g., 2048 and RSA). We collected the key
usage and extended key usage options (e.g., email signing,
9 https://www.dovecot.org/
10 The secret salt was only accessible to the IT staff and not to theresearchers
11 Used if the email subdomain did not exist anymore, and the original
purpose was unclear when we performed our experiments.

Researchers

2

University IT Services

Verify and Test on own Mail

Develop Data
Collection and
Pseudonymization

Report Errors
Verify and Execute
ExecuteCheck for Errors

1
Develop Analysis

ExecuteLocal Pipeline Test
with dedicated sample

8

Primary
6b

7
ExecuteSecondary

5

4

Transfer

3

Pseudonymization
Assertion failed,
6a whole mail removed

Fig. 1. Illustration of our data collection and evaluation pipeline.

certificate signing, code signing), and the number of valid
email addresses for each certificate. Finally, we collected thecomplete certificate chain, including all metadata for all signing certificates. We did not pseudonymize the serial numbers
for non-leaf certificates.
For PGP, we collected the key type (e.g., public key ora sub-key), the signature algorithm, and key length. For
elliptic curve keys, we collected curve information as well
as pseudonymized key IDs, and creation and expiration dates.
For extended PGP keys, we collected update dates. If a key
included subkeys we also stored the above data for the sub
keys.
The pipeline dropped all data (like email subject, email
content, non-key attachments) that we excluded from ouranalysis.
B. Data Cleaning
We included all 81,647,559 emails and 37,463 email user
accounts at our university from January 1994 to July 2021 in
the initial analysis (cf. Section V-A). However, we excluded
some emails and email user accounts based on the following
procedure:
Processing Errors. Parsing a dataset that spans millions ofemails from over two decades that were generated and sent bymany different email clients and versions poses a significant
challenge. Hence, we were not able to successfully parse
all emails. Our parser failed to parse 0.09% emails in thedataset. Due to privacy restrictions, we were not allowed tofurther investigate root causes of parser failures. However,Appendix A provides more details on S/MIME and PGP
related parsing errors.
Inactive Email Accounts. Our initial data set included 37,463
total user accounts. However, we identified 18,302 inactive

email accounts for which we did not find any sent emails.
Of them, 17,928 email accounts received but did not send
emails and 374 did neither receive nor send emails. Many
students prefer to use their private email accounts instead of
their automatically created university email accounts, leaving
them inactive.
Invalid Dates. We excluded 307,680 (0.38% of our dataset)emails with obviously forged header dates, e.g., year <1994
or > 2021 (after data collection) and emails for which the dateparser failed.
C. Ethical Concerns and Data Privacy
To conduct the large scale measurement study on email
data, our institutions, and specifically the institution where
the data was collected and evaluated, did not require formal
ethical review for this type of study. Therefore, we did not
involve an ethics review committee. However, we followed
our institutions’ guidelines for good scientific practice, which
includes ethical guidelines. Here, the institutions specificallyplace the burden of determination of whether research is
ethical on the respective researchers. We intensively discussed
within and outside our research team to determine possible
concerns with this research project, and whether this project
would be feasible. We concluded that in addition to following
laws and our institutions’ ethics requirements, we should also
follow the de facto ethics standards of the S&P community.
The data used in this study can be described as pseudonymized
data derived from human subjects, as mentioned in the Callfor Papers.
In addition to ethics, we made sure to address all legal
aspects of our research to adhere to strict German privacy
protection laws and the European General Data Protection
Regulation (GDPR). Therefore, we involved the data pro-tection officer and the works committee of the institution
where the data was collected and evaluated, as required by
the German data protection regulations. We developed thedata collection plan jointly with the data protection officer,
with the goal to protect users’ privacy and adhere to thestrict German data protection regulations and the regulationsin the state of Lower Saxony. After more than a year of
multiple discussions and hearings, we agreed on the presented
data collection plan (cf. details in Section IV-A). After the
involved authorities had rigorously assessed the legal situation
based on the GDPR, German data protection laws, and state
law of the involved authorities, we were allowed to analyze
pseudonymized metadata of all users at our institution without
requiring user consent. Additionally, our legal counsel decreedthat the benefit of our research to society outweighed therisks to individuals. We concur with the assessment thatanswering our research questions is beneficial for future endto-end encryption research, which ultimately benefits society,and that there was no harm done to any participants basedon possibly re-identifiable metadata. Importantly, we will not
publish the metadata we collected and only an encrypted copy
will be stored at the university data center for ten years without
access by researchers to follow good scientific practice. As iscommon in research, we only publish aggregate data, and noemail accounts can be re-identified through the publicationof this paper. As part of the joint development of our datacollection plan, we decided to take the following measures
to protect users’ (metadata) privacy and adhere to the GDPR,
German, and state laws:
The involved researchers never had access to raw data.
The data collection pipeline was executed by the university’s IT staff who operate the email servers andhave access to the backup data. They transferred thepseudonymized data to the researchers to a secure server.
• We reduced the amount of data to the absolute minimum
we required to investigate our research questions.
• We used cryptographically secure hash functions withsalts unavailable to the researchers to pseudonymize userdata.
• At all times pseudonymized data (cf. Table IV in the
appendix) was only stored on secured university servers.
• We did not and will not share any data other than theaggregate numbers in the paper with anyone outside theteam of involved researchers.
• We assured the data protection officer and the works
committee that we would not take any actions to deidentify users.
•

Data Set. Our dataset includes email data from January 1994
to July 2021 covering 81,647,559 emails of 37,463 users froma large Germany University with more than 30,000 students
and 5,000 employees. However, the dataset might not becomplete. We could not include emails that users deleted fromtheir accounts; similarly, we did not include emails sent todeleted accounts. Some research groups and departments havetheir own email servers; their emails were also not included
in our dataset. Therefore, the dataset should not be assumed
to include all emails that were sent or received from January
1994 to July 2021 at our university. Additionally, we do notassume that our dataset is representative for all email data
in Germany or globally. Instead, we think our data set mightoverreport the use of email encryption since most email users
in the dataset are highly educated and our university offers
free S/MIME certificates to all email users.
However, we think our dataset is one of the largest and mostvaluable for the type of analysis we perform and our results
are a valuable contribution to the security research community.
Data Analysis. We could not analyse all cryptographic details, e.g., we could not verify digital signatures since we were
not allowed to parse the body content of emails and we could
not extract details for certificates or signatures that were usedin encrypted emails, since this data is also encrypted.
We tested our pipeline thoroughly but still missed someedge cases that inevitably arise in mail software that evolves
over a long time span. While our pipeline was able to process
99.91% of all emails, processing failed for the remaining
0.09%. Errors during S/MIME and PGP parsing were logged
separately. We encountered 1,199 S/MIME and 23,168 PGP
emails where parsing failed (cf. Appendix A for more details).
We deemed this margin of error tolerable compared to the highorganizational costs of refining and repeating the entire process
once more.
Distinction Between Send and Receive. The dataset we
evaluated did not contain information whether an email wassent or received. To still group emails into sent and received
emails, one approach would be to group emails based on thefolder they were stored in. However, this would introduce
challenges such as email clients using different names for sent
folders or users using their own folder names. Therefore, we
decided to identify emails based on multiple parameters. Sent
emails were not allowed to contain a return_path header,since it is added by outgoing mail servers and emails were
only allowed to go through at most one mail relay12 . Most
incoming emails have more than five email relays.

D. Limitations

Mail Client Behaviour. The macOS and iOS clients AppleMail, iPhoneMail and iPadMail generally identify themselvesusing the X-Mailer header in mails, but the copy placed in
the sent folder by these clients does not contain this header.
As a result, these clients could only be correctly detected onreceived emails, their sent mails are included in the ”No User

Like every measurement study, our work comes with several
limitations:

12 One mail relay header is added when the webmail client of the university
is used for sending mails. Regular mail clients do not add this header.

The above data protection precautions ensured the necessary
compromise between user privacy and data utility in order toperform the analyses on which we report in Section V.

Agent” group. The ticket system used by the university data
center automatically deletes mails in its inbox and does notplace a copy in the sent folder and as a result only the answered
tickets are available in our dataset.
E. Replication Package
To improve the replicability of our work, we provide areplication package including the following material: (a) thecomplete processing pipeline consisting of multiple Python
scripts to process and pseudonymize emails from Dovecot
mail servers, (b) the analysis scripts to replicate our results
on different datasets, and (c) the agreement with our data protection officer. Due to the sensitive nature of our measurement
study, we cannot make raw email data available. We hope thisreplication package helps future studies to better compare andposition themselves to our work; and hope others replicate ourwork on different email datasets to improve our community’sunderstanding of the use of email encryption. The replication
package is available on our website at [39].
V. R ESULTS
We provide a detailed analysis of the email corpus wecollected and the adoption of S/MIME and PGP from 1994 toJuly 2021 below.
A. Dataset Summary
In total, we analyzed metadata for 81,647,559 emails from37,089 email accounts. Overall, the university’s email users
exchanged 40,540,140 (49.67%) emails internally.
Figure 2 illustrates the use of email at our university
in the past 27 years. While we found only 350 emails in
1994, we detected an almost exponential growth and found
17,190,472 emails in 2020. This development reflects theenormous relevance of email as a communication tool andis in line with reports on the global use of email13 .
Use of Email Encryption and Signatures. We found
2,334,042 (2.86%) emails that were either encrypted or signed
using S/MIME or PGP. We identified a huge difference
between the use of email encryption and signatures.
46,973 (0.06%) emails were encrypted. 26,105 (55.57%)
emails were encrypted using S/MIME and 20,868 (44.43%)of them were encrypted using PGP. In contrast, 2,287,922(2.8%) emails were signed. 2,040,794 (89.2%) of them weresigned using S/MIME and only 247,128 (10.8%) were signed
using PGP.
Figure 2 illustrates the use of S/MIME and PGP between
1994 and 2020. We found the first S/MIME signed email in1998 and the first S/MIME encrypted email in 1999. The
first PGP signed email appeared in 1994 and the first PGP
encrypted email in 1997.
Key Insights: Dataset.
We saw an exponential growth of the use of email between 1994
and 2020.
• Only 0.06% of emails were encrypted.
•

13 cf. https://www.emailisnotdead.com/

Fig. 2. Rise of email, S/MIME and PGP over time at our university.

••

2.8% of emails were signed.
S/MIME was more widely used than PGP.

B. S/MIME Certificates and PGP Keys
Below, we give an overview of the S/MIME certificates and
PGP keys we found. This includes certificates and keys frominternal and external senders. Overall, we were able to collect
9,765 S/MIME certificates, 3,741 primary PGP keys and 3,840
sub keys (cf. Table II for details).
S/MIME Certificates. All but one certificate that used anelliptic curve encryption algorithm supported the RSA encryption algorithm.
2048 bits was the most widely used RSA key size (91.58%);
5.54% of the RSA keys had 4096 bits. In 237 cases we saw
1024 bits RSA keys; 6 RSA keys had 512 bits. While thelast 512 bits RSA key we found was created in 2010, we saw
2 1024 bits RSA keys created in 2020.
7,472 (76.52%) certificates supported the SHA-256 signature algorithm. However, we also found outdated signaturealgorithms including SHA-1 (2,028; 20.77%) or MD5 (148;
1.52%). Surprisingly, we found 11 certificates issued in 2020
using SHA-1. The last certificate using MD5 was generated
in 2017.
5,194 (53.19%) of all certificates expired in 2020 or earlier.
The mean validity period for S/MIME certificates was 3.13
years (sd= 2.70) ranging overall from a minimum of 4.00
weeks to a maximum of 99.99 years. 6,953 (71.20%) certificates were created between 2015 and 2020 with a peak of
1,654 certificates (16.94% of all S/MIME certificates) in 2019.
Overall, we found 671 different issuer names. However,
1,150 (11.78%) certificates had no issuer. The most prominent issuer was the DFN issuing 3,209 (32.86%) certificates.
622 (6.37%) were issued by our university. Another German
university issued 563 (5.77%) of all S/MIME certificates wefound. In 332 (3.40%) cases, a distinct issuer only signed asingle certificate. 89 (0.91%) issuers signed only two certificates. 137 of them (32.54%) had no root CA. In total, weS/MIME
Sent

PGP
Received

Sent

Total
Received

Sent

Received

SignedEncrypted

SignedEncrypted

SignedEncrypted

SignedEncrypted

Total

356,330

9,358

1,684,464

16,747

69,9508,197

176,983

12,633

16,660,280

64,952,315

ClientThunderbird
Outlook
Ticketsystem
AppleMail
Evolution
Mutt/NeoMutt
Outlook-Express
Claws Mail
iPhone/iPad-Mail
MailMate
Other
No Useragent

231,475
78,7360?1
11,479
0
7
0
?1
2,361
1,270
31,002

6,311
1,675
0
?1
190
29
0
0
?1
8
3
1,142

508,423
258,013311,743
58,752
16,366
50
7,355
145
5,275
3,526
13,397501,419

7,103
3,728
1
2,157
341
31
31
0
104
3
128
3,120

46,407
325
0
?1
232
564
0
1,654
?1
63
2,804
17,901

5,775
22
0
?1
48
1
0
207
?1
2
221
1,921

85,904
5,528
223
20,722
1,387
11,506
2,452
2,918
7
131
8,658
37,547

6,825
52
0
1,408
74
1
16
161
8
3
2,167
1,918

8,206,215
3,102,760
4?1
26,224
6,066
29,065
2,573
?1
5,296
2,572,631
2,708,758

11,875,969
7,785,073
1,259,208
2,268,757
70,482
141,263447,775
21,640723,482
8,364
9,715,649
30,634,653

Operating System2
Windows
Linux
Mac
iOS
Android
WebmailUnknown
No Useragent

167,350
60,827
5,909
1
81
135
91,02531,002

2,183
3,985
170
0
1
3
1,874
1,142

326,747
164,926
88,1445,294
66
831
597,037
501,419

2,555
4,379
2,397
104
0
8
4,184
3,120

38,751
8,797
2,126
0
34
10
2,331
17,901

2,531
3,372
172
0
101
2
98
1,921

66,78623,06924,8497167
322
24,23637,547

2,589
4,191
1,841
8
110
2
1,974
1,918

7,150,676
799,179
340,599695
84,744
2,378,940
3,196,689
2,708,758

10,679,520
1,457,578
3,057,579
726,209120,115
2,425,629
15,851,032
30,634,653

Usergroup
Scientific
Staff
NX Internal
Student
External

237,579
106,434
9,903
1,700
714

3,992
5,318
20
28
0

808,539684,791
121,243
67,813
2,078

8,376
8,155
129
71
16

66,483
2,122
77
1,242
26

6,708
1,134
39
311
5

115,388
49,111
7,153
4,903
428

7,796
3,864
436
536
1

11,776,198
3,265,602
970,160
393,685
72,25639,504,464
17,560,401
4,960,051
2,703,240
224,159

Affected emails

>= 5%

> 4%

> 3.5%> 3%> 2.5%> 2%> 1.5%> 1%> 0.5%<= 0.5%

1 AppleMail and the iOS mail client miss the X-Mailer header in the sent folder. Hence, we identify their client as ”No Useragent“.
2 Not all clients store the operating system as part of the Useragent/X-Mailer field. If not available, we identify them as ”Unknown“.

TABLE I
T OP 10 CLIENTS OF S/MIME AND PGP USERS AT OUR UNIVERSITY.

identified 495 root CAs. 274 of them were only the root of
one certificate. The Deutsche Telekom was the root CA of
6,342 (64.95%) certificates. In contrast, 1,150 (11.78%) wereself-signed and not linked to a CA.

Fig. 3. Distribution of primary PGP keys over time.

PGP Keys. PGP keys can be split into primary and sub keys.
Each primary key can have multiple sub keys. Below, we focus
on primary keys. We found 3,741 primary and 3,840 sub keys.
Primary keys had on average 1.03 sub keys with a maximumof 10.
Most PGP keys used RSA (3,169; 84.71%) and 2048 bits
(928; 24.81% keys) or 4096 bits (2,015; 53.86% keys) keys,

followed by DSA (323; 8.63%) with mostly short key sizes<= 1024 bit (290; 7.75%) for signatures and Elgamal sub
keys for encryption.
PGP key expiration dates can be set and extended by theuser themselves. We saw this in 219 (5.85%) cases witha maximum of 4 extensions. 1,234 (32.99%) keys had no
expiration date, while 1,344 (35.93%) keys were expired. The
average validity period for PGP keys was 4.67 years (sd 3.39)
ranging from zero weeks to 85.82 years.
The majority (276; 85.45%) of DSA keys were created
before or in 2010 and used 1024 bit keys. 278 (86.07%) had
no expiration date set. The last two DSA keys were createdin 2019 with 3072 bit keys.
928 (24.81%) of the RSA keys used 2048, 2,015 (53.86%)4096 bits. Only 6 (0.16%) had 1024 bit keys. 2048 bit was
the most widely used key size (60-70% per year) between
2010 and 2013. 4096 bit was most dominantly used from 2014
and peaking in 2017 with 86.18%. Interestingly, in 2018 thepicture changes again and 2048 and 3072 bit keys appeared
more frequently again with 2048 bits taking back the lead in
2020, although the amount of RSA keys is starting to decline infavor of EdDSA. We attribute this phenomenon to the releaseof Autocrypt in 2018 which uses RSA 2048 bit keys by default
to avoid the 10kB header limit. [6].

# S/MIME

Cryptographic Algorithms1

RSA

DSA

ElGamalValidity Period

EC
ECDH
EdDSA
mean
std. dev.
medianminmaxunlimited

512
1024
2048
3072
4096
8192
768
1024
2048
3072
768
1024
2048
3072
4096
521
256
256
(years)(years)(years)(weeks)(years)
-

9,765
6
237
8,942
32
541
5
1
3.13
2.70
2.99
4.00
99.99
-

# PGP
Primary
Sub
3,741
6
928
208
2,015
5
1
290
17
15
249
4.67
3.39
4.99
0.00
85.82
1,234

3,840
6
950
226
2,078
2
1
2
2
1
61
186
13
53
247
2
4.62
3.68
4.99
0.00
85.82
1,240

1 Key sizes in bits

TABLE IID ETAILS FOR S/MIME CERTIFICATES AND PGP KEYS IN OUR DATASET

We found the first four EdDSA keys in 2017, followed bynine in 2018, 31 in 2019 and 193 in 2020, almost reaching
the number of RSA keys (207). 53 (21.29%) of the EdDSA
keys did not have an expiration date set.
Until 2000, all 33 PGP mails were hashed with MD5.
Between 2004 and 2015, SHA1 was most widely used in
62,273 (80.96%) PGP mails.
We found the first use of SHA2-256 in 2006 with asteady increase until 2016 when it became the most widelyused algorithm with a total of 125,973 mails (68.54% since
2016; 50.22% overall). SHA2-512 first appeared in 2008, hasconstantly been growing since, and has been the second mostused hash algorithm since 2016 (44,859; 25.94% since 2016;20.26% overall). The remaining hash algorithms SHA-224 (7;<0.01%) and RIPE-MD160 (132; 0.05%) were almost nonexistent in our dataset.
Key Insights: S/MIME and PGP.
RSA is the most widely used key algorithm.
A key size of 2048 bits was used most often with S/MIME.
PGP keys used 4096 bits most often, although newer PGP keys usedless secure 2048 bits.
• About one third of the PGP keys did not have an expiration date set.
• The Deutsche Telekom was root CA for 64.95% of all S/MIME
certificates.
•••

C. User Interaction with S/MIME and PGP
Overall, we identified 37,463 unique user accounts including 374 accounts with no emails at all and 18,302 users who

never sent an email. Below, we focus our analysis on theremaining 19,161 active email users who have sent at least
one email. Active users sent 688.93 emails (ranging from 1
to 182,379 emails) and received 3,249.81 (ranging from 0 to
2,881,904 emails) on average.
Based on the subdomain of user accounts’ postboxes weassigned user accounts to the groups scientific, staff, students,
external and NX internal. We assigned administrative staff ofa research group to scientific, assigned subdomains that wereno longer available when we collected and analyzed the data
to NX internal and external to accounts that are hosted by
the university but belong to external research projects. 9,053(24.41%) of our users were in the scientific user group, 2,169(5.85%) in staff and 19,002 (51.23%) were students (cf. Table I
for a distribution of emails for all user groups).
Below, we focus on user behavior. We aim to understandreal world implications of the common usability challenges(cf. [5], [20], [21], [27], [45]) with a focus on:• General use of S/MIME and PGP.
• Use of S/MIME and PGP with multiple clients.
• Distributing S/MIME certificates and PGP keys.
• Long-term S/MIME certificate and PGP key management.
• Leakage of private keys.
General Use of S/MIME and PGP. 94.54% of the active
users never used S/MIME or PGP for sending email. In
contrast, 62.59% of our active users received at least one email
signed or encrypted using PGP or S/MIME.
375 users (1.96% of the active users) sent at least one
encrypted email. While 167 users used PGP exclusively and
159 used S/MIME exclusively, 49 used both. 1,047 users(5.46% of all active users) signed at least one email. 446 users
PGP exclusively, 455 users used S/MIME exclusively, and 146
users used both.
S/MIME users signed 33.58% of their emails on averageafter using S/MIME for the first time but encrypted only 1.26%of their emails on average. PGP users signed 4.90% of theiremails on average but encrypted only 0.96% of their emails.
Breaking down S/MIME and PGP use into user groups, weobserved that staff users used S/MIME to sign their emails
more often than scientific users (3.26% vs 2.02%) and eighttimes more often than students (0.43%) on average. In contrast,
the amount of PGP signed emails is smaller across all usergroups, with the scientific user group being at the top with
0.56%, the staff users with 0.06%, and students with 0.32%
on average.
For more details including the use of encryption, pleaserefer to Table I.
Use of Multiple Email Clients. To identify email clients,
we relied on user agent metadata in emails. We were able todetect email clients for 48,269,184 (59.14%) emails. Hence,
we could detect the email client for 13,951,522 (83.74%)
emails sent by our users, for 14,492 (82.55%) encryptedand 377,377 (88.53%) signed emails. However, 33,343,411(40.86%) emails did not contain user agent information.

The most common client used for signing and encryption
was Thunderbird in combination with Enigmail for PGP
support14 . It was used to send 289,968 (65.35%) S/MIME and
PGP emails in our dataset. 80,758 (18.20%) of the S/MIME
and PGP emails were sent using Outlook. Table I provides
more details on the top 10 email clients.
Using multiple email clients posed additional burden on
the widespread use of S/MIME and PGP since users haveto synchronize their keys and certificates across all clients.
For 8,828 (46.07%) out of 19,161 users, we could detect
multi-client use. Table III provides an overview of multiclient use across all users and S/MIME and PGP users.
Figures 5 and 6 compare how S/MIME and PGP users employ
signatures and encryption between single- and multi-client
users. Our data implies that single-client users sign emails
more frequently than multi-client users. For S/MIME users,
the number of signed emails decreases from median 62.25%
for single-client users to median 1.58% for dual-client andto 1.03% for triple-client users. However, for power users
with more than three clients the number of signed emails
increases to median 17.48%. The majority (74.90%) of multiclient users tend to use one of their clients for email signatures
exclusively and not employ signatures with other clients, while
only 25.10% of the multi-client users used email signatures
across different clients. In contrast, single-client PGP users
only signed 0.36% of their emails on average. However, we
did not find considerable differences between single and multiclient PGP users (cf. Table III for more details).
S/MIME Rendezvous. In addition to multiple clients, thedistribution of S/MIME certificates and PGP keys between
users poses another significant challenge for the adoption ofemail encryption. We focus on S/MIME certificate distribution,
since S/MIME clients automatically attach certificates including public keys to signed emails. However, plain PGP clients15
do not automatically attach their public keys to signed emails
but require users to manually attach public keys or look themup on PGP key servers.
Signed S/MIME emails were the most frequent cryptographic emails in our dataset. Therefore, we were able toinvestigate users’ interactions with S/MIME certificates theyreceived from their communication partners. Without additional effort for key exchange, S/MIME users who received
S/MIME certificates from others could encrypt future emails
to those senders. Below, we report on such behavior of users
in our dataset. 601 users (3.14% of all internal active users)
sent at least one signed email. Overall, we identified 374
rendezvous where both sender and recipient exchanged public
S/MIME keys due to sending signed emails to each other.
Most emails (64.08%) between S/MIME rendezvous partners
were signed. However, only 3.36% of all emails between
rendezvous partners were encrypted on average. Once one
rendezvous partner had sent a first encrypted email, 13.95%
of all following emails were encrypted on average. Figure 4
14 Native PGP support was added to Thunderbird 78 (June 2020)
15 These clients do not have Autocrypt support

illustrates the distribution of signed and encrypted emails
between S/MIME rendezvous partners.

Fig. 4. Distribution of signed and encrypted emails between S/MIME
rendezvous partners. While they signed most of their emails, they encrypted
only few. However, receiving an encrypted email had a positive impact on
encrypting future emails between rendezvous partners.

Long-term S/MIME and PGP Key Management. Below,
we report on the long term use of S/MIME certificates inour dataset. In particular, we focus on the replacement of
outdated certificates. Overall, we identified 680 university
email addresses with at least one S/MIME certificate which
was actively used. 496 (72.94%) of them were valid until
December 2020. 203 (29.85%) of these email addresses had
two or more certificates associated. On average, they used 2.86
certificates – one used up to 27 certificates.

Fig. 5. Distribution of S/MIME signatures and encryption for users with one
or multiple clients.

Since certificates expire and need to be replaced withnew ones, users need to create, set up, and distribute them
ideally very close to the expiration date of the old certificate.
Overall, we found 364 certificate rollovers. In 271 cases,
the new certificates had a longer expiration period than theold certificates. 229 of the certificate rollover events we
detected occurred in time before the old certificates expired.
On average, they happened 55.05 weeks before the old ones
expired. Figure 7 shows that the majority of users often create

S/MIME

PGP

Emails signed1

Emails encrypted1

Emails signed1

Emails encrypted1

clients2

1

2

3

4+

1

2

3

4+

1

2

3

4+

1

2

3

4+

mean
std. dev.
25%
50%
75%
max

53.55
44.99
1.59
62.25
100.00100.00

23.50
35.79
0.14
1.58
45.03
100.00

28.17
36.09
0.13
1.03
57.92
99.48

33.22
35.46
0.10
17.48
67.91
98.61

1.39
7.17
0.00
0.00
0.00
67.59

0.33
1.88
0.00
0.00
0.00
20.00

2.57
11.38
0.00
0.00
0.24
85.85

1.02
2.66
0.00
0.06
0.79
20.17

14.08
27.05
0.08
0.36
10.00
100.00

6.01
18.14
0.04
0.13
0.57
93.75

2.08
8.84
0.03
0.10
0.32
72.61

4.39
13.90
0.03
0.09
0.42
91.23

2.53
12.91
0.00
0.00
0.00
100.00

0.97
2.91
0.00
0.00
0.00
20.53

0.72
2.65
0.00
0.00
0.00
22.58

0.75
1.81
0.00
0.00
0.48
13.52

1 Mean, std. dev, percentiles and max values in %2 Number of clients used by single users.

TABLE IIID ISTRIBUTION OF SIGNED AND ENCRYPTED EMAILS FOR MULTI - CLIENT USERS .

is not necessarily a security issue as long as the passphrase toprotect the key is adequate.
Key Insights: S/MIME and PGP users.
More than 94% of all active users never used S/MIME or PGP.
S/MIME users signed six times more of their emails than PGP users
on average.
• Using two to three different clients decreased the likelihood ofsigning emails by 51.76%.
• On average, less than 3% of all emails between users who had
exchanged S/MIME certificates previously were encrypted.
• Leakage of private keys via email does not seem to pose an issue.
••

VI. D ISCUSSION
Fig. 6. Distribution of PGP signatures and encryption for users with one or
multiple clients.

a new certificate shortly before the previous certificate expires.
However, we also detected 42 certificate rollover events that
occurred after the old certificates expired. S/MIME could
neither send signed nor receive encrypted emails in this time
period. On average, the late rollover events occurred 70.64
weeks after the expiration dates.

Fig. 7. Distribution of the time around certificate expiration for certificate
renewals in weeks. Most certificates were renewed around one year before they
expired due to an expired root CA certificate. The second most certificateswere renewed one week before they expired.

Leakage of Private Keys. Overall, we encountered three
instances of private PGP keys (and their private sub keys)
being sent via email in 2015, 2017, and 2018. All three keys
were sent by the users to themselves. One of those was afreshly created PGP key (i.e., less than one week old). ThisIn this section, we discuss the implications of our results
and recommendations for the future development of email
encryption in five key areas: Limited use of email encryption,use of insecure keys, impact of certain events, challenges ofusing multiple clients, and lack of opportunistic encryption.
Limited Use of Email Encryption. As illustrated in Section V-A, we observed that only a very small fraction of emails
in our dataset was encrypted (0.06%) or signed (2.8%). While
we saw an exponential growth of the use of email overall(cf. Figure 2), the fraction of encrypted emails remained
consistently small. Our results also imply that S/MIME wasmore widely used than PGP for both signing and encrypting
emails. Although these findings do not come unexpected in
the light of previous work [22], [26], [31], [32], [35], we
think they can serve as ground truth to confirm previous
user study results and have value for future development. The
small fraction of email encryption and the fact that the useof email encryption did not grow with the overall use of
emails suggest that both S/MIME and PGP are niche tools thatare mostly used by a small number of security-aware users.
However, the difference between the use of S/MIME and PGP
seems to give grounds for optimism. Our institution provides
an S/MIME infrastructure and encourages its use to improvesecurity and privacy without advertising it aggressively. Thesefindings are in line with results of previous works [1], [2]which find that many users do not use end-to-end encryption
for security reasons but adopt it along with other features.
Therefore, future development should look into adding morevalue to the use of email encryption on top of security andprivacy. For example, more widely accepting digitally signed

emails for administrative processes could increase the valueof email encryption for a broader set of users.
Use of Insecure Keys. For both S/MIME and PGP, wefound only small numbers of insecure or outdated keys (cf.
Table II). The majority of keys used sufficient key sizes (>=2048 bits) for RSA and Elgamal and >= 1024 bits for the
DSA algorithm. However, we found a significant number ofkeys that had no expiration date set or were used after their
expiration date. In particular, most PGP keys were affected(32.99% had no expiration date and 35.93% were expired).
While the widespread adoption of secure algorithms andsufficient key sizes is positive, the prevalence of key expiration
issues is a matter of concern. These findings suggest that
initial key generation by email encryption clients and plugins
works well and supports email users in setting up sufficiently
strong encryption keys. However, they also illustrate that tool
support to prevent the use of insecure and expired keys has
limits. Therefore, future development should look into better
prevention mechanisms that nudge users into only using secureand still valid keys.
Impact of Certain Events. Our longitudinal field data allows
us to shed light on the impact of specific events in time on
the use of email encryption. In the following, we focus on two
events: The Snowden revelations in 2013 and the COVID-19
pandemic in 2020. While this list is not exhaustive, both events
had significant impact on the security community.
In 2013, Edward Snowden leaked the mass-surveillance
program of the NSA and other security agencies [23], [24].
Users of modern digital communication tools were made awareof the significance of end-to-end encryption for information
security. Compared to 2012 (0.035%), the use of email encryption doubled to 0.07% in the following years. While the impactof the revelations on absolute numbers is limited, the use of
email encryption still significantly increased. This incident
illustrates that awareness campaigns may positively affectthe adoption of end-to-end encryption which is in line withan unprecedented growth of Signal users after a WhatsApp
controversy in January 202116 .
During the COVID-19 pandemic, we found that more emails
were exchanged than before. This can be explained by the factthat most of the institution’s staff worked from home and many
administrative processes that had been paper-based before thepandemic were digitalized in 2020. Hence, for all emails wenoted an increased amount of emails sent (39% compared to2019). However, the number of S/MIME-signed emails also
increased by almost 76% from 255,104 to 449,646 compared
to 2019. In contrast, the amount of encrypted emails slightly
dropped from 0.061% in 2019 to 0.050% and 0.051% in 2020
and 2021, respectively. Users might have had problems setting
up and using email encryption from home and accessing theuniversity’s technical support during the pandemic. Similarly,
about one-third fewer certificates were issued to users in 2020
compared to the previous year (cf. Figure 10).
16 cf. https://www.reuters.com/article/us-signal-users-idUSKBN29I27U

Key Management Challenges. We identified multiple key
management challenges. While we found limited use of email
encryption in general, the use of multiple clients posed
additional challenges for S/MIME users in our dataset (cf.
Section V-C) – single client S/MIME users signed more than
60% while users with two clients only signed 1.58% of theiremails, which might be caused by a lack of tool support
for transferring S/MIME setups between clients. Similarly,
we identified challenges for long-term key and certificate
management. 42 out of 364 certificate rollovers occurredmonths after the certificates’ expiration dates, which made it
impossible to produce valid signatures in the meantime. Better
tool support for easier key rollovers could contribute to earlier
certificate updates for a broader set of users. However, we also
identified a positive aspect of key management. Only threeemails had private keys attached, all of which were sent tothe users’ own mailboxes. Hence, in no case was private key
material leaked to unauthorized users. Overall, while our data
implies that key management needs to be improved, private
key leakage through email attachments was not an issue forour users.
Missed Opportunities. In addition to the key management
issues above, we identified the initial distribution and use ofkeys and certificates to be challenging for users. In particular,
using already exchanged public keys between users for further
email encryption was a surprising issue we found. While we
identified mutual public S/MIME key exchanges between 374
users, they only encrypted 13.95% of their future emails. Overall, 318,214 emails in our dataset could have been encrypted
between these users without any additional key exchange.
Here, our data illustrates that an automated mechanism for
more efficient opportunistic encryption similar to Autocrypt
could potentially to help to increase the number of encryptedemails.
VII. C ONCLUSION
In this work, we presented the first analysis of a large corpus
of longitudinal email data for thousands of users at a large
German university. We were able to confirm common beliefs
and results from previous work in the security community:
Only few users used email encryption to secure only a small
fraction of their emails. We identified key management to bechallenging in particular in the context of multiple clients, keyrollovers and key exchange. Based on our evaluation, we makesuggestions for improving email encryption adoption. Overall,
we hope our investigation provides a data driven motivation
for future work to improve both the security and usability ofemail encryption solutions.
ACKNOWLEDGMENT
The authors would like to thank the staff at the Leibniz
University IT Services at Leibniz University Hannover.

R EFERENCES[1] R. Abu-Salma, E. M. Redmiles, B. Ur, and M. Wei, “Exploring User
Mental Models of End-to-End Encrypted Communication Tools,” in 8th
USENIX Workshop on Free and Open Communications on the Internet(FOCI 18). Baltimore, MD: USENIX Association, 2018.
[2] R. Abu-Salma, M. A. Sasse, J. Bonneau, A. Danilova, A. Naiakshina,
and M. Smith, “Obstacles to the Adoption of Secure Communication
Tools,” in 2017 IEEE Symposium on Security and Privacy, SP 2017,
San Jose, CA, USA, May 22-26, 2017, 2017, pp. 137–153.
[3] O. Akgul, W. Bai, S. Das, and M. L. Mazurek, “Evaluating In-Workflow
Messages for Improving Mental Models of End-to-End Encryption,” in30th USENIX Security Symposium (USENIX Security 21). USENIX
Association, Aug. 2021, pp. 447–464. [Online]. Available: https://www.usenix.org/conference/usenixsecurity21/presentation/akgul[4] T. Alrashed, A. H. Awadallah, and S. Dumais, “The Lifetime of Email
Messages: A Large-Scale Analysis of Email Revisitation,” in Proc. 2018
Conference on Human Information Interaction & Retrieval, ser. CHIIR’18. New York, NY, USA: Association for Computing Machinery,
2018, p. 120–129.
[5] E. Atwater, C. Bocovich, U. Hengartner, E. Lank, and I. Goldberg,“Leading Johnny to Water: Designing for Usability and Trust,” inEleventh Symposium On Usable Privacy and Security (SOUPS 2015).
Ottawa: USENIX Association, Jul. 2015, pp. 69–88.
[6] Autocrypt team, “Autocrypt Level 1: Enabling encryption, avoiding
annoyances,” https://autocrypt.org/level1.html.
[7] N. Avigdor-Elgrabli, R. Gelbhart, I. Grabovitch-Zuyev, and A. Raviv,“More than Threads: Identifying Related Email Messages,” in Proceedings of the 27th ACM International Conference on Information
and Knowledge Management, ser. CIKM ’18. New York, NY, USA:
Association for Computing Machinery, 2018, p. 1711–1714.
[8] W. Bai, D. Kim, M. Namara, Y. Qian, P. G. Kelley, and M. L. Mazurek,“Balancing Security and Usability in Encrypted Email,” IEEE Internet
Computing, vol. 21, no. 3, pp. 30–38, 2017.
[9] W. Bai, M. Namara, Y. Qian, P. G. Kelley, M. L. Mazurek, and D. Kim,
“An Inconvenient Trust: User Attitudes toward Security and Usability
Tradeoffs for Key-Directory Encryption Systems,” in Twelfth Symposium
on Usable Privacy and Security (SOUPS 2016). Denver, CO: USENIX
Association, Jun. 2016, pp. 113–130.
[10] J. Callas, L. Donnerhacke, H. Finney, D. Shaw, and R. Thayer,“OpenPGP message format (RFC 4880),” 2007.
[11] J. Clark, P. C. van Oorschot, S. Ruoti, K. E. Seamons, andD. Zappala, “Securing Email,” CoRR, 2018. [Online]. Available:
http://arxiv.org/abs/1804.07706[12] D. Davis, “Compliance Defects in Public Key Cryptography,” in Proceedings of the 6th USENIX Security Symposium, San Jose, CA, USA,
July 22-25, 1996. USENIX Association, 1996.
[13] Z. Durumeric, D. Adrian, A. Mirian, J. Kasten, E. Bursztein,
N. Lidzborski, K. Thomas, V. Eranti, M. Bailey, and J. A. Halderman,“Neither Snow Nor Rain Nor MITM...: An Empirical Analysis of Email
Delivery Security,” in Proceedings of the 2015 Internet Measurement
Conference, ser. IMC ’15.
New York, NY, USA: Association for
Computing Machinery, 2015, p. 27–39.
[14] S. Dusse, P. Hoffman, B. Ramsdell, L. Lundblade, and L. Repka,“RFC2311: S/MIME Version 2 Message Specification,” 1998.
[15] M. Elkins, D. D. Torto, R. Levien, and T. Roessler, “RFC3156: MIME
Security with OpenPGP,” 2001.
[16] S. Fahl, M. Harbach, T. Muders, M. Smith, and U. Sander, “Helping
Johnny 2.0 to Encrypt His Facebook Conversations,” in Proc. 8th
Symposium on Usable Privacy and Security (SOUPS’12). ACM, 2012.
[17] D. Fisher, A. J. Brush, E. Gleave, and M. A. Smith, “Revisiting
Whittaker & Sidner’s ”Email Overload” Ten Years Later,” in Proceedings of the 2006 20th Anniversary Conference on Computer Supported
Cooperative Work, ser. CSCW ’06. New York, NY, USA: Association
for Computing Machinery, 2006, p. 309–312.
[18] I. D. Foster, J. Larson, M. Masich, A. C. Snoeren, S. Savage, andK. Levchenko, “Security by Any Other Name: On the Effectivenessof Provider Based Email Security,” in Proceedings of the 22nd ACM
SIGSAC Conference on Computer and Communications Security, ser.
CCS ’15. New York, NY, USA: Association for Computing Machinery,
2015, p. 450–464.
[19] S. L. Garfinkel, D. Margrave, J. I. Schiller, E. Nordlander, and R. C.
Miller, “How to Make Secure Email Easier to Use,” in Proceedings of
the SIGCHI Conference on Human Factors in Computing Systems, ser.

CHI ’05. New York, NY, USA: Association for Computing Machinery,
2005, p. 701–710.
[20] S. L. Garfinkel and R. C. Miller, “Johnny 2: a user test of key
continuity management with S/MIME and Outlook Express,” in Proc.
1st Symposium on Usable Privacy and Security (SOUPS’05). ACM,
2005.
[21] S. L. Garfinkel, J. I. Schiller, E. Nordlander, D. Margrave, and R. C.
Miller, “Views, Reactions and Impact of Digitally-Signed Mail in eCommerce,” in Financial Cryptography and Data Security, A. S. Patrickand M. Yung, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg,
2005, pp. 188–202.
[22] S. Gaw, E. W. Felten, and P. Fernandez-Kelly, “Secrecy, Flagging, andParanoia: Adoption Criteria in Encrypted Email,” in Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems, ser. CHI’06. New York, NY, USA: Association for Computing Machinery,
2006, p. 591–600.
[23] G. Greenwald and E. MacAskill, “NSA Prism program taps into user
data of Apple, Google and others,” The Guardian, vol. 7, no. 6, pp.
1–43, 2013.
[24] G. Greenwald, E. MacAskill, and L. Poitras, “Edward Snowden: thewhistleblower behind the NSA surveillance revelations,” The Guardian,
vol. 9, no. 6, p. 2, 2013.
[25] A. Lerner, E. Zeng, and F. Roesner, “Confidante: Usable Encrypted
Email: A Case Study with Lawyers and Journalists,” in 2017 IEEE
European Symposium on Security and Privacy (EuroS&P).
Los
Alamitos, CA, USA: IEEE Computer Society, apr 2017, pp. 385–400.
[26] J. R. P. Mauriés, K. Krol, S. Parkin, R. Abu-Salma, and M. A. Sasse,“Dead on Arrival: Recovering from Fatal Flaws in Email Encryption
Tools,” in The LASER Workshop: Learning from Authoritative Security
Experiment Results (LASER 2017). USENIX Association, Oct. 2017,
pp. 49–57.
[27] S. E. McGregor, E. A. Watkins, M. N. Al-Ameen, K. Caine, and
F. Roesner, “When the Weakest Link is Strong: Secure Collaboration in
the Case of the Panama Papers,” in 26th USENIX Security Symposium(USENIX Security 17). Vancouver, BC: USENIX Association, Aug.
2017, pp. 505–522.
[28] A. Peterson, “Edward Snowden sent Glenn Greenwald this video
guide about encryption for journalists. Greenwald ignored it.”https://www.washingtonpost.com/news/the-switch/wp/2014/05/14/
edward-snowden-sent-glenn-greenwald-this-video-guide-aboutencryption-for-journalists-greenwald-ignored-it/, 2014.
[29] V. Roth, T. Straub, and K. Richter, “Security and usability engineeringwith particular attention to electronic mail,” International Journal of
Human-Computer Studies, vol. 63, no. 1, pp. 51–73, 2005.
[30] S. Ruoti and K. Seamons, “Johnny’s Journey Toward Usable Secure
Email,” IEEE Security & Privacy, vol. 17, no. 6, pp. 72–76, 2019.
[31] S. Ruoti, J. Andersen, S. Heidbrink, M. O’Neill, E. Vaziripour, J. Wu,
D. Zappala, and K. Seamons, “”We’re on the Same Page” A Usability
Study of Secure Email Using Pairs of Novice Users,” in Proc. CHI
Conference on Human Factors in Computing Systems (CHI’16), 2016.
[32] S. Ruoti, J. Andersen, T. Hendershot, D. Zappala, and K. E. Seamons, “Private Webmail 2.0: Simple and Easy-to-Use Secure Email,”in Proceedings of the 29th Annual Symposium on User Interface
Software and Technology, UIST 2016, Tokyo, Japan, October 16-19,
2016, J. Rekimoto, T. Igarashi, J. O. Wobbrock, and D. Avrahami, Eds.
ACM, 2016, pp. 461–472.
[33] S. Ruoti, J. Andersen, T. Monson, D. Zappala, and K. Seamons, “A
Comparative Usability Study of Key Management in Secure Email,” inProceedings of the Fourteenth USENIX Conference on Usable Privacy
and Security, ser. SOUPS ’18. USA: USENIX Association, 2018, p.
375–394.
[34] S. Ruoti, J. Andersen, D. Zappala, and K. E. Seamons, “Why Johnny
Still, Still Can’t Encrypt: Evaluating the Usability of a Modern
PGP Client,” CoRR, vol. abs/1510.08555, 2015. [Online]. Available:
http://arxiv.org/abs/1510.08555[35] S. Ruoti, N. Kim, B. Burgon, T. W. van der Horst, and K. E. Seamons,“Confused Johnny: when automatic encryption leads to confusion andmistakes,” in Symposium On Usable Privacy and Security, SOUPS ’13,
Newcastle, United Kingdom, July 24-26, 2013, 2013, pp. 5:1–5:12.
[36] S. Sheng, L. Broderick, C. A. Koranda, and J. J. Hyland, “WhyJohnny Still Can’t Encrypt: Evaluating the Usability of Email Encryption
Software,” in Proc. 2nd Symposium on Usable Privacy and Security(SOUPS’06). ACM, 2006.

[37] Signal, “Signal Support - Is it private? Can I trust it?”https://support.signal.org/hc/en-us/articles/360007320391-Is-it-privateCan-I-trust-it-, visited 02/01/2021.
[38] C. Stransky, D. Wermke, J. Schrader, N. Huaman, Y. Acar, A. L.
Fehlhaber, M. Wei, B. Ur, and S. Fahl, “On the Limited Impact ofVisualizing Encryption: Perceptions of E2E Messaging Security,” inSeventeenth Symposium on Usable Privacy and Security (SOUPS 2021).
USENIX Association, Aug. 2021, pp. 437–454. [Online]. Available:
https://www.usenix.org/conference/soups2021/presentation/stransky[39] C. Stransky, O. Wiese, V. Roth, Y. Acar, and S. Fahl, “Companion
Website - EMail Paper,” https://publications.teamusec.de/2022-oaklandemail/, 2021.
[40] O. Tange, “GNU Parallel - The Command-Line Power Tool,” ;login:
The USENIX Magazine, vol. 36, no. 1, pp. 42–47, Feb 2011. [Online].
Available: http://www.gnu.org/s/parallel[41] H. Tankovska, “Statista - Number of e-mail users worldwide from2017 to 2024,” https://www.statista.com/statistics/255080/number-of-email-users-worldwide/, 2021.
[42] A. Ulrich, R. Holz, P. Hauck, and G. Carle, “Investigating the OpenPGPWeb of Trust,” in European Symposium on Research in Computer
Security. Springer, 2011, pp. 489–507.
[43] WhatsApp
LLC,“end-to-end
encryption
WhatsApp,”https://blog.whatsapp.com/end-to-end-encryption/?lang=en,visited02/01/2021.
[44] S. Whittaker and C. Sidner, “Email overload: exploring personal information management of email,” in Proceedings of the SIGCHI conferenceon Human factors in computing systems, 1996.
[45] A. Whitten and J. D. Tygar, “Why Johnny Can’t Encrypt: A Usability
Evaluation of PGP 5.0,” in Proc. 8th Usenix Security Symposium(SEC’99). USENIX Association, 1999.
[46] P. Zimmermann, “PGP Version 2.6.2 User’s Guide,” https://web.pa.msu.
edu/reference/pgpdoc2.html, Oct. 1994.

Fig. 8. PGP parsing errors over the years. The marked area is the parsing
errors of the top 5 users of the year.

A PPENDIX
A. S/MIME and PGP Parsing Errors
During the parsing of the mails, we encountered parsing
errors for 0.09% of all emails.
While the error rate for S/MIME is consistently low (cf.
Fig. 9), the error rate for PGP is higher (cf. Fig. 8). Overall, we
encountered parsing errors for up to 1.17% of the S/MIME and22.09% of the PGP emails per year. While we cannot provide
in-depth insights into the parsing errors due to ethics and dataprotection concerns, Figures 8 and 9 show that most of theparsing errors for both S/MIME and PGP were caused by thefive most active users of the respective years. For example,
we saw most parsing errors for S/MIME in 2005 and PGP in
2004 where our data set included 1,367 S/MIME emails and
47 users and 1,521 PGP emails and 129 users. In this year, thethree most active S/MIME users contributed all parsing errors
for S/MIME and the five most active PGP users contributed
306 parsing errors (88.26% of all PGP parsing errors). Even
more extreme is the peak for PGP email related parsing errors
in 2004 which were mostly caused by a single KMail 17 user.
While parsing errors in general are disappointing since they
add noise to our results, the fact that the vast majority of errors
was caused by a small group of or even single users alleviates
their negative effects. Our evaluation focuses on a per-user andnot a per-email view. A small number of users whose emails
could not be parsed, for example, only has a low impact on
the overall data concerning the use of multiple email clients.
17 cf. https://apps.kde.org/kmail2/

Fig. 9. S/MIME parsing errors over the years. The marked area is the parsing
errors of the top 5 users of the year.

B. Pseudonymization Table
Table IV illustrates the pseudonymization we applied to thedata.
C. Certificates issued for our university
Figure 10 shows the certificates that were issued by the DFN
PKI in each year up until February 19, 2021.

Header-Data

FormatMessage ID
User
User group
SenderReceivers
CC list
BCC list
Date
ClientFolderSHA-256 with salt
SHA-256 with salt
Categorized
SHA-256 with salt
SHA-256 with salt
SHA-256 with salt
SHA-256 with salt
Bracketed into week
Raw value
Categorized

S/MIME-Data

FormatSerial Number
Not Valid Before and After
Issuer Info
Signature AlgoKey size
Key type

SHA-256 with salt1
Bracketed into weeks1
Raw value
Raw value
Raw value
Raw value

PGP-Data2

FormatKeyID
Creation date
Expiration date
Type of key
LengthKey Algo
Digest Algo

SHA-256 with salt
Bracketed into week
Bracketed into week
Raw value
Raw value
Raw value
Raw value

1 Applied to the leaf certificate
2 Applied to both primary and sub key info

TABLE IV
C OLLECTED METADATA FOR ALL EMAILS INCLUDING THE STORAGE
FORMAT. F OR DATA PROTECTION REASONS AND IN CONSULTATION WITHOUR UNIVERSITY ’ S DATA PROTECTION OFFICER AND THE IT STAFF , WE
CHOSE TO PSEUDONYMIZE SOME VALUES USING A HASH FUNCTION OR
CATEGORIES .

Fig. 10. X.509 certificates issued by the DFN PKI for our university from2005 to 2020 using two different root certificates G1 and G2. G1 expired in
2019. Includes certificates issued until February 19, 2021.