Using ePADD to Appraise, Process, and Provide Access toHistorically and Culturally Valuable Email Collections
Tutorial Abstract
J. Schneider
Stanford University
557 Escondido Mall
Stanford, CA 94305
USAjosh.schneider@stanford.edu

ABSTRACT

1 ePADD SOFTWARE

This tutorial will provide participants with the knowledge andexperience to begin to implement ePADD at their institution inorder to appraise, process, and provide access to email archives
in their collections. There will be a brief lecture and discussionthat focuses on the motivation for development of the software,and a comparison of the software in the context of other tools
and services. Several technical concepts will be introduced. The
remainder of the tutorial will be devoted to demonstration andhands-on exercises taking users through specific modules andfunctionalities.
Participants will learn how to ingest email in ePADD from
MBOX files or through IMAP, the processes ePADD carries out
upon ingest (de-duplication, name resolution, named entity
recognition/ classification), how to instruct a donor on
appraising and transferring their own email, how to use browse
and search features to screen for and restrict confidential orsensitive messages and attachments, how to annotate messages,
how to assign authority records to correspondents using ePADD,
and how to export mail between ePADD modules. Participants
will also learn how researchers can use these same tools within
ePADD to discover, browse, and search relevant messages,
including how they can review and request specific messages
and attachments.
The tutorial will also include discussion on overcoming
potential implementation challenges, as well as opportunities toparticipate in ePADD‘s development. The tutorial will bebeneficial for all those with a responsibility or interest inprocessing and/or providing access to email archives.

ePADD is free and open-source computational analysis softwarethat allows individuals and institutions to appraise, process, andmake discoverable and fully accessible for research email ofpotential historical or cultural value. [2] The software primarily
accomplishes this goal by incorporating techniques fromcomputer science and computational linguistics, including
natural language processing, named entity recognition, andother statistical machine learning-associated processes [1].

CONCEPTS• Computing Methodologies → Artiﬁcial Intelligence;
Natural language processing • Computing Methodologies →Machine Learning • Information Systems → World WideWeb; Web applications; Internet communications tools; EmailKEYWORDS
Acquisition, Archival appraisal, Archival processing, Archives,
Descriptive metadata, Email, Named entity recognition, Naturallanguage processing, Privacy, Redaction, Screening, Web access

2 ePADD SOFTWARE ARCHITECTURE
2.1 Appraisal
Appraisal provides donors, curators, and archivists with a toolset
to review and manage an email archive prior to accessioning itto a repository. ePADD can gather email from multiple sources.
Upon ingest, ePADD de-duplicates messages, resolves
correspondent names from the address book, and extracts finegrained entities using a custom NLP toolkit. These
functionalities and others enable users to determine therelevance and importance of email messages, identify and flag
confidential, restricted, or legally-protected information, andimpose access restrictions prior to transfer.
2.2 Processing
Processing is designed for an archivist to further perform allfunctions included in the Appraisal module, including scanningfor confidential, restricted, or legally-protected information, aswell as other tasks that prepare the archive for discovery by anddelivery to end users, such as reconciliation of correspondents
and extracted entities with established authority records.
2.3 Discovery
Discovery is designed to run under a standalone web server, andallows researchers to browse and search a redacted emailcollection prior to physically traveling to a repository’s reading
room to access the full corpus. Only metadata from theprocessed email archive is published online.

2.4 Delivery
Delivery provides users with access to the full contents of theunrestricted portions of a processed email archive, including
attachments, from a managed workstation in a repository’s
reading room.

3 TUTORIAL FORMAT
There will be a brief lecture and discussion that focuses on themotivation for development of the software, and a comparison of
the software in the context of other tools and services. Several
technical concepts will be introduced. The remainder of thetutorial will be devoted to demonstration and hands-on exercises
taking users through specific modules and functionalities, aswell as a brief discussion on overcoming potential
implementation challenges, as well as opportunities toparticipate in ePADD‘s development.

4 INTENDED AUDIENCE
The tutorial will be beneficial for all those with a responsibility
or interest in appraising, processing, and/or providing access toemail archives, or who are interested in familiarizing themselveswith the technologies utilized to accomplish these goals, in orderto potentially apply them to other file genres or formats.

5 EXPECTED LEARNING OUTCOMES
This tutorial will provide participants with the knowledge andexperience to begin to implement the free and open source
software ePADD at their institution in order to appraise, process,and provide access to email archives in their collections.
Participants will learn how to ingest email in ePADD from
MBOX files or through IMAP, the processes ePADD carries out
upon ingest (de-duplication, name resolution, named entity
recognition/ classification), how to instruct a donor on
appraising and transferring their own email, how to use browse
and search features to screen for and restrict confidential orsensitive messages and attachments, how to annotate messages,
how to assign authority records to correspondents using ePADD,
and how to export mail between ePADD modules. Participants
will also learn how researchers can use these same tools within
ePADD to discover, browse, and search relevant messages,including how they can review and request specific messages
and attachments.
Participants will also become aware of the resources that areavailable for learning more about the software and overcoming
potential implementation challenges, as well as opportunities toparticipate to participate in the user community and join inePADD‘s development.

6 INSTRUCTOR BIOGRAPHY
Josh Schneider is Assistant University Archivist at Stanford
University, where he acquires and supports researcher use ofStanford University records, faculty papers, and materials
documenting campus and student life. His case study onappraisal of electronic records appeared in the latest volume of
the Society of American Archivists’ Trends in Archival Practiceseries. Josh is also Community Manager for ePADD, an opensource software package that uses named entity recognition andother NLP-driven processes to support the appraisal, processing,
discovery, and delivery of email archives. Josh serves on theeditorial boards of The American Archivist, Journal of Western
Archives, and the blog of SAA’s Electronic Records Section(BloggERS!). He received an MLIS from Simmons College and aBA in Philosophy from Brown University.

ACKNOWLEDGMENTS
ePADD development is managed by Stanford University’sDepartment of Special Collections & University Archives, part of
Stanford University Libraries [3]. The ePADD development teamis composed of Glynn Edwards, Peter Chan, Josh Schneider, andSudheendra Hangal [4]. Development work is carried out incollaboration with partners at Harvard University, theMetropolitan New York Library Council (METRO), University ofIllinois at Urbana-Champaign, and University of California,Irvine. Funding for current ePADD development is provided
through an Institute of Museum & Library Studies (IMLS)
National Leadership Grant (NLG) for Libraries. Development for
the initial 2015 release of ePADD was primarily funded by theNational Historical Publications and Records Commission(NHPRC).

REFERENCES[1] Email: Process, Appraise, Discover, Deliver -- ePADD Phase 2. Project Proposal,
National Leadership Grant for Libraries. Retrieved June 27, 2017, from Instituteof Museum and Library Services: https://www.imls.gov/grants/awarded/lg-7015-0242-15[2] ePADD repository, 2017. Retrieved June 27, 2017, from Github:https://github.com/EPADD/epadd[3] ePADD software, 2017. Retrieved June 27, 2017, from Stanford University
Libraries: http://library.stanford.edu/projects/epadd[4] Hangal, S., et al. 2014. Historical research using email archives in special
collections. Proceedings of ACM CHI Conference on Human Factors in
Computing Systems. Toronto, Canada. Retrieved June 27, 2017, from Stanford
University: https://mobisocial.stanford.edu/papers/chi2015.pdf