Collections As Data: Preservation to Access to Use to Impact
Short Paper
J. Mears

A. Potter

K. Zwaard

Library of Congress
United States
jame@loc.gov

Library of Congress
United States
abpo@loc.gov

Library of Congress
United States
kzwa@loc.gov

ABSTRACTThe Library of Congress has many millions of objects it has either
digitized or acquired in born-digital formats. The first priority is
to preserve, process, and provide access to digital collections via
loc.gov. Most of these collections are free and available to anyone
around the world, and their existence online represent theculmination of a huge amount of time, effort, and expertise. The
Library of Congress has embarked upon an effort to maximize theuse of our digital collections, promote innovation at theinstitution, and grow national capacity for cultural memory. This
paper will describe how developing and executing a programaround promoting collections as data has enabled the expansion ofinternal and external partners, deepened the exploration and value
proposition of digital collections, engaged a broader community,and provided new skills to librarians who are working withcollections as data. The programs that will be outlined will
include the development of a digital scholarship lab which seeks
to provide support for computational engagement with collections,
testing different training models for digital skill building, andlaunching Innovator-in-Residence programs that showcase howdata analysis techniques can change digital scholarship and digital
curation landscapes. These partnerships and pilot projects build on
the digitization and preservation efforts of the Library bybroadening the scope of engagement and providing compellinguse cases that amplify the impact of digital collections.

KEYWORDS
Data Analysis, Digital Scholarship, Outreach, Education &Training, Impact of Preservation

1

LAUNCHING NATIONAL DIGITAL
INITIATIVESThe National Digital Initiatives division was founded in October
2016 and operates in the National and International Outreachservice unit under National Programs. The core staff came fromthe retired National Digital Information Infrastructure andPreservation program after a realignment of the Library ofCongress service units. The founding principle of NDI is to honor

the knowledge, skills, and effort of the Library of Congress staff
who build our digital library. Much of this is invisible work. The
acquisition, transfer, processing, and preservation of petabytes ofdigital collections is central to the Library’s mission and purpose
and represent an incredible asset that belongs to the American
public. NDI also inherits the network building and community
engagement success of the NDIIPP program.
NDI’s mission is to broaden awareness of the Library’s
innovation and use of its digital resources through outreach andexternal partnerships. We seek to work closely with other service
units and divisions to reach new audiences, spark innovative
projects, and work together on shared problems. To execute thisstrategy, three program areas were defined: 1) Expand the use of
Library of Congress digital resources, 2) Incubate, encourage, andpromote digital innovation, and 3) Grow national capacity forcultural memory.

2

ENABLING NEW USES OF THE DIGITALCOLLECTION

2.1

Collections as Data

The Library of Congress and other libraries have been serving
digital collections online for over a decade. However, the model
for which these resources are accessed largely recreates an analog
experience of reading a page at a time or browsing through acollection of photographs. With modern computing power and theemergence of data-analysis tools, these digital collections can be
explored more deeply and reveal more connections. In order totake advantage of computation, collections must be madeavailable in a form that can be recognized by computers, atransformation not unlike moving from creating a card catalog to
the MARC record schema. Obviously, the collection must bedigitized but often more than a just a digitized image is needed tomake full use of the object. Optical character recognition thatreflects underlying text or metadata that describe the content in a
structured way, like using ISO date and formats, latitude andlongitude coordinates, or tags according to a taxonomy, are dataLibrary of Congress, March 2017, Washington, D.C. USAthat widely available tools can use to create visualizations andperform analysis.
For users that want to work with a large number of digital files
and download them in bulk rather than one at a time, The Libraryof Congress provides processed bulk-data-download derivatives
for two collections. The MARC Distribution Service provides allofThe
LibraryofCongresscatalogrecords
andthe(https://www.loc.gov/cds/products/marcDist.php)
National Digital Newspaper Program provides the raw OCR textof newspaper pages from the Chronicling America project(http://chroniclingamerica.loc.gov/ocr/). There are additional
collections for which there is an approximation of bulk data
access for anyone with the technical capacity for scripting and theaptitude to understand and crawl an API or a site. Affixing thetext "?fo=json" to the end of many URLs in https://loc.gov results
in a JSON API that can get some researchers started(see http://www.loc.gov/pictures/?fo=json). XML sitemaps areavailablefor,currently,
277
digitalcollections(https://www.loc.gov/collections/sitemap), providing informationsuch as the item URLs, last modified date, and frequency of
updates (e.g. https://www.loc.gov/collections/franklin-piercepapers/?c=1000&sp=1&fo=sitemap). Similarly, https://id.loc.gov
has a number of formats available for crawling a large number ofdatasets, but the bulk download of data is not generally available.
The World Digital Library (https://api.wdl.org) has a welldocumented API which serves a similar purpose for thosecollections. There are collections which do not have an API,
documented or otherwise, that allows crawling or web harvesting.
Some very motivated researchers have previously used this kindof interaction to build datasets, or to extract particular curated
selections, and to make them available elsewhere.
Locations away from our main website occasionally havecapabilities like one of those described above. For example, theLibrary of Congress participates in the Flickr Commons(https://www.flickr.com/commons) which comes with some
capabilities of using the Flickr API for downloading collections.
These few examples represent a small percentage of the roughly
300 collections and approximately 2 million items, in multiple
content formats, available on loc.gov but not optimized foranything other than interactive browsing and various degrees ofsearchability. One of the goals of NDI is to promote standards andpractices around providing access to collections as data so that thefull value of computation can be leveraged to bring even moreawareness of the knowledge and creativity contained in theworld’s libraries.

2. 2 Exploring Digital Scholarship
Emerging disciplines — like data science, data journalism anddigital humanities that take advantage of new computing tools andinfrastructure — provide a model for creating new levels of accessto library collections. Visualizing historical events and
2

J. Mears et al.
relationships on maps, with network diagrams and analysis of
thousands of texts for the occurrence of words and phrases are a
few examples of what’s possible. NDI is actively exploring how
to support these and other kinds of interactions with the Library’s
vast digital holdings.
Michelle Gallinger and Daniel Chudnov were asked by NDI to
study how libraries and other research centers have developed
services that use computational analysis, design and engagement
to enable new kinds of discovery and outreach. Their report(PDF)[1], was just released. For the report, they interviewed
researchers and managers of digital scholarship labs and worked
with Library staff on a pilot project that demonstrated how thecollections could be used in data analysis. This work resulted inconcrete recommendations to the Library on how to approachsetting up a Lab at the Library of Congress. These
recommendations could also be helpful to other organizations who
may be thinking of establishing their own centers for digitalscholarship and engagement.
The recommendations from Gallinger and Chudnov would
warrant a paper on its own but we will review some of theirfeature recommendations: First, that the Lab should be service
oriented to outside users. Second, that the Lab support users togrow and develop their research. Third, that the Lab be an
educational space, and fourth that the Lab enable organizationaltransformation. As research, scholarship, and resources become
more digitally based, the services provided by the Lab will, in the
beginning, be groundbreaking but eventually become part of thenormal services offered by the Library. It is this transformation of
how librarians perform reference services for the digitalcollections—services that include identifying available data,
working with that data to deliver specific a specific corpus,
establishing provenance for the data, and helping to answer
research questions with the data—that NDI hopes to cultivate in adigital scholars lab.
Concurrently, NDI is co-leading an internal group charged withstudying how the Library of Congress can programmaticallyenable digital scholarship for its collections. The group iscompleting a paper in 2017 that describes the challenges andopportunities inherent in the Library of Congress’ collections andoperations. Much of the information in the Collections as Datasection of this paper comes from that group’s efforts.

2.3

Engaging Communities

Using Gallinger and Chudnov’s report and recommendations as
guidance, an on-site digital exploration center is an opportunity toopen the Library’s digital holdings up for enhanced scholarly
inquiry. It would also serve as a point of engagement for moregeneral users for topic or visual exploration It would serve
numerous communities:x
Scholars interested in using Library of Congress digitalcollections as data to create visualizations, explore

Collections as Data: Preservation to Access to Use to Impact

Library of Congress, March 2017, Washington, D.C. USA

digital scholarship, and pursue computationally-assisted
research.
Teachers and students who want to use Library ofCongress data in their STEM or humanities classwork
and projects.
Artists, writers and other creatives who want to accessour public domain content to remix and reuse the
Library’s digital materials in new ways.
On-site and virtual users who want bulk access to digitalcollection data for analysis or use in other applications.

and introduces them to low or no-cost computational tools to
explore digital library collection data sets. Over two days, 61
librarians and researchers attended, most with no priorprogramming experience. They were given instruction around
MALLET, a topic modeling tool, Gephi, a network analysis
visualization tool, OpenRefine, a data editing tool, and Carto, a
mapping tool. According to post-event surveys, attendees
confirmed that the event was valuable in orienting them tocomputational methodologies and new research and processing
possibilities for their digital collections.

In developing coordinating services, the Library will work
collaboratively to design orientation and introductory classes on
digital scholarship methods and tools to help introduce theconcept of using library collections as data. The courses will be inparallel to professional development trainings for Library staff
who will be delivering services, especially reference services, in
new ways.

Developing and promoting a program to develop skills aroundcollections as data has enabled the expansion of internal andexternal partners, deepened the exploration and value propositionof digital collections, engaged a broader community, and provided
new skills to librarians who are working with collections as data.

x

x

x

2.4

Skill-building

Providing access to digital library collections as data requires new
skills for librarians and using digital library collections similarly
requires new skills and modes for researchers. NDI is exploring
how to offer educational opportunities for both of these groups togain these new skills. In February 2017, 40 librarians, archivists
and data wranglers came to the Library of Congress to learn
advanced skills in managing digital collections. The Library
hosted the Software Carpentry workshop, inviting staff from theinstitution, the DC Public Library and other federal libraries forhands-on learning in the programming language Python, theversion-control software Git and the command-line interface
Bash. Software Carpentry is a volunteer, non-profit organizationthat provides short, intensive workshops to help researchers
automate tasks and manage information. It started with scholars in
the physical sciences who found that traditional graduate
programs were not preparing them for the challenges of working
with data for their research products. Software Carpentry
workshops have lately been adapted for social sciences, thehumanities and libraries.
Attendees from the workshop immediately saw a way to apply
their new skills. “I can see an opportunity to use scripts toimprove researchers’ experience in the reading room,” said
Kathleen O’Neill, a senior archives specialist in the Manuscript
Division. “For those researchers with limited experience withdigital collection material, we could provide a library of simple
scripts to search, analyze and report on the born-digital collection
material.”
Later in the year, NDI partnered with George Mason and George
Washington University Libraries to host a program titled “Hackto-Learn”. There is clear demand for hands-on computational
experience yet librarians are often under represented at events likehackathons. So the organizers worked together to develop an
inclusive hackathon that utilizes the skills librarians already have

3

SHOWCASING DATA ANALYSISTECHNIQUES

3.1

Innovators in Residence

While 3 million people access the collection every month on the
loc.gov website, there is untapped potential of using the digitalcollections in other ways, be it for data analysis to support digitalscholarship, by an artist seeking to remix and reuse content tocreate new art or commerce, or a journalist in need of
authoritative data for in-depth reporting.
NDI has launched an Innovator-in-Residence program to support
innovative uses of our collections and partnerships with
universities, scholarly societies, artists, corporations, and otherorganizations. A wide variety of programs would be included in a
broad residency program:
x

x

x

x

x

A program that would support the work of digital
humanities scholars at the Library of Congress, whichhave been successful at other large libraries.
A challenge grant program that would run contests forinnovative uses of digital collections and make small
awards to winners, modeled after Federal Data
Challenges.
A program that would support software developers tocome to the Library and build tools and services thatmake innovative uses of Library collections.
A fellowship program to support hybrid teams to use
Library of Congress collections and data to support
information for the American people. A journalist anda social scientist working with a historian working on
a project that will be published widely in a
newspaper, magazine or book.
A program to support artist to create art based on the
Library’s collection.

In 2017 two multimedia artists were selected to create digital art
pieces based on the Library of Congress digital collections. In
3

Library of Congress, March 2017, Washington, D.C. USA

J. Mears et al.

addition to creating the works, they will present their art to the
public and Library staff in a workshop.

3.2

Crowdsourcing application

NDI piloted the Innovator-in-Residence program at the Librarywith an opportunity for staff to apply for a short-term assignment.
Two staff members were selected. One, Chris Adams, focused on
exploring automatic image identification and fleshing out theconcept of labs.loc.gov. The other, Tong Wang, created a proofof-concept crowdsourcing application. Tong wanted to show how
we could leverage open source tools created by other members of
ourcommunity(specifically,
the
Scribehttp://scribeproject.github.io/ transcription framework) and theChronicling
AmericaAPI(http://chroniclingamerica.loc.gov/about/api/) to engage the publicwith library collections and to enhance the usability of our digitalmaterial. The application he created is called 1000 Words and it
invites users to identify cartoons, photographs, and other images
in historic newspapers and to describe them. This would allow
those images to be searched for the first time and for data sets tobe created. For example, the Library could offer to researchers acollection World-War-I-era cartoons, which previously would
require flipping through thousands of newspaper pages to collect.

3.3

EventsWe have hosted several events in an effort to demonstrate what
we mean when we talk about collections as data and provoke
exploration of our collections computationally for our staff andmembers of the public.
We invited programmers, artists, entrepreneurs, activists,
researchers and librarians finding new ways to connect withdigital collections to share best practices and lessons learned
during Collections as Data: Stewardship and Use Models to
Enhance Access, a public conference held at the Library ofCongress in September 2016. From tracking the popularity ofbible verses in historic newspapers to mapping everywhere you've
ever walked on your smartphone, speakers such as Jer Thorp fromthe Office for Creative Research, Pinboard founder Maciej
Ceglowski, and Director of the Five College Digital Humanities
Initiative Marisa Parham discussed the methods, skills, ethicalconsiderations and community participation necessary to find newmeaning in digitized and born digital cultural heritage collections.
The topic resonated with many. The event’s hashtag #AsData
trended #2 on Twitter the day of the event, receiving over 8500
livestream views. The more than 450 people came, representing
over 15 different countries from universities, public libraries,
galleries, federal institutions, and newspapers such as the
Washington Post and New York Times. Tweets and verbal
feedback revealed attendees appreciated the holistic, nuanced,
cross-disciplinary consideration of data.

4

Figure 2: #AsData poster designed by Oliver Baez Bendorf.

The program was so successful that we will host a second
Collections as Data Symposium in July, showcasing the impact ofcomputational research through a series of successful case studies
across disciplines and practitioners.
To facilitate learning and discovery, we were the host site forArchives Unleashed 2.0 in 2016, a web archive datathon series
facilitated by Matthew Weber, Ian Milligan and Jimmy Lin inwhich teams of researchers used a variety of analytical tools toquery web-archive data sets in the hopes of discovering some
intriguing insights before their 48-hour deadline is up. Aside fromhosting, the Library of Congress participated on the Hackathon
teams and by providing collections for the teams to work with.
In an effort to centralize participatory opportunities for local
District of Columbia residents, develop best practices forcommunity engagement, and support sharing of information
across DC cultural heritage institutions, NDI is co-hosting a
Citizen Public History Fair in September with NARA, theSmithsonian Transcription Center, DC Public Library Special
Collections, United States Holocaust Memorial Museum, theFolger Shakespeare Library, the Lincoln Theatre and theAnacostia Community Museum. The fair will showcase or demo
volunteer project opportunities including digital crowdsourcing
initiatives and feature lightning talks from curators andresearchers. Following the event, a series of crowdsourcing
programs by our institutions will take place throughout the year inan effort to develop a robust community of practice aroundhistorical resources.

Collections as Data: Preservation to Access to Use to Impact

Library of Congress, March 2017, Washington, D.C. USA

In 2018, NDI will also serve as co-host of several high-profile
meetings at the Library of Congress that support the collections asdata vision. These include the 2018 Code4Lib Annual meeting,the 2018 IIIF Annual meeting, a HathiTrust Research Center
workshop, and a workshop in conjunction with a grant funded
DARIAH-EU project on digital heritage and culture.

4

CONCLUSION

The Library of Congress has millions of digital items available to
the American Public via the loc.gov web site. Millions of people
from all over the world visit our vast collections online. An
additional 1.6 million people visit the Library of Congress
Jefferson building each year [2]. The efforts of the National
Digital Initiatives to support the practice of providing data level
access to digital collections provides opportunities to enhance thepreservability of digital collections and increase their usefulness
to the communities we serve. When digital collections areleveraged as data, the public spaces, technical environment,
training, tools and expert support are all modernized and mademore relevant. These partnerships and pilot projects describedabove build on the digitization and preservation efforts of the
Library by broadening the scope of engagement and providing
compelling use cases that amplify the impact of digitalcollections.

REFERENCES[1] D. Chudnov and M. Gallinger. 2016. Library of Congress Lab: Library ofCongress Digital Scholars Lab Pilot Project Report. Washington, DC.
[2] OCIO Web Metrics & Analytics Service Team. Library of Congress.2016.
Quarterly Key Metrics. http://staff.loc.gov/sites/webmetrics/reports/

5