# Presentation: 319649

## NDIIPP: Future Directions and Relevance to Other Countries

- Beth Dulabahn
- Office of Strategic Initiatives
- Library of Congress
- November 7, 2004

**Notes:**
Good morning. I’m Beth Dulabahn, from the Librtary of Congress, and on behalf of Laura Campbell I’ll be talking the  Library’s National Digital Information Infrrastructure and Preservation Program, or NDIIPP.

So what I’d like to do today is talk very briefly about NDIIPP, what we’ve been doing since the inception of the program, and where we’re going.

## NDIIPP Legislation and Funding

- Created by federal legislation (PL 106-554) in December 2000
- $100 million in appropriated funds
- Up to $175 million potentially invested
- $75 million of the appropriated funds subject to $ for $ match from non-federal sources

**Notes:**
As many of you know, the NDIIPP Program was created by  Congress at the end of 2000.  Congress appropriated $100 million dollars for NDIPP, $5 million of which it gave to the Library outright, $20 million of which it pledged subject to Congressional approval, and the remaining $75 million of which was subject to a one-fo-one match from non-federal sources.

## NDIIPP Goals

- Develop a national digital collection and preservation strategy
- Work with government agencies, libraries, archives, and other stakeholders to establish partnerships and form networks
- Help identify and preserve at-risk digital content
- Support development of tools, models, and methods for digital preservation

**Notes:**
So what is it that NDIIPP hopes to accomplish? First and formost, it aims to develop a national digital collection and preservation strategy. To accomplish this, it needs to work with other agencies to form partnerhships and networks. And we recoginize that these partnerships most extend beyond our national borders to include international partners as well.

Because the amount of digital content far exceeds the capacity of any single institution to collect and preserve it, we will rely on those partnerships to help identify and presrve the content.

--And through NDIIPP we will provide support for the development of tools and techniques for acquiring and sustaining digital content.

## Who’s Included?

- Key federal agencies (including NARA, NLM, NAL, Commerce)
- Libraries, archives, and other cultural institutions
- Non-profit entities (e.g., RLG, OCLC, CLIR)
- Non-federal and commercial sectors (e.g., motion picture studios, record labels, publishers)

**Notes:**
What was unique about Congress’s intent was that LC was to involve other federal agencies, cultural heritage institutions and related non-profits, as well as a variety of commercial entities, who for the most part had not come to the table to talk about cooperating in the area of digital preservation

Unique for LC
International cooperation not emphasized

But despite this, it is very clear that the problems and issues don’t have national borders, so we see this program as very much part of an international effort to address the issues that face all of usl
Already mentioned: Participation in the International Internet Preservation Consortium
Technical specifications for open source tools to harvest and preserve web content
Share experiences and best practices
UK Digital Preservation Coalition

## Initial Activities

- Established the NDSAB in Spring 2001
- Representation from federal agencies, industry, research libraries and foundations
- Convened Stakeholder Meetings in Fall 2001
- 3 workshops including representatives from the commercial sector
- Commissioned environmental scans
- Conducted Scenario Planning sessions
- Defined preservation architecture space
- Facilitated a National Science Foundation research agenda
- Developed Plan & presented to Congress 2003

**Notes:**
Basically, the first two years were a period of listening in learning, as we consulted broadly to find out what others saw as critical problems and issues. 

Intellectual property and liability issues
Collaborative collecting policies
Understanding user communities
Balance between preservation and access
Commercial entities—interested, concerned, cautious.
Areas that stakeholder want NDIIPP (and LC) to focus on:

Selection: Explore coordinated collecting agreements, selection factors
IP: Among the most challenging of all barriers; necessary to separate preservation and access for certain content—but for whom is content preserved?
Business models:  Who pays?  What incentives are needed for owners to want have content preserved?  Key is building a sustainable digital preservation infrastructure
Standards:  Support development of new preservation standards (PDF/A); investigate use of specific preservation techniques (emulation, migration, etc.)
Communication: Build a national constituency for digital preservation; sell the key concept that preservation is a necessary concern during each phase of the digital life cycle
Preservation architecture: A general approach to support federated preservation activities and other relationships between institutions; basic concept is to build an interoperable infrastructure that can be build in pieces over time

## NDIIPP Strategy

- Iterative approach
- Learn by doing

**Notes:**
Overall strategy during  period of performance (8 yrs.)

I think it’s fair to say that we’ve concluded the initial phase of NDIIPP, ;which focused on collecting information and surveying the landscape. We’re now anxious to move on the next phase, the basic strategy for which is to learn by doing.  We’re making an initial round of investments, and expect to make subsequent investments based on what we’ve learned.
Iterative strategy—Learning by doing.  Initial round of investments and  subsequent investments based on what we’ve learned.

## NDIIPP Focus Areas

*Preservation architecture*
- Digital preservation research
- Network of preservation partners

**Notes:**
To date, the NDIIPP program has concentrated on three main areas: 
Context how 3 areas fit together
Preservation Architecture:  The intent of which is to establish an environment of defined goals and responsibilities to achieve interoperability
Research: develop tools and techniques to leverage and make process of preservation more efficient
Third, a Network of partners to share responsibility for content & preservation

## Preservation Architecture

- Architecture is a conceptual framework to guide development of national preservation network
- Build upon current work
- Provide basis for communication about the problem space

**Notes:**
Intent is to provide a common framework for talking about the preservation space at a high conceptual level, not solve the technical problems.

## Slide 9

**Notes:**
This is the visual representation of the framework, which has three layers—a top layer that represents the access functionality that needs to exist; a middle layer that represents the content collection management; and the bottom layer that represents the functionality required to literally store and manage the bits.

## Archive Ingest and Handling Test

- Document steps taken to ingest heterogeneous set of digital content
- Testers have digital preservation capabilities: Johns Hopkins, Old Dominion, Harvard, Stanford
- Each to work with GMU 9/11 Archive
- 12gb with 57,000 objects  associated metadata
- Diverse file formats
- Identify metrics for ensuring preservation of data integrity/document logical integrity

**Notes:**
I talked earlier about the NDIIPP Strategy of “Learning by Doing”  , and the Archive Ingest and Handling test is a good example of how we’re proceeding with that. 4 univiersities—Harvard, Standord, Old Dominion, and Johns Hopkins are all working with the same body of content, which is the George mason University archive. George Mason wantged to deposit the archive with LC, anjd subsequently agreed to have the archive used as the testset in this ingest and handling test.  The archive has the advantage of being relatively small, but containing a variety of file formats.

## Archive Test Activities

- Accept archive as-is
- Generate metadata
- Ingest content into the archive
- Migrate/emulate content as applicable to each archive
- Export content/metadata out of the archive

## Archive Test Outcomes

- List of logical candidate architectures for future work
- Understanding of the strengths and weaknesses of each
- Rough dollar cost figures for each solution
- Grasp of the possibilities and challenges of federating

**Notes:**
What we hope to get from the test is a sense of the strengths and weaknesses of the various architectures in their ability to support archiving and interoperability at a very basis level.

Martha Anderson is the LC person most involved with this test, so please see her if you have more questions about it.

## Research Program

- Joint LC/National Science Foundation (NSF) digital preservation research grants program
- NSF administers
- Agenda shaped by *It’s About Time* report
- Looking to fund cutting edge research to address major needs
- Proposals due 9/14/2004; peer review panels evaluating submissions
- Grants to be announced early 2005

**Notes:**
Over 60 proposals—very interesting set of proposals

## NDIIPP Partnerships

- Agreements finalized Sept. 2004
- Eight consortia involving over 30 partners
- State government entities
- Universities
- Cultural heritage institutions
- Private & not-for-profit
- Awards totaling $15+ million
- Projects expected to last 3 years
- Aim is to spur collaboration in selecting and preserving at-risk content

## Diversity of Content

- Funded projects will work with diverse and challenging content, including
- TV broadcasts and digital video
- Web sites
- State and local government information
- Geospatial data
- Social science data

## NDIIPP Partner Projects with Web Harvesting Components

- The Web at Risk: A Distributed Approach to Preserving Our Nation’s Political Cultural Heritage
- California Digital Library, lead
- University of North Texas, New York University
- State & local gov’t publications, political campaigns, labor history, former federal agency sites
- Develop tools for selection and acquisition

**Notes:**
8 partneerhsip,, there are two that are of special iterrest for attendees at this conference. Out of those 6

TheWeb at Risk is a project led by the California Digitial Library.. Tools development is a major component of this project.

## NDIIPP Partner Projects with Web Harvesting Components (Cont.)

- Exploring Collaborations to Harness Objects with a Digital Environment for Preservation
- University of Illinois at Urbana-Champaign, lead
- OCLC; Tufts University; Michigan State, Arizona, Connecticut, Illinois, North Carolina, and Wisconsin State libraries
- Collect Web-based state gov’t agency publications
- Develop tools for selection, acquisition, and access
- Test existing repository architectures (Dspace, Greenstone, FEDORA, Eprints)

## The Next NDIIPP Area of Investment...

- State and Local Governments
- Currently in planning phase
- Plan to award $$ to states through IMLS

## Achievements at End of 5 Years

- A network of 50-75 partners
- Vast archive of at-risk content
- Recommendations to Congress on long-term governance of a national digital preservation

**Notes:**

Over 100 TB of data.

Recommendations to Congress on long-term governance of a national digital preservation p. including costs and areas of future development

## Slide 20