# Presentation: 912403

## Digital Archives at the National Library of Medicine

**A presentation at the MLA Session**
**Lighting the Path: Digital Repositories in the Real World**
**May 24, 2004**
- by Diane Boehr
- Cataloging Unit Head, National Library of Medicine,
- National Institutes of Health,
- Health & Human Services
- boehrd@mail.nlm.nih.gov

## Scope

- Historical medical works
- The NLM Archive
- PubMed Central

**Notes:**
Today I will be discussing work that NLM is doing for digitizing/archiving initiatives in three main areas: historical medical works being digitized and mounted on the Web by the History of Medicine Division; the NLM Archive for born digital and digitized resources published by the Library on its Web site, that are determined to have permanent value, and a little bit about journal articles being archived in PubMed Central.  There is another major NLM digitizing project, Profiles in Science, which I will not be discussing today
Since Betsy Humphreys and Ed Sequiera will be giving a detailed presentation on Wednesday afternoon on e-journal archiving, I will not be talking about that aspect of PMC, and will instead focus on the planning and implementation of the library's other two initiatives and the effects all three of these projects have had on the Cataloging Section's workload and workflow. 

## Considerations as you begin a project

- It will take much longer than you anticipate
- You will learn a great deal about topics outside your normal work duties
- Be willing to take baby steps and make a start
- It is very rewarding to see the fruits of your labor

**Notes:**
If you wait until every detail is determined and in place, nothing may ever happen.

## HMD Projects

- Historical Anatomies
- Medicine in the Americas

## Historical Anatomies

- http://www.nlm.nih.gov/exhibition/historicalanatomies/home.html
- Provides high-resolution downloadable scans of selected important images from illustrated anatomical atlases dating from the 15th to the 20th century
- Titles and images selected by Michael North, Head of Rare Books and Early Manuscripts

**Notes:**
A site designed to broaden access to NLM's collection of historical images 

## Historical Anatomies

- Consists of large JPEGs and** **zoomable digitized images from the books and a brief bibliographical and historical introduction to each title

## Technical details

- The imaging for this project is contracted out
-  The contractor makes archival quality TIFF files (800 ppi resolution) and from that, thumbnail and JPEG images are made for the site, using Adobe Photoshop
-  Zoomifyer Pro is used to create the pan and zoom images
- The TIFF files are backed up on CD-ROMs

**Notes:**
Zoomifyer Pro was selected since it is an inexpensive, off the shelf, software product 
CD-ROM back-ups will be stored in Iron Mountain along with our master microfilms and motion pictures. 

## Search and retrieval

- Individual images do not have any metadata associated with them at this time
- Bibliographic citations on the site match the LocatorPlus records
-  As the focus of the site is selected individual images from the books, rather than the entire text, there are currently no links from the LocatorPlus records for the individual titles to images on the Web site

**Notes:**
Metadata may be a future addition to the site.
This project was done independently, without input from the Cataloging Section. 
Current NLM policy on linking is to provide links only when they lead to full text.

## Sample screen

## Medicine in the Americas

- Monographic original source materials on the development of medicine in New World published prior to 1914 are being digitized in their entirety
- (http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=Books)

## Technical details

- Digitizing is being done in-house
- Books are scanned, and from the initial scan a photocopy and a TIFF file are created
- Photocopies are scanned to create OCR Word text files, which are then manually reviewed and cleaned up to create a searchable, downloadable PDF text** **in modern font
- TIFF file is used to create the typeface and layout of the original published work

## Technical details

- Mounting of these texts on the Web and the XML coding of the Word files done using the NLM Bookshelf platform
- Bookshelf developed by NCBI for medical texts supplied by publishers in SGML, or other desktop publishing formats
- Platform has an existing template that allows the record creators to easily input metadata without needing to know XML

## Search and Retrieval

- Bookshelf site only supports keyword searching
- Standard bibliographic data from LocatorPlus and brief historical data is included with the text
-  Catalog records have hot links to the Bookshelf site

**Notes:**
Therefore standardized headings and terminology can be used in the keyword searches for better retrieval.

## Slide 14

**Notes:**
Six titles are currently available on the site.  This is the Short Contents view.
The Search feature searches all the historical works as if they were one title.  Currently no way to limit to one specific title.

## Slide 15

**Notes:**
Full contents and PDF modern font view

## Slide 16

**Notes:**
A view of the bibliographic information.  Scrolling down will get you to MeSH subjects.

## Slide 17

**Notes:**
TIFF original font view

## Timeframes

- Both projects went from planning to implementation in about one year, although both projects will be adding more material to their sites
- Use of standard, off the shelf products or existing  technologies made implementation easier

**Notes:**
For Medicine in the Americas use of existing NCBI technology meant a lot of procedural issues were already resolved and the biggest problem was finding a scanner to make scans at the desired resolution. 

## NLM Archives

- A site to store material of permanent value that has been published on the NLM Web site, but is now outdated or superseded
- Searchable, yet clearly distinguished from current material

**Notes:**
Archiving NLM's own electronic resources has been a much slower project to get off the ground, because we were developing so many of the concepts and procedures from the ground up.   However, I am glad to announce that after almost 5 years of planning, the NLM archive is now a reality, and NLM has begun developing the policies and procedures that will assure that resources of permanent value on our Web site will be remain available 

## What do we mean by permanent?

- Three aspects to permanence were identified:
- 1) Identifier validity: The extent to which the given name or identifier will always provide access to the same resource
- 2) Resource availability: The extent to which a given resource is guaranteed to remain available in electronic form
- 3) Content invariability: The extent to which the content of the resource could change

**Notes:**
#3) Is the resource subject to revision or replacement; is it static, stable, or growing in content? 

## NLM Permanence Ratings

- Four categories of permanence have been defined:
- 1)  Permanent, unchanging content:  NLM has made a commitment to keep this resource permanently available.  Its identifier will always provide access to the resource.  Its content will not change.

**Notes:**
From highest to lowest
(e.g. a clinical alert; official minutes)

## NLM Permanence Ratings

- 2)  Permanent, stable content:  NLM has made a commitment to keep this resource permanently available.  Its identifier will always provide access to the resource.  Its content is subject only to minor corrections or additions.

**Notes:**
(e.g.  An online exhibit; fact sheets).  A resource in this category will be moved to the archives any time a major revision is made to the content, and a new version will be issued on the current Web site.

## NLM Permanence Ratings

- 3)  Permanent, dynamic content:  NLM has made a commitment to keep this resource permanently available.  Its identifier will always provide access to the resource.  Its content could be revised, replaced.

**Notes:**
(e.g.  a database such as PubMed; training manuals).  Resources in this category remain permanently available on the NLM Web site, but the version you look at today may look different in the future or contain new material.

## NLM Permanence Ratings

- 4)  Permanence not guaranteed:  NLM has made no commitment to retain this resource.  It could become unavailable at any time.  Its identifier could be changed.

**Notes:**
(e.g.  General announcements,  FAQs)

## Workflows

- Permanence ratings are assigned when a resource is promoted to the NLM Web site
- Default permanence ratings are generated based on the category to which the resource belongs
- Resource creators use a template which adds basic metadata, in addition to the category and permanence rating

**Notes:**
Additional metadata is added to the record to enhance discovery and retrieval

Permanence ratings can be viewed on the bottom of the resource, above the footer
Currently 30 categories of possible document types available

## Templates

- Metadata input template is a feature of TeamSite, our Web content management software
- No knowledge of HTML is needed to use these templates
- Minimal set of required fields, with default values or drop-down menus supplied wherever possible

## Required metadata

| 1) Title |  7)   Rights |
| --- | --- |
| 2) Heading |  8)   Contact e-mail |
| 3) Date first published |  9)   Language |
| 4) Date last modified | 10)  Document category |
| 5)  Next scheduled review date | 11)  Permanence level  |
| 6)  Publisher | 12)  URL |

**Notes:**
Optional: Subjects
      (Keyword or MeSH) 
Record creators may optionally supply subject terms, either keywords or MeSH to the initial metadata set.  Permanence guarantor (NLM) is automatically supplied by TeamSite for all resources.  
Date first published and date last modified automatically comes up with today’s date on initial record creation.  After the record is saved, date first published is locked.  Next scheduled review date is always set for one year from today.  Publisher (NLM) and URL are automatically generated.  Language defaults to English.
Drop down menu for document categories, which triggers a default permanence rating.

## Slide 28

**Notes:**
A screenshot of the TeamSite template

- The NLM metadata set is based on Dublin Core, with some local adaptations
- The full scheme may be seen at
- http://www.nlm.nih.gov/tsd/cataloging/metafilenew.html

## Workflows

- Every resource has the minimal metadata assigned by the resource creator
- Permanent resources are routed to the Cataloging Section
- Complete MARC bibliographic records are created
- Includes standardized access points, including MeSH and an NLM classification number
- Accessible in LocatorPlus
- Distributed to the utilities and other NLM licensees.

## Workflows

- The enhanced metadata created in Cataloging is then added back to the header information of the online resource
- Preliminary metadata and the enhanced versions can be seen by clicking on "View source"

**Notes:**
We are still working on the process of getting enhanced metadata back into the headers

## Slide 32

**Notes:**
For example, here is a typical announcement that might appear on the NLM website; a resource that is not permanent.

## Slide 33

**Notes:**
In the bottom left corner of every page you can see a portion of the metadata.

## Basic metadata

**Notes:**
Click on view source and you will see all the available metadata

## Slide 35

**Notes:**
Here is an example of a permanent resource (Permanent, stable content)

## Enhanced metadata

**Notes:**
And here is the enhanced metadata, with MeSH, personal name entries, summary, etc.

## Archive Design

- Separate, distinct, but integral part of the NLM Web site
- Searchable with standard NLM search software: Mindserver from Recommind

**Notes:**
The archive itself has been designed as a branch of the main NLM Web site.  After investigating many possible archive models, NLM determined that the best way to ensure proper migration of all resources and allow searching and retrieval of archived items, was to keep the archive as a separate, distinct, but integral part of the NLM Web site 

Mindserver is also used to search the main site.

## Archive contents

- Out-of-date resources--older material that was once up on the site, but is no longer of current interest
- Earlier versions of current documents that have undergone major revisions

**Notes:**
When an updated version of permanent, stable resource is promoted to the Web, the existing version is moved to the archive.  If a document undergoes several revisions, users will be able to find each of the earlier versions stored in the archives and trace the entire history through the use of the Replaces and Replaced by links.

## Slide 39

## Slide 40

## Slide 41

## Still to come

- Archiving non-HTML files, such as PDF, video and audio clips, etc.
- Archiving resources from areas in the library which do not get promoted through TeamSite

**Notes:**
The  NLM archive is a case where we have had to take the baby steps I mentioned earlier, in implementation. Right now, procedures are in place to only to archive HTML documents. 

Because we currently depend on TeamSite for the templating and archiving procedures, resources from areas of the library that do not use TeamSite are not yet included in the process, although our long term goal is to include all of the library's electronic publications. 

## Impact on Cataloging

- PubMed Central (PMC)
- A bibliographic record must exist in the NLM catalog before a journal is added to PMC
- Records must be created if the title is not already in the catalog
- Downloaded from OCLC
- Skeletal record created from local template
- High-priority, 24 hr. turnaround time
- Records are then fully cataloged

**Notes:**
Since almost all of the systems in use throughout the library rely on data in the bibliographic records in the ILS to extract the data they need, the responsibilities of the Cataloging Section have expanded with these new initiatives. 
Journals cannot be added to PubMed Central (PMC) until a bibliographic record is entered in the ILS.  Therefore Cataloging is among the first to be notified when a new title will be added.  If there is not already an existing record for the journal title in the NLM catalog, a technician in the Catalog Maintenance Unit creates a skeletal record, either by downloading existing copy from OCLC or using a template developed in the Section.  Since no work can proceed until at least a skeletal record exists in LocatorPlus, this task is given a very high priority and we try to complete it within 24 hours.  The title is then passed on to a cataloger who will complete the descriptive cataloging and assign MeSH terms and any additional necessary access points.

## Impact on Cataloging

- PMC
- If the title is already in the catalog, holdings must be updated
- Indicate the title is available in PMC
- Range of issues
- Any embargo periods

**Notes:**
Because PMC is continually adding not only new titles, but expanding the backfile of titles in the database, the holdings information in the LocatorPlus records is updated on a regular basis, as earlier issues of a title are added to PubMed Central.  This is another new, high-priority task for the Maintenance Unit.  Keeping this information up-to-date not only publicizes the data to the public, but ensures that ILL requests for articles from these titles will be identified as freely available to requesting libraries.

## Impact on Cataloging

- NLM Archive
- Cataloger creates core level MARC records for any new resource on the NLM Web site rated Permanent
- View the site, as well as utilize metadata supplied by record creator for descriptive data
- Supply MeSH and NLM classification
- Establish authorized name headings in the national authority file
- Transfer this enhanced metadata back to the resource

**Notes:**
Cataloging is responsible for creating a core-level MARC record in LocatorPlus for any new resource which goes up on the NLM Web site with a rating of Permanent (Dynamic, Stable, or Unchanging).   The Cataloging Section is automatically notified whenever a Permanent resource is promoted to the Web, and the information is given to a cataloger who creates the bibliographic record, using the metadata provided by the record creator, as well as viewing the actual resource.  These are equivalent to any other record in LocatorPlus, with standardized name and title access points, classification numbers, and subject headings using MeSH. 
Right now, transferring the enhanced metadata back to the resource itself is a manual process that must also be done by the Cataloging Section, although we hope we might be able to automate this process in the future, if we have a volume of material that would justify writing the programming to support this.

## Impact on Cataloging

- HMD projects
- Minimal impact on Cataloging
- Books being digitized already have records in the catalog
- HMD has its own cataloging staff who can make links between existing catalog records and digitized material

## Impact on Cataloging

- Despite the increased workload, we think archiving projects are enhanced when catalogers are involved in the projects
- Catalogers increase their knowledge by becoming involved in these projects

**Notes:**
Catalogers learn more about the library as whole, becoming involved in areas outside of cataloging, and achieve personal growth by learning about topics beyond traditional cataloging.