# Presentation: 366738

## What Agencies Should Know About PDF/A-1

**What Agencies Should Know About PDF/A-1**

- April 6, 2006
- Mark Giguere
**mark.giguere@nara.gov**

## Introduction

- Agenda
- Why long term preservation of PDF is an issue
- Overview of PDF/A-1 and the ISO Process
- Discussion of PDF/A-1 Standard and NARA’s Transfer Guidance for Permanent PDF records
- Roles of both PDF/A-1 and the NARA’s PDF Transfer Guidance in Federal recordkeeping
- Conclusion and Questions

**Notes:**
Although never formally identified, the “/A” to most of the people involved in the standards activity represents Archive.  

## Long-term preservation of PDF is an issue

- Wide use of PDF
- PDF is a ubiquitous open format for electronic documents
- Proprietary, but with publicly available specification
- Much important information maintained in PDF
- Permanent archival records, in some cases
- The feature-rich nature of PDF can  complicate preservation efforts

## PDF Not a Suitable Archival Format

- PDF itself is not suitable as an archival format
- Some features not compatible with current archival requirements
- Not necessarily self-contained
- Encryption
- All PDFs are not created equal
- Long-term solution needed
- Permanent archival records, in some cases
- Administrative Office of U.S. Courts initiated idea for an ISO Standard based on PDF (PDF/A)

**Notes:**
For example: 
Can include features not compatible with current archival requirements
PDF documents are not necessarily self-contained.  
Some PDF files depend on system fonts and other content drawn from outside the file.  
As technology changes, these external dependencies can cause information to be lost. 
PDFs can be encrypted and require passwords for rendering
Additionally, because there are many PDF development tools on the market, there is inconsistency in the file format.  
This means that future migration of PDF files could be difficult because archivists won’t necessarily know “what’s under the hood”.  
With so much important information all over the world being maintained as PDF, we needed a long-term solution to ensure that digital PDF documents remain accessible for long periods of time. 
Administrative Office of U.S. Courts initiated idea for an ISO Standard based on PDF (PDF/A)

## Overview of PDF/A-1 and the ISO Process

- Multi-part ISO International Standard
- ISO 19005-1:2005, *Document management – Electronic document file format for long-term preservation – Part 1: Use of PDF 1.4 (PDF/A-1)*
- Part 2 (19005-2) intended to bring PDF/A into conformance with PDF 1.6
- Part 3 (19005-3) intended to address dynamic content (e.g., Java Script)
- And additional future parts, as necessary

**Notes:**
Part 1 based on PDF 1.4, 
The standard is being published in parts so that new parts can be added without obsolescing previous parts.  
PDF/A-1 refers to the format defined by Part 1 of the standard which deals with PDF 1.4
Part 2 and later parts will be based on a later versions of PDF and/or may define archiving requirements for more complex content types. 
Work has already begun on: 
PDF/A Part 2 which will be based on PDF 1.6 (This subsumes PDF 1.5).  
PDF/A Part 3 which may address dynamic content such as Java Script. 
The PDF/A JWG is creating
Application Notes 
Listing of Frequently Asked Questions (FAQs) 
Will be made publicly available to assist developers of PDF/A applications to better understand the requirements of the file format and provide implementation guidance. 

## PDF/A-1  Approach

- PDF/A-1 specifies:
- The subset of PDF components, from the PDF 1.4 Reference), that are either required, restricted, or prohibited, **and**
- How these components may be used by software

**Notes:**
The PDF/A Standard is organized to mirror the PDF 1.4 Reference.  
In developing the standard, the JWG assigned each section of the PDF Reference to a sub-committee of experts who identified PDF components that could complicate archival preservation and prohibited or restricted them. 
For each section, they defined the set of PDF objects that may be used in PDF/A, and in many cases added restrictions to their use.  
They also specified PDF objects that may not be used in PDF/A files.  In some cases, they defined how a PDF/A conformant reader must handle these objects. 
Adobe Systems, Inc., makes the PDF specification publicly available. The PDF/A standard places constraints on its use to make it suitable for the long-term preservation of electronic documents. 
A conforming PDF/A document may include any valid PDF 1.4 feature that is not explicitly forbidden by this standard. 

## PDF/A-1  Requirements

- Disallows or limits features that could complicate long term preservation, and
- Maximizes:
- Device independence
- Can be reliably and consistently rendered without regard to the hardware/software platform
- Self-contained
- Contains all resources necessary for rendering
- Self-documenting
- Contains its own description
- Transparency
- Amenable to direct analysis with basic tools

**Notes:**
Disallows or limits features that could complicate long term preservation, and
Device Independence:  PDF/A requires device independent components so that the static visual appearance can be reliably and consistently rendered and printed without regard to the hardware or software platform used.  The GRAPHICS clause, for example, incorporates requirements from PDF/X to ensure predictable color rendering.  PDF/A also prohibits the use of components not defined in PDF Reference 1.4. 
Self-Containment:  Everything that is necessary to render or print a PDF/A file must be contained within the file.  The FONTS clause requires that all fonts used are embedded in the file.  A PDF/A conforming writer must always embed fonts, and a conforming reader must always use the embedded fonts.  Embedding fonts means that the file will be rendered using the fonts intended and not using fonts residing on the local workstation, for example. The standard warns creators that not all fonts can be legally embedded and that legal restrictions on embedding should be determined.
Self-Describing Files:  PDF/A requires Adobe Extensible Metadata Platform (XMP) be used for embedding metadata in PDF files.  Again, to allow flexibility of implementation, PDF/A provides recommendations for documenting file attributes such as: File identifier, File provenance, Font metadata, and allows non-XMP schemas to be included, as long as they are embedded.  Implementers can use XMP in a variety of ways to include information about electronic records within the file itself.  Having metadata embedded in the file can increase the informational value of electronic documents and enhance the future researcher’s understanding of the document.
Transparency:  Level A conforming PDF/A files provide text “in natural reading order” so that the file can be read with basic text editing tools, such as MS Notepad.  This supports access to the informational content of PDF/A files, even without the benefit of a PDF/A reader. 

## PDF/A-1  Table of Contents 

- 1 Scope
- 2 Normative References
- 3 Terms and Definitions
- 4 Notation
- 5 Conformance Levels
- 6 Technical Requirements
- 6.1 File Structure
- 6.2 Graphics
- 6.3 Fonts

- 6.4 Transparency
- 6.5 Annotations
- 6.6 Actions
- 6.7 Metadata
- 6.8 Logical Structure
- 6.9 Interactive Forms
- Informative annexes
- Annex A - PDF/A-1 Conformance Summary
- Annex B - Best Practices for PDF/A
- Bibliography

**Notes:**
The PDF/A Standard is organized to mirror the PDF 1.4 Reference. 
This is just a flash slide for overview purposes.

## Two Conformance Levels

- Level A - Promotes the creation of PDF/A files with rich semantic and structural information,
- Uses “Tagged PDF” and Unicode character maps
- Level B - Allows less complex files such as scanned images.
- Includes all requirements of 19005-1 minimally necessary to preserve the visual appearance
- Does not require users to define structure or other descriptive information.

**Notes:**
PDF/A supports two conformance levels to promote the creation of PDF/A files with rich semantic and structural information, and to also allow less complex files such as scanned images.  
The two levels of conformance are referred to as Level A and Level B.  
Level A uses “Tagged PDF” and Unicode character maps to preserve the document's logical structure and content text stream in natural reading order, and 
Level B includes all requirements of 19005-1 minimally necessary to preserve the visual appearance.  
While Level A should support a higher level of document preservation service and confidence over time, Level B allows PDF/A conformance without requiring users to define structure or other descriptive information. 

## Annexes of the Draft PDF/A Standard

**Informative Annexes** provide supplemental information including:
- Summary of the PDF structures and components disallowed, required, or limited
- Best Practices for PDF/A-1
- Guidelines for capturing or converting electronic documents to PDF/A-1
- To replicates the exact quality and content of source documents
- Required for compliance with NARA’s PDF Transfer Guidance

**Notes:**
Informative Annexes 
Annex A -  Quick reference of PDF/A requirements
Annex B - Guidelines for capturing or converting to electronic documents to PDF/A - to ensure that resulting PDF/A documents retain their quality and integrity as records.  
To meet this critical archival need, PDF/A capture or conversion processes should replicate the exact content and quality of the source document within the PDF/A file.  Following are examples of specific software requirements that accomplish this:
3⁄4	PDF/A writers should not use lossy compression, subsampling, downsampling, or any other process that either alters the content or degrades the quality of source data in the PDF/A document
3⁄4	Software should not substitute searchable text, based on optical character recognition, for the original scanned text within the bit-mapped image of documents that are scanned to PDF/A from paper or converted to PDF/A from image formats
NOTE	Optical character recognition processes may involve loss of data through imprecise interpretation of scanned characters.

## PDF/A-1 Dos and Don’ts

- PDF/A-1 Dos:
- •	Embed fonts
- •	Device-independent color
- •	XMP metadata,
- •	Tagging

- PDF/A-1 Don’ts:
- •	Encryption
- •	LZW Compression
- •	Embedded files
- •	External content references
- •	Transparency
- •	Multi-media
- •	JavaScript

**Notes:**
PDF/A - Overview of Requirements
Dos 
Embedded fonts, character encoding
Device-independent rendering of graphics
XMP metadata  “Adobe XML Metadata Framework” 
Two levels of conformance
Level A (e.g., Tagged PDF, UNICODE Mapping)
Level B (e.g. No Tagged PDF)
Don’ts
Uniform file format (header, trailer, no encryption)
No LZW - foreign patent issue 
External actions restricted, no dependence on external content 
Transparency prohibited
Readers not required to act on hyperlinks, but may
Annotations restricted, no multi-media 
No Java Script 
Forms based on appearance, not data

## NARA’s Expectations for PDF/A

- PDF/A-1 should address some of the PDF archival issues and enable PDF records to be maintained longer as PDF
- Standard maintained by ISO, not just vendors
- Agencies should implement PDF/A-1 along with records management policies and procedures
- Such as....
- NARA’s PDF Transfer Guidance
- AOUSC’s document management program

**Notes:**
PDF/A-1 should address some of the PDF archival issues and enable PDF records to be maintained longer as PDF 
Standard maintained by ISO, not just vendors 
Agencies should implement PDF/A-1 along with records management policies and procedures 
Such as....
NARA’s PDF Transfer Guidance 
AOUSC’s document management program The intent of PDF/A was not to claim that PDF-based solutions are the best way to preserve electronic documents.  

## How NARA is Addressing PDF 

**Issued PDF Transfer Guidance**
- Allowing agencies to transfer permanent records to NARA in PDF In March of 2003, NARA
**Participating in PDF/A ISO Standard Development**
- To influence the process
- To gain knowledge

## Transfer Format versus File Format

**NARA’s transfer guidance and PDF/A-1 have a similar **
**goal  .....**to ensure that valuable electronic information in PDF is not lost.
**But different purposes:**
**Transfer Format** - NARA’s PDF Transfer Guidance
- Specifies NARA _transfer_ requirements
- Applies to _existing_ and future records in PDF
**File Format** - The PDF/A ISO Standard (PDF/A-1)
- Specifies a subset of the PDF file format
- More format reliability/fewer in “bells & whistles”
- PDF should be maintained longer as PDF (e.g., within agencies)

**Notes:**
The Goal of both NARA’s transfer guidance and PDF/A 
To ensure that valuable electronic information maintained in PDF is not lost 
How they differ in purpose::
NARA’s PDF Transfer Guidance - Transfer Format 
Specifies requirements for transferring permanent PDF records to NARA
Applies to existing and future Federal records in PDF so that NARA can accept and process these records
The PDF/A ISO Standard (PDF/A) - File Format 
Specifies a subset of the PDF file format that is more suitable for long term preservation than traditional PDF
Will allow PDF records to be maintained longer as PDF (e.g., within agencies)
Fewer bells and whistles
Minimized migration requirements

## Scope and Usage 

**NARA’s PDF Transfer Guidance**
- Usage: Instructions on what is required to transfer existing permanent PDF records to NARA.
- Scope
- Applies to permanent records
- PDF 1.0 - 1.4
- Addresses quality criteria, laws and regulations, transfer documentation, NARA contact information
**PDF/A-1 ISO Standard **
- Usage: Programming specification to create and process the file format
- Scope
- Applies to one aspect of long term preservation (i.e., file format)
- PDF 1.4
- Addresses how to use the PDF 1.4 reference to create and process a flavor of PDF that is more amenable to long term preservation.
- Should be used as one piece of the archival puzzle

**Notes:**
NARA’s PDF Transfer Guidance
Usage: Transfer existing permanent PDF records to NARA 
Scope: 
Applies to records in PDF scheduled as permanent
Supplements Federal Regulations
Incorporates PDF versions - PDF 1.0 - 1.4
Additionally includes quality criteria, laws and regulations, transfer documentation, NARA contact information  
Will be upgraded to include later versions of PDF in the future
PDF/A ISO Standard 
Addresses one aspect of long term preservation (i.e., file format) 
Based on PDF 1.4 only. Addresses how to use the PDF 1.4 reference to create and process a flavor of PDF that is more amenable to long term preservation.
Should be used as one piece of the archival puzzle
PDF/A, alone, does not ensure accurate reflection of source material in the creation process.
Successful implementation depends on:
Records management policies and procedures 
Additional requirements (e.g., such as those prescribed in NARA’s  Transfer Guidance).
Compliance with QA processes

## Requirements - PDF/A and NARA’s PDF Transfer Guidance

**Embedded fonts **
- PDF/A-1 and NARA’s PDF Transfer Guidance both require that fonts be embedded
- NARA guidance phases in requirements for workstation resident fonts.
**Encryption **
- PDF/A-1 and NARA’s PDF Transfer Guidance both prohibit encryption
- NARA guidance phases in requirement as long as we can open, view and print

**Notes:**
Embedded fonts 
PDF/A and NARA’s PDF Transfer Guidance both require that fonts be embedded.  
Everything that is necessary to render or print a PDF/A file must be contained within the file. 
Embedding fonts means that the file will be rendered using the fonts intended and not using fonts residing on the local workstation, for example. 
Not all fonts can be legally embedded so agencies must first determine any legal restrictions on embedding fonts they use.
NARA accepts transfers PDFs that do not include fonts resident in operating system, for documents created before March 31, 2003 (the day the transfer guidance was released.)  Provided agencies time to phase-in new requirements.
Encryption 
PDF/A and NARA’s PDF Transfer Guidance both prohibit encryption. This means that access to the file is not reliant upon User IDs and/or Passwords. 
Agencies that require access controls can provide these access controls outside of the file format.
NARA accepts PDFs with encryption as long as we can open, view, and print them, for documents created before March 31, 2003

## Special Features
PDF/A-1 restricts special features 
Embedded files, external links, Java Script
PDF/A-1 promotes tagged PDF as a higher level of conformance
NARA evaluates special features on a case-by-case basis at the time of scheduling

Metadata/Documentation 
PDF/A requires that embedded metadata must be in Adobe XMP
NARA requires transfer documentation (e.g., SF-258), and would evaluate embedded metadata at the time of scheduling

- PDF/A-1 restricts special features
- Embedded files, external links, Java Script
- PDF/A-1 promotes tagged PDF as a higher level of conformance
- NARA evaluates special features on a case-by-case basis at the time of scheduling
**Metadata/Documentation **
- PDF/A requires that embedded metadata must be in Adobe XMP
- NARA requires transfer documentation (e.g., SF-258), and would evaluate embedded metadata at the time of scheduling

- Requirements - PDF/A and NARA’s PDF Transfer Guidance

**Notes:**
Special Features
PDF/A prohibits embedded files because such files rely on external software for rendering.  In the future, some software programs could be unavailable, and the information within embedded files could be lost.
PDF/A also prohibits external links and Java Script so that the file is not dependant on external content or functionality.
PDF/A supports “Tagged PDF” and Unicode character maps to preserve the document's logical structure and content text stream in natural reading order
Because of the diverse ways agencies use PDF files, NARA evaluates the acceptability of these and other special features on a case-by-case basis at the time of scheduling.
Metadata
PDF/A requires Adobe Extensible Metadata Platform (XMP) be used for embedding metadata in PDF files. 
Implementers can use XMP in a variety of ways to include information about electronic records within the file itself.  
NARA requires transfer documentation (e.g., SF-258), and would evaluate the value and acceptability of XMP or other embedded metadata at the time of scheduling.

## Quality Requirements
PDF/A-1 as a file format does not address quality/creation requirements such as exact replication of source material
Informative Annex B - identifies recommended creation guidelines 
Agencies must implement these guidelines to comply with NARA’s PDF transfer guidance
NARA’s PDF Transfer Guidance includes 
quality requirements regarding scanning quality, 
lossy compression 
substitution of characters with OCR’d text

- PDF/A-1 as a file format **does not address quality/creation requirements such as exact replication of source material**
- Informative Annex B - identifies recommended creation guidelines
**Agencies must implement these guidelines to comply with NARA’s PDF transfer guidance**
- NARA’s PDF Transfer Guidance includes
- quality requirements regarding scanning quality,
- lossy compression
- substitution of characters with OCR’d text

- Requirements - PDF/A and NARA’s PDF Transfer Guidance

**Notes:**
As a file format standard, PDF/A would provide for flexibility of implementation. 
However archivists and records managers were concerned that implementers would incorrectly assume that by simply maintaining information as PDF/A, long term preservation would be guaranteed.   For example: PDF/A alone does not guarantee exact replication of source material

Text in the Introduction emphasizing:
For long term preservation of electronic documents, PDF/A must be implemented in conjunction with policies and procedures, including quality assurance procedures to ensure acceptable replication of source material.  
Also....
Informative Annex B, Best Practices for PDF/A, provides recommendations and best practices for capturing or converting documents to PDF/A.
Annex B is consistent with NARA’s transfer guidance which specifies:
- quality requirements regarding scanning quality, 
- Prohibits lossy compression 
- Prohibits substitution of bitmapped characters with OCR’d text

## For records in PDF, agencies need to understand that:
PDF/A-1 is one option for long-term preservation of electronic documents
PDF/A-1, by itself, does not guarantee exact replication of source material
Agencies must implement PDF/A-1 in conjunction with additional requirements to meet NARA standards for transferring permanent records to NARA (i.e., NARA’s PDF Transfer Guidance)

- PDF/A-1 is one option for long-term preservation of electronic documents
- PDF/A-1, by itself, _does not guarantee_ exact replication of source material
- Agencies must implement PDF/A-1 in conjunction with additional requirements to meet NARA standards for transferring permanent records to NARA (i.e., NARA’s PDF Transfer Guidance)

- Take Away

## More Information is Available 

- More information on NARA’s PDF Transfer Guidance on NARA’s Web Site
- http://www.archives.gov/records-mgmt/initiatives/pdf-records.html
- More information on PDF/A on AIIM Web Site
- _[http://](http://www.aiim.org/standards.asp?ID=25013)__[www.aiim.org/standards.asp?ID](http://www.aiim.org/standards.asp?ID=25013)__[=25013 ](http://www.aiim.org/standards.asp?ID=25013)_
- Contact Susan Sullivan at _[susan.sullivan@nara.gov](mailto:susan.sullivan@nara.gov)_

## Questions/Discussion