MEMORANDUM

To: Chair, Interagency Committee on Government Information,\
Office of Management and Budget,\
Executive Office of the President

From: Chair, Categorization of Government Information Working Group

Subject: Transmittal of Subsection 207(d) Recommendations

On behalf of the Categorization of Government Information Working Group,
I am pleased to transmit our recommendations to the Interagency
Committee on Government Information (ICGI). These recommendations are
offered pursuant to the E‑Government Act of 2002, Section 207
(\"Accessibility, Usability, And Preservation of Government
Information\"). Subsection 207(d)(1) of that Act requires ICGI to
submit recommendations to the Director of the Office of Management
and Budget (OMB) by December 17, 2004 on:

> \"the adoption of standards, which are open to the maximum extent
> feasible, to enable the organization and categorization of Government
> information in a way that is searchable electronically, including by
> searchable identifiers; and in ways that are interoperable across
> agencies\"; and on: "the definition of categories of Government
> information which should be classified under the standards".

The provisions in Subsection 207(d) of the Act are understood to require
ICGI recommendations in four distinct areas: a definition of which
Government information should be categorized; a standard for searchable
and persistent identifiers to be applied to items of categorized
government information; a standard set of categories (i.e.,
\"bibliographic attributes\") for categorizing government information;
and, an open standard for interoperable search of government information
so categorized.

The four areas were pursued in parallel, using an iterative approach to
maximize public input and review. First, in each of the four areas, a
Statement of Requirements was drafted, posted for public review, and
revised based upon comments received. An initial draft Recommendation
was then developed, posted for public comment, and revised based upon
comments received. Attached hereto are the four detailed Recommendation
documents as they were revised, plus summaries of comments received for
Recommendation drafts.

The following are the eight key recommendations appropriate for action
by the OMB Director, given here as concise statements abstracted from
the detailed documents.

**Recommendation 1. Define \"categorizable Government information\"
under Section 207 --**\
The following definition is recommended:

> Categorizable Government information means any information product,
> regardless of form or format, that a U.S. Federal agency discloses,
> publishes, disseminates, or makes available to the public, as well as
> information produced for administrative or operational purposes that
> is of public interest or educational value. This includes information
> created or exchanged within or between agencies. Not included are
> Federal government information holdings explicitly provided in law as
> so constrained in access that even a reference to the holding is kept
> from public view for a specified period of time.

**Recommendation 2. Immediately adopt the \"Handles\" standard for
identifiers --** In the short run, the U.S. Federal Government should
adopt Handles as a standard approach to meet immediate searchable
identifier requirements. Handles are the most widely deployed and
functional scheme for searchable identifiers. Adoption of Handles
requires management of the top level naming authority and support of the
Federal Handle Registry infrastructure. The naming management should be
provided by those organizations managing the Domain Naming System for
the U.S. Federal portion of the global Internet, i.e., the General
Services Administration (GSA) and the Defense Information Systems Agency
(DISA). In addition, the Government Printing Office (GPO) is well
positioned to perform these functions under the National Bibliography
Program. []{.mark}

**Recommendation 3. Eventually adopt the Uniform Resource Name standard
for identifiers --** In the long run, the U.S. Federal Government should
adopt an approach to searchable identifiers based on the Uniform
Resource Name (URN) and including a Resolution Discovery System. The URN
approach allows integration of existing unique and persistent identifier
schemes, e.g., Handles, International Standard Book Number (ISBN),
Digital Object Identifier (DOI), and Persistent URL (PURL). The URN
approach also accommodates future schemes, yet to be defined. Initial,
informal discussions with industry experts indicate that the
implementation of a Resolution Discovery System and its integration into
existing browsers is very feasible, if the government sends a clear
indication that it is desirable.

**Recommendation 4. Assert the essential need for continuity in
bibliographic practice --** It is a requirement under long‑standing law
and policy that all Federal government information resources must have
an appropriate bibliographic treatment so that they are citable, whether
the resource is electronic or otherwise. All major styles for
bibliographic citations now include guidance on bibliographic citations
for electronic information resources. With contemporary information
technology, agencies have flexibility in achieving these requirements
for bibliographic treatment at reasonable cost. Human catalogers are now
supported by information technology that can reformat, generate, or
suggest some of the bibliographic values and so deliver information
citations as part of the search and retrieval process.

**Recommendation 5. Assert the ongoing need for diligence in
cataloging** - Advances in information science and technology are
providing new tools to enhance the efficiency of machine-aided
cataloging and thereby supplement traditional techniques for
bibliographic cataloging. However, regardless of how technology may
change the manner in which cataloging is accomplished, the fundamental
responsibilities of agencies are unchanged. Technological change and the
E‑Government Act must not be viewed as justification for Federal
agencies to be less diligent or precise in cataloging government
information.

**Recommendation 6. Support automated collection of electronic
government information** - Federal agencies must be responsive to the
needs of intermediaries providing public access to government
information, including other government agencies as well as external
parties of all kinds. Intermediaries use many tools to develop
searchable collections of government information, including
the ubiquitous "Web crawlers", news feeds, and government directories.
To help intermediaries develop collections at reasonable cost, Federal
agencies should help lead the development and adoption of standards for
electronic information metadata and network protocols. Metadata formats
vary by technology and file format, and agency decisions on particular
elements of metadata also vary by content of the file and status under
law and policy (e.g., security classification, Federal record, etc.).
Federal agencies should have a simple tool for authors and other
catalogers to decide what metadata to support.

**Recommendation 7. Establish minimum categories for search services** -
The citation of an information item is composed of bibliographic
attributes (title, author, date, etc.), and these should be supplemented
with various categories that are useful for search and retrieval
(subject, place, audience, identifier, etc.). To satisfy the needs of
searchers for government information, all interoperable search services
for Federal government information must be capable at minimum of
searching by five distinct conceptual categories: Identifier, Subject,
Agency Creator, Title, and Publication Date. In addition, search
services for Federal government information should provide for searching
by other criteria, including: Place, Audience, and Keywords. As noted in
the detailed recommendation on interoperable search across agencies,
such a capability of a search service is separate from whether specific
values of searchable categories may exist for each item in any given
collection. For instance, although a search service may be capable of
searching for items by Title, that service may be used on a collection
of aerial photographs wherein the items do not have individual titles.

**Recommendation 8. Adopt the ISO 23950 international standard for
interoperable search** - It is recommended that the U.S. Federal
Government adopt the ISO 23950 international standard to enhance
interoperability among networked systems that aid in the discovery of
and access to government information. This mature search and retrieval
standard allows for traditional bibliographic catalogs to be integrated
as appropriate with electronic information resources of many kinds and
different formats. Although this recommendation does not require new law
or policy, the role of GSA should be clarified as necessary to assure
that future search technology procured by Federal agencies will be
compliant with the ISO 23950 international standard. The oversight role
of OMB under existing policy should also be updated to reflect the
effects of the E-Government Act and other changes.

After ICGI provides recommendations in December 2004, Section 207(d)
specifies that the OMB Director will issue policies by December 2005
requiring agencies to use standards. This includes "determining
priorities and developing schedules for the initial implementation of
the standards by agencies." It is suggested that OMB develop or update
agency implementation documents during 2005 for interoperable search and
during 2005-2006 for the other standards. In December 2006 OMB could
then evaluate agencies in their initial compliance with such policies.

Before closing, I note that it has been my honor to lead the
Categorization of Government Information Working Group since its
inception in January, 2004. I would like to thank the many people who
contributed significantly to the efforts of the group. In particular,
the following individuals deserve special thanks for their diligence,
leadership, and accomplishments:

> Gil Baldwin, TC Evans, and Gretchen Schlag of the Government Printing
> Office\
> Jim Erwin of the Defense Technical Information Center\
> Bob Haycock, at OMB through April and now in Department of the
> Interior\
> Andy Hoskinson of the Industry Advisory Council\
> Richard Huffine of the Environmental Protection Agency\
> Lucian Russell of Computer Sciences Corporation

/signed/

Eliot Christian

Chair, Categorization of Government Information Working Group

Attachments:

Attachment A. Definition Recommendation - as revised after public
comment\
Attachment B. Identifier Recommendation - Summary of Comments Received\
Attachment C. Identifier Recommendation - as revised after public
comment\
Attachment D. Categories Recommendation - Summary of Comments Received\
Attachment E. Categories Recommendation - as revised after public
comment\
Attachment F. Interoperable Search Recommendation - Summary of Comments
Received\
Attachment G. Interoperable Search Recommendation - as revised after
public comment

Categorization of Government Information (CGI) Working Group\
U.S. Federal Interagency Committee on Government Information

# Defining What Government Information Is To Be Categorized: Recommendation

# December 3, 2004

## 1. Recommendations

**These recommendations satisfy the requirements set forth in
"Requirements for Enabling the Identification, Categorization and
Consistent Retrieval of Government Information" August 5, 2004,
<http://www.cio.gov/documents/ICGI/CGI-Requirement-040805.doc>**

**1.1** The U.S. Federal Government should adopt a high-level definition
of what Government information is to be categorized.

**1.2** The recommended definition is focused on final, published,
information products, *produced by or for* the U.S. Federal Government:

**Categorizable Government** **Information** means any information
product, regardless of form or format, that a U.S. federal agency
discloses, publishes, disseminates, or makes available to the public, as
well as information produced for administrative or operational purposes
that is of public interest or educational value. This includes
information created or exchanged within or between agencies. Not
included are Federal government information holdings explicitly provided
in law as so constrained in access that even a reference to the holding
is kept from public view for a specified period of time. 

**1.3** While all government information will be not accessible to the
public, awareness of its existence and the applicable restrictions on
access should be.

Cases will occur in which the publishing agency may limit access to
Government information or to the descriptive metadata about certain
products to certain audiences for a specified period of time, due to
security, privacy, or other records management reasons.

**1.4 Granularity of Application**

The intent of these recommendations is to allow agencies some
flexibility in the application of categorization to their information
assets. At a minimum, an agency should be required to categorize
information at the "product" or publication level. Optionally, an agency
may categorize more granular units of publications, such as individual
documents, chapters, pages, etc. In making the initial determination
whether a specific product should be categorized, an agency should
consider if that product is "citable." If it is citable it should be
categorized, assigned a searchable identifier, etc. An agency should
also consider if categorizing a particular product or information asset
supports the agency's core mission.

## Appendices

## 1. Background

## In order for the categorization of Government information to add value for the information user, it should meet several general major requirements: 

## Enhance public access to Government information resources. 

## Render a predictable level of granularity among the search returns from decentralized data sources. 

## Be a realistic mandate for Government entities, many of which operate with less than optimal levels of funding or IT support, to carry out. 

## Be compatible with existing information characterization and retrieval mechanisms.

## Be flexible enough to allow for technological advances in information management, publishing, or discovery and retrieval. 

## The U.S. Federal Government seeks to enhance search interoperability by adopting common standards, as required under the E-Government Act of 2002, Section 207 \"Accessibility, Usability, and Preservation of Government Information.\" Paragraph 207(d)(1) of the E- Government Act (44 U.S.C. Chapter 36) requires that the Interagency Committee on Government Information (ICGI) submit recommendations to the Director of the Office of Management and Budget (OMB) on:

-   the adoption of standards, which are open to the maximum extent
    feasible, to enable the organization and categorization of
    Government information in a way that is searchable electronically,
    including by searchable identifiers; and in ways that are
    interoperable across agencies;

-   the definition of categories of Government information which should
    be classified under the standards; and

-   determining priorities and developing schedules for the initial
    implementation of the standards by agencies.

### 2. Alternative Definitions Considered and Implications

### 2.1 Government Publication

Historically several relevant definitions of Government information have
been codified. One of these definitions is that of "Government
publication" found at 44 U.S. Code 1901, the governing statute for the
Federal Depository Library Program:

> As used in this chapter "Government publication" means informational
> matter which is published as an individual document at Government
> expense, or as required by law.

This language, derived from the paper documents era, excludes the growth
areas of Federal electronic information. Entire categories of Government
information, such as dynamic data, audio or video files, statistical
data, remote sensing data, and more are ignored by a definition that
emphasizes the fixed "documentary" nature of legacy print products.

### 2.2 Federal Record

Another relevant statutory definition is that for Federal records, found
at 44 USC 3301:

> ... "records" includes all books, papers, maps, photographs, machine
> readable materials, or other documentary materials, regardless of
> physical form or characteristics, made or received by an agency of the
> United States Government under Federal law or in connection with the
> transaction of public business and preserved or appropriate for
> preservation by that agency or its legitimate successor as evidence of
> the organization, functions, policies, decisions, procedures,
> operations, or other activities of the Government or because of the
> informational value of data in them. Library and museum material made
> or acquired and preserved solely for reference or exhibition purposes,
> extra copies of documents preserved only for convenience of reference,
> and stocks of publications and of processed documents are not
> included.

This language underlies the work of the National Archives and Records
Administration (NARA) safeguarding the records on which the American
people depend for documenting their individual rights, for ensuring the
accountability and credibility of their national institutions, and for
analyzing their national experience. Today more of these records are
being electronically created and maintained than ever before, and NARA
anticipates exponential growth in the number of electronic records to be
maintained and made accessible in the coming years. This statutory
definition includes potentially billions of email messages and other
work products.

### 2.3 Public Information

A manageable middle ground is needed which, while recognizing the need
to protect national security interests and personal privacy rights, is
sufficiently broad to encompass information dissemination formats yet to
be invented, but focuses on published information. Such language is
found in the 44 USC 3502 definition of public information, at paragraph
12:

> \[T\]he term "public information" means any information, regardless of
> form or format, that an agency discloses, disseminates, or makes
> available to the public.

A consequence of adopting this definition could be to exclude from CGI
information products that were produced for an internal agency audience,
but that are also of public interest. This concept is codified in 44 USC
1902, which requires that:

> Government publications, except those determined by their issuing
> components to be required for official use only or for strictly
> administrative or operational purposes which have no public interest
> or educational value and publications classified for reasons of
> national security, shall be made available ... for public information.

## 3. Assumptions and Constraints

The goal of agreeing upon, and ultimately implementing, a definition of
what information is to be categorized, is to enable users to obtain
comprehensive results when searching for Government information.

### 3.1 Scope of Definition

Searchers of government information need to find tangible resources
(i.e. printed documents, maps, CDs, or DVDs) as well as intangible
(online electronic) resources produced by or for the Government. The
definition of resources to which categorization is applicable should not
be so all-encompassing as to be unmanageable. For that reason it is
recommended that information products *about* the Government, such as
television news coverage of Government activities, be excluded. For
similar reasons, applying categorization to objects *owned by* the
Government, or owned by other parties and *loaned to* the Government,
such as museum artifacts, should be excluded. An overly broad definition
of Government information risks creating a requirement so burdensome to
the Government that the goal of improved public access will be
jeopardized.

**3.2 Limited Exclusion for Restricted Information Resources**

The Federal government generally does not constrain access to or use of
its holdings and the data and information are in the public domain. Yet
there are a range of constraints that may apply to any particular
holding.

Use constraints such as copyright restrictions may apply in certain
cases specifically allowed under law, such as patents.

Access constraints may apply to certain security classified information,
proprietary information, personal information, litigation-related
information, and other particular cases.

For example, there is certain information for which access is restricted
to authorized public citizens such as (1) Information restricted to
private citizens eligible to receive that data, (2) information limited
to government contractors, (3) information limited to state and local
governments. It is important that these types of information also be
within the scope of the recommended definition.

Even when information may be withheld from disclosure, publication, or
dissemination the public has a right to know about its existence. The
only information out of scope for this discussion are those few Federal
government information holdings explicitly provided in law as so
constrained in access that even a reference to the holding is kept from
public view. 

## 4. Review Process Used

### 4.1 Review Process

GPO's initial document "Defining What Government Information Is to Be
Categorized" was drafted following an initial meeting at GPO on March
16, 2004, and was posted to the Web on March 30, 2004. This initial
version appears at <http://www.gpoaccess.gov/cgiwg/pdf/cgiwgroup.pdf>.

GPO solicited public comments by sending an invitational email to
various audiences, including:

-   American Library Association

-   American Association of Law Libraries

-   Association of Research Libraries

-   CENDI

-   Depository Library Council to the Public Printer

-   Interagency Committee on Government Information

Public comments received were considered during the process of revising
the draft.

**4.2 Comments Received**

GPO has received comments on the Recommendations from:

-   Walter Warnick, April 12, 2004.

-   Patrice McDermott, on behalf of the American Library Association,
    Office of Government Relations, April 29, 2004 and September 2004.

-   Mary Alice Baish, May 17, 2004.

-   Susan Sullivan, on behalf of the Electronic Records Policy Working
    Group (ERPWG), May 3, 2004

-   J. Timothy Sprehe, September 17, 2004

-   Nancy Kremers, Defense Information Systems Agency, November 9, 2004

-   Alan Vander Mallie, International Trade Administration, November 19,
    2004

-   Michael Daconta, Department of Homeland Security, November 19, 2004

-   Tricia S. Wellman, Office of Information and Privacy, Department of
    Justice, November 3, 2004, and subsequent telephone conversations on
    Nov. 29 and Dec. 1, 2004

-   Diane C. Schenk, on behalf of Jerry E. Williams, Small Business
    Administration, December 1, 2004

-   Michael Pasek, Dept. of State, December 1, 2004

-   John McGing, Social Security Administration, December 3, 2004

-   Kim Nelson, Chief Information Officer, Environmental Protection
    Agency, December 3, 2004

The following is a categorized summary of comments on the Searchable
Identifier Recommendation (posted at
http://www.cio.gov/documents/ICGI/recommendation.html ).

### Feasibility and Costs

**Kimberly T. Nelson** (Assistant Administrator and Chief Information
Officer, Office of Environmental Information, U.S. Environmental
Protection Agency) commented

Could either the Government Printing Office (GPO) or the General
Services Administration (GSA) play the role of government-wide resolvers
and name-space registrars? Does this replace the role of persistent URLs
(PURLS) that GPO has been assigning for some time now?

While we support the objective of having a way of finding information
regardless of its specific location/url stability, we question how
expensive this proposed approach will be, not only in developing systems
to do it, but in staff time to attend to the information (metadata)
requirements. We are also unclear what it might mean for our search
engine capability \-- whether the ability to track persistent
identifiers would be a federal search capability, or whether each agency
would be responsible for adopting/installing such a capability. This is
a worry more than an objection, but it does have to do with the
feasibility of implementing the recommendations.

Editor response: GPO or GSA could provide government-wide resolvers and
name-space registration. The recommendation suggests the Defense
Information System Agency (DISA) and GSA provide name-space registration
because of their similar role in the Domain Name System (DNS). While the
recommendation suggests the implementation of Handles as a default
searchable identifier scheme for agencies that have not implemented a
scheme, the overall recommendation accommodates other schemes under the
Uniform Resource Name (URN) umbrella.

Although there will be costs associated with the implementation of
searchable identifiers, they do not seem excessive on a government wide
basis. Currently, commercial concerns are offering searchable identifier
management and value added services, e.g. the maintenance of
relationships between searchable identifiers, for \$1.00 per identifier,
per year. Therefore, the estimate of \$.25 per identifier, per year
seems reasonable considering federal economies of scale and the initial
lack of value adding at the infrastructure level.

Finally, the searchable identifier would only be one more piece of
metadata to be included in search engine indexes.

**Alan Vander Mallie**, (ITA Web Manager & Section 508 Coordinator Web
Services, Office of the Chief Information Officer, International Trade
Administration, U.S. Department of Commerce) commented:

We agree with the five recommendations as stated. However, we encourage
the consideration of additional costs under Section 3 -- Implications.
The cost estimate of \$.3-1M per year may initially cover the Handle
name space management and operation of a robust GHR but some wording and
cost estimates ought to be included for developing and operating local
namespaces or supporting cross-agency initiatives where budgeting may be
a significant factor. There is no doubt that the establishment of a
general naming and resolution framework will help avoid costs and
improve efficiencies for the Federal Government. The question as always
is how to identify funds for projects that are cross-agency in nature,
above the agency level but not at the pinnacle.

Editor response: The final version of the recommendations contains
additional cost information. However, the overall costs are primarily a
function of the number of information objects requiring searchable
identification.

**Eleanor Frierson**, (Deputy Director, National Agriculture Library)
commented:

> 1\) Under the \"Implications\" section: \"Policy may be required to
> specify the type and granularity of the digital objects requiring
> searchable identifiers and associated metadata.\"  We did not know
> what was meant by this statement, particularly the use of the word
> \"granularity\". This document probably should contain a statement as
> to what unit or level of a publication will require a handle. For
> example, NAL has digitized the *Journal of Agronomy*   Can NAL provide
> one handle for the entire run of 25 years, or do we need to provide
> metadata and a handle for each article or each year of the journal
> run?  Factors such as staffing and costs associated with registering
> handles and creating metadata need to be taken into   account in
> determining this policy.

Editor's response: The Searchable Identifier Recommendations were
modified to clarify \"Granularity\" as the \"unit or level of
publication\".

Ultimately, the organization or agency applying the Handle will
determine the content level requiring categorization and searchable
identifiers. In addition, the number of items requiring categorization
and searchable identifiers will have a direct impact on agency
implementation costs. However, on the positive side, widespread use of
searchable identifiers should drastically reduce the number of content
versions requiring storage and management. In addition, the application
of searchable identifiers to content will make that content much more
useful and facilitate value adding. Consequently, agencies may enjoy
increased efficiency and some reduced costs.

> The second concern is who will coordinate this program? We had
> concerns as to how the overall management of handle naming process
> will be administered.  With so many agencies and subunits of agencies
> within the government, this could be a major headache. How many
> management layers will be involved?   What role will each departmental
> library play in this program? What role will each department\'s
> information management and policy staff play in this process?

Editor's response: The recommendations specify that GSA will coordinate
the top level name spaces (for Handles the top level prefixes, e.g.
100.2) for non-DOD agencies and that DISA coordinate name spaces for
DOD. The lower level name spaces will be coordinated within the
agencies, perhaps, by the CIO. This could be done in a centralized, top
down manner. Alternately, it could be done in a distributed manner
similar to the Domain Name System (DNS). GSA and DISA coordinate the top
level DNS name spaces, e.g. dtic.mil. Undoubtedly, the exact name space
assignment process will be agency specific. Of course, within the URN
environment, certain identifiers, e.g. ISBN and ISSN, are, by
definition, unique.    

> The third issue has to do with compliance to this government-wide
> mandate to create handles.  We do not believe this program can be
> successfully implemented unless the e-publishing process includes
> machine-generated metadata or a machine-controlled metadata
> requirement that forces author to create metadata before their
> documents are considered as published.  As well, the handle creation
> process should also be machine controlled and seamless. One can hardly
> be other than skeptical that this program can be successfully
> implemented government-wide unless the registration process is made as
> easy as possible, yet has adequate controls to assure compliance.

Editor's response: Although a tightly controlled e-publishing process
would be optimal, a government-wide, standard process is not feasible at
this point. However, vendors have indicated, that if the government
adopts a standard searchable identifier approach, they will integrate
that approach into their products, e.g. browsers and content management
systems. If so, products could be developed to provide the desired level
of e-publishing control and support.

Joan Hoffman, (Web Content Services Team, OCIO/IRSD) commented:

As stated in the draft document, this type of universal naming standard
can not be adopted until browsers actually can read identifiers
formatted by such a standard. In addition, existing Web content
management systems used in government and the private sector are written
to support the current http URL standard and hence do not support this
standard. This would impose a significant cost burden on government
agencies who are implementing or plan to adopt off-the-shelf Web content
management solutions in the near future.

Editor's response: Browser can currently read Handles when the full path
to the proxy server is provided or a plug-in is utilized. Therefore,
Handles can be implemented immediately and are compatible with existing
Web content management systems. URNs will require the implementation of
a RDS.

Even if the latest browsers were to adopt such a standard, we could not
force users to upgrade instantly and thus would be faced with having to
provide backwards compatibility for possibly a long period of time.

Editor's response: Any browser modification designed to support an RDS
would be made available as a downloadable upgrade.

The standard does not address the issue of session persistence. Some
government information systems require the establishment of a persistent
session between the browser and the server before the server will supply
a requested document. This requires a multi-step process that can not be
codified into a single URN for system security reasons-even though the
information itself is not classified.

Editor's response: In the case of Handles, resolution only provides the
address of the information object. Negotiation for access to the object
is between the user's browser and the target repository. Consequently,
persistent sessions are not a problem. The URN RDS will only identify
the resolver for a particular searchable identifier scheme. Therefore,
the agency should not use any particular resolution scheme that does not
support persistent access.

IGCI made several passing references to the cost of these requirements.
However we recommend that IGCI perform a full cost and alternatives
analysis, obtain OMB certification of the analysis, and identify funding
sources in conjunction with this sweeping recommendation that imposes
significant burden on government agencies and users. Such a cost
analysis should include the cost of upgrading existing document and Web
content management systems used by various agencies of the government,
including any associated security and systems documentation, to support
this standard.

Editor's response: Web content management systems will not need to be
upgraded to support searchable identifiers.

The standard does not address the issue of compliance waivers. There are
likely many cases in which agencies (such as ours) will seek waivers
because of cost or other issues that make implementation impractical.
The waiver process should be fully vetted and documented in advance of
the implementation of this standard.

The suggestion that open source systems be used for this purpose has
significant security and stability implications. It also does not
account for the new documentation requirements for systems required by
OMB. In light of the new FISMA requirements, open source solutions
should be fully documented before being recommended to the agencies. The
cost of this would likely make the overall burden similar to what would
be incurred by obtaining a commercial product.

Editor's response: The Handle system deployed at the Defense Technical
Information Center (DTIC) is compliant with Defense Information System
Agency (DISA) Security Technical Implementation Guides (STIGs).

### Implementation Policy

**Mike Pasek** (US Department of State, IRM/BPC/RG) commented:

The recommendations talk about adopting standards to provide long-term
access, however, there is no discussion of how long users expect or
should expect such information to be available to them, and that\'s
probably because it will vary so much. Records retention, and therefore
availability, is based on NARA approved appraisals of the value and
utility of the government information, including electronic information
available on the Internet. We recognize that retention and disposition
requirements would not directly be affected by established standards for
searchable identifiers, so users need be aware there may be limits as to
how long categories of information may be available on a website. In
other words, what is accessible one day may not be accessible the next.
And there is some recognition of that in the background discussion of
one of the recommendations.

Editor's response: The metadata associated with the searchable
identifier should indicate how long the information object will be
available.

**Owen Ambur**, (Co-Chair xmlCoP
[[http://xml.gov/]{.underline}](http://xml.gov/)) commented:

\"Policy may be required to specify the type and granularity of the
digital objects requiring searchable identifiers and associated
metadata. Policy may also be required to specify the existence of a
minimal set of policy metadata. In addition, policy will be required for
the specification of Handle namespaces. Finally, policy may be required
to specify levels of performance and information assurance.\"

I certainly would be supportive of such policies. However, I believe it
is

\*more\* important simply to automatically \*measure\* and publicly
\*report\* agency performance in providing metadata for each of its Web
pages than it is to establish \*policy\* about it. Ideally, both should
be done, but if a choice is to be made with respect to which to do
first, it is more important to make reality salient (measure and report)
to its stakeholders than merely to express a wish (policy) in writing.
If reality is made salient and the results are important to its
stakeholders, they will take appropriate action. The same cannot be said
with respect to the mere expression of policy.

Editor response: The referenced policy is intended to provide guidance
to agencies on the appropriate level of information object requiring
categorization and the assignment of a searchable identifier.

### Technical Approach

**Michael Mealling**, (VeriSign Research & Development) commented:

The Handle system has some definite scalability and \"root server\"
control issues that will probably cause problems fairly quickly. There
is considerable work being done in the W3C on some thing called RDDL and
SPARQL which have already demonstrated the ease with which they can
duplicate what handle does but using non-proprietary web services and
RDF.

At the same time I realize that some in the USG need something sooner
rather than later. So I\'d modify the recommendation to something like
\"those U.S. Federal Government departments that require immediate
solutions should adopt the Handle System assuming the previous
recommendations of adherence to the URN framework are adopted. Those
departments that are not experiencing an immediate need should work
toward merging any existing namespaces into the URN framework and
deploying the RDS resolution infrastructure.\"

Editor response: The Handle system has been implemented at a number of
federal agencies and within the publishing community as Digital Object
Identifiers (DOIs) without the problems identified by Mr. Mealling.
However, since Mr. Mealling has extensive experience with the
specification of URNs and the Dynamic Delegation Discovery System (DDDS)
his concerns need to be investigated. However, he does endorse the
Handle recommendation as long as it is clear the ultimate goal is the
adoption of a URN framework and the deployment of a Resolver Discovery
System (RDS). This is the essence of recommendations 1-3.

**Chris Ott**, (Director of Government Sales, Epok, Inc.) commented:

I read with interest \"Recommendations on Open Interoperable Standards
for Searchable Identifiers\", but I was somewhat surprised that XRIs
(eXtensible Resource Identifiers) weren\'t mentioned. As you may know,
XRIs are the result of the OASIS effort to define abstract identifiers

Editor response: XRIs were considered. When fully specified, XRIs will
provide an elegant longer term approach that is compatible with the
URN/Handle recommendations.

**Michael C. Daconta**, (Metadata Program Manager, Department of
Homeland Security)

This recommendation suggests the development of a URN scheme, and a
global URN resolver system. This is an old debate that hinges on
people's desire to resolve the "404 problem" with current web
addressing; however, it is debatable whether that problem exists within
government domains that, by nature, are more permanent and where there
also exists satisfactory workarounds. The central point of contention is
that this suggests the need for something new, outside the currently
successfully mechanisms used today in the World Wide Web. The costs of
that new systems have not been satisfactorily justified.

Editor response: Although it is possible to manage Uniform Resource
Locators (URLs) persistently, in fact agencies and the private sector
are not doing it well. Consequently "404 errors" continue to be a major
problem both in the federal and private sectors. In addition, the
primary cost of providing persistent identification is in the management
of the identifiers, whether Handles, URNs, or URLs. The implementation
of searchable identifiers will incur additional costs because of the
additional discipline required to maintain persistent access to the
information objects. Searchable identifiers provide a useful framework
for conceptualizing and managing this requirement.

Additionally, this recommendation did not consider the extensible
Resource Identifier (XRI) recommendation from OASIS. The DHS Information
Analysis and Infrastructure Protection (IAIP) division has suggested a
pilot using XRI for identification of intelligence information.
Unfortunately, XRIs also do not have a resolver mechanism; however, they
do provide a mapping to HTTP URIs.

Editor response: Subsequent to the publishing of the draft
recommendation XRIs were considered. Although XRIs offer an increased
level of functionality over URNs, their specifications are still in
development. In addition, when and if XRIs are shown to be applicable to
the searchable identifier requirement, the URN/Handle approaches are
upwardly compatible with XRIs.

I do not believe this group adequately examined the simpler approach of
just using HTTP URIs for universal identification with a proxy scheme
for non-web resources. An HTTP URI is globally resolveable and with the
establishment of a government-wide PURL resolver can be guaranteed to be
permanent. The creations of complex URN schemes only hamper adoption.
Lastly, the necessity for a resource to be "location independent" is
also debatable in that an HTTP URI that does not point to the actual
resource could point to a metadata record about the resource and thus we
can have a single unified identifier scheme that just returns varying
representations.

Editor response: One of the primary searchable identifier requirements
was to specify a framework that integrates existing identifier schemes,
e.g. Handles, DOIs, PURLs, ISBNs, and ISSNs. Specification of "http" as
the solution does not facilitate that integration. In addition, PURLs as
currently implemented do not provide global resolution.

**Betsy L. Humphreys**, (Associate Director for Library Operations,
National Library of Medicine) commented:

Thank you for the opportunity to review the recommendations for
standards for searchable identifiers. We commend the Working Group for
tackling this difficult challenge. The Web still lacks a working
infrastructure for ensuring unique identification and persistent access
to digital objects. We agree that this is certainly an area in which an
appropriately designed government initiative could aid progress.
However, we have serious concerns about mandating a single technical
approach at this stage and believe that a pilot project that is more
technologically neutral is the best way to determine how best to achieve
the end goal.

A rush to implement an operating resolution system for the U.S.
government based on the Handle system seems ill-advised, for the reasons
we outline below. We think it could hinder rather than assist progress
toward the desirable goal of a universally used resolution/permanence
system. There are too many unanswered questions, with respect to both
technology and sensible policy, for a centrally-mandated and controlled
system to have much chance of success. In contrast, a more modestly
conceived demonstration test-bed could have a very positive impact on
developments.

Specific comments:

Sections 1 and 4-7 of the proposal provide much lucid background
material and pithy technical information that will prove helpful to
general readers. The term \"searchable identifier\" is not defined, but
is described in an earlier document by the same principal author
(Erwin), where it is associated with three properties:

1\) the identifier provides a unique identifier for a digital object;

2\) persistent access is provided for the object;

3\) the identifier allows retrieval of metadata about the object.

We think assignment of a unique identifier is a separate process (with
its own set of problems) distinct from providing persistent access. The
latter is more a question of institutional commitment and longevity than
a technical issue.

Editor response: Agreed. Ideally, the assignment of globally unique
identifiers is separate from their subsequent resolution. However,
searchable identifier schemes have typically addressed both
requirements. In addition, institutional commitment is absolutely
required for the successful implementation of searchable identifiers.
However, relying on URLs and redirection has not been a successful
strategy. In addition, it is a strategy that does not address advancing
technology.

The document uses the term \"Resolution Discovery System,\" or RDS. We
suggest using \"Resolver Discovery Service\" (or System) as the full
form of RDS, since it is the term that appears more frequently in the
literature.

Editor response: Agreed. The recommendations now utilize the term
"Resolver Discovery System".

Among the competing proposed solutions to the problem of using a unique
identifier to retrieve a digital object, most pursue a three-stage
process:

1\) the identifier for an object is presented to a RDS, which returns an
address for a Resolver that holds in its knowledge base the address for
the object in question (this is the \"resolution\" step).

2\) the identifier is presented to the Resolver, which returns a network
address for the digital object.

3\) the object itself is retrieved, using the address provided by the
Resolver.

In the comments below, these will be referred to as Stages 1, 2, and 3.
We will also use \"resolution\" to refer to the entire process (Stages
1-3) as does the proposal (although perhaps this term would be more
appropriately restricted to stage 1 alone).

Our primary concerns are with the proposal\'s \"Recommendations\".
Dealing with each recommendation by number:

1\) The URN certainly has the virtue of being an Internet
quasi-standard, though it is by no means in widespread use.

2\) Implementing a unique identifier resolution infrastructure is quite
a good idea (subject to important caveats, below).

3\) In a resolution system, ready access to metadata certainly seems a
good idea.

4\) This is the most troublesome recommendation. In our view, there is
no good reason to mandate use of the Handle system. The statement that
it is \"the most \[\...\] widely deployed\" Internet searchable
identifier \[\...\] system\" is unsubstantiated; even if true, the
Handle System has been deployed in what is still a relatively small and
well-circumscribed domain (commercial publishing) relative to the larger
problem. The assertion that it is the \"most fully functional\" such
system is potentially misleading, as it has only been exercised within
that restricted domain.

Editor response: Currently, the Handle/DOIs identify over 12M
information objects. In addition, with the implementation of the Advance
Distributed Learning (ADL) Registry, the number of information objects
identified by Handles/DOIs will rise to 22M. In contrast, the next most
widely deployed searchable identifier system, PURLS, have been assigned
to approximately 600K information objects.

The assertion that the Handle system is the most fully functional system
is based on its ability to globally resolve any Handle or DOI through
the GHR. PURL resolution is limited to the local resolver.

There is insufficient experience to provide confidence in the long-term
effectiveness of the Handle system, and there are important risks
associated with committing to it. Although client software is open
source, the server software does not appear to be so. The server code
and protocols are large and complex, with many dependencies, which would
be a liability in keeping up with rapidly evolving Internet technology.
The problem space the server addresses could be provided more simply
using generic, open source methods that are agnostic with respect to the
form of identifier used. (NOTE: Preliminary discussions are currently
underway between the principal stakeholders in URNs, ARKs, and PURLs
with an aim to create an agnostic global infrastructure using these
simple identification methods).

As the document points out, the Handle\'s centralized name space
assignment is not scalable for deployment within the U.S. government,
let alone the world. The same applies to the Handle system\'s
centralized server architecture.

Editor response: The Handle Server software is available without charge
to the government and the server code, in our experience, is easy to
maintain. In addition, CNRI has agreed to distribute the name space
assignment methodology to meet government needs. Finally, the overall
Handle architecture is highly distributed, not centralized. The "root"
GHR only comes into play if the Handle/DOI local resolver is not
explicit. In other words, the GHR is the Handle resolver of last resort.

5\) Implicit in the earlier description of how a fully operational RDS
service would operate, there are at least two levels at which names must
be assigned.

First, someone (for example, a publisher) has to assign names to digital
objects. Second, someone has to assign identifiers to the bodies
assigning names to objects. This second task is presumably the one
referred to in recommendation 5, for which role DISA and GSA are
recommended. It is unclear that these agencies are the optimal
candidates for this task. It is advisable to identify stakeholders in a
global resolution service and the relative numbers of documents to be
handled in different domains. It is quite possible that other bodies,
such as national libraries, will be logical participants. It is not
clear whether this naming authority should be centralized, or delegated
among multiple agencies, perhaps along the lines of the nature or origin
of the documents being considered.

Editor response: Ensuring the global uniqueness of the searchable
identifier is absolutely essential to the scheme's success. Unique
identifiers can be assigned in a centralized manner, e.g. ISSN and ISBN.
However, for the potentially billions of government information objects,
a centralized approach does not scale. Consequently, it is essential
that the name space assignments be tightly controlled at the government
wide level and the agency level. DISA and GSA perform the high level
name space assignment function for DNS in the military and other
government domains. In addition, the Government Printing Office (GPO) is
well positioned to perform these functions under the National
Bibliography Program. At the agency level, it is assumed that the
agencies will control the name space assignment. Consequently, some
agencies may control it through the CIO. Others, through a national
library.

There are a number of significant related issues that would have to be
dealt with in implementing a resolution service, that are not addressed
at all by the recommendations. Although the syntax of URLs may be
well-defined, rules for their use (referred to by some as their
semantics) are not. When attempting to assign unique identifiers to
digital objects, several issues arise:

\(a\) There is debate about the appropriate level of document
granularity to identify uniquely. (touched upon in section 3,
\"Implications\"). There have been serious proposals put forward to
label figures, tables, lists of references, and other distinct
components of a document as individual retrievable objects in their own
right.

\(b\) Although currently active publishers may provide a single source
for digital access to their holdings, many other documents may be
available in multiple versions, languages, and formats, and from
multiple different information providers. It is not clear how to
properly apply identifiers in such instances, nor how to resolve
multiple instances of the same identifier where there may be different
policies, charging schemes, or preferences for access.

(c)It is not clear how stability of content for a digital object can be
assured (authors could potentially edit documents post hoc, for example,
to remove errors. This is the classic problem of \"non-repudiable\"
content, which the digital signature community is concerned with).

Editor's response: These are issues that will need to be addressed by
agency policy. However, the implementation of searchable identifiers
provide an approach for addressing them

Use of existing (and possibly new, additional) metadata standards will
be required for many aspects of a resolution service.

The discussion of costs in section 3 does not mention ongoing
maintenance and production expenses, nor the costs to agencies to
implement the resulting system.

Recognizing the above problems, which by no means constitute a complete
list, but also agreeing that the world needs good resolution and
permanence services, how can government contribute to progress with this
problem? We think it advisable to propose development of a more modest
prototype system. A small number of participants could be selected so as
to provide a variety of document types (traditional print publications,
internal documents, \...) and unique naming schemes. We would not
recommend restricting test bed participants to U.S. government agencies
(which would arbitrarily exclude a wealth of experience available in
other sectors and foreign sites). A government-only system could evolve
in ways that would be unworkable for other communities, which would
likely develop parallel systems. Reconciling such systems could prove
difficult, and if one of the competing systems was widely adapted, the
government would likely end up dropping its own system, at considerable
cost.

As the proposal points out in section 5, the URN framework can support
Handles, ARKS, PURLs, etc., so we recommend designing the system using
URNs, and selecting a limited number of unique identifier schemes to
embed within URNs (perhaps ARKs, Handles, PURLs, and \"native\" URNs,
such as those being used by the Nordic National Libraries). Develop
simple open-source Stage 1 and Stage 2 resolvers. Establish a naming
authority regime. Modify a cross-platform open-source web client (such
as Mozilla) to resolve URNs. Then perform scaling studies of the
resulting prototype, and document the experiences with forging policies
within and between test-bed participants.

Such a test-bed could well answer some of the vexing questions that
should be addressed prior to tackling government-wide resolution. At the
very least, it should help create a coherent list of the formidable
problems that will be encountered in developing a government-wide
resolution infrastructure such as that proposed here, and pave the way
for orderly and controlled progress toward it.

Editor response: Of the searchable identifier schemes mentioned, Handles
is the most functional and widely deployed. Other approaches, e.g. ARKs
and XRIs promise greater functionality. However, they are not widely
deployed. The searchable identifier technology, except for the lack of
an RDS, is not really an issue. Consequently, it is not clear, what
would be learned from a test bed. Implementation of searchable
identifiers will primarily require a change of thinking and a higher
level of discipline in the management of information objects.
Consequently policy and process will be paramount.

### Miscellaneous Issues

**Larry Lannom**, (Director of Information Management Technology,
Corporation for National Research Initiates) commented:

Reading your draft closely...I have one small nit in your description of
the handle system. At several points you reference \'a GHR\' as in
\'operation of a GHR.\' By definition there is only one, since it's the
root of the handle system. I know we\'ve discussed delegation of
authority so you\'re right in the sense that DTIC or someone else could
run what in essence would be the root node of a number of Local Handle
Services underneath it but I know we wouldn\'t call that a GHR because
we trademarked the phrase Global Handle Registry. We don\'t really have
a term yet for what I think you\'re referencing.

Editor response: The recommendations 4-5 were modified to use the term
Federal Handle Registry (FHR).

**Jerry E. Williams** (Acting Chief Information Officer, U.S. Small
Business Administration) asked:

What is the full scope of recommendation #5?  Should \".ed\" and
\".com\" domains be included since these domains may also include
government related data from contractors or consultants?

Editor response: The .edu and .com domains will be accommodated through
the GHR managed by the Corporation for National Research Initiates
(CNRI).

### 1 Background 

One of the early and persistent criticisms of the Internet is the
impermanence of content. Digital objects appear one day only to
disappear the next. Although it is technically possible to maintain
digital content persistence through scrupulous Uniform Resource Locator
(URL) maintenance, in practice, \"Error 404s\" are all too commonplace.

Therefore, researchers proposed the assignment of \"names\" to digital
objects that could subsequently be resolved to actual physical
locations. Under this approach the digital object\'s name would remain
constant while the associated physical location(s) could change. The
Internet community adopted the term, \"Uniform Resource Name\" to
describe this approach.

In October 1995, Keith Moore hosted a meeting at the University of
Tennessee for research groups interested in URNs. One of the key
concepts that emerged from the meeting was that for URNs \"...the
resolution system must be separate from the way names are assigned\"
\[1\]. This concept was dubbed the \"Knoxville Framework\" by the
attendees. The Knoxville Framework provides a mechanism for
incorporating existing naming and resolution schemes into a URN
framework and also encourages the development of new approaches to take
advantage of changing requirements and technologies.

In the meantime, the URN functionality requirements and syntax were
refined by the Internet Engineering Task Force (IETF) in a number of
Request for Comments (RFCs). In addition, several RFCs proposed
approaches for resolver \"discovery\", e.g. Resolver Discovery System
(RDS).

Although the URN and Knoxville Framework approaches are conceptually
attractive and support the long-term incorporation of multiple naming
and resolution approaches, actual deployment has lagged. Currently,
browsers do not support URN resolution. Therefore, several non-URN
searchable identifier schemes have emerged including the Persistent URL
(PURL) developed by the Online Computer Library Center (OCLC), the
Handle System developed by the Corporation for National Research
Initiatives (CNRI), the Digital Object Identifier (DOI) based on the
Handle System and popularized within the commercial publishing industry
by the International DOI Foundation (IDF), and the Archival Resource Key
(ARK) scheme developed by John Kunze and implemented at the University
of California and prototyped at the U.S. National Library of Medicine.
Although proponents point out that their schemes could be made
compatible with the URN framework, all, taking a pragmatic approach, are
based on http.

The public cannot effectively use ephemeral and unauthoritative
government information. Consequently, Congress emphasized the important
role of searchable identifiers in the E-Government Act of 2002. For the
purpose of these recommendations, the functional intent of searchable
identifiers, persistent identifiers, and URNs are conceptually very
similar.

**2 Recommendations**

The U.S. Federal Government should adopt a searchable identifier
standard to provide long-term persistent access to digital government
information through a global naming and resolution framework. The
searchable identification standard should not only be flexible enough to
remain viable as technology changes, but also be specific enough to
provide real, near term functionality and authoritative access to
government information.

• Recommendation 1: The overall searchable identifier standard should be
based on URNs as described in RFC 1737 \[4\] and RFC 2141 \[5\]. In
addition, the standard should attempt to achieve the goals of the
Knoxville Framework, i.e. separation of naming and resolution to
encourage the introduction of multiple, competing, and innovative
approaches.

Ultimately, the searchable identifier framework must be flexible enough
to easily incorporate new naming approaches and changes in technology.
The URN syntax can support the definition of multiple, competing naming
schemes or namespaces, e.g. Organization for Advancement of Structured
Information (OASIS), International Standard Book Numbers (ISBN), and
National Bibliography Numbers (NBN). The URN can also support existing
and future naming schemes by providing a syntax that facilitates the
generation of globally unique identifiers.

• Recommendation 2: A URN Resolver Discovery System (RDS) should be
developed and implemented.

Although URNs provide a flexible syntax for generating globally unique
identifiers, URNs are not resolvable through standard browsers.
Consequently, while URNs are attractive from a flexibility point of
view, they have limited practical utility. Several proposals have been
specified over the years to utilize the Domain Name Service (DNS) to
support URN resolver discovery. RFC 2168 \[10\] suggested use of the
Naming Authority Pointer DNS Resource Record (NAPTR). Most recently,
RFCs 3401-3404 \[6-9\] suggested a Dynamic Delegation Discovery System
(DDDS). To date, an operational, scaleable RDS has not been developed.
However, an operational RDS is technically feasible and essential for
the effective support of multiple URN resolution methods.

Initial informal industry feedback indicates that the development of an
RDS and its integration into popular browser software is very doable.
However, it is only doable, if the U.S. Federal Government sends a clear
signal to industry that such development and integration is essential to
the government\'s information management and dissemination needs.

• Recommendation 3: Naming and resolution schemes should support
standard, intuitive access to digital object metadata.

Searchable identification coupled with metadata supports the effective
use, reuse and value-adding of digital content. The metadata should
include both descriptive and policy metadata. Minimally, policy metadata
should describe what users can expect in terms of a digital object\'s
permanence. However, the policy metadata should also provide information
about digital object differentiation, i.e. alert the user to the
existence of other versions or disseminations of the same logical
object. In addition, the policy metadata should contain information on
digital object modification through parameterization as described by
Kunze \[3\]. Since the appropriate metadata is often genre specific, URN
registration should specify metadata appropriate for a particular URN
scheme.

• Recommendation 4: Although the URN framework and the implementation of
an RDS are recommended to facilitate long-term support of multiple
identification schemes and changing technology, the U.S. Federal
Government should immediately adopt the Handle System as the default
searchable identification scheme for agencies that have not already
implemented a searchable identification scheme.

The Handle System, as described in RFC 3651 \[2\], is the most fully
functional and widely deployed Internet searchable identifier naming and
resolution system. In addition, since it was developed under a grant
from the Defense Advanced Research Agency (DARPA), it is available free
to government organizations. Finally, since it was implemented using
open source software, it is relatively inexpensive to deploy and can be
enhanced to meet future requirements.

However, adoption of the Handle System should include the integration of
the Handles into the URN framework, standard, intuitive access to
digital object metadata, and the modification of the Handle System to
support fully distributed name space allocation. Currently, Handles name
space assignment is centrally controlled. A fully centralized name space
assignment approach will not scale for a government-wide implementation.
In addition, funding and operational support of a Federal Handle
Registry (FHR) will be required to achieve maximum functionality. A FHR
provides the critical capability to resolve any Handle if the local
Handle resolver is not known.

Initial, informal industry feedback indicates that the tighter
integration of Handle resolution support into popular browser software
is, like the integration of an RDS, very doable with a clear indication
of U.S. Federal Government interest. Consequently, support of both an
RDS and the integration of Handle resolution support into browser
software would provide a general naming and resolution framework for
which the Handle System would be an operational reference
implementation.

• Recommendation 5: The U.S. Federal Government should designate
organizations to manage the allocation of Handle namespaces and to
operate a FHR.

Since the Defense Information Systems Agency (DISA) and the General
Services Administrations (GSA) currently manage the allocation of the
.mil and .gov Internet domains, they would be logical choices to perform
the allocation of high level Handle namespaces and to oversee the
operation of a FHR. In addition, the Government Printing Office (GPO) is
well positioned to perform these functions under the National
Bibliography Program.

### 3 Implications

• Policy: Policy may be required to specify the type and granularity,
i.e. unit or level of publication, of the digital objects requiring
searchable identifiers and associated metadata. Policy may also be
required to specify the existence of a minimal set of policy metadata.
In addition, policy will be required for the specification of Handle
namespaces. Finally, policy may be required to specify levels of
performance and information assurance.

• Oversight: The Office of Management and Budget (OMB) should review,
coordinate, and approve the high level Handle name space assignment
scheme at the highest level, e.g. the DISA and GSA, to ensure
consistency and uniqueness across the U.S. Federal Government. In
addition, OMB should ensure the adequate funding and robust operation of
the FHR.

• Policy: Policy may be required to specify the type and granularity of
the digital objects requiring searchable identifiers and associated
metadata. Policy may also be required to specify the existence of a
minimal set of policy metadata. In addition, policy will be required for
the specification of Handle namespaces. Finally, policy may be required
to specify levels of performance and information assurance.

• Oversight: The Office of Management and Budget (OMB) should review,
coordinate, and approve the high level Handle name space assignment
scheme at the highest level, e.g. the DISA and GSA, to ensure
consistency and uniqueness across the U.S. Federal Government. In
addition, OMB should ensure the adequate funding and robust operation of
the FHR.

• Estimated Costs:

URN implementation:

\- URN registration and implementation of an RDS \$ 1.0M

URN Operation:

\- RDS \$ .2M

Handles implementation:

**-** FHR implementation \$ .3M

Handles Operation:

\- Top level Handle name space management \$ .2M

\- FHR operation \$ .5M

\- Agency namespace management\
(per agency annual cost \$ .1M, assuming 200 agencies) \$ 20.0M

\- Sub-agency level policy development, identifier\
assignment, maintenance, i.e. updating the physical\
address of the information object and local resolution\
services, and on-going training (\$.25 per object, per\
year, assuming 20M objects) \$ 5.0M

Total Implementation: \$ 1.3M

Total Annual Operation: \$ 25.0M

Total Annual Operation per agency (assuming 200 agencies): \$ .13M

At the sub-agency level, individual organizations will have to develop
searchable identifier policy, operate local namespaces, maintain
searchable identifier records, create digital object metadata, and
provide training.

• Benefits: Reliable searchable identifiers and associated metadata will
provide the basis for increased functionality and \"value adding\" by
government and commercial organizations. The implementation of
searchable identifiers will also reduce the number of duplicate
information items on the Internet and provide a basis for informed
decision making through the use of authoritative information Web
Services raises the possibility of long-term, standard approaches to
digital content preservation.

• Priorities and schedule: Recommendations 4-5 should be implemented on
a priority basis. Specification of the Handle System as an interim
searchable identification standard, the establishment of DISA/GSA Handle
naming authorities, and support of a robust FHR can be accomplished by
the end of FY06. Concurrently, funding should be provided for the design
and development of an RDS to support existing and future URN
implementations. The RDS will eliminate the need to reference proxy
servers or to develop multiple plug-ins, thereby, greatly improving
searchable identifier usability by the public.

**4 Base Requirements**

These recommendations satisfy all of the requirements identified in the
*CGI Requirements for Enabling the Identification, Categorization and
Consistent Retrieval of Government Information* that was posted for
public comments and revised over the period August - September 2004
\[http://www.cio.gov/documents/ICGI/CGI-Requirement-040805.doc\].

**Major Requirements**

+----------------+-------------------------------------+---------------+
| *              | **Paraphrased Statement of          | *             |
| *Requirement** | Requirement**                       | *Supporting** |
|                |                                     |               |
|                |                                     | **Reco        |
|                |                                     | mmendations** |
+----------------+-------------------------------------+---------------+
| 7.6 (paragraph | Global uniqueness. The same         | 1, 4, and 5   |
| 1)             | identifier will never be assigned   |               |
|                | to two different resources.         |               |
+----------------+-------------------------------------+---------------+
| 7.6 (paragraph | Support distributed naming and      | 1, 2, 4 and 5 |
| 1)             | resolution. Since information is    |               |
|                | created in a highly distributed     |               |
|                | manner, it is essential that any    |               |
|                | identifier scheme support           |               |
|                | distributed naming or               |               |
|                | identification.                     |               |
+----------------+-------------------------------------+---------------+
| 7.6 (paragraph | Support both tangible and           | 1             |
| 2)             | intangible objects.                 |               |
+----------------+-------------------------------------+---------------+
| 7.6 (paragraph | Utilize an open, extensible         | 1, 2, 3, 4    |
| 2)             | architecture. Since persistently    | and 5         |
|                | identified objects will exist into  |               |
|                | perpetuity, the identification      |               |
|                | scheme must be open and adaptable   |               |
|                | to changing technology.             |               |
+----------------+-------------------------------------+---------------+
| 7.7 (paragraph | Provide persistent access to        | 1, 2, 4 and 5 |
| 1)             | digital information objects         |               |
|                | regardless of the current status of |               |
|                | the organization that created,      |               |
|                | named, or previously maintained     |               |
|                | them. In other words, address all   |               |
|                | aspects of the government           |               |
|                | information life cycle, i.e.,       |               |
|                | creation, long-term management and  |               |
|                | access, and permanent preservation. |               |
+----------------+-------------------------------------+---------------+
| 7.7 (paragraph | Be robust. The searchable           | 1, 2, 4 and 5 |
| 2)             | identifier scheme must provide      |               |
|                | highly reliable access to           |               |
|                | authoritative information objects.  |               |
+----------------+-------------------------------------+---------------+
| 7.7 (paragraph | Be compatible, to the greatest      | 1 and 2       |
| 2)             | extent possible, with existing and  |               |
|                | emerging persistent identification  |               |
|                | standards for intangible and        |               |
|                | tangible objects. In addition,      |               |
|                | leverage existing and globally      |               |
|                | unique identifier schemes, e.g.,    |               |
|                | ISSN, ISBN, UPC, etc.               |               |
+----------------+-------------------------------------+---------------+
| 7.8 (paragraph | Be scalable in terms of identifier  | 1, 2, 4 and 5 |
| 1)             | assignment and resolvability.       |               |
|                | Ultimately, billions of objects     |               |
|                | will be persistently identified. In |               |
|                | addition, persistent identification |               |
|                | leads to information aggregation.   |               |
|                | However, information aggregation is |               |
|                | only possible if objects can be     |               |
|                | instantaneously resolved and        |               |
|                | accessed.                           |               |
+----------------+-------------------------------------+---------------+
| 7.8 (paragraph | Be easy to use. In other words, be  | 2             |
| 1)             | resolvable by the end user with     |               |
|                | minimal, or ideally no, additional  |               |
|                | knowledge beyond the object's name  |               |
|                | or identifier.                      |               |
+----------------+-------------------------------------+---------------+
| 7.8 (paragraph | Support multiple machine and user   | 1 and 2       |
| 1)             | interfaces, e.g. browsers and bar   |               |
|                | code readers.                       |               |
+----------------+-------------------------------------+---------------+
| 7.8 (paragraph | Be human readable.                  | 1             |
| 1)             |                                     |               |
+----------------+-------------------------------------+---------------+
| 7.8 (paragraph | Support information object metadata | 3             |
| 2)             | to be used for object discovery,    |               |
|                | digital rights management,          |               |
|                | specification of inter-object       |               |
|                | relationships, and other services.  |               |
+----------------+-------------------------------------+---------------+
| 7.8 (paragraph | Reference digital object metadata   | 3             |
| 2)             | with a standard syntax, e.g.        |               |
|                | urn:ark:100.20/doc?                 |               |
+----------------+-------------------------------------+---------------+

### 5 Alternatives Considered for the URN Recommendations (1-3)

Several alternatives were considered:

• Selection of a single searchable identifier scheme.

• Utilization of the Extensible Resource Identifier (XRI)

• Utilization of the Uniform Resource Identifier (URI).

Single Searchable Identifier Scheme:

Although specific, deployed schemes, e.g. Handles, are required to
support the searchable identifier requirements of the E-Government Act
of 2002, in the short-term, a single searchable identifier approach
cannot leverage existing searchable identifier deployments, meet the
government\'s long-term requirements, and provide an optimal response to
changing technology. Therefore, a generalized approach that integrates
multiple schemes and encourages competition and innovation is essential
for meeting the long-term searchable identifier requirement.

Extensible Resource Identifier:

The Extensible Resource Identifier (XRI) specification is being
developed by the Organization for the Advancement of Structured
Information Standards (OASIS). The OASIS XRI Technical Committee (TC)
approved XRI Generic Syntax and Resolution 1.0 as a Committee Draft in
January 2004. The TC is currently working on version 1.1 of these
specifications. XRIs are designed to extend the functionality of URIs
and provide both persistent and re-assignable identification.
Unfortunately, XRIs are not ready for deployment. In addition, the XRI
specification was developed in the private sector without substantial,
or any, federal government involvement. Finally, while potentially
powerful, the XRI syntax is daunting and would represent a steep
learning curve for agency deployment. Fortunately, XRIs are compatible
with URIs, URNs, and Handles. Therefore, searchable identifiers
initially assigned under one of those schemes would be upwardly
compatible with XRIs when deployed.

Uniform Resource Names (URNs):

Under the \"classical\" view there were two URI types: URLs and URNs. It
was expected that other types would be defined. However, the only other
type ever proposed was the Uniform Resource Citation (URC).

The \"contemporary\" view is that individual schemes or namespaces can
all be URIs \[11\]. Consequently, some searchable identifier scheme
proponents question the utility of registering their schemes as URNs,
preferring the URI designation. Under this approach, \"hdl\" or \"ark\"
would be designated as a URI scheme or namespace. Currently, there are
at least 84 registered and unregistered URI schemes including http, ftp,
gopher, ldap, and urn.

On the other hand, URNs are defined as \"... resource identifiers with
the specific requirements for enabling location independent
identification of a resource, as well as longevity of reference\" \[RFC
3406\]. This is the definition of a searchable identifier. Therefore,
grouping all searchable identifier schemes under the URN designation,
facilitates functional standardization and registration. In addition,
URN grouping, as opposed to the utilization of \"flat\" URI space mixing
identifier schemes of differing functionality, makes the development of
a searchable identifier RDS easier.

The syntax of the URN is as follows:

urn:\<NID\>:\<NSS\> where NID is the Namespace Identifier and NSS is the
Namespace Specific String.

Consequently, this URN approach allows non-interoperable schemes such as
PURLs, Handles, ARKs, and ISSNs to assign unique identifiers within a
global URN framework. For example, an organization using Handles may
assign 100.2/ADA123456 as a unique identifier. An organization using
ARKs could inadvertently assign the same identifier. However, since the
identifier schemes are explicitly identified, there is no ambiguity.

urn:hdl:100.2/ADA123456

urn:ark:100.2/ADA123456

Although the NSS is identical, the different NIDs make these two URNs
globally unique. Consequently, URNs support both flexible naming and the
incorporation of legacy or new unique identification schemes. Finally,
once a RDS is developed, the NID will identify the appropriate
resolution facility.

To summarize, the URN syntax accommodates both existing and future
searchable identifier schemes, supports the distributed assignment of
globally unique identifiers, and simplifies the development and
operation of a RDS.

### 6 Alternatives Considered for the Handles Recommendations (4-5)

Although a URN framework with an RDS is recommended for the long-term,
short-term searchable identification support requires the implementation
and support of an operational searchable identifier scheme that can
provide global resolution.

Currently, the Handles system enjoys the broadest implementation coupled
with the highest level of functionality. The latter statement is based
on the Handle System\'s ability to globally resolve both Handles and
DOIs from any Handle or DOI resolver. In other words, if a specific,
local Handle/DOI resolver cannot resolve a particular resolution
request, that request is redirected to the FHR. Since the FHR is aware
of all registered Handle/DOI resolvers, it can redirect the request to
the appropriate resolver. In addition, since the Handle System, was
developed under a grant from the Defense Advanced Research Agency
(DARPA), it is available free to government organizations. Finally, the
Handle System has been implemented using open source software.
Therefore, it should be relatively inexpensive to maintain.

PURLs also enjoy widespread use and the PURL server software is
available free from OCLC. However, PURLs do not support global
resolution. PURL resolution is limited to a particular PURL resolver.

DOIs, which are based on the Handle concept and software, provide
additional functionality developed by partners known as Registry
Agencies (RAs). However, the RA enhancements are proprietary and
therefore, do not provide a solid basis for a government directed and
managed searchable identifier infrastructure. In addition, over time the
question may arise as to who \"owns\" the searchable identifier, the RA
or the RA\'s customer.

ARKs provide a conceptually elegant searchable identifier approach.
However, ARKs lack a broadly deployed base. ARKs are currently
operationally deployed at the University of California and are being
prototyped at the U.S. National Library of Medicine.

Finally, as previously mentioned, all of these persistent identification
schemes coupled with URN naming and an RDS could provide searchable
identifier functionality.

### 7 Review Process Used

These recommendations were informed by the draft requirements document
comments; RFCs addressing URNs, Handles, ARKS, and RDS; the CENDI
persistent identification white paper; CENDI organizational feedback;
extensive work with the Handles system; the OASIS XRI specifications;
interaction and consultation with the OASIS XRI Technical Committee, the
International DOI Foundation (IDF), and IDF Registration Agencies; and
discussions with experts in the field of searchable identifiers.

**8 Notes and References**

1\. Arms, W., \"Uniform Resource Names, A Progress Report, \" D-Lib
Magazine, February 1996.
\[http://www.dlib.org/dlib/february96/02arms.html\]

2\. Sun, S., Reilly, S., Lannom, L. \"RFC 3651 - Handle System Namespace
and Service Definition.\" \[http://www.faqs.org/rfcs/rfc3651.html\]

3\. Kunze, J. and Rogers, R.P.C. "The ARK Persistent Identifier Scheme,"
Internet Draft, 31 July 2004.
\[http://www.ietf.org/internet-drafts/draft-kunze-ark-08.txt\]

4\. Sollins, K. and Masinter, L. "RFC 1737 -- Functional Requirements
for Uniform Resource Names," December 1994.
\[http://www.w3.org/Addressing/rfc1737.txt\]

5\. Moats, R. \"RFC 2141 - URN Syntax,\" May 1997.
\[http://www.faqs.org/rfcs/rfc2141.html\]

6\. Mealling, M. \"RFC 3401 - Dynamic Delegation Discovery System (DDDS)
Part One: The Comprehensive DDDS,\" October 2002.
\[http://www.faqs/org/rfcs/rfc3401.html\]

7\. Mealling, M. \"RFC 3402 - Dynamic Delegation Discovery System (DDDS)
Part Two: The Algorithm,\" October 2002.
\[http://www.faqs/org/rfcs/rfc3402.html\]

8\. Mealling, M. \"RFC 3403 - Dynamic Delegation Discovery System (DDDS)
Part Three: The Domain Name System (DNS) Database,\" October 2002.
\[http://www.faqs.org/rfcs/rfc3403.html\]

9\. Mealling, M. \"RFC 3404 - Dynamic Delegation Discovery System (DDDS)
Part Four: The Uniform Resource Identifiers (URI),\" October 2002.
\[http://www.faqs.org/rfcs/rfc3403.html\]

10\. Daniel, R. and Mealling, M. \"RFC 2168 - Resolution of Uniform
Resource Identifiers using the Domain Name System,\" June 1997.
\[http://www.faqs.org/rfcs/rfc2168.html\]

11\. Joint W3C/IETF Planning Interest Group. \"URIs, URLs, and URNs:
Clarifications and Recommendations 1.0,\" September 2001.
\[http://www.w3.org/TR/uri-clarification/\]

12\. OASIS XRI Technical Committee. \"XRI Generic Syntax and Resolution
Specification 1.0\", Committee Draft, January, 2004.
\[http://www.oasis-open.org/committees/download.php/5109/xri-syntax-resolution-1.0-cd.pdf\]

Air Force CIO commented:

Thank you for the opportunity to comment on the E-Gov Act draft ICGI
recommendations. AF‑CIO has no input for the attached documents.
Negative response.

DOD CIO commented:

Context paragraph: Please add the following sentence: The
recommendations in this document will contribute to the government data
assets being visible, accessible, and understandable.

Recommendations paragraph 3: It would be helpful if ICGI provides a
sample or prototype to promote consistency.

Recommendations paragraph 4, second subparagraph: Remove \"agency\" as a
qualifier of Creator. There may be data assets held by a government
organization not originally created by, nor for that agency.

Request deleting the word \"publication\". Although publication date is
important, the actual creation date is much more important. It is
important to know when the ICGI paper was created rather than when it
got published on the web. It may be created in August but not published
until next February.

Recommendations paragraph 4, Implications: The third sentence was
deleted as the term \"substantial is relative based on the agency size.
These recommendations do not entail substantial new investments in the
management of government information.

GSA CIO commented:

We propose that the work and efforts of the ICGI and FEAPMO-DRM
communities be aligned, and should be providing the exact same solution,
either to or from the FEAPMO-DRM, since these appear to be activities of
a very similar if not equivalent nature. ...Our comments on the DRM
assert that the information sharing at the core of DRM goals and
objectives need categorical keys that are contextually grounded, and
that assertion is repeated here to enhance the 'subject' as part of the
only context carrying metadata described in the 'minimum categories for
search services' part of the 'Standards for Categorizing Government
Information' document. We describe a policy driven approach to
establishing categories around collaborations and their relationship to
the FEA Reference Models. ...To summarize here, all G2G, G2B, G2C
collaborations are expressed as an instance of an open standard
collaboration metamodel that provides a formality suitable for employing
as a government ontology that describes role interactions including the
information entities that we seek to enable access to exist within this
framework. We show how the entire FEA taxonomy, and the FEAPMO Line of
Sight which traces a specific path through each of the Reference Models
(including the DRM and its' information entities which are analogous to
the information that ICGI seeks to categorize and facilitate access to)
is seen as an 'aspect' of this collaboration. ...If we exploit the
momentum of semantic technologies, then we can consider the impact of
natural language processing on unstructured data to determine standard
syntactical expressions of semantic foundations such as RDF's
subject/predicate/object. From there, we might further derive that these
relationships can be seen as an instance of some ontology, perhaps
expressed in OWL and based on collaborative role interactions, and
exploit existing capabilities to automate reasoning over these to
further distinguish the uniqueness or fitness of information for some
usage scenario.

DOI commented:

in your draft Recommendation on Standards for Categorizing Government
Information, I am pleased to see the following:

\"To satisfy the requirements of searchers for government information,
all interoperable search services for government information must be
capable at minimum of searching by five distinct conceptual categories:
Identifier, Subject, Agency Creator, Title, and Publication Date. In
addition, search services for government information should provide for
searching by other criteria, including: Place, Audience, and Keywords.\"

\"\... anticipate recommendations of the ICGI Electronic Records
Management Working Group in metadata requirements for electronic
records, and recommendations of the ICGI Web Content Management Working
Group for metadata associated with Web pages.\"

I hope the XML subgroup that my colleague Tom McClay is heading up will
specify XML names, definitions, and an XML schema for those metadata
elements \-- particularly those to be associated with (embedded within)
Web pages. I also hope those metadata elements (XML metatags), together
with any controlled vocabularies associated with them, will be made
readily available for discovery and use in an XML registry.

HUD commented:

First, since these recommendations will have to be implemented by a wide
and varied audience, it is essential that they be written in plain
language. Anyone lacking the specific technical expertise in the area of
library science would have a difficult time determining what needed to
be done and why. If those who need to be involved in implementation
cannot understand what\'s to be done, they won\'t do it.

Second, if we understand this task correctly, it is overwhelming. The
thought of trying to categorize all government information in a
meaningful and practical way is daunting, both in time and resources. So
if this is going to be \"doable,\" it would a good idea to break it down
into steps. Here\'s what we could achieve in 3 years. Here\'s what we
could achieve in 5 years. (So, for example, the first step might be to
categorize entire websites\--HUD.GOV is primarily housing and community
development subjects\--then phase in major entry points, then down to
the actual page level of detail.) Pick off the most easily done.

We\'re especially happy to see the recommendation to use Dublin Core
metadata for use on website content. This matches up with the
recommendations from the Web Standards community and is what is being
used by many other governments around the world. It also seems to us
that this is the simplest, yet still robust, way of beginning to
categorize the information on government websites and make that content
easier for citizens to find.

We\'re also supportive of using the vendor-specific metadata (e.g.,
Adobe PDF format, Microsoft Office properties, etc.). Any time we can
leverage existing capabilities we will be more sucessful in the ultimate
outcome.

One potential area of concern is in the first recommendation. While on
the surface, the need for using a standard bibliographic practice in
categorizing government information sounds like a no-brainer, the actual
implementation could well be problematic. At HUD, as in the vast
majority of Federal websites, content creation is done by a very diverse
audience. Coming up with a standardized set of bibliographic references
that everyone understands will be difficult. For example, agreeing upon
the appropriate usage of a particular audience\--for example,
students\--can mean many things to many different people. Subjects will
be even more troublesome. It would be very useful to have resources
identified and concrete implementation strategies suggested to help us
make this really work well.

We appreciate the effort that has gone into this and understand that
this is a huge undertaking. We\'re just looking for a bit of clarity in
the overall vision/strategic direction of the effort, as well as the
practical outcomes.

NASA commented:

Congratulations to the team on putting these together. NASA will support
this effort as appropriate. NASA would like to make the following
comments:

1\. Some thought must be given to legacy content and how the standard
categorization process is applied to this info which does not have
categories.

2\. Make sure these recommendations are easily adopted as part of the
day-to-day operation of content generation or review so that it is not
an undue burden on the Agency.

**Comments:** This draft document will be retired on completion of a
recommendation by the U.S. Federal Interagency Committee on Government
Information (ICGI), in December 2004. Until December 5, 2004, comments
on this document may be sent to the editor, Richard L. Huffine at
<huffine.richard@epa.gov>.

**Background:** These particular recommendations address the requirement
for open, interoperable standards for categorizing Government
information, pursuant to the E‑Government Act of 2002, Section 207
(\"Accessibility, Usability, And Preservation of Government
Information\"). Subsection 207(d)(1) specifies that the ICGI shall
submit recommendations on \"the adoption of standards, which are open to
the maximum extent feasible, to enable the organization and
categorization of Government information in a way that is searchable
electronically, including by searchable identifiers; and in ways that
are interoperable across agencies\" and on "the definition of categories
of Government information which should be classified under the
standards".

**Context:** Under Subsection 207(d)(1) of the E-Government Act, ICGI is
recommending a common definition of what U.S. Federal government
information is to be categorized. These categorization recommendations
would apply to each item encompassed within that definition. All
government information meeting that definition will be identified
persistently, following the specifics in a separate ICGI recommendation
on searchable identifiers. These categorization recommendations include
a category for such identifiers. Federal agencies will also implement
interoperable search services, in keeping with another required ICGI
recommendation. These categorization recommendations will be supported
by that search service standard.

The Categorization of Government Information Working Group (CGI WG) also
supports the efforts of the Electronic Records Policy Working Group and
the Web Content Management Working Group in establishing metadata
standards for the specific types of information products under the
purview of their aspects of the E-Gov Act. All ICGI Working Groups are
encouraged to utilize these four recommendations where applicable in
achieving their goals, and to participate in developing the recommended
decision tree as a tool for further analysis of Web site materials and
electronic records.

**[Recommendations\
]{.underline}**

**1. Assert the essential need for continuity in bibliographic practice
--** Bibliographic practice has its own long traditions, and it is
essential from a public policy perspective that governments build on
that practice in the context of locating government information. It is a
long-standing requirement under law and policy that all government
information resources must have an appropriate bibliographic treatment
so that they are citable, whether those resources are electronic or
otherwise. However, many people perceive that bibliographic treatment of
government information at the Federal level is not accomplished evenly
or comprehensively.

Part of the problem is that government agency publishers of electronic
information are not well aligned with government records managers, nor
are government agency publishers of electronic information well aligned
with the communities of bibliographic practice within or external to
government. At the U.S. Federal level, these are critical issues for the
Government Printing Office (GPO) and the National Archives and Records
Administration (NARA), among others.

The inadequate bibliographic treatment of government information at the
Federal level has been exacerbated with the steep rise in the proportion
of government information that is electronic. Yet, newly available
technology provides some basis for optimism. As described in the
recommendation for interoperable search across agencies, agencies now
have great flexibility in how the requirement for bibliographic
treatment can be achieved in practice. Through the technique of
"semantic mapping", differences in the particular handling of
bibliographic elements is no longer a fundamental barrier to searching
across a diverse collection of resources. Traditional bibliographic
catalogs of information resources can be integrated as appropriate with
electronic information resources of many kinds in hundreds of different
formats. Another very helpful development is that all major styles for
bibliographic citations now include guidance on bibliographic citations
for electronic information resources.

**2. Assert the ongoing need for diligence in cataloging** - Advances in
information science and technology are providing new tools to enhance
the efficiency of machine-aided cataloging and thereby supplement
traditional techniques for bibliographic cataloging. However, regardless
of how technology may change the manner in which cataloging is
accomplished, the fundamental responsibilities of agencies are
unchanged. In particular, technological change and the E‑Government Act
must not be viewed as justification for Federal government agencies to
be less diligent or precise in cataloging of government information.

**3. Support automated collection of electronic government
information** - Federal agencies must be responsive to the needs of
intermediaries using automated tools to enhance public access to
government information. Such intermediaries may be other government
agencies as well as external parties of all kinds. Among the many tools
they use to develop searchable collections are the ubiquitous "Web
crawlers", news feeds, and government directories.

To help intermediaries develop collections at minimal cost, Federal
agencies should lead the adoption of standards for electronic
information metadata and network protocols. In that regard, agencies
should support the Open Archives Initiative Protocol for Metadata
Harvesting (OAI‑PMH). OAI-PMH defines a simple mechanism for collecting
records containing metadata distributed across many networked sources.
The metadata that is collected may be in any format that is agreed upon
by the community, including through not limited to Dublin Core elements.
For publishers of government information, OAI-PMH offers a simple way to
make metadata available to community collections. In the case of a
collection compiled using OAI-PMH and offered by a Federal government
agency, other ICGI recommendations are also involved. The search service
at such a collection would support the recommended interoperable search
service, with the recommended minimum categories. Also, any items in the
collection that meet the recommended definition of government
information would each have the recommended persistent identifier.

Federal agencies should also support the mapping of common metadata
schemes to bibliographic categories, especially the Dublin Core metadata
scheme, as well as many technology-specific metadata (e-mail records,
geospatial data, etc.) or vendor-specific metadata (Adobe Portable
Document Format, Microsoft Office properties, etc.). Dublin Core (ISO
15836) is a basic set of metadata elements designed for electronic
documents. In the context of locating government information at a very
broad level, the bibliographic orientation of this set of elements is
especially relevant. The element set is typically elaborated with
additional restrictions, rules, and interpretations as needed in
particular applications.

While metadata formats vary by technology and file format, agency
decisions on what particular elements of metadata to use vary by the
content of the file and its status under law and policy (e.g., security
classification, Federal record, etc.). Federal agencies should have a
simple tool such as a decision tree to help authors and other catalogers
to determine when a document (or information object) needs to be
categorized and what elements should be used to categorize that
material. This tool would be helpful in addressing the issue of
granularity identified in the initial requirement. It would also assist
in determining the refinements for elements (e.g., What dates are
relevant for a particular document type?), and in determining the
appropriate taxonomies that should be applied. The tool should
differentiate not only among "Web site" and "electronic record" but
across a fairly long list of different information resource types that
need distinct bibliographic treatment as government information. Such a
list should be developed and should identify for each type exactly what
are the required or optional elements. The development of such a tool
should involve records managers, Web site managers, and information
managers from a variety of disciplines within government.

**4. Establish minimum categories for search services** -- As noted
above, bibliographic communities have rich terminologies for
bibliographic cataloging. Yet, a small subset of bibliographic elements
used with citations stands out as so common as to be almost universal.
It is a deliberate choice from a public policy perspective to focus on
that small set of common bibliographic elements in recommending the
minimum categories for Federal search services.

To satisfy the requirements of searchers for government information, all
interoperable search services for government information must be capable
at minimum of searching by five distinct conceptual categories:
Identifier, Subject, Agency Creator, Title, and Publication Date.
In addition, search services for government information should provide
for searching by other criteria, including: Place, Audience, and
Keywords.

As noted in the recommendation on interoperable search across agencies,
such a capability of a search service is separate from whether specific
values of searchable categories may exist for each item in any given
collection. For instance, although a search service may be capable of
searching for items by Title, that service may be used on a collection
of aerial photographs wherein the items do not have individual titles.

**[Implications]{.underline}**

Adoption of these recommendations across the Federal government will
provide the framework that enables categorization. These recommendations
are thus responsive to the requirements of Section 207 of the
E-Government Act, but these recommendations do not themselves entail
substantial new investments in the management of government information.
These recommendations are aligned with other ICGI recommendation as
noted elsewhere in this document. They also anticipate recommendations
of the ICGI Electronic Records Management Working Group in metadata
requirements for electronic records, and recommendations of the ICGI Web
Content Management Working Group for metadata associated with Web pages.

**[Background]{.underline}**

The scope of this recommendation was debated for some months in a
subgroup of the CGI WG. Those discussions were strongly informed by the
library background of the subgroup members. These discussions also
proceeded in parallel with discussions in the other CGI WG subgroups and
other working groups within ICGI, that were concurrently addressing
related aspects of government information. Some cross-representation and
regular liaison reports helped to assure convergence of approaches.
Also, the preliminary draft product of this subgroup was converged with
other the work of other CGI WG subgroups by the CGI WG chair.

**[Alternatives Considered]{.underline}**

This CGI WG subgroup considered whether to establish a specific core set
of elements that would be required for all government information. This
approach has been adopted by several other governments to date. However,
the subgroup realizes that there is great diversity across the range of
government information, and no dogmatic set of elements could fully
address that diversity in practice. For example, letters and other
correspondence rarely have a formal "title" and different information
products rely on different events to establish a \"creation date\". The
interoperable search service approach offers a practical way to
accommodate such diversity, as it allows managers of government
information to apply locally appropriate techniques in support of
government-wide search access by common bibliographic metadata.

This subgroup discussed issues involving \"granularity\", i.e., what
should be cataloged when items of government information are parts of
other items of government information? This issue is treated in the CGI
WG draft recommendation on the definition of government information.

This subgroup also discussed whether full text indexing of information
products might be an adequate approach to categorization. While this CGI
WG subgroup sees great value in the ability to find information products
based on the full text of their contents, this access was not seen as a
replacement for categorization. The recommendation as stated supports
both search interoperability and the interoperable use of metadata that
is created and managed for specific information products.

The following is a categorized summary of comments on the Search
Interoperability Recommendation (posted at
http://www.cio.gov/documents/ICGI/recommendation.html ). In addition to
the written comments received during the announced public review period
through September 27, a few noteworthy conversations are also summarized
here.

### Alignment with Law and Policy

**Patrice McDermott** (Deputy Director, Office of Government Relations,
American Library Association) commented:

I am writing on behalf of the American Library Association to commend
the Working Group on its recommendation that the federal government
should adopt a search service standard to enhance interoperability among
networked systems. This recommendation is completely in line with what
those of us who worked on the E‑Government Act from its very beginning
had in mind. Our intent in working for the legislation was to make it
possible for the public to search, find, get access to and use
government information products across the government. While a search
service standard will not accomplish all these things, it is essential
that the public have faith that it is finding all government information
on a topic or concern.\
\
As the recommendation notes in Section 3, the library community has
worked together to provide seamless access to information through search
service standards. The section succinctly lays out all the reasons for
the government to adopt this recommendation: this is good government
both in the sense of accountability to the public and in the sense of
effective government.\
\
We strongly urge OMB to adopt and implement this recommendation.

**Lori Lisowski** (Director, Policy and Communications Staff, NARA)
stated:

NARA supports this recommendation. It makes practical sense to have a
government-wide search standard. We support leveraging existing
standards and technologies for search interoperability of online
government information and of a scalable and flexible approach.

Following a discussion at an open meeting of the CGI WG, **Andy
Hoskinson** (Industry Advisory Council) developed a definition of
\"search technology\" as that term might be used in the context of
Federal acquisitions. He reports that IAC members did not object to his
proposed definition of search technology as:

Any software application or information system that has the primary
purpose of facilitating discovery and retrieval of information.

### Feasibility and Cost

**Jon Venverloh** (Senior Manager, Google Federal) commented:

Google agrees that the additional cost to government of the \'additional
support for the search service standard\' costs \'thousands per year.\'
Such work likely includes modifying front end applications to compose
search queries in different ways to interoperate with each search
engine.

In conversation with the editor, Jon further clarified that vendors
would not make major architectural changes to their software, but would
readily support the common standard through a software gateway. The
editor notes that such a gateway approach to implementation is
completely consistent with the recommendation. Gateways are already a
very common technique for technology vendors to support this and many
other standard interfaces.

**John Joaquin** (Convera) stated:

Convera is committed to continuing its support of the government market
and would quickly support this effort.

**Beverly Godwin** (GSA) recommended that CGI WG take a look at the
search mechanism provided by the \"Science.Gov\" initiative. This prompt
led to contact with **Carl Sylvia** at Deep Web Technologies, which
provides the search technology underlying Science.Gov. Carl was asked to
review the recommended search standard and provided this response:

I have taken a few moments to review the Z39.50 \[ISO 23950\] SRW/SRU
standards, and I am quite excited by what I have discovered. I agree
that with a minimal amount of effort our current search technology will
be able to take advantage of the SRU functionality. In addition, are
currently in the process of re-architecting our deep-web search
technology solution and this architecture will be able to make full use
of the full Z39.50 SRW implementation (as it will be a web-services
based implementation).

I would be very interested in discussing the possibility of applying our
search technologies to collections that are currently supporting the
Z39.50 standards, both with our current implementation, and in the
future as we begin deployment of our new web-services based
architecture.  In particular, I would be interested in discussing Z39.50
support on collections that we currently support through science.gov,
and any additional collections that we may include in future search
portals.

**Walt Warnick** (Office of Scientific and Technical Information at the
Department of Energy) discussed search interoperability issues with the
CGI WG. After discussion, all agreed that the recommendation would
recognize that a mix of human and machine-aided categorization is
appropriate with contemporary information technology. There is also an
expectation that more innovations will be forthcoming in the automated
analysis of text to support even more effective searching in the future.

**John Shirey** (Environmental Protection Agency) initially expressed a
concern about recommending the ISO 23950 search standard because most
ISO 23950 implementations in libraries have used a special Internet
protocol rather than the ubiquitous \"HTTP\" (HyperText Transfer
Protocol). John withdrew his objections when he looked into the new ISO
23950 profile that does in fact use HTTP, and also provides simplified
and human-readable query syntax for search.

### Benefits in Cost Savings

**Raul Valdes-Perez** (President and co-Founder of Vivisimo, Inc.)
stated that implementation of these recommendations would provide
\"substantial information-delivery and cost benefits\" by simplifying
the task of creating and maintaining cross-agency search facilities.

**Andy Hoskinson** asserted: \"From a benefits standpoint, the cost
savings achieved through improved information sharing between federal
agencies and other entities (state and local governments, private sector
entities, etc.) would be staggering.\"

**Lori Lisowski** asked: \"what are the costs of not doing this,
especially with free commercial search tools already available?\"\
\
The editor responded: \"the costs of continuing the current lack of
interoperability include various lost opportunities for better
efficiency across agencies, across levels of government, and in managing
technology evolution.\" The editor also pointed out that free commercial
search tools will continue to play a major role in public access to
government information, but those search tools operated by Federal
government agencies should also be interoperable (typically via software
gateways).

### Benefits in Information Accessibility

**Jon Venverloh** stated:

We at Google think the CGI Working Group\'s mission is important and
agree that improved search capabilities will reap significant rewards
for citizens and civil servants.

**Andy Hoskinson** predicted that implementing the recommendation would

greatly enhance the effectiveness, accountability, and transparency of
government through improved citizen access to government information. It
will accomplish this by providing open, non-proprietary, and uniform
interfaces to government search services that make it easy for both
public and private sector entities to \'connect the dots\' between
information maintained by different federal government agencies

**Gladys Cotter** (Associate Chief Biologist for Information, USGS)
commented:

The search interoperability standard will greatly aid access to data and
information produced by all sectors of society. Implementation of this
standard will move us closer to the vision of fully integrated
information access and retrieval.

**Lori Lisowski** asked: \"Would this interoperability function provide
greater levels of service, such as digging into the data in
databases?\"\
\
The editor responded: \"this interoperability function can be used to
provide greater levels of service. However, it should be understood that
\'digging into the data in databases\' can be achieved with the
recommended standard only with respect to those databases holding
certain types of information.\"

### Role of Government

**John Joaquin** (Convera) stated:

Search interoperability is a much needed capability for government
users. Proprietary search technology, like that offered by Convera and
others, provides considerable value to the knowledge worker.
Unfortunately the value collected in product specific systems (indexes)
is difficult to access from other search technologies. Given the nature
of the search market it very unlikely that vendors will come together to
solve this problem using a standards-based approach.

A customer-driven approach is, in my opinion, the best way to a
solution. This approach leverages the government\'s buying power.
Ideally policy-setting organizations would promulgate a standard that
would require/recommend agencies to support when making investments in
search tools. This approach will get the attention of industry and
provide a business reason to adopt said standard.

**Andy Hoskinson** commented:

there is no monetary incentive that I am aware of for private sector
companies to collaborate on a search interoperability standard \[\...\]
it is appropriate for the Government to take the lead in this area.

### Intergovernmental Implications

**John Borras** (Director, Technology Policy, e-Government Unit, UK)
stated his full support for the recommendation. John noted that this
recommendation builds on work done in the OASIS e‑Government Technical
Committee wherein there was

lengthy debate within the committee and amongst our various technical
advisors about the all the issues and the committee has now published
it\'s own similar recommendation for the attention of all
Governments\...Clearly the more we in government across the globe
implement this standard then the better chance we have of meeting our
stated goal of true interoperability.

**Ivan DeLoatch** (Staff Director, Federal Geographic Data Committee)
expressed

strong endorsement of this recommendation for standard-based
interoperable search. In particular, the use of the ISO 23950 standard
is critical to leveraging the broad interoperability currently
implemented in many levels of government in the context of geospatial
data. This includes: the US National Spatial Data Infrastructure as
mandated by law and directed by policy, the international Global Spatial
Data Infrastructure that now includes more than fifty national spatial
data infrastructure, and the more recent Global Earth Observations
System of Systems currently being planned for international use.

### Effect on Search Technology

**Raul Valdes-Perez** stated that providers of search services will be
able to

independently evolve an individual search resource without the
substantial overhead of worrying how changes to the resource will impact
cross-agency search engines that depend on it

**Andy Hoskinson** drew attention to

the likelihood that this recommendation, if implemented, will spawn a
robust private-sector "cottage industry\" around the discovery,
mediation, and aggregation of federal government information and related
information obtained through state/local and private sector channels

### Technology Approach

**David McDonald** (Microsoft Consulting Services, Public Sector) stated

I am pleased to see this initiative and am a strong supporter of
government search interoperability. Inclusion of the Search/Retrieve Web
Service Profile to FIPS 192 is strongly recommended. A standard web
services interface enables software developers to more easily create
solutions that include search and retrieval of government information.
Standardizing the URL and query syntax using CQL as stipulated by the
SRU service as described in this recommendation is a good first step
\[\...\] Government agencies should be encouraged to implement the full
XML/SOAP interfaces.

**John Barden** (Head, Reference & Research Services, William T. Muse
Law Library, University of Richmond School of Law) commented:

This is a very promising initiative. Is it possible, to go one-step
beyond the search standards identified here to incorporate \[\...\]
proximity searching

The editor responded that the recommended standard does support
proximity searching, among other useful features not discussed in this
document.

**Anne Washington** (Librarian, Office of Legislative Information,
Congressional Research Service, Library of Congress) stated:

There is a prevalence of structured and semi-structured information held
by governments. Here on Capitol Hill, there are several projects
underway to deliver legislation in XML. I know that we aren\'t alone in
our effort to make full-text government documents available in XML.
\[The Requirements document\] refers to common query mechanisms that the
interoperable search standard should address. This might be a place to
refer to the XML Query language (XQUERY). It is a single mechanism that
addresses full-text, markup and fielded documents. XQuery is a standard
being developed by the W3C

The editor responded:

From the perspective of interoperable search, we cannot expect a
searcher to know the details of any particular document schema such as a
convention for markup of legislation. Indeed, we cannot expect a
searcher to know which markup language is used with any particular
collection of documents. In the case of XQuery, the searcher needs to
know that the documents being searched are to be treated as XML
documents, and further that they are structured according to a
particular XML Schema.

\[\...\] interoperable search is directed at the problem of broad-scale
interoperability. It should be seen as supplementing, not supplanting,
other search mechanisms such as the use of XQuery on legislative
documents in a particular schema.

**Jon Venverloh** commented:

the international standards referenced in section 3 do not appear to
address relevance of documents that have been retrieved. That is, these
standards may address recall but do not address precision. How relevance
is determined is the primary differentiate among search engines and
represents the bulk of research and development costs for the major
search engines such as Google, Yahoo, and others.

The editor responded that the recommended standard intentionally
excludes standardization of relevance ranking. Search services compliant
with this standard do in fact use a wide range of ranking approaches and
ranking is an area where search technology vendors compete actively.

**Jon Venverloh** further commented that:

query analysis, syntax and results are handled differently from one
search engine to another. Search engines compete to some degree on
usability. Google spends a substantial amount of its R&D resources on
continually improving usability.

The editor responded that the recommendation is supportive of active
competition among vendors with respect to query analysis and results
handling as that is the primary way that progress is made in balancing
\"precision versus recall\" in information search. The standard
recommended here does not overly constrain those different approaches.
It only provides a standard syntax for those certain components of a
query and result set that are common across the great majority of search
technologies.

**Jon Venverloh** also contended that:

a standardized query format \[\...\] would limit unique features and
innovations among some search engines \[\...\] the proposed standards do
not seem to anticipate the different technological approaches that are
inherent\--and beneficial to end users\--in a market with multiple
competing search engines.

The editor corrected this misimpression and pointed out that the
recommended standard does provides support for unique features and
innovations, and that such extensions are widely used in practice.

### Agency Implementation Specifics 

**Lori Lisowski** asked: \"What are the practicalities of being able to
accomplish this?\"\
\
The editor responded: \"As a practical matter, search interoperability
is typically provided by adding a standard gateway service onto whatever
similar service already exists. This is usually a matter for the search
technology vendor to address.\"

**Lori Lisowski** also asked: \"How much preparatory work would agencies
be required to do in order to enable such a service?\"\
\
The editor responded: \"Agencies would usually need to work with the
vendor to provide the \'semantic map\' which relates locally-known
elements to the standard search concepts. This is the same type of work
that agencies already perform whenever they customize a search to use
any agency-specific features.\"

**Lori Lisowski** further asked: \"How would it deal with false
positives?\"\
\
The editor responded:

A \'false positive\' might exist when an item in any result set is
perceived to satisfy the given search criteria and yet be regarded as
not a very relevant item. In the discipline of \'information
retrieval\', this aspect is measured as \'precision\' and it is
understood that any relevance ranking makes a particular trade-off
between precision and \'recall\'. This standard is silent with regard to
the variety of ranking approaches and algorithms. This is currently an
active area of competition among information retrieval technologies.

### General Expressions of Support

**Kate Dolan** (The Enterprise Standards Program, National Security
Agency): \"I just wanted to say that we are happy to see the support of
standards usage.\"

**Aline Dolan** (Naval Hospital, Jacksonville): \"Excellent. Definitely
the right way to go. Thank you very much for your efforts to keep our
services accessible to all.\"

### 1 Recommendation

The U.S. Federal Government should adopt a search service standard to
enhance interoperability among networked systems that aid in the
discovery of and access to government information. The adopted search
service standard should be the ISO 23950 international standard, thereby
providing a high degree of interoperability across many communities of
practice and types of data and information holdings. This recommendation
follows existing law and policies of the U.S. Federal Government,
positioning the standard search service as a supplement to other search
mechanisms as may be needed for reasons other than broad scale
interoperability.

### 2 Implications 

**Policy** - No additional policy action is needed to implement the
standard search service recommended here. The U.S. Federal Government
already has law and policy mandating a standard search service as part
of the Government Information Locator Service established by law (United
States Code Chapter 44, Section 3511). Corresponding Federal policy (OMB
Memorandum 98-5) required a standard search service to be used for
locating government information. That standard search service is
described in Federal Information Processing Standard (FIPS) 192-1, which
is required to be cited in procurements of search technology by Federal
agencies. FIPS 192 adopted a profile \[GILS\] of the international
standard recommended here, ISO 23950. Similar law, policy, and standards
exist for geospatial data in the United States (i.e., E-Government Act
section 216, OMB Circular A-16, and the Geospatial profile \[GEO\] of
ISO 23950).

**Oversight** - As noted in existing policy, GSA and OMB oversight
should be exercised to assess and enforce existing law and policy
requiring search technology procured by Federal agencies to comply
with FIPS 192. FIPS 192 should be updated to include the newly available
\"Web services\" profile of ISO 23950 known as \[SRW\].

**Cost** - There is an ongoing operational cost to government in
supporting any standardized search service, but this would be
essentially the same as what is entailed in setting up non-standard
search services. When first introduced, the support of a search service
standard may prompt an add-on cost in acquisitions of search technology,
but such a cost would be a small percentage of what the U.S. Federal
government spends on disseminating government information. For example,
an Internet portal for government information may cost millions per
year, while the additional support for the search service standard on
that portal may be thousands per year. Also, after search technology
vendors have implemented the standard interface once, their costs for
supporting additional implementations should be very minor.

### 3 Background 

An information index helps a searcher to find resources. Such an index
usually covers just one collection of resources, yet searchers often
want to search across multiple collections. The ability to search
multiple, separately operated indexes is called \"search
interoperability\". Amazingly, libraries worldwide already offer
interoperable search across their many thousands of collections. This
search interoperability is based on a carefully negotiated international
standard supported by the major vendors of information retrieval
technology. And, the standard addresses far more than mere \"word in
text\" search\--the standard includes sophisticated methods needed for
precise searching of collections holding millions of diverse resources.

Clearly, libraries worldwide are meeting their customers\' needs to have
seamless access to information. The same cannot be said of governments.
Agencies no longer rely on libraries as the backbone of information
dissemination\--they offer information directly to the public via their
own indexes and directories of Web pages, databases, and a diverse range
of specialized services. It is true that the amount of accessible
government information is growing at a healthy rate, but the need for
people to confidently search for government information across agencies
and levels of government is not being met. The problem is not so much
the amount of information. The problem is that few governments have yet
focused on search interoperability.

Because public access to government information is the basis of
effective, accountable and transparent government, interoperability of
government search facilities is essential. Adoption of a search service
standard would serve the public interest by making government
information more readily accessible through the diverse community of
government information providers. Search interoperability also generates
government-wide efficiencies: from increased information sharing, and
from lowered costs for mechanisms needed to merge information from
multiple government sources. Efficiencies accrue within each single
government organization, as well. For instance, a search service
standard provides some \"future-proofing\" against changes in search
technology. With standards-based search, the periodic migration to new
search technology is not so disruptive, and it is also easier to
maintain access to legacy holdings.

Governments at all levels worldwide are major producers and consumers of
data and information, encompassing many communities of practice and
types of data and information holdings. Because governments both depend
upon and foster a competitive intermediary market for information
dissemination and service delivery, government support of broad scale,
standards-based interoperability is essential. In that regard,
governments must promote an information search interface that is
non-proprietary, fair, and stable. By acquiring products that support an
international standard search service, governments will encourage a fair
and competitive market for products, and maximize agency choice.

The E-Government Act of 2002 requires the U.S. Federal Government to
enhance search interoperability by adopting a common standard. Section
207 (\"Accessibility, Usability, And Preservation of Government
Information\"), requires that the Interagency Committee on Government
Information submit recommendations to the Director of the Office of
Management and Budget (OMB) on \"the adoption of standards, which are
open to the maximum extent feasible, to enable the organization and
categorization of Government information in a way that is searchable
electronically, including by searchable identifiers; and in ways that
are interoperable across agencies\".

### 4 Base Requirements

This Recommendation satisfies all of the mandatory and desirable
requirements as given in the Statement of Requirements for Search
Interoperability, posted for public comments and revised over the period
February - April, 2004 ( http://www.search.gov/interop/requirements.html
).

**Paraphrased Statement of Requirement:**

-   Supports different levels of access control, such as restrictions by
    service, session, distributed resource, database, record, or data
    element.

-   Supports authentication of user identity through an ancillary
    service (e.g., e‑Authentication).

-   Supports verification of the integrity of delivered data, metadata,
    or other information must be able to be verified as well.

-   Supports the search service standard for library catalogs accessible
    over network technologies, \[ISO 23950\] (identical to ANSI/NISO
    Z39.50)

-   Supports the library standard for catalog records, Machine-Readable
    Cataloging

-   Supports access to data without mandating proprietary technologies,
    nor proprietary vocabularies or thesauri

-   Can be readily accommodated by leading search products, including
    Internet search engines

-   Supports search of information that may be unstructured (often
    called \"full-text\"), semi‑structured (typically represented with
    inline \"markup\"), or structured (sometimes known as \"fielded\").

-   Supports search of HTML meta element contents and other varieties of
    metadata embedded within particular types of files (e.g., PDF,
    e-mail, etc).

-   Supports customizable search of other varieties of structured
    metadata through common mechanisms such as SQL and LDAP.

-   Provides for interoperable search across locators for information
    and collections of information

-   Interoperable with the international standard search service
    supporting the U.S. National Spatial Data Infrastructure
    Clearinghouse of geospatial data

-   Implementable over the Internet using TCP/IP, HTTP/HTTPS, HTTP GET
    and HTTP POST

-   Precisely defined as to how searches are expressed and communicated
    between a client component and a server component, including a query
    language, a query syntax, and standardization of a result set schema

-   Specified in an interface definition language such as Web Services
    Definition Language

-   Supports searching of structured information using a nested Boolean
    query, e.g., (date \> \'20040101\') AND ((subject = \'earthquake\')
    OR (subject = \'temblor\')).

-   Supports the usual sets of data structures (word, phrase, date,
    URL.) and relations (equal, greater than, less than).

-   Includes a query evaluation function to handle \"abstract concepts\"
    (e.g., name, category, date) according to what they mean
    semantically rather than merely how they may be labeled
    syntactically.

-   Supports abstract concepts that are produced by semantic mapping
    without requiring any particular semantic mapping technique

-   Supports gateway to Internet Anonymous FTP Archive (IAFA) file
    system catalogs and Distributed Authoring and Versioning for the Web
    (WebDAV)

-   Adopts readily to the underlying data model of named properties and
    property sets that is defined for objects addressable by software

-   Already in production use for searching metadata variants such as
    Dublin Core Metadata Initiative, ISO 15836 Encoded Archival
    Description, and ISO 8879Standard Generalized Markup Language (SGML)

-   Compatible with many and diverse approaches to compiling collections
    of information, without mandating any particular approach

-   Supports interoperable search of business and services registries,
    modeled on ISO 11179 Metadata Registries, ebXML, or the Universal
    Description, Discovery, and Integration (UDDI) model

-   Scalable in terms of supporting arbitrarily complex searches

-   Scalable in not foreclosing concurrent searches on multiple servers

-   Extensible to search tasks with unusual data structures and
    relations, definable through profiles or equivalent

-   Provides extension mechanisms to nurture innovation in areas not yet
    ready for the broadest level of standardization

-   Has been in use worldwide in many languages

-   Supports negotiation between client and server as to each other\'s
    language capabilities for the session

-   Supports character set negotiation, with Latin-1 as a minimum for
    U.S. Federal Government applications

### 5 ISO 23950 Overview 

ISO 23950, the international standard for information search and
retrieval, defines a particular set of network client-server
\"services\". The definition is powerful enough to accommodate the most
commonly required search functions over a broad range of search
facilities, including the requirements stated above. This section
provides an general overview of ISO 23950 using the particular variety
of ISO 23950 known as the \[SRW\] (Search and Retrieve for the Web)
profile.

By analogy to a restaurant, a network service in operation is like a
waiter handling a dinner order from a customer. Just as a customer is
not expected to give step-by-step instructions to the kitchen, the
ISO 23950 service allows the client to precisely specify the request but
does not allow the client to specify the exact procedure for satisfying
the request. This is an important feature for security as well as for
broad interoperability. Clients have no more control than necessary, and
clients need not know execution details.

For example, a searcher who wants to find what the Library of Congress
may have on \"fruit\" can send an ISO 23950 search request that looks
like this:

http://z3950.loc.gov:7090/voyager?operation=searchRetrieve&version=1.1&query=fruit

This SRW search syntax uses the Internet standard for URL\'s (RFC 1738).
The search request has two component parts: a \"base URL\" and a
\"searchpart\", separated by a question mark (\"?\"). The base
URL identifies the server host and port (here, \"z3950.loc.gov:7090\")
and the ISO 23950 service (here, \"voyager\"). The searchpart consists
of parameters separated by \"&\", each with the structure
\"key= value\". The names of the parameters from the ISO 23950 service
description are the \"key\" strings within the URL. (In this example,
the keys are \"operation\", \"version\", and \"query\".)

The ISO 23950 definition of a standard search syntax provides an obvious
level of interoperability. The example search statement could be applied
to several popular Internet search services in this way:

http://www.google.com/search?operation=searchRetrieve&version=1.1&query=fruit

http://search.yahoo.com/search?operation=searchRetrieve&version=1.1&query=fruit

http://alltheweb.com/search?operation=searchRetrieve&version=1.1&query=fruit

http://www.altavista.com/web/results?operation=searchRetrieve&version=1.1&query=fruit

http://vivisimo.com/search?operation=searchRetrieve&version=1.1&query=fruit

Without ISO 23950, a searcher would need to use the particular syntax
invented by each search technology vendor:

http://www.google.com/search?hl=en&ie=UTF‑8&oe=UTF‑8&q=fruit&\
btnG=Google+Search

http://search.yahoo.com/search?fr=fp-pull-web-t&p=fruit

http://alltheweb.com/search?cat=web&cs=utf8&q=fruit&\_sb_lang=pref

http://www.altavista.com/web/results?q=fruit&kgs=1&kls=0

http://vivisimo.com/search?query=fruit&v%3Asources=Web&x=0&y=0

This is only a trivial example of the variety of search syntaxes
supported by technology vendors, especially as most also support Boolean
operations with fielded searches. The bewildering variety of search
syntax has become a major barrier to search interoperability among
Internet search vendors, just as it was among library catalog search
vendors before agreements were reached on the international search
standard in the 1990\'s.

Following here is a bit more detail about the \"Common Query Language\"
\[CQL\] syntax used in the \"searchpart\" of an SRW search URL
introduced above. In CQL, a query can be as simple as an unqualified
single term (\"fruit\" in the example above). Queries also may be joined
together using the Boolean \"and\", \"or\" operators, as in the
following example:

(bird or dinosaur) and (feathers or scales)

The Boolean \"not\" is used as a binary operator, finding records which
contain \"this but not that\". For example,

dinosaur not reptile

would find records which contain the word \"dinosaur\" but not the word
\"reptile\"\'.

In addition to queries targeted at whole records, queries can be limited
to a particular part of the records being searched. These searchable
parts are called \"indexes\" in CQL. For example, limiting a search to
the \"author\" index would find matches on the names of authors. An
index is specified in CQL as part of a set of indexes, in recognition
that different communities of practice sometimes have unique indexes.
For instance, both the bibliographic and the heraldry communities might
wish to name a \"title\" index, but those indexes would have different
meanings.

In ISO 23950 and CQL, an \"index\" is an abstract concept. A CQL query
that limits a search to the \"author\" index can be executed in various
ways by the server application. For an e-mail collection, the author
index may contain values taken out of the \"from\" field of e-mail
messages; For a news clipping collection, the author index may contain
values taken out of the \"by-line\" field in the news stories. This
abstraction is very important for achieving search interoperability.

### 6 Alternatives Considered 

**Single Portal** - From a management control perspective, there is a
certain attraction to having just one system encompassing all relevant
information. The manager of a portal might then focus just on
\"operability\" issues within the manager\'s own control, relegating
\"interoperability\" to \"someone else\'s problem\". Such a single
portal could be physically distributed, using various mechanisms for
\"pulling\" or \"pushing\" information, metadata, and update signals
among distributed components of a logically centralized system. Yet,
from a public policy perspective, the very idea of a single, master
portal is unrealistic. Any effective government organization must
accommodate relationships with other levels of government and with other
public and private sector information sources. Consequently, any single
portal must co-exist with other information portals, and so must support
a degree of interoperability. From a technology perspective also,
interoperability is a more appropriate approach. There are simply too
many mutually incompatible search mechanisms already in place to imagine
that any single solution could provide customized interfaces to all of
them. The need for interoperability is even more obvious when one
considers that many of those custom interfaces would be shaped by
distinct vendors who can change interface specifications at will.

**Common Data Model** - In the early days of mainframe computer systems,
it was common to envision an enterprise-wide \"management information
system\" that mandated a common data model applied to all enterprise
information systems. This approach is less stringent than subordinating
all systems into some master, all-encompassing system, but it still does
require central administration of an abstract and complex model shared
by all interacting systems. In practice, this approach suffers much the
same difficulties found in the \"single portal\" approach. Today\'s
reality is that any government organization must accommodate a great
variety of in‑house and external actors who evolve their component
systems independently. These largely independent systems already have
their own data models that often have little in common, even when a
single vendor has supplied the systems software.

**Applications Programming Interface (API)** - Software is an integral
part of most government information systems and software is implemented
through programs. Designers of complex systems usually divide software
into modules that are each provided with a published interface with
well-defined entry points for application programmers. Unfortunately,
such an API approach must be tailored to each distinct programming
environment. Now that there are many operating platforms and programming
languages, the programming interfaces needed for broad interoperability
have become too numerous to be manageable. However, the current
\"services‑oriented architecture\" approach underlying the present
recommendation does build on the programming discipline of the API
approach. The important difference is that \"services\" are based on the
characteristics of a network interface between interacting systems,
rather than being based on characteristics of the programming interface.
This is a great advantage for the set of problems encountered in
information search and retrieval.

**Structured Query Language (SQL)** - SQL has a long history of use,
starting with the first management information system efforts several
decades ago. When combined with an appropriate network service such as
ODBC (Open DataBase Connectivity), SQL can be used as part of a
services-oriented architecture. However, SQL by itself does not include
the essential idea of search indexes as an abstract mapping against
actual content structures. Also, SQL is oriented toward query of
database tables rather than Information Retrieval against very large
collections. An SQL query would result in a table having all records
that satisfy the search constraints; Information Retrieval would build a
\"result set\" giving a rank-ordered listing of records that satisfy a
search request, any of which might be actually retrieved in a separate
operation. Nevertheless, SQL is often used very effectively in
combination with the ISO 23950 international standard search service
recommended here.

### 7 Review Process Used

An iterative approach identified requirements for a search service
standard. A draft Statement of Requirements was posted publicly, with
comments of the major stakeholders invited, on the Internet
at http://www.search.gov/interop/requirements.html . Several revisions
were made over the months February - April, 2004. The initial version of
the Statement of Requirements was based on a recommendation in February
2003 of the E-Government Technical Committee of the Organization
for the Advancement of Structured Information Standards \[OASIS\]. The
OASIS recommendation was informed by, among others, an April 2002 white
paper titled \"Interoperability Strategy: Concepts, Challenges, and
Recommendations\" by the Industry Advisory Council \[IAC\], Enterprise
Architecture Shared Interest Group. (Focused on promoting
government-industry partnerships, the Industry Advisory Council
represents professionals from over 400 leading information technology
companies.)

### 8 Notes and References

\[CQL\] Common Query Language is described at
http://www.loc.gov/z3950/agency/zing/cql/

\[GILS\] E. Christian, Application Profile For The Government
Information Locator Service (GILS), http://www.gils.net/prof_v2.html ,
April 1997.

\[GEO\] D. Nebert, Z39.50 Application Profile for Geospatial Metadata or
\"GEO\", http://www.blueangeltech.com/Standards/GeoProfile/geo22.htm ,
May 2000.

\[IAC\] Dodd, John, at al, Interoperability Strategy: Concepts,
Challenges, and Recommendations,
http://www.search.gov/IAC-Interoperability.pdf , April 2003.

\[ISO 23950\] ANSI/NISO Z39.50-1995, Information Retrieval (Z39.50):
Application Service Definition nd Protocol Specification,
http://lcweb.loc.gov/z3950/agency , 1995.

\[OASIS\] The OASIS E-Government Technical Committee recommendation
concerning Search ervice nteroperability is available at\
http://www.oasis-open.org/committees/download.php/5846/wd-egov-searchservice-CD.pdf

\[SRW\] Search/Retrieve Web Service home page is
http://www.loc.gov/z3950/agency/zing/srw/