NISTIR 8084
 

The NIST Plan for Providing 

Public Access to Results of 

Federally Funded Research 

Katherine E. Sharpless 

Regina L. Avila 

Sally S. Bruce 

Wo L. Chang 

Virginia Covahey 

Robert M. Dimeo 

A. Kirk Dohne 

Heather M. Evans 

Aaron P. Fein 

Donna J. Kimball 

Andrea M. Medina-Smith 

Alan E. Munter 

James K. Olthoff 

Dianne L. Poster 

Kathleen M. Roberts 

Susannah B. Schiller 

John Henry J. Scott 

Barbara P. Silcox 

James A. St. Pierre 

Mark D. Stiles 

This publication is available free of charge from: 

http://dx.doi.org/10.6028/NIST.IR.8084 
This publication is available free of charge from http://dx.doi.org/10.6028/NIST.IR.8084 
 

 

 

 

 

 

 

 

 

 

NISTIR 8084 

The NIST Plan for Providing 

Public Access to Results of 

Federally Funded Research 

Katherine E. Sharpless, Special Programs Office 

Regina L. Avila, Information Services Office 

Sally S. Bruce, Standards Coordination Office 

Wo L. Chang, Information Technology Laboratory 

Virginia Covahey, Public Affairs Office 

Robert M. Dimeo, NIST Center for Neutron Research 

A. Kirk Dohne, Engineering Laboratory 

Heather Evans, Program Coordination Office 

Aaron P. Fein, Physical Measurement Laboratory 

Donna J. Kimball, Laboratory Programs 

Andrea M. Medina-Smith, Information Services Office 

Alan E. Munter, NIST Center for Neutron Research 

James K. Olthoff, Physical Measurement Laboratory 

Dianne L. Poster, Laboratory Programs 

Kathleen M. Roberts, Information Technology Laboratory 

Susannah B. Schiller, Office of Information Systems Management 

John Henry J. Scott, Material Measurement Laboratory 

Barbara P. Silcox, Information Services Office 

James A. St. Pierre, Information Technology Laboratory 

Mark D. Stiles, Center for Nanoscale Science and Technology 

This publication is available free of charge from http://dx.doi.org/10.6028/NIST.IR.8084 

September 2015 

U.S. Department of Commerce 

Penny Pritzker, Secretary 

National Institute of Standards and Technology 

Willie May, Under Secretary of Commerce for Standards and Technology and Director 
This publication is available free of charge from http://dx.doi.org/10.6028/NIST.IR.8084 
 

 

 

 

 

 

Table of Contents 

Preface......................................................................................................................................... 1
 

National Institute of Standards and Technology Plan for Providing Public Access to the 

Results of Federally Funded Research ........................................................................................ 2
 

Appendix A. Members of the committees responsible for drafting NIST’s plan for providing
	

public access to results of federally funded research. ............................................................... 18
 

Appendix B. Feedback received from NIST staff’s review of draft Public Access Plan and 

responses provided by the Data Policy Group. ......................................................................... 19
 

Appendix C. Public feedback received in response to the Federal Register Notice of July 7, 

2015........................................................................................................................................... 26
 

Appendix D. NIST Data Taxonomy and Actions/Consequences for Data Categories............ 61
 

 

 

 

 

 

 

 

 

 

 

 

 

 

Preface 

In February 2013 the White House Office of Science and Technology Policy (OSTP) issued a 

memorandum requiring federal agencies to develop a plan for “Increasing Access to the Results 

of Federally Funded Scientific Research.”1 A draft plan for providing public access was 

developed by the Data Policy Group and Publications Working Group (Appendix A). The draft 

was posted for NIST staff review and comment in the summer of 2014. A summary of staff 

feedback and responses prepared by the Data Policy Group and the Publications Working Group 

is provided in Appendix B. 

The NIST Plan for Providing Public Access to the Results of Federally Funded Research was 

approved by OSTP in December 2014. The plan is provided in the body of this report. The plan 

was posted for public comment, announced in a Federal Register Notice2 on July 7, 2015. All 

public feedback received is provided in Appendix C. This feedback will inform changes to the 

NIST directives for managing public access to the results of federally funded research, available 

at www.nist.gov/data. 

The Data Policy Group also developed a data taxonomy for categorizing NIST data. The 

taxonomy is included in NIST O 5701.00 Managing Public Access to Results of Federally 

Funded Research. It is provided in Appendix D of this NISTIR to provide context for some of 

the points raised by NIST staff (Appendix B). 

This report serves as the historical record of documents as originally approved. 

1 https://www.whitehouse.gov/sites/default/files/microsites/ostp/ostp_public_access_memo_2013.pdf 

2 http://www.gpo.gov/fdsys/pkg/FR-2015-07-07/pdf/2015-16508.pdf 

1
 
 

 

 

National Institute of Standards and Technology
 

Plan for Providing Public Access to the
 

Results of Federally Funded Research
 

1. PURPOSE 

This document establishes a plan to enable public access to the results of research funded wholly 

or in part by NIST; NIST’s Public Access Policy will be informed by this Public Access Plan. 

To the extent feasible and consistent with law, agency mission, resource constraints, U.S. 

national, homeland, and economic security, and the objectives listed below, NIST intends to 

make freely available to the public, in publicly accessible repositories, all peer-reviewed 

scholarly publications and associated data arising from unclassified research and programs 

funded wholly or in part by NIST. Subject to the same conditions and constraints listed above, 

NIST will also promote the deposit of scientific data arising from unclassified research and 

programs, funded wholly or in part by NIST, to make it available free of charge unless otherwise 

excepted, in publicly accessible databases. NIST’s Public Access Plan promotes the following 

objectives: 

	 Establish NIST’s commitment to providing public access to scientific research results 

	 Support governance of and best practices for managing peer-reviewed scholarly
 

publications and digital scientific data across NIST 

	 Ensure effective access to and reliable preservation of NIST peer-reviewed scholarly 

publications and digital scientific data for use in research, development, education, and 

scientific discovery 

	 Enhance innovation and competitiveness by maximizing the potential to create new 

business opportunities. 

2. SCOPE 

The NIST Public Access Plan applies to the results of research funded wholly or in part by NIST, 

presented in peer-reviewed scholarly publications and as research data, defined in Circular A110 of the Office of Management and Budget as the “recorded factual material commonly 

accepted in the scientific community as necessary to validate research findings.” 

Not considered research data in Circular A-110, and therefore not covered by this plan, are: 

	 Laboratory notebooks, results of preliminary analyses, drafts of scientific papers, plans 

for future research, peer review reports, communications with colleagues, or physical 

objects, such as laboratory specimens; 

	 Trade secrets, commercial information, or other materials necessary to be held 

confidential by a researcher until they are published, or similar information that is
 

protected under law; and
 

	 Personnel and medical information and similar information the disclosure of which 

would constitute a clearly unwarranted invasion of personal privacy. 

2
 
 

 

 

NIST will protect confidentiality and personal privacy and will recognize proprietary interests, 

business confidential information, and intellectual property rights, avoiding significant negative 

impact on intellectual property rights, innovation, and U.S. competitiveness. 

Implementation will be prospective and will not apply to NIST peer-reviewed scholarly 

publications published and research data created before NIST’s Public Access Policy takes 

effect. However, NIST will endeavor to make legacy information publicly available and ensure 

its preservation as soon as practicable. A process will be created to engage with customers to 

help facilitate and prioritize information release. 

3. APPLICABILITY 

The NIST Public Access Plan applies to the following groups: 

	 All NIST employees who publish peer-reviewed scholarly material and data as part of 

their employment, including full- and part-time employees, temporary government 

employees, and special government employees; 

	 Awardees from non-NIST organizations that publish peer-reviewed scholarly material 

and data through activities funded wholly or in part by NIST through a grant, 

cooperative agreement, contract, or other agreement. This includes but is not limited 

to states, localities, regulated parties, non-profit and volunteer organizations, 

contractors, cooperative agreement holders, grantees, cooperating Federal agencies, 

intergovernmental organizations, universities, and other educational institutions. For 

activities funded by multiple sources with differing public access requirements, the 

provisions of this plan will apply unless otherwise specified by NIST in its funding 

documents. 

4. REQUIREMENTS 

To the extent feasible and consistent with law, agency mission, resource constraints, U.S. 

national, homeland, and economic security, and the objectives listed below, NIST intends to 

make freely available to the public, in publicly accessible repositories, all peer-reviewed 

scholarly publications and associated data arising from unclassified research and programs 

funded wholly or in part by NIST. Subject to the same conditions and constraints listed above, 

NIST will also promote the deposit of scientific data arising from unclassified research and 

programs, funded wholly or in part by NIST, free of charge unless otherwise excepted, in 

publicly accessible databases. 

All proposals or plans for activities that will generate scientific data using NIST funding will be 

required to (1) adhere to a Data Management Plan (DMP) that describes how scientific data 

generated through the course of the proposed work will be shared and preserved or (2) explain 

why data sharing and/or preservation are not within the scope of this plan. 

	 Reasonable costs for data preservation and access may be included in grant proposals or 

project plan budgets for contracts. Grantee and contractor DMPs will be reviewed as part 

of the technical evaluation process. 

	 NIST managers will be required to ensure staff compliance with the requirements of 

DMPs, including those for preservation and discoverability. NIST managers will ensure 

that DMPs are considered in the context of employees’ performance plans and 

evaluations. 

3
 
 

 

 

 

 

	 Non-compliance with requirements by staff and funding recipients will result in 

penalties; policies will evolve over time.
 

Authors of peer-reviewed scholarly publications are required to submit to the NIST public access 

archive system metadata and their copies of final peer-reviewed journal manuscripts within the 

scope of this plan once the manuscript is accepted for publication. In lieu of the author’s version 

of the final peer-reviewed manuscript, NIST will also accept the final published article, as 

formatted by the journal, provided the author has the right to submit the published version. 

NIST’s plan further requires that the final manuscript, which has been peer-reviewed and 

accepted for publication, be freely available to the public no later than 12 months following 

publication. 

5. AUTHORITY 

NIST’s authority to require broad public access to the results of federally funded research stems 

from multiple sources, including, but not necessarily limited to, those below. 

	 Executive Office of the President, Memorandum for the Heads of Executive Departments 

and Agencies: Increasing Access to the Results of Federally Funded Scientific Research, 

dated February 22, 2013, requires that the direct results of federally funded scientific 

research, including that of awardees, be made available to and useful for the public, 

industry, and the scientific community. Available at 

http://www.whitehouse.gov/sites/default/files/microsites/ostp/ostp_public_access_memo 

_2013.pdf (Accessed 10 February 2014.) 

	 Memorandum for the Heads of Executive Departments and Agencies – Transparency and 

Open Government, Executive Office of the President, January 21, 2009, requires that be 

transparent, participatory, and collaborative. 

http://www.whitehouse.gov/the_press_office/TransparencyandOpenGovernment 

(Accessed 8 April 2014.) 

	 Open Government Directive (M10-06) issued on December 8, 2009 directs executive 

departments and agencies that, “to increase accountability, promote informed 

participation by the public, and create economic opportunity, each agency shall take 

prompt steps to expand access to information by making it available online in open 

formats.” http://www.whitehouse.gov/open/documents/open-government-directive 

(Accessed 10 February 2014.) 

	 The Office of Management and Budget (OMB) Memorandum M-13-13 Open Data 

Policy – Managing Information as an Asset dated May 9, 2013, requires that agencies 

collect or create information in a way that supports downstream processing and 

dissemination (e.g., use of machine-readable and open formats and extensible metadata). 

http://www.whitehouse.gov/sites/default/files/omb/memoranda/2013/m-13-13.pdf 

(Accessed 8 April 2014.) 

	 The Office of Management and Budget (OMB) Circular A-110, Uniform Administrative 

Requirements for Grants and Agreements with Institutions of Higher Education, 

Hospitals, and Other Non-Profit Organizations (section 36) specifies “The Federal 

awarding agency(ies) reserve a royalty-free, nonexclusive and irrevocable right to 

reproduce, publish, or otherwise use the work for Federal purposes, and to authorize 

4
 
 

 

 

 

 

 

 

 

 

others to do so.” 

February 2014.) 

http://www.whitehouse.gov/omb/circulars_a110/ (Accessed 10 

 The Office of Management and Budget (OMB) Circular A-130, Management of Federal 

Information Resources, states that “[t]he open and efficient exchange of scientific and 

technical government information ... fosters excellence in scientific research and 

effective use of federal research and development funds.” 

http://www.whitehouse.gov/omb/circulars_a130_a130trans4/ (Accessed 10 February 

2014.) 

 The Freedom of Information Act (FOIA; 5 U.S.C. 552) provides for public access to the 

records of the federal government. http://www.gpo.gov/fdsys/pkg/USCODE-2011-

title5/html/USCODE-2011-title5-partI-chap5-subchapII-sec552.htm (Accessed 10 

February 2014.) 

 National Institute of Standards and Technology Act (15 U.S.C. 271, Chapter 7) states the 

responsibility of NIST to “compile, evaluate, publish, and otherwise disseminate general, 

specific, and technical data resulting from the performance of the functions specified in 

this section or from other sources when such data are important to science, engineering, 

or industry, or to the general public, and are not available elsewhere.” 

http://www.gpo.gov/fdsys/pkg/USCODE-2011-title15/html/USCODE-2011-title15-

chap7-sec271.htm (Accessed 24 January 2014) and 

http://www.nist.gov/director/ocla/upload/NIST-Organic-Act.pdf (Accessed 10 April 

2014) 

 Copyright Act (17 U.S.C. 105, Section 105) provides that “copyright protection under 

this title is not available for any work of the United States Government.” 

http://uscode.house.gov/view.xhtml?req=(title:17%20section:105%20edition:prelim 

(Accessed 24 January 2014) 

 The E-Government Act of 2002 (44 U.S.C. 101) has among its primary purposes the 

promotion of “the use of the Internet and emerging technologies within and across 

Government agencies to provide citizen-centric Government information and services” as 

well as providing “enhanced access to Government information and services.” 

http://www.gpo.gov/fdsys/pkg/PLAW-107publ347/pdf/PLAW-107publ347.pdf 

(Accessed 8 April 2014) 

 Public Law 90-396 Standard Reference Data Act provides for publication and copyright 

of Standard Reference Data. http://www.nist.gov/srd/upload/publiclaw90-396.pdf 

(Accessed 24 January 2014) 

 Public Law 104-13, The Paperwork Reduction Act of 1995 has as one of its key purposes 

to “ensure the greatest possible public benefit from and maximize the utility of 

information created, collected, maintained, used, shared, and disseminated by or for the 

federal government.” http://www.reginfo.gov/public/reginfo/pra.pdf (Accessed 8 April 

2014.) 

 Public Law 111-358, Section 103, The America COMPETES Reauthorization Act of 

2010 outlines the responsibility of a working group of the National Science and 

Technology Council “to coordinate Federal science agency research and policies related 

5
 
 

 

 

to the dissemination and long-term stewardship of the results of unclassified research, 

including digital data and peer-reviewed scholarly publications, supported wholly, or in 

part, by funding from the Federal science agencies.” 

http://www.gpo.gov/fdsys/pkg/PLAW-111publ358/pdf/PLAW-111publ358.pdf 

(Accessed 10 February 2014.) 

NIST’s Public Access Plan and resulting policies do not rescind any other Department of 

Commerce or NIST policies or guidance and do not alter or supersede existing law or 

regulations, including NIST’s fee recovery authority for the provision of calibrations and 

Standard Reference Materials (15 USC 275c) and Standard Reference Data (15 USC 271-278e), 

which is further articulated in Public Law 90-396, the Standard Reference Data Act. 

6. ROLES AND RESPONSIBILITIES 

The NIST Director 

	 Controls and manages NIST’s Policy and Order on Managing Public Access to Results of 

Federally Funded Research. 

 Ensures coordination of the management of public access to results of federally funded 

research with non-NIST organizations, as applicable. 

Associate Director for Laboratory Programs (ADLP) 

 Implements and provides oversight for maintenance of, and compliance with, NIST’s 

Policy and Order on Managing Public Access to Results of Federally Funded Research. 

 Ensures the availability of appropriate resources for managing public access to results of 

federally funded research. 

 Reviews, approves, and evaluates the effectiveness of NIST OU and Office plans for 

managing public access to results of federally funded research. 

 Ensures compliance with NIST’s Policy and Order on Managing Public Access to Results 

of Federally Funded Research. 

 Coordinates collaboration and cooperation on implementation of the NIST’s Policy and 

Order on Managing Public Access to Results of Federally Funded Research across NIST 

and with the Department of Commerce and other federal agencies. 

	 With the Associate Director for Management Resources (ADMR) and the Associated 

Director for Innovation and Industry Services (ADIIS), coordinates with relevant OUs 

and Offices in their infrastructure planning and implementation to promote 

interoperability across NIST. 

	 With the ADMR and Chief Information Officer (CIO), coordinates with relevant OUs 

and Offices in their infrastructure planning and implementation to promote 

interoperability across NIST. 

	 With the ADMR and CIO, coordinates collaboration and cooperation on implementation 

of this plan across NIST, and with the Department of Commerce and other Federal 

agencies. 

Associate Director for Management Resources (ADMR) 

	 Facilitates the provision of NIST-level infrastructure to manage public access to results of 

federally funded research. 

6
 
 

 Ensures the development and deployment of training, awareness, and outreach activities 

pertaining to the management of public access to results of federally funded research. 

 With the ADLP and ADIIS, coordinates with relevant OUs and Offices in their 

infrastructure planning and implementation to promote interoperability across NIST. 

 Oversees the activities of the Chief Information Officer and the Directors of the 

Information Services Office and Office of Acquisition and Agreements Management in 

supporting NIST’s Policy and Order on Managing Public Access to Results of Federally 

Funded Research, as applicable. 

	 With the ADLP and CIO, coordinates collaboration and cooperation on implementation 

of this plan across NIST, and with the Department of Commerce and other Federal 

agencies. 

Associate Director for Innovation and Industry Services 

	 Oversees the activities of the Directors of the Advanced Manufacturing National Program 

Office, the Baldrige Performance Excellence Program, the Economic Analysis Office, the 

Hollings Manufacturing Extension Partnership, the Technology Innovation Program, and 

the Technology Partnership Office in supporting NIST’s Policy and Order on Managing 

Public Access to Results of Federally Funded Research, as applicable. 

NIST Chief Information Officer (CIO) 

	 Manages NIST-level information technology infrastructure to support NIST’s provision 

of public access to results of federally funded research. 

	 Ensures that the NIST Enterprise Data Inventory (EDI) is available to NIST employees 

and that NIST inventory records are provided to the Department of Commerce and 

government-wide inventories in the necessary format, per Office of Management and 

Budget requirements. 

 Supports NIST OU and Office Directors’ responsibilities (see Section VI.6. of this 

Order), as applicable. 

 With the ADLP and ADMR, coordinates with relevant OUs and Offices in their 

infrastructure planning and implementation to promote interoperability across NIST. 

 With the ADLP and ADMR, coordinates with other agency CIOs and with the Federal 

CIO Council to promote interoperability across agencies. 

Director, Information Services Office 

 Works with the Office of Information Systems Management (OISM) to ensure 

implementation and operation of the NIST EDI. 

 Curates metadata for NIST scholarly publications and scientific research data for publicly 

available repositories. 

 Manages creation and maintenance of persistent identifiers for NIST Technical Series 

Publications. 

 Develops data citation methods to facilitate attribution to NIST scientific data sets. 

 Provides consultation and educational materials for NIST employees on managing data 

and providing public access to results of federally-funded research, including use of the 

NIST EDI, and the NIST review process, as applicable, for results of federally funded 

research that are intended for public dissemination. 

7
 
 

	 Facilitates search and access to metadata for NIST data or final published articles or 

NIST Technical Series Publications for the public. 

	 Supports NIST OU and Office Directors’ responsibilities, as applicable 

Director, Office of Acquisition and Agreements Management (OAAM) 

	 Works with the Directors of NIST OUs and Offices to ensure that activities funded 

wholly or in part by NIST to a non-NIST organization through a grant, cooperative 

agreement, contract, or other agreement include requirements for managing data and 

publications consistently with the NIST directives for Managing Public Access to Results 

of Federally Funded Research, as specified by NIST in the terms and conditions of the 

grant, cooperative agreement, contract, or other agreement with the non-NIST 

organization, beginning October 1, 2015. 

Directors of the OUs and Offices that produce scientific data 

 Implements ADLP-approved plan to manage public access to results of federally funded 

research within his/her OU or Office. 

 Works with other offices, e.g., OISM and the Information Services Office, to manage 

public access to results of federally funded research. 

 Reviews data prior to making it publicly available; authority to carry out this 

responsibility may be delegated to the Division Chief or equivalent. 

 Ensures that his/her OU or Office prioritizes the discoverability and publication of 

applicable OU or Office datasets based on stakeholder needs and resources required. 

 Provide oversight for implementation of the OU-/Office-level plan by units (such as 

divisions, programs, or projects) within the OU/Office. 

 Evaluate the effectiveness of units with the OU/Office in meeting the objectives of this 

plan 

 Coordinate with ADLP, ADMR, and CIO in infrastructure planning and implementation 

to promote interoperability across NIST. 

Supervisory Employee within an OU or Office 

	 Ensures activities under his/her direction are in compliance with his/her OU or Office 

plans to manage public access to results of federally funded research. 

	 Ensures employees under his/her supervision meet employee-level requirements of 

his/her OU or Office plans to manage public access to results of federally funded 

research. 

	 Works with OAAM to ensure that activities funded wholly or in part by NIST to a nonNIST organization through a grant, cooperative agreement, contract, or other agreement 

include requirements for managing data and publications consistently with the NIST 

directives for Managing Public Access to Results of Federally Funded Research, as 

specified by NIST in the terms and conditions of the grant, cooperative agreement, 

contract, or other agreement with the non-NIST organization, beginning October 1, 

2015. 

Non-Supervisory Employee 

	 Complies with the employee-level requirements of his/her OU or Office plans to manage 

public access to results of federally funded research: 

8
 
 prepares and executes DMPs as specified by the OU or Office plans to manage 

public access to results of federally funded research, as applicable, 

 provides metadata for NIST data to the NIST EDI or other publicly available 

repositories, as applicable, 

 if data are tagged as available to the public in the EDI, provides data in open formats 

via publicly available repositories or upon request and to the extent feasible, directly 

to the requestor, free of charge unless otherwise excepted, and 

 provides publications dated October 1, 2015 and later to the NIST public access 

archive system no later than 12 months following publication. 

	 Works with OAAM to ensure that activities funded wholly or in part by NIST to a nonNIST organization through a grant, cooperative agreement, contract, or other agreement 

include requirements for managing data and publications consistently with the NIST 

directives for Managing Public Access to Results of Federally Funded Research, as 

specified by NIST in the terms and conditions of the grant, cooperative agreement, 

contract, or other agreement with the non-NIST organization, beginning October 1, 

2015. 

Awardees and their institutions: 

	 Ensure that authors and investigators comply with all terms and conditions of awards, 

including compliance with the NIST Public Access Policy. 

7. IMPLEMENTATION 

This plan establishes a framework for identifying, managing, and preserving the results of 

federally funded research so as to make them publicly accessible as peer-reviewed publications 

and digital data. NIST’s guiding principles for implementation include the following: 

	 Create flexible approaches and infrastructure to accommodate a wide range of results of 

scientific research as well as a diversity of stakeholders including funded researchers, 

universities, libraries, publishers, industry, civil society, and any other users of NIST 

research results. Policies, processes, and infrastructure that provide meaningful access to 

the results of NIST-funded research for this full range of stakeholders will be developed. 

	 Optimize search, archival, and dissemination features to encourage innovation in 

accessibility and interoperability while ensuring long-term stewardship of the results of 

federally funded research. 

	 Plan for change as the types and volume of scientific information produced with NIST 

funding expands. Extensible and evolvable solutions that can accommodate new needs 

on an ongoing basis are required. NIST will track and respond to continuing changes in 

digital technologies when planning to make research results publicly accessible. 

	 Provide appropriate leadership to promote and enhance NIST’s reputation for highquality output, willingness to work in partnership, and responsiveness to stakeholders. 

Policy 

NIST will adopt a systematic approach to implement a Public Access Policy that includes the 

following: 

9
 
 

 

	 NIST-wide and OU-/Office-level processes for the continual update and evaluation of 

public access and data management policies to ensure that these remain effective and 

relevant into the future and that the evaluation takes into account the relative values of 

long-term preservation and access and its associated costs and administrative burden. 

	 Full and open consultation and cooperation with stakeholders, including those in the 

private sector, through the formation of public-private partnerships with foundations and 

other research-funding organizations, to improve compatibility and access to publications 

and data and explore new approaches to maintain and improve NIST’s public access and 

data management policies. 

	 Public discovery and download of peer-reviewed publications and associated data free of 

charge no later than 12 months following publication. 

	 Attribution of publications to authors, journals, and original publishers. 

	 Effective data management planning for all NIST-funded activities that produce scientific 

data. 

	 Public discovery and access to NIST scientific data. 

	 Clear guidance and access to appropriate education and training materials for NIST staff 

and NIST-funded extramural researchers to help them comply with NIST policies. 

Publications 

NIST will establish a public access archive system to enable the submission of metadata and 

final, peer-reviewed manuscripts or final publications that includes the following functionalities: 

 Allows authors to submit and manage manuscripts directly with the NIST public access 

archive system or through the funding agreement’s NIST Program Official. 

	 Allows submission by the author, the publisher, or the manager of the funding agreement. 

	 Accepts manuscripts in a variety of formats compatible with the current state-of the-art in 

repository architecture. 

	 Accepts any additional files of figures, tables, data files, or supplementary information 

included with the manuscript. 

	 Provides flexible and multiple approaches to manuscript submission. 

NIST will partner with the National Institutes of Health (NIH) to utilize the existing PubMed 

Central (PMC) repository system to serve as the repository of full-text peer-reviewed scholarly 

publications for NIST, leveraging off of the well-established search, archival, and dissemination 

features of PMC. The NIST interface to PMC will: 

 Enable the storage, organization, and management of metadata and contents of peerreviewed publications and associated data collected or submitted under NIST’s Public 

Access Policy. 

	 Be established using an architecture and follow industry standards that facilitate open 

government, enable integration, be machine readable in non-proprietary or widely 

distributed archival formats, and promote interoperability and accessibility. 

10
 
 

 

 

 

 

 

 

 

 

 

	 Have the capacity to integrate peer-reviewed scholarly publications with appropriate 

scientific databases. 

	 Be accessible from NIST websites. 

	 Enable NIST to monitor compliance with the Public Access Policy. 

The NIST interface to the PMC public access archive system will ensure easy search, analysis, 

and download of the full text of peer-reviewed scholarly publications arising from research 

funded by NIST. Public access to the full text will be provided without charge no later than a 

12-month embargo period following publication, although NIST reserves the right to shorten or 

extend the embargo period. Such an extension would be announced in the Federal Register. Full 

public access to publications’ metadata upon first publication will be provided through PMC in a 

format that ensures interoperability with current and future search technology. NIST’s 

responsibilities to ensure public access include: 

	 Properly maintaining the interface system to the PMC archive system to ensure that it is 

reliably available through the Internet. 

	 Using existing archives that are trusted, reliable providers of peer-reviewed scholarly and 

technical literature and are available through the Internet. 

	 Ensuring that the system is accessible to people with disabilities and compliant with 

Section 508a of the Rehabilitation Act (29 USC 794d). 

	 Including the features and capabilities to meet the criteria outlined in the Office of 

Science and Technology Policy Director’s Memorandum dated February 22, 2013. 

	 Providing annual notice in the Federal Register to allow NIST’s customers to petition for 

changing the embargo period in the following year, for publications in a specific field, by 

providing evidence that the current embargo period does not provide a public benefit and 

is inconsistent with the objectives articulated in the OSTP memo. 

All of the material available from the PMC site is provided by the respective publishers or 

authors. Almost all of it is protected by U.S. and/or foreign copyright laws, even though PMC 

provides free access to it. (Public domain material3 is an exception.) Users of PMC are directly 

and solely responsible for compliance with copyright restrictions and are expected to adhere to 

the terms and conditions defined by the copyright holder. Transmission, reproduction, or reuse 

of protected material, beyond that allowed by the fair use principles of the copyright laws, 

requires the written permission of the copyright owners. U.S. fair use guidelines are available 

from the U.S. Copyright Office at the Library of Congress.4,5 

PMC has two services that may be used for automated retrieval and bulk downloading of a 

subset of articles and all the metadata from the PMC archive. The PMC Open Archives 

Initiative (OAI) service and the PMC File Transfer Protocol (FTP) service are the only services 

that may be used for automated downloading of articles in PMC and only a subset of the total 

PMC content, the PMC Open Access Subset6

, is included. Publishers can set licenses allowing 

3 http://www.ncbi.nlm.nih.gov/pmc/about/copyright/#public-domain
 4 For fair use guidelines, see http://www.copyright.gov/fls/fl102.html
 5 For more information, see http://www.ncbi.nlm.nih.gov/pmc/about/copyright/.
 6 http://www.ncbi.nlm.nih.gov/pmc/tools/openftlist/
 

11
 
 

bulk download when they send final published articles directly to PMC. (Articles that are 

available through the PMC OAI and FTP services are still protected by copyright but are 

distributed under a Creative Commons or similar license that generally allows more liberal use 

than a traditional copyrighted work.) 

By partnering with the NIH and its established PubMed Central publication archive, NIST 

ensures the permanent preservation and long-term accessibility of metadata and its peerreviewed scholarly publications free of charge. 

NIST will take a staged approach to providing public access to publications via PMC. A pilot 

exercise, which will include the NIST Journal of Research and the Journal of Physical and 

Chemical Reference Data, will be conducted in year one, establishing the infrastructure for 

transferring metadata and publications to the repository. In year two, deposit of NIST-authored 

peer-reviewed publications will be operational. In year three, extramural publications of 

scientific research funded wholly or in part by NIST will be deposited. 

Data 

To the extent feasible and consistent with applicable law and policy, agency mission, resource 

constraints, U.S. national, homeland, and economic security, and the objectives listed above, 

digitally formatted scientific data resulting from unclassified research supported wholly or in part 

by Federal funding will be stored and publicly accessible to search, retrieve, and analyze. 

NIST’s plan for providing public access to data consists of three components: data management 

plans (DMPs), an Enterprise Data Inventory (EDI), and a Common Access Platform providing a 

public access infrastructure. Work began with a pilot implementation developed according to the 

guidance provided in the Project Open Data component of OMB memorandum M-13-13 and the 

deadline it set of November 9, 2013 for initial implementation. This pilot was initiated with a 

review of NIST reference data and the selection of an appropriate pilot set representative of the 

diversity of data types and domains across NIST. Persistent identifiers and metadata have been 

provided for some of this reference data, and the work is continuing. Lessons learned in this 

pilot will inform the development of NIST’s Enterprise Data Inventory (EDI), described below. 

Generation of data management plans has also begun, providing documentation of plans for 

storage, archival, and accessibility for NIST’s multiple types of data. At a minimum, data 

management plans (DMPs) must contain a summary of activities that generate data, a summary 

of the data types generated by the identified activities, a plan for storage and preservation of the 

data, and a plan describing whether and how data generated will be reviewed and made available 

to the public. 

The EDI is a catalog of the datasets that are generated via NIST-sponsored research to enable 

researchers to link those datasets to the scientific literature, other datasets, etc. The metadata 

describing the scientific data contained in the catalog will include, at a minimum, the common 

core metadata schema in use by the federal government, found at https://project-opendata.cio.gov/. This catalog is part of the comprehensive public listing of agency data that was 

required by the Executive Order of May 9, 2013 and OMB Memorandum M-13-13. The JSON 

file for the current public listing of datasets is provided at http://www.nist.gov/data/index.cfm; 

93 NIST datasets are currently available via data.gov, including a complete listing of NIST 

Standard Reference Data (SRD), which has been critically evaluated using documented 

procedures under the requirements of the Standard Reference Data Act. The NIST EDI serves 

12
 
 

 

 

not as a repository of study data but as an index containing information that describes a data set 

(i.e., metadata) and information about where and how to access the data. An interagency 

technical advisory group has been assembled to provide input to this effort and ensure that the 

reference implementation meets the needs of a wide range of stakeholders. 

The final component, the Common Access Platform (CAP), will use the information gained in 

the first two phases to put in place production-level infrastructure and populate it with persistent 

identifiers and metadata for all publicly available NIST data. The CAP is expected to provide 

for interoperability within NIST and potentially with other federal agencies. This production 

infrastructure will be subject to continuing evaluation, refinement, and revision. NIST will 

assess the long-term needs for preservation of scientific data in fields that the agency supports, 

and outline options for developing and sustaining repositories for scientific data in digital 

formats, taking into account the efforts of public- and private-sector entities. 

All grants, contracts, and cooperative agreements will include requirements for data management 

planning consistent with the goals of the NIST plan. Terms and Conditions will include 

language that requires scientists seeking funding to describe how and where they will make their 

data available to the public and explicitly describe how they will make the data that underlies 

scientific publications available for discovery, retrieval, and analysis. 

Outreach and Education 

In coordination with other agencies and the private sector, awareness and support training, 

education, and workforce development related to NIST’s plans to provide public access to the 

results of federally funded scientific research, including scientific data management, analysis, 

storage, preservation, and stewardship, will be provided to NIST staff and those outside NIST 

who are working on NIST-funded scientific research. 

8. METRICS, COMPLIANCE, AND EVALUATION 

NIST will develop metrics that evaluate compliance with NIST’s Public Access Policy. 

Possible metrics include: 

	 Number of intramural and extramural papers (i.e., articles submitted from NIST grants 

and contracts) made available to the public per year, 

	 Percentage of intramural and extramural papers for which datasets were made available 

immediately upon publication, 

	 Percentage of intramural and extramural papers available in full text after the embargo 

period, per year, 

	 Number of datasets added to the Enterprise Data Inventory per year. 

	 Number of datasets made public per year. 

	 Percentage of staff and grantees in compliance with requirements. 

NIST will utilize data from PubMed Central, other reference sources, grant and cooperative 

agreements, and contract reports to determine compliance; compliance will be enforced through 

annual performance reviews at both staff and management levels. 

DMPs of extramural scientists seeking funding will be evaluated, and the quality of DMPs will 

be considered in funding decisions. 

13
 
 

 

 

9. PUBLIC-PRIVATE PARTNERSHIP 

NIST will take advantage of PubMed Central, which is an existing and accepted public-private 

partnership. Its primary means of disseminating the results of federally funded research is 

through private, peer-reviewed journals rather than through reports published by the federal 

government. The use of a non-proprietary archival language maximizes interoperability between 

public and private platforms, making creative reuse of metadata and contents of publications 

possible. The same potential for reuse exists for NIST’s data and associated metadata. Value to 

all stakeholders is enhanced, and unnecessary duplication of existing mechanisms is avoided. 

The impact of federal research investments is maximized through public access to the results of 

that research. 

10. INTERAGENCY COORDINATION 

NIST will coordinate with other agency partners through the following mechanisms. 

	 NIST is participating in the interagency publications and data implementation working 

groups convened by OSTP to enable interagency coordination in responding to the 

requirements of the February 2013 public access memo. 

	 NIST is an active participant in the Federal Networking and Information Technology 

Research and Development (NITRD) program, including the Big Data Senior Steering 

Group. More than 15 Federal agencies and offices currently participate in the Senior 

Steering Group and its sub-groups for data technologies, research projects, challenges 

and competitions, and workforce development. 

	 NIST has established an interagency Technical Advisory Group to provide input on the 

pilot Common Access Platform reference implementation for integrated access to 

distributed data repositories (see section 7 above). 

	 NIST is coordinating with NIH for the use of the PubMed Central publications repository 

and adherence to established PubMed Central requirements. 

	 NIST will explore the development of a research data commons, a federated system of 

research databases, along with other Departments and Agencies for storage, 

discoverability, and reuse of data with a particular focus on making the data underlying 

the conclusions of peer-reviewed scientific publications resulting from federally funded 

scientific research available for free at the time of publication. 

11. PUBLIC NOTICE 

NIST will work with other executive agencies in publishing a generalized announcement of the 

public access plan in the Federal Register soliciting comment from federally funded researchers, 

universities, libraries, publishers, users of federally funded research results, civil society groups 

and the general public. NIST will post its final Public Access Plan for public comment on the 

NIST website. 

12. UPDATE AND RE-EVALUATION OF THE PLAN 

The plan will be evaluated annually and updated as necessary until NIST’s Public Access Policy 

is implemented. 

14
 
 

13. TIMELINE FOR IMPLEMENTATION 

Key milestones are outlined in each implementation category below. 

DATA PUBLICATIONS 

POLICY 

FY13 August 2013 Submit draft plan to OSTP Submit draft plan to OSTP 

FY14 March 2014 Feedback received from OSTP Feedback received from OSTP 

FY14 May 2014 Submit revised draft plan (data and publications combined) to OSTP 

FY14 June 2014 NIST staff review of NIST Public Access Plan 

FY15 December 2014 NIST Public Access Policy effective 

FY15 December 2014 NIST Public Access Plan posted for public review 

FY15 January 2015 Address public comments in NIST Public Access Plan and Policy, as 

appropriate 

INFRASTRUCTURE 

FY13 August 2013 Data Management Plan (DMP) 

template drafted 

FY14 November 2013 

Initial pilot Enterprise Date 

Inventory (EDI) system 

operational per OMB M-13-13 

Letter of Intent submitted to NIH 

to utilize PubMed Central as NIST 

repository 

FY14 February 2014 

Initial draft of NIST extensible 

metadata schema for scientific 

data 

FY14 May 2014 DMP Tool demo development Draft Business Process for internal 

Editorial Review System 

FY14 June 2014 

Draft reviewed by Editorial 

Review Boards and NIKE 

working group 

FY14 July 2014 

Develop metadata registry and 

handle resolver for Common 

Access Platform (CAP) 

FY14 August 2014 Full DMP Tool development 

Interagency Agreement with NIH 

established for deposit of peerreviewed publications in PubMed 

Central 

15
 
 

 

 

DATA PUBLICATIONS 

FY15 November 2014 Develop data-type registry for 

CAP 

Pilot for depositing example 

journal types to PubMed Central 

established 

FY14 September 2014 

Demonstrated EDI concept to 

Data Policy Group and stood up 

EDI Customers Working Group 

Processes and plans developed to 

facilitate submission of NISTauthored journal articles to NIH 

PubMed Central. 

FY15 December 2014 Data citation recommendations 

made 

FY15 December 2014 Standard language developed Terms and Conditions for grants and 

contracts 

FY15 February 2015 Enterprise Data Inventory (EDI) 

operational 

FY16 October 2015 Common Access Platform (CAP) 

operational 

Editorial Review System for 

publications operational 

PROCESSES 

FY15 October 2014 Data Management Plans required 

for all NIST-funded research 

FY15 February 2015 

Metadata for publicly available 

datasets entered into EDI (and 

datasets shown in data.gov as 

appropriate) 

FY16 October 2015 Standard language regarding public access to data and publications 

included in Terms and Conditions for grants and contracts 

FY16 October 2015 

Submission of NIST intramural 

journal articles to repository is 

operational 

FY17 October 2016 

Submission of NIST extramural 

journal articles to repository is 

operational 

16
 
 

 

DATA PUBLICATIONS 

OUTREACH AND EDUCATION 

FY14 October 2013 Information and input web site 

launched 

FY14 December 2013 Pilot OU Town Hall outreach 

event 

FY14 September 2014 Initial educational resources in 

place 

NIST Town Hall meeting, NIST 

Connections articles, FAQs on 

internal website 

FY15 December 2014 Pilot training session(s) to get 

feedback on EDI 

FY15 September 2015 NIST Town Hall meeting, NIST 

Connections articles 

FY16 October 2015 

Meetings with individual awardees 

and NIST contacts to discuss 

public access to research data and 

publications generated via grants 

and contracts 

14. RESOURCES 

NIST has identified base funding resources within its Scientific and Technical Research and Services 

Appropriations in order to ensure initial implementation, scale-up, and continued operation of the NIST 

system to make publications and data publicly available. 

15. DOCUMENT HISTORY 

Draft approved by OSTP and OMB, 04 December 2014 

Plan posted to www.nist.gov/data, 03 April 2015 

17
 
 

 

 

 

 

 

 

 

 

 

 

 

 

Appendix A. Members of the committees responsible for drafting NIST’s plan for providing 

public access to results of federally funded research. 

Data Policy Group 

Chair – Wo Chang 

Laboratory Programs – Dianne Poster 

Center for Nanoscale Science and Technology – Mark Stiles 

Engineering Laboratory – Kirk Dohne 

Information Technology Laboratory – James St. Pierre, Kathleen Roberts 

Material Measurement Laboratory – John Henry Scott 

NIST Center for Neutron Research – Robert Dimeo, Alan Munter 

Office of Information Systems Management – Susannah Schiller 

Physical Measurement Laboratory – James Olthoff, Aaron Fein 

Program Coordination Office – Heather Evans, Jason Boehm 

Special Programs Office – James Whetstone, Katherine Sharpless 

Standards Coordination Office – Sally Bruce 

Publications Working Group 

Chair – Katherine Sharpless 

Laboratory Programs – Donna Kimball 

Information Services Office – Barbara Silcox, Regina Avila 

Office of Information Systems Management – John Venable, Joseph Kau 

Public Affairs Office – Virginia Covahey 

18
 
 

 

 

 

 

 

Appendix B. Feedback received from NIST staff’s review of draft Public Access Plan and 

responses provided by the Data Policy Group. 

1. It seems NIST's Public Access Plan is not extensive enough and does not fully address 

the ambitious goals and spirit set by OSTP. NIST should do much more. 

It is true that the White House directives set out some very ambitious goals for Federal agencies. 

NIST is committed to meeting these goals, by phasing in new systems and requirements over the 

coming years. To enhance NIST’s current methods of making research data and scientific 

publications available through various mechanisms (e.g., journals, FTPs, NIST website, etc.), 

NIST plans to systematically 

	 identify available datasets by creating Data Management Plans (DMPs) in which 

consequence levels for data preservation, review, and discoverability by applying the 

NIST Data Taxonomy will be mapped, 

	 assign and link persistent identifiers (PIDs) to all publications and datasets 

	 develop the NIST Enterprise Data Inventory (EDI), which will provide a metadata 

catalog containing the dataset descriptions – allowing discovery and access – and a type 

registry that describes the dataset content and structure, and 

	 deposit NIST scientific publications (and associated data if available) in PubMed Central 

and the Federal Digital System (FDSys), two well-established repositories. 

2. The NIST Public Access Plan does not comply with the OSTP and OMB memos that 

state that all research data related to a publication must be open, machine-readable, and 

digitally accessible to the public at the time of publication. 

Under our plan, NIST articles and reports will be discoverable through Google, Google Scholar, 

FDsys and PubMed Central (PMC), and scientific/technical article databases. A search 

mechanism that searches both FDsys and PMC will be available from the NIST website. One of 

the goals of the OSTP requirement is to provide free access to federal research results to the 

public at large, not just to other scientists who are experienced in conducting literature searches, 

therefore it is important that publications be discoverable in multiple ways. Supplementary 

information and datasets associated with publications can be stored with publications in both 

FDsys and PMC. Workflows are currently under development. Metadata will be available as 

soon as a paper is published. After a paper is published, the link to the publisher’s website via 

the digital object identifier (DOI) is provided on the PMC site and readers will be able to access 

the paper via this link if the publisher permits access. 

Furthermore, by implementing the Common Access Platform (CAP) with persistent identifiers 

(PIDs), the PID can be open, machine-readable, and digitally accessible and provide very fine 

granularity for: 

a.	 Data value – provide syntactic and semantic definition of a data point: e.g., a spreadsheet 

cell contains the value 32, which is an integer with description for degrees “in Celsius”. 

b.	 Dataset – provide information such as a general description of the data, content structure, 

application program interface (API) to access dataset, etc. 

c.	 Data repository – provide information for access rights, what network protocol is used, 

etc. 

19
 
 

 

 

 

 

 

Therefore, the CAP can support any standard and proprietary digital format. Once the datasets 

are being assigned a persistent identifier (PID), publications can easily make reference to related 

datasets using PIDs just like DOIs. 

As the NIST Public Access Plan takes effect, and the above-mentioned infrastructure is in place, 

NIST laboratories should also be considering how to improve our ability to provide open, 

machine-readable research data. This effort will require OU leadership to provide additional 

resources and guidance for prioritization based on factors including the reasonableness of effort 

and the stakeholder need. 

3. Based on the NIST Public Access Plan, it appears that NIST will be behind other 

agencies in providing open access to publications and data, and does not take advantage of 

expertise and knowledge available in other agencies. 

NIST is coordinating the Interagency Technical Advisory Group (iTAG) with other Federal 

agencies (DOE, National Archives and Records Administration, Smithsonian, Treasury, Census) 

for gathering operational requirements and insights on how we can maximize access to scientific 

and technical data. The objective is to improve the public's ability to locate and access 

appropriate digital assets while enabling proper long-term stewardship of these assets by 

optimizing archival functionality and (where appropriate) leveraging existing institutional 

repositories, public and academic archives, and community and discipline-based repositories of 

scientific and technical data, software, and publications. 

4. The NIST Public Access Plan seems to be focused on the researchers and data 

generators and not the data consumers or other beneficiaries of the plan. Specifically, are 

researchers the best choice for gatekeepers who decide which data will be discoverable? 

It is important to consider the consumers of NIST data as we implement the NIST public access 

plan. The purpose of the plan and associated documents such as the Data Taxonomy is to provide 

direction and guidance to NIST researchers. Planning for the management and sharing of NISTgenerated data is an individual responsibility. The OU directors and their management teams are 

responsible for deciding which categories of data will be discoverable, and this will vary from 

OU to OU using the taxonomy as a starting point for discussion. However, as stated in the 

taxonomy and plan, OUs must consider stakeholder needs when prioritizing which data sets to 

make publicly available. 

5. The NIST Public Access Plan does not provide a solution that will permit NIST data to 

be discovered across different scientific domains. 

By using the CAP architecture with PIDs as the basic fabric for research data and scientific 

publications, NIST has the unique opportunity of integrating all NIST’s internal and external 

scholarly publications, technical reports, scientific data, and even software for discoverability 

and consumption within as well as across domains. Furthermore, the CAP architecture also 

enables federation of datasets between other agencies’ datasets once these datasets are also 

associated with PIDs. 

6. The NIST Public Access Plan does not provide a solution that will interoperate with the 

mainstream infrastructure in the field of scholarly publications, e.g., integration with 

CrossRef. 

20
 
 

 

 

 

 

When NIST authors submit an article for publication in a journal, publishers participating with 

CrossRef acquire the information that is necessary for use of CrossRef, FundRef, and OrcID. 

That information is obtained regardless of where (or whether) an author’s copy of the paper is 

placed in a repository. The version of record (the publisher’s version with the DOI) will still be 

available directly from publishers’ websites. Access to NIST publications will be possible in two 

ways: (1) the way it occurs now (which is essentially the same approach as provided by 

CHORUS) whereby a reader can access the citation and possibly the article itself if that is 

permitted by the publisher and (2) via PMC or FDSys. 

7. CHORUS would be a better publication repository than PubMed Central because it is 

free to NIST, requires no additional work by authors, provides proper indexing for 

searches, and provides the publisher’s version of record to readers. 

CHORUS and PMC are complementary tools by which we can make our publications available, 

and it is likely that we will use both. We have met with representatives of CHORUS several 

times to discuss options. CHORUS is a portal to publishers’ websites. While CHORUS 

provides citation information and a link to the journal, access to the content of the article can 

often only be obtained by paying a toll (i.e., if a library subscribes to the journal, if the reader 

pays to obtain a copy of a paper, or if the author has paid to make the paper open access) or by 

waiting out an embargo. By using PMC and FDSys as our repositories, we can make the content 

of papers immediately available if all of the papers’ authors are federal employees. (Some 

journals allow federal employees to deposit the version of record in agency repositories.) 

Because of the possible limited access to publications via CHORUS, NIST – like other federal 

agencies including DOE – will also make publications available through a separate repository 

(PMC for peer-reviewed papers, FDSys for NIST Publication Series and non-peer-reviewed 

papers). DOE funds their own repository; NIST has established an interagency agreement with 

NIH so that we can use their repository rather than establishing one of our own. 

NIST’s editorial review processes and requirements are currently being updated. An Editorial 

Review System to replace Nike is being sought, and authors will have to update citation 

information and upload final, peer-reviewed versions of their papers to the new system, just as 

they are required to do now. Depending on a given publisher’s agreement with PMC and/or 

NIST, the publisher may deposit the published version or the author’s version with PMC, or the 

NIST library will do so. PMC clearly marks papers to distinguish the author’s and final 

published versions. Authors will not upload papers to PMC. NIST will index publications in 

FDsys and PMC so that papers in both repositories can be found in a single search. 

The logical endpoint of the work that we do using federal funds is to make results of the work 

available to those who have paid for it. In the past, we ordered reprints; more recently, we might 

have paid a publisher to provide open access to a paper. We can continue to do that, making the 

content of papers available on the publishers’ websites and accessible via CHORUS. We can 

also post papers (as copyrights permit) to other websites. But papers will also be available 

through PMC and FDSys. The only expected additional cost in terms of authors’ time will be 

that associated with the review of XML-formatted manuscripts in PMC. 

21
 
 

 

 

 

 

 

8. The NIST Public Access Plan takes effect in October 1, 2014, but the plan does not state 

clearly what this means for data. The NIST Public Access Plan has a phase-in period for 

publications but not for data. Is this approach realistic? 

NIST will proceed in three phases to implement NIST Public Access Plan. The first phase is the 

implementation of Data Management Plans (DMPs). All data generated beginning October 1, 

2014 must be covered by a DMP that contains a minimum of four elements: (a) a summary of 

data-generation activities, (b) a summary of data types, (c) plans for an appropriate level of data 

preservation, and (d) a description of the appropriate level of data access. The NIST laboratories 

will be responsible for implementing a system for generating DMPs, which may include the use 

of an online tool developed by OISM. For questions about each laboratory’s requirements for 

generating DMPs, contact your DPG representative (http://inet.nist.gov/adlp/howdoi/publicaccess.cfm). 

Phase two involves the Enterprise Data Inventory (EDI). To help make our data more 

discoverable, during FY 2015, NIST will generate a catalog of our datasets, housed within the 

new EDI. It will consist of metadata (i.e., descriptions) of discoverable NIST datasets. 

Laboratory-specific guidelines will help NIST staff determine which datasets should be listed in 

the EDI. If you are interested in participating in testing the EDI, contact NIST Data Coordinator 

Wo Chang (wo.chang@nist.gov). As datasets are discovered and we learn more about our 

customers’ needs, we will prioritize datasets for availability through a publicly accessible 

repository. As the NIST Public Access Plan takes effect, making actual datasets (and associated 

domain-specific metadata) available online will require OU leadership to provide additional 

resources and guidance for prioritization based on factors including the reasonableness of effort 

and the stakeholder need. 

The final phase is that publications will be made freely available. All NIST scholarly and 

technical publications with a publication date of October 1, 2015 or later must be submitted to 

the NIST public-access archive system within 12 months following publication. During FY 2015 

this system will be developed and piloted. The initial pilot will focus on inclusion of two 

journals—the NIST Journal of Research and the Journal of Physical and Chemical Reference 

Data—into the public-access archive system. Other journals will be added to the archive over the 

course of FY 2015. Publications resulting from NIST-funded grants will be deposited in the 

archive beginning October 1, 2016. If you are interested in participating in the pilots, contact 

Open Access Officer Katherine Sharpless (katherine.sharpless@nist.gov). 

9. How will the requirements for DMPs be implemented? 

OU directors are responsible for implementing DMPs within their organizations. Some 

laboratories already have program and project management tools in place that will integrate 

DMP content, others may choose to use the online tool that OISM is developing, and some 

laboratories may determine that they will build a new collection tool. For OU-specific DMP 

implementation, please contact your supervisor or your OU DPG representative 

(http://inet.nist.gov/adlp/howdoi/public-access.cfm). 

10. Are guest researchers covered by NIST Public Access Plan? 

Yes, providing public access will be a NIST policy, and guest researchers agree to abide by 

NIST policies when they sign their guest researcher agreements. 

22
 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

11. How will data that is generated by collaborators or research partners outside of NIST 

be treated under this plan? 

The project lead will be responsible for determining how data will be treated. NIST will not 

release datasets that have been generated by a collaborator unless both parties are in agreement. 

Similarly, in cases where multiple OUs work together on a common activity, the OU with the 

main role in directing the activity has discretionary authority to manage the DMPs and associated 

data. In the case of grant recipients, expectations for data management will be addressed by 

language in Terms and Conditions, which is currently under discussion. 

12. Is deposition of data into a repository required by NIST Public Access Plan, either for 

NIST staff or for our external collaborators or partners? Will NIST stand up data 

repositories for this plan? If so, will they be on-premises, associated with the publications 

repositories, or elsewhere? No repository exists at NIST to repose or disseminate scientific 

data in digital formats, nor does NIST have a plan to provide one. 

If data has been categorized as discoverable, it must be made available. At this time, that might 

occur through a deposit in PubMed Central or in FDSys as supplementary information associated 

with a publication, posting on the NIST website, or by requiring a direct request to the NIST 

researcher who is responsible for the data. Data usage and requests for datasets will allow us to 

identify and prioritize data that should be added to repositories, and will allow us to develop our 

own repositories as appropriate. A centralized data repository is currently being explored; a 

detailed procedure on how to deposit datasets is still in development. 

13. What is the dividing line between data and metadata? 

Data is ‘recorded factual material commonly accepted in the scientific community as necessary 

to validate research findings.’ Metadata (administrative) is the information that would allow you 

to locate that data, much like a card catalog was once used to locate a book in a library. Domainspecific metadata (description of conditions experiment under which the data or modeling result 

were generated) are not part of the administrative metadata. 

14. Metadata creation or capture can be a very time intensive operation. What resources 

are being deployed at NIST to assist NIST staff with this task? 

The goal of the NIST Public Access Plan is to minimize administrative burden on staff 

researchers while being responsive to White House directives. OMB has developed the Common 

Core, a minimum set of required metadata, which is used to catalog datasets generated by the 

funded agency; note that this is not an instrument-level, domain-specific repository but is 

administrative metadata that describes the datasets (e.g., title and description of dataset, name 

and contact information of responsible staff member) and information about where and how to 

access the data through a simple interface. The oversight implementation of the EDI, including 

approval chains, will be developed in the coming months and communicated to all staff. 

15. How will data be reviewed by ERB? There are no details on how review of data will be 

implemented. 

The technical quality of the data itself is the responsibility of the creator, producer and/or 

distributor; its review is delegated to the OU that created, produced, and/or distributes it. ERB 

will be responsible for review of associated documentation. Examples: 

23
 
 

 

 

 

 

 

 

 

 

 

 

 

 

(1) Information about SRD 100 is located at http://www.nist.gov/srd/nist100.cfm.	 ERB did 

not review the webpage or SRD 100 itself but did review the user’s guide, which is 

available from the SRD 100 page and at http://www.nist.gov/srd/upload/Users-GuideSESSA-Version-1-3.pdf. 

(2) A paper is published with data provided as supplementary information. ERB would 

review the entire publication ‘package’ as they normally do. The supplementary 

information would be checked by the ERB sponsor (and by other reviewers) to confirm 

that it met NIST editorial and policy requirements (e.g., SI usage, appropriate 

uncertainties), but the data itself would not be closely examined by the ERB sponsor or 

the board. Technical review is conducted at the group and division levels and by ERB 

readers. 

16. Where can I learn more about the NIST-XM metadata requirements? 

The Data Dictionary for NIST Extensible Metadata Scheme is based on the OMB Common 

Core, available at http://project-open-data.github.io/schema/. The NIST dictionary is awaiting 

finalization of the OMB recommendations. 

17. The NIST data taxonomy is not actually a taxonomy, but rather a description of NIST 

data types. 

Dictionary.com defines taxonomy as “a classification into ordered categories” and we believe 

that is, in fact, what the NIST data taxonomy is. 

18. A list of examples for each category in the NIST data taxonomy would be helpful. 

Some examples of each type of data: 

 Standard Reference Data – defined by Standard Reference Data Act; see 

http://www.nist.gov/srd/ 

 Reference Data – for example, see http://www.nist.gov/pml/data/index.cfm or special 

datasets that NIST provides on the SRD website 

	 Resource Data – data used to develop an economic impact report, 

http://www.nist.gov/director/planning/upload/report03-1.pdf 

 Published Results – results provided in a published paper or on a website or in a publicly 

available dataset 

 Publishable Results – results of an analysis that are provided in a management-reviewed 

report that is intended for internal use only 

 Derived Data – peak areas obtained through integration of raw data, concentrations that 

those peaks represent 

 Working Data – raw data coming off an instrument 

19. The delineations between some of the data categories in the NIST data taxonomy are 

unclear, and some appear to overlap each other. 

Yes, the delineations between some of the categories are meant to be flexible. The OUs will 

determine which of their datasets fit into which category. 

20. Using a data classification system such as provided in the NIST data taxonomy may not 

be practical in the long term. 

The data taxonomy is a starting point, and it can be modified if that should be necessary. 

24
 
 

 

 

 

 

 

 

 

21. In the section on consequence levels, the classes are referred to as “grades of data”,

which along with the triangular nature of the diagram may provide a false impression of a 

data quality hierarchy. 

The second paragraph of the Taxonomy document states that although the taxonomy takes the 

form of a pyramid, the categories are not strictly hierarchical. 

22. What levels of data in the data taxonomy are intended for public release? 

The data taxonomy outlines minimum discoverability requirements for NIST data, based on the 

seven NIST data types (figure 4). However, these are minimum, or baseline, requirements that 

are a starting point for OUs to make decisions about data set discoverability. The data taxonomy 

does not explicitly discuss public release of data, but datasets that are listed in the EDI must be 

made publicly available unless they are identified in the EDI as restricted for public release. 

23. The consequence levels for preservation appear to address only data backup, not true 

preservation methods, and they appear to be redundant with existing NIST IT Security 

plans. 

Although there appears to be redundancy with NIST IT Security plans, we have learned that not 

all NIST researchers are backing up their data. The preservation consequence levels should 

reinforce the need to do that. We understand that there is a difference between backup and 

preservation, but depending on the requirements established within the OUs, backup may be all 

that is necessary for some types of data. 

24. The discoverability consequence levels require a PID be applied to some unpublished 

data. Since most of this data will reside on NIST-internal IT systems, won’t this be a 

problem? 

The PID is a pointer to a location of the data, which can be updated if the data are moved to a 

NIST IT system designed for public access. 

25. How will data usage statistics (e.g., views/downloads/use) be tracked and then 

attributed (i.e., like citations to a journal)? Will these be included in the performance 

evaluation process? 

This is still being discussed but data management plans will be considered in the context of 

performance reviews. 

26. The NIST response to the OSTP states that it will be posted for review in July 2014 and 

that public comments will be addressed in September 2014. Isn't this timeline unrealistic? 

We are slightly behind schedule as we await OMB’s review of our draft plan. 

27. Section 14 of the NIST response to the OSTP states that sufficient resources exist at 

NIST to implement the plan on an ongoing basis, but the plan does not provide cost 

estimates for implementation. Is it appropriate to make this statement? 

The NIST Director will ensure that resources are available for implementation of the plan. 

25
 
Appendix C. Public feedback received in response to the Federal Register Notice of July 7, 

2015. 

Comment Number Submitter 

1 American Society of Mechanical Engineers (ASME) 

2 Association of American Publishers (AAP) and International Association 

of Scientific, Technical, and Medical Publishers (STM) 

3 Clearinghouse for the Open Research of the United States (CHORUS) 

4 Inside Public Access 

5 Jean Public 

6 John Wiley & Sons 

7 RELX Group 

26
 

 

 

 

 

 

 

 

 

 

 

 

 

Two Park Avenue tel 1.212.591.7000 

New York, NY fax 1.212.591.7674 

10016-5990 U.S.A. www.asme.org 

August 20, 2015 

Katherine Sharpless 

National Institute of Standards and Technology 

100 Bureau Drive Stop 4701 

Gaithersburg, MD 20899-4701 

Re: National Institute of Standards and Technology Plan for Providing Access to the Results of Federally Funded 

Research (Federal Register Doc. 2015-16508) 

Dear Ms. Sharpless: 

ASME appreciates the opportunity to comment on the National Institute of Standards and Technology (NIST) Plan 

for Providing Access to the Results of Federally Funded Research (Federal Register Doc. #2015-16508.) 

With over 140,000 members, ASME (the American Society of Mechanical Engineers) is the largest mechanical 

engineering professional organization in the world. Since its founding in 1880, ASME has worked to advance public 

safety and the quality of life throughout the world. ASME’s reputation as a “neutral convener” has been earned 

over these many decades by its deliberate embrace of all stakeholders in the consensus process and in facilitating a 

robust technical peer review process built on integrity and honesty. 

ASME publishes 29 technical journals periodically highlighting the latest engineering research. Our organization 

annually publishes about 3,500 journal articles each year, authored by approximately 9,000 authors and 

researchers, many of whom are US researchers who acknowledge support from the US government. All of which 

are reviewed by some 7,500 subject matter expert editors and reviewers. ASME journals offer the highest quality 

peer-reviewed literature in their respective field of mechanical engineering. In terms of conferences proceedings, 

ASME publishes some 8,000 papers annually, representing the work of some 20,000 authors and the involvement 

of some 15,000 subject matter expert organizers and reviewers. 

ASME endorses the dissemination of the results of all peer-reviewed research, including research supported by 

federal funding, but it must be done in a manner that is sustainable for the publishing community. It is critical to 

protect the authors’ rights to their intellectual property, as well as the critical functions of peer review. Over the 

years, our organization has invested in technologies and innovations that enable and preserve high-quality digital 

peer review, production, distributions, interoperability and discovery of the latest scientific and scholarly works. 

ASME, along with over 400 other members of the scholarly and professional publishing community, is a member of 

the Association of American Publishers (AAP). AAP has submitted comments to NIST providing detailed responses 

to the questions raised in your Request for Comments. We strongly endorse their letter and are in agreement with 

their recommendations, although we would like to elaborate on two of the challenges associated with 

implementing public access. 

Federal investments in scientific research are vital contributors to our nation’s economy and our national security. 

For decades, the U.S. has reaped the benefits of effective public-private partnerships. The best approach for 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Katherine Sharpless 

August 20, 2015 

Page 2 

achieving greater public access for federally funded research is to continue to support public-private partnerships 

which will result in the broad dissemination of materials that analyze and interpret research while preserving the 

critical peer reviewed material, which is considered the "gold standard" of scientific communication and content 

We urge you to partner with us in participating in the Clearinghouse for Open Research of the United States 

(CHORUS), a multi-publisher portal providing access to journal articles reporting on government-funded research. 

ASME, along with over 100 scholarly publishers, is a member of CHORUS which leverages existing infrastructure, 

tools, and services that support sharing, access, discoverability, reporting, and preservation. CHORUS also reduces 

the compliance burden on authors, and in turn increases compliance with public access goals. 

ASME, along with the other scholarly publishers who are engaged in CHORUS, would welcome the opportunity to 

develop a partnership with NIST that would provide access to federally funded, peer reviewed research articles, 

while preserving the scholarly value of the peer reviewed version of record and maintaining an ever-growing 

archive in perpetuity. CHORUS would advance access without any new federal investments or burdensome 

requirements on researchers or their institutions. The U.S. Department of Energy (DoE) is currently utilizing 

CHORUS as a component of its model for providing public access to peer-reviewed articles that report on DOEfunded research. 

We appreciate that your plan also provides an opportunity to be able to petition to lengthen embargo periods, as 

well as your recognition of the need to “effectively promote the quality and sustainability of scholarly publications 

while meeting the objectives of public access.” In the comments provided by AAP, they provide various examples 

of evidenced based research, but a study that was prepared by Dr. Philip Davis entitled “Journal Usage Half-Life,” 

specifically addresses usage half-lives of journals both within and across subject disciplines, including engineering. 

As stated in his study, “Just 3% percent of journals had usage half-lives of 12 months or less, which ranged from as 

low as 1% for Life Sciences Journals to as high as 6% for Engineering journals.” Dr. Davis analyzed usage data of 

more than 2,800 journals from 13 scholarly publishers to show that the median half-life of journals is distributed 

between 24 and 60 months. In the case of engineering and technology subjects, the median half-life is greater than 

36 months. We urge you to take his findings into consideration. 

In the absence of any evidence to support 12 months as an appropriate embargo period, ASME recommends a 24 

month embargo to ensure that we can fulfill our mission to the engineering community, as well as authors and 

scholars worldwide. 

We appreciate the opportunity to provide you with our comments on the NIST Public Access Plan. We would 

welcome the opportunity to engage with NIST to discuss ways to determine appropriate and consistent 

implementation of embargo periods for specific scientific fields, and how to maximize the effectiveness of CHORUS 

to ensure the full benefits, reduced costs and decreased administrative burdens to NIST. 

Thank you for the opportunity to present our views. Please contact Kathryn Holmes, Director, ASME Government 

Relations (holmesk@asme.org; 202.785.7390) if we can be of assistance. 

Sincerely, 

Philip DiVietro 

Managing Director, ASME Publishing 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

August 20, 2015 

Katherine Sharpless 

National Institute of Standards and Technology 

100 Bureau Drive Stop 4701 

Gaithersburg, MD 20899-4701 

public-access@nist.gov 

Re: Request for Public Comment: National Institute of Standards and Technology Plan for Providing 

Public Access to the Results of Federally Funded Research (Federal Register Doc. 2015-16508) 

The Professional and Scholarly Publishing Division of the Association of American Publishers (AAP/PSP) 

and the International Association of Scientific, Technical, and Medical Publishers (STM) appreciate the 

opportunity to comment on the National Institute of Standards and Technology (NIST) Plan for Providing 

Public Access to the Results of Federally Funded Research (the Plan). AAP/PSP and STM are the major US 

and international trade associations for professional and scholarly publishers; like NIST, our 150+ 

members are guided by a commitment to advancing science, standards, and technology. Our members 

focus on creating and preserving the best scholarly communication, validated through peer review and 

disseminated worldwide to inspire new avenues of thought and advance discovery and innovation. 

AAP/PSP and STM members include non-profit professional societies, commercial publishers, and 

university presses that create books, journals, computer software, databases, and electronic products in 

virtually all areas of human inquiry and activity. Collectively, they represent tens of thousands of 

publishing employees, editors and authors, and other professionals throughout the country who 

regularly contribute to the advancement of American science, medicine, learning, culture and 

innovation. They comprise the bulk of a $10 billion publishing industry that contributes significantly to 

the U.S. economy and enhances the U.S. balance of trade. 

Our members publish the vast majority of materials used in the U.S. by scholars and other professionals 

in the sciences and other areas of scholarship, and they are the worldwide disseminators, archivists and 

shapers of the scholarly record in both print and electronic form. They make significant intellectual 

contributions and investments that improve the quality, discoverability, and availability of peerreviewed articles and other publications. A major goal of our members’ publishing activities is to help 

produce and provide access to high-quality peer-reviewed articles in a useful and user-friendly digital 

environment that enables researchers and other readers to discover, analyze, and link to the latest 

breakthroughs and developments in scholarly research. In particular, publishers of scientific and medical 

journals have, for more than 100 years, played an integral role in building and documenting the U.S. 

scientific research enterprise. Our members therefore are integral partners with the scholarly research 

community in the U.S. and with NIST as it seeks to promote research and innovation. 

1
 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

AAP/PSP, STM, and our members have supported the principle that the public should have access to 

articles that report on federally funded research. AAP/PSP publicly supported the February 22, 2013 

Executive Office of the President Office of Science and Technology Policy memo on “Increasing !ccess to 

the Results of Federally Funded Scientific Research” (the OSTP memo), and our members have been 

working for years on efforts to promote sustainable public access. These efforts include free or low-cost 

access to articles for target communities through Research4Life (in partnership with the United 

Nations), the Emergency Access Initiative (in partnership with the National Institutes of Health), 

patientINFORM (in partnership with health advocacy organizations), and patientACCESS, among others. 

They also include innovative business models like article rental and delayed access that allow for easy 

free or low-cost access in a sustainable system. Many of our members voluntarily provide free access to 

all articles that they publish after a delay that is appropriate for their journals’ disciplines and practices; 

Our members, as well as AAP/PSP and STM on their behalf, have participated in and supported many 

public-private partnerships to deliver value to the public, and they are supporting the collaborative 

effort of CHORUS (the Clearinghouse for Open Research of the United States) to deliver public access in 

a way that minimizes costs for the public, agencies, researchers, and publishers alike. 

In this spirit, we welcome the opportunity to comment on NIST’s Public Access Plan. While we 

understand that the Plan has been carefully crafted and integrates comments from stakeholders 

throughout the process, there are many details that will still need to be worked out in the 

implementation stage and in future revisions to the plan; We appreciate NIST’s commitment to having 

comments received “inform NIST as it revises its directives that implement the Plan;” 

The comments below focus first on the second question prompt, as our comments here inform those 

that follow. We then treat the other three prompts in the order provided. All comments and 

recommendations are meant to support NIST’s successful implementation of the Plan in collaboration 

with all scholarly communication stakeholders, advancing public access while ensuring a sustainable 

publishing environment. 

What are the biggest challenges to implementing a public access policy, and how can these challenges 

be addressed? 

1.	 Free access to articles has the potential to destabilize scholarly communication. NIST should 

maintain its commitment to proceed carefully, incrementally, and in close consultation with 

stakeholders to avoid unintended consequences 

We recognize that the implementation schedule takes an incremental approach and provides for regular 

reviews of the impact of the policy, which we believe is appropriate. We also appreciate the opportunity 

for all who are affected by the policy to comment, including through this Federal Register notice. In 

particular, we appreciate that the evaluation will consider “the relative values of long-term preservation 

and access and its associated costs and administrative burden;” 

Several sections of the Plan indicate that NIST will regularly consult with the scientific community, 

including publishers, and maintain ongoing discussions. Our associations and their members have 

engaged in consultations with a coalition of agencies at the National Academies (in May 2013) and the 

Forum on Open Science and look forward to continued engagement. We would welcome even more 

open communication as the Plan is implemented. 

2
 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

2.	 The burdens of implementation on researchers, research institutions, publishers, and NIST could 

be considerable and grow exponentially. Consider more extensive public-private partnerships to 

reduce burdens and support the sustainability of scholarly communication. 

The Plan’s commitment that NIST “will establish a public access archive system” may have prevented 

NIST from considering options that could achieve its goals at lower cost and regulatory burden. We hope 

that NIST will consider leveraging existing resources where possible, as some other agencies have done 

in the development of their plan. 

Although the Plan refers to PubMed Central as a public-private partnership, publishers have often felt 

that the implementation of PMC and the NIH Public Access Plan do not fully take their concerns into 

account. Some of these concerns include the diversion of traffic from publisher platforms,1 the difficulty 

in obtaining usage data on publisher content accessed through the National Library of Medicine (NLM), 

and the reformatting of published content for delivery through NLM. In addition, some of the tools 

available to PMC under the NIH Public Access Policy and mentioned in the NIST plan may not be 

available for articles reporting on research funded by NIST, which are not covered by current publisher 

agreements with NLM. 

We would welcome the opportunity to work with NIST to address some of these issues, consistent with 

the Plan’s commitment to exploring new approaches and partnerships and the “guiding principle” that 

NIST may make changes to accommodate new technologies and find solutions that scale with the 

volume of research and public access solutions. In particular, NIST should consider solutions offered by 

non-governmental organizations, particularly CHORUS (the Clearinghouse for Open Research of the 

United States), that could significantly reduce the regulatory burden of the Plan, lower costs for the 

government and grantees, and support the sustainability of scholarly communication. 

The potential regulatory burden of the Plan should not be underestimated. According to the Association 

of !merican Universities, it takes “23 steps and several emails for authors to submit manuscripts to 

PubMed entral” to comply with the National Institutes of Health’s Public !ccess Policy, and others have 

suggested that the requirement is even more burdensome. A major study of the experience in the 

United Kingdom in implementing public access mandates quantifies the burden on UK researchers. The 

study found that the cost to research organizations of implementing mandates put in place by Research 

Councils UK and the Higher Education Funding Councils in 2013/14 was at least £9.2m and an amount of 

time equivalent to 110 fulltime staff members; The study also found that the compliance burden “falls 

disproportionally on smaller institutions;” NIST should take steps to minimize any unnecessary costs and 

burdens in its implementation. 

Distributed systems like CHORUS may also help with issues of integrating articles with appropriate 

databases and interoperability. Our members have found that integrating links to articles hosted by 

other publishers is possible in a single management system, and that services like CrossRef (which 

underlie some CHORUS services) facilitate such linking. CHORUS directly supports current policies and 

practices of researchers and publishers in providing access to articles and leverages resources already 

1 See Davis PM. 2013. Public accessibility of biomedical articles from PubMed Central reduces journal readership—retrospective 

cohort analysis. FASEB Journal 27 (7): 2536-2541 http://dx.doi.org/10.1096/fj.13-229922 and Davis PM. 2012. The Effect of 

Public Deposit of Scientific Articles on Readership. The Physiologist 55: 161-5 http://www.theaps.org/mm/Publications/Journals/Physiologist/2010-present/2012/October.pdf 

3
 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

invested in by publishers. We understand that NIST is working on a pilot project with CHORUS and hope 

that this will lead to a true public-private partnership. 

The Plan’s staged approach to implementation should allow NIST to fully consider CHORUS and other 

possible options for providing access to support the research community in fully enabling public access. 

In contrast to the burdens reported for compliance with PMC, such solutions could enable automated 

compliance, reduce unnecessary federal investments, and allow researchers and their institutions to 

focus on research rather than administrative tasks. 

We would also appreciate clarification of the requirement that authors submit copies of manuscripts to 

the NIST public access archive. In particular, this appears to conflict with the description of the utility 

NIST envisions from PMC, and would prevent researchers and NIST from taking advantage of existing 

infrastructure to minimize burdens for researchers, funded institutions, and publishers. Where possible, 

we recommend that NIST utilize and build on existing, independent, third-party services, including 

distributed systems like CHORUS and archive solutions like Portico and CLOCKSS. 

3.	 Differences in scholarly communication practices among disciplines means that NIST’s public 

access policy will impact some fields more than others. NIST should provide additional 

opportunities to change the embargo for fields that have longer article lifecycles. 

We appreciate that the Plan includes an opportunity to petition for changing the embargo, and that the 

first mention of a 12-month embargo is modified by the right for NIST to change the embargo. We hope 

such an adjustment will be considered quickly, as the initial 12-month embargo for all disciplines is 

inconsistent with evidence and the need to sustain the quality, integrity, and availability of high-quality 

peer-reviewed articles reporting on scientific research. 

Rather than setting a 12-month embargo for all articles, an evidence-based policy would recognize the 

differences among practices in various fields and set embargoes appropriately. According to a study by 

Dr. Phil Davis evaluating usage patterns in more than 2800 journals across 10 disciplines,2 half of the 

lifetime downloads for the articles published in a volume occur three years or more after publication for 

the majority of journals and this “half-life” varies considerably by discipline. This is important 

information, because usage is a key criterion for library subscription decisions.3 Surveys4 and direct 

communication with librarians further indicate that short embargoes could reduce the incentive to 

subscribe. 

These results are consistent with the experiences of some of our members and with what is known 

about the use of articles by researchers. The American Psychological Association (APA) found that less 

than 16 percent of the usage of APA psychology journals occurs in the first year, and the American 

Mathematical Society (AMS) found that only 10 percent of the citations in the mathematics literature 

were to articles published in the previous three years combined. In addition, as reported by NSFsupported researchers at Indiana University, some papers in some fields can “remain dormant for years 

2 Phil Davis, “Journal Usage Half-Life;” www.publishers.org/usagestudy. 3 For a survey of the research on cancellations related to usage data, see J; Williamson, P; Fernandez, and L; Dixon, “Factors in 

Science Journal ancellation Projects: The Roles of Faculty onsultations and Data,” Issues in Science and Technology 

Librarianship 78, Fall 2014. http://www.istl.org/14-fall/refereed4.html. 4 See, e.g., http://blog.alpsp.org/2009/10/alpsp-survey-of-librarians-report.html 

4
 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

and then suddenly explode with great impact upon the scientific community;”5 This indicates that usage 

varies significantly between and among journals. 

The importance of such evidence to the sustainability of journals and maintenance of subscriptions is 

supported by experience. To cite a few examples: 

 The Genetics Society of America piloted a 3-month embargo for GENETICS and had “a high rate 

of subscription cancellations”;

6 

 The American Journal of Pathology tried a 6-month embargo, and “subscription renewals 

declined precipitously”;

7 and 

	 The Journal of Clinical Investigation found a too-short embargo unsustainable. After a 10-year 

experiment that saw the journal lose 40 percent of its institutional subscriptions, it had to 

reinstitute the subscription model to survive.8 

Although each of these examples involves an embargo shorter than 12 months, each of them also 

involves a journal that publishes in the health sciences, which is the fastest-moving field and has the 

highest level of federal support. We would expect that journals in other fields would have similar issues 

with uniformly imposed 12 month embargoes. 

Throughout the world, funders have implemented policies that recognize inherent differences among 

the practices of different disciplines, and we urge NIST to do the same. As one example, in the United 

Kingdom, policymakers have instituted as a starting point a 24-month embargo for articles in social 

science and humanities journals and 12 months for other disciplines. We recommend that NIST use the 

Davis study and other evidence to set differentiated embargoes by discipline, as suggested in the OSTP 

memo. 

We also recommend that NIST consider expanding its criteria and approach to petitions for changing the 

embargo; We call your attention to language in the National Science Foundation’s Public !ccess Plan, 

which calls for “factually and statistically based evidence that a change < will more effectively promote 

the quality and sustainability of scholarly publications while meeting the objectives of public access;” 

We also recommend that you allow any stakeholder, not just “NIST customers,” to initiate such a 

petition. 

The wrong policy carries the risk of undermining the quality and sustainability of scholarly 

communication and thereby reducing the availability to NIST-funded researchers of established, highquality journals in which to publish. 

4.	 The costs for public access could quickly escalate, even as resources for research are constrained. 

NIST must ensure adequate resources are available to support allowable costs for access to 

publications and data. 

Existing Department of Commerce practice allows grant proposals to include funds for communicating 

the results of the research, and the Plan notes that “costs for data preservation and access may be 

5 NSF News from the Field, “Like Sleeping eauty, Some Research Lies Dormant for Decades, IU Study Finds,” May 25, 2015. 

http://www.nsf.gov/news/news_summ.jsp?cntn_id=135258&WT.mc_id=USNSF_195&WT.mc_ev=click. 6 http://www.whitehouse.gov/sites/default/files/microsites/ostp/scholarly-pubs-(%23293)%20gsa.pdf. 7 http://www.whitehouse.gov/sites/default/files/microsites/ostp/scholarly-pubs-(%23259)%20ASIP%20response.pdf. 8 Reported in http://scholarlykitchen.sspnet.org/2009/02/26/end-of-free-access. 

5
 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

included in grant proposals or project plan budgets for contracts;” Although the Plan does not mention 

costs for publication explicitly, we hope that such costs for publishing and broadening access to articles, 

if applicable, continue to be allowed. 

In addition, grantees frequently need supplemental funds for publication or for data access that were 

unforeseen at the time of the proposal, or for expenses that have been incurred after the grant period 

has ended. We encourage NIST to make sure that sufficient funds are available to cover all such 

expenses and also that “no-cost extensions” of grants are available without prejudice to enable the full 

communication of research findings whenever articles may be ready to be published. 

5.	 Public access policies could undermine intellectual property and copyright, and affect American 

competitiveness by shifting costs to the US that are currently borne worldwide. Such potential 

negative impacts should be regularly assessed and addressed. 

We appreciate the Plan’s commitment that “NIST will<recognize <; intellectual property rights, avoiding 

significant negative impact on intellectual property rights, innovation, and U;S; competitiveness;” Such a 

commitment needs to be followed with regular assessment and adjustment to the policy where there 

could be impact. 

The Plan acknowledges that even when free access to content is provided , such content is still 

protected by copyright law. As the Plan is implemented, it must continue to allow authors, publishers, 

and other copyright holders to set licenses that are appropriate to the material covered, and avoid any 

overreach in terms of usage rights requirements. Appropriate licenses, together with appropriate 

embargoes, can help support the sustainability of scholarly publishing and ensure the quality and 

integrity of the scholarly record. 

Inasmuch as the Plan is designed to promote U.S. economic competitiveness, care must also be taken to 

combat piracy by clearly communicating license terms and taking steps to prevent bulk downloads 

where the license does not allow such use. 

What are the best practices (from academia, industry, and other stakeholder communities) in 

managing public access of data? 

In addressing issues related to public access to data, flexibility is key. As with scholarly communication 

practices with respect to articles, different fields have unique research practices. The Plan implicitly 

recognizes this by focusing on Data Management Plans (DMPs), rather than creating universal 

requirements for all researchers. 

In addition to publisher efforts to create and disseminate publications that report on and analyze the 

latest research, publishers have considerable experience and have made significant investment in digital 

technology, metadata standards, and tools to help users understand and work with data. Publishers 

support the discoverability and reuse of scholarly data and are actively working with researchers and 

standards organizations to develop tools and processes to ensure the availability and utility of such data. 

NIST should capitalize on existing standards and practices in different fields, and avoid creating new and 

conflicting requirements. 

6
 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

As with sharing publications, there are considerable burdens to formatting data, tagging it with 

metadata, and otherwise preparing for the sharing of data. Publishers have found that collaborative 

private sector solutions can minimize costs and burdens for all involved, and we encourage NIST to 

leverage existing efforts. Publishers are working with repositories to better link to data, ensure quality 

citation, and develop metadata standards, and we look forward to continuing our partnerships with 

research communities in ensuring the quality and integrity of available data collections. 

NIST could learn from initiatives already underway to standardize metadata and provide links between 

sources of research information. Issues related to expanding access, managing data, minimizing 

compliance costs, and other policy questions are already being worked through by various groups 

engaged with the issue. We encourage the continued evolution of programs that are working to improve 

data stewardship and public access to data. These include the Research Data Alliance (RDA), CrossRef, 

DataCite, Opportunities for Data Exchange (ODE), APARSEN, and the NISO/NFAIS Supplementary Journal 

Articles Material Project, among others. Such collaborative approaches provide the best way forward 

towards broad access to and preservation of digital data. 

It is critical that NIST continue to 

distinguish between data and various 

types of presentation of data and 

preserve and respect intellectual property 

protection and copyright ownership as 

appropriate. The Data Publications 

Pyramid displayed here,9 derived from 

open science pioneer Jim Gray’s e-science 

pyramid, provides a model for 

understanding how research data can be 

presented in a variety of ways with 

increasing levels of curation and analysis. 

NIST has already acknowledged the 

different levels of the pyramid by 

referring to “peer-reviewed publications 

and associated data” separately from 

“NIST scientific data;” !s the Plan is 

Publications

with

data

Processed Data and 

Data 

Representations

Data Collections and

Structured Databases

Raw Data and Data Sets

(1) Data 

contained and 

explained within

the article

(2) Further data 

explanations in 

any kind of 

supplementary

files to articles

(3) Data 

referenced from

the article and 

held in data 

centers and 

repositories (4) Data 

publications, 

describing

available datasets

(5) Data in 

drawers and on

disks at the 

institute

The Data Publication Pyramid

implemented, NIST must continue to distinguish between information products at different levels of the 

pyramid and work with all stakeholders, including primary researchers, secondary researchers, 

publishers, libraries and data centers, to create clear rules and protocols for the sharing of data. A 

collaborative approach will ensure that the needs of each stakeholder group are addressed and that the 

progress of science is not impeded. In particular, the need to expand incentives for providing broad and 

timely access to new data must be balanced with the need to preserve incentives for researchers to 

interpret and analyze their results through curation and peer-reviewed publication. 

Rather than imposing an inflexible mandate, NIST should focus on supporting and encouraging the 

development of cyber infrastructure, standards for the structure of data and metadata, navigation tools 

9 As appearing in the October 17, 2011 Report on Integration of Data and Publications, a report of Opportunities for Data 

Exchange which brings together stakeholders including researchers, publishers, libraries and data centers to support a more 

connected and integrated scholarly record. Full report available at http://www.alliancepermanentaccess.org/wpcontent/uploads/downloads/2011/11/ODE-ReportOnIntegrationOfDataAndPublications-1_1.pdf 

7
 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

and applications to achieve discoverability and interoperability and ensuring appropriate and 

sustainable funding for innovation and long-term stewardship. These policies should be developed in 

collaboration with all key stakeholders involved in the presentation, analysis, deposit, storage, and 

preservation of data. NIST should promote a comprehensive framework for reliable digital data 

preservation, access, and interoperability through the promotion of standards and clear rules developed 

by the scholarly community. NIST could also support pilot projects, data curation programs, and 

interpretation initiatives for the relevant scholarly disciplines. 

With regard to the top of the data pyramid and data associated with articles, NIST should encourage 

data to be deposited in accordance with the policies of the publication and standard citation practices, 

such as those being developed by the National Information Standards Organization (NISO) and others. In 

addition, NIST should recognize the need for different repositories for different communities. This is 

consistent with distributed approaches that have been successful throughout the research community 

and which could be successful for publications as well. 

What criteria could the NIST laboratories use in prioritizing the public release of datasets? 

As noted above, NIST should proceed carefully in imposing new burdens on researchers and in 

developing the capacity to accurately and consistently provide quality access to datasets. The best way 

to proceed would be to prioritize first those data that are already being made available and/or which 

already have proper formatting and tagging to ensure their integrity and utility. These data tend to be 

those towards the top of the pyramid. Publishers are already working with researchers to make such 

material more widely available and link them to articles that put the data in context and provide 

additional background on its collection. 

NIST could also prioritize those data that are most useful to the public. Statistics on the use publicly 

available data can be obtained from existing data repositories or the use of material already available to 

NIST. Where the usage data is available to NIST through its own means, it would be helpful for NIST to 

share such data with content providers or the broader public. Publishers have helped develop the 

COUNTER system for accurate usage data collection on the usage of publications, and the sharing of 

COUNTER-compliant data by NIST would help improve the release and usage of datasets. 

How can NIST improve its plan to provide greater public access to NIST data? 

In addition to the recommendations above, the best way to improve the plan is through continued clear 

communication and engagement with scholarly community. The Plan consistently and repeatedly 

indicates that NIST will be actively engaged in consultation with all parts of the scholarly community, 

and we are encouraged that the Plan explicitly notes that implementation will take place over time. We 

hope that this will enable NIST to carefully assess the impact of its policies and adjust the plan as 

needed, in consultation with all stakeholders. We all urge NIST to consider both positive and negative 

effects on scholarly communication and the availability of diverse, high-quality peer-reviewed articles 

and research data in any analysis of the Plan. The high-quality peer reviewed articles that our members 

produce represent significant investments by publishing organizations to improve, disseminate, 

interpret, and steward those reports, and the ability to continue making those investments and ensuring 

the quality and integrity of the scholarly record depends on the sustainability of the publishing 

enterprise. 

8
 
 

 

 

 

 

 

 

 

 

 

Thank you for the opportunity to comment on the Plan. We look forward to future opportunities to 

provide input on and collaborate in ensuring access to high-quality peer-reviewed scholarly 

communication. 

Sincerely, 

John Tagler Michael Mabe 

Vice President & Executive Director Chief Executive Officer 

Professional & Scholarly Publishing International Association of STM Publishers 

Association of American Publishers, Inc. Prins Willem Alexanderhof 5 

71 Fifth Avenue, New York, NY 10003 The Hague, 2595 BE 

455 Massachusetts Ave, Washington, DC 20001 The Netherlands 

jtagler@publishers.org Mabe@stm-assoc.org 

Phone: 212 255-1407 Phone: +44 1865 339321 

9
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

August 21, 2015 

Katherine Sharpless 

National Institute of Standards and Technology (NIST) 

100 Bureau Drive Stop 4701 

Gaithersburg, MD 20899-4701 

RE: National Institute of Standards and Technology Plan for Providing Public Access to the 

Results of Federally Funded Research 

CHORUS (Clearinghouse for the Open Research of the United States), a US 501(c)(3) non-profit 

organization, appreciates this opportunity to respond to the Request for Comments on the NIST Public 

Access Plan. 

By leveraging existing infrastructure, tools, and services that support access, sharing, discoverability, 

reporting, and preservation, CHORUS helps research funders effectively and sustainably facilitate public 

access. CHORUS members and signatories collectively produce the vast majority of published peerreviewed journal articles. We currently track an estimated 9,000 journals that our members collectively 

publish and monitor over 135,000 articles reporting on funded research for public access and long-term 

availability, more than 30,000 of which are already freely accessible to the public. CHORUS reports this 

data via 24 agency dashboards, which also provide access to each article we monitor. CHORUS has made 

excellent progress working with funding agencies to support their approach to providing public access. 

The US Department of Energy and Smithsonian Institution have signed participation agreements with 

CHORUS, and the NSF, NASA, NOAA have all directly mentioned CHORUS in their plans. We are in 

active negotiations with and, in some cases, are pursuing pilot projects with eight other U.S. federal 

agencies. 

Like NIST, CHORUS is committed to delivering a public-access solution that avoids duplication of 

effort, minimizes administrative costs, and maximizes compliance. We look forward to continuing our 

collaboration with NIST in a pilot project to achieve these important objectives. 

What are the biggest challenges to implementing a public access policy, and how can these 

challenges be addressed? 

As NIST develops and implements its public-access policy, it will be critical to ensure that its policy 

achieves the goal of expanding public access without imposing undue administrative burden and costs on 

taxpayers and researchers. We applaud NIST’s recognition of this challenge and ongoing commitment to 

engage in “full and open consultation and cooperation with stakeholders,” “create flexible approaches” to 

accommodate the range of research and diversity of stakeholders, “plan for change,” and adopt “evolvable 

solutions” that respond to new needs and digital technologies, and develop processes for the “continual 

update and evaluation” of policies to ensure effectiveness and relevance while also weighing the “costs 

and administrative burden.” The CHORUS-NIST collaboration is guided by these principles and 

represents a unique and important opportunity to leverage innovative technologies to better serve the 

public need. We greatly appreciate the openness of NIST leadership to partnering for the public benefit. 

CHOR, Inc. | 72 Dreyer Avenue, Staten Island, New York 10314 | 917-251-9235 | www.chorusaccess.org 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

NIST has also indicated that it will require its funded investigators submit an electronic version of final 

peer-reviewed journal manuscripts to PubMed Central (PMC) upon acceptance for publication. We are 

hopeful that NIST can help to actively support robust public-private partnership and to engender a more 

open collaboration around public access policy and policy implementation than is currently the case 

between PMC and the publishing community. One way would be to work with NIH to embrace open 

international standards – particularly DOIs, CrossRef’s FundRef, ORCID, and COUNTER usage 

statistics – to improve interoperability between its PMC infrastructure and other platforms on which 

versions of the same articles appear. In our unique role as a non-profit organization set up to provide 

agencies and publishers with a sustainable solution to delivering public access to published articles 

reporting on federally funded research, CHORUS is well-placed to help facilitate and support this richer 

engagement among NIST, NIH, and the full range of other stakeholders, including libraries, universities, 

scholarly societies, and research organizations. 

What are the best practices (from academia, industry, and other stakeholder communities) 

in managing public access of data? How can NIST improve its plan to provide greater 

public access to NIST data? 

CHORUS supports NIST’s efforts to encourage researchers to make their research data freely available, 

with easily discoverable reuse restrictions. CHORUS engages with stakeholders to develop tools and 

services to support researchers with data management, and to help funding bodies with compliance 

tracking. We would welcome the opportunity to collaborate with NIST and other stakeholders – national 

and international, and across all disciplines and sectors – to leverage existing systems, experience, and 

expertise, in order to ensure minimal duplication of effort and maximum efficiency. 

In particular, the need to develop – and continue to develop – data standards is critical. We strongly 

encourage NIST to actively partner with the organizations already overseeing the development and 

management of these standards and to recommend and deploy existing tools such as DOIs, FundRef, and 

ORCID to grant recipients for their data management. CHORUS would be very interested in working 

with the NIST, publishers, data-archive managers, and other stakeholders on developing a data-validation 

system, as well as mechanisms to connect articles and related datasets, for example, via developing 

publishers’ systems to enable authors to submit their data to an appropriate archive and simultaneously 

link this to an article. I work with the CrossRef–DataCite Pilot, SHARE, and the RDA-WDS Publishing 

Data Services Working Group, and CHORUS is in discussions with the RMap 

Project, Dataverse, Figshare, and Dryad.

Sincerely,

Howard Ratner 

Executive Director, CHORUS / CHOR, Inc. 

h.ratner@chorusaccess.org 

917-251-9235 

CHOR, Inc. | 72 Dreyer Avenue, Staten Island, New York 10314 | 917-251-9235 | www.chorusaccess.org 

 

To: Katherine Sharpless, 

NIST 

By email to public-access@nist.gov 

August 12, 2015 

Inside Public Access comments on the NIST Plan for Providing 

Public Access to the Results of Federally Funded Research (NIST 

Public Access Plan). 

NIST officially requested comments on its Public Access plan in the July 7, 2015 Federal 

Register. Inside Public Access is a newsletter and consulting service with a deep interest 

in the US Public Access program. We are especially interested in making Public Access 

work as well as possible, while minimizing the burden on the research community. 

We are therefore pleased to make the following comments: 

The legal background here is that the Paperwork Reduction Act (PRA), which NIST 

specifically cites, requires that all federal agencies obtain a clearance before they collect 

information from the public. Requiring authors to submit copies of their published 

articles is certainly an information collection under PRA. 

This request for comment is presumably a step in the clearance process, or at least feeds 

into it. NIST may even change its Public Access Policy in response. In any case it will 

then submit an Information Collection Request (ICR) to OIRA. Central to the ICR is a 

so-called burden estimate, which typically focuses on the hours of public labor required 

to understand and comply with the collection requirements. 

The point is that this is the time for interested parties to raise whatever issues they have 

with NIST's proposed Public Access Policy. The primary issue areas are the need for the 

articles, how to maximize their value and how to minimize the burden of providing them. 

CHORUS obviously plays into the burden issue, so NIST needs to justify not using 

CHORUS. 

Statutory authority for the collection may also be an issue because there is no clear 

authority given by Congress for the US Public Access program. It was created by an 

Executive Branch memo. NIST needs to address this issue, as their Authority section 

provides no actual statutory authority. 

Whether the government has a right to demand and publish these copyrighted articles 

may also be an issue, in view of the potential for damaging the copyright holders. This 

brings the copyright issue to the fore and NIST needs to address this issue. 

Just to elaborate, I have been talking to the Energy Department’s Office of Science about 

this issue. They claim that their right to collect and publish journal articles is based on the 
acquisition regulation 2 CFR 200.315(b). 

http://www.gpo.gov/fdsys/pkg/CFR-2014-title2-vol1/xml/CFR-2014-title2-vol1-sec200-

315.xml 

Here is the text of that regulation: 

“(b) The non-Federal entity may copyright any work that is subject to copyright and was 

developed, or for which ownership was acquired, under a Federal award. The Federal 

awarding agency reserves a royalty-free, nonexclusive and irrevocable right to reproduce, 

publish, or otherwise use the work for Federal purposes, and to authorize others to do so.” 

As I read this the journal article (the work) has to have been written (developed) under 

the award. The award contracts all have specific end dates. Journal articles are often 

written after the contract end date, because one does not write up the results until the 

research is over. Such articles are not developed under the award. 

As I read this the government has no right to these post-contract articles, but NIST is 

going to collect and publish them under the US Public Access program. I suppose one 

can argue that the research was part of the development, but then what is being 

copyrighted is the ideas not the text and that is incorrect. The copyright only begins with 

the writing. 

The basic point is that the NIST Public Access program is a separate information 

collection. Submitting published articles to a funding agency is clearly an information 

collection, because the agency gets the information. But so is one researcher sharing data 

with another, when that activity is mandated by agency rule, as it is under the Public 

Access program. So the data sharing requirements will also have to be cleared through
 

OIRA. 

Every proposal must include what is called a data management plan or DMP. The DMP 

says how the researcher will make available and share their data, which then becomes a 

contract commitment if they are awarded a grant. 

It is true that preparing a DMP is part of the burden of a proposal, but there is also the 

potentially much larger burden of preparing and sharing data. It can be very laborious to 

prepare data for others to use. It can require a lot of refining, formatting, documentation 

and explanation. 

The point is that NIST should be estimating how much sharing will go on and what the
 

burden is. It appears that this is not being done. More broadly, all of the agency Public 

Access programs that use the data management plan requirement should be clearing the 

burden of data sharing with OIRA. 

How to estimate (and minimize) the burden of mandated data sharing is a research 

problem, one that NIST should take seriously. There are really just two questions to be 

answered in estimating burden. First, how often will required sharing happen? Second, 

what will the average burden likely be? 
Note that the DMP data sharing requirement does not end when the research contract 

ends, far from it. This is also true of the requirement to submit published articles, but data 

requests can come long after publication. Article usage data suggests that data requests 

may well come many years after publication. So the estimate of how many mandated data 

sharings will occur needs to take into account this potential longevity. 

As to the average burden per request, here the problem is the extreme vagueness of the 

requirement. What is emerging as the standard requirement language is something to the 

effect of "whatever the community normally requires to validate the research" which 

NIST specifically cites. 

As regulatory language this is hopelessly vague, so hopefully standard practices will 

develop. It does seem to be leading in some cases to discipline specific agency research 

programs developing their own specific data sharing rules. 

The specialization of data sharing rules may well be happening at other agencies as well, 

or develop in the future as the Public Access program matures. It may make sense 

scientifically but it also makes burden estimation more complex. It may also make the 

goal of minimizing burden more difficult to pursue, given the multiplicity of discipline 

specific requirements. 

A related concern is that this vague DMP requirement can be taken to mean that the data 

to be shared is whatever the requestor wants. The danger here is that imposing burden via 

endless requests can be used as a political weapon. 

We already see this in some cases with Freedom of Information Act (FOIA) requests. But 

while FOIA puts the burden on the agency, Public Access data requests put the burden on 

the public, namely the researcher. Data sharing rules may be needed to limit this sort of 

abuse. 

Note too that the fact that a researcher deposits their data in a repository does not mean 

that their burden ends. People requesting data from the repository may well then ask the 

researchers for explanations, more documentation, or even more data. The vagueness of 

the DMP requirements may contribute to this. 

It seems likely that the burden of Public Access data sharing mandates may be large, even 

very large. The agencies and OIRA need to take this potential burden seriously and take 

steps to minimize it. The first step is to do proper burden estimating. NIST can take the 

lead in this important research. 

Respectfully submitted by 

David Wojick, Ph.D. 

http://insidepublicaccess.com 

davidwojick@insidepublicaccess 




 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

August 21, 2015 

Katherine Sharpless 

National Institute of Standards and Technology (NIST) 

100 Bureau Drive Stop 4701 

Gaithersburg, MD 20899-4701 

RE: National Institute of Standards and Technology Plan for Providing Public Access to the Results of 

Federally Funded Research 

I am writing on behalf of John Wiley & Sons, the leading American publisher of scientific research. Wiley 

was founded in 1807 and has 2,600 employees across the country. We publish over 1,600 journals 

across all major scholarly disciplines and partner with over 1,200 non-profit professional and scholarly 

societies which represent nearly 15 million researches around the world. 

Wiley is committed to working with funders, researchers and other stakeholders to expand access to 

high-quality, peer-reviewed articles. Together we can develop sustainable models that expand access 

and strengthen the system of scholarly communication, recognizing the investments publishers make in 

managing peer-review, publishing and disseminating articles, and preserving research for the future. 

We appreciate the opportunity to provide comments on the NIST Plan for Providing Public Access to the 

Results of Federally Funded Research in response to the Federal Register notice on July 7, 2015. 

What are the best practices (from academia, industry, and other stakeholder communities) in 

managing public access of data? 

Wiley is committed to working with its authors and agencies to enable greater access to digital data. In 

June 2015 Wiley announced a groundbreaking partnership with the data repository organization 

Figshare to integrate data sharing within existing journal workflows and article publication. The new 

data sharing service is being piloted with a number of titles and rolled out over the following few 

months along with new data citation and data sharing policies. This will ensure that more data is 

accessible, sharable and reproducible under a creative commons license from articles on Wiley Online 

Library without any cost to the author or readers. 

We look forward to working with NIST as it seeks to promote data sharing over the coming years. In 

particular, we appreciate that NIST will be “taking into account the efforts of public- and private-sector 

entities” as it outlines repository options for scientific data in digital formats. As these efforts advance 

we encourage NIST to consult with stakeholders, align with ongoing activities in the research community 

and leverage existing infrastructure instead of creating duplicative federal repositories. 

What are the biggest challenges to implementing a public access policy, and how can these challenges 

be addressed? 

111 River Street, Hoboken, NJ 07030-5774, U.S.
 

T +1 201 748 6000
 

F +1 201 748 6088
 

www.wiley.com
 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Embargoes and Petitions 

One of the key challenges in developing a sustainable public access policy will be to ensure that it 

provides sufficient time for publishers to recoup their significant investments in the peer-review and 

publishing processes. Indeed, the long-term success of “green” public access policies depends on a 

vibrant subscription base to support private sector investments. 

To this end, we are concerned that the initial 12-month embargo chosen by NIST for all journal articles 

covered by the policy could be too short for many publishers and societies to recoup their investments 

in the peer-review and publishing processes and continue to produce the highest-quality scientific 

research literature. Disciplines vary significantly in their publishing cultures and usage patterns, and 

what may be appropriate for certain rapidly-moving fields of research may be unsustainable for others. 

A variable approach to embargoes is essential to accommodate this diversity and ensure sustainability. 

Such an approach has been taken by other funders around the world including policymakers in the UK, 

who set an initial 12-month embargo for STEM disciplines and a 24-month embargo for the arts, 

humanities and social sciences. 

We appreciate that NIST has, in accordance with the OSTP memorandum, included a petitions 

mechanism to adjust this embargo period. This is a critical safeguard to ensure the NIST plan does not 

harm the system of scholarly communication and can respond flexibly to accommodate the diversity of 

research and stakeholders. However, we are concerned that the criteria for adjusting the embargo 

period – “providing evidence that the current embargo period does not provide a public benefit and is 

inconsistent with the objectives articulated in the OSTP memo” – diverges from the primary purpose of 

such a period, which is to preserve the sustainability and quality of scholarly publications. As a potential 

alternative approach, we would highlight the criteria outlined in the National Science Foundation public 

access plan, which calls for petitions to present evidence that a change in the embargo period “will more 

effectively promote the quality and sustainability of scholarly publications while meeting the objectives 

of public access.” In addition, we would recommend that NIST allow any stakeholder to make such a 

petition, not just “NIST’s customers.” 

Such a measured, reflective, and stakeholder-driven approach will be essential to develop a sustainable 

public access policy and adjust as needed to address challenges in the course of implementation. 

Implementation and Repositories 

NIST has indicated that it will use the NIH's PubMed Central (PMC) repository to implement its public 

access plan. As PMC can act in direct competition with other stakeholders and unnecessarily duplicate 

functions already being performed by other actors, we would encourage NIST to explore other more 

cost-effective options for implementing its public access plan that leverage journals’ existing 

infrastructure. Wiley and a coalition of other publishers and societies, who collectively publish the vast 

majority of articles reporting on federally-funded research, are committed to helping agencies 

implement their public access plans through the Clearinghouse for the Open Research of the United 

States (CHORUS). Distributed systems like CHORUS can save taxpayer dollars, avoid unnecessary 

duplication and reduce compliance burdens. We look forward to future discussions on how CHORUS 

can be leveraged to support implementation of agency public access plans. 

If NIST does use PMC, we would encourage it to consider ways in which the system could be improved 

upon: 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

	 If PMC is used, rely on relevant industry standards rather than creating proprietary identifiers and 

link to authoritative article versions if the version of record is not available within PMC. The current 

approach of using PMC IDs instead of industry-standard DOIs encourages traffic to remain within 

PMC. 

	 If PMC is used, clearly display copyright information where relevant and provide usage statistics for 

manuscripts deposited in PMC by publishers on behalf of authors. The National Library of Medicine 

(NLM) does not communicate effectively to stakeholders and partners critical information, including 

usage of accepted manuscripts posted on PMC and clear guidance on article copyrights, both 

ownership and re-use terms. 

	 If PMC is used, do not expand services in ways that dilute its mission and effectiveness, such as the 

creation of derivative products and services without consultation with publishers and other 

stakeholders and establishing overseas mirror repositories when other access and repository 

options already exist. 

	 If PMC is used, consider developing additional services to improve public access to its databases, 

such as API access to Medline/PubMed. Enabling programmatic access to database content will 

increase dissemination of publically-funded material and will encourage the development of 

innovative products and services based upon NLM data. 

Intellectual Property Protection 

Strong intellectual property protections underpin the information economy and support the ability of 

rightsholders to continue to publish high-quality, peer-reviewed scientific literature. As such, we 

appreciate NIST’s commitment to “recognize/.intellectual property rights, avoiding significant negative 

impact on intellectual property rights, innovation, and U.S. competitiveness.” In particular we 

appreciate NIST’s clear acknowledgment that articles made freely accessible are still protected by 

copyright laws. To ensure the sustainability of scholarly publishing and prevent the undermining of 

copyright, it will be important to continue to allow rightsholders to set appropriate license terms, ensure 

such license terms are clearly communicated, and take steps to prevent bulk downloads and 

infringement in cases where licenses do not allow such use. 

How can NIST improve its plan to provide greater public access to NIST data? 

Wiley looks forward to partnering with NIST as it implements its initial public access plan. Experience 

from around the world suggests that the adoption of any such plan is never a simple endeavor, and that 

both expected and unexpected challenges will emerge over the course of this process. To respond 

nimbly to these challenges and make improvements and adjustments to its plan, it will be important 

that NIST continue to adopt a flexible approach to public access and engage openly and regularly with 

the scholarly community. 

Thank you for your consideration of these comments regarding the NIST public access plan. We look 

forward to collaborating with NIST as it refines and implements its public access plan and remain 

committed to strengthening the system of scholarly communication in support of researchers and 

funders in the United States and around the world. 
 

 

Sincerely, 

Andrew A Tein 

Vice President, Global Government Affairs
 

Wiley
 

Office: +1 201 748 7751
 

e-mail: antein@wiley.com
 

 

 

 

 

 

 

 

August 21, 2015 

Dr. Willie E. May 

Director 

National Institute of Standards and Technology 

100 Bureau Drive Stop 4701, 

Gaithersburg, MD 

20899-4701 

Re: Comments on the National Institute of Standards and Technology’s Public Access Plan 

Dear Dr May: 

RELX Group (formerly Reed Elsevier) welcomes the opportunity to comment on the National Institute 

of Standards and Technology’s (NIST) Public Access Plan. RELX Group is one of the world’s largest 

providers of professional information solutions, with more than 13,000 U.S. employees and more than 

28,000 employees worldwide. 

Elsevier, a RELX Group company, is a world-leading provider of scientific, technical and medical 

information products and services. We serve 30 million scientists, students and health and 

information professionals worldwide. We help advance science and health by providing world-class 

information and innovative tools that help customers make critical decisions, enhance productivity, 

and improve outcomes 

Elsevier publishes more than 2,000 journals, including The Lancet and Cell, and a growing portfolio of 

open access titles. We also publish close to 20,000 book titles, including major reference works such as 

Gray’s Anatomy and titles from osby and Saunders. lsevier’s online solutions include Scienceirect, 

Scopus, Reaxys, linicaley and osby’s ursing Suite, which enhance the productivity of science and 

health professionals by providing quick and efficient access to published research, and the SciVal suite 

and ai’s Pinpoint Review, which help research and health care institutions deliver better 

outcomes more cost-effectively. 

Elsevier is an integral partner to the scholarly research community in the United States. Our US 

Elsevier workforce includes about 3,000 people spread across 15 offices in 10 states. We publish 

about 350,000 journal articles each year, many of which are authored by US researchers and of which 

approximately 35,000 acknowledge support from the US government. We are the single largest 

publisher of journal articles that report on research funded by the Department of Energy. In addition, 

we publish over 250 journals in partnership with US scholarly societies, such as The Combustion 

Institute, Hydrogen Energy Publications and The International Energy Initiative. Elsevier publishes 

18% of articles which report on research funded by the U.S. Department of Health & Human Services 

(HHS) each year. 

RELX Group 1150 18th St. NW, Suite 600 Telephone: 202.857.9112 www.relxgroup.com 

Washington, DC 20036 Fax: 202.857.8294 
 

 

 

 

 

 

 

 

 

 

 

 

 

RELX Group and Elsevier support the principle that the public should be able to access publicly-funded 

research and we demonstrate this support through a broad array of sustainable public access options 

in our 2000+ journals, and through our universal access initiatives. We provide free or very low-cost 

access to hundreds of medical journals via patient information services such as Patient Inform and 

patientACCESS. In addition, through a program called Research4Life, we provide free or low-cost 

access to Elsevier journals and books to more than 6,000 institutions in more than 100 developing 

countries. 

lsevier also participates in the ’s mergency !ccess nitiative. This important initiative provides 

temporary free access to full text articles from major biomedicine titles to healthcare professionals, 

librarians, and the public affected by emergencies and disasters such as the ebola outbreak and the 

aftermath of the Nepal earthquake. 

We also welcome the collaborative public-private partnership with federal agencies in the 

Clearinghouse for Open Research of the United States (CHORUS), a multi-publisher portal to provide 

access to journal articles reporting on government-funded research. 

The principle of ensuring access to published research by means of public access is one that we 

entirely support. What is at issue is not the principle of public access, but the form that it should take 

in order to make it work in a sustainable fashion. 

RELX Group is pleased to provide the following comments on the NIST Public Access Plan. 

1.	 Sustainable Public Access Policies with Variable Embargoes Are Essential to Scientific 

Research and High-Quality Scholarly Communication 

We are very concerned that NIST’s public access plan includes a “one-size-fits-all” twelve-month postpublication embargo for journal articles in all disciplines. Embargo periods are necessary because the 

subscription business model is relied on to support the making available of versions of articles outside 

of a subscription. Selling subscriptions enables publishers to recoup a return on investment in our journals 

and to continue to reinvest to support the scholarly communications system. When versions of articles are 

made available too early, this reduces the incentive for our customers to subscribe to journals and therefore 

makes it difficult for us to secure a return on our investment. Embargo periods therefore need to be set 

according to the usage patterns we see in journals, specifically not too early so that it is worth a customer 

continuing to subscribe. 

Setting embargos at the optimum level requires careful consideration and the aim is to strike a balance 

between making a version of an article publically available but not too early so as to undermine the 

continuing viability of a journal. Studies have shown that when embargo periods are set too low, the 

incentive to buy a journal decreases. Studies have similarly shown that usage patterns, which help determine 

the time at which it becomes viable for a version of an article to become publically available, vary between 

disciplines, but also within different disciplines. This was a key conclusion of a comprehensive usage study 

of over 2800 journals across 10 disciplines, conducted by Dr. Philip M. Davis.1 Looking across all 

1 Journal Usage Half-Life By Philip M. Davis, Ph.D. Phil Davis Consulting; 

http://www.publishers.org/_attachments/docs/journalusagehalflife.pdf 

RELX Group 1150 18th St. NW, Suite 600 Telephone: 202.857.9112 www.relxgroup.com 

Washington, DC 20036 Fax: 202.857.8294 
 

 

 

 

 

 

 

 

 	 

 

disciplines, Davis found that the majority of journals had a usage half-life of 3 years or more, and that 

only 3 percent of journals had half-lives of less than 12 months. Usage half-life is defined as the time 

taken for articles to reach half of their total number of downloads. 

KEY FINDINGS FROM JOURNAL USAGE HALF-LIFE BY DR. PHILIP M. DAVIS 

These results are consistent with previous studies on journal usage in different disciplines. For 

example, in their 2012 submission to the White House Office of Science and Technology Policy, the 

American Psychological Association (APA) noted that less than 16% of the usage of APA psychology 

journals occurs in the first year. Similarly, an analysis in 2009 by the American Mathematical Society 

found that only 10% of the citations in the mathematics literature were to articles published in the 

previous three years combined. Further, a study of journal publishing in the humanities and social 

sciences concludes that, given the comparatively long life of articles in those fields, the imposition of 

embargo periods that are being adopted for biomedical journals could threaten the sustainability of 

humanities and social science journals.2 

These studies clearly demonstrate that a 12 month embargo period would be unsustainable in some 

disciplines and that variation in embargo periods are needed to take account of differences within and 

between disciplines. With this evidence in mind, we urge NIST to reconsider its approach and allow for 

variable embargo periods on a discipline-specific basis, using the evidence provided alongside other 

2 “The uture of Scholarly 
ournals Publishing !mong Social Science and umanities !ssociations,” Report on a study funded by a 

Planning Grant from the Andrew W. Mellon Foundation (February 2009), available at: 

http://www.nhalliance.org/bm~doc/hssreport.pdf 

RELX Group 1150 18th St. NW, Suite 600 Telephone: 202.857.9112 www.relxgroup.com 

Washington, DC 20036 Fax: 202.857.8294 
 

 

 

 

 

 

 

 

 

 

 

 

relevant data. This will ensure the continuation of robust and dynamic, high-quality scientific and 

scholarly communication system. 

Additionally, we are encouraged that NIST has included a process to allow publishers to petition for an 

exception to the established embargo where they can provide supporting evidence demonstrating that 

the established embargo is not viable. This is consistent with the February 22, 2013 OSTP memo on 

Increasing Access to the Results of Federal Funded Scientific Research that directs agencies to provide 

a mechanism for stakeholders to petition for changing an established embargo. 

2.	 The NIST Plan Will Incur Unnecessary Costs by Duplicating Efforts Provided by the Privatesector, including the Clearing House for the Open Research of the United States (CHORUS). 

RELX Group and Elsevier welcome opportunities to enhance delivery of public access through 

collaboration and strongly urge NIST to consider partnership approaches to public access which avoid 

duplication of effort and minimize both costs and administration for researchers and their institutions. 

One of the significant collaboration opportunities in facilitating public access is via the CHORUS service 

(http://www.chorusaccess.org/), which leverages existing infrastructure, tools, and services that 

support sharing, access, discoverability, reporting, and preservation. 

CHORUS also reduces the compliance burden on authors, and in turn increases compliance with public 

access goals. 

We are concerned that NIST has not given consideration to how this new approach presents 

opportunities for cost-savings within federal agency budgets and for institutions receiving federal 

agency research support. A planned collaboration with CHORUS would achieve this aim, and also the 

OSTP objective of leveraging existing infrastructure and fostering public-private partnerships with 

scientific journals relevant to the agency’s research. One example of successful collaboration to date 

includes the partnership between the U.S. Department of Energy (DoE) and CHORUS, which is 

currently leveraging open standards, distributed networks, and established infrastructure to enable 

agency indexing of articles and advance access to publicly available research articles that acknowledge 

DOE funding (http://www.chorusaccess.org/us-department-of-energy-and-chorus-sign-participationagreement-to-advance-public-access/). We would urge NIST, either independently or through its 

partnership with NIH and PMC, to explore the opportunities that the CHORUS service can provide for 

facilitating public access. 

3.	 The NIST Plan Should Promote Public-Private Partnerships 

We encourage NIST to look for additional opportunities to leverage public-private partnerships. We 

believe collaboration between our organizations to achieve shared goals could be significantly 

improved in the following ways: 

i.	 Elsevier urges NIST to share COUNTER-compliant distributed usage statistics for 

manuscripts which report on NIST funded research in PMC so that publishers can continue 

to report on impact and usage to authors and to their institutions that subscribe to these 

publications and pay their publication costs. 

RELX Group 1150 18th St. NW, Suite 600 Telephone: 202.857.9112 www.relxgroup.com 

Washington, DC 20036 Fax: 202.857.8294 
 

 

 

 

 

 

 

 

 

 

 

 

 

ii. It is also critical that NIH cease reformatting and enhancing manuscripts to make them 

appear more like, and substitute for, the final version-of-record of articles. This duplicative 

process only adds cost to PMC and its partnering agencies like NIST. 

iii. Finally, it is essential that PMC ensure readers are presented with the best version of the 

article available, which means that entitled users are transparently linked to the final 

published version of articles reporting on NIST funded research. 

4. NIST’s Approach to a Data Repository Solution Should Allow for Optimal Data Use 

RELX Group and Elsevier support the principle that raw research data should be freely available, and 

have both a research data policy and a number of tools and services to support researchers who wish 

to share and access data. We therefore support NIST’s vision for data sharing; however, we feel 

strongly that researchers should be free to deposit their data in repositories best suited for their 

research and where they feel their data can be most discoverable and accessible, best preserved, and 

analyzed using discipline-specific tools as often provided by discipline-specific repositories. 

Specifying where data should be deposited could limit the opportunity for flexible approaches to data 

submission, access, understanding, and use. 

We would also welcome reassurance from NIST that federal research data repositories will be looking 

to collaborate with publishers and other stakeholders to help make data discoverable and 

interpretable in the context of associated publications. Some examples of lsevier’s collaborations 

with data repositories to date can be found here. 

We appreciate the opportunity to provide you with our comments on the NIST Public Access Plan. We 

would welcome the opportunity to engage with NIST to discuss ways to determine appropriate and 

consistent implementation of administrative intervals for specific scientific fields, and how to 

maximize the effectiveness of CHORUS to ensure the full benefits, reduced costs and decreased 

administrative burdens to NIST. 

Sincerely, 

Steven M. Manzo 

Vice President 

Global Government Affairs Americas 

RELX Group 

RELX Group 1150 18th St. NW, Suite 600 Telephone: 202.857.9112 www.relxgroup.com 

Washington, DC 20036 Fax: 202.857.8294 
 

 

 

 

 

 

 

 

Appendix D. NIST Data Taxonomy and Actions/Consequences for Data Categories 

I. PURPOSE 

The purpose of this taxonomy is to define a collection of terms and concepts that describe classes 

and categories scientific data arising from unclassified research and programs funded wholly or 

in part by NIST1

, as well as policy requirements, actions, and consequences that might apply to 

those categories as a result of requirements expressed in the Office of Science and Technology 

Policy (OSTP) Open Data Memorandum, OMB Memorandum M-13-13, and Executive Order 

13642. (See NIST Policy P 105.01.) In the context of these requirements, research data is 

defined as “the recorded factual material commonly accepted in the scientific community as 

necessary to validate research findings, but not any of the following: preliminary analyses, drafts 

of scientific papers, plans for future research, peer reviews, or communications with 

colleagues.”2 

Although the categories in the NIST data taxonomy are arranged in a pyramid, they are not 

strictly hierarchical. Categories range from working data to standard reference data (SRD) (see 

Figure 1). The goal of this document is to achieve a shared understanding of the data 

management space at NIST, not to make policy choices or to define requirements or recommend 

procedures. This vocabulary is intended to enable discussions among NIST management and 

technical staff to support NIST’s data management Policy and Order. 

1 A non-NIST organization that publishes scholarly and technical material, including data, through activities funded 

wholly or in part by NIST through a grant, cooperative agreement, contract, or other agreement, must manage public 

access to published scholarly and technical material, including data, as agreed to by NIST and that organization in 

the terms and conditions of the grant, cooperative agreement, contract, or other agreement between NIST and the 

non-NIST organization. 

2 For purposes of this policy, NIST is adopting the definition of “research data” provided in 2 C.F.R. §200.315 

(e)(3). http://www.gpo.gov/fdsys/pkg/CFR-2014-title2-vol1/pdf/CFR-2014-title2-vol1-sec200-315.pdf 

61
 
SRD

Resource Data

Derived Data

Reference Data

Working Data

Published Results

Publishable Results

Figure 1. Data pyramid describing the categories of NIST data, 

ranging from “Working Data” to “Standard Reference Data (SRD)” 

II. DATA CATEGORIES 

The main categories envisioned for NIST data have been arranged in the form of a “data 

pyramid,” (Figure 1) recognizing that in general the volume of data decreases as you move from 

the bottom of the pyramid toward the top. This is an oversimplification, and several competing 

dimensions for characterizing and distinguishing data classes have been combined into this one 

view for reasons of simplicity and compactness. However, this simplified diagram (Figure 1) 

provides a useful breakdown of data classes for the narrow context of discussing data 

management plans and NIST data curation and dissemination policies. 

Several classes of data are described in the pyramid, with the following definitions: 

1. Working Data 

The digital equivalent of entering data in a laboratory notebook. Working data may be raw 

observational data that is acquired directly from an instrument or a measurement system, or 

digital values acquired or generated during experiments or simulations. In some cases the 

researcher responsible for generating the working data may determine that this data has 

immediate value and is worth preserving, or the researcher may expect that the data will have 

value after it has been manipulated or further evaluated, and the data has the potential to develop 

into a publication or will be used to draw conclusions. In other cases working data may be 

recognized as not appropriate for broader use in its present form. It may have value to the data 

62
 
 

 

 

 

 

 

 

 

 

 

producers and their collaborators, but it should be recognized that the data could be easily 

misinterpreted by people not closely involved in its production because some metadata and 

important facts about its status or acquisition are not readily available beyond the immediate 

research team (e.g., adequate metadata for re-purposing is not attached to the data itself, or 

expending resources to codify needed metadata is not justified, etc.). 

2. Derived Data 

Underpins the conclusions provided in a publication or report. Derived data comes from 

working data that has been manipulated, analyzed, processed, or evaluated in some way. The 

data must have passed some minimal (perhaps ad hoc) evaluation and be considered by the 

responsible researcher (typically the data producer) to be ready for the next steps in the workflow 

or project/product development effort. 

3. Publishable Results 

All final or summary results that comply with relevant NIST policies (e.g., SI units, uncertainty 

statements), that have been reviewed internally and approved by an appropriate NIST authority, 

and that could be published either in a scientific publication or as a standalone data product. 

4. Published Results 

Results that are publishable and that are contained in a document that has been reviewed and 

approved for publication by the necessary NIST organizational authorities, submitted to its 

intended publisher, and made public. 

5. Resource Data 

Data used to underpin, support, or defend decisions, actions, or positions of NIST. 

6. Reference Data (RD)3 

Data similar in many characteristics to SRD, sharing features of organization, documentation, 

and evaluation with SRD. The primary difference between RD and SRD is that reference data is 

not distributed under the authority of the Standard Reference Data Act 

7. Standard Reference Data 

Data that has been collected from documented sources, organized, critically evaluated using a 

procedure that is documented, and distributed, as described in the Standard Reference Data Act. 

The Standard Reference Data Act defines standard reference data as “quantitative information, 

related to a measureable physical or chemical property of a substance or system of substances of 

known composition and structure, which is critically evaluated as to its reliability under [the 

3 It should be noted that the definitions for standard reference data and reference data in this document are similar 

but not identical to those in the International Vocabulary of Metrology (VIM). There are two key differences: the 

definition of standard reference data is adopted from the SRD Act, and both definitions are broader than those in the 

VIM since the VIM only refers to measured data. The VIM defines reference data as being “related to a property of 

a phenomenon, body, or substance, or to a system of components of known composition or structure, obtained from 

an identified source, critically evaluated, and verified for accuracy.” The scope of reference data as used in this 

document expands beyond physical and chemical properties. 

63
 
 

 

 

 

 

provisions of the Standard Reference Data Act].”4 Standard Reference Databases are 

copyrightable, and NIST may secure copyright in them. 

III. ACTION/CONSEQUENCE LEVELS 

While the section on Data Categories is intended to define a variety of classes or grades of data 

that exist at NIST, this section defines corresponding requirements or consequences that should 

be considered when formulating data policy. As explained earlier, the purpose of this taxonomy 

document is not to impose these consequences or required actions on the categories, but merely 

to provide a vocabulary that simplifies discussion of assigning such requirements to various 

categories. Expressions of NIST data policy (e.g., NIST Directives, Guidance Memoranda, OU 

policies, etc.) should contain statements that map these consequence levels onto specific 

categories. Incorporating this taxonomy document as a reference into policy documents that 

delineate such mappings will simplify those statements of policy and reduce their ambiguity. 

Figure 2. Mapping preservation consequence levels onto the NIST data categories. 

SRD

Resource Data

Derived Data

Reference Data

Working Data

Published Results

Publishable Results

Level

3

3

3

2

2

1

1

1. Preservation Consequence Levels Defined 

Consistent with NIST Administrative Manual Subchapter 2.06 Records Management,5 the NIST 

Records Retention Schedule6 for Scientific and Technological Records,7 and the General 

4 15 U.S.C. § 290a, Standard Reference Data Act, http://0-www.gpo.gov.librus.hccs.edu/fdsys/pkg/USCODE-1995-

title15/pdf/USCODE-1995-title15-chap7A.pdf. 5 http://inet.nist.gov/mando/directives/206.cfm 

6 http://inet.nist.gov/mando/nist-records-schedule.cfm 

7 http://inet.nist.gov/mando/services/upload/Items-25-32-Scientific-and-Technological-Records.pdf 

64
 
 

 

 

 

 

 

Records Schedule8 for Input Records, Output Records, and Electronic Copies,9 the following 

preservation consequence levels correspond to the Data Categories in the data pyramid (See 

Figure 2): 

1.	 No preservation requirements, 

2.	 Individual user responsible for preservation of data, 

3.	 Data must be backed up using a tested/automated process (i.e., proof that restoration is 

possible).10 

SRD

Resource Data

Derived Data

Reference Data

Working Data

Published Results

Publishable Results

Level

3

3

2

3

2

1

1

Figure 3. Mapping review consequence levels onto the NIST data categories. 

2.	 Review Consequence Levels Defined 

The following review consequence levels correspond to the Data Categories in the data pyramid 

(See Figure 3.): 

1.	 No additional review requirements, 

2.	 Technical aspects of the data must be reviewed and approved within the OU following 

OU policies. 

3.	 Review by other appropriate NIST authorities (e.g., WERB, ODI) is required. 

8 http://www.archives.gov/records-mgmt/grs.html 

9 http://www.archives.gov/records-mgmt/grs/grs04-3.pdf 

10 The data are backed up periodically, but the backup frequency is left unspecified and commercial backup 

technologies such as Tivoli Storage Manager are employed, OR the data are backed up at the level of OISM Central 

File Services Tier 2, OR the data are backed up at the level of OISM Central File Services Tier 1. 

65
 
 

 

 

 

 

 

 

 

3.	 Discoverability Consequence Levels Defined 

The following discoverability consequence levels correspond to the Data Categories in the data 

pyramid (See Figure 4.): 

1.	 No discoverability requirements, 

2.	 Metadata values must be entered into the NIST Enterprise Data Inventory (i.e., the 

NISTXM11 metadata) and a Persistent Identifier (PID) minted for the dataset, 

3.	 Metadata values in the NIST Enterprise Data Inventory are made publicly accessible. 

SRD

Resource Data

Derived Data

Reference Data

Working Data

Published Results

Publishable Results

Level

3

3

3

3

2

1

1

Figure 4. Mapping discoverability consequence levels onto the NIST data categories. 

In addition to the guidance above, additional considerations should also be applied by each OU 

and Office to prioritize the availability of datasets based on factors including stakeholder need, 

the reasonableness of effort required to make the data available, and other relevant factors. 

IV. RELEVANCE OF THE NIST IT SYSTEM SECURITY PLANS 

There is a very close relationship between NIST data and the information technology (IT) 

systems used to store, utilize, and exchange that data. Further, extensive NIST policy governing 

11 The NIST Extensible Metadata Schema is a definition of the minimum metadata values to be associated with 

NIST datasets. Formerly known as the “NIST Common Core,” the NISTXM is a very minor extension of the OMBrequired Common Core metadata fields. 

66
 
 

 

 

 

 

 

 

 

 

 

 

IT systems has already been defined, and NIST has numerous special publications and Federal 

Information Processing Standards (FIPS) for the benefit of the nation, pursuant to the Federal 

Information Security Management Act (FISMA) of 2002 and other legislation relative to 

information technology. 

Federal law12 defines the three components of a widely accepted model for discussing 

information security, including both IT security and information assurance: 

a.	 Integrity: guarding against improper information modification or destruction, including 

ensuring information nonrepudiation and authenticity; 

b.	 Confidentiality: preserving authorized restrictions on access and disclosure, including 

means for protecting personal privacy and proprietary information; and 

c.	 Availability: ensuring timely and reliable access to and use of information. 

This security model applies to information, information systems, and related resources (including 

user information such as research results), and therefore is much broader than just NIST data. 

However, these concepts are relevant to data generated by federally funded research. 

FIPS Publication 19913 defines three levels of potential impact on organizations and individuals 

should there be a breach of security (i.e., in this context a loss of confidentiality, integrity, or 

availability of the data). The potential impact can be LOW, MODERATE, or HIGH if the loss of 

confidentiality, integrity, or availability could be expected to have a limited adverse effect, a serious 

adverse effect, or a severe/catastrophic effect on organizational operations, organizational assets, or 

individuals. These impact levels are clarified and amplified in FIPS 199. When federally funded 

research is intended for publication, its INTEGRITY and CONFIDENTIALITY impacts are 

LOW since the unauthorized modification or disclosure of the data would have a limited adverse 

effect on NIST operations, assets, and individuals. However, if the federally funded research 

contains business or personally identifiable information, proprietary information, or other 

sensitive information prior to publication, CONFIDENTIALITY is deemed MODERATE and 

therefore requires more stringent security controls. Business or personally identifiable 

information, proprietary information, or other sensitive information must never be published or 

otherwise made public. The data categorization and security controls must be documented 

within the respective NIST OU system security plan. 

Preservation of records may be accomplished through various means. (NIST staff can see ‘How 

do I backup my data’ for more information.) 

12 See 44 U.S.C § 3542 – Definitions.
 13 FIPS 199, “Standards for Security Categorization of Federal Information and Information Systems,” NIST,
	

February 2004, available at http://csrc.nist.gov/publications/fips/fips199/FIPS-PUB-199-final.pdf.
 

67