U.S. DEPARTMENT OF LABOR 

Office of the Chief Information Officer 

Guidelines for Ensuring and 

Maximizing the Quality, Objectivity, 

Utility, and Integrity of Information 

Disseminated by the Department of 

Labor 

October 1, 2002 
U.S. Department of Labor Information Quality Guidelines 

TABLE OF CONTENTS 

INTRODUCTION......................................................................................................................... 1

Background..................................................................................................................................1 

Purpose.........................................................................................................................................1 

SCOPE AND APPLICABILITY................................................................................................. 2

QUALITY MANAGEMENT PRINCIPLES ............................................................................. 3

1996 SAFE DRINKING WATER ACT AMENDMENTS FOR SAFETY AND HEALTH 

RISK ANALYSIS PRINCIPLES ................................................................................................ 4

BLS GUIDELINES FOR INFORMING USERS OF INFORMATION QUALITY AND 

METHODOLOGY ....................................................................................................................... 4

INFORMATION QUALITY RESPONSIBILITIES ................................................................ 5

Chief Information Officer (CIO) .................................................................................................5 

Agency Heads..............................................................................................................................5 

INFORMATION CATEGORIES............................................................................................... 5

Influential.....................................................................................................................................5 

Non-influential.............................................................................................................................7 

PRIVACY AND SECURITY INFORMATION AND PUBLIC DISCLOSURE ................... 7

INFORMATION QUALITY ASSURANCE PROCESS .......................................................... 7

INFORMATION COMPLAINT AND APPEALS PROCESS................................................. 8

Overview of Information Complaint and Appeal Process...........................................................9 

TRACKING AND REPORTING INFORMATION COMPLAINTS AND APPEALS...... 10

APPENDIX I: INFORMATION QUALITY GLOSSARY..................................................... 12

APPENDIX II: ADAPTING THE PRINCIPLES UNDER THE 1996 SAFE DRINKING 

WATER ACT AMENDMENTS FOR SAFETY AND HEALTH RISK ANALYSES......... 15

APPENDIX III: BLS GUIDELINES FOR INFORMING USERS OF INFORMATION 

QUALITY AND METHODOLOGY........................................................................................ 20

APPENDIX IV: SAMPLE INFORMATION QUALITY ASSURANCE TECHNIQUES 

AND METHODS ........................................................................................................................ 23
U.S. Department of Labor Information Quality Guidelines 

INTRODUCTION 

Background 

On December 21, 2000, Congress passed Section 515 of the Treasury and General Government 

Appropriations Act for Fiscal Year 20011 (the Act). OMB issued final guidance2 for 

implementing the Act, which required all Federal agencies to: 

• Issue information quality guidelines ensuring and maximizing the quality, objectivity, utility, 

and integrity of information, including statistical information, disseminated by the 

Department; 

• Establish administrative mechanisms allowing affected persons to seek and obtain correction 

of information maintained and disseminated by the Department that does not comply with the 

OMB guidelines; and 

• Report to the Director of OMB the number and nature of complaints received regarding 

compliance with the OMB guidelines, including how the complaints were resolved. 

The OMB guidance directed Federal agencies to publish draft information quality guidelines on 

their web sites by April 1, 2002. OMB further clarified on March 4, 2002, that the web site 

publishing date was extended to May 1, 2002. 

The Department of Labor (DOL) posted its draft guidelines to its web site on May 1, 2002. On 

May 22, 2002, DOL posted an appendix addressing the adaptation of the Safe Drinking Water 

Act to the development of risk assessments. DOL published notice of the guidelines in the 

Federal Register on May 1, 2002, and required that comments be submitted by May 31, 2002. 

DOL extended the comment period to June 30, 2002, in response to public requests and to 

OMB’s decision to extend the submission date for agencies to provide revised guidelines to 

OMB. 

These guidelines represent changes made to the draft DOL guidelines posted for public comment 

on May 1, 2002, and incorporate revisions based upon feedback from OMB, public comments, 

and internal Departmental review. 

Purpose 

The purpose of these guidelines is to establish Departmental guidance for implementing an 

Information Quality program at DOL. This Information Quality guidance is intended to enhance 

the quality of the information disseminated by DOL. 

 

1 Public Law 106-554, App. C. 

2 See Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Information

Disseminated by Federal Agencies; Notice; Republication; Office of Management and Budget, Federal Register Vol.

67, No. 36, pp. 452-460, February 22, 2002. 

 1 
U.S. Department of Labor Information Quality Guidelines 

SCOPE AND APPLICABILITY 

These guidelines are intended, within the context of laws administered and enforced by DOL, to 

meet the information quality objectives set forth in OMB’s guidelines. They are intended to 

improve the internal management of the Federal Government. They are not intended to impose 

any binding requirements or obligations on DOL or the public or to create any right or benefit, 

substantive or procedural, enforceable at law by a party against the United States, its agencies, 

officers, or any person. They are not intended to provide any right to judicial review. 

These guidelines reflect this Department's commitment to information quality as an important 

management objective that takes its place alongside other Departmental objectives, such as 

ensuring the success of agency missions, observing budget resource priorities and restraints, and 

providing information to the public. Agencies should strive to assure that these goals reinforce 

each other as much as is practicable. Where an agency believes that they conflict, it should, 

consistent with its legal responsibilities, attempt to reconcile them in a manner that the agency 

believes will best serve the public interest and help the agency meet its statutory or program

obligations. Program efficiency must be a critical goal as DOL agencies carry out their 

responsibilities under these guidelines. Thus, for example, it may not be in the public interest for 

agencies to devote significant resources to correcting information where the expenditure of such 

resources is not, in the agency's view, cost effective in light of the significance of the data and 

the agency's more pressing priorities and obligations. 

The DOL’s pre-dissemination reviews apply to information that DOL first disseminates on or 

after October 1, 2002. Other aspects of these guidelines, including the information correction 

process apply on or after October 1, 2002, with respect to information that DOL disseminates on 

or after October 1, 2002, regardless of when DOL first disseminated the information. 

Information means any communication or representation of knowledge such as facts or data, in 

any medium or form, including textual, numerical, graphic, cartographic, narrative, or 

audiovisual forms. Dissemination includes agency initiated or sponsored distribution of 

information to the public. It does not include agency citation to or discussion of information that 

was prepared by others and considered by the agency in the performance of its responsibilities, 

unless an agency disseminates it in a manner that reasonably suggests that the agency agrees 

with the information. Agency sponsored distribution of information covers instances where an 

agency has directed a third party to disseminate information or where the agency has the 

authority to review and approve the information before release. By contrast, if an agency funds 

research, but the researcher decides whether or not to disseminate the results, the agency has not 

“sponsored” the dissemination, and the information is not subject to these guidelines. In these 

instances, agencies should direct the researcher to include an appropriate disclaimer in the 

publication. Similarly, the guidelines would not cover publications of their research findings by 

Departmental employees or Federal grantees or contractors when published in the same manner 

as their academic colleagues. Again, the researchers should include an appropriate disclaimer 

noting that the views are theirs and not necessarily those of DOL. 

 2 
U.S. Department of Labor Information Quality Guidelines 

These guidelines do not apply to the following: 

• Information intended to be limited to distribution to government employees, or DOL 

contractors, or grantees; 

• Government information intended to be limited to intra- or inter-agency use or sharing of 

information, such as strategic plans, performance plans, program reports, operating plans, or 

budgets; 

• Responses to requests for Departmental records under the Freedom of Information Act, the 

Privacy Act, the Federal Advisory Committee Act, or other similar laws; 

• Correspondence or other communications with individuals or organizations; 

• Press releases (except where the press release itself is the primary source of the information); 

• Congressional testimony; 

• Archival records; 

• Public filings; 

• Dissemination of information through subpoenas or adjudicative processes, such as those 

recognized under the Administrative Procedure Act or established pursuant to regulation; 

provided, however, that information originally disseminated through such vehicles could 

subsequently become subject to these guidelines to the extent it is re-disseminated more 

broadly through other vehicles.

• Information clearly represented as opinion and not an official agency or Departmental 

representation; 

• Policy guidance recommendations or statements or summaries of agency policies, 

procedures, or programs; 

• Statements of legal policy or interpretation, including briefs filed with courts or 

administrative bodies; and 

• Final agency decisions, settlements in litigation and descriptions of these settlements, or 

determinations of legal force and effect, such as wage determinations. 

These guidelines apply to all agencies of DOL, except to the extent that agencies have adopted 

tailored agency-specific guidelines. 

For a glossary of numerous terms and their definitions used throughout the rest of this document, 

please consult Appendix I. The definitions are from the OMB guidance for the Act. 

QUALITY MANAGEMENT PRINCIPLES 

Every agency should establish information quality, as defined in OMB and DOL information 

quality guidelines, as a performance goal. Quality includes the “utility,” “objectivity,” and 

“integrity” of the information. The level of quality should be “appropriate to the nature and 

timeliness of the information to be disseminated” and will be affected by the nature of the 

underlying data. In considering utility, agencies should evaluate the usefulness of particular 

information to those expected to use it. The information also should be objective--“accurate, 

reliable, and unbiased,” and presented “in an accurate, clear, complete, and unbiased manner.” 

Agencies also should protect the integrity of information from unauthorized access or revision. 

 3 
U.S. Department of Labor Information Quality Guidelines 

These objectives and guidelines are to be interpreted consistent with DOL’s statutory 

obligations. Where agencies are disseminating information of a scientific, financial, or statistical 

nature, they should use sound statistical and research methods to develop and analyze the data. 

Depending on the type of information disseminated and consistent with statutory and 

confidentiality restrictions, agencies should identify the sources of the information and where 

appropriate, the supporting data, models, and error sources. 

Where agencies develop and disseminate “influential” scientific, financial, or statistical

information, they should provide a higher level of transparency about data and methods. Unless 

prevented by confidentiality, legal constraints, or other compelling interests, the level of 

transparency should be such that qualified third parties could reproduce the information. In 

identifying what kinds of information may be subject to reproducibility standards, agencies 

should use commonly accepted scientific, financial, or statistical standards. Agencies are 

encouraged to make arrangements that will permit appropriate public access to the related 

original and supporting data and analytical results. Regarding analytical results in situations 

where agencies do not permit access to data or methods due to other compelling interests, such 

as confidentiality protections, agencies should, unless otherwise prohibited by law, generally 

disclose their data sources (at whatever levels of generality are needed to preserve necessary 

confidentiality), quantitative methods and assumptions that have been employed, and the types of 

robustness checks used to assure the quality of results. 

1996 SAFE DRINKING WATER ACT AMENDMENTS FOR SAFETY AND HEALTH 

RISK ANALYSIS PRINCIPLES 

With regard to analysis of risks to human health, safety, and the environment maintained or 

disseminated by agencies, the Occupational Safety and Health Administration (OSHA) and the 

Mine Safety and Health Administration (MSHA), in performing risk analysis, are hereby 

adapting the standards contained in the Safe Drinking Water Act, as set forth in Appendix II. 

DOL does not anticipate that any other agencies will be performing such analysis of risks for 

their programs. However, to deal with unforeseen contingencies, DOL hereby adopts the Safe 

Drinking Water Act standards with respect to all programs other than OSHA and MSHA. 

Should it be necessary in the future for another DOL agency to perform such an analysis, DOL 

will consider, at that time, whether it is appropriate to adapt the Safe Drinking Water Act 

standards. 

BLS GUIDELINES FOR INFORMING USERS OF INFORMATION QUALITY AND 

METHODOLOGY 

Appendix III contains the Bureau of Labor Statistics (BLS) Guidelines for Informing Users of

Information Quality and Methodology supplement to the overall Departmental guidelines. These 

supplemental guidelines reaffirm BLS commitment to both OMB and DOL information quality 

guidelines. Moreover, as part of the Interagency Council on Statistical Policy, BLS supports the 

Council's commitment to information quality. 

 4 
U.S. Department of Labor Information Quality Guidelines 

INFORMATION QUALITY RESPONSIBILITIES 

It is the responsibility of all Departmental agencies to make information quality an important 

goal in every phase of a product’s development. The following responsibilities pertain to the 

implementation of DOL’s information quality guidelines. 

Chief Information Officer (CIO) 

• Maintain a leadership role in overseeing the implementation of these guidelines and in 

providing guidance to the agencies on information quality matters. 

• Develop and submit to OMB the annual report on the number, nature, and resolution of

complaints. 

• Coordinate, as appropriate, with other Federal organizations on cross-agency information 

quality issues. 

Agency Heads 

• Apply, consistent with applicable statutes and regulations, DOL’s information quality 

policies, procedures, and guidance to Department-sponsored information products that an 

agency has direct authority to control. 

• Ensure that, where Department-sponsored information does not necessarily reflect the views 

of DOL, an appropriate disclaimer will be included. 

• Ensure that in its submissions to the OMB under the Paperwork Reduction Act, the agency 

demonstrates how it is attempting to provide that information will be collected, maintained, 

and used in a way consistent with OMB and DOL information quality standards. 

INFORMATION CATEGORIES 

Per OMB’s guidance, information means any communication or representation of knowledge 

such as facts or data, in any medium or form, including textual, numerical, graphic, cartographic, 

narrative, or audiovisual forms. This definition includes information that an agency disseminates 

from a web page, but does not include the provision of hyperlinks to information that others 

disseminate. This definition does not include opinions, where the agency’s presentation makes it 

clear that what is being offered is someone’s opinion rather than fact or the agency’s views on 

information of the kind that is subject to these guidelines.

DOL has identified two categories of information that are disseminated to the public, with the 

level of quality control and review being greater for influential information than for noninfluential information. Whether information is influential is to be determined on an item-byitem basis rather than by aggregating multiple studies, documents, or other informational items 

that may influence a single policy or decision. 

Influential 

Definition: This category contains scientific, financial, or statistical information when agencies

can reasonably determine that dissemination will have or does have a clear and 

substantial impact on important public policies or important private sector 

decisions. 

 5 
U.S. Department of Labor Information Quality Guidelines 

To be influential, information should have a clear and substantial impact. A clear impact is one 

that is determined by the agency to have a high probability of occurring. A substantial impact is 

one that meets the levels of significance described below. 

In rulemaking, influential information is scientific, financial, or statistical information that the 

agency believes will have a clear and substantial impact on the resolution of one or more key 

issues in an economically significant rulemaking, as that term is defined in section 3(f)(1) of 

Executive Order 12866. 

In non-rulemaking contexts, DOL should consider two factors in relation to each other – breadth 

and intensity – in determining whether information is influential. These factors need to be 

considered together. Information that has a low cost or modest impact on a limited range of 

affected parties is less likely to be influential than information that can have a very costly or 

crucial impact on a broad range of parties. Of course even information that has a low cost or 

modest impact on any one party can be influential if it can impact a broad range of parties -- for 

example, an action that could cost an individual employer only $20 could nevertheless be 

influential if it impacts the vast number of employers in the United States, since it would then 

have an aggregate effect in excess of $100 million. Within that framework, in considering 

whether information has a high intensity impact, agencies should use as a benchmark the $100 

million figure used to determine whether a rule is economically significant. It should be noted 

that the definition of “influential” applies to information itself and not to the decisions that the 

information may support. Even if a decision or action by an agency is very important, a 

particular piece of information supporting it may not be influential, for example, because it is 

cumulative to other information or because it involves legal or policy issues. 

Moreover, if it is merely arguable that an impact will occur, or if it is a close judgment call, then 

the impact is probably not clear and substantial. The "influential" designation is intended to be 

applied to information only when clearly appropriate. Agencies should not designate 

information products or types of information as influential on a regular or routine basis. Nor 

should agencies place an "influential" label on the title page or text of an information product. 

Examples: Principal economic indicators, such as Consumer Price Index, the Employment 

Situation, and Producer Price Index; the Private Pension Bulletin; and 

Unemployment Insurance Weekly Claims data. 

 6 
U.S. Department of Labor Information Quality Guidelines 

Non-influential 

Definition: All information disseminated to the public that does not meet the criteria set forth in 

the influential information definition.

Examples: Fact sheets (e.g., Disaster Unemployment Assistance, Injury Trends in Mining), 

OSHA Construction Resource Manual, technical information issuances, annual 

reports, and studies (e.g., Pension and Health Benefits of American Workers, 

Coverage Status of Workers under Employer-Provided Pension Plans, and Study of 

401(k) Plan Fees and Expenses, Study of Health Insurance Coverage of the 

Unemployed). 

PRIVACY AND SECURITY INFORMATION AND PUBLIC DISCLOSURE 

Regardless of the category of information, all agencies will comply with the Privacy and 

Security Statement posted on DOL’s web site. DOL is strongly committed to maintaining the 

privacy of information and the security of its computer systems. With respect to the collection, 

use, and disclosure of information, DOL makes every effort to ensure compliance with 

applicable Federal laws, including, but not limited to, the Privacy Act of 1974, the Paperwork 

Reduction Act of 1995, the Trade Secrets Act, and the Freedom of Information Act. DOL 

reaffirms its commitment to keep the public appropriately informed. 

As part of its efforts to ensure and maintain the integrity of the information disseminated to the 

public, DOL’s IT security policy and planning framework is designed to protect information 

from unauthorized access or revision and to ensure that the information is not compromised 

through corruption or falsification. 

INFORMATION QUALITY ASSURANCE PROCESS

Departmental agencies should use the information quality assurance process described below to 

maximize the quality of information disseminated. Agencies should use information quality 

assurance processes that are appropriate to the complexity and importance of the product being 

developed. Agencies may use appropriate pre-existing information quality assurance processes 

that are at least as effective as those of DOL or OMB. 

The quality assurance process should begin at the inception of the product development process. 

At the initiation of the product development process, agencies should consult existing 

Departmental and agency information quality assurance guidelines. Agencies should determine 

the information category of the product to be developed, the level of quality assurance needed, 

and the appropriate techniques required to maximize and ensure information quality.

There are numerous techniques and methods agencies can utilize to ensure they consistently 

produce and disseminate quality information. Appendix IV provides some sample techniques 

and methods derived from industry best practices. Agencies should use the information quality 

assurance techniques and methods that they determine are most appropriate for their information 

products. If agencies choose to conduct a formal, independent, external peer review of data and 

analytical results, the peer review should meet the following general criteria: (a) peer reviewers 

 7 
U.S. Department of Labor Information Quality Guidelines 

should be selected primarily on the basis of necessary technical expertise; (b) peer reviewers 

should be expected to disclose to agencies prior technical or policy positions they may have 

taken on the issues at hand; (c) peer reviewers should be expected to disclose to agencies their 

sources of personal and institutional funding (private or public sector); and (d) peer reviews 

should be conducted in an open (made public) and rigorous manner. 

Agencies should incorporate the selected quality assurance techniques into the project 

development schedule. Throughout the product's development, agencies should ensure that 

quality assurance decisions are defensible and appropriate to the category of information 

involved. The product may be subject to internal agency quality controls and any appropriate 

Departmental reviews before being disseminated to the public. For example, if an agency 

decides to post the information on the DOL web site, it should adhere to DOL’s and its own 

Public Web Site Review and Clearance Process. Agencies should incorporate lessons learned 

into future product development activities so as to improve DOL’s overall quality management 

process. 

INFORMATION COMPLAINT AND APPEALS PROCESS 

Because DOL is committed to information dissemination programs based on high standards of 

quality, it recognizes the value of public input. DOL therefore encourages the affected public to 

suggest improvements in Departmental information quality practices and to contact it when 

particular disseminated information may not meet the OMB guidelines and the guidelines set 

forth above. DOL believes that in most cases, informal contacts would be appropriate. 

Sometimes agencies and affected persons may find it helpful to resolve concerns about 

information in a more structured way and may choose to follow a more formal process. DOL 

will make available to the public a list of officials to whom complaints and appeals should be 

sent and where and how such officials may be reached. Affected persons may submit such 

complaints and appeals to the contact point in the DOL agency responsible for the information. 

Each agency may designate one or more officials to review information complaints and another 

official or officials who will be responsible for appeals if the complainant is dissatisfied with the 

initial response to the complaint. The agency should provide that the official conducting the 

second level review is not the same official who responded to the initial request or from the same

office that prepared the information in question. In determining the level of the person 

designated to respond to appeals, the agency may wish to consider such factors as the 

qualifications of the person and the significance of the information in question. An agency may, 

within its discretion, wish to consider the designation of a panel to resolve appeals involving 

influential information, when the agency believes that such an appellate board is needed and is 

an efficient way to provide expertise or perspective or otherwise to improve the resolution of the 

appeal. Designated agency officials may consult with other agency or Departmental offices, as 

the agency may deem appropriate to the resolution of the complaint. 

The purpose of the information complaint and appeal process is to deal with information quality 

matters, not to resolve underlying substantive policy or legal issues. 

 8 
U.S. Department of Labor Information Quality Guidelines 

As is the case with other provisions of these guidelines, the process is intended to improve the 

internal management of the Federal Government. It is not intended to create any right or benefit, 

substantive or procedural, enforceable at law by a party against the United States, its agencies, 

officers, or any person. It is not intended to provide any right of judicial review. Concerns 

regarding information in a rulemaking must, except as provided below, be presented in the 

rulemaking in accordance with the rulemaking’s procedures. 

Overview of Information Complaint and Appeal Process

Affected persons may indicate their interest in following a more structured complaint and appeal 

process by expressing that interest to the concerned agency. Complainants should: 

• Identify themselves and indicate where and how they can be reached; 

• Identify, as specifically as possible, the information in question; 

• Indicate how they are affected by the information about which they are complaining; 

• Carefully describe the nature of the complaint, including an explanation of why they believe 

the information does not comply with OMB, Departmental, or agency-specific guidelines; 

and 

• Describe the change requested and the reason why the agency should make the change. 

Failure to include this information may result in a complainant not receiving a response to the 

complaint or greatly reducing the usefulness or timeliness of any response. Complainants should 

be aware that they bear the burden of establishing that they are affected persons and showing the 

need and justification for the correction they are seeking, including why the information being 

complained about does not comply with applicable guidelines. 

In deciding how to handle complaints, agencies should be especially mindful of their legal 

obligations, program priorities, resource constraints, and their duty to use resources efficiently. 

For example, agencies have important responsibilities to issue rules and provide compliance 

guidance to the public. Agencies must administer the complaint and appeal process consistent 

with these obligations and their responsibilities to carry them out in an expeditious manner. 

Any structured process would not apply to an agency’s archival information or to public filings. 

Agencies may choose not to respond to complaints about claimed defects that are frivolous or 

unlikely to have substantial future impact. 

Where procedures exist for dealing with information quality issues, agencies may consider 

resolving complaints by referring them to these procedures. For example, complaints about the 

quality of information in a rulemaking are ordinarily to be submitted and handled in accordance 

with rulemaking procedures. As agencies consider information quality issues within the context 

of a rulemaking, they are reminded of their primary responsibility to resolve these issues in a 

manner consistent with the Administrative Procedure Act and the substantive statute pursuant to 

which the rule is being issued. In unusual circumstances involving an information product 

related to a rulemaking, agencies should consider an information complaint under these 

information correction procedures. When the agency disseminates a study, analysis, or other 

information prior to the final agency action or information product, requests for correction 

 9 
U.S. Department of Labor Information Quality Guidelines 

should be considered prior to the final agency action or information product in those cases if the 

agency determines that an earlier response would not unduly delay issuance of the agency action 

or information product and the complainant has shown a reasonable likelihood of suffering 

actual harm from the agency's dissemination if the agency does not resolve the complaint prior to 

the final agency action or information product. In deciding what action may be appropriate in 

these unusual circumstances, agencies should consider the factors previously discussed in these 

guidelines. They also may consider: (1) the impact of the information on the complainant; (2) 

the extent to which the complainant's concerns have been rendered moot as a result of actions 

taken by the agency; (3) the mechanisms available under the Administrative Procedure Act or 

other laws to resolve complainant's concerns; and (4) the public interest to be served in pursuing 

further action on the complaint. 

Where an agency responds directly to a complaint, it should respond in the manner that it deems 

most suitable, whether by letter, telephone, email, or otherwise. 

Agencies should try to respond to complaints and appeals within sixty (60) days of their receipt, 

unless they deem a response within this time period to be impracticable. If an agency believes 

that more time is required to decide how to respond to a complaint or appeal, it should estimate 

the time needed and notify the complainant within the 60-day period of the reasons for the delay 

and the time that it estimates that a decision will be reached. Once the agency had decided how 

to address the complaint, it should notify the complainant. 

If a complainant is dissatisfied with the initial response to the complaint, he or she may submit 

an appeal to the designated contact point in the agency responsible for the information. 

A complainant may appeal within forty-five (45) days of the date the agency notified the 

complainant how it would handle the complaint or one hundred and five (105) days from the date 

on which an agency or agencies first received the complaint, whichever is later. The appeal 

request should contain the same contact and descriptive information that was provided in the 

original complaint and the specific reasons why the initial agency response was not satisfactory. 

Once an appeal decision has been rendered by the agency, it should notify the complainant. 

In processing initial complaints and appeal requests, DOL and its agencies should be flexible and 

take into account, among other things, the nature, significance, and volume of complaints, the 

agency’s particular program needs, and available review mechanisms. 

TRACKING AND REPORTING INFORMATION COMPLAINTS AND APPEALS 

The Office of the Chief Information Officer (OCIO) is responsible for reporting the results of the 

Department’s information quality efforts as required by OMB guidance. OCIO anticipates using 

a system to centrally track and report complaints and appeals. 

In accordance with procedures to be specified by the CIO, DOL will establish on its web site or 

agency web sites an information quality site to keep the public informed about information 

quality on a timely basis. The purpose of the information quality web site would be to inform

the public about the agency's information quality practices and procedures. The information 

 10 
U.S. Department of Labor Information Quality Guidelines 

quality web site should include access to the agency's information quality guidelines and an 

easy-to-understand explanation of the agency's procedures regarding complaints (which will 

include an explanation of how a person may file a complaint and, subsequently, an 

administrative appeal of the agency's response to the complaint). The information quality web 

site also could contain other types of information, such as a description of significant corrections 

that the agency has made as a result of the information complaint and appeal process. Each 

agency should determine the content of this information page based on its mission, activities 

subject to the guidelines, and the expected level of interest by members of the public. 

 11 
U.S. Department of Labor Information Quality Guidelines 

APPENDIX I: INFORMATION QUALITY GLOSSARY 

OMB provides the following definitions in its guidance for the Act. 

1. “Quality” is an encompassing term comprising utility, objectivity, and integrity. Therefore, 

the guidelines sometimes refer to these four statutory terms, collectively, as “quality.” 

2. “Utility” refers to the usefulness of the information for its intended users, including the 

public. In assessing the usefulness of information that the agency disseminates to the public, 

the agency needs to consider the uses of information not only from the perspective of the 

agency but also from the perspective of the public. As a result, when transparency of 

information is relevant for assessing the information’s usefulness from the public’s 

perspective, the agency must take care to ensure that transparency has been addressed in its 

review of the information. 

3. “Objectivity” involves two distinct elements, presentation and substance. 

a. “Objectivity” includes whether disseminated information is being presented in an 

accurate, clear, complete, and unbiased manner. This involves whether the information is 

presented within a proper context. Sometimes, in disseminating certain types of 

information to the public, other information must also be disseminated in order to ensure 

an accurate, clear, complete, and unbiased presentation. Also, the agency needs to 

identify the sources of the disseminated information (to the extent possible, consistent 

with confidentiality protections) and, in scientific, financial, or statistical context, the 

supporting data and models, so that the public can assess for itself whether there may be 

some reason to question the objectivity of the sources. Where appropriate, supporting 

data should have full, accurate, transparent documentation, and error sources affecting 

data quality should be identified and disclosed to users. 

b. In addition, “objectivity” involves a focus on ensuring accurate, reliable, and unbiased 

information. In a scientific, financial, or statistical context, the original and supporting 

data shall be generated, and the analytical results shall be developed, using sound 

statistical and research methods. 

i. If data and analytic results have been subjected to formal, independent, external peer 

review, the information may generally be presumed to be of acceptable objectivity. 

However, this presumption is rebuttable based on a persuasive showing by the 

petitioner in a particular instance. If agency-sponsored peer review is employed to 

help satisfy the objectivity standard, the review process employed shall meet the 

general criteria for competent and credible peer review recommended by OMB-OIRA 

to the President’s Management Council (9/20/01) 

(http://www.whitehouse.gov/omb/inforeg/oira_review-process.html), namely, “that 

(a) peer reviewers be selected primarily on the basis of necessary technical expertise, 

(b) peer reviewers be expected to disclose to agencies prior technical/policy positions 

they may have taken on the issues at hand, (c) peer reviewers be expected to disclose 

to agencies their sources of personal and institutional funding (private or public 

sector), and (d) peer reviews be conducted in an open and rigorous manner.” 

ii. If an agency is responsible for disseminating influential scientific, financial, or

statistical information, agency guidelines shall include a high degree of transparency 

 12 
U.S. Department of Labor Information Quality Guidelines 

about data and methods to facilitate the reproducibility of such information by 

qualified third parties. 

A. With regard to original and supporting data related thereto, agency guidelines 

shall not require that all disseminated data be subjected to a reproducibility 

requirement. Agencies may identify, in consultation with the relevant scientific 

and technical communities, those particular types of data that can practicably be 

subjected to a reproducibility requirement, given ethical, feasibility, or 

confidentiality restraints. It is understood that reproducibility of data is an 

indication of transparency about research design and methods and thus a 

replication exercise (i.e., a new experiment, test, or sample) shall not be required 

prior to each dissemination. 

B. With regard to analytic results related thereto, agency guidelines shall generally 

require sufficient transparency about data and methods that an independent 

reanalysis could be undertaken by a qualified member of the public. These 

transparency standards apply to agency analysis of data from a single study as 

well as to analyses that combine information from multiple studies. 

i Making the data and methods publicly available will assist in determining

whether analytic results are reproducible. However, the objectivity standard 

does not override other compelling interests such as privacy, trade secrets,

intellectual property, and other confidentiality protections. 

ii In situations where public access to data and methods will not occur due to

other compelling interests, agencies shall apply especially rigorous robustness 

checks to analytic results and document what checks were undertaken. 

Agency guidelines shall, however, in all cases, require a disclosure of the 

specific quantitative methods and assumptions that have been employed. 

Each agency is authorized to define the type of robustness checks, and the 

level of detail for documentation thereof, in ways appropriate for it given the 

nature and multiplicity of issues for which the agency is responsible. 

C. With regard to analysis of risks to human health, safety, and the environment 

maintained or disseminated by the agencies, agencies shall either adopt or adapt 

the quality principles applied by Congress to risk information used and 

disseminated pursuant to the Safe Drinking Water Act Amendments of 1996 (42 

U.S.C. 300g-1(b)(3)(A) and (B)). Agencies responsible for dissemination of vital 

health and medical information shall interpret the reproducibility and peer review 

standards in a manner appropriate to assuring the timely flow of vital information 

from agencies to medical providers, patients, health agencies, and the public. 

Information quality standards may be waived temporarily by agencies under 

urgent situations (e.g., imminent threats to public health or homeland security) in 

accordance with the latitude specified in agency-specific guidelines. 

4. “Integrity” refers to the security of information – protection of the information from 

unauthorized access or revision, to ensure that the information is not compromised through 

corruption or falsification. 

5. “Information” means any communication or representation of knowledge such as facts or 

data, in any medium or form, including textual, numerical, graphic, cartographic, narrative, 

 13 
U.S. Department of Labor Information Quality Guidelines 

or audiovisual forms. This definition includes information that an agency disseminates from

a web page, but does not include the provision of hyperlinks to information that others 

disseminate. This definition does not include opinions, where the agency’s presentation 

makes it clear that what is being offered is someone’s opinion rather than fact or the agency’s 

views. 

6. “Government information” means information created, collected, processed, disseminated, 

or disposed of by or for the Federal Government. 

7. “Information dissemination product” means any book, paper, map, machine-readable 

material, audiovisual production, or other documentary material, regardless of physical form

or characteristic, an agency disseminates to the public. This definition includes any 

electronic document, CD-ROM, or web page. 

8. “Dissemination” means agency-initiated or sponsored distribution of information to the 

public (see 5 CFR 1320.3(d) (definition of “Conduct or Sponsor”). Dissemination does not 

include distribution limited to: government employees or agency contractors or grantees; 

intra- or inter-agency use or sharing of government information; and responses to requests 

for agency records under the Freedom of Information Act, the Privacy Act, the Federal 

Advisory Committee Act, or other similar law. This definition also does not include 

distribution limited to: correspondence with individuals or persons; press releases; archival 

records; public filings; subpoenas; or adjudicative processes. 

9. “Influential” when used in the phrase “influential scientific, financial, or statistical 

information” means that the agency can reasonably determine that dissemination of the 

information will have or does have a clear and substantial impact on important public 

policies or important private sector decisions. Each agency is authorized to define 

“influential” in ways appropriate for it given the nature and multiplicity of issues for which 

the agency is responsible. 

10. “Reproducibility” means that the information is capable of being substantially reproduced, 

subject to an acceptable degree of imprecision. For information judged to have more (less) 

important impacts, the degree of imprecision that is tolerated is reduced (increased). If 

agencies apply the reproducibility test to specific types of original or supporting data, the 

associated guidelines shall provide relevant definitions of reproducibility (e.g., standards for 

replication of laboratory data). With respect to analytic results, “capable of being 

substantially reproduced” means that independent analysis of the original or supporting data 

using identical methods would generate similar analytic results, subject to an acceptable 

degree of imprecision or error. 

 14 
U.S. Department of Labor Information Quality Guidelines 

APPENDIX II: ADAPTING THE PRINCIPLES UNDER THE 1996 SAFE DRINKING 

WATER ACT AMENDMENTS FOR SAFETY AND HEALTH RISK ANALYSES 

When disseminating influential information in the context of analyses of safety, health, or 

environmental risks, the final OMB guidelines instruct agencies to “...adopt or adapt the quality 

principles applied by Congress to risk information used and disseminated pursuant to the Safe 

Drinking Water Act Amendments of 1996 (42 U.S.C. 300g-1(b)(3)(A) & (B)).” According to 

the preamble to OMB’s final guidelines (67 F.R. 375), these principles reflect a “...basic 

standard of quality for the use of science in agency decision making” and “...a basic quality 

standard for the dissemination of public information about risks of adverse health effects.” 

Specifically, 42 U.S.C. 300g-1(b)(3)(A) states that “to the degree that an agency action is based 

on science, the [EPA] Administrator shall use “(i) the best available, peer-reviewed science and 

supporting studies conducted in accordance with sound and objective scientific practices; and (ii) 

data collected by accepted methods or best available methods (if the reliability of the method and 

the nature of the decision justifies use of the data).” Under 42 U.S.C. 300g-1(b)(3)(B), “the 

Administrator shall ensure that the presentation of information on public health effects is 

comprehensive, informative, and understandable.” Finally, in documents made available to the 

public to support regulation, this section of the Safe Drinking Water Act Amendments requires 

such documents to specify the following, to the extent practicable: 

(i) each population addressed by any estimate of public health effects; 

(ii) the expected risk or central estimate of risk for the specific populations; 

(iii) each appropriate upper-bound or lower-bound estimate of risk; 

(iv) each significant uncertainty identified in the process of the assessment of public health 

effects and studies that would assist in resolving the uncertainty; and 

(v) peer-reviewed studies known to the Administrator that support, are directly relevant to, 

or fail to support any estimate of public health effects and the methodology used to 

reconcile inconsistencies in the scientific data. 

Within the Department of Labor, analyses of safety and health risks are performed primarily by 

the Occupational Safety Health Administration (OSHA) and the Mine Safety and Health 

Administration (MSHA). Such analyses have generally been done only in connection with 

promulgating safety and health rules; as such, risk analyses disseminated by these agencies are 

subject to statutory requirements governing the bases for regulatory decision making as well as 

the public rulemaking process. 

DOL is adapting the principles of the Safe Drinking Water Act Amendments for both health and 

safety risk analyses. For health analyses, the principles will be adapted as follows: 

1. In taking agency actions that are based on the use of science in the analysis of health risks,

the agency shall use 

a. the best available peer-reviewed science and supporting studies conducted in 

accordance with sound and objective scientific practices; and

 15 
U.S. Department of Labor Information Quality Guidelines 

b. data collected by accepted methods or best available methods (if the reliability of the 

method and the nature of the decision justifies use of the data), including; 

i. exposure data such as that generated by enforcement activity, contained in 

published literature, and submitted to the rulemaking record; and 

ii. testimony and comment from experts familiar with the underlying scientific 

information related to the risk analysis and other relevant information in the 

rulemaking record. 

2. In the dissemination of public information about risks, the agency shall ensure that the 

presentation of information about risk effects is comprehensive, informative, and 

understandable, within the context of its intended purpose. 

3. In a quantitative analysis of health risks made available to the public, the agency shall 

specify, to the extent practicable: 

a. each population addressed by any estimate of public health effects; 

b. the expected risk or central estimate of risk for the specific populations; 

c. each appropriate upper-bound or lower-bound estimate of risk; 

d. each significant uncertainty identified in the assessment of public health effects and 

studies that would assist in resolving the uncertainty; and 

e. information, data, or studies, peer-reviewed where available, known to the agency that 

support, are directly relevant to, or fail to support any estimate of risk effects and a 

discussion that reconciles inconsistencies in the data or information, and explains the 

rationale used by the agency to rely on the data or information used for the risk 

analysis. 

For safety risk analyses, the principles will be adapted as follows: 

1. In taking agency actions that are based on the use of science in the analysis of safety risks,

the agency shall use 

a. the best available statistical data from surveys of fatalities, injuries, and illnesses, and 

the best available peer-reviewed science and supporting studies that describe the nature 

of the safety risks being addressed; 

b. data collected by accepted methods or best available methods (if the reliability of the 

method and the nature of the decision justifies use of the data), including; 

i. incident reports compiled from an agency’s information collection or

enforcement activities; 

ii. incident or accident investigation reports provided by the public or private 

sectors; 

iii. relevant analyses of such information or data, peer reviewed where available; 

and 

 16 
U.S. Department of Labor Information Quality Guidelines 

iv. testimony of experts familiar with the causal nature of fatalities, injuries, or 

illnesses being addressed in the safety risk analysis and other relevant 

information in the rulemaking record. 

2. In the dissemination of public information about safety risks, the agency shall ensure that 

the presentation of information is comprehensive, informative, and understandable, within 

the context of its intended purpose. 

3. In a quantitative risk analysis of safety risks made available to the public, the agency will 

specify, to the extent practicable: 

a. the agency’s best estimate of the size of the population at risk of such effects by 

industry sector; 

b. the agency’s best estimates of the total number and or rate of fatalities, injuries, or 

illnesses that occur each year and that are relevant to the safety risks being addressed;

c. the possible range in the agency’s best estimate of the number or rate of fatalities, 

injuries, or illnesses, taking into account possible uncertainties in the data underlying 

the estimate; 

d. data gaps and other significant uncertainties identified in the assessment of risk effects 

and the kind of data or information that would assist in reducing uncertainty; and 

e. information, data, or studies, peer-reviewed if available, known to the agency that 

support, are directly relevant to, or fail to support any estimate of risk effects and a 

discussion that reconciles inconsistencies in the data or information, and explains the 

rationale used by the agency to rely on the data or information used for the risk 

analysis. 

With regard to statutory requirements, Section 6(b)(5) of the OSH Act of 1970 and Section 

101(a)(6)(A) of the Federal Mine Safety and Health Act of 1977 require the Secretary to set 

health standards, in part, “on the basis of the best available evidence,” and that development of 

standards be based upon “research, demonstrations, experiments, and such other information as 

may be appropriate.” Section 6(b)(5) and Section 101(a)(6)(A) also state that “...[i]n addition to 

the attainment of the highest degree of health and safety protection for the employee, other 

considerations shall be the latest available scientific data in the field...” Furthermore, Section 

6(f) of the OSH Act mandates that the Secretary’s determinations be considered conclusive “if 

supported by substantial evidence in the [rulemaking] record considered as a whole.” 

Thus, the OSH Act and Mine Act reflect the basic principle underlying the requirements of the 

Safe Drinking Water Act Amendments – that agency actions be based on the best scientific 

information available at the time of the agency action. OSHA’s and MSHA’s risk assessments 

disseminated in past health rulemakings have relied on the kinds of scientific information 

described in the Amendments, i.e., “peer-reviewed science and supporting studies” as well as 

other data that the agency considers were collected by “accepted methods or the best available 

methods.” The agencies recognize that peer review adds significant value to a scientific study. 

However, in developing risk assessments to support rulemakings, the agencies also consider all 

other information submitted to the record, including expert testimony, written comments from

the scientific community on data and other information contained in the record, including risk 

 17 
U.S. Department of Labor Information Quality Guidelines 

analyses conducted by rulemaking participants and submitted to the record. In those instances 

where agencies are compelled to take actions to protect the public from serious risks absent the 

availability of peer-reviewed scientific literature, DOL believes it consistent with its adaptation 

of the SDWA principles, as well as applicable statutes, to take such action based on data 

collected by “accepted methods or best available methods” so long as the nature of the action 

justifies the use of the data. 

Part of what can be considered the risk analysis in the context of the Safe Drinking Water Act 

Amendments also appears in OSHA’s and MSHA’s Economic Analyses for proposed and final 

health rules. The Economic Analysis includes an analysis of worker exposures to the health 

hazard of interest, estimates of the sizes of the exposed worker populations in affected industry 

sectors, and an analysis of the numbers of exposure-related illnesses that occur in those 

populations and the numbers of illnesses potentially avoided by the new standard. In past 

rulemakings, OSHA and MSHA have found relatively few peer-reviewed studies available from

which the agencies could reliably construct exposure profiles for all or most affected industry 

sectors. Information and data typically relied upon by the agencies to conduct these analyses 

include exposure data generated by enforcement activity, exposure data submitted to the record 

by industry or labor organizations, industry studies conducted by the National Institute for 

Occupational Safety and Health (NIOSH), and data obtained by the agencies or their contractors 

during the conduct of site visits to industrial facilities. In addition, OSHA has usually relied on 

statistics published by the Bureau of Labor Statistics (BLS) or the U.S. Bureau of the Census to 

develop estimates of the size of the population at risk. 

Economic and cost data are normally not available from peer-reviewed studies. Such data often 

comes from industry sources that require confidentiality, suppliers of equipment and services, 

and industrial manuals. Data on profits, sales, and other operations data come from census 

sources, Standards and Poors, SRI International and similar sources, surveys, extrapolations from

site visits, and industry and government reports. Data quality is met in these circumstances by 

clearly specifying sources, making available non-confidential information, and providing the 

spreadsheets and algorithms used by the agencies so persons can reproduce the analysis. 

Analyses of safety risks conducted by OSHA and MSHA to support safety standards are quite 

different from health risk analyses in terms of the kinds of data and information generally 

available to the agencies. The goal of a safety risk analysis is to describe the numbers, rates, and 

causal nature of injuries related to the safety risks being addressed. OSHA and MSHA have 

historically relied on injury and illness statistics from BLS, combined with incident or accident

reports from enforcement activities, incident or accident reports submitted to the record from the 

private or public sectors, testimony of experts who have experience dealing with the safety risks 

being addressed, and information and data supplied by organizations that develop consensus 

safety standards (such as the American National Standards Institute or the ASTM International). 

Thus, DOL’s adaptation of the Safe Drinking Water Act principles for safety risk analysis 

reflects the use of injury and illness statistics as a primary source of data, but also calls for the 

use of peer-reviewed scientific data and supporting studies where they are available. 

In disseminating its health and safety risk analyses for proposed and final rules, it has been 

OSHA’s and MSHA’s practice to state clearly its reasons for using the kinds of information and 

 18 
U.S. Department of Labor Information Quality Guidelines 

data described above; this is necessary to demonstrate that the agencies have relied on the “best 

available evidence” in making its conclusions. Because of the requirements of rulemaking 

procedures to consider all evidence and comment placed in the record by interested parties, the 

Department intends to adapt the principles set forth in the 1996 Safe Drinking Water Act 

Amendments to reflect that agencies must consider their statute and case law and data and 

evidence contained in the rulemaking docket, provided the agencies clearly state the reasons for 

relying on particular data and evidence in the risk analysis. That is, in addition to “peerreviewed science and supporting studies” and “data collected by accepted methods or best 

available methods,” agencies may consider expert testimony, public comment, and other data 

and information contained in the rulemaking record, and may rely on such testimony and 

information in their risk analyses provided that the agencies clearly communicate their rationale 

for selecting such data and information and why it is consistent with statutory requirements to 

use the best available information. 

The principles outlined by the Safe Drinking Water Act Amendments also contain a specification 

for reporting results of risk analyses, as described above. For health risk analyses, OSHA and 

MSHA have historically reported their “best estimate” of the risk to workers exposed to a health 

hazard; this has typically been an estimate that the agencies refer to as a “maximum likelihood” 

estimate derived from the statistical procedure of fitting a mathematical exposure-response curve 

to dose-response data. The agencies also typically have reported statistical upper limits of their 

estimates of risk. The industry and exposure profiles presented in the Economic Analysis 

provide estimates of the populations at risk, by affected industry sector. Finally, during the 

course of rulemaking, OSHA and MSHA must consider and address data, expert testimony, and 

public comment that deal with uncertainties in the risk assessment and with conflicting scientific

evidence. As part of demonstrating that it has relied on the “best available evidence”, the agency 

must also clearly present its reasons for accepting certain studies or data and rejecting others, and 

reconcile apparent discrepancies or conflicts in the available data to the extent possible. These 

practices are consistent with the reporting principles described by the Safe Drinking Water Act 

Amendments, as well as the obligations of the OSH Act and the Mine Act. 

These general principles also apply to OSHA’s and MSHA’s reporting of results and conclusions 

from analyses of safety risks; that is, the agencies make every effort to reliably estimate the sizes 

of the populations at risk and the magnitude of the safety risk presented to workers, and to 

explain uncertainties and apparent discrepancies in the available data. However, as described 

above, the methods and underlying data relied on for safety risk analyses are often different from 

that for health risk analyses; thus, DOL has adapted the language of the Safe Drinking Water Act 

principles as applied to the dissemination of information on safety risks to reflect the kinds of 

results typically obtained from a safety risk analysis. 

 19 
U.S. Department of Labor Information Quality Guidelines 

APPENDIX III: BLS GUIDELINES FOR INFORMING USERS OF INFORMATION 

QUALITY AND METHODOLOGY 

The Office of Management and Budget (OMB) has issued government-wide information quality 

guidelines in accordance with Section 515 of the Treasury and General Government

Appropriations Act for Fiscal Year 2001. The purpose of these guidelines is to ensure and 

maximize the quality, utility, objectivity, and integrity of information disseminated by Federal 

agencies. The guidelines direct each Federal agency to issue its own Section 515 guidelines. As 

part of the Department of Labor, BLS follows DOL’s information quality guidelines, as well as 

the OMB Guidelines. Moreover, as part of the Interagency Council on Statistical Policy, BLS 

supports the Council’s commitment to information quality. The following BLS Guidelines for 

Informing Users of Information Quality and Methodology supplement the DOL guidelines that 

apply to all Departmental agencies. 

The Bureau of Labor Statistics is the principal fact-finding agency for the Federal Government in 

the broad field of labor economics and statistics. BLS is an independent national statistical 

agency within the Department of Labor that collects, processes, analyzes, and disseminates 

essential statistical data to the American public, business, and labor. BLS also serves as a 

statistical resource to the Department of Labor. 

BLS data must satisfy a number of criteria, including relevance to current social and economic 

issues, timeliness in reflecting today’s rapidly changing economic conditions, accuracy, 

consistently high statistical quality, and impartiality in both subject matter and presentation. 

As a Federal statistical agency, BLS conducts work in an open environment. Major changes in 

program design, scope, or methods are discussed in advance with users and advisory committees 

and described in published materials. Fair information practices are used, such as maintaining 

the confidentiality of individual responses. Confidentiality of the information that respondents 

furnish is assured by protecting the microdata, combining the data reported, and issuing the 

findings in summary tables, analyses, and reports. BLS values cooperation with data users and 

consults with a broad spectrum of users of its data in order to make its products more useful. As 

part of its customer pledge to the public, BLS promises to help users understand the uses and 

limitation of the data. 

BLS applies statistical information quality principles provided in guidance from the Office of 

Management and Budget (OMB Statistical Policy Directives, for example) as well as the 

National Research Council’s Principles and Practices for a Federal Statistical Agency. 

Moreover, all BLS information products are subjected to a multi-stage review before they are 

disseminated to the public. 

A key component in ensuring information quality is integrity, or the protection of data from

corruption through unauthorized access. BLS data integrity guidelines spell out procedures to 

protect the confidentiality of BLS records, the process of data collection, and various security 

measures. 

 20 
U.S. Department of Labor Information Quality Guidelines 

To inform users about information quality and methodology, BLS provides descriptions of the 

methods and procedures used to develop and produce its statistical products. These descriptions 

are prepared at various levels of complexity and comprehensiveness to address the wide range of 

user needs. Summary level technical notes are usually included with news releases. For most 

programs, a periodical of record contains more comprehensive technical material. In addition, 

Major Programs of the Bureau of Labor Statistics provides a summary description of data 

availability, coverage, sources of data, reference periods, major uses, and forms of publication. 

The BLS Handbook of Methods covers most major programs and is updated every few years. 

A major purpose for providing users with information on methodology is to assist them in 

determining whether the data adequately meet their needs both in terms of closeness of concept 

and range of statistical error. 

BLS makes the information it disseminates and the methods used to produce this information as 

transparent as possible, so they could, in principle, be reproduced by qualified individuals. In 

practice, however, most estimates included in the BLS information products are not directly 

reproducible by the public because the underlying data used to produce them contain confidential 

information about individual respondents. The transparency, therefore, has the related goal of 

providing enough information about methodology for the public to understand the information 

and to have confidence in its preparation. 

The level of documentation on methodology may differ among statistical programs based on 

type of data (from households or establishments), frequency of collection (monthly, annually, 

one-time), expected uses of the information, budget, and how long the survey has been in 

existence. The type of survey, census, or data collection process also may affect the existence of 

generally accepted evaluation methods and data collection protocols may affect the consistency 

of documentation. 

Most of the information on information quality and methodology are available in both print and 

electronic form to assist the broad range of users. Current descriptions of specific BLS surveys 

and programs are available at http://www.bls.gov/bls/descriptions.htm. This page provides links 

to the relevant sections of the BLS Handbook of Methods. For most programs, the Handbook

provides a variety of information that, as appropriate, may include a background summary, a 

description of the concepts, the sources of data and collection methods, the sampling and 

estimation procedures, and the uses and limitations of the statistics. Additional related 

information may be available from a program’s home page, and links to these home pages are 

included with the descriptions. 

As part of its commitment to information quality, BLS encourages communication with its users. 

In addition to formal advisory councils from the business, labor, and academic communities, 

BLS fosters discussions with the public at large by making it easy to reach staff by a variety of 

formats, including phone, mail, and email. A customer service guide is published annually with 

the names, phone numbers, and email addresses of subject matter specialists who can answer 

technical questions about the information BLS issues. Every page on the BLS web site has a link 

to a subject matter contact, a technical contact, and a general feedback contact. Every print 

 21 
U.S. Department of Labor Information Quality Guidelines 

publication also contains contact information. For more information on how to contact BLS, see 

its contact page. 

Affected persons who believe that BLS has disseminated information that does not meet its 

guidelines, or those of the DOL or OMB and who wish to follow a formal complaint process, 

may send their complaint to the point of contact that BLS is designating. 

 22 
U.S. Department of Labor Information Quality Guidelines 

 23 

APPENDIX IV: SAMPLE INFORMATION QUALITY ASSURANCE TECHNIQUES 

AND METHODS 

The table below provides some sample techniques and methods derived from industry best 

practices. This is not intended to be an exhaustive list. Agencies should select and apply 

techniques and methods depending on the complexity, influence, and subject matter of each 

information product. 

Techniques and 

Methods 

Definition Applicability 

Peer Review An independent assessment of the technical and 

scientific merit of research by individuals 

knowledgeable in the particular subject of interest and 

with no unresolved conflict of interest. 

Peer review is an appropriate technique for 

reviewing scientific studies and economic 

analyses. 

Certification Process of reviewing information prior to official 

release to ensure that erroneous data are not released, 

or to identify data of marginal quality. It is often 

conducted concurrently with an interpretative analysis 

of the data. 

Certification is an appropriate technique for 

statistical programs. 

Performance Measures Numerical indicators of the progress of the 

development of information. 

Performance Measures should be used to help 

management track the development of an 

information product and improve information 

quality, but they generally should also be used 

in conjunction with other, more rigorous quality 

assurance techniques. 

Check Lists A specific, step-based plan designed to ensure all 

appropriate actions are taken. 

Notes the steps in production that can identify 

inconsistencies, mistakes, or weaknesses, and 

ensure completeness. 

Bureau of Labor Statistics (BLS) 

Handbook of Methods 

Publication explaining how the BLS obtains and 

prepares the economic data it publishes. 

Applicable to statistical data collected by the 

BLS; and includes approaches and 

methodologies that could be appropriate for 

other agencies conducting surveys.