SUBMITTED ELECTRONICALLY VIA EMAIL TO SP2000-02@NIST.GOV

February 26, 2018

Ms. Lisa Carnahan

National Institute of Standards and Technology

100 Bureau Drive

Stop 2100

Gaithersburg, MD 20899

Dear Ms. Carnahan,

On behalf of UL LLC, I am pleased to submit comments with respect to the National Institute of 

Standards and Technology’s (NIST) request for comments on NIST Special Publication 2000-02: 

Conformity Assessment Considerations for Federal Agencies.

UL is a global, independent, safety-science company that has championed progress and safety for 

more than 120 years. Guided by our mission, UL’s 14,000 professionals promote safe working and 

living environments for all people. UL uses research, standards, and conformity assessment to 

continually advance and meet ever-evolving safety challenges, and partner with businesses, 

manufacturers, retailers, trade associations, and international regulatory authorities to provide 

solutions and address the risks of increasingly complex global supply chain.

UL supports NIST’s efforts to fulfill its responsibility under the National Technology Transfer and 

Advancement Act (NTTAA) and Office of Management and Budget (OMB) Circular A-119 to provide 

guidance to US federal agencies on conformity assessment issues. In that respect, while updating the 

ABCs of Conformity Assessment (Special Publication 2000-01) and issuing this Considerations

document are helpful, NIST should also proceed with revising the 2000 Guidance on Federal 

Conformity Assessment as codified in 15 CFR 287 to ensure it remains aligned with the 2016 revision 

to OMB A-119 and these newer NIST publications.

UL also applauds NIST for focusing on the importance of federal agencies engaging stakeholders 

when considering the conformity assessment activities on which it will rely. NIST has modeled this 

best practice in its own outreach to stakeholders in the development of this document by holding a 

public workshop in February 2017 and participating in numerous discussions with conformity 

assessment bodies, trade associations, industry associations, and others over the course of many 

months. UL is pleased that NIST also published notice of the availability of this document – and the 

companion ABCs – in the Federal Register.

Nevertheless, UL has identified some significant concerns with the Considerations document as 

currently drafted. These are outlined below:

1. Throughout the document, NIST misses opportunities to reinforce the spirit and principles 

of OMB A-119 (2016) with respect to conformity assessment. In executing its OMB A-119 

responsibility to “coordinate Federal, State, and local standards activities and conformity 

assessment activities with private sector standards development and conformity standards 

activities,” NIST often has limited its coordinating role to regulators, with inconsistent engagement 

with the private sector. To NIST’s credit, the efforts to reach out to private sector stakeholders in 

the development of the Considerations document are a step in the right direction; in continuing 

such outreach, however, NIST should seek to be inclusive and transparent and avoid “cherrypicking” amongst the private sector voices it consults. Additionally, NIST should avail itself of the 

many opportunities throughout the Considerations to reinforce OMB A-119 and NTTAA principles 
National Institute of Standards and Technology (NIST)

Conformity Assessment Considerations for Federal Agencies (SP 2000-02)

Page 2

around agencies leveraging private sector conformity assessment expertise and activities where 

appropriate to meet the agency objectives/needs and otherwise consistent with law.

2. While OMB A-119 is method-neutral when it comes to conformity assessment, the NIST 

Considerations document fails to take a neutral approach in describing methods of 

conformity. There are a number of examples peppered throughout the document that use 

prejudicial language to paint first party conformity assessment (i.e., supplier’s declaration of 

conformity, SDoC) favorably while challenging the potential costs and efficiency of more “robust” 

and “independent” methods such as third-party. This bias also runs counter to the conformity 

assessment neutral position of the Office of the US Trade Representative in the negotiation of 

trade agreements, and specifically that of the horizontal Technical Barriers to Trade chapter.

NIST does federal agencies a disservice by its failure to describe various methods of conformity 

assessment neutrally. Instead of using language like “trade-friendly” to describe SDoC (thereby 

implying that third-party should always be the last resort), NIST should instead offer federal 

agencies a set of questions they can leverage that would lead them into their own determination 

as to what the appropriate level of conformity assessment is needed to meet the objective at 

hand, manage risks, and deliver confidence needed. UL recommends NIST consider and 

incorporate the set of questions presented to them by the International Federation of Inspection 

Agencies (IFIA), outlined below:

 Is a high level of confidence required?

 Is the perceived risk high?

 Is there a documented history of (industry) compliance?

 Is there a documented history of (industry) non-compliance?

 Do regulatory authorizing/statutory provisions provide severe penalties and an effective 

deterrent?

 How strong is the need for impartiality?

 Are there voluntary, market-driven schemes that address confidence needs?

 Are there relied upon international schemes that can be leveraged?

 Specific to products:

o Are products regulated primarily manufactured in countries with a history of risk 

factors and other issues?

o Are products manufactured in complex and fragmented supply chains?

o Is there evidence that product liability is an effective deterrent?

o What are the societal risks of non-compliant products?

o Who bears the cost of market surveillance?

o How likely is the need for recall or correction action?

o How effective is the model in supporting anti-counterfeiting enforcement?

3. The Considerations document confuses rather than clarifies the distinctions between 

conformity schemes/programs and bodies and the role of each in conformity assessment.

For example, lines 980 through 986 present a combination of factors related to conformity 

assessment bodies and conformity assessment schemes. The factors related to bodies are 

competence of personnel and adequacy of facilities and equipment. The factors related to 

schemes are adequacy of the product standard, the number and type of testing and inspection 

methods used, size of the sample and types of sampling methods, use of quality management 

system requirements for producers of the object undergoing conformity assessment, and the 

nature/extent of surveillance activities. Also, in lines 1024-1027 surveillance is implied to be a 

choice of the conformity assessment body when surveillance is one of the key elements of the 

scheme.
National Institute of Standards and Technology (NIST)

Conformity Assessment Considerations for Federal Agencies (SP 2000-02)

Page 3

4. The role of surveillance in conformity assessment is consistently mischaracterized in the 

draft Considerations. One such example is found on lines 545-546, where NIST describes the 

role of surveillance as “provid[ing] confidence in ongoing conformity once initial conformity has 

been determined” when, in reality, it would be more accurate to state that the goal of surveillance 

is to “maintain the validity of the attestation.” When coupled with the misconception perpetuated 

in the ABCs document that conformity assessment ensures that specific products, processes, 

services, systems, persons, or bodies always fulfill requirements, federal agencies could be led to 

wrongly believe that conformity assessment guarantees compliance when, in actuality, it provides 

assurance by creating an incentive to fulfill requirements – and to do so on an ongoing basis.

5. The document confuses rather than clarifies the distinction between management 

systems/management system requirements and requirements in ISO/IEC Standards for 

conformity assessment bodies (ISO/CASCO Standards). In lines 1277 through 1286 the 

Considerations document erroneously describes ISO CASCO standards as setting management 

system requirements. ISO CASCO standards set requirements for the competence, consistency, 

and impartiality of conformity assessment bodies. These standards include one section of 

requirements labeled “Management System Requirements.” The CASCO standards set these 

requirements to ensure the conformity assessment body establishes a management system to 

self-assess and assure the ongoing fulfillment of requirements in the rest of the standard. To 

imply that ISO CASCO standards are management system standards or are wholly management 

system requirements is a serious misstatement.

UL affirms NIST’s desire to provide a better understanding of conformity assessment to US federal 

agencies looking to it for guidance. While well-intentioned, the current Considerations falls short of 

achieving the clarity necessary to provide optimal value to federal agencies. Absent NIST 

reconsidering some elements of the approach it has taken, UL offers specific recommendations in the 

attached Appendix to help resolve inaccuracies, address some of the aforementioned areas, and 

bring greater clarity and alignment with internationally accepted conformity assessment concepts. In 

addition, UL recommends that NIST review resources developed by the private sector (such as UL’s 

series of courses on “Conformity Assessment Essentials”) for consideration as potential training tools 

for federal agencies. UL would be happy to discuss access to our materials with you.

Again, thank you for giving the conformity assessment stakeholder community and the general public 

the opportunity to review and comment on the Considerations and the companion ABCs. UL looks 

forward to continuing to collaborate with NIST to help support federal agencies considering conformity 

assessment programs. Please do not hesitate to contact me if you have any questions regarding 

these comments.

Sincerely,

Ann M. Weeks

Vice President, Global Government Affairs

UL LLC

1850 M Street NW, Suite 1000

Washington, DC 20036

Email: Ann.Weeks@ul.com

Phone: +1 202.296.1435
National Institute of Standards and Technology (NIST)

Conformity Assessment Considerations for Federal Agencies (SP 2000-02)

Page 4

Appendix - Specific Recommendations

1. Introduction

Federal Policy Context of this Document:

Recommendation #1: With respect to lines 146-148, NIST should consider framing this policy by 

using the language taken directly from OMB A-119 (2016) as follows:

Agencies should also design conformity assessment programs with the objectives of further outcomes 

that are closely aligned with market dynamics and otherwise maximize net benefits to society. In this 

context, agencies should recognize the possible contribution of private sector conformity assessment 

activities. When properly conducted, conformity assessments conducted by private sector conformity 

assessment bodies can increase productivity and efficiency in government and industry, expand 

opportunities for international trade, conserve resources, improve health and safety, and protect the 

environment.

 Rationale: In describing the policies for Federal agencies outlined in OMB Circular A-119 in 

lines 138-155, NIST cites most of the OMB A-119 language verbatim, with the exception of 

the third bullet where it extracts the first sentence on “market dynamics” and “net benefits to 

society” without offering the subsequent context provided in OMB A-119. Such context is 

important because it underscores the importance of leveraging the private sector, which is 

the very spirit of the NTTAA and OMB A-119.

2. Elements of a Conformity Assessment Program

Objectives and Goals:

Recommendation #2: Amend lines 413-418 to read as follows (Text recommended to be deleted is 

indicated by strikethrough; text recommended to be added is indicated by underline):

The basic principle of conformity assssessment10, see Figure 1, is that an object of conformity

(i.e., product, service organization, process, or person, etc.) is evaluated to determine if

demonstrate it meets requirements. The determination is based on, which creates evidence 

of conformity (e.g., a test report, inspection report, audit report). An organization may attest (a 

statement to convey assurance) to this conformity that fulfillment has been demonstrated

based on the determination evidence of conformity and the activities used to create it. 

Surveillance activities support continued validity of the attestation for on-going conformity of 

the product, service, organization process, or person assurance of fulfillment of requirements.

 Rationale: These proposed edits more accurately define the principle of conformity by using 

internationally accepted conformity assessment language. In addition, these modifications 

would help dispel any misconception that conformity assessment ensures or guarantees 

compliance instead of a demonstration that requirements have been fulfilled.

Recommendation #3: Amend the language in Figure 1 (The Conformity Assessment Process) as 

follows:

1. In the “Attestation” box, modify the language to read “Who says performance has been 

demonstrated?

2. In the “Surveillance” box, modify the language to read “What about assurances next 

week?”
National Institute of Standards and Technology (NIST)

Conformity Assessment Considerations for Federal Agencies (SP 2000-02)

Page 5

 Rationale: UL recommends these edits to provide greater alignment between the NIST 

Considerations document and conformity assessment language accepted internationally (via 

ISO definitions/terminology).

Recommendation #4: On line 441, strike the phrase “in the conformity” and replace with “as a basis 

for”.

 Rationale: Test results are used as a basis for attestation; this edit simply underscores this.

Recommendation #5: On line 474, after “conformity assessment” insert the word “activity”.

 Rationale: This edit would clarify that inspection is a specific activity of conformity 

assessment, rather than encompassing the entirety of conformity assessment.

Recommendation #6: On lines 478 and 479, modify the sentence to read “Many inspection programs 

are the basis for product markings such as the US Department of Agriculture meat grades or 

certificates which attest to the conformity of inspected products.

 Rationale: Inspection, by definition in ISO/IEC 17000 Annex A, does not include issuance of 

an attestation. Issuance of an attestation is a required element of conformity assessment, but 

is differentiated as a separate activity from inspection.

Audit:

Recommendation #7: On line 486, before “management systems” insert the phrase “for example”.

 Rationale: Audit activity does not apply solely to management systems, so it is more 

appropriate to use management systems as an example than imply audit activity is limited to 

such systems.

Recommendation #8: Consider deleting lines 488 to 501.

 Rationale: A description of auditing as a conformity assessment activity is confusing when it 

includes such detailed information about management systems. Management systems are 

only one type of object that can be effectively evaluated by auditing. For example, financial 

auditing can be the basis for an attestation that an accounting system meets specified 

requirements. If detailed information about management systems is needed in the 

Considerations document, then it should be placed in some other location and not combined 

with auditing.

Attestation:

Recommendation #9: On lines 504-505, after the word “consider” insert “fulfillment of” and replace the 

word “met” with “demonstrated”.

 Rationale: Use of the terminology around “fulfillment” and “demonstration” is consistent with 

ISO principles and internationally accepted and used terminology.

Recommendation #10: On line 507, strike the word “typically”.
National Institute of Standards and Technology (NIST)

Conformity Assessment Considerations for Federal Agencies (SP 2000-02)

Page 6

 Rationale: The word “typically” is not needed as an attestation is based on a review of 

conformity assessment activities.

Recommendation #11: On line 508, after the phrase “verification of” insert the following: “the 

suitability, adequacy, and effectiveness of”.

 Rationale: The term “review” is specifically defined in ISO/IEC 17000 and that definition 

should be utilized.

Certification:

Recommendation #12: On line 524 strike the word “organization” and replace it with the word 

“system”.

 Rationale: This edit is intended to bring additional consistency to how conformity assessment 

is characterized throughout the document and to greater align the NIST Considerations with 

international (ISO) principles and terminology.

Recommendation #13: On line 537, strike the word “conducted” and replace it with the word “issued”.

Also delete “and includes some form of surveillance activity”.

 Rationale: Per ISO/IEC 17000, certification is a type of attestation – not a conformity 

assessment process. As a result, a certification is issued, not conducted. A certification 

scheme (a conformity assessment scheme in which the attestation is certification) can 

include surveillance activities. Failure to maintain the accuracy of these concepts will only

lead to confusion when federal agencies (or others) seek to establish conformity assessment 

in practice.

Surveillance:

Recommendation #14: On line 544, strike the word “conformity” and replace it with “assurance”.

 Rationale: Making this suggested change would help dispel any misconceptions users may 

have that conformity assessment provides a guarantee of compliance rather than an 

assurance that specified requirements have been fulfilled. 

Recommendation #15: On lines 545-546, strike the phrase “provide confidence in ongoing conformity 

once initial conformity has been determined” and replace it with the phrase “maintain the validity of 

the attestation”.

 Rationale: This change is recommended to more accurately describe the function and value 

of surveillance and to ensure such activities are properly characterized in the document, per 

the point (#4) UL raises in its “General Observations.” 

Recommendation #16: On lines 563-568, the description of first, second, and third parties mistakenly 

tracks with the parties to a transaction for the object of conformity. These lines should be rewritten to 

reflect first, second and third parties have a broader meaning to conformity assessment:

 First party: individual or group supplying the object of conformity or having the same interests 

in the object of conformity (e.g., investors, business partners of the supplier such as contract 

manufacturers, advertisers, etc.); 
National Institute of Standards and Technology (NIST)

Conformity Assessment Considerations for Federal Agencies (SP 2000-02)

Page 7

 Second party: individual or group with a need for confidence or assurance that specified 

requirements are fulfilled (This is not just the purchaser – it can include underwriters of 

insurance for the purchaser, distributors, and retailers who deliver the object to the end 

purchaser, and regulators who seek to protect the interests of the purchaser or end user of 

the object); and

 Third party: individual or group whose interests are independent of first and second parties 

(using this definition of third parties, the concept of independence does not need to be 

addressed again – rather, the more critical requirements for impartiality can be the focus).

 Rationale: Effective concepts of first, second, and third parties that can capture all individuals 

and groups related to a conformity assessment are much more powerful in helping federal 

agencies consider conformity assessment.

Listing:

Recommendation #17: On line 583, strike the words “is not in itself” and replace with the words “may 

or may not be” before the word “attestation”.

 Rationale: It is possible for a listing to be an actual attestation. It is also possible that a listing 

is a pointer to the actual attestation, and in this case a series of listings is more commonly 

thought of as a Directory.

Requirements and Specifications

D. Determine Confidence Point

Recommendation #18: The term “Confidence point” is a poor choice in lines 773 through 823. What is 

being described is optimal conformity assessment – it delivers needed confidence considering the 

consequences of noncompliance and the cost of conformity assessment. Discussion of risk should be 

part of a discussion of the consequences of noncompliance, not separate from it. In lines 813-815 

additional factors to consider are erroneously labeled as part of risk considerations. Rather, these are 

factors that contribute to the confidence/assurance the conformity assessment delivers.

 Rationale: Confusing and mixed-up concepts will make consideration of conformity 

assessment more difficult for federal agencies.

Conformity Assessment Roles, Activities and Policies

A. Define the Conformity Assessment Model

Recommendation #19: Strike lines 879-891 in their entirety.

 Rationale: NIST should remove references to “independence” in their discussion of “steps 

and factors in arriving at a conformity assessment model”. Models are not independent – only 

bodies are. The preferred descriptions UL has offered of first, second, and third parties under 

Recommendation #16 give the concepts needed to discuss the relative merits of conformity 

assessment schemes performed by first, second, and third parties. The concept of 

“independence” is effectively captured in these descriptions and should not be referenced 

again here or elsewhere (e.g., line 901). UL believes that the key question (i.e. “step” or 

“factor”) in arriving at a conformity assessment model is to determine whether a first party, 

second party, or third party performs the conformity assessment activities.
National Institute of Standards and Technology (NIST)

Conformity Assessment Considerations for Federal Agencies (SP 2000-02)

Page 8

Recommendation #20: On line 903, after the sentence ending “objectives” add the following 

sentence: “Similarly, the use of a less robust model does not always lead to a program that is 

optimally effective and/or efficient for achieving conformity assessment program goals and meeting 

broader objectives.”

 Rationale: As UL noted in its “General Observations,” one of the significant flaws with the 

Considerations document as drafted is its failure to achieve neutrality with respect to the 

methods of conformity assessment described. The sentence in lines 901-903 unfairly calls 

out use of “the most independent” (problematic in itself per the rationale provided in 

Recommendation #19) and “most robust model” (often third-party) as not always leading to a 

program that is effective or efficient. UL argues that same argument could be applied to less 

robust models. In reality, different methods exist and are applied based on what is needed to 

manage risks and reach the level of confidence required to meet the need or objective at 

hand. The method – irrespective of level of robustness – is only effective and efficient when it 

delivers the assurance required in an appropriate and timely manner.

Recommendation #21: On lines 932-934, delete “and provide the market with acceptable confidence”.

 Rationale: There is no role for the federal government to weigh in on market needs for 

confidence. Federal government regulators need only look after their own confidence needs. 

Hopefully, existing market-related conformity assessment can also meet regulator confidence 

needs, which makes existing conformity assessment more valuable and makes the overall 

market more efficient. However, the regulator should not get involved in the market’s 

confidence needs – for example, the need of a distributor to be confident of the fulfillment of 

requirements for the product he/she is distributing. Overburdening regulators with roles and 

concerns that do not belong to them is a hindrance to their consideration of conformity 

assessment.

Recommendation #22: On line 960, strike the sentence “Reliance on an SDoC is considered to be a 

trade-friendly approach to conformity”.

 Rationale: The use of this sentence by NIST, a federal government agency, prejudices 

potential readers and users of the Considerations to default to SDoC when, in many 

instances, it may be more appropriate to consider or apply a different method of conformity 

assessment. The use of SDoC, second party, or third party is dependent upon what 

regulators determine is needed and/or what markets demand by way of assurance. When 

SDoC is enough to satisfy those needs, there is not a market for third-party to provide 

services; conversely, where greater assurance is necessary, there is little pertinence for 

SDoC. The language in the current draft gives the perception that NIST is passing judgment 

the various methods of conformity assessment rather than arming federal agencies with a set 

of tools or questions to arrive at that determination on their own, with the consultation of the 

US Trade Representative. Additionally, the subsequent sentence (“From a manufacturer’s 

perspective...”) is adequate to communicate why industry may prefer this method over 

others. 

Recommendation #23: On lines 1083-1096, suggest NIST replaces these lines with an actual 

example of surveillance.

 Rationale: What is being described is a regulatory inspection activity, not surveillance to 

assure the ongoing validity of an attestation. While certainly a very valuable activity that 
National Institute of Standards and Technology (NIST)

Conformity Assessment Considerations for Federal Agencies (SP 2000-02)

Page 9

utilizes an activity that is also utilized in conformity assessment (inspection), it likely falls 

outside the field of conformity assessment since there is no attestation involved.

Recommendation #24: On lines 1277-1291 the document indicates the program owner should 

develop competence requirements. However, ISO CASCO standards generally require the conformity 

assessment body to develop specific competence requirements for their staff. The text should be 

amended to more appropriately reflect that.

 Rationale: Giving this message to federal regulators will confuse them when they then read 

ISO CASCO standards for conformity assessment bodies. Consideration should be given 

proper characterization of the ISO CASCO standards, per point #5 made in UL’s “General 

Observations” section.

Recommendation #25: Lines 1320-1323 should be rewritten to more closely align with ISO CASCO 

requirements.

Rationale: As currently written, the text indicates that accreditation bodies use specific 

competence requirements in addition to ISO CASCO standards. This is generally untrue. ISO 

CASCO requirements actually put the obligation on the conformity assessment body to set 

specific competence requirements. In the most recent revision of ISO/IEC 17011 for 

accreditation bodies, ISO CASCO clearly indicated that accreditation bodies attest to 

fulfillment of requirements including competence requirements – competence is not attested 

outside of fulfillment of requirements. Further, the text indicates accreditation bodies attest if 

appropriate. This is incorrect since accreditation is a type of attestation – it is not possible to 

accredit without attesting. Inconsistencies between this document and the ISO CASCO 

standards will hinder federal agencies’ consideration of conformity assessment.

If you have any questions regarding these recommendations, please contact Sarah Owen, Global 

Government Affairs Manager, at sarah.owen@ul.com or +1 202.530.6163.