1300 North 17th Street ▪ Suite 900

Arlington, Virginia 22209

Tel: 703.841.3200

Fax: 703.841.3392

www.medicalimaging.org

1

October 14, 2020

P. Jonathon Philips, Et Al.

Information Access Division 

National Institute of Standards and Technology 

44100 Bureau Drive (Mail Stop 8940) 

Gaithersburg, MD 20899

Re: Draft NISTIR 8312, “Four Principles of Explainable Artificial Intelligence”

Dear Mr. Philips,

As the leading trade association representing the manufacturers of medical imaging equipment 

and radiopharmaceuticals, the Medical Imaging & Technology Alliance (MITA) believes in the 

importance of innovative artificial intelligence (AI) applications to empower the healthcare 

sector. MITA Members have decades of experience developing and deploying AI which learns 

and performs specialized tasks for patients. As leaders in the field, we support the National 

Institute of Standards and Technology (NIST) in its pursuit of its goal to clarify explainability for 

AI applications and enable public trust in AI technology. The following comments, including 

those enumerated in Appendix A, are provided in support of that goal.

One difficulty we encountered in the draft document was clarity surrounding the chosen 

terminology for each pillar. “Explanation” as a pillar of explainability was particularly 

challenging. We would recommend NIST consider revising the label of each pillar to more 

accurately reflect the achievement of the characteristics highlighted in each pillar concept: Does 

an explanation exist? Is it meaningful? Is it accurate? Does it contain knowledge limits? For 

example, “Explanation” could be rewritten “Explanation Generated”. Additionally, NIST should 

consider moving “Explanation Accuracy” to be next to the “Explanation” section as they are 

closely related to each other. 

We also suggest that the definition of Knowledge Limits on line 169 be aligned with the 

definition on line 228. The current phrasing on line 169 focuses on what the application can do, 

but since the term is “Knowledge Limits”, we believe the focus should be on what the product 

cannot do. We suggest using the text from 228 in line 169: “The systems identify cases they were 

not designed or approved to operate, or their answers are not reliable.” 

Similarly, the types of explanations iterated in section 3 (line 245) seem to refer to types, or 

kinds, of explanation, but instead give examples of instances where an explanation might be 

necessary. We would recommend that NIST consider renaming the section to clarify that these 
2

are instances of explanation and separate the latter portion of the section (from line 299) to 

discuss explanation detail level.

We found Sections 5 and 6 to be both accurate and informative. The evaluation of machine 

explainability and human explainability provided clear support for what is reasonable to expect 

from machine learning algorithms. We encourage NIST to incorporate the conclusions drawn 

from these sections into the introduction.

Innovative AI applications have the power to significantly improve patient care. This publication

will be an important step to foster innovation. We support NIST in its efforts and look forward to 

continued engagement across the Federal government to ensure the primacy and excellence of AI 

in the United States healthcare sector.

Sincerely,

Patrick Hope

Executive Director, MITA

MITA is the collective voice of medical imaging equipment and radiopharmaceutical 

manufacturers, innovators and product developers. It represents companies whose sales 

comprise more than 90 percent of the global market for medical imaging technology. These 

technologies include: magnetic resonance imaging (MRI), medical X-Ray equipment, computed 

tomography (CT) scanners, ultrasound, nuclear imaging, radiopharmaceuticals, radiation 

therapy equipment, and imaging information systems. Advancements in medical imaging are 

transforming health care through earlier disease detection, less invasive procedures and more 

effective treatments. The industry is extremely important to American healthcare and noted for 

its continual drive for innovation, fast-as-possible product introduction cycles, complex 

technologies, and multifaceted supply chains. Individually and collectively, these attributes 

result in unique concerns as the industry strives toward the goal of providing patients with the 

safest, most advanced medical imaging currently available.
3

Line # COMMENTS Proposed change (if any)

254-258 Explainability for societal 

acceptance is in line with the 

European Commission's definition 

of principle of explainability:

Explicability is crucial for building 

and maintaining users’ trust in AI 

systems. This means that processes 

need to be

transparent, the capabilities and 

purpose of AI systems openly 

communicated, and decisions – to 

the extent

possible – explainable to those 

directly and indirectly affected. 

Without such information, a decision 

cannot be duly

contested. An explanation as to why 

a model has generated a particular 

output or decision (and what 

combination

of input factors contributed to that) 

is not always possible. These cases 

are referred to as ‘black box’ 

algorithms and

require special attention. In those 

circumstances, other explicability 

measures (e.g. traceability, 

auditability and

transparent communication on 

system capabilities) may be required, 

provided that the system as a whole 

respects

fundamental rights. The degree to 

which explicability is needed is 

highly dependent on the context and 

the severity

of the consequences if that output is 

erroneous or otherwise inaccurate." 

It is recommended to capture 

explainability concepts from the 

European Commission's Ethics 

Guidelines for Trustworthy AI

Capture explainability concepts from the 

European Commission's Ethics Guidelines 

for Trustworthy AI
4

160 "output is the result of a query to an 

AI system": The output of and AI 

system is not necessarily as result of 

a query, could be a periodic output 

also, unqueried.

Change sentence to: “The output may the 

result of a query or may be a periodic 

output (unqueried).”