October 15, 2020 

National Institute of Standards and Technology 

100 Bureau Drive 

Gaithersburg, MD 20899 

Re: Draft NISTIR 8312: Four Principles of Explainable Artificial Intelligence 

Dear NIST, 

BlackBerry commends NIST’s leadership and transparent, consensus-based process to set “a 

plan for Federal engagement in developing technical standards and related tools” to respond to 

the Executive Order (EO 13859), “Maintaining American Leadership in Artificial Intelligence”. 

We hope these efforts lead to better understanding of the theoretical capabilities, limitations, and 

risks of AI, and will advance the foundational research seeking to build trust in AI systems by 

improving their accuracy, reliability, security, robustness, and explainability. 

We are encouraged by the recent AI workshop Series and appreciate the opportunity to comment 

on the publication of draft NISTIR 8312 “Four Principles of Explainable Artificial Intelligence”. 

BlackBerry supports NIST’s foundational AI research to cultivate trust in the design, 

development, and use of AI. Please find our Comments to NISTIR 8312 in the attached 

Excel file. Below is an overview of our response. 

General Comments 

We agree with two foundational positions expressed in the NISTR 8312 draft, namely: that 

explainability is one of several properties that enable trust in AI systems, and that approaches 

taken to explainable AI will be improved by considering viewpoints, methods, and lessons from 

multiple disciplines, including computer science, engineering, and psychology. 

The First Principle: “Explanation” 

The first principle is described in terms of obligation (Line 173). We respectfully express our 

concern that this description is too constrictive to be applied for all AI systems across the board. 

Not all the AI systems are required to produce explanation. Further investigation is required to 

advance understanding of when and to whom an explanation needs to be provided for what 

purpose. We propose updating the description in terms of how the first principle can be satisfied. 

BlackBerry Corporation

3001 Bishop Drive, Suite 400, San Ramon, California, 94583 USA. tel: +1 (925) 242-5660 fax: +1 (925) 242-5661 

Trademarks, including but not limited to BLACKBERRY, EMBLEM Design, BBM and BES are the trademarks or registered 

trademarks of BlackBerry Limited, used under license, and the exclusive rights to such trademarks are expressly reserved. 

1 
 

 

Types of Explanations 

We generally agree that there are multiple types and purposes of explanations. There is trade-off 

between explanation detail and time constraints. However, further work is required to 

characterize explanation, meaningfulness and explanation accuracy for types or purposes of 

explanations. Referring to the use case of movie recommendations, we note that such 

explanations are not only for the owner’s benefit but also for the user’s. For example, an 

explanation based on the user’s previous choices may increase his or her willingness to accept a 

movie recommendation. 

Next Steps for Progress 

We propose that to facilitate advancement of explainable AI methods necessary for complex, 

real-world systems, the following challenges must receive careful attention and priority 

consideration: 

1. Establishing a common framework and definitions for explainable AI. A common 

taxonomy and terminology is a necessary first step. 

2. Characterizing explanation, meaningfulness and explanation accuracy for types and 

purposes of explanations. 

3. Identifying, cataloging, and establishing a prioritization of limitations and challenges to 

overcome with respect to explainability in systems utilizing AI. 

Conclusion 

BlackBerry encourages NIST’s continued leadership in foundational AI research to build trust in 

AI systems. We welcome the opportunity to comment on NISTIR 8312 and offer Mr. Takashi 

Suzuki tsuzuki@blackberry.com to respond to questions about BlackBerry’s response. 

Respectfully submitted, 

 Takashi Suzuki,

 Senior Director, Standards 

 BlackBerry Corporation 

BlackBerry Corporation

3001 Bishop Drive, Suite 400, San Ramon, California, 94583 USA. tel: +1 (925) 242-5660 fax: +1 (925) 242-5661 

Trademarks, including but not limited to BLACKBERRY, EMBLEM Design, BBM and BES are the trademarks or registered 

trademarks of BlackBerry Limited, used under license, and the exclusive rights to such trademarks are expressly reserved. 

2