Revision of Guidelines on Assessment of AI Reliability in the Field of Plant Safety
The Guideline provides the methodology to fulfill AI’s desired quality in terms of safety and productivity
enhancement, in order to promote the usage of AI in the field of plants where safety is considered vital.

The Guideline has seen wider adoption since its publication in November 2020. With these adoptions in consideration,
real-life examples of guideline adoption are hereby provided, as well as a revision of the Guideline to promote its
appropriate adoption.

KEY POINTS OF REVISION
1. CLARIFIED the relationship between AI safety
evaluation and plant safety evaluation based on
traditional risk evaluation methods e.g. HAZOP,FMEA※
2. ADDED explanatory comments to the Format for
Recording Actions Taken, enhancing its usability
3. PROVIDED Q&A comments regarding common
questions and uncertainties in adopting the Guideline
4. PROVIDED practical examples of the Template for
Reliability Records, based on actual reliability
assessment pursuant to the Guideline(Seven examples in total, covering all five use cases)

Version 1

Expanded via revision

Chapter 2Reliability Assessment
Structure

Relationship with
HAZOP、FMEA etc.
CLARIFIED

Chapter 3
Use Cases
Chapter 4
Flow of Utilizing
the Guidelines
Template for
Reliability Records

Additional explanation
PROVIDED

Comments ADDEDNEW CONTENTS“Q&A,” “PracticalExamples”（※）IEC 61882:2016 Hazard and operability studies (HAZOP studies) - Application guide; IEC 60812:2018 Failure modes and effects analysis (FMEA and FMECA)0cf. Structure of the Guidelines on Assessment of AI Reliability in the Field of Plant Safety
The Guideline provides quality management methods using the “three types of quality” in terms of reliability evaluation,
as well as “five uses cases” regarding plants.
This allows plant owners and AI vendors to resolve the challenges regarding AI reliability evaluation.
– Plant owners can: explain the AI’s reliability within and without the company; set adequate requirements for vendors.
– AI vendors can: explain their AI’s reliability to plant owners; share plant owners’ vision regarding the requirements.
Three types of qualities, evaluation method, and requirements of reliability evaluation

Chapter 2

Structure of reliabilityevaluation*• Three types of qualities ofreliability evaluation• Method and requirements for reliability evaluation

Quality in use

1. Setting the quality in use

External quality

2. Setting the external quality

Internal quality

4. Confirming the level ofinternal quality

Flow of utilizing theGuidelines

3. Setting the level of external quality

Specific examples ofutilizing the Guidelines

5. Confirming and implementing
requirements for internal quality* Based on the structure of the 1st edition of Machine Learning Quality Management Guideline by AIST
Chapter 3• Use cases covered

Use cases in the field ofplant safety

1.
2.
3.
4.

Prediction of pipeline thickness
Diagnosis of pipeline imagery
Diagnosis of facilities deterioration
Detection and diagnosis of earlysigns of abnormality
5. Operational optimization

Chapter 4• Specific application of reliability evaluation based on use cases
 Relationship between the AI and other systems⁃ “External safety mechanism” to monitor and correct the output ofthe AI⁃ Independent system to ensure safety separately from the input,processing, and output of the AI⁃ Human involvement in the judgement and reactions of the AI Examples of quality settings (incl. Levels of safety to be expectedfrom the AI)

Real-world examples of utilizing AI in the field of plant safety• Utilizing entities/people incharge• People in charge indevelopment and operation
phases• Actions to take indevelopment and operation
phases1