International Symposium: Risk–based Management of Energy Infrastructure Tohoku Forum for Creativity
Session 2: Advanced Maintenance

Development and evaluation of an anomaly detection systemusing System Invariant Analysis Technology(SIAT) for sound data

Tomoya SOMA
NEC Digital Technology Development Laboratories
Akiko SASAKI (Presenter)
NEC Corporation, AI Analytics Department© NEC Corporation 2023

Table of contents
01

Challenges faced in introducing AI

02

Overview of System Invariant Analysis Technology(SIAT)

03

Case study- Thermal power generation boiler monitoring- Inspection of Spacecraft “Orion”

04
05

Abnormality detection by soundSensing Devices- Acoustic Analysis Gateway- Optical Fiber sensing

Section013© NEC Corporation 2023

Challenges
faced in introducing AI

Challenges in utilizing AI at equipment maintenance sites
Growing interest in AI, but manufacturing companies are facing challenges torealize utilization of AI on-site
Voices often heard in field

User can not handle AI Lack of human resources whohandles AI
 No data scientist

Accuracy does not improve Take time to maintain accuracyof AI

Incomprehensible judgment basis4© NEC Corporation 2023

 User can’t understand thereason why AI judged anomalyImportant points in introducing AI
Point is that AI efforts can be maximized by focusing on the fieldIs it highly accurate?
Quickly and accurately find state changes from large
amounts of complex data

Is the result easy to understand?
Provide users with the basis of anomaly judgement
Highly explainable AI

Can user operate AI?
Easy to use, user can take initiative in tuning5© NEC Corporation 2023

Benefits Current On-site data can be handled Statistics and IT expertise is notrequired No need to change your organization
or operationsSection

02

Overview of SIAT

SIAT: System Invariant Analysis Technology

6© NEC Corporation 2023

Overview of System Invariant Analysis Technology(SIAT)
This technology focuses on the relationships between sensors and detects abnormal
behaviors very early and with high accuracy
Modeling
Automatic relationship calculation
and relational expression creation

Period representing the usual situation

Normal state model

Anomaly detection
Detect signs of anomalies at an early stage bycomprehensively looking at all relationships
Deviation from “usual”
between A & B

Residual threshold(automatically set)

ResidualMonitoring the anomalies ofrelationships
ReSensor AA

7TimeAB

10:00
10:01
10:02
10:03

1.0
2.0
3.0
2.0

2.0
4.0
6.0
4.0

© NEC Corporation 2023

Expected value from relational expression Actual value

BSensor B

B(t)=2×A(t)Relational expressions(automatically created)

Overview of System Invariant Analysis Technology(SIAT)
This technology focuses on the relationships between sensors and detects abnormal
behaviors very early and with high accuracy
Modeling
Automatic relationship calculation
and relational expression creation

Anomaly detection
Detect signs of anomalies at an early stage bycomprehensively looking at all relationships

Easy-to-understand in manufacturing&statemaintenancesites
Normalmodel

Deviation from “usual”
between A & B

Not require data scientists
ResidualA

User interface for use for on-site
TimeA

B

10:00

1.0

2.0

10:02
10:03

3.0
2.0

6.0
4.0

B

© NEC Corporation 2023

Monitoring the anomalies ofrelationships
ReSensor A
Actual value

Rich10:01
client tooland
Python interface for analysts
2.0
4.0
8

Residual threshold(automatically set)

B(t)=2×A(t)Relational expressions(automatically created)

Sensor B

Expected value from relational expression

SIAT in Manufacturing and Plants
Various use cases such as Anomaly detection, Predictive maintenance, and Quality
deterioration are increasing in areas with high economic and social impact

Socialinfrastructure(Bridge, highway etc.)

Power plants

IT system
data centers,
telecom networks

9© NEC Corporation 2023

Cars, trains,
aircrafts, Space craft,etc.

Manufacturing plants(Chemical, assemblyplants etc.)

Sensordata

safetyefficiency
OperationlogsSection

03

Case study

1. Thermal power generation boiler monitoring
2. Inspection of Spacecraft “Orion”

10© NEC Corporation 2023

1. Thermal power generation boiler monitoring
Visualization of state changes (modulation of metal temperature due to clinker generation, etc.) in theboiler furnace over time and from the viewpoint of the part
Results
SIAT
Anomaly score graph(Transition of invariantdestruction)

Anomaly case
Accumulation of clinker inthe furnace gets worse

Normal caseNormally, the site returns tooriginal state when theclinker peels off11© NEC Corporation 2023

1 month later

2 month later Cleanup due to clinker ashaccumulation, panel
blockage, etc. in areas
where the degree ofanomaly score is high The state of the site thatcannot be confirmed fromthe outside can bevisualized by the collapseof the invariant

Section

03

Case study

1. Thermal power generation boiler monitoring
2. Inspection of Spacecraft “Orion”

12© NEC Corporation 2023

Thermal vacuum test of spacecraft "Orion"
1. Detects small abnormalities that are different from the model in the normal state, detects the possibility of failuresuch as abnormalities and events at an early stage.
2. Identify the root cause of the failure by modeling the entire system
3. By modeling the status monitoring data history, the entire system can be grasped even in a complicated system.
Contributing to strengthening support for various missions and speeding up aircraft manufacturingCollection and accumulation oftime series data
Accumulate time-series data collected fromapproximately 150,000 sensors in all subsystem
in a thermal vacuum test

Modeling the normal state

Early detection of possible failure

Automatically build over 22 billion relationships

Using the model under normal conditions,

between sensors to model the normal behavior

compare with the results of the subsequentof a spacecraft

thermal vacuum test. Make sure all subsystems
are working properly

13© NEC Corporation 2023

Section0414© NEC Corporation 2023

Abnormality detection bysound3.1 Common challenges of Anomaly detection using sound data
1. Processing Back Noise Background noise must be reduced in advance by various methods.
 Various companies use various methods, but if the frequency band of the abnormal sound overlaps with the frequency band of the noise,
the abnormal sound may be muffled.

2. Sensing (sound collection)
 Specialized microphones are likely to be used in many cases, tend to be expensive.
 Installation and wiring of recording equipment is difficult
 When centralized monitoring is used, there is a high possibility of issues with the data collection network.

3. Abnormality detection by AI Most likely to use neural network technology such as deep learning after extracting features
 Judgment is limited to OK/NG

4. Response after abnormality detection
 Current solutions can only detect abnormalities and cannot analyze the cause. If the characteristics of the sound that caused theabnormality are not known, countermeasures are likely to be time-consuming.
 Difficult to identify similar anomalies when they occur
15© NEC Corporation 2023

3.2 Capability of SIAT Anomaly detection using sound data
1. Processing Back Noise Learning is possible with noise included Noise is also defined as the "usual state“

2. Sensing (sound collection)
 General microphone installation only Easy to get started with inexpensive equipment
 No need to modify existing equipment

3. Abnormality detection by AI
 It is possible to detect which frequency characteristics are abnormal

4. Response after abnormality detection
 Similar events can be determined from the frequency characteristics of anomalies and trends

16© NEC Corporation 2023

3.3 Model Creation Steps
1.
2.

Decompose PCM recorded wav data into frequencies using FFT
Automatically extract invariant relationships between frequencies using SIAT

1. WAV data

2. CSV data

Frequency decomposition by FFT

Model the relationship between frequencies

Assumes that focusing on the invariant relationship between frequencies creates ananomaly detection model that is robust to ambient noise
17© NEC Corporation 2023

3.4 Anomaly detection
26th May

27th28th(1) Model in one hour (16:45-17:45) on May 26

Model creation period (26th May 16:45-17:45)

Analysis period (27th – 28th)

Bursting sound (28th 17:34–18:27)

Normal state (27th May 12:02-12:11)

Just before the abnormality (28th May 17:25-17:34)
Mechanicalsound changes

18© NEC Corporation 2023

(2) Check the state of the model in the normal period.
• Anomaly score is stable near 0• It is thought that the model learns including various surroundingnoises.

(3) Verify whether anomaly detection is possible using anomaly periods• Abnormality score is clearly different from the normal period• It is thought that the abnormal noise is detected at a level thatcannot be detected by humans.

3.5 Anomaly detection
Machinestarting soundAbnormal state (28th May 17:35-17:44)
Bursting sound(4) Abnormalities• Abnormality occurs in the red frame area.
• Subsequent anomaly score remains high and transitions(5) Failed inverter is shut down
• Abnormality score becomes low and remains stable

The system was able to accurately capture abnormal noises even in areas
with high ambient noise.
19© NEC Corporation 2023

3.6 Usage scenarios & Needs for Anomaly detection by soundInspection (Good/Fail)
Mechanization of sensory inspections that rely on veterans

Monitoring (detection of predictive failure)
Real-time monitoring• Standardization of inspections• Constant noise monitoring• Shortening of inspector training period• Detection of signs of failure of rotating machines• Explanation of results based on evidence• Detection of abnormalities in power supply equipment• Prevention of missed inspection• Reduction of the burden of inspection patrols

Abnormality detection during manufacturing• Optimization of tool replacement time
• Abnormality detection of materials

Watching over/preventing crime
Privacy-conscious monitoring

Medical assistance in the medical field• Detection of anomalies beyond visual range• Reduce the workload of laboratory technicians• Realization of monitoring that cannot be done with images• Identification of conditions by voice and sound• Identification of sounds of objects breaking/screams

20

Medical field© NEC Corporation 2023

Section05Sensing Devices

1. Acoustic Analysis Gateway
2. Optical Fiber sensing

21© NEC Corporation 2023

Acoustic Analysis Gateway for Equipment Operation monitoring
Acoustic Analysis Gateway is hardware that records, analyzes, and transfers sound fromvarious devices over a wide band.
Acoustic data is gathers by a dedicated waterproof
microphone, and frequency is decomposed by FFTand transferred to a host computer (server, cloud,etc.). Compact and easy to install.
Specialized microphones that detect sounds in a
wider than the audible range allow you to record
sounds around your equipment or attach them
directly to the equipment for direct recording of thevibrations of them.
By decomposing broadband sounds into eachfrequency band for analysis, the operating status ofcustomers' equipment can be grasped in detail.
22© NEC Corporation 2023

Section

05

Sensing Devices

1. Acoustic Analysis Gateway
2. Optical Fiber sensing

23© NEC Corporation 2023

Features Fiber Sensing
Vibration-centric sensing technology using optical fiber High-density (several meters granularity) vibration/temperature sensing over awide area (tens of kilometers)
 High environmental tolerance without being affected by electromagnetic noise Does not require communication equipment or lines to transmit observation data Does not require power supply to the sensing point
Low power consumption and easy maintenance Can be used in explosion-proof/high radiated areas(assuming that the device is kept out of the controlled area)
 Heat-resistant by utilizing optical fiber(may be usable at 700°C)
 Requires know-how on how to install the fiber24© NEC Corporation 2023

NEC vs Conventional Methodology
NEC (DAS)

Conventional (FBG)

Measuring Method

 Capable of multipoint measurement withordinary unprocessed optical fiber (SM) The number of points varies depending on thesensing frequency, but 5,000 points can bemeasured at a 1kHz sampling rate The frequency and number of points can bechanged by software Processed fiber allows multi-point measurement up to
maximum of 5 to 6 points

Measuring Accuracy± a few nm (theoretically ± a few pm)± a few nm (limited to distortions)

Transmission Range

~80km

~30km

Flexibility of Sensing
Points

Flexibility in changing measurement points due tosoftware-assisted repositioning

No flexibility due to the need to pre-fabricate themeasurement points into fiber

Cost Equivalent or cheaper equipment compared toFBG
 Cheaper overall due to the use of generalpurpose fiber

Expensive
Due to difficulty to process fiber, the cost increases

25© NEC Corporation 2023

Applications (Examples in Industrial Usages)
 Intrusion detection for critical facilities Condition monitoring by sound/abnormal sound detection
 Human voice and abnormal sound detection and condition identification (AI application)

 Continuous monitoring of infrastructure facilities (bridges, buildings, and otherstructures) Crack detection in high-pressure/high-temperature piping
 High-pressure steam piping in power plants
 Continuous inspection of piping (especially in explosion-proof areas) in chemical plants

 Vibration measurement in explosion-proof areas
 Use as low-cost vibration sensor

 Detection of cracks and bulges in CFRP tanks
 Unattended monitoring of hydrogen stations

26© NEC Corporation 2023

High judgment accuracy

Easy to understand basis of judgment

Easy to use on manufacturing site

27

© NEC Corporation 2023