Development of Malicious Hand Behaviors Detection Method by Movie Analysis
Authors: Demachi Kazuyuki , Chen Shi 
Published: October 2018

An urgent lesson learned from Fukushima Daiichi accident is what can happen by natural disaster can also occur by human design. The accident raised a fear that terrorists could cause a similar accident by acts of sabotage against nuclear power plant (NPP) and it is noticeable that threats of terrorism for nuclear security are increased after the accident. When considering sabotage, the prime threat to nuclear power plants, due attention should be paid to sabotage by insiders. Generally, insiders are the individuals with authorized access to nuclear facilities in transport who could attempt unauthorized sabotage. They could take advantage of their access authority and knowledge, to bypass dedicated physical protection elements or other provisions [1]. Thus, we should value the catastrophic consequences of the attack or act of insider sabotage which may lead to loss of safety functions of NPP.

International Atomic Energy Agency (IAEA) indicated that the physical protection system (PPS) of a nuclear facility should be integrated and effective against both sabotage and unauthorized removal. The primary PPS functions are deterrence, detection, delay and response. It is noticeable that if detection failed, delay and response would become invalid. Thus, detection of insiders’ sabotage should be enhanced. Considering current countermeasures of PPS to insiders’ sabotage, the most significant challenge is how to distinguish ordinary maintenance behaviors and malicious behaviors since some malicious behaviors may hidden in ordinary maintenance behaviors. It appears that hand behavior has high contribution to human activity and a significant portion of maintenance behaviors and malicious behaviors.

In this study, we proposed a hand behavior detection algorithm for insiders’ malicious behaviors for nuclear security [2]. We focused on the fact that the hand shape is uniquely determined by the fingertip coordinates. First, the depth image of the hand was captured with Kinect v2, and after removing the five fingers were remained by removing the palm and wrist parts, and the five fingers were identified using the K-means clustering [3], and the farthest point of each finger from wrist pixel was taken as the fingertip coordinates. The fingertip coordinates of the five fingers were combined for 60 frames to be time-series data, and this was used as the training data of the neural network. Time-series data obtained from five kinds of behaviors of five hands was used for training data. For the machine learning method, the Stacked-Auto Encoder (SAE) [4–5] which is one of popular methods was used. It extracts the feature of input data at intermediate layer of the first stage. In the second layer, the extracted feature is input and its feature is extracted to be used as the input of the softmax layer for pattern classification.

Meanwhile, a real-time fingertip tracking system was developed and time-series data of each fingertip was successfully obtained with 29.8fps using MATLAB whose CPU was Intel Xeon Processor E5-2630v4 (25M Cache, 2.20 GHz). Moreover, a time-series data analysis based behavior recognition method was developed and all assumed malicious behaviors were detected with high accuracy (82.555% in overall) and speed (0.0023 seconds per frame) in the same computing environment. Also, robustness of the behavior recognition method was verified.