Patent Publication Number: US-2022222402-A1

Title: Information processing device, information processing method, and information processing program

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
     This application is a continuation application of International Application No. PCT/JP2020/037029, filed on Sep. 29, 2020 which claims the benefit of priority of the prior Japanese Patent Application No. 2019-184139, filed on Oct. 4, 2019, the entire contents of which are incorporated herein by reference. 
    
    
     FIELD 
     The present invention relates to an information processing device, an information processing method, and an information processing program. 
     BACKGROUND 
     Deep learning has initially attracted attention by achieving high discrimination performance that is close to human beings or superior to human beings in a field of image processing. At the present time, usefulness of deep learning is confirmed in subjects of dealing with a wide variety of data including typical time series data such as sensor data in addition to moving image processing and voice/language processing. Thus, demonstration experiments have been planned by using data collected from actual facilities in a manufacturing industry including a chemical industry, and have produced certain results in subjects such as quality prediction or anomaly detection in a system. 
     Presently, at the time of introducing a model of machine learning including deep learning into actual facilities, it is often examined whether there is certain validity in a relation between an input and an output learned by the model from a scientific/technological viewpoint, or whether the relation agrees with experience knowledge of human beings that have conventionally observed manufacturing processes thereof, for example. On the other hand, many of models of machine learning attracting attention in recent years represented by deep learning have a complicated structure, and it is difficult for human beings to intuitively understand the relation between an input and an output of the model or processing of the model for an individual input, and the model is often called a “black box”. 
     Thus, interpretation of models of machine learning has been researched from an early age mainly in a field of image processing (for example, refer to Non Patent Documents 1 to 3), and a method in such a field of image processing that is simply expanded has been used for a model of deep learning for dealing with time series data. 
     For example, the technique disclosed in Patent Document 1 uses a sensitivity map for displaying an importance of an input component in a machine learning model used for monitoring a production process. 
     Furthermore, the technique disclosed in Patent Document 1 supports monitoring processing by calculating an importance for individual pieces of input data, the importance indicating that which of characteristic amounts contributes to determination of a model. In addition, the technique disclosed in Non Patent Document 4 smoothes calculated importances by using a method obtained by getting an idea from SmoothGrad as one type of an explanation method in image processing for grasping a temporal change in the importances of respective input sensors in a soft sensor using a model of deep learning.
     Non Patent Document 1: D. Baehrens, T. Schroeter, S. Harmeling, M. Kawanabe, K. Hansen, and K. Muller, “How to Explain Individual Classification Decisions”, Journal of Machine Learning Research 11, 1803-1831. (2010).   Non Patent Document 2: D. Smilkov, N. Thorat, B. Kim, F. Viegas, and M. Wattenberg, “SmoothGrad: removing noise by adding noise”, arXiv preprint, arXiv 1706. 03825. (2017).   Non Patent Document 3: M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic Attribution for Deep Networks”, ICML 2017.   Non Patent Document 4: Kiritoshi, Izumitani (2019). “Time-Smoothgrad: A Method Extracting Time-Varying Attribution for Neural Networks”, JSAI 2019.   Patent Document 1: Japanese Laid-open Patent Publication No. 2019-67139   

     In the techniques disclosed in Patent Document 1 and Non Patent Document 4, an importance of each input component of a model is obtained while considering a one-to-one relation between each input component of the model and each output value of the model. That is, in a case of using sensor data as an input, the importance of each input component of the model is obtained while considering a one-to-one relation between each sensor value and the output value of the model at each time. The model is, for example, an estimation model that has performed learning in advance. 
     In a case of using a gradient of the input as the importance, when only a value of a sensor A ten minutes before the present time slightly varies, a degree of variation in an estimated value of a soft sensor as an output is assumed to be the importance of the value of the sensor A ten minutes before the present time. 
     This assumption is unnatural especially in a case of using time series data as an input for the model. For example, in a real system, when a value that is measured by the sensor A ten minutes before the present time is caused to slightly vary, it can be considered that a value that is measured by the sensor A after that time will also slightly vary. It can be considered that a value that is measured at the same time or after that time by another sensor B will also vary in conjunction with a slight variation in the sensor A. 
     However, in the related art, a correlation and a causal relation among input components of the model are scarcely considered in calculating the importance of each input component of the model. 
     For example, the following exemplifies a case of using, as an input, sensor data of a manufacturing industry such as a chemical industry. A monitor who observes a real system may empirically know that “when a value measured by the sensor A is operated, a value that should be estimated by the soft sensor ten minutes later will vary”, or may grasp a correlative characteristic such that “the value that should be estimated by the soft sensor tends to be linked with a value of the sensor B five minutes before”. 
     On the other hand, in a model of deep learning, each input component is automatically processed, or automatically picked and chosen to be useful for estimation. Due to this, even if the model appropriately grasps an input/output relation of the real system, the monitor cannot immediately recognize that the obtained importance agrees with an experience of the monitor related to the correlation and the causal relation among the input components, so that the monitor needs to examine a relation between the importance and experience knowledge of himself/herself again in many cases. 
     The same problem has arisen in a case of actually introducing such importance display systems into a monitoring system of a production process. For example, in the related art, when high importances are assigned to only part of some input components that vary in conjunction with each other in a real system, the monitor cannot recognize the relation among the input components in some cases. In a case in which the importance is assigned to an input component that cannot be operated in a production process, the monitor cannot grasp influence of an input component that can be operated in the production process on an estimated value, and various devices are difficult to be operated in the production process in some cases. Thus, in the related art, an effect of the importance is extremely limited in some cases. 
     SUMMARY 
     It is an object of the present invention to at least partially solve the problems in the related technology. 
     According to an aspect of the embodiments, an information processing device includes: processing circuitry configured to: acquire a plurality of pieces of data; input the pieces of data to a model as input data, and in a case of obtaining output data output from the model, calculate an importance of each component of the input data with respect to the output data based on the input data and the output data; acquire a binding coefficient indicating a relevance among components of the input data; calculate a bound importance obtained by applying the binding coefficient to the importance; and create information indicating the bound importance of each input item. 
     The above and other objects, features, advantages and technical and industrial significance of this invention will be better understood by reading the following detailed description of presently preferred embodiments of the invention, when considered in connection with the accompanying drawings. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG. 1  is a block diagram illustrating a configuration example of an information processing device according to a first embodiment; 
         FIG. 2  is a diagram for explaining an outline of information processing performed by an information processing device illustrated in  FIG. 1 ; 
         FIG. 3  is a diagram illustrating an example of a heat map image indicating distribution of bound importances; 
         FIG. 4  is a diagram for explaining dimension reduction; 
         FIG. 5  is a diagram for explaining an example of a causal relation among input/output variables; 
         FIG. 6  is a flowchart illustrating an example of a processing procedure of information processing according to the first embodiment; 
         FIG. 7  is a diagram exemplifying a relevance among components of input data; 
         FIG. 8  is a diagram exemplifying input/output of data of a pre-learned estimation model; 
         FIG. 9  is a block diagram illustrating a configuration example of an information processing device according to a modification of the first embodiment; and 
         FIG. 10  is a diagram illustrating a computer that executes a computer program. 
     
    
    
     DESCRIPTION OF EMBODIMENTS 
     The following describes embodiments of an information processing device, an information processing method, and an information processing program according to the present application in detail based on the drawings. The information processing device, the information processing method, and the information processing program according to the present application are not limited to the embodiments. Hereinafter, in a case of describing A that is a vector, a matrix, or a scalar as “{circumflex over ( )}A”, for example, “{circumflex over ( )}A” is assumed to be equal to a “symbol obtained by writing “{circumflex over ( )}” right above “A””. In a case of describing A that is a vector, a matrix, or a scalar as “˜A”, for example, “˜A” is assumed to be equal to a “symbol obtained by writing “˜” right above “A””. 
     First Embodiment 
     The following embodiment describes a configuration of an information processing device  10  according to a first embodiment and a procedure of processing performed by the information processing device  10  in order, and lastly describes an effect of the first embodiment. 
     Configuration of Information Processing Device 
     First, the following describes the configuration of the information processing device  10  with reference to  FIG. 1 .  FIG. 1  is a block diagram illustrating a configuration example of the information processing device according to the first embodiment. 
     Herein, for example, the information processing device  10  acquires a plurality of pieces of data acquired by a sensor installed in a facility to be monitored such as a factory or a plant. The information processing device  10  then uses the pieces of acquired data as inputs, and estimates a state of the facility to be monitored by using a pre-learned estimation model for estimating an anomaly in the facility to be monitored. 
     The information processing device  10  uses data of respective sensors input to the pre-learned estimation model and output data output from the pre-learned estimation model to calculate a contribution degree (importance) of each sensor to an output value. The importance indicates a degree of contribution of each input to an output. It is meant that, as an absolute value of the importance is larger, an influence degree of the input with respect to the output is higher. Herein, as the importance, the information processing device  10  obtains the importance of each input component with respect to a pre-learned model while considering a relevance among input components of the pre-learned estimation model. Specifically, the information processing device  10  acquires a binding coefficient indicating a relevance among components of input data of the pre-learned estimation model, and applies the binding coefficient to the importance to calculate and output the bound importance including the relevance among the input components of the pre-learned estimation model. 
     As a result, the monitor of the facility to be monitored can confirm the bound importance already including the relevance among the input components. Thus, the monitor does not necessarily examine a matching degree between the importance and an experience of the monitor himself/herself related to the correlation and the causal relation among the input components, the examination having been conventionally requested. In other words, the information processing device  10  outputs the bound importance already including the relation among the input components, so that a load of interpreting the bound importance on the monitor can be reduced. 
     As illustrated in  FIG. 1 , the information processing device  10  includes a communication processing unit  11 , a control unit  12 , and a storage unit  13 . The following describes processing performed by each unit included in the information processing device  10 . 
     The communication processing unit  11  controls communication related to various kinds of information exchanged with a connected device. For example, the communication processing unit  11  receives a plurality of pieces of data as processing targets from another device. Specifically, the communication processing unit  11  receives a plurality of pieces of sensor data acquired in the facility to be monitored. The communication processing unit  11  also transmits, to the other device, a state of the facility to be monitored estimated by the information processing device  10 . The information processing device  10  may communicate with the other device via a communication network, or may operate in a local environment without being connected to the other device. 
     The storage unit  13  stores data and computer programs requested for various kinds of processing performed by the control unit  12  and includes a data storage unit  13   a , a pre-learned estimation model storage unit  13   b , and a pre-learned relation model storage unit  13   c . For example, the storage unit  13  is a storage device such as a semiconductor memory element including a random access memory (RAM), a flash memory, and the like. 
     The data storage unit  13   a  stores data collected by an acquisition unit  12   a  described later. For example, the data storage unit  13   a  stores data from sensors disposed in target appliances in a factory, a plant, a building, a data center, and the like (for example, data such as a temperature, a pressure, sound, and vibration). The data storage unit  13   a  may store any type of data constituted of a plurality of real values such as image data, not limited to the data described above. 
     The pre-learned estimation model storage unit  13   b  stores the pre-learned estimation model (estimation model). The pre-learned estimation model is a model in which an input and an output are set corresponding to a problem to be solved. For example, the pre-learned estimation model is an estimation model for estimating an anomaly in the facility to be monitored. Specifically, the pre-learned estimation model is an estimation model for solving a problem for estimating a certain indicator for a certain product in a factory. In this case, the pre-learned estimation model is a model using sensor data within a certain time width acquired from a manufacturing process as input data, and outputting a value of an indicator at a time that is shifted, by a certain time, from an end time of the time width of the input data, the model that has been learned by using a method of statistics/machine learning. 
     Estimation means a procedure of estimating an unknown value by using a known value in a sense from a viewpoint of statistics/machine learning. Specific examples of this estimation model may include existing models such as a model that does not broadly consider a time-series property such as a simple linear multiple regression model or a model obtained by applying appropriate regularization thereto, a model that can be converted into a state space model that allows an exogenous variable such as a vector autoregressive model, or a model of various neural networks such as a feedforward type, a convolutional type, and a recurrent type, or deep learning performed by using a combination thereof. 
     The pre-learned relation model storage unit  13   c  stores the pre-learned relation model (relation model). The pre-learned relation model is a model that has learned a relevance among components of the input data with respect to the pre-learned estimation model. For example, the pre-learned relation model has learned a relevance among a plurality of pieces of sensor data in advance by using a method of statistics/machine learning based on the pieces of sensor data acquired on a time-series basis in the facility to be monitored. The pre-learned relation model uses the input data for the pre-learned estimation model as an input, and outputs a group of values of binding coefficients corresponding to respective input components of the input data. The binding coefficient is a coefficient indicating the relevance from a value of each component to a value of each component of the input data. 
     The control unit  12  includes an internal memory for storing requested data and computer programs specifying various processing procedures and executes various kinds of processing therewith. For example, the control unit  12  includes the acquisition unit  12   a , an estimation unit  12   b , an importance extraction unit  12   c  (a first importance calculation unit), a coefficient acquisition unit  12   d , a binding unit  12   e  (a second importance calculation unit), and a visualization unit  12   f  (a creation unit). Herein, the control unit  12  is, for example, an electronic circuit such as a central processing unit (CPU), a micro processing unit (MPU), and a graphical processing unit (GPU), or an integrated circuit such as an application specific integrated circuit (ASIC) and a field programmable gate array (FPGA). 
     The acquisition unit  12   a  acquires a plurality of pieces of data. For example, the acquisition unit  12   a  acquires a plurality of pieces of sensor data acquired in the facility to be monitored. Specifically, the acquisition unit  12   a  periodically (for example, every minute) receives multivariate time-series numerical data from a sensor installed in the facility to be monitored such as a factory or a plant, and stores the data in the data storage unit  13   a . Herein, the data acquired by the sensor is, for example, various kinds of data such as a temperature, a pressure, sound, and vibration related to a device or a reactor in the factory or a plant as the facility to be monitored. The data acquired by the acquisition unit  12   a  is not limited to the data acquired by the sensor but may be image data or numerical data input by a person, for example. 
     The estimation unit  12   b  inputs a plurality of pieces of data to the model as input data, and obtains output data (hereinafter, referred to as estimated output data) output from this model. The estimation unit  12   b  uses the pieces of sensor data acquired by the acquisition unit  12   a  as input data to be input to the pre-learned estimation model for estimating the state of the facility to be monitored, and obtains the estimated output data output from the pre-learned estimation model as an estimated value. 
     Specifically, the estimation unit  12   b  obtains, as the estimated output data, an estimated value related to the state of the facility to be monitored after a certain time set in advance elapses. For example, the estimation unit  12   b  obtains, as the estimated value, a presumed value of a specific sensor in the facility to be monitored. The estimation unit  12   b  may also calculate an abnormality degree from the presumed value that is output as described above. The estimated output data is output to the importance extraction unit  12   c , and is visualized by the visualization unit  12   f  as part of an output from the information processing device  10 . 
     The importance extraction unit  12   c  extracts the importance for output data of each component of the input data based on the input data and the output data output from the pre-learned estimation model. The importance extraction unit  12   c  calculates a group of values of importances for the output data of respective components of the input data by using one or more of the input data, the pre-learned estimation model, and the output data output from the pre-learned estimation model. For example, the importance extraction unit  12   c  inputs, as the input data, a plurality of pieces of the sensor data to the pre-learned estimation model for estimating the state of the facility to be monitored, and calculates the importance for each sensor based on the input data and the output data in a case of obtaining the output data output from the pre-learned estimation model. Hereinafter, the importance calculated by the importance extraction unit  12   c  is referred to as an input importance. 
     The following describes a specific example of calculating the input importance. For example, in the pre-learned model for calculating the output value from the input value, the importance extraction unit  12   c  uses a partial differential value related to each input value of the output value or an approximate value thereof to calculate the input importance for each sensor at each time. For example, the importance extraction unit  12   c  assumes the pre-learned estimation model as a function, and calculates the importance by using a method such as a sensitivity map, integrated gradients, SmoothGrad, or Time-Smoothgrad using gradients in values of the input data. The importance extraction unit  12   c  may also calculate the importance by using a method of applying perturbation to the input data to calculate variation in the estimated output data, for example, a method of applying sufficiently small perturbation to each component of the input or a method of occlusion and the like. 
     The coefficient acquisition unit  12   d  acquires a binding coefficient indicating a relevance among the components of the input data of the pre-learned estimation model. The coefficient acquisition unit  12   d  acquires the binding coefficient by using the pre-learned relation model. The coefficient acquisition unit  12   d  inputs a value of each component of the input data to the pre-learned relation model, and acquires a group of values of binding coefficients indicating the relevance with respect to values of the respective components of the input data output from the pre-learned relation model. 
     The binding unit  12   e  calculates the bound importance by applying a corresponding binding coefficient to the input importance extracted by the importance extraction unit  12   c . The binding unit  12   e  uses the group of values of the input importances and a group of values of the binding coefficients as inputs, assigns weight to each component of the input data by a value of a corresponding binding coefficient, and calculates a group of values obtained by adding up values of corresponding input importances, that is, a group of values of bound importances. The group of values of bound importances is part of the output from the information processing device  10 . 
     The visualization unit  12   f  creates information indicating the estimated value estimated by the estimation unit  12   b  (for example, the abnormality degree), the bound importance calculated by the binding unit  12   e , and the binding coefficient acquired by the coefficient acquisition unit  12   d . For example, the visualization unit  12   f  creates and visualizes an image indicating the estimated value, the bound importance, and the binding coefficient. 
     For example, the visualization unit  12   f  displays the abnormality degree calculated by the estimation unit  12   b  as a chart screen. The visualization unit  12   f  also displays a graph indicating progression of the bound importance for each piece of the sensor data. The visualization unit  12   f  also obtains items having a relevance higher than a predetermined value among input items based on the binding coefficient, and creates information indicating the items in association with each other. For example, in the graph indicating the progression of the bound importance of each piece of the sensor data, the visualization unit  12   f  causes pieces of discrimination information of sensors having a relevance higher than the predetermined value to be framed, displayed with blinking, or displayed in the same color. 
     Alternatively, the visualization unit  12   f  displays the fact that there are sensors having a relevance higher than the predetermined value, and the discrimination information of the sensors having the relevance. The visualization unit  12   f  may also create a heat map indicating distribution of the bound importances of the respective input items per unit time. 
     Procedure of Information Processing 
     Next, the following describes an outline of information processing performed by the information processing device  10  with reference to  FIG. 2 .  FIG. 2  is a diagram for explaining the outline of the information processing performed by the information processing device  10  illustrated in  FIG. 1 . 
     In  FIG. 2 , a sensor and a device for collecting signals for operation and the like are attached to a reactor or a device in a plant to acquire data at every fixed time.  FIG. 2  ( 1 ) illustrates progression of process data collected by the acquisition unit  12   a  from the sensor A to a sensor E. The estimation unit  12   b  estimates an anomaly after a certain time elapses by using the pre-learned estimation model (refer to  FIG. 2  ( 2 )). The visualization unit  12   f  then outputs time series data of an estimated abnormality degree as a chart screen (refer to  FIG. 2  ( 3 )). 
     The importance extraction unit  12   c  extracts an input importance for a predetermined output value for each sensor at each time by using the process data input to the pre-learned estimation model and the output value from the pre-learned model (refer to  FIG. 2  ( 4 )). The coefficient acquisition unit  12   d  then inputs values of respective components of the process data to the pre-learned relation model, and acquires a group of values of binding coefficients indicating a relevance with respect to the value of each component (refer to  FIG. 2  ( 5 )). The binding unit  12   e  calculates the bound importance by applying a corresponding binding coefficient to the input importance extracted by the importance extraction unit  12   c  (refer to  FIG. 2  ( 6 )). 
     The visualization unit  12   f  then displays a chart screen indicating progression of the bound importance of the process data of each of the sensors A to E with respect to estimation (refer to  FIG. 2  ( 7 )). As illustrated in  FIG. 2  ( 7 ), the visualization unit  12   f  may display sensors having a high relevance by surrounding, with frames W 1  and W 2 , pieces of discrimination information “D” and “E” of the sensors D and E having a relevance higher than the predetermined value. In a case in which the relevance between the sensors is directed, the visualization unit  12   f  may display the fact that the relevance is directed, not limited to a case in which the relevance between the sensors is undirected. For example, in a case in which the sensor C-&gt;the sensor D is higher than the predetermined value but the sensor D-&gt;the sensor C is lower than the predetermined value, the visualization unit  12   f  may display text describing that fact. A display method for the sensors is merely an example, and is not limited to displaying with a frame or text.  FIG. 3  is an example of a heat map image indicating distribution of the bound importances. As illustrated in  FIG. 3 , the visualization unit  12   f  may create and display a heat map indicating distribution of the bound importances of the respective input items. 
     Example of Calculating Binding Coefficient 
     Next, the following describes an example of calculating the binding coefficient. It is assumed that the pre-learned estimation model is provided in advance. A pre-learned estimation model f (refer to Expression (1)) is a model that has learned a relation between an input x=(x 1 , . . . , x n ) ϵ R n  and an output y E R from data in advance. It is assumed that estimated data {circumflex over ( )}y=f (x)ϵR. 
         f:x     ŷ   (1)
 
     Regarding estimation at the time of giving the input x=(x 1 , . . . , x n ) to the model f, it is assumed that the importance extraction unit  12   c  can calculate an input importance a i =a i  (x; f) of each component of the input by using an existing method as described above. 
     In the first embodiment, the binding unit  12   e  calculates a bound importance ˜a as represented by the expression (2) by using a group of the binding coefficients (c i   i′ ) i,i′  acquired by the coefficient acquisition unit  12   d , and outputs the bound importance. 
         ãi   i ( x;f )Σ i′   c   i   i′   a   i′ ( x;f )  (2)
 
     As a method of calculating the input importance a i =a i  (x; f), for example, there is known a method of using a partial differentiation coefficient (gradient) represented by the expression (3). 
     
       
         
           
             
               
                 
                   
                     
                       ∂ 
                       f 
                     
                     
                       ∂ 
                       
                         x 
                         i 
                       
                     
                   
                   ⁢ 
                   
                     ( 
                     x 
                     ) 
                   
                 
               
               
                 
                   ( 
                   3 
                   ) 
                 
               
             
           
         
       
     
     As an improved method for calculating the input importance, a method of integrating the gradient by an appropriate path (Integrated Gradient), a method of smoothing the gradient by noise (SmoothGrad), and Time-Smoothgrad are suggested. Time-Smoothgrad is an improved method compatible with a case in which an input is part of time series data. 
     Subsequently, the following specifically describes a method of calculating the binding coefficient (c i   i′ ) i,i′  requested for deriving a bound importance ˜a i =˜a i  (x; f). First, as a first example, the following describes an example of obtaining the binding coefficient by using a manifold hypothesis. 
     In the field of machine learning, there is a case of assuming that lower-dimensional representation satisfying an appropriate hypothesis is present regarding a region that may be taken by the input data. This is called the manifold hypothesis. In a case of this example, for example, it is assumed that an input variable x=(x 1 , . . . , x n )ϵR n  can be decomposed as represented by the expression (4) by using a low-dimensional latent variable z=(z 1 , . . . , z m )ϵR m  (m&lt;n). It is assumed that represents noise satisfying an appropriate condition. 
         x={circumflex over (x)} ( z )+ϵ  (4)
 
     Actually, a group of a function z (x) from the input variable to a corresponding latent variable and a function {circumflex over ( )}x (z) from the latent variable to a reconstructed input variable are learned by the pre-learned relation model. This example describes an example of calculating a group of binding coefficients in a case in which representation by a latent variable of an input variable is given, that is, the group of two functions described above is given in Principal Component Analysis (PCA) or an AutoEncoder (AE). 
       FIG. 4  is a diagram for explaining dimension reduction. For example, in a case of performing dimension reduction using principal component analysis, it is assumed that the latent variable corresponding to the input can be calculated by the expression (5) with a coefficient matrix W=(w ij ) i,j ϵR n×m , and it is assumed that the input reconstructed from the latent variable can be calculated by the expression (6). 
         z   j ( x;W )=Σ i   x   i   w   ij   (5)
 
         {circumflex over (x)}   i ( z;W )=Σ j   z   j   w   ij   (6)
 
     At this point, for example, the coefficient acquisition unit  12   d  calculates the binding coefficient as follows. A partial differentiation coefficient ∂{circumflex over ( )}x i′ /∂x i  is assumed to be c i   i′ . For example, in a case of principal component analysis exemplified herein, ∂z j /∂x i =w ij  and ∂{circumflex over ( )}x i′ /∂z j =w i′j  are satisfied, so that a binding coefficient c i   i′  is represented by the expression (7) using a chain rule. 
     
       
         
           
             
               
                 
                   
                     c 
                     i 
                     
                       i 
                       ′ 
                     
                   
                   = 
                   
                     
                       
                         ∂ 
                         
                           
                             x 
                             ^ 
                           
                           
                             i 
                             ′ 
                           
                         
                       
                       
                         ∂ 
                         
                           x 
                           i 
                         
                       
                     
                     = 
                     
                       
                         
                           ∑ 
                           j 
                         
                         ⁢ 
                         
                           
                             
                               ∂ 
                               
                                 
                                   x 
                                   ^ 
                                 
                                 
                                   i 
                                   ′ 
                                 
                               
                             
                             
                               ∂ 
                               
                                 z 
                                 j 
                               
                             
                           
                           ⁢ 
                           
                             
                               ∂ 
                               
                                 z 
                                 j 
                               
                             
                             
                               ∂ 
                               
                                 x 
                                 i 
                               
                             
                           
                         
                       
                       = 
                       
                         
                           ∑ 
                           j 
                         
                         ⁢ 
                         
                           
                             w 
                             
                               i 
                               ⁢ 
                               j 
                             
                           
                           ⁢ 
                           
                             w 
                             
                               
                                 i 
                                 ′ 
                               
                               ⁢ 
                               j 
                             
                           
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   7 
                   ) 
                 
               
             
           
         
       
     
     The binding coefficient c i   i′  calculated as described above is interpreted as a degree of variation of the input variable in an input variable x i′  direction under restriction of the manifold hypothesis at the time of trying to cause the input variable to vary in an x i  direction, and the binding unit  12   e  binds the input importance. 
     This example also corresponds to a case in which dimension reduction is performed on the input data by Latent Semantic Analysis (LSA), Non-negative Matrix Factorization (NMF), and the like, or a case in which a given pre-learned estimation model is a model that can obviously return latent representation to a space of original input data using Linear Discriminant Analysis (LDA), Partial Least Squares (PLS), Canonical Correlation Analysis (CCA), and the like. 
     Next, the following describes an example of obtaining the binding coefficient by using the causal relation as a second example. When the causal relation between variables of input/output is known, for example, when inference has been performed by appropriate statistical causal analysis, a group of binding coefficients (c i   i′ ) i,i′  is calculated by using this relation. 
     In this example, described is an example of calculating the group of the binding coefficients in a case in which the causal relation among input/output variables is described to be already known by using Structural Equation Modeling (SEM). Specifically, the causal relation between each input variable and output variable is assumed to be unknown. 
       FIG. 5  is a diagram for explaining an example of the causal relation among input/output variables. The causal relation among the input/output variables is assumed to be given by structural equations illustrated in  FIG. 5 . In this case, for example, the binding coefficient is calculated as follows. 
     It is assumed that c i   i =1 is always satisfied. When i≠i′, for each directed path from x i  to x i′  on a directed graph illustrated in  FIG. 5 , the sum total of values obtained by multiplying a directed side constituting the directed path by a coefficient of a corresponding structural equation is assumed to be c i   i′ . 
     For example, directed paths from x 1  to x 4  on the exemplified directed graph include three paths including {(x 1 , x 2 ), (x 2 , x 3 ), (x 3 , x 4 )}, {(x 1 , x 3 ), (x 3 , x 4 )}, and {(x 1 , x 4 )}, so that the binding coefficient c 1   4  is represented by the expression (8). 
         c   1   4   =b   1   2   b   2   3   b   3   4   +b   1   3   b   3   4   +b   1   4   (8)
 
     For example, a directed path from x 3  to x 1  is not present on the directed graph illustrated in  FIG. 5 , so that c 3   1 =0 is satisfied. 
     In this way, the information processing device  10  interprets the calculated binding coefficient c i   i′  as a degree of variation of the input variable x i′  at the time of operating the input variable x i , and binds the input importance. 
     As a third example, the following describes an example of not performing binding. Formally, it is assumed that c i   i =1 and c i   i′ =0 (i≠i′) are always satisfied irrespective of a value of an input x. 
     In this case, ˜a i  is represented by the expression (9), and the bound importance is identical to a related input importance. In this sense, the bound importance is a concept truly including the related input importance. 
         ã   i =1· a   i +Σ i′≠i 0 ·a   i′   =a   i   (9)
 
     Processing Procedure of Information Processing 
     Next, the following describes an example of a processing procedure performed by the information processing device  10 .  FIG. 6  is a flowchart illustrating an example of the processing procedure of information processing according to the first embodiment. 
     As exemplified in  FIG. 6 , when the acquisition unit  12   a  acquires data (Step S 1 ), the estimation unit  12   b  inputs the data to the pre-learned estimation model, and estimates, for example, an anomaly after a certain time elapses based on the output data of the pre-learned estimation model (Step S 2 ). The visualization unit  12   f  displays an image visualizing time series data of the estimated value of the abnormality degree (Step S 3 ). 
     The importance extraction unit  12   c  then extracts the input importance with respect to a predetermined output value for each input item at each time by using the input data input to the pre-learned estimation model and the output value from the pre-learned model (Step S 4 ). 
     The coefficient acquisition unit  12   d  then inputs a value of each component of the input data to the pre-learned relation model, and acquires a group of values of binding coefficients each indicating the relevance with respect to the value of each component (Step S 5 ). The binding unit  12   e  calculates the bound importance by applying a corresponding binding coefficient to the input importance extracted by the importance extraction unit  12   c  (Step S 6 ). The visualization unit  12   f  then displays an image indicating the bound importance of each input item (Step S 7 ). 
     Effect of First Embodiment 
     In this way, the information processing device  10  acquires the binding coefficient indicating the relevance among the components of the input data of the pre-learned estimation model, and applies the binding coefficient to the importance to calculate and output the bound importance including the relevance among the input components of the pre-learned estimation model. 
       FIG. 7  is a diagram exemplifying the relevance among the components of the input data.  FIG. 8  is a diagram exemplifying input/output of data of the pre-learned estimation model. Specifically, in a case of the time series data exemplified in  FIG. 7 , a causal relation or physical limitation may be present among the components of the input (refer to  FIG. 7  ( 1 )). Thus, in the first embodiment, the binding coefficient indicating the relevance among the components of the input data of the pre-learned estimation model is acquired, and the binding coefficient is applied to the input importance to calculate the bound importance (refer to  FIG. 7  ( 2 )). In this way, in the first embodiment, even with the pre-learned estimation model that may be called a black box, the bound importance reflecting the relevance among the input components of the model can be obtained (refer to  FIG. 8  ( 3 )). 
     As a result, the monitor of the facility to be monitored can confirm the bound importance to which the relevance among the input components is already applied. Thus, the monitor does not necessarily examine a matching degree between the importance and an experience of the monitor himself/herself related to the correlation and the causal relation among the input components, the examination having been conventionally requested. In other words, the information processing device  10  outputs the bound importance to which the relevance among the input components is already applied, so that a load of interpreting the bound importance on the monitor can be reduced. 
     In the first embodiment, it is possible to obtain items having a relevance higher than the predetermined value among the input items of the input data of the pre-learned estimation model based on the binding coefficient. In this case, the monitor can easily recognize the relevance among the input components, and in a case of monitoring a production process, the monitor can appropriately grasp influence of the input component that can be operated on the estimated value, and appropriately operate various devices. 
     Modification 
       FIG. 9  is a block diagram illustrating a configuration example of an information processing device according to a modification of the first embodiment. In an information processing device  10 A illustrated in  FIG. 9 , the pre-learned relation model storage unit  13   c  stores a first pre-learned relation model  13   d  and a second pre-learned relation model  13   e.    
     For example, the first pre-learned relation model  13   d  has performed learning to calculate the binding coefficient from mapping of dimension reduction. The second pre-learned relation model  13   e  is a model that learns a causal relation between input variables and calculates the binding coefficient. In a case of acquiring the binding coefficient, the coefficient acquisition unit  12   d  switches between the first pre-learned relation model  13   d  and the second pre-learned relation model  13   e  to be used depending on a monitoring mode. 
     For example, in a case of a normal mode, the coefficient acquisition unit  12   d  calculates the binding coefficient by using the first pre-learned relation model  13   d . In this case, the information processing device  10 A outputs the bound importance including an influence degree from each component of the input data on the output of the pre-learned estimation model. Due to this, the monitor can monitor that the output of the pre-learned estimation model is susceptible to a variation of which component of the input data based on an output result of the information processing device  10 A, which is useful for previously finding a risk factor for stable operation in the production process. 
     In a case of an abnormal mode in which a certain anomaly occurs in the estimated output data, the coefficient acquisition unit  12   d  switches the model to the second pre-learned relation model  13   e  to calculate the binding coefficient. In this case, the information processing device  10 A outputs the bound importance that is bound by the causal relation between the input variables. Thus, the monitor can monitor a variation that can be caused in a value of the pre-learned estimation model by a device that can be operated in the actual production process among the input components based on the output result of the information processing device  10 A. As a result, by controlling the device that can be operated, the monitor can previously examine preventing an anomaly from occurring in the actual production process, or a measure for a case in which an anomaly actually occurs. 
     System Configuration and Like 
     The components of the devices illustrated in the drawings are merely conceptual, and it is not required that they are physically configured as illustrated necessarily. That is, specific forms of distribution and integration of the devices are not limited to those illustrated in the drawings. All or part thereof may be functionally or physically distributed/integrated in arbitrary units depending on various loads or usage states. All or optional part of the processing functions performed by the respective devices may be implemented by a CPU or a GPU and computer programs analyzed and executed by the CPU or the GPU, or may be implemented as hardware using wired logic. 
     Among pieces of the processing described in the present embodiment, all or part of the pieces of processing described to be automatically performed can be manually performed, or all or part of the pieces of processing described to be manually performed can be automatically performed by using a known method. Additionally, the processing procedures, control procedures, specific names, and information including various kinds of data and parameters described herein or illustrated in the drawings can be optionally changed unless otherwise specifically noted. 
     Computer Program 
     It is also possible to create a computer program describing the processing performed by the information processing device described in the above embodiment in a computer-executable language. For example, it is possible to create a computer program describing the processing performed by the information processing device  10  or the information processing device  10 A according to the embodiment in a computer-executable language. In this case, the same effect as that of the embodiment described above can be obtained when the computer executes the computer program. Furthermore, such a computer program may be recorded in a computer-readable recording medium, and the computer program recorded in the recording medium may be read and executed by the computer to implement the same processing as that in the embodiment described above. 
       FIG. 10  is a diagram illustrating the computer that executes the computer program. As exemplified in  FIG. 9 , a computer  1000  includes, for example, a memory  1010 , a CPU  1020 , a hard disk drive interface  1030 , a disk drive interface  1040 , a serial port interface  1050 , a video adapter  1060 , and a network interface  1070 , which are connected to each other via a bus  1080 . 
     As exemplified in  FIG. 10 , the memory  1010  includes a read only memory (ROM)  1011  and a RAM  1012 . The ROM  1011  stores, for example, a boot program such as a Basic Input Output System (BIOS). As exemplified in  FIG. 9 , the hard disk drive interface  1030  is connected to a hard disk drive  1090 . As exemplified in  FIG. 9 , the disk drive interface  1040  is connected to a disk drive  1100 . For example, a detachable storage medium such as a magnetic disc or an optical disc is inserted into the disk drive  1100 . As exemplified in  FIG. 9 , the serial port interface  1050  is connected to a mouse  1110  and a keyboard  1120 , for example. As exemplified in  FIG. 10 , the video adapter  1060  is connected to a display  1130 , for example. 
     Herein, as exemplified in  FIG. 9 , the hard disk drive  1090  stores, for example, an OS  1091 , an application program  1092 , a program module  1093 , and program data  1094 . That is, the computer program described above is stored in the hard disk drive  1090 , for example, as a program module describing a command executed by the computer  1000 . 
     The various kinds of data described in the above embodiment are stored in the memory  1010  or the hard disk drive  1090 , for example, as program data. The CPU  1020  then reads out the program module  1093  or the program data  1094  stored in the memory  1010  or the hard disk drive  1090  into the RAM  1012  as needed, and performs various processing procedures. 
     The program module  1093  and the program data  1094  related to the computer program are not necessarily stored in the hard disk drive  1090 , but may be stored in a detachable storage medium, for example, and may be read out by the CPU  1020  via a disk drive and the like. Alternatively, the program module  1093  and the program data  1094  related to the computer program may be stored in another computer connected via a network (a local area network (LAN), a wide area network (WAN), and the like), and may be read out by the CPU  1020  via the network interface  1070 . 
     The embodiments and the modification thereof described above are included in the technique disclosed herein, and also included in the invention disclosed in CLAIMS and equivalents thereof. 
     According to the present invention, an importance of each of input components of a model can be obtained while considering a relevance among the input components of the model. 
     Although the invention has been described with respect to specific embodiments for a complete and clear disclosure, the appended claims are not to be thus limited but are to be construed as embodying all modifications and alternative constructions that may occur to one skilled in the art that fairly fall within the basic teaching herein set forth.