Abstract:
A method and system of forecasting reliability of an asset is provided. The method includes identifying peer units of the asset by using selected criteria, performing a search for the peer units based upon the selected criteria, and constructing local predictive models using the peer units. The method also includes estimating the future behavior of the asset based upon the local predictive models and dynamically updating the local predictive models to reflect at least one change in the criteria.

Description:
STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH AND DEVELOPMENT  
       [0001]     The U.S. Government may have certain rights in this invention pursuant to contract number 621-004-S-0031 awarded by the Defense Advanced Research Projects Agency (DARPA) of the Department of Defense (DoD). 
     
    
     BACKGROUND  
       [0002]     The present disclosure relates generally to asset reliability forecasting and, in particular, to asset reliability forecasting and event-based asset selection.  
         [0003]     Selecting the most appropriate assets for use in an operation requires some degree of knowledge of the individual units of a particular group of assets, such as age, service/maintenance history, etc. For example, given a fleet of commercial trucks it may be useful to know each vehicle&#39;s repair history before determining which vehicle should be assigned to a particular operation. In this example, a newer truck that has had minimal repair issues may be chosen for a cross-country operation, while an older truck with a less-than-optimal repair record may be selected for an intra-state operation. Other information that may be useful in the selection process includes logistical information (e.g., the current location of a vehicle with respect to the starting point and destination of the operation), the sensitivity of the operation (e.g., valuable, fragile, or perishable cargo), environmental considerations (e.g., extreme heat, rough terrain), and time-sensitive considerations, to name a few.  
         [0004]     For large groups of assets, these determinations can become complex and fraught with error. Moreover, for particular types of assets and operations, the various information elements used in making these determinations oftentimes change dynamically over time, making the asset selection process even more uncertain.  
         [0005]     What is needed, therefore, is a way to identify and select the most appropriate assets for operations or events and to predict future performance of the assets based upon changing criteria over time.  
       BRIEF SUMMARY  
       [0006]     Exemplary embodiments include a method and system of forecasting reliability of an asset is provided. The method includes identifying peer units of the asset by using selected criteria, performing a search for the peer units based upon the selected criteria, and constructing local predictive models using the peer units. The method also includes estimating the future behavior of the asset based upon the local predictive models and dynamically updating the local predictive models to reflect at least one change in the criteria.  
         [0007]     The system includes a host system and a storage device in communication with the host system. The storage device stores asset data. The system also includes an asset integrity engine executing on the host system. The asset integrity engine performs a method. The method includes identifying peer units of the asset by using selected criteria, performing a search for the peer units based upon the selected criteria, and constructing local predictive models using the peer units. The method also includes estimating the future behavior of the asset based upon the local predictive models and dynamically updating the local predictive models to reflect at least one change in the criteria. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0008]     Referring to the exemplary drawings wherein like elements are numbered alike in the accompanying FIGURES:  
         [0009]      FIG. 1  is a system upon which the asset reliability forecasting processes may be implemented in exemplary embodiments;  
         [0010]      FIG. 2  is a flow diagram illustrating a process for implementing the asset reliability forecasting processes in exemplary embodiments;  
         [0011]      FIG. 3  is a graphical depiction of a state space for a cluster of potential peers associated with a selected probe in exemplary embodiments;  
         [0012]      FIG. 4  is a graphical depiction of a state space for a cluster of potential peers associated with a selected probe after applying a similarity function in exemplary embodiments;  
         [0013]      FIG. 5  is a diagram of a portion of the asset integrity engine and corresponding algebraic interpretations in exemplary embodiments;  
         [0014]      FIG. 6  is a flow diagram describing a process for computing a fitness function using a wrapper approach and a filter approach in exemplary embodiments;  
         [0015]      FIG. 7  is a diagram evolutionary architecture used in implementing evolutionary algorithms to tune the parameters of structural elements in exemplary embodiments; and  
         [0016]      FIG. 8  is a graphical depiction of time-based performance results of peers generated via the asset integrity engine in exemplary embodiments. 
     
    
     DETAILED DESCRIPTION OF THE EMBODIMENTS  
       [0017]     In accordance with exemplary embodiments, asset reliability forecasting processes provide the ability to select the most reliable units, or assets, within a group of assets by formulating it as a classification and prediction problem. The prediction of each unit&#39;s remaining life is based on the identification of “peer” units, i.e., assets with similar utilization and maintenance records that are expected to behave similarly to the unit under consideration. With these peers, local predictive models are constructed for estimating the unit&#39;s remaining life. Evolutionary algorithms (EAs) may be used to develop the criteria for defining peers and the relevance of each criterion in evaluating similarity with the unit. Each individual in the EA&#39;s population characterizes an instance-based fuzzy model that is used to predict the unit&#39;s remaining life. The precision of the selection of units with best-expected life provides the fitness value.  
         [0018]     The asset reliability forecasting processes may be applicable to any type of device or apparatus (e.g., electromechanical systems), such as commercial vehicles (e.g., passenger jets, cargo vehicles), military vehicles (e.g., tanks, jet fighters, ships and submarines, etc.). It should be noted that the fleet need not be limited to mobile assets, and therefore it could be a “fleet” of medical imaging equipment (e.g., CT and MRI scanners, XR machines), or any other suitable system. The assets are described herein with respect to vehicles of a fleet.  
         [0019]     The concept of similarity in determining asset reliability is dynamic over time. The reliability of an asset, as described herein, is evaluated in terms of the mission or operation at hand. For example, given a mission that has a set of requirements (e.g., time or duration, hazardous environmental conditions, number of assets needed, etc.), what percentage of assets assigned to that mission will be able to complete the mission without encountering a critical failure. This reliability determination is complicated by factors (e.g., a new type of operation is employed or new equipment platforms are introduced to the asset fleet and insufficient data exists on how the assets will behave in that environment).  
         [0020]     Turning now to  FIG. 1 , a system upon which the asset reliability forecasting processes may be implemented will now be described. The system of  FIG. 1  includes a host system  102  in communication with one or more data sources  104  over a network  106  and storage device  108 .  
         [0021]     Host system  102  may be implemented using one or more servers or suitable high-speed processors operating in response to a computer program stored in a storage medium accessible by the server or servers. The host system  102  may operate as a network server (e.g., a web server) to communicate with network entities such as data sources  104 . The host system  102  may handle sending and receiving information to and from network entities, e.g., data sources  104  and may perform associated tasks.  
         [0022]     Host system  102  may also operate as an application server. In accordance with exemplary embodiments, the host system  102  executes one or more computer programs to perform asset reliability forecasting processes. These one or more computer programs are referred to collectively herein as an asset integrity engine  110 .  
         [0023]     As previously described, it is understood that separate servers may be utilized to implement the network server functions and the application server functions of host system  102 . Alternatively, the network server and the application server may be implemented by a single server executing computer programs to perform the requisite functions described with respect to host system  102 .  
         [0024]     The asset integrity engine  110  may include a user interface (UI)  112  for enabling individuals to perform activities, such as configuring the asset feature information, similarity parameters, and weighting parameters.  
         [0025]     Storage device  108  may be implemented using a variety of devices for storing electronic information. It is understood that the storage device  108  may be implemented using memory contained in the host system  102 , or it may be a separate physical device. The storage device  108  is logically addressable as a consolidated data source across a distributed environment that includes network  106 . Information stored in the storage device  108  may be retrieved and manipulated via the host system  102 . In an exemplary embodiment, the host system  102  operates as a database server and coordinates access to application data including data stored on storage device  108 .  
         [0026]     Storage device  108  stores a variety of information and content relating to assets of the entity implementing the asset reliability forecasting processes. Examples of the types of information stored in storage device  108  and managed by the asset integrity engine  110  may include asset prediction files, asset evaluation and performance files, and asset selection files. One or more databases may be utilized for organizing this information. For example, the organization or entity of host system  102  may maintain database records for each of its assets which provide, e.g., maintenance, repair, and utilization information, etc.  
         [0027]     The asset integrity engine  110  may access information available from external data sources  104  and utilize this information in generating and providing asset reliability predictions and performance information to requesting individuals. External data sources  104  refer to sources of information that are external to the host system  102 , and may be provided by a third party. The external data sources  104  may be implemented using one or more servers operating in response to a computer program stored therein or in a storage medium accessible by the server or servers (e.g., in a manner similar to that described above with respect to host system  102 ).  
         [0028]     The data sources  104  are used to train the model and validate the results of testing. Sources of data may include design and engineering data (e.g., model, configuration, date of manufacture, date of service, upgrades, software modifications, etc.), recommendation data from remote monitoring and diagnostics services (e.g., time-stamped records of when abnormal patterns in the fault data were detected, leading to a recommendation issued by monitoring service entity), maintenance data from repair shops (e.g., repair actions that successfully fixed the problem), utilization data from an entity utilizing the assets (odometer miles, megawatt-hours, hours spent motoring, cumulative engine hours, percentage of time spent in each gear setting, etc.), and other relevant data sources.  
         [0029]     Network  106  may be any type of known network including, but not limited to, a local area network (LAN), a wide area network (WAN), a global network (e.g. the Internet), a private network (e.g. an Intranet), and a virtual private network (VPN). The network  106  may be implemented using a wireless network or any kind of physical network implementation known in the art. Network entities (e.g., external data sources  104 ), may be coupled to the host system  102  through multiple networks (e.g., intranet and Internet) so that not all network entities are coupled to the host system  102  through the same network. One or more of the network entities and the host system  102  may be connected to the network  106  in a wireless fashion.  
         [0030]     Turning now to  FIG. 2 , a flow diagram describing a process for implementing the asset reliability forecasting processes in exemplary embodiments will now be described.  
         [0031]     At step  202 , a probe, or query, is identified. At step  204 , a search and retrieval of one or more data sources  104  is performed. This includes finding all database instances whose behavior is similar to the probe. The instances reflect the probe&#39;s potential peers (e.g., as points in an n-dimensional feature space). A probe Q has an associated n-dimensional vector of values for each potential attribute. A similar n-dimensional vector characterizes each unit u i  in the group. Furthermore, each unit has an attached vector O(u i )=[D 1,i , D 2,i , D k(i),i ] containing its historic operational availability durations: u i =[x 1,i , x 2,i , x n,i ]; O(u i )=[D 1,i , D 2,i , . . . , D k(i),i ]. For each dimension i, a Truncated Generalized Bell Function, TGBF i (x i ; a i , b i , c i ), centered at the value of the probe c i , which represents the degree of similarity along dimension i, as shown in  FIG. 2 . Since the parameters c i  in each TGBF i  are determined by the values of the problem, each TGBF i  has only two free parameters, a i  and b i , to control its spread and curvature, or slope. In the retrieval step, an instance is extracted from the database if all of its features are within the support of the TGBFs. Thus, the retrieval step may be formalized. P(Q), the set of potential peers of Q to be retrieved, is composed of all units within a range from the value of {u j , j=1, . . . , m|u j εN(Q)} and N(Q), a neighborhood of Q, is defined by the constraint ∥x i,Q −x i,j ∥&lt;R i  for all potential attributes i for which their corresponding weight is non-zero. R i  is half of the support of the TGBF i , centered on the probe&#39;s coordinate x i,Q . A graphical depiction of a state space  300  for a cluster of potential peers associated with a selected probe is shown in  FIG. 3 .  
         [0032]     At step  206 , a similarity function is applied to the instances retrieved at step  204 . The similarity function is a dynamic concept that may frequently change over time. Each TGBF i  is a membership function representing the degree of satisfaction of constraint A i (x i ). Thus, TGBF i  measures the closeness of an instance around the probe value X i,Q  along the i th  attribute. For a potential peer P j , we evaluate S ij =TGBF(x ij ; a i , b i , x i,Q ), its similarity with the probe Q along each attribute i. The values (a i , b i ) may be design choices initially selected manually, and later determined by an evolutionary algorithm. In order to find the most similar instances that are closest to the probe along all n attributes, a similarity measure defined as the intersection (minimum) of the constraint-satisfaction values is used:  
         S   j     =       Min   ⁢         n             i   =   1           ⁢     {     S     i   ,   j       }       =     Min   ⁢         n             i   =   1           ⁢       {     TGBF   ⁡     (         χ   iI     ;     a   i       ,     b   i     ,     χ   i     ⁢     ,   Q       )       }     .             
 
         [0033]     This equation implies that each attribute or feature is equally important in computing similarity. In order to consider each criterion to have a different relevance in that computation, a weight w i  may be attached to each attribute A i  and the similarity measure is extended between P j  and the probe Q using a weighted minimum operator:  
               S   j     =     Min   ⁢         n             i   =   1           ⁢     {     Max   ⁡     [       (     1   -     w   i       )     ,     S     j   ,   i         ]       }                   =     Min   ⁢         n             i   =   1           ⁢     {     Max   ⁡     [       (     1   -     w   i       )     ,     TGBF   ⁡     (         χ   i     ;     a   i       ,     b   i     ,     χ     i   ,   Q         )         ]       }                 
 
 where w i ε[0,1]. The set of values for the weights {w i } and of the parameters {(a i , b i )} are important design choices that impact the proper selection of peers. Initially, they may be selected manually. Subsequently, they may be derived using evolutionary search techniques as described further herein. A graphical depiction of a state space  400  for a cluster of potential peers associated with a selected probe after applying a similarity function is shown in  FIG. 4 . 
 
         [0034]     Using the selected similar instances, local predictive models are created for forecasting each unit&#39;s (asset&#39;s) remaining life at step  208 . Each local model is used to generate an estimated value of the predicted variable y. For example, assume that for a given probe Q, a number of peers m have been retrieved, P j (Q), j=1, . . . , m. Each peer P j (Q) has a similarity measure S j  with the probe. Furthermore, each peer P j  has a track record of operational availability between failures O(P j )=[D 1j , D 2j , . . . , D k(j),j ]. Each peer P i (Q) will have k(j) availability pulses in its track history. For each peer P j , the duration of the next availability duration y j =D k(j)+1j  is sought. The prediction of all peers {D k(j)+1j }(j=1, . . . , m) is combined to estimate the availability duration y Q  for the probe Q. The next availability duration D k(j)+1j  from the operational availability vector O(P j )=[D 1j , D 2j , . . . , D k(j),j ] by using an exponential average α that gives relevance to the most recent information, namely:  
               Y   j     =     D         k   ⁡     (   j   )       +   1     ,   j                   =       ⁢       D   _         k   ⁡     (   j   )       ,   j                   =       α   ×     D       k   ⁡     (   j   )       ,   j         +       (     1   -   α     )     ×         D   _           k   ⁡     (   j   )       -   1     ,   j       ⁡     [       where   ⁢           ⁢       D   _       1   ,   j         =     D     1   ,   j         ]                       =           (     1   -   α     )         k   ⁡     (   j   )       -   1       ⁢     D     1   ,   j         +         ∑     k   ⁡     (   j   )           t   =   2       ⁢         (     1   -   α     )         k   ⁡     (   j   )       -   t       ×   α   ×     D     t   ,   j                       
 
         [0035]     At step  210 , the model outputs are normalized and aggregated in order to determine a final output. The individual predictions {D k(j)+1j }(j=1, . . . , m) of the peers P j (Q) are combined to generate the prediction of the next availability duration, D next ,Q for the probe Q. This aggregation is referred to herein as the similarity weighted average and may be determined by computing the weighted average of the peers&#39; individual predictions using their normalized similarity to the probe as a weight:  
         Y   Q     =       D     Next   ,   Q       =             ∑     j   =   1     m     ⁢       S   j     ×     Y   j             ∑     j   =   1     m     ⁢     S   j         ⁢           ⁢   where   ⁢           ⁢     y   j       =       D         k   ⁡     (   j   )       +   1     ,   j       .             
 
 A diagram of a portion of the asset integrity engine  500  and corresponding algebraic interpretations is shown in  FIG. 5 . 
 
         [0036]     At step  212 , the accuracy of the model outputs is evaluated. Given the role played by the weights {w i }, search parameters {(a i , b i )} and by exponent α, it is desirable to create a methodology that could generate the best values according to selected metrics (e.g., classification precision). A primary performance metric may include the ability of a classifier to select the best units at any given time. Approaches used in defining the top units may include a fixed percentage approach or a fixed number approach. In addition, baselines calculated for measuring the increase in capability provided by the algorithms may include a random baseline (e.g., first baseline measured the expected performance if selection of the best N units were done randomly, a worse case scenario) and a heuristics baseline (e.g., a second baseline represents the best performance achieved by single or multiple heuristics which were used to rank the assets and pick the best N units).  
         [0037]     Evolutionary search techniques may be employed to develop and maintain the fuzzy instance based classifier. Using a wrapper methodology or filter technique, evolutionary algorithms are defined for tuning the parameters of a classifier, as well as for tuning a structural search via attribute (feature) selection and weighting. A process for computing a fitness function using a wrapper approach and a filter approach is shown in  FIG. 6 . Note that it is also possible to use a fitness function that combines wrapper characteristics, e.g. best classifier&#39;s performance, with filter characteristics, e.g. parsimony in chromosome representation (least amount of required information).  
         [0038]     At step  214 , an evolutionary algorithm is performed using the results of the evaluation described in step  212 . Evolutionary algorithms are composed of a population of individuals (e.g., chromosomes), each of which contains a vector of elements that represent distinct tunable parameters within the fuzzy instance based classifier configuration. Examples of tunable parameters may include the range of each parameter used to retrieve neighbor instances and the relative weights associated with each parameter used for similarity calculations.  
         [0039]     Each chromosome specifies a vector of weights [w 1 , w 2 , . . . w D ] and defines an instance of the attribute, or feature, space used by its associated classifier. If w i ε{0,1}, attribute selection is performed (i.e., a crisp subset of the universe of potential attributes is selected). If w i ε[0,1], attribute weighting is performed (i.e., a fuzzy subset of the universe of potential attributes is defined). 
 
[ w   1   w   2    . . . w   D ]└( a   1   , b   1 ), ( a   2   , b   2 ), . . . , ( a   D   , b   D )┘[α]
 
 where w i ε[0,1] for attribute weighting or w i ε[0,1] for attribute selection 
    D=Cardinality of universe of features U,D=|U| 
       d   =       ∑   i   D     ⁢     w   1           
 
 (fuzzy) cardinality of selected features 
    (a i , b i )=Parameters for TGBF i  and     α=Parameter for Exponential Average    
 
         [0043]     Thus, the first part of the chromosome, containing the weights vector [w 1 , w 2 , . . . w D ], defines the attribute space (e.g., the fuzzy instance based classifier structure), and the relevance of each attribute in evaluating similarity. The second part of the chromosome, containing the vector of pairs [(a 1 , b 1 ), . . . (a i , b i ), . . . (a D , b D )] defines the parameter for the retrieval and similarity evaluation. The last part of the chromosome, containing the parameter α, defines the local model.  
         [0044]     The fitness function may be computed using the wrapper approach or filter technique of  FIG. 6 . For each chromosome, its corresponding fuzzy instance based classifier is instantiated. Following a leave-one-out approach, the fuzzy instance based classifier may be used to predict the expected life of the asset under question (e.g., using the formula represented in step  210 ). Steps  204 - 210  may be repeated for all assets. The results may be sorted in decreasing order (e.g., via the predicted duration D next, Q . A top number or percentage of this sorted group (e.g., top 20%) may be selected. The fitness function of the chromosome represents the precision of the classification, or TP/(TP+FP), where TP=true positives (correct classifications) and FP=false positives (incorrect classifications). An evolutionary architecture used in implementing evolutionary algorithms to tune the parameters of structural elements, via attribute or feature selection and weighting is shown in  FIG. 7 .  
         [0045]     A graphical depiction of time-based performance results  800  (three time-based data slices) of peers generated via the asset integrity engine is shown in  FIG. 8 . The data segmentation allows evaluation of the affects of environmental, operational, or maintenance changes on the model and its performance, as well as an understanding of how the incremental acquisition of data may improve the performance of the learning techniques employed in the evolutionary process.  
         [0046]     As described above, asset reliability forecasting processes provides a fuzzy peer-based approach for performance modeling combined with an evolutionary framework for model maintenance. The asset reliability forecasting processes provide the ability to select the most reliable units, or assets, within a group of assets.  
         [0047]     While the invention has been described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the scope of the invention. In addition, many modifications may be made to adapt a particular situation or material to the teachings of the invention without departing from the essential scope thereof. Therefore, it is intended that the invention not be limited to the particular embodiment disclosed as the best or only mode contemplated for carrying out this invention, but that the invention will include all embodiments falling within the scope of the appended claims. Moreover, the use of the terms first, second, etc. do not denote any order or importance, but rather the terms first, second, etc. are used to distinguish one element from another. Furthermore, the use of the terms a, an, etc. do not denote a limitation of quantity, but rather denote the presence of at least one of the referenced item.