Patent Publication Number: US-11656606-B2

Title: Site-wide operations management optimization for manufacturing and processing control

Description:
BACKGROUND 
     The present invention generally relates to programmable computing devices, and more specifically, to computing devices, computer-implemented methods, and computer program products configured to optimize operations management for manufacturing and processing control. 
     Large scale manufacturing and processing facilities use complex computer-based control systems to regulate the sub-processes and overall processes used to produce goods. A control system is a set of electronic and mechanical devices that monitor real-time data and regulate each sub-process of the production process. The control systems control the output of one sub-process to be an optimal input of one or more ensuing sub-processes. Sensors perform real-time monitoring of each input, sub-process mechanism, and output and provide the data to the control system. The control system monitors the sensor data and can causes modifications to the input or mechanisms if there is an issue with the output. 
     SUMMARY 
     Embodiments of the present invention are directed to a site-wide operations management optimization for manufacturing and process control. A non-limiting example computer-implemented method includes selecting an optimization algorithm for the control system of a processing plant based on whether the control system is guided by a linear-based predictive model or a non-linear-based predictive model, in which a gradient is available. Calculating a set variable using the optimization algorithm. Predicting an output based on the calculated set variable. Comparing an actual output at the processing plant to the predicted output. Suspending a physical process at the processing plant in response to the actual output being a threshold value apart from the predicted output. 
     Other embodiments of the present invention implement features of the above-described method in computer systems and computer program products. 
     Additional technical features and benefits are realized through the techniques of the present invention. Embodiments and aspects of the invention are described in detail herein and are considered a part of the claimed subject matter. For a better understanding, refer to the detailed description and to the drawings. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The specifics of the exclusive rights described herein are particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other features and advantages of the embodiments of the invention are apparent from the following detailed description taken in conjunction with the accompanying drawings in which: 
         FIG.  1    illustrates a block diagram of components of a manufacturing and process control system in accordance with one or more embodiments of the present invention; 
         FIG.  2    illustrates an exemplary diagram of a network of oil processing plants in accordance with one or more embodiments of the present invention; 
         FIG.  3    illustrates an exemplary network representation of processing plants in accordance with one or more embodiments of the present invention; 
         FIG.  4    illustrates a flow diagram of a process for optimizing a manufacturing and process control system in accordance with one or more embodiments of the present invention; and 
         FIG.  5    illustrates a block diagram of a computer system for use in implementing one or more embodiments of the present invention. 
     
    
    
     The diagrams depicted herein are illustrative. There can be many variations to the diagrams or the operations described therein without departing from the spirit of the invention. For instance, the actions can be performed in a differing order, or actions can be added, deleted, or modified. Also, the term “coupled” and variations thereof describes having a communications path between two elements and does not imply a direct connection between the elements with no intervening elements/connections between them. All of these variations are considered a part of the specification. 
     DETAILED DESCRIPTION 
     One or more embodiments of the present invention provide computer-implemented methods, computing systems, and computer program products for optimizing a throughput production of each production facility of a network of production facilities. 
     Modern manufacturing and processing facilities are arranged as a network of remote plants, in which each plant completes a step towards converting raw materials into marketable products. Each plant in the network includes a self-contained set of inputs and outputs for each of the plant&#39;s sub-processes. The control systems use predictive models to guide the overall process by monitoring the relationship between various set-points (controllable variable values), material inflows, and throughput. For example, oil processing involves multiple remote facilities that each perform separate functions include separating residuals from crude oil, converting the residuals into marketable products, and treating the marketable products to remove any contaminants. The individual control systems use predictive models to determine an expected throughput from each oil processing facility and regulate the facilities based on a comparison of actual throughput to predicted throughput. 
     To ensure that the control systems produce the maximum amount of marketable products, large-scale optimization measures are implemented into the control systems. The large-scale optimization methods need to account for operational restrictions and non-linear relationships between inflows and outflows. Conventional optimization methods rely on industry-specific models that are too costly to be portable to other industries or, in some instances, unavailable. In some instances, conventional optimization models, are either implicitly defined or define processes as a black box. As a result of poor portability and black box approximations, the conventional optimization methods are unable to account for randomness in raw materials or unexpected breakdowns in a process. 
     One or more embodiments of the present invention address one or more of the above-described shortcomings by providing computer-implemented methods, computing systems, and computer program products to optimize the operation of a production plant of a multi-plant network. The optimization can be achieved by deriving optimal set-points and flows for an entire process, recasting the multi-plant network into a surrogate network of regression transformers that capture the essential process input to output relationships at each plant. The representation is coupled with operational constraints to develop a prediction-optimization formulation plant-wide optimization. 
     Turning now to  FIG.  1   , a manufacturing and process control system  100  is generally shown in accordance with one or more embodiments of the present invention. The control system  100  is a computer-based system that regulates the operation of connected electrical and mechanical devices via the use of feed-forward or feedback control loops. The control system  100  is operable to detect a predictive model type used by a sub-control system of a plant and determine the optimal set values based on the predictive model type. The control system  100  further monitors the inputs and outputs of the process performed at the plant, and adjusts the process based on not achieving the desired throughput. The control system  100  is in operable communication with a network of sub-control systems, in which each sub-control system is an overall control system of a processing plant of a network of processing plants. As illustrated, the control system  100  is in operable communication with a first sub-control system  102 , a second sub-control system  104 , and a third sub-control system  106 . Each of the first, second, and third sub-control system  102   104   106  includes a respective fleet of sensors  108  and at least one actuator  110 . The actuator  110  is operable to move or control a machine involved in at least one step of the process performed at the processing plant Furthermore, at least one sensor of the fleet of sensors  108  is operable for sensing and collecting data from the actuator  110  and transmitting the data back to the control system  100 . It should be appreciated that although  FIG.  1    illustrates three sub-control systems  102   104   106 , the control system  100  can be in operable communication with any number of sub-control systems used to control a network of processing plants. 
     The control system  100  is in operable communication with each respective first, second, and third sub-control system  102   104   106  of each processing plant of the network of processing plants. Each of the first, second, and third sub-control systems  102   104   106  applies a predictive model (e.g., a regression model) to calculate a predicted throughout at the plant. For the purposes of explanation, the first sub-control system  102  is discussed herein. However, this discussion applies to any sub-control system of the network of processing plants. The first sub-control system  102  uses a predictive model, for example, a regression model to collect data from the processing plant and calculate the optimal throughput. The control system  100  can detect the type of predictive model used by the first sub-control system  102  and determine whether the predictive model is a linear-based predictive model or a gradient-based model. The control system  100  can detect the type of predictive model based on accessing a processing plant network database and read the type of predictive model. The control system  100  can also access the first sub-control system  102  and extract characteristics of the predictive model and compare the characteristics to a dictionary of predictive models. 
     For each the sub-control system, a prediction model is used to learn the relationship between the control variables and the output. Whether the sub-control system is applying a linear predictive model or a non-linear predictive model, the control system  100  can model the network of processing plants as follows (the model is illustrated in further detail at  FIG.  3   ). As shown in  FIG.  3   , the plant is represented as a hierarchy structure with L layers having upstream-downstream operation relations.  FIG.  3    is a feed-forward network representation for the plant. A circular node in a layer of the network represents a plant (i.e., a sub-system), where a regression function is built. A rectangular node represents all operational constraints such as maintaining inventory levels and limits on the adjustment of variables from the preceding time period. The relationship in these rectangular nodes are often linear. 
     The predictive model ƒ l  can be a linear-based predictive model or a highly non-linear-based predictive models, in which a gradient is available. The linear-based predictive model type can include linear-based regression functions, decision trees, piece-wise linear-based regression functions, gradient-boosted trees, multivariate adaptive regression splines (MARS), fully connected feed-forward network (FFN), and other similar models. The highly non-linear predictive model (gradient-based predictive model) can include deep neural networks (DNN), support vector machines (SVN), and other similar models. Regardless of the type of predictive model, the predictive model can be built through machine learning and executed on a neural network. The predictive model can be trained using industry-specific historical data. 
     The plant consists of a consecutive network of processes (e.g., sub-control systems), where processes at the l-th layer in  FIG.  3    can be described by the following vector-valued regression function:
 
ƒ l :( z   l−1   ,x   l )∈   k     l−1     +m     l     →y   l ∈   n     i    
 
where x l ∈   m     l    are the controllable variables (e.g., set-points) and z i−1 ∈   k     l−1    are the uncontrollable variable (e.g., the inflow from a previous process or predecessor plant).
 
     The vector y i ∈   nl  are the outflows from processes at the l-th layer. As described above, k 0 =0; that is, the decision variables for the first layer are only set-points. The control system  100  can find the optimal set-points xi and flow rates (y l , z l ) to maximize processing plant production or some other target variable. The relationship between inputs and an output for an industrial plant is often complex, a non-linear prediction model should be used to capture the complexity rather than a linear model. A single-period model can be described as the nonconvex constrained optimization problem below: 
                     min   X         ∑     i   =   1     L             ϕ   l     (     y   l     )               (   1   )                           s   .   t   .               y   1     =       f   1     (     x   1     )       ,                         y   l     =       f   l     (       z     l   -   1       ,     x   l       )       ,       ∀       l     =   2     ,   …         ,   L   ,                       A   l     ⁢     y   l       +       B   l     ⁢     z   l         ≤     b   l       ,       ∀       l     =   1     ,   …         ,     L   -   1     ,                     x   _     l     ≤     x   l     ≤       x   _     l       ,       ∀       l     =   1     ,   …         ,   L   ,         
where X=(x l , . . . , x L ; y L ; z l , . . . , z L−1 ), and the objective function Σ l=1   L ϕ l (y l ) depends on the outputs for each process. Where  x   l  and  x   l  are upper and lower bounds on control set-points. The regression model needs to be cognizant of operational domain constraints. Therefore, model parameter matrices A l , B l  and vector b l  are used to capture interactions between nodes within two consecutive network layers (plants). One example for operational constraints to be included in A lyl +B lzl ≤b l  is the tank storage constraint for the fourth tank, T 4 ,  214  (i.e., l=2), as seen in  FIG.  2   :
 
   u     2   −u   2   0   ≤y   2   −z   2   1   −z   2   2   ≤ū   2 − 2   0 ,  (2)
 
where u 2   0  is the initial storage level,  u   2  and ū 2  are the lower and upper limits for storage tank T 4 .
 
     The second nonconvex program for multi-period planning horizon T being described as follows: 
                     min       X   t     ,     u   l   t             ∑     t   =   1     T             ∑     l   =   1     L             ϕ   l     (     y   l   t     )                 (   3   )                             s   .   t   .             (     1   -   α     )     ⁢     u   l     t   -   1               ≤     u   l   i     ≤       (     1   +   α     )     ⁢     u   l     t   -   1                         (     1   -   β     )     ⁢     x   l     t   -   1         ≤     x   l   t     ≤       (     1   +   β     )     ⁢     x   l     t   -   1                       u   l   t     =       u   l       -   t     -   1       +     y   l   t     -     z   l   t                       u   _     l     ≤     u   l   t     ≤       u   _     l                     X   t     ∈     𝓍   t       ,                 (       l   =   1     ,   …         ,   L   ,     t   =   1     ,   …         ,   T     )     ,         
where x l  is the set of constraints for the t-the period having a form of (1),  u   l  and ū l  are tank storage limits. The parameters α, β∈[0,1] is used to limit the amount of adjustment for a set-point from the time period t to the period t+1. The initial operating points X 0  and u 0  at time t=0 are assumed to be fixed and given, possibly from the current operation time.
 
     Conventional non-linear optimization algorithms devoted to a nonconvex program cannot guarantee the generation of a global minimizer, and they predict based on a local minimizer rather than a global minimizer. In some industries such as the oil and gas industry, a small improvement to solution techniques can have a significant economic impact; for example, a 5% increase in control efficiency can results in billions of dollars of additional revenue for the oil and gas industry. The herein described computer-implemented methods, computing systems, and computer program products utilize a mixed-integer linear program (MILP) to obtain a globally optimal solution to the nonconvex (possibly nonsmooth) problem within a reasonable run time. 
     To solve optimization problems (1) and (3), the present disclosure classifies the type of prediction functions into: piece-wise linear functions such as random forest and differentiable functions such as convolutional neural networks. For piece-wise linear functions, these problems can be solved by using a mixed-integer linear program. For gradient-based functions, these problems can be solved by using a mixed-integer linear program can be solved by a nonlinear optimization algorithm. 
     For partition regressions based on piece-wise linear approach such decision trees, multivariate adaptive regression splines (MARS), and decision lists, nonconvex programs formulate (for example, (1) and (3)) are formulated as a respective mixed-integer linear program (MILP). The mixed-integer linear representation is used for each respective regression function ƒ l . As exemplary illustrations, a MILP model for three models: Decision tree-based ensemble models, multivariate adaptive regression splines (MARS), and feed-forward deep neural networks are shown. 
     A tree ensemble model combines predictions from multiple decision trees ƒ t (x). A decision tree uses a tree-like structure to predict the outcome for an input feature vector x. The t-th regression tree in the ensemble model has the following form: 
                       f   t     (   x   )     =       ∑     i   =   1       M   t           (         a     t   ,   i     T     ⁢   x     +     b     t   ,   i         )     ·     I   ⁡   (     x   ∈     R     t   ,   i         )                 (   4   )               
where R t,I , . . . , Rt, M t  represent a partition of feature space, and I is the indicator function. In practice, individual decision trees often suffer from high variance predictions and can overfit the training data, which lead to a poor out-of-sample predictive accuracy, if there is no restriction in the size of tree. By using the bagging techniques, the tree ensemble regression function outputs predictions by taking the weighted sum of multiple decision trees as:
 
                       f   ⁡   (   x   )     =       ∑     t   =   1     T             λ   t     ⁢       f   t     (   x   )           ,           (   5   )               
where λ t  is the weight for the decision tree ƒ t (x). Parameters a t,i , b t,i , R t,i  and λ t  have been learned from the historical data.
 
     An individual decision tree regression y=ƒ t (x) consists of leaf nodes L and branching nodes B. For each leaf node lϵL, a linear model of the form r l (x)=w l   T +c l  has been learned from the training data based on the points assigned to the leaf node. A branching node lϵB is represented by a hyperplane a l   T x+b l , where if a l   T x+b l &lt;0 then the point x will follow the left branch from the node, otherwise it splits into the right branch. Since the topology of the tree is fixed, for each feature vector x, there is a unique path leading to a leaf node lϵL from the root of the tree. Let N L (l) denote the ancestor nodes of l where the left branch is followed on the path from the root to l, and let N R (l) denote the set of right branch ancestors on the path. The binary variable e l ϵ{0,1}, lϵL, indicates if x is assigned to leaf node l then e l =1. Exactly one leaf node is selected for a data point x, where 
     
       
         
           
             
               
                 
                   
                     
                       ∑ 
                       
                         ℓ 
                         ∈ 
                         ℒ 
                       
                     
                       
                     
                       e 
                       ℓ 
                     
                   
                   = 
                   L 
                 
               
               
                 
                   ( 
                   6 
                   ) 
                 
               
             
           
         
       
     
     To determine the unique path routing to a leaf node, with the help of the indicator variable e t , the following constraints are enforced for modeling the splitting at branching nodes:
 
 a   k   T   x+b   k   ≤M   1 (1 −e   )−ϵ,∀ ∈ ,k∈   L ( )
 
 a   k   T   x+b   k   ≥−M   2 (1 −e   ),∀ ∈ ,k∈   R ( ),  (7)
 
Where M 1  and M 2  are big positive numbers, and a very small ϵ&gt;0. The decision tree regression y=ƒ t (x) can be represented as a mixed-integer bi-linear model
 
                     y   =       ∑     ℓ   ∈   ℒ               e   ℓ     (         w   ℓ   T     ⁢   x     +     e   ℓ       )         ,           (   8   )               
The bi-linear term e l (w l   T +c l ) can be reformulated to a set of linear equations. Assume y l   L ≤w l   T +c l ≤y l   U  for some constants y l   L  and y l   U . Then y l =e l (w l   T +c l ) is equivalent to
 
 y   l   L   e   l   ≤y   l   ≤y   l   U   e   l  
 
 w   l   T   x+e   l   −y   l   U (1− e   l )≤ y   l   ≤w   l   T   x+e   l   −y   l   L (1− e   l ).  (9)
 
     Based on the above, control system  100  uses the following mixed-integer linear representation for a decision tree-based model regression y=ƒ t (x): 
     
       
         
           
             
               
                 
                   
                     f 
                     ⁡ 
                     ( 
                     x 
                     ) 
                   
                   = 
                   
                     
                       ∑ 
                       
                         i 
                         = 
                         1 
                       
                       T 
                     
                       
                     
                       
                         λ 
                         t 
                       
                       ⁢ 
                       
                         
                           ∑ 
                           
                             ℓ 
                             ∈ 
                             ℒ 
                           
                         
                         
                           
                             y 
                               
                           
                           ℓ 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   10 
                   ) 
                 
               
             
           
         
       
       
         
           
             
               
                 
                   s 
                   . 
                   t 
                   . 
                 
               
               
                 
                   
                     
                       ∑ 
                       
                         ℓ 
                         ∈ 
                         ℒ 
                       
                     
                       
                     
                       e 
                       ℓ 
                     
                   
                   = 
                   1 
                 
               
             
           
         
       
       
         
           
             
               
                 
                   
                     a 
                     k 
                     T 
                   
                   ⁢ 
                   x 
                 
                 + 
                 
                   b 
                   k 
                 
               
               ≤ 
               
                 
                   
                     M 
                     1 
                   
                   ( 
                   
                     1 
                     - 
                     
                       e 
                       ℓ 
                     
                   
                   ) 
                 
                 - 
                 e 
               
             
             , 
             
               ∀ 
               
                 ℓ 
                 ∈ 
                 ℒ 
               
             
             , 
             
               k 
               ∈ 
               
                 
                   𝒩 
                   L 
                 
                 ( 
                 ℓ 
                 ) 
               
             
           
         
       
       
         
           
             
               
                 
                   
                     a 
                     k 
                     T 
                   
                   ⁢ 
                   x 
                 
                 + 
                 
                   b 
                   k 
                 
               
               ≥ 
               
                 - 
                 
                   
                     M 
                     2 
                   
                   ( 
                   
                     1 
                     - 
                     
                       e 
                       ℓ 
                     
                   
                   ) 
                 
               
             
             , 
             
               ∀ 
               
                 ℓ 
                 ∈ 
                 ℒ 
               
             
             , 
             
               k 
               ∈ 
               
                 
                   𝒩 
                   R 
                 
                 ( 
                 ℓ 
                 ) 
               
             
           
         
       
       
         
           
             
               
                 y 
                 ℓ 
                 L 
               
               ⁢ 
               
                 e 
                 ℓ 
               
             
             ≤ 
             
               y 
               ℓ 
             
             ≤ 
             
               
                 y 
                 ℓ 
                 U 
               
               ⁢ 
               
                 e 
                 ℓ 
               
             
           
         
       
       
         
           
             
               
                 
                   w 
                   ℓ 
                   T 
                 
                 ⁢ 
                 x 
               
               + 
               
                 e 
                 ℓ 
               
               - 
               
                 
                   y 
                   ℓ 
                   U 
                 
                 ( 
                 
                   1 
                   - 
                   
                     e 
                     ℓ 
                   
                 
                 ) 
               
             
             ≤ 
             
               y 
               ℓ 
             
             ≤ 
             
               
                 
                   w 
                   ℓ 
                   T 
                 
                 ⁢ 
                 x 
               
               + 
               
                 e 
                 ℓ 
               
               - 
               
                 
                   y 
                   ℓ 
                   L 
                 
                 ( 
                 
                   1 
                   - 
                   
                     e 
                     ℓ 
                   
                 
                 ) 
               
             
           
         
       
       
         
           
             
               
                 e 
                 ℓ 
               
               ∈ 
               
                 { 
                 
                   0 
                   , 
                   1 
                 
                 } 
               
             
             , 
             
               ∀ 
               
                 ℓ 
                 ∈ 
                 ℒ 
               
             
             , 
           
         
       
     
     For a multivariate adaptive regression spline (MARS), the control unit  100  considers the regression spine fitting of degree 1: h(x)=α 0 +Σ i=1   N α i h i (x), where α 1  are scalars and h i (x)=max{w i   T +c 1 ,0}. The control unit  100  applies the following linear representation: 
     
       
         
           
             
               
                 
                   
                     h 
                     ⁡ 
                     ( 
                     x 
                     ) 
                   
                   = 
                   
                     
                       α 
                       0 
                     
                     + 
                     
                       
                         ∑ 
                         
                           i 
                           = 
                           1 
                         
                         N 
                       
                         
                       
                         
                           α 
                           i 
                         
                         ⁢ 
                         
                           y 
                           i 
                         
                       
                     
                   
                 
               
               
                 
                   ( 
                   11 
                   ) 
                 
               
             
           
         
       
       
         
           
             
               
                 
                   s 
                   . 
                   t 
                   . 
                 
               
               
                 
                   
                     
                       y 
                       i 
                     
                     ≥ 
                     
                       
                         
                           w 
                           i 
                           T 
                         
                         ⁢ 
                         x 
                       
                       + 
                       
                         c 
                         i 
                       
                     
                   
                   , 
                   
                     i 
                     = 
                     1 
                   
                   , 
                   … 
                       
                   , 
                   N 
                 
               
             
           
         
       
       
         
           
             
               
                 y 
                 i 
               
               ≤ 
               
                 
                   ( 
                   
                     
                       
                         w 
                         i 
                         T 
                       
                       ⁢ 
                       x 
                     
                     + 
                     
                       c 
                       i 
                     
                   
                   ) 
                 
                 + 
                 
                   
                     M 
                     3 
                   
                   ⁢ 
                   
                     e 
                     i 
                   
                 
               
             
             , 
             
               i 
               = 
               1 
             
             , 
             … 
                 
             , 
             N 
           
         
       
       
         
           
             
               
                 y 
                 i 
               
               ≤ 
               
                 
                   M 
                   4 
                 
                 ( 
                 
                   1 
                   - 
                   
                     e 
                     i 
                   
                 
                 ) 
               
             
             , 
             
               i 
               = 
               1 
             
             , 
             … 
                 
             , 
             N 
           
         
       
       
         
           
             
               
                 e 
                 i 
               
               ∈ 
               
                 { 
                 
                   0 
                   , 
                   1 
                 
                 } 
               
             
             , 
             
               
                 y 
                 i 
               
               ≥ 
               0 
             
             , 
             
               i 
               = 
               1 
             
             , 
             … 
                 
             , 
             N 
             , 
           
         
       
     
     For a feed-forward deep neural network (DNN), for example a rectified linear unit (ReLU) deep neural network with K+1 layers. Indexed from 0 to K, which is used to model a non-linear function ƒ(x):    n     0   →   n     K    with n K =1. For each hidden layer 1≤k≤K−1, the output vector x k  is computed as x k =σ(W k x k−1 +b k ), where σ is the ReLU function, W k ϵ   n     K     ×n     k−1   , b k ϵ   n     K   . For each layer k, assume there exists L k , U k ϵ  such that L k e k ≤W k x k−1 +b k ≤U k e k , e k =(1, . . . , 1)ϵ   n     K   . Therefore, the MILP model for a deep neural network is: 
                               x   k     -     s   k       =         W   k     ⁢     x     k   -   1         +     b   k         ,             k   =   1     ,   …         ,     K   -   1                   (   12   )                             x   k     ,       s   k     ≥   0     ,             k   =   1     ,   …         ,     K   -   1                                 z   k     ∈       {     0   ,   1     }       n   k         ,             k   =   1     ,   …         ,     K   -   1                                 x   k     ≤       U   k     ⁢     z   k         ,             k   =   1     ,   …         ,     K   -   1                                 s   k     ≤       -     L   k       ⁢     (     1   -     z   k       )         ,             k   =   1     ,   …         ,     K   -   1                         y   =       f   ⁡   (     x   0     )     =         W   K     ⁢     x     K   -   1         +     b   K           ,         
where two new variables are introduced: s k ϵ   +   n     k    and z k ϵ{0,1} n     k   . The control unit  100  can use the same approach to have a MILP representation for other RE-type activation functions, including Leaky ReLU and Parametric ReLU.
 
     The control unit  100  can further utilize a model to determine whether additional materials are needed to be added or removed for a process. For a regression function in the form y=f(x), the i-th material commitment is as follows:
 
 x   i ∈[ l,u ] or  x   i =0  (13)
 
For some upper bound u and some lower bound l. The condition can be modeled as:
 
                     l   ⋆   s     ≤     x   i     ≤     u   ⋆   s             (   14   )                       t   -     u   ⁡   (     1   -   s     )       ≤     x   i     ≤     t   -     l   ⁡   (     1   -   s     )                   l   ≤   t   ≤   u               s   ∈     {     0   ,   1     }           
These equations can be added to a MLIP model. The control unit  100  can then cause a physical material to be added or removed from a process based on the solution.
 
     The control unit  100  can further determine whether a process needs to be enabled or disabled. For a process commitment either y=ƒ(x) or y=0. The control unit  100  assumes that the capacity of a process when operating is a≤ƒ(x)≤b. The following equations are then added to the model a plant&#39;s status: 
                     a   ⋆   v     ≤   y   ≤     b   ⋆   v             (   15   )                         f   ⁡   (   x   )     -     b   ⁡   (     1   -   v     )       ≤   y   ≤       f   ⁡   (   x   )     -     a   ⁡   (     1   -   v     )                   a   ≤     f   ⁡   (   x   )     ≤   b               v   ∈     {     0   ,   1     }           
The control unit  100  can then solve for the MILP model to determine whether a process should be enabled or disabled.
 
     The control system  100  can solve for the set variable and flow variable based on applying a mixed-integer linear program, when the predictive models are piece-wise linear functions such as random forest, multivariate adaptive regression splines and fully connected feed-forward network. 
     In the event that the sub-control system is applying a non-linear predictive model, the control system  100  can apply the following primal-dual optimization algorithm. The control system  100  can solve the above referenced model by a two-level augmented Lagrangian method when gradients of ƒ l  are available. The control system  100  can treat non-linear local constraints y l =ƒ l (⋅) and the linear coupling constraints A lyl +B lzl ≤b l  differently. The control system  100  applies an augmented Lagrangian method to non-linear coupling constraints in the outer loop. The control system  100  applies a multi-block alternating direction method of multipliers (ADMM) to the linear coupling constraints at the inner level. 
     In order to apply a multi-block ADMM to a single period-model, the control system  100  adds a slack variable u l  for the linear constraint, such that
 
 A   lyl   +B   lzl   +v   l   −b   l   +u   l =0; and  u   l =0.  (16)
 
The control system  100  then solves the optimization problem using an augmented Lagrangian method with both constraints, y l =ƒ l (⋅) and u l =0 by the following algorithm. First, the control unit  100  can consider the following subproblem in the augmented Lagrangian method
 
                     min   ⁢         F   ⁢     (       u   ^     ,     v   ^     ,     x   ^     ,     y   ^     ,     z   ^     ,       λ   ^     k     ,         v   ^     k     ;     β   k       ,     γ   k       )       =       ϕ   ⁡   (     y   L     )     +         (     λ   1   k     )     T     ⁢     (       y   1     -       f   1     (     x   1     )       )       +         β   k     2     ⁢              y   1     -       f   1     (     x   1     )            2       +       ∑     l   =   2     L               (     λ   l   k     )     T     ⁢     (       y   l     -       f   l     (       z     l   -   1       ,     x   l       )       )         +         β   k     2     ⁢       ∑     l   =   2     L                  y   l     -         f   l     (       z     l   -   1       ,     x   l       )     )        2       +       ∑     l   =   1       L   -   1                 (     v   l   k     )     T     ⁢     u   l         +         γ   k     2     ⁢       ∑     l   =   1       L   -   1                    u   l          2                           (   17   )                                                        s   .   t   .                ⁢     A   l       ⁢     y   l       +       B   l     ⁢     z   l       +     v   l     -     b   l     +     u   l       =   0     ,               ∀   l     =   1     ,   …         ,     L   -   1     ,                                                   x   _     l     ≤     x   l     ≤       x   _     l       ,         y   _     l     ≤     y   l     ≤       y   _     l       ,       ∀   l     =   1     ,   …         ,   L   ,                                             z   _     l     ≤     z   l     ≤       z   _     l       ,       v   l     ≥   0     ,       ∀   l     =   1     ,   …         ,     L   -   1.             
The augmented Lagrangian method is provided below where h({circumflex over (x)}, ŷ, {circumflex over (z)})=[y 1 −ƒ 1 (y 1 ), y 2 −ƒ 2 (z 1 , y 2 ), . . . y l −ƒ L (z L−1 ,y L )].
 
     
       
         
           
               
             
               
                   
               
               
                 ALGORITHM 1: Two-level ALM 
               
               
                   
               
             
            
               
                   
               
            
           
           
               
               
               
            
               
                 1: 
                 Initialize starting points (û 0 ,{circumflex over (v)} 0 ,{circumflex over (x)} 0 ,ŷ 0 ,{circumflex over (z)} 0 ); 
                   
               
               
                   
                 β 1 , γ 1  &gt; 0, ω ∈ (0, 1), r &gt; 1; index k ← 1; 
                   
               
               
                 2: 
                 while some stopping criterion is not satisfed do 
                   
               
               
                 3: 
                  /* Inner level problem */ 
                   
               
               
                 4: 
                  Solve (20) by a multi-block ADMM to get 
                   
               
               
                   
                  (û k ,{circumflex over (v)} k ,{circumflex over (x)} k  ŷ k ,{circumflex over (z)} k ,μ k ) for ({circumflex over (λ)} k ,β k ) and ({circumflex over (v)} k ,γ k ); 
                   
               
               
                 5: 
                  if ∥h({circumflex over (x)} k ,ŷ k ,{circumflex over (z)} k )∥ ≤ ω∥h({circumflex over (x)} k − 1 ,ŷ k −1 ,{circumflex over (z)} k −1 )∥ then 
                   
               
               
                 6: 
                   {circumflex over (λ)} k +   1  ← {circumflex over (λ)} k  + β k h({circumflex over (x)} k , ŷ k , {circumflex over (z)} k ), β k +   1  ← β k ; 
                   
               
               
                 7: 
                  else 
                   
               
               
                 8: 
                   {circumflex over (λ)} k +   1  ← {circumflex over (λ)} k , β k +   1  ← rβ k ; 
                   
               
               
                 9: 
                  end if 
                   
               
               
                 10: 
                  if ∥û k ∥ ≤ ω∥ū k −   1 ∥ then 
                   
               
               
                 11: 
                   {circumflex over (v)} k + 1  ← {circumflex over (v)} k  + γ k  û k , γ k + 1  ← γ k ; 
                   
               
               
                 12: 
                  else 
                   
               
               
                 13: 
                   {circumflex over (v)} k + 1  ← {circumflex over (v)} k , γ k + 1  ← rγ k ; 
                   
               
               
                 14: 
                  end if 
                   
               
               
                 15: 
                  k ← k − 1; 
                   
               
               
                 16: 
                 end while 
                   
               
               
                   
                   
                 (18) 
               
               
                   
               
            
           
         
       
     
     The theoretical convergence results for the two-level augmented Lagrangian method when combining with a multi-block ADMM can be seen as follows. The first assumption is that F(û,{circumflex over (v)},{circumflex over (x)},ŷ,{circumflex over (z)},{circumflex over (λ)} k ,{circumflex over (v)} k ,{circumflex over (β)} k ,{circumflex over (γ)} k ) is continuously differentiable. The second assumption is that the inner level ADMM outputs an approximate solution {(û k , v k , {circumflex over (x)} k ,ŷ k ,{circumflex over (z)} k ,û k ) to the following conditions 
                       d   1   k     ∈       ∇     f   ⁡   (         x   ^     k     ,       y   ^     k     ,       z   ^     k       )       +       ∇       h   ⁡   (         x   ^     k     ,       y   ^     k     ,       z   ^     k       )     T       ⁢     (         λ   _     k     +       β   k     ⁢     h   ⁡   (         x   ^     k     ,       y   ^     k     ,       z   ^     k       )         )       +       N   𝒳     (         x   ^     k     ,       y   ^     k     ,       z   ^     k       )     +       C   T     ⁢     μ   k           ,       d   2   k     ∈       μ   k     +       N     𝒳   ^       (       v   ^     k     )         ,   
         A   ⁢       y   ^     k       +     B   ⁢       z   ^     k       +       v   ^     k     -   b   +       u   ^     k       =     d   3   k       ,           v   ^     k     +       γ   k     ⁢       u   ^     k       +     μ   k       =   0     ,           (   19   )               
Where
 
lim k→∞   ∥d   i   k ∥=0 for  i= 1,2,3,ƒ( {circumflex over (x)},ŷ,{circumflex over (z)} )=φ( y   L )  (20)
 
Also where C=[0 A B], which is a matrix associated with ({circumflex over (x)}, ŷ, {circumflex over (z)}) in the linear constraint. X and  X  are the box constraints for ({circumflex over (x)}, ŷ, {circumflex over (z)}) and {circumflex over (v)}, respectively.
 
     For the last block of inner level ADMM, the linear constraint related to û is an identity matrix and the objective function is a convex and quadratic function. Under mild conditions on the functions f, h and the matrices A, B, the multi-block ADMM algorithm can converge (subsequently) to a stationary point of the subproblem (17) and therefore the second Assumption can be satisfied. 
     The multi-period formulation (3) can be compactly expressed as follows: 
                     min       w   1     ,   …         ,     w   T             ∑     t   =   1     T             ϕ   i     (     w   t     )               (   21   )                           s   .   t   .               w   l     ∈     Ω   t       ,     t   =   1     ,   …         ,   T                             M     t   -   1       ⁢     w     t   -   1         +       N   t     ⁢     w   t       +     c   t       =   0     ,         
where w=(X, u, p, q, r, s) and the constraint set Ω t  is related to X t , bounds for u, and positivity constraints for p, q, r, and s. The parameters M t , N t , and c t  are determined from the coupling constraints.
 
     The phrases “neural network” and “machine learning” broadly describes a function of electronic systems that learn from data. A machine learning system, engine, or module can include a machine learning algorithm that can be trained, such as in an external cloud environment (e.g., a cloud computing environment), to learn functional relations between inputs and outputs that are currently unknown. In one or more embodiments, machine learning functionality can be implemented using a predictive model, having the capability to be trained to perform a currently unknown function. In machine learning and cognitive science, neural networks are a family of statistical learning models inspired by the biological neural networks of animals, and in particular, the brain. Neural networks can be used to estimate or approximate systems and functions that depend on a large number of inputs. 
     The predictive model can be embodied as so-called “neuromorphic” systems of interconnected processor elements that act as simulated “neurons” and exchange “messages” between each other in the form of electronic signals. Similar to the so-called “plasticity” of synaptic neurotransmitter connections that carry messages between biological neurons, the connections in the predictive model that carry electronic messages between simulated neurons are provided with numeric weights that correspond to the strength or weakness of a given connection. During training, the weights can be adjusted and tuned based on experience, making the control system  100  adaptive to inputs and capable of learning. After being weighted and transformed by a function determined by the network&#39;s designer, the activation of these input neurons are then passed to other downstream neurons, which are often referred to as “hidden” neurons. This process is repeated until an output neuron is activated. The activated output neuron determines which character was read. 
     Referring to  FIG.  2   , an exemplary diagram  200  of a network of oil processing plants in accordance with one or more embodiments of the present invention is shown. In connection with  FIG.  2   , optimization can involve devising production strategies that maximize synthetic crude oil (SCO) production, under both normal operations as well as unplanned upsets and breakdowns. Starting with mined oil sands, the first step is to extract bitumen as froth and store it in a storage tank. The first plant, P 1 ,  202  and the second plant, P 2 ,  202  are two parallel froth and diluted bitumen production plants. The first tank, T 1 ,  206  and the second tank, T 2 ,  208  are intermediate storage tanks. The third tank, T 3 ,  210  stores diluted bitumen. The diluted bitumen is passed through an upgrading third plant, P 3 ,  212 , to produce low-quality SCO that gets stored in a fourth tank, T 5 ,  214 . This is an intermediate product that is either sold directly to refineries or is further processed in a fourth plant, P 5 ,  216  and a fifth plant, P 6 ,  218  that are parallel upgraders that produce different grades of synthetic crude oil. 
     Each of the first, second, third, fourth, and fifth plants  202 ,  204 ,  212 ,  216 ,  218  are controlled by a sub-control system. Each sub-control system employs a respective predictive model to calculate predicted throughputs at each respective plant. The predictive models can each be a same type of predictive model or at least one of the first, second, third, fourth, and fifth plants  202 ,  204 ,  212 ,  216 ,  218  can employ a different type of predictive model. 
     Referring to  FIG.  3   , an exemplary network representation  300  of processing plants described in  FIG.  2    in accordance with one or more embodiments of the present invention is shown. The representation  300  is viewed as a hierarchal structure with L layers having upstream-downstream operation relations. The first node  302 , second node  304 , third node  306 , fourth node  308 , and fifth node  310  represent a respective plant (i.e., a sub-system) from  FIG.  2   , where a predictive model is built. The first rectangular node  312  and the second rectangular node  314  represents all the operational constraints, such as maintaining inventory levels and limits on the adjustment of variables from the preceding time period. The relationship in these rectangular nodes  312   314  are often linear. In some embodiments of the present invention, these rectangular nodes  312   314  represent a volume of fluid in one or more storage tanks. It should be appreciated that  FIGS.  2  and  3    only illustrate one particular arrangement of a network of plants, however, the herein described computer-implemented methods, computing systems, and computer programs products are applicable to any number of network arrangements. 
     Referring to  FIG.  4   , a computer-implemented process  400  for optimizing a manufacturing and process control system in accordance with one or more embodiments of the present invention is shown. In accordance with embodiments of the invention, the process  400  is implemented by the control system  100 . At block  402 , the control system  100  determines a type of predictive model used by a sub-control system of a plant of a network of plants. The network of plants being arranged for the purposes of completing separate processing stages of one or more products. The predictive model can be a linear-based predictive model or a non-linear-based predictive model. The control system  100  can determine the type of predictive model by accessing the sub-control system and analyzing the predictive model. The control system  100  can also access a network database with a description of each predictive model used at each plant of a network of plants. 
     At block  404 , the control system  100  selects an optimization algorithm based on the type of predictive model used by the sub-control system of the plant. Regardless of type of predictive model, the control system  100  used the selected optimization algorithm to solve a model of the plant to determine optimal set variables and outflow values at the plant. In the event that the plant uses a linear-based predictive model, the control system  100  uses a mixed-integer linear program to solve for the optimization. If the plant uses a non-linear-based predictive model, the control system takes into account both linear and non-linear constraints and solves the optimization through an augmented Lagrangian method. 
     At block  406 , the control system  100  solves for the optimum values for the set-points and outflow values. As described above, the control system  100  uses a mixed-integer linear program to solve for the optimization of a linear model. Alternatively, the control system  100  uses an augmented Lagrangian method to solve the optimization of a non-linear predictive model. 
     At block  408 , the control system  100  applies the optimum set-points and outflow values to determine and optimum throughput value for the plant. The control system  100  further receives actual throughput values from sensors arranged on or about the plant. The sensor values can include a volume, quality, or time of processing of fluid processed by the plant. The control system  100  further compares the measured throughput to the predicted throughput. 
     At block  410 , the control system  100  determines whether the measured throughput values are within a threshold value of the predicted threshold value. The threshold value can include an upper bound and a lower bound. The control system  100  can further determine whether the measured value is within the upper bound and the lower bound. At block  412 , if the measured value is within the upper bound and the lower bound, the control system  100  takes no further action. However, if the measured value is not within the upper bound and lower bound, the control system  100  can suspend one or more processes at the plant. The control system  100  can cause an actuator at the plant to cease to function, thereby suspending the process. 
     It is understood that the present disclosure is capable of being implemented in conjunction with any other type of computing environment now known or later developed. For example,  FIG.  5    depicts a block diagram of a processing system  500  for implementing the techniques described herein. In examples, the processing system  500  has one or more central processing units (processors)  521   a ,  521   b ,  521   c , etc. (collectively or generically referred to as processor(s)  521  and/or as processing device(s)). In aspects of the present disclosure, each processor  521  can include a reduced instruction set computer (RISC) microprocessor. Processors  521  are coupled to system memory (e.g., random access memory (RAM)  524 ) and various other components via a system bus  533 . Read only memory (ROM)  522  is coupled to system bus  533  and may include a basic input/output system (BIOS), which controls certain basic functions of the processing system  500 . 
     Further depicted are an input/output (I/O) adapter  527  and a network adapter  526  coupled to the system bus  533 . I/O adapter  527  may be a small computer system interface (SCSI) adapter that communicates with a hard disk  523  and/or a storage device  525  or any other similar component. I/O adapter  527 , hard disk  523 , and storage device  525  are collectively referred to herein as mass storage  534 . Operating system  540  for execution on processing system  500  may be stored in mass storage  534 . The network adapter  526  interconnects system bus  533  with an outside network  536  enabling processing system  500  to communicate with other such systems. 
     A display (e.g., a display monitor)  535  is connected to the system bus  533  by display adapter  532 , which may include a graphics adapter to improve the performance of graphics intensive applications and a video control. In one aspect of the present disclosure, adapters  526 ,  527 , and/or  532  may be connected to one or more I/O busses that are connected to the system bus  533  via an intermediate bus bridge (not shown). Suitable I/O buses for connecting peripheral devices such as hard disk controls, network adapters, and graphics adapters typically include common protocols, such as the Peripheral Component Interconnect (PCI). Additional input/output devices are shown as connected to system bus  533  via user interface adapter  528  and display adapter  532 . An input device  529  (e.g., a keyboard, a microphone, a touchscreen, etc.), an input pointer  530  (e.g., a mouse, trackpad, touchscreen, etc.), and/or a speaker  531  may be interconnected to system bus  533  via user interface adapter  528 , which may include, for example, a Super I/O chip integrating multiple device adapters into a single integrated circuit. 
     In some aspects of the present disclosure, the processing system  500  includes a graphics processing unit  537 . Graphics processing unit  537  is a specialized electronic circuit designed to manipulate and alter memory to accelerate the creation of images in a frame buffer intended for output to a display. In general, graphics processing unit  537  is very efficient at manipulating computer graphics and image processing and has a highly parallel structure that makes it more effective than general-purpose CPUs for algorithms where processing of large blocks of data is done in parallel. 
     Thus, as configured herein, the processing system  500  includes processing capability in the form of processors  521 , storage capability including system memory (e.g., RAM  524 ), and mass storage  534 , input means such as keyboard  529  and mouse  530 , and output capability including speaker  531  and display  535 . In some aspects of the present disclosure, a portion of system memory (e.g., RAM  724 ) and mass storage  534  collectively store the operating system  540  to coordinate the functions of the various components shown in the processing system  500 . 
     Various embodiments of the invention are described herein with reference to the related drawings. Alternative embodiments of the invention can be devised without departing from the scope of this invention. Various connections and positional relationships (e.g., over, below, adjacent, etc.) are set forth between elements in the following description and in the drawings. These connections and/or positional relationships, unless specified otherwise, can be direct or indirect, and the present invention is not intended to be limiting in this respect. Accordingly, a coupling of entities can refer to either a direct or an indirect coupling, and a positional relationship between entities can be a direct or indirect positional relationship. Moreover, the various tasks and process steps described herein can be incorporated into a more comprehensive procedure or process having additional steps or functionality not described in detail herein. 
     One or more of the methods described herein can be implemented with any or a combination of the following technologies, which are each well known in the art: a discrete logic circuit(s) having logic gates for implementing logic functions upon data signals, an application specific integrated circuit (ASIC) having appropriate combinational logic gates, a programmable gate array(s) (PGA), a field programmable gate array (FPGA), etc. 
     For the sake of brevity, conventional techniques related to making and using aspects of the invention may or may not be described in detail herein. In particular, various aspects of computing systems and specific computer programs to implement the various technical features described herein are well known. Accordingly, in the interest of brevity, many conventional implementation details are only mentioned briefly herein or are omitted entirely without providing the well-known system and/or process details. 
     In some embodiments, various functions or acts can take place at a given location and/or in connection with the operation of one or more apparatuses or systems. In some embodiments, a portion of a given function or act can be performed at a first device or location, and the remainder of the function or act can be performed at one or more additional devices or locations. 
     The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, element components, and/or groups thereof. 
     The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. The embodiments were chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated. 
     The diagrams depicted herein are illustrative. There can be many variations to the diagram or the steps (or operations) described therein without departing from the spirit of the disclosure. For instance, the actions can be performed in a differing order or actions can be added, deleted or modified. Also, the term “coupled” describes having a signal path between two elements and does not imply a direct connection between the elements with no intervening elements/connections therebetween. All of these variations are considered a part of the present disclosure. 
     The following definitions and abbreviations are to be used for the interpretation of the claims and the specification. As used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having,” “contains” or “containing,” or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a composition, a mixture, process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but can include other elements not expressly listed or inherent to such composition, mixture, process, method, article, or apparatus. 
     Additionally, the term “exemplary” is used herein to mean “serving as an example, instance or illustration.” Any embodiment or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments or designs. The terms “at least one” and “one or more” are understood to include any integer number greater than or equal to one, i.e. one, two, three, four, etc. The terms “a plurality” are understood to include any integer number greater than or equal to two, i.e. two, three, four, five, etc. The term “connection” can include both an indirect “connection” and a direct “connection.” 
     The terms “about,” “substantially,” “approximately,” and variations thereof, are intended to include the degree of error associated with measurement of the particular quantity based upon the equipment available at the time of filing the application. For example, “about” can include a range of ±8% or 5%, or 2% of a given value. 
     The present invention may be a system, a method, and/or a computer program product at any possible technical detail level of integration. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention. 
     The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire. 
     Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device. 
     Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++, or the like, and procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user&#39;s computer, partly on the user&#39;s computer, as a stand-alone software package, partly on the user&#39;s computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user&#39;s computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instruction by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention. 
     Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions. 
     These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks. 
     The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks. 
     The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions. 
     The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments described herein.