## 5 Integration of Pruning for Stream-Lined Control
This elimination of weights continues until 270 weights have been eliminated and only five weights remain. The resulting percent change in the fitness function as a function of the removed weights is shown in Figure 7(a). The resulting number of removed filters as a function of removed weights is shown in Figure 7(b). Next, the optimal brain surgeon (OBS) algorithm was considered [41]. OBS uses the Hessian matrix, \(H\), to create a vector of saliencies, \(L\), where each \(q\)th entry in the vector is defined as  
\[L_{q}=\frac{1}{2}\frac{w_{q}^{2}}{[H^{-1}]_{qq}}, \tag{20}\]  
Figure 8: Percent difference in the fitness function from the unpruned, full-state weighted control matrix for the three pruning methods ((**a**), note that a negative value indicates an improved performance) and the total number of filters removed through pruning (**b**). given that \(w_{q}\) is the \(q\)th weight in the weight vector, \(\hat{W}\). The inverse Hessian matrix is calculated as  
\[H_{r+1}^{-1}=H_{r}^{-1}-\frac{H_{r}^{-1}\cdot X^{[r+1]}\cdot X^{[r+1]T}\cdot H_{ r}^{-1}}{X^{[r+1]}\cdot X^{[r+1]T}\cdot H_{r}^{-1}}, \tag{21}\]  
such that \((\cdot)^{-1}\) is the inverse function, \(r\) is the iteration number, and \(\mathbf{X}\) is a vector of partial derivatives. As the standard application of OBS is for multi-layer networks, \(\mathbf{X}\) is truncated to only include derivatives with respect to the input-to-output layers, similar to the hidden-to-output weights discussed in [41]. For the weighted control algorithm, the output layer uses a linear activation function, which reduces it down to \(X^{[r]T}=\hat{W}*z(r)\), where \(z(r)\) is the output of the bandpass filters on the \(r\)th iteration. In this instance, the number of iterations equals the number of time-steps in the simulation. As specified in [41], the weight with the smallest saliency is used to update all other weights by deriving a weight change vector,  
\[\delta w=-\frac{w_{q}}{\left[H^{-1}\right]_{qq}}H^{-1}\cdot e_{q}\, \tag{22}\]  
and adding this to the weight vector, \(\hat{W}\). In Equation (22), \(e_{q}\) is the unit vector in weight space corresponding to the \(q\)th weight \(w_{q}\). After updating all weights, the identified weight is removed from the matrix. The algorithm continues to eliminate weights in this manner until there is an unacceptable amount of incurred error, at which point the network could be retrained. To understand the effect of removing weights, the stopping error for the algorithm was incrementally increased without implementing any retraining until 270 weights had been removed. The resulting percent change in the objective function as a function of the removed weights is shown in Figure 8a. The resulting number of removed filters as a function of the removed weights is shown in Figure 8b. Finally, the third method that was considered for eliminating extraneous weights is the minimum error method. In this method, one weight is temporarily removed and the reduced weighted control matrix is evaluated using the fitness function (Equation (19)). This weight is then inserted back into the matrix, the next weight is removed, and the fitness function is evaluated. After all weights are considered, the weight that results in the smallest fitness function is permanently removed. This is repeated for all remaining weights until 270 weights have been removed. The resulting percent change in the fitness function as a function of the removed weights is shown in Figure 8a. The resulting number of removed filters as a function of the removed weights is shown in Figure 8b. As can be seen from Figure 8a, there is a non-linear relationship between the weights and the fitness function for all three methods. In particular, up to 223 weights can be removed for any of the three methods and less than a 5% degradation in control effectiveness occurs when comparing the full state to the pruned state. After that, the percent difference rapidly increases for the minimal-value deletion pruning and OBS methods.