# Comparison of Single Control Loop Performance Monitoring Methods
## 3 Results  
### Case 1. Identification of Faults with Different CPM Methods  
Kullback-Leibler divergence, histogram intersection, and Euclidean distance are used in this case to compare the selected reference dataset to a testing set selected from a sliding window of one day (8640 data points). The training data are selected from the beginning of the simulation with a size of 50,000 data points. The start of the dataset is known to represent the normal behavior of the process and can thus provide an accurate reference for the methods. Additionally, the number of bins for the histograms used is set to 8.  
In practice, identifying normal behavior of the process is a challenge for accurate monitoring in industrial processes. For this purpose, the utilized simulator allows for a fault-free scenario when identifying the normal process behavior and provides means for estimating the performance of the chosen CPM indices.  
In Figure 5, the differences between the reference data (first 50,000 data points) and a testing set selected with a sliding window of size 8640 (1 day) can be seen for the histogram intersection method. For the _HI_ method, the high index values indicate good performance as the statistical properties of the test set are close to the reference data obtained during a normal control loop performance. Slight changes in the metrics occur even in normal operation (days 6-15), due to the randomly chosen setpoints. During the periods with faults, the index clearly deviates from the values in normal operation. After the simulated faults have ceased, the index returns to nominal value range, indicating a good performance of the CPM method. Similar performance was observed for the Kullback-Leibler divergence and Euclidean distance metrics.  
Figure 4: Faults and setpoints during a simulation run of 40 days.  
However, with these methods, properties of the training data can affect the resulting metrics. Deviations from the training data have a significant effect and retraining may be necessary to adapt to an evolving process environment. Moreover, selection of the size of the sliding window for the metrics affects the resolution of the results. With a larger window size, the observation of a fault might be delayed as a lower proportion of the window is from faulty data. Determining alarm limits is dependent on the process as tolerances can vary.  
Identification of an individual fault was considered by comparing a period of the simulation where one fault was present with an equal-sized duration from the fault-free period. In Figure 6a, this comparison for the OCE method was performed and presented as a boxplot for normal, fault-free data, and separately for the five fault scenarios. A high \(OCE_{total}\) value corresponds to good control, whereas lower values indicate decreased control loop performance. It can be seen that the OCE method can separate all of the faulty situations from the normal operation, as the notches of the plot (95% confidence) do not overlap. The process was repeated for all metrics (boxplots presented in Appendix A, Figures A1-A7) and qualitatively compiled in Table 1. The fault was considered to be identified when the index significantly differed from the normal behavior in the expected direction. As indicated in Figure 6b, _ISE_ shows a lower index value for the fault scenarios' quantization and valve stiction, although it is expected that the integrated error value would increase in the presence of fault. According to Table 1, among the tested CPM methods, _KL_, _HI_, _ED_, \(OCE_{p}\), and \(OCE_{total}\) could detect the decreased control loop performance simulated in Case 1.  
Figure 5: Histogram intersection between reference data and a sliding window of 1 day. Individual faults enabled between the first two vertical lines. Faults disabled after the last vertical line. The lines overlap on days 6–15 and 32–40.