# Classifying X-Ray Tube Malfunctions: AI-Powered CT Predictive Maintenance System
## 6 Experiments
### Experiments with CNN  
For CNNs, we decided to try two different approaches:  
The first approach consists of giving a CNN trimmed and normalised data. Here, the goal is for CNNs to learn how to extract features themselves and find the hidden patterns in the data. The second approach is based on the assumption that DWT coefficients might not be representative enough of the features or may contain unnecessary features. In many works, DWT coefficients are only used as an intermediate step and more features are extracted over time. In this case, the goal is for a CNN to learn additional needed processing on top of the DWT. Another possible upside is that data processed with DWT could be easier to learn for CNNs and allow for easier/faster convergence. During the experiments, we constantly monitored the validation accuracy, and the best model of every epoch was always saved after the epoch and loaded during the final evaluation. Together with limiting the number of epochs, it has the approximate effects of the early stopping mechanism. As a loss function, binary cross-entropy was utilised with the Adam optimiser. For a VGG-type CNN, we utilise 5-fold cross-validation with possible duos/triplets of filters in consecutive blocks:  
* 8, 16
* 16, 32
* 32, 64
* 64, 128
* 8, 16, 32
* 16, 32, 64 \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Parameters** & **Wavelet** & **Accuracy/Micro F1-Score** \\ \hline LR & c = 1, penalty = L1 & coif10 & 84\% \\ \hline SVC & 16 different setups & coif10 & 84\% \\ \hline RF & Estimators = 100 criterion = “gini”, Minimal number of samples for & & \\  & leaf = 9 & coif3 & 68.4\% \\ \hline DT & criterion = “gini”, Minimal number of samples for leaf = 7 & db5 & 65.4\% \\ \hline GBDT & Estimators = 1000, Subsample = 0.75 Depth = 5/7/9, Learning & & \\  & rate = 0.2 & coif2 & 70.6\% \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Performance of classifiers when DWT coefficients were used as features. * 32, 64, 128
* 64, 128, 256  
This choice was made for several reasons. Firstly, we did not have enough data to train a 1D CNN as massive as VGG16 or 19 from scratch, so we proceeded with continuous slashing of VGG blocks. This approach brought us to a setup with three blocks--64, 128, and 256. This was the largest network that did not either diverge or heavily oscillate on our data. Compared to the original VGG architecture, the 256 filter block did not have 3 but only 2 convolutional layers. We also tweaked the architecture with a global average pooling layer, followed by a 32-neuron dense layer, a dropout layer with a 20% chance, and finally, an activated dense layer with one neuron. From here, we decided to experiment with the number of filters and modules. For the normalised and trimmed data, with no prior feature extraction, the VGG-like network setup with 32 and 64 filters performed the best, achieving an accuracy of 75.4%. For the combination with DWT, we first needed to select candidate wavelets for different sizes of filters. We have experimented with (16, 32), (32, 64), and (64, 128) filter combinations. We noticed that VGG-style architecture was a fast learner on our data and rarely ever improved over epoch 50. Hence, we first tried to train the 3 selected VGG-style architectures on all selected wavelets (same as in Section 6.3). Afterwards, based on the validation accuracy, we picked the best-performing wavelet for every of the 3 filters. During this experiment, the best-performing coefficients of wavelets for the filters were \"sym2\", \"haar\", and \"coiff7\", respectively. Afterwards, we cross-validated all filter combinations mentioned above on the 3 selected wavelets. As we have been training different filter sizes and numbers of blocks, we trained for 100 epochs. The best performance with this approach was achieved by combining a 32 and 64-filter (two-module) network and the \"coiff7\" wavelet, with an accuracy of 78.6%.