In recent years, Machine Learning (ML) techniques have gained popularity in Struc- tural Health Monitoring (SHM). These have been particularly used for damage detection in a wide range of engineering applications such as wind turbine blades. The outcomes of previous research studies in this area have demonstrated the capabilities of ML for ro- bust damage detection. However, the primary challenge facing ML in SHM is the lack of interpretability of the prediction models hindering the broader implementation of these techniques. For this purpose, this study integrates the novel Shapley Additive exPlana- tions (SHAP) method into a ML-based damage detection process as a tool for introducing interpretability and, thus, build evidence for reliable decision-making in SHM applications. The SHAP method is based on coalitional game theory and adds global and local inter- pretability to ML-based models by computing the marginal contribution of each feature. The 1 Interpretable machine learning in damage detection using Shapley Additive Explanations contribution is used to understand the nature of damage indices (DIs). The applicability of the SHAP method is ﬁrst demonstrated on a simple lumped mass-spring-damper system with simulated temperature variabilities. Later, the SHAP method has been evaluated on data from an in-operation V27 wind turbine with artiﬁcially introduced damage in one of its blades. The results show the relationship between the environmental and operational variabilities (EOVs) and their direct inﬂuence on the damage indices. This ultimately helps to understand the difference between false positives caused by EOVs and true positives resulting from damage in the structure. NOMENCLATURE SHM Structural health monitoring O&M Operation and maintenance WT Wind turbine EOV Environmental and operational variabilities ML Machine learning CNNs Convolutional Neural Networks SHAP Shapley Additive exPlanations LIME Local Interpretable Model-agnostic Explanations DSF Damage sensitive features MD Mahalanobis distance DI Damage index MSE Mean squared error XGBoost eXtreme Gradient Boosting 1 INTRODUCTION Structural health monitoring (SHM) refers to the process of identifying changes in the integrity of a structure based on observations and measurements that describe its current state. SHM has sparked the interest of researchers due to the untapped potential for safety and reliability im- 2 Interpretable machine learning in damage detection using Shapley Additive Explanations provements and operation and maintenance (O&M) cost reductions. Among the diversity of meth- ods adopted by researchers, data-driven vibration-based structural health monitoring (VSHM) is a promising approach and is continuously being studied. The primary goal of VSHM is to detect damage in a structure through the following steps [1]: (1) Measure vibration responses from the structure; (2) Extract damage sensitive features (DSFs), i.e. metrics that contain useful information to characterize the state of the structure; (3) Select a method to process the DSFs and calculate a damage index (DI); and (4) compare the DI with a predeﬁned threshold, thereby identifying out- liers that indicate potential damage in the structure. For the last two steps, commonly, a statistical outlier analysis method is implemented such as the Mahalanobis distance (MD) to identify discor- dant outliers in a feature set [2]. However, one of the prevailing challenges in damage detection is the inﬂuence of environmental and operational variabilities (EOVs) on DSFs when monitoring structures and collecting vibration responses. Several studies have demonstrated how EOVs can camouﬂage the presence of damage and, ultimately, hinder damage detection [3–6]. To address this challenge, researchers have investigated the use of machine learning (ML) techniques to identify and mitigate the inﬂuence of EOVs. In the past decade, numerous studies have implemented these techniques showing promising results when monitoring structures, iden- tifying failures or damage [7, 8]. A study by Santos et al. [9] compared several ML algorithms, namely one-class support vector machine, support vector data description, kernel principal com- ponent analysis and greedy kernel principal component analysis to detect damage in a three-story aluminum frame structure under the inﬂuence of EOVs. The results have shown reliability in recog- nizing patterns and clustering the observations from the healthy structure, thereby mitigating the effects of EOVs and ultimately increasing the damage detection accuracy. Abdeljaber et al. [10] presented a computationally inexpensive system using 1D Convolutional Neural Networks (CNNs) for real-time damage detection and localisation. The approach enables the automatic extraction of optimal DSFs and was tested on eight structural cases derived from large-scale experiments on a simulator. The results showed the high level of generalization of this approach and the advantage of eliminating the need for manual model or parameter tuning on feature extraction. In the ﬁeld of wind energy, Zhang et al. [11] proposed a data-driven framework using random forests (RFs) 3 Interpretable machine learning in damage detection using Shapley Additive Explanations in combination with extreme gradient boosting (XGBoost) for fault detection in wind turbines. The former is used to rank the DSFs according to their relevance and, later, the top-ranking features are used to train the XGBoost algorithm. The approach was tested on numerical simulations of three different types of wind turbines (onshore and offshore) in different operating conditions. This approach outperformed the support vector machine method when dealing with multi-dimensional features sets in terms of avoiding model overﬁtting. As seen, ML techniques have been successfully implemented in real case applications. Nev- ertheless, the ML algorithms adopted in these studies for VSHM frameworks are often referred to as black-box models and they provide little to no information about the decision-making pro- cess. The lack of interpretability and understanding regarding how these models detect changes in structural integrity and how DSFs are used by these models to make a certain prediction is one of the reasons hindering the widespread adoption of these techniques for industrial applications. Enabling interpretability in ML-based VSHM frameworks can help building evidence for reliable decision-making in SHM applications. In particular, differentiating between novelties caused due to the inﬂuence of EOVs or damage in the structure can help reducing false alarms. The issue of interpretability when using ML techniques has been addressed previously in ﬁelds outside engineering [12]. In particular, the Shapley Additive exPlanations (SHAP) approach pro- posed by Lundberg et al. [13] has lately attracted attention in the research community. The authors adopted a model agnostic representation of feature importance which is estimated by Shapley values [14, 15] and developed an algorithm to estimate the Shapley values in a computationally efﬁcient way. SHAP is an additive feature attribution method that deﬁnes the output of a model as the sum of the real values attributed to each input feature. In other words, the Shapley values represent the marginal contribution of each feature to the prediction of the model. Lundberg et al. [13] have also shown that the Shapley values are the only way to assign feature importance while maintaining two important properties: Local accuracy : The sum of the feature attributions is equal to the output of the function one is seeking to explain Consistency : Changes in the model where the impact of a feature increases will never de- 4 Interpretable machine learning in damage detection using Shapley Additive Explanations crease the attribution assigned to that feature. SHAP has been implemented in ﬁelds such as medicine and ﬁnance. For example, Lundberg et al. [16] used the SHAP technique to understand the outputs of a ML-based system predicting the risk of hypoxaemia during anaesthesia care from electronically recorded intraoperative data. Similarly, in the ﬁeld of ﬁnance, Bussmann et al. [17] employed the SHAP technique to explain the risks associated to peer to peer lending credit in regulated ﬁnancial services. The interpretable model enables lending entities to classify potential borrowers into risky and non-risky borrowers based on common ﬁnancial characteristics identiﬁed by SHAP as highly inﬂuencial including earn- ings before interest and proﬁt before taxes for non-defaulted companies and total assests and shareholders funds for defaulted companies. In both cases, the added interpretability helped un- derstanding the predictions made by the ML models and highlighted the potential of interpretable ML for reasoning in predictive models. Interpretable ML techniques such as SHAP have also been adopted in engineering applica- tions, albeit less frequently. For instance, Parsa et al. [18] used SHAP to analyse the causes for trafﬁc accidents predicted by a XGboost classiﬁcation model. By implementing SHAP , the study found that trafﬁc related features such as the vehicle speed had a relative high impact on the pre- diction of an accident. Furthermore, non-trafﬁc related features such as demographic, network, land use and the weather conditions were identiﬁed as important predictors for the occurrence of an accident. This information can facilitate policy decisions related to e.g. speed limits, urban planning, among others. For SHM applications, Lim et al. [19] estimated different damage levels for several bridges based on monitored conditions of their decks using an XGBoost classiﬁcation model. Their data was retrieved from the Korean Bridge Management System and corresponded to 142,439 deck inspection records of 2388 bridges. For their analysis, the authors collated 53 variables including identiﬁcation factors such as region, shape, vehicle weight limit, among others; structural factors such as deck material, thickness and strength, etc.; trafﬁc factors and inspection factors such as age, damage type and condition rating of the damage. Further, SHAP was adopted to capture the inﬂuence of the features used for the classiﬁcation. Their results showed that major inﬂuencing 5 Interpretable machine learning in damage detection using Shapley Additive Explanations factors inﬂuencing damage to bridges include age, average daily truck trafﬁc and vehicle weight limit. Onchis et al. [20] used Local Interpretable Model-agnostic Explanations (LIME) and SHAP in combination to characterize the location and depth of damage in cantilever beams. Through the added interpretability and a novel stability index, the authors provided trust in models prediction. The ﬁndings emphasized the beneﬁts of using Shapley values for model interpretability in addi- tion to ML-based techniques and their contribution to the decision-making process for preventive maintenance. These studies have shown that interpretable ML techniques improved the pattern recogni- tion properties of the predictive models providing supplementary information to select the most adequate predictors (i.e. predictors with the highest contribution). Enabling interpretability in ML- based VSHM frameworks help building evidence for reliable decision-making in SHM applications. However, to the authors’ best knowledge, this issue has not been reviewed in the scientiﬁc litera- ture, particularly, for interpreting the effect EOVs in continues VSHM for damage detection. Damage detection in structures can often not be treated as a supervised classiﬁcation problem, due to the lack of knowledge about the damage. Therefore, this study introduces a novel frame- work to understand the cause of novelties identiﬁed by an MD-based DI in a semi-supervised framework. For this, an XGBoost regression model is trained to learn the relationship between DSFs, EOVs and MD-based DIs. The XGBoost regression is selected due to their performance and low computational time [21]. Furthermore, the process is complemented with the SHAP ap- proach for enhanced understanding of the variabilities in the DIs identiﬁed. The proposed frame- work yields understanding when detecting outliers that correspond to a false alarm or damage in a structure. The framework is ﬁrst evaluated on a simple lumped mass-springer-damper system with simulated temperature variabilities. The simulation explores and highlights the capabilities of SHAP to interpret the effect of temperature in a dynamic system. Later, the framework is demon- strated on data gathered from an in-operation V27 WT with artiﬁcially introduced damage in one of its blades to evaluate and identify the nature of DIs (e.g. EOVs or damage). The paper is organized such that section 2 introduces the framework for interpretable damage detection followed by a simulated case study in section 3 and an experimental case study on an 6 Interpretable machine learning in damage detection using Shapley Additive Explanations operating wind turbine in section 4. Concluding remarks are provided in section 5. 2 METHODOLOGY Damage sensitive features (Covariance-based)Measured environmental and operational variabilities (e.g., temperature) Novelty index (MD-based)ML-based regression model (XGBoost)Interpretation through Shapley additive explanationPredictors Target Fig. 1: Proposed ﬂow for interpreting the MD-based DI This section elaborates on the steps proposed to determine the cause of novelties identiﬁed through an MD-based damage detection approach. First, extracted damage sensitive features (DSFs) are used to calculate the MD-based DI. Then, an XGBoost regression model is trained considering the DSFs and measured environmental variabilities as predictors and the MD-based DI as the prediction target. Finally, SHAP is included in the analysis to interpret the XGBoost model and thus interpret the contribution of each DSF and measured environmental parameters to the DI. A schematic representation of the proposed framework is illustrated in Fig. 1. 2.1 Covariance-based damage sensitive features Using the covariance as a DSF for damage detection has shown promising results in several studies [22–24] and is, therefore, adopted in this study. The covariance of the vibration responses is used to measure the phase and amplitude relationship between the responses at different sen- sor locations, which characterizes the state of the structure. 7 Interpretable machine learning in damage detection using Shapley Additive Explanations 2.2 Mahalanobis distance-based damage index The Mahalanobis distance (MD) measures the distance between a point and a distribution of a multi-dimensional space [25]. Worden et al. [2] introduced the use of the MD as a means of outlier identiﬁcation for damage detection in structures. To calculate the MD-based DI, DSF vectors xxxiare obtained from the measured responses and gathered into a matrix XXX= [xxx1;xxx2;:::;xxxi;:::;xxxM];XXX2 RLM, whereLis the length of the DSF vector and Mis the number of observations. The MD is calculated as follows: yi=di(xxxi;DDD) =q (xxxiDDD)T1 DDD(xxxiDDD); (1) whereyiis the MD between the DSF vectors xxxiat observation iand the DSF matrix DDD2RLE withEM.DDDis the subset of XXXused for training. DDD2RLis the mean of the observations in DDD andDDD2RLLis the covariance between observations in DDD. 2.3 XGBoost decision trees regression model XGBoost is a scalable machine learning (ML) technique for tree boosting developed by Chen and Guestrin [21]. The algorithm uses nested decision trees to improve efﬁciency and learning in the model [26]. Whilst decision trees use simple conditional statements to classify the features into target labels based on particular attributes of the features, XGBoost uses multiple decision trees in parallel, thereby optimizing performance and improving the accuracy of the classiﬁcation. Each decision tree within the XGBoost algorithm learns from the previous decision tree gradually improving the model and building a strong learner as shown in Fig. 2 Let( 1;y1);( 2;y2);:::;( i;yi);:::;( M;yM)be a data set with Mobservations containing the DSFs including measured EOVs i2RLand the corresponding prediction target yi2R. The tree ensemble model is the summation of Kadditive functions with the estimated prediction ^yideﬁned 8 Interpretable machine learning in damage detection using Shapley Additive Explanations Sample observation PredictionIteration Fig. 2: Basic structure of gradient boosted decision trees as follows: ^yi=KX k=1fk( i); fk2F; (2) wherefkis the independent tree structure of Ktrees andFthe tree space. Different sets of functions are trained by minimizing the following objective function: J(t)=MX i=1l(^yi;yi) +tX k=1 (fk) (3) where the ﬁrst regularization term lis a loss function which measures the difference between the prediction ^yiand the prediction target yiat iteration step t(ortthtree). The second term is the complexity of the model and is deﬁned as: (fk) = P+1 2PX j=1w2 j; (4) 9 Interpretable machine learning in damage detection using Shapley Additive Explanations wherePis the number of leaves in the tree, wjthe score of a leave, the complexity scalar of each leaf andthe scaling parameter to penalize the complexity. Unlike in general gradient boosting decision trees, for XGBoost the second-order Taylor expansion is taken to the loss function. Using the mean squared error (MSE) as a loss function, the tthtree can be derived as follows: J(t)MX i=1[giwq( i)+1 2(hiw2 q( i))] + P+1 2PX j=1w2 j =PX j=1[(X i2Ijgi)wj+1 2(X i2Ijhi+)w2 j] + P:(5) by deﬁning Ij=fijq( i) =jgas the instance set assigned to the jthleaf.Ijrepresents all the data samples in leaf node j;giandhiare the ﬁrst and second derivatives of the MSE loss function and data points are assigned to the corresponding leaf with the function q(). Further, by letting Gj=P j2IjgiandHj=P j2Ijhi, optimizing the objective function can be transformed to ﬁnding the minimum of a quadratic function. The optimal weight w jfor thejthleaf is computed by w j=Gj Hj+(6) for a given structure q(xi) J=1 2TX 1Gj Hj++ T (7) where Eq. 7 is used as a scoring function to measure for a given tree structure q. To evaluate the best split during the training and determine when the splitting is stopped, the following gain score is calculated: Gain =1 2" GL2 HL++GR2 HR+GL+GR2 HL+HR+#  ; (8) 10 Interpretable machine learning in damage detection using Shapley Additive Explanations where the ﬁrst two terms refer to the summation of the left and right tree respectively and are compared to the score of the original leaf, in third term. The score of the generated leafs is penalized by . If the gain is negative, the tree stops growing and returns to the previous split, otherwise it continues improving the model. In other words, after every split the decision tree (DT) model is evaluated based on the two regularization terms: The ﬁrst term in Eq. 3 is responsible for prediction accuracy during the training and the second term considers the complexity of the DT. As high tree complexity can cause over-ﬁtting, this is penalized. 2.4 Shapley Additive Explanations for XGBoost decision trees The Shapley Additive Explanations (SHAP) method is used in this study to interpret the output of the model described in section 2.3. The SHAP method is an additive feature attribution method that deﬁnes the output of a model as the sum of the real values attributed to each input feature. Additive feature attribution methods have an explanation model adeﬁned as a linear function of binary features as in the following equation: a(z) = 0+LX l=1 lzl (9) wherezl2f0;1gL,Lis the length of the DSF vector and l2I Ris the feature attribution values. The variables zlrepresent a feature being observed ( zl= 1) or unknown ( zl= 0). The SHAP method deﬁnes f (S) =E[f( )j S], whereScontains a set of non-zero indexes inz. To compute the feature attribution value lfor each input feature, SHAP uses the classic Shapley values from game theory as in the following equation: i=X SNnflgjSj!(LjSj1)! L![f (S[flg)f (S)]; (10) whereNis the set of all input features. To interpret the XGBoost model described in section 2.3, SHAP for tree-based models, i.e., 11 Interpretable machine learning in damage detection using Shapley Additive Explanations TreeShap, as proposed by Lundberg et. al. [27] is implemented. 3 SIMULATED LUMPED SIX DEGREE OF FREEDOM SYSTEM To evaluate the proposed framework in application to VSHM, let us ﬁrst consider a simple dynamic system. Figure 3 shows a six degrees of freedom (DOF) lumped mass-spring-damper system consisting of six masses m1tom6connected to each other by springs with stiffnesses k1tok6and dampers with the coefﬁcients c1toc1. The system is ﬁxed to the ground on one of the sides. The mass and damper coefﬁcients are given the same value of 2kgand0:01Ns2 m respectively. m4 m5 m3 m2 m1 c1 c2 c3 c4 c5 c6k1 k2 k3 k4 k5 k6 m6 Fig. 3: A six DOF mass-spring system To model the effect of EOVs on the system’s dynamic, we assume the temperature Tiaffects the spring stiffnesses. For simplicity, the following rule is used: k1=k2=k3=k4=k5=k6= 0:1Ti; (11) The system is excited by an impulse force applied to ﬁrst mass and, after the impulse, the masses vibrate freely. The resulting time series with 4350 samples with a sampling frequency of 1321Hzare used for calculating the covariance which are used as DSFs (see section 2.1). In total we simulate 2000 observations. At each observation, the temperature is set according to Figure 4. It ranges from approximately 7.5C to 15.5C, where extreme weather events are implemented as sudden temperature drops of 4C between observations 1100-1150 and 1400- 1450. To simulate damage, after 1950 observations, the stiffness k3is reduced by 5%. For each 12 Interpretable machine learning in damage detection using Shapley Additive Explanations observation, the stiffnesses are set according to Eq. 11, and the DSF are calculated. The number of observations per simulated state is summarized in Table 1. 0 250 500 750 1000 1250 1500 1750 2000 Observation68101214Temperature °CExtreme weather event Fig. 4: Simulated temperature ﬁeld with sudden temperature drops highlighted with a grey back- ground Healthy Damage TotalReference Testing Observations 1250 700 50 2000 Table 1: Partition of data: Simulated data set 3.1 Damage detection: MD-based DI Following the framework for continues monitoring presented in previous studies [24, 28], the MD-based DI, Eq. 1 is computed with only a subset of healthy observations, in this case the ﬁrst 1250 are used as a reference for the inverse covariance matrix DDD. The DI in Fig. 5 shows an increase for observations 1100-1150 and 1400-1450, where the temperature drops 4C. Further, the DI of observations 1950-2000 increase due to the 5% stiffness reduction of spring k3. 3.2 Interpretability To understand the difference between novelties caused by the stiffness reduction and the inﬂuence of temperature effects, a regression model is built taking the temperature and DSFs 13 Interpretable machine learning in damage detection using Shapley Additive Explanations 0 250 500 750 1000 1250 1500 1750 2000 Observation101102Mahalanobis Distance, yiIntroduced damage Extreme weather event CalculatedCalculated, damage Threshold 80% Fig. 5: MD-based damage index as predictors and the MD-based DI as the prediction target. The XGBoost regression model is trained as described in section 2.3 using observations from the healthy and damaged states of the system. The aim of adopting a regression model in this study is to maintain a high prediction accuracy of the regression model to reproduce the MD-based DI. For this, the hyperparameters (i.e., maximum depth, minimum child weight, gamma, subsample, colsample by tree, alpha for L1 regularization, and learning rate) are identiﬁed by a random search. The search is performed until a performance accuracy R2>0:99was reached, where R2is square of the sample correlation coefﬁcient between the predicted and the calculated MD. The parameters are summarized in Table 2. For each randomly selected values in a given range a maximum of 1000 iteration is set and the training stops if the mean squared error does not improve for 50 iterations to avoid overﬁtting. 0 250 500 750 1000 1250 1500 1750 2000 Observation101102Mahalanobis DistancePredicted, yi Introduced damage Extreme weather eventCalculated, yi Calculated, damage Fig. 6: Predictions from the XGBoost regression model for 6DOF mass-spring-damper system 14 Interpretable machine learning in damage detection using Shapley Additive Explanations Hyperparameter Description From To Interval 6DOF - Best parameters Maximum depth The maximum depth of which each tree is built 1 10 1 7 Minimum child weight The minimum sum of instance weight of all the observationsrequired in a child 1 5 1 3 Gamma Controls the minimum loss reduction required to make a node split 0 1 0.1 0 Subsample The number of observations randomly sampled at each tree 0.1 1 0.1 0.4 Colsample by tree The number of features selected to build each tree 0 1 0.1 0.6 Regularization alpha L1 regularization value on weights 0.01 0.3 0.01 0.19 Learning rate Stepsize on updating the weights 0.01 0.3 0.01 0.22 Table 2: Summary of hyperparameters and ranges chosen for the random search Figure 7a summarizes the Shapley values obtained for the 6DOF lumped mass-spring-damper system XGBoost model for the observations 1 to 1950, while Figure 7b refers to the summary of the observations 1950 to 2000. The features are ranked in descending order. It can be seen that the variance of sensor 3 has the largest marginal contribution on the model’s prediction and the covariance of sensor 3 and 5 the lowest. On the horizontal axis, the contribution of the feature can be observed indicating a positive or a negative impact on the models prediction. For example, the temperature feature in Figure 7a shows that low values (colored blue) have a positive impact on the models prediction, i.e. these increase the model’s predicted index, whilst lower values have a negative impact, decreasing the model’s predicted index. This suggests that low temperatures cause a high MD-based DI and vice-versa. Looking at the last 50 observations in Figure 7b, where the stiffness reduction is introduced, the contribution of the temperature to these predictions drops from ranks 4 to 6. This indicates that the predictions of these observations are not dominated by the inﬂuence of the temperature ﬁeld on a global scale. Looking at the local Shapley values tempfor the temperature in Figure 8, it can be observed that observations 1100-1150 and 1400-1450 have particularly high Shapley values. This indicates that these observations are caused by temperature. Conversely, observations 1950 to 2000 have a negative impact on the model’s prediction indicating that these novelties did not occur due to the inﬂuence of temperature. Figure 9a and 9b show the Shapley values for observations 1100-1150 and 1400-1450 re- spectively in a global impact overview of the features contribution. In these ﬁgures, it is clear that the temperature is the most inﬂuential feature for the predictions. The temperature drops of 4°C introduced into the 6-DOF simulated system which were responsible for the increased DI can be identiﬁed as false alarms by observing the Shapley values. 15 Interpretable machine learning in damage detection using Shapley Additive Explanations 0 50 100 150 200 SHAP value (impact on model output)cov_3_5cov_2_5cov_5_6cov_4_4cov_2_2cov_2_6cov_5_5cov_3_6cov_2_3cov_3_4cov_1_6cov_1_1cov_1_2cov_1_5cov_2_4cov_6_6tempcov_1_3cov_1_4cov_3_3 LowHigh Feature value (a) 20 10 0 10 20 30 40 50 60 SHAP value (impact on model output)cov_3_5cov_5_5cov_4_4cov_3_6cov_2_6cov_3_4cov_5_6cov_2_5cov_6_6cov_2_3cov_2_2cov_1_6cov_1_1cov_1_4tempcov_1_2cov_1_5cov_2_4cov_1_3cov_3_3 LowHigh Feature value (b) Fig. 7: Summary contribution plot for (a) observations 1 to 1950 with no implemented stiffness reduction and (b) observations 1950 to 2000 with stiffness reduction 0 250 500 750 1000 1250 1500 1750 2000 Observation65.0 43.4 21.7 0.021.743.465.086.7108.4130.1151.7173.4Temp. contribution, temp Fig. 8: Shapley explanation for temperature in the 6DOF mass-spring-damper system XGBoost regression model 16 Interpretable machine learning in damage detection using Shapley Additive Explanations 0 50 100 150 200 SHAP value (impact on model output)cov_5_6cov_5_5cov_2_6cov_2_4cov_3_5cov_4_5cov_3_4cov_2_2cov_1_6cov_4_4cov_1_5cov_3_6cov_1_2cov_2_3cov_6_6cov_1_1cov_1_4cov_1_3cov_3_3temp LowHigh Feature value (a) 20 0 20 40 60 80 100 SHAP value (impact on model output)cov_2_5cov_5_5cov_4_5cov_2_3cov_2_4cov_3_5cov_1_2cov_3_4cov_2_2cov_1_1cov_4_4cov_1_6cov_1_5cov_3_6cov_6_6cov_2_6cov_1_4cov_1_3cov_3_3temp LowHigh Feature value (b) Fig. 9: Summary contribution plot for extreme weather events between (a) observations 1100-1150 and (b) observations 1400 to 1450 17 Interpretable machine learning in damage detection using Shapley Additive Explanations 4 CASE STUDY: OPERATIONAL WIND TURBINE V27 In this section, we demonstrate the capabilities of the proposed framework to detect damage and explain the DI in a continuous monitoring system installed in an operating wind turbine. The experiment was conducted by Tcherniak et al. [24] between November 28th, 2014 and March 12th, 2015. In the experiment, one of the blades in the operating Vestas V27 wind turbine in Figure 11a is equipped with 11 accelerometers. For this study, the 8 accelerometers shown in Figure 10 placed on the leading and trailing edge are considered. Additionally, the electromechanical actuator shown in Figure 11c was mounted close to the root of the blade to excite the blade. During the monitoring campaign, a total of 24,693 observations were recorded considering a healthy, damaged and a repaired WTB. During the monitoring campaign, artiﬁcial damage was introduced by creating a trailing edge opening and gradually expanding it from an initial length of 15cm in damage scenario 1 (see Figure 11b), to 30cm in scenario 2 and 45cm in scenario 3. Finally, the WTB was repaired and further observations were recorded. 3.5m 2m 2m 2m13m DamageActuator 6m#2#5#7 #1 #4 #8#10 #1 1 15cm 30cm 45cm Accelerometer Fig. 10: Sketch of the equipment on the blade of the Vestas V27 wind turbine The framework is demonstrated on observations corresponding to the 32 rpm operational mode of the turbine. For the purpose of damage detection, the data is classiﬁed into two sets, namely healthy and damaged. The former contains solely observations recorded after repairs as there are nearly three times more observations than those recorded prior to the introduction of the artiﬁcial damage. Therefore, in this study, the observations considered as healthy are ob- tained from the repaired observations which consists of 2927 observations. The data classiﬁed as 18 Interpretable machine learning in damage detection using Shapley Additive Explanations damaged correspond to the observations that were recorded during the three damage scenarios. Further, the observations are ﬁltered by a band-pass ﬁlter with cut-off frequencies of 700 and 1200 Hz. A summary of the number of observations in each class is presented in Table 3. Healthy Damage TotalReference Testing 15cm 30cm 45cm Observations 2000 639 66 117 105 2927 Table 3: Partition of observations in the Vestas V27 data set (a) (b) (c) Fig. 11: (a) Vestas V27 wind turbine (b) First damage implemented by a 15cm trailing edge opening around 6m from the tip of the blade (c) Electromagnetic actuator mounted close to the root of the blade for mechanical excitation 4.1 Damage detection: MD-based DI The DI shown in Figure 12 is computed in the same manner as described in section 2.2 with the obtained covariance-based DSF vectors. The ﬁrst 2000 observations are used to create a reference to compute the MD as shown in Eq. 1. Outliers in the DI can be identiﬁed at around observations 360-675 and 1080-1215 (highlighted in grey). These observations will be the focus of further interpretation. The DI for these observation causes false alarm and, thus, misinterpreted damage. 19 Interpretable machine learning in damage detection using Shapley Additive Explanations 0 500 1000 1500 2000 2500 Observation101Mahalanobis Distance, yiIntroduced damage Extreme weather eventCalculated 15cm damage30cm damage 45cm damageThreshold 80% Fig. 12: MD-based DI for wind turbine blade V27 4.2 Interpretability To understand the potential cause of these novelties and how they differ from the identiﬁed damage after observation 2639, the XGBoost regression is built. DSFs and measured EOVs such as temperature, wind direction and wind speed are included. The XGBoost regression model is adopted from section 3.1 with the hyperparameters shown in Table 4. The model’s predictions with anR2>0:99are shown in Fig. 13. Hyperparameter Description From To Interval V27 - Best parameters Maximum depth The maximum depth of which each tree is built 1 10 1 8 Minimum child weight The minimum sum of instance weight of all the observationsrequired in a child 1 5 1 1 Gamma Controls the minimum loss reduction required to make a node split 0 1 0.1 0 Subsample The number of observations randomly sampled at each tree 0.1 1 0.1 0.6 Colsample by tree The number of features selected to build each tree 0 1 0.1 0.5 Regularization alpha L1 regularization value on weights 0.01 0.3 0.01 0.08 Learning rate Stepsize on updating the weights 0.01 0.3 0.01 0.14 Table 4: Summary of hyperparameters and ranges chosen for the random search The global inﬂuence, i.e., the Shapley values, of the DSFs and EOVs are shown in Figure 14a and 14b. The Shapley values presented for the healthy state of the WTB in Figure 14a show that the variance of accelerometer 7 has the highest inﬂuence on the model’s predictions, followed by temperature. This indicates that the DI is highly inﬂuenced by temperature while the turbine has no artiﬁcial damage introduced. We also can identify that temperature values close or below (see Figure 15a) have a positive inﬂuence suggesting that low temperatures cause a high MD-based DI. This is consistent with the SHAP results obtained from the simulation in section 3.1, where the 20 Interpretable machine learning in damage detection using Shapley Additive Explanations 0 500 1000 1500 2000 2500 Observation101Mahalanobis distancePredicted, yi Introduced damageCalculated, yi 15cm damage30cm damage 45cm damage Fig. 13: Predictions from the XGBoost regression model for the V27 MD-based DI drops in temperature caused an increase in the MD-based DI. EOVs such as wind direction and wind speed have a rather small inﬂuence on the model’s prediction. Conversely, the signiﬁcance of the temperature dropped for observations 2639-2927 as shown in Fig. 14b. The majority of Shapley values for observations during the damaged state of the WTB ﬂuctuate around or below 0 indicating a low inﬂuence on the model’s prediction. This leads to the conclusion that these novelties are not caused by the effect of temperature. For local interpretability and to gain insight into the contribution of the temperature measure- ments shown in Figure 15a to the model’s predictions, the corresponding Shapley values for each of the observations are shown in Figure 15b. Looking into the previous false alarms identiﬁed between observations 360-675 and 1080-1215, it is clear that these outliers are inﬂuenced by the positive Shapley values contribution of temperature to the model’s outputs. When looking at the observations after 2639, when the artiﬁcial damage was introduced, the temperature effect is rather negligible with Shapley values ﬂuctuating around zero. Focusing on observations 360-675 in Figure 16a and 1080-1215 in Figure 16b, the Shapley values for temperature are dominating throughout all features conﬁrming the inﬂuence of EOVs on the model’s outputs. Low temperature values have a positive impact on the prediction where higher values have a rather low impact. Nevertheless, limitations can be found in observations around 2400 where false alarm is present but a rather minor contribution of weather is found slightly reaching a neutral contribu- tion of zero. This indicates, that the model is able to predict these observations but does not use 21 Interpretable machine learning in damage detection using Shapley Additive Explanations 2 0 2 4 6 SHAP value (impact on model output)Cov11_11Cov1_10Cov1_11Cov2_5Cov8_11WindSpeedWind_DirectionCov4_7Cov1_8Cov7_8Cov7_11Cov1_5Cov1_1Cov5_7Cov10_11Cov2_11Cov4_11Cov8_8TemperatureCov7_7 LowHigh Feature value (a) 2 0 2 4 6 8 10 12 SHAP value (impact on model output)Cov1_10Cov8_11Cov1_11Cov5_8Cov1_2Cov5_5Cov1_8Cov4_10Cov7_8Cov5_7Cov11_11Cov4_7TemperatureCov7_11Cov4_11Cov2_11Cov7_10Cov10_11Cov8_8Cov7_7 LowHigh Feature value (b) Fig. 14: Summary contribution plot for (a) observations 1-2639 for the healthy state of the structure and (b) observations 2639-2927 with artiﬁcially introduced damage the weather as a signiﬁcant contributor. 22 Interpretable machine learning in damage detection using Shapley Additive Explanations 0 500 1000 1500 2000 2500 Observation4 2 024681012Temperature, °C (a) 0 500 1000 1500 2000 2500 Observation0.5 0.00.51.01.52.02.53.0Temp. contribution, temp Introduced damage Extreme weather event (b) Fig. 15: (a) Measured temperature during the measurement campaign and (b) Shapley explana- tion for the temperature in the XGBoost regression model for damage detection in the operating V27 WTB 23 Interpretable machine learning in damage detection using Shapley Additive Explanations 1.0 0.5 0.0 0.5 1.0 1.5 2.0 2.5 3.0 SHAP value (impact on model output)WindSpeedCov1_8Cov1_10Cov10_10Cov8_10Cov7_11Cov4_10Cov4_7Cov1_1Cov1_5Cov1_11Cov8_8Cov7_8Cov2_5Cov5_7Cov10_11Cov4_11Cov2_11Cov7_7Temperature LowHigh Feature value (a) 2 0 2 4 6 SHAP value (impact on model output)Cov1_10Cov8_10Cov1_5Cov1_2Cov7_11Cov4_5Cov1_11WindSpeedCov4_8Cov5_7Cov10_11Cov1_1Cov2_11Cov7_8Cov1_8Cov8_8Cov4_7Cov4_11Cov7_7Temperature LowHigh Feature value (b) Fig. 16: Summary contribution plot for (a) observations 360-675 where the ﬁrst signiﬁcant outliers were identiﬁed and (b) observations 1080-1215 where second outliers were identiﬁed 24 Interpretable machine learning in damage detection using Shapley Additive Explanations 5 CONCLUSIONS This study introduced a framework to distinguish novelties due to the inﬂuence of environmen- tal and operational variabilities (EOVs) from those caused by damage in a structure. This frame- work addresses the lack of interpretability and transparency in ML techniques used for vibration- based structural health monitoring (VSHM). The proposed framework was ﬁrst evaluated on a 6-degrees-of-freedom lumped mass-springer-damper system with simulated temperature variabil- ities and, later, demonstrated on a full-scale experiment in an in-operation wind turbine blade with artiﬁcially introduced damage. The results indicate that ML models such as XGBoost can successfully learn the relationship between damage sensitive features (DSFs), EOVs and Mahalanobis distance-based damage in- dices. The complementary use of interpretable ML, in this case the Shapley Additive exPlanations (SHAP) approach, enabled the determination of the contributions of the DSFs and EOVs to the predictions from the ML model. This process provided transparency and eased the understand- ing of the inﬂuence of EOVs on the damage indices. The SHAP method estimated the marginal contributions of EOVs such as temperature and enabled the identiﬁcation of false positives. The re- sults suggest that the proposed framework can strengthen the conﬁdence in the decision-making process and contribute to the widespread adoption of ML techniques for industrial applications. Furthermore, the proposed framework can increase reliability and safety and reduce operation and maintenance costs in the wind industry.