Probabilistic Fusion Approach for Robust Battery Prognostics

Authors: jokin  alcibar
mondragon  unibertsitatea
ekhi  zugasti
aitor  aguirre-ortuzar
jose i aizpurua
Venue: NeurIPS_cc_2024_Workshop_BDU
Published: 2024-09-04 17:12:56

Abstract: Batteries are a key enabling technology for the decarbonization of transport and energy sectors. The safe and reliable operation of batteries is crucial for battery-powered systems. In this direction, the development of accurate and robust battery state-of-health prognostics models can unlock the potential of autonomous systems for complex, remote and reliable operations. The combination of Neural Networks, Bayesian modelling concepts and ensemble learning strategies, form a valuable prognostics framework to combine uncertainty in a robust and accurate manner. Accordingly, this paper introduces a Bayesian ensemble learning approach to predict the capacity depletion of lithium-ion batteries. The approach accurately predicts the capacity fade and quantifies the uncertainty associated with battery design and degradation processes. The proposed Bayesian ensemble methodology employs a stacking technique, integrating multiple Bayesian neural networks (BNNs) as base learners, which have been trained on data diversity. The proposed method has been validated using a battery aging dataset collected by the NASA Ames Prognostics Center of Excellence. Obtained results demonstrate the improved accuracy and robustness of the proposed probabilistic fusion approach with respect to (i) a pseudo-Bayesian model averaging, (ii) a pseudo-Bayesian model averaging with Bayesian bootstrapping, and (iii) a point prediction stacking strategy based on different BNNs.

Main paper: 
 introduction: 
 Batteries are crucial for a sustainable, carbon-free economy. The development of accurate battery Remaining Useful Life (RUL) prediction models is particularly important for reliable energy strategies and cost-effective solutions. The estimation of the state-of-health (SOH) is a key activity for the design of RUL prognostics models, focusing on capturing aging dynamics and health state (Toughzaoui et al., 2022) . SOH-based prognostics capture battery ageing and health estimation which are crucial indicators for addressing the degradation that impacts capacity and increases safety risks like overheating (Wang et al., 2022) . Thus, precise SOH monitoring and forecasting are imperative for safe and efficient operation of battery-operated systems (Zhao et al., 2023) .Recent data-driven approaches have focused on modeling the capacity degradation of lithium-ion (Li-ion) batteries. Toughzaoui et al. (2022) developed a Convolutional Neural Network (CNN)-Long short-term memory (LSTM) architecture, and Wei and Wu (2023) presented a graph CNN complemented by dual attention mechanisms for the estimation of SOH and RUL of batteries. Due to the variability inherent in the battery manufacturing process, it is essential to quantify this uncertainty to ensure robust and reliable prognostics predictions Abdar et al. (2021) ; Nemani et al. (2023) .In the broader machine learning context, ensembles of probabilistic models have been utilized to capture complex uncertainties. Fan et al. (2017) introduced a Bayesian posterior predictive framework for weighting ensemble climate models. Cobb et al. (2019) present a new machine learning retrieval method based on an ensemble of BNNs. In this scenario, the overall output from the ensemble is treated as a Gaussian mixture model. However, models are equally weighted with no adaptation to the observed data. Zhang et al. (2022) present a Bayesian Mixture Neural Network (BMNN) for Li-ion battery RUL prediction. However, the absence of a weighted model combination limits the analysis of individual model contributions. Alternatively, Bai and Chandra (2023) described a Bayesian ensemble framework using gradient boosting and Markov Chain Monte Carlo sampling, while Dai et al. (2023) demonstrated a robust Bayesian fusion method within a sequential Monte Carlo algorithm, enhancing uncertainty quantification in predictions.In this context, inspired by the promising ability of probabilistic ensemble models to capture model uncertainty, the main contribution of this research is the development of a probabilistic model fusion approach for battery SOH predictions. Bayesian convolutional neural networks (BCNNs) are used as base models for SOH prediction, and the developed fusion approach integrates individual BCNN probabilistic predictions. The fusion strategy balances between precision and reliability of individual predictions, adopting an optimal tradeoff between accuracy and uncertainty of predictions through the proposed stacking approach. The proposed approach has been compared with (i) stacking BCNN models using mean information, (ii) Pseudo BMA, and (iii) a Pseudo-BMA variant stabilized with Bayesian bootstrap, called Pseudo-BMA+. Obtained results confirm that the proposed framework infers accurate, well-calibrated, and reliable probabilistic predictions, which improve predictive performance and contribute to estimate uncertainty in a robust and reliable manner in complex data-driven tasks. The proposed approach has been tested and validated with the publicly available NASA's battery dataset Saha and Goebel (2007) . 
methodology: 
 The proposed framework integrates BCNNs with probabilistic ensemble strategies to generate accurate predictions with robust uncertainty quantification, leveraging Bayesian modeling and ensemble strategies. The approach is divided into offline and online stages. From battery datasets, the offline process completes data pre-processing and model training. In the online process, trained models are stacked according to computed weight and stacking criteria. The outcome is a one-step-ahead probabilistic capacity estimate. Figure 1 shows the high-level block diagram of the proposed approach. The approach is generally divided into offline and online phases. During the offline phase, starting from a battery dataset with different run-to-failure trajectories on the same type of batteries, different base models are designed through a training strategy which seeks diversity in the training set to develop complementary predictive models.BCNN combines feature extraction capabilities of classical CNN models with the uncertainty quantification of Bayesian theory. This fusion enables robust predictions by capturing model uncertainty, crucial for reliable decision-making in complex scenarios. The proposed architecture is built using variational layers of TensorFlow Probability in Python Dillon et al. (2017) . Detailed architecture is provided in the Appendix A.Model diversity is crucial for effective ensemble models Nam et al. (2021) . Accordingly, the training set for each battery model is modified to learn different battery aging properties. Using the leave-oneout (LOO) strategy, K diverse BCNN models are built from K run-to-failure trajectories, changing the training set in each iteration (cf. Figure 1 ). Each model is trained on all batteries except one, held as a test set. This LOO approach enhances models' ability to generalize across battery types and manufacturing conditions. This stage completes the offline training process, which results in a set of BCNN models, M = {BCN N 1 , BCN N 2 , . . . , BCN N K } .During the online phase, the proposed stacking of predictive distribution strategy is designed and tested. The proposed approach takes as input individual base models [cf. Eq. (2.1)] and monitored data up to the prediction instant t, which is used to forecast the probability density function (PDF) of the capacity at t + 1, ŷP DF (t + 1). The objective of the stacking process is to integrate the predictive distributions of different base models and propagate all the information end-to-end.For comparison and benchmarking purposes, we implemented alternative ensemble methods: Pseudo-Bayesian Model Averaging (Pseudo-BMA) (cf. Appendix C.1), an enhanced variant of Pseudo-BMA stabilized using Bayesian bootstrap, known as Pseudo-BMA+ (cf. Appendix C.2) and Stacking Point Prediction (SPP) (cf. Appendix C.3).The optimal way to combine a set of Bayesian posterior predictive distributions is by using the logarithmic score Yao et al. (2018) . This method maximizes the average log-likelihood of the observed data, which is a proper scoring rule used to evaluate the accuracy of probabilistic forecasts. It measures the accuracy of a forecast and penalizes overconfidence and underconfidence in the predicted probability. The logarithmic score is defined as follows:ŵ = arg max w 1 N N i=1 log K k=1 w k p(y i | y -i , M k ) + λ reg K k=1 w 2 k (1)where N denotes the total number of data points and K denotes the total number of base models. The LOO predictive distribution for each model, i.e. p(y i | y -i , M k ), is used to compute the model's prediction for the data point i. To avoid overfitting, a regularization term λ reg is added.In the Bayesian framework, stacking extends beyond averaging point predictions by combining multiple Bayesian posterior predictive distributions. This approach develops a stacking model that leverages the strengths of various predictive models [cf. Eq. (2.1)], enhancing the overall predictive accuracy. Stacking predictive distributions enables fusion of uncertainties from various models into a unified predictive framework. This approach improves forecast accuracy and offers a comprehensive evaluation of uncertainty associated with forecasts, providing advantages across diverse decisionmaking scenarios. The fundamental equation governing this process is defined as follows:p(ỹ|y) = K k=1 ŵk p(ỹ|y, M k ) (2)where p(ỹ|y) represents the aggregate probability estimation based on the ensemble model, ω k denotes the weight assigned to the k-th component within the ensemble, and p(ỹ|y, M k ) refers to the probabilistic forecast generated by each base model.Online forecasting computes one-step ahead predictions. To forecast battery capacity at instant t + 1, previous data until instant t is used, plus an uncertainty factor expressed as noise. This data includes voltage V (t) and temperature T (t) at instant t, as well as a Gaussian noise term ϵ distributed as N (0, σ = 1), which introduces variability in the data's progression over time. The one-step ahead capacity distribution prediction is thus defined as :EQUATIONwhere f (.) denotes the designed ensemble model, and ŷP DF (t + 1) is the distribution of the capacity estimate at t + 1. 
results: 
 To evaluate the proposed approach, different ensemble strategies are compared to evaluate their strengths and identify the most suitable approach. Table 1 presents a comparative analysis in terms of accuracy and probabilistic metrics.A notable observation from the results in Table 1 is the variance between the proposed ensemble approach (cf. Figure 1 ) and the benchmarking ensemble models (cf. Appendix C) in specific scenarios.For batteries #5, #6 and #7, the proposed approach exhibited superior outcomes, particularly in probabilistic metrics such as Negative Log-Likelihood (NLL) (cf. Appendix D.2) and Continuous Rank Probability Score (CRPS) (cf. Appendix D.1). This suggests that the method produces not only accurate point estimates but also well-calibrated probability distributions. This is particularly valuable for battery health prognostics, where understanding prediction uncertainty is crucial for informed decision-making. 1 , showcasing the different ensemble methods used for forecasting the capacity degradation of battery #5. The pseudo-BMA method displays a relatively narrow credible interval, though its accuracy is not very precise, especially at the beginning. The pseudo-BMA+ method shows similar uncertainty quantification to pseudo-BMA, with slightly improved accuracy. In contrast, the stacking of point predictions method yields the least favorable results in both accuracy and uncertainty quantification. Notably, the stacking of predictive distributions method provides the most comprehensive results, combining high accuracy with narrow credible intervals.These observations highlight that although pseudo-BMA and pseudo-BMA+ present good results, the stacking of predictive distributions is generally superior due to its connection with the logarithmic score. This approach avoids potential issues such as identification problems arising from the lack of strict propriety associated with the energy score, and eliminates the need for additional smoothness assumptions required by other proper local scoring rules (Gneiting and Raftery, 2007) .  
discussion: 
 The proposed research work demonstrates that the stacking of predictive distributions based on a Bayesian framework improves the accuracy and robustness of predictions compared with pseudo-BMA, pseudo-BMA+ and point prediction stacking approach. However, before drawing definitive conclusions about the application of the proposed solution in real-world applications,Credible intervals indicate the uncertainty associated with the data and the model (see Figure 2 ). Enhancing robustness involves reducing these intervals by minimizing uncertainty. This can be achieved by increasing observations to refine model uncertainty and using priors, such as maximum entropy or weakly informative priors, to tighten credible intervals.To analyze larger fleets of batteries, clustering similar batteries or using hierarchical modeling with a global model and specific group models would be more efficient than leave-one-out methods, enabling better data diversity and scalable adaptations. 
conclusion: 
 This work presented a novel probabilistic fusion approach for battery state-of-health prognostics, combining BCNNs with Bayesian ensemble stacking techniques. The proposed stacking of predictive distributions method demonstrated superior performance on the NASA battery dataset, particularly in terms of probabilistic metrics. Results showed improved accuracy and well-calibrated uncertainty estimates compared to alternative probabilistic fusion methods, highlighting the potential of this approach for robust battery health monitoring.The main contribution lies in the effective integration of model diversity and uncertainty quantification, enabling more reliable decision-making in battery management systems. However, the generalization to diverse battery types and operational conditions requires further investigation. Despite this limitation, the proposed method represents a significant progress in probabilistic battery prognostics, with potential applications in other health monitoring domains. 
 
 Appendix: BCNN models are a Bayesian extension of the classical Convolutional Neural Network (CNN) models to include uncertainty associated with parameter estimation. This requires modification of the classical backpropagation algorithm through Bayesian techniques. The incorporation of uncertainty into the model is achieved by treating weights as random variables and applying Variational Inference to approximate posterior distributions. This results in a more robust model that predicts the complete probability density function (PDF).BCNN models have been selected to improve the robustness and accuracy of model prediction with respect to classical CNN models. To this end, BCNNs make use of probabilistic distributions to model parameters and the uncertainty related to their training process, and prior distributions to incorporate previous knowledge, generate uncertainty estimations and mitigate over-fitting Blundell et al. (2015) .In contrast, the classical learning models, e.g. non-Bayesian CNN models, focus on maximum likelihood estimation (MLE) and they overlook prior and posterior distributions. This leads to increasing error and decreasing model robustness in high uncertainty contexts, e.g. out-of-distribution data or manufacturing drifts.The architecture of the BCNN models is shown in Figure 3 and it is defined as follows:Convolution 1D Reparameterization Adaptive Avg. Pooling Layer • Distribution Lambda: this layer is responsible for producing the final results given the inputs and the learned weights from the previous layers. The output layer consists of two neurons representing the mean, ŷ and variance, σ2 , in order to quantify the expected value and its associated uncertainty. To ensure a positive variance, the neuron is activated using an exponential function.The effectiveness of the proposed method has been tested using a battery dataset from the NASA Ames Prognostics Center of Excellence (Saha and Goebel, 2007) .A subset of available battery data has been selected, focusing on batteries #5, #6, #7 and #18. Each battery is operated under various conditions including charging, discharging, and impedance analysis.Throughout the charge and discharge cycles, temperature, current, and voltage were meticulously recorded. During charging, a constant current mode at 1.5 A was maintained until the voltage reached 4.2 V, followed by a switch to constant voltage mode until the current dropped to 20 mA. Discharge cycles involved a constant load mode at 2 A until the voltage levels reached 2.7 V, 2.5 V, 2.2 V and 2.5 V for batteries #5, #6, #7 and #18, respectively. The experiment ended once the battery capacity decreased by 30%. These batteries had a maximum capacity of 2Ah with an end-of-life capacity set at 1.4Ah. Figure 5 shows variations in capacity degradation rates for identical batteries. This is an indicator of uncertainty inherent in the manufacturing process, which affects SOH estimates. The accuracy of the regression is measured by mean squared error (MSE), while negative log likelihood (NLL) assesses model performance by quantifying prediction probabilities. Finally, The correctness of probability predictions is assessed through the continuous ranked probability score (CRPS).D.1 Continuous Ranked Probability Score (CRPS)CRPS can be formally expressed as a quadratic measure of discrepancy between the predicted Cumulative Distribution Function (CDF), F (•), and the observed empirical CDF for a given scalar observation y (Zamo and Naveau, 2018) :EQUATIONwhere I(x ≥ y i ) is the indicator function, which models the empirical CDF.To obtain a single score value from Eq. ( 14), a weighted average is calculated for each individual observation of the test set (Gneiting et al., 2005) :EQUATION)where N denotes the total number of predictions.NLL metric assesses probabilistic models by using the likelihood concept, which indicates how likely the observed data is given model parameters (Bosman and Thierens, 2000) . Likelihood (L) is the product of each observation's probability density function (PDF), expressed mathematically asEQUATIONwhere θ denotes model parameters and X includes N data points. NLL is preferred for optimization since minimizing NLL is equivalent to maximizing the log-likelihood, facilitating the discovery of model parameters that best explain the observed data, represented byEQUATIONWorkshop on Bayesian Decision-making and Uncertainty, 38th Conference on Neural Information Processing Systems (NeurIPS 2024). 



Reviews: review: title: title: Accept
main_review: main_review: In this work, the authors develop a probabilistic model fusion approach for battery state-of-health predictions. The proposed approach combines Bayesian convolutional neural networks and ensemble methods to reliably predict uncertainty. The novelty of the approach is relatively limited. It is unclear what the core differences are from Alcibar et al. (2024). However, I think the core contribution is the successful application to battery SOH, which is interesting and non-trivial.

Minor suggestions:
Please add abstract before introduction.
Please emphasize and discuss the difference and similarity between the proposed method and existing ones.
title: title: Solid Submission with some work needed in data application
main_review: main_review: This paper demonstrates a novel combination of bayesian model averaging and bayesian convolutional neural networks in order to better model battery health over time. Care has been made to take advantage of the uncertainty estimate that bayesian neural networks give and a robust model averaging approach using multiple types of model averaging has been performed. 

While the application section, which demonstrates the performance of the algorithm on a publicly available dataset seems well done, there are two points that could do with improvement. First, the citation for the dataset "Saha, B. and Goebel, K. (2007). Nasa ames prognostics data repository. NASA Ames, Moffett Field, CA, USA." points to the location of all prognosis datasets, but the site itself has multiple battery datasets and each gives their own separate citation. For replicability, please update the citation to the more specific dataset. Second, the model comparisons and evaluations were not compared to any external benchmarks so it is difficult to know how the performance of this approach compares to existing methods. 

However, overall, I believe the paper is well written, interesting, and well demonstrates bayesian decision making using AI.