# Deep Lifelong Learning Optimization Algorithm in Dense Region Fusion
###### Abstract  
Deep lifelong learning models can learn new information continuously while minimizing the impact on previously acquired knowledge, and thus adapt to changing data. However, existing optimization approaches for deep lifelong learning cannot simultaneously satisfy the following conditions: unrestricted learning of new data, no use of old data, and no increase in model parameters. To address this problem, a deep lifelong learning optimization algorithm based on dense region fusion (DLLO-DRF) is proposed. This algorithm first obtains models for each stage of lifelong learning, and divides the model parameters for each stage into multiple regions based on the parameter values. Then, based on the dispersion of the parameter distribution, dense regions are dynamically obtained from the divided regions, and the parameters of the dense regions are averaged and fused to optimize the model. Finally, extensive experiments are conducted on the self-labeled transmission line defect dataset, and the results show that DLLO-DRF has the best performance among various comparative algorithms. lifelong learning; deep neural network; model integration; object detection +
Footnote †: journal:  
LFL [6], and DMC [7] use the output of the previous model as a soft label for the previous task to reduce forgetting and knowledge transfer. The second approach aims to enable neural networks to review previously learned knowledge through data replay. This is achieved by employing a standard first-in-first-out (FIFO) buffer and selectively storing old knowledge in long-term memory. The third approach involves expanding the network's capacity to retain more knowledge. This can be accomplished by employing redundant parameters in large deep networks, allowing them to accommodate the learning of new tasks while preserving previously acquired knowledge. These above approaches offer the advantage of not requiring explicit consideration of previously trained models when new data arrives. However, these approaches also present several challenges. Firstly, they often entail imposing restrictions on the learning of new data during model training. Secondly, they rely on using old data to \"review\" the model, which can be cumbersome. Lastly, expanding the model's parameter count to retain more knowledge may lead to increased complexity. Inspired by the theory of model integration technology, we propose an optimization method for deep lifelong learning based on dense region fusion (DLLO-DRF). This method aims to enhance the performance of the final lifelong learning model by effectively integrating the model parameters from each stage of lifelong learning [8; 9; 10]. By searching for model weight parameters that possess the capability to express data at all stages of lifelong learning, we can achieve significant improvements in the performance of lifelong learning models. This paper makes the following key contributions:  
* A deep lifelong learning optimization method, DLLO-DRF, is proposed, and its corresponding objective function is designed. This method analyzes the differences in the distribution of model parameters in different stages of lifelong learning, and then dynamically integrates these model parameters to obtain weight parameters that effectively capture the overall data patterns. * Dense region fusion is introduced into the objective function of DLLO-DRF, and the corresponding algorithm is designed. * Comparative experiments are conducted using self-labeled transmission line defect datasets, and with various deep object detection algorithms. The results show that DLLO-DRF effectively improves the performance of deep neural network models under lifelong learning. The following sections are organized as follows: Section 2 provides a review of existing works that are relevant to the subject matter discussed in this paper. Section 3 presents the materials and methods in detail, including the objective function and inference process of DLLO-DRF, and experimental material. Section 4 focuses on presenting the obtained results in detail.