# Population Dynamics in Genetic Programming for Dynamic Symbolic Regression
## 1 Introduction  
The integration of machine learning and artificial intelligence into various production systems, where data are continuously accumulated throughout the course of production processes, has become very common. This has led to the implementation of prediction models for various purposes, including estimating product qualities for accelerated product development, estimating the remaining useful lifetime of machines to facilitate timely maintenance scheduling, and acquiring comprehensive insights into production processes. In the conventional paradigm of static machine learning scenarios, the process involves obtaining training data, training a prediction model using a machine learning algorithm, and subsequently utilizing the trained model, as illustrated in Figure 1. This approach involves aggregating data over an extensive time frame and applying it to machine learning algorithms to generate fixed prediction models. However, in scenarios where prediction models need to be adapted during the ongoing production process to account for shifts and regime changes, it becomes imperative to re-fit these models to the most recent data. Figure 1: In traditional static data modeling, the machine learning algorithm processes the training data once, yielding a singular output model. This necessitates a shift from static modeling to dynamic modeling, where the data are inherently dynamic, changing over time. Despite this dynamic nature, the modeling process remains essentially static, repeating over changing data, as illustrated in Figure 2. While faster classical machine learning algorithms like Random Forests [1] or Support Vector Machines [2] can be employed in such scenarios by simply re-executing them, slower algorithms like genetic programming (GP) [3; 4] or training deep neural networks (NNs) [5] are often deemed impractical due to the runtime implications associated with their re-execution. Rather than resorting to the complete re-execution of machine learning algorithms, a more efficient approach involves incorporating previous results to leverage existing knowledge, thereby reducing the runtime significantly. For instance, when using neural networks, one can optimize the network by seeding the re-optimization process with the weights of the latest model, rather than seeding it with new, random weights. In the realm of population-based evolutionary algorithms, these can operate in an open-ended fashion, continually evolving and adapting to changing datasets. In this dynamic modeling scenario, the training data are perceived as dynamic, changing over time, with the machine learning algorithm continuously running and outputting prediction models dynamically, as illustrated in Figure 3.  
In industrial applications, the deployment of certain models is constrained due to their black-box nature. Notably, deep neural networks are often undesired as their predictions lack transparency, making it challenging for experts to comprehend the underlying prediction processes. This complicates the validation and interoperability of such models. On the contrary, white-box models, represented as symbolic models, are easily interpretable by humans, enabling validation by experts and enhancing the overall acceptance of machine learning models. The advantage of symbolic models lies in their simplicity, as they can be easily implemented in any system or programming language, given their nature as mathematical expressions. Additionally, symbolic models allow for various mathemat  
Figure 3: In dynamic modeling, the training data undergo changes throughout the machine learning algorithm, enabling continuous model output that adjusts to evolving data or improves with the discovery of more effective models. Figure 2: In scenarios with evolving data over time, one may need to re-run the entire machine learning algorithm to obtain a newly adapted model. ical transformations such as simplifications and the definition of derivatives, providing additional benefits in specific scenarios.