# Population Dynamics in Genetic Programming for Dynamic Symbolic Regression
## 2 Materials and Methods
### Population Dynamics  
Given that benchmark instances are generated based on a sum of weighted terms, our aim is to analyze the presence and significance of these terms within the current population. While an exact occurrence analysis using subtree matching would be challenging, especially for complex generating functions such as those in Friedman-based benchmarks, we approach our analysis based on the known variables for each term. Our focus lies on two aspects:  
Variable frequency, measuring the relative occurrence of a variable within the entire population.  
Variable impact, measuring the importance of a variable, based on Breiman [1]'s permutation feature importance.  
The variable frequency is computed across the entire population, while variable impacts are defined on an individual basis. The impact of a variable is determined by measuring the reduction in a model's accuracy when the variable is absent from the dataset. This absence is simulated by shuffling the variable, disrupting its relationship with the target and other correlated inputs. To obtain variable impacts for the entire population, we calculate the mean impact over all individuals for each feature. Importantly, we bear in mind the fact that individual feature importance essentially measures the drop in model accuracy, and low-accuracy models will contribute less to the overall impact. Thus, the impact calculated over the population is not inflated by models with low accuracy. Alternatively, scaling the impact with model accuracy is possible, but it could lead to a doubly scaled measure.  
In summary, variable frequency indicates the presence of terms in the population, irrespective of their relevance to the regression task. On the other hand, variable impacts assess  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Name** & **Type** & **Train. Size** & **Test Size** & **Features** & **Complexity** \\ \hline W1 & [PERSON] et al. [41] & 100 & 10 & 3 & low \\ W2 & [PERSON] et al. [41] & 100 & 10 & 8 & low \\ W3 & [PERSON] et al. [41] & 100 & 10 & 10 & medium \\ F1 & [PERSON] [42] & 1000 & 100 & 3 & high \\ F2 & [PERSON] [42] & 1000 & 100 & 6 & high \\ \hline \hline \end{tabular}
\end{table}
Table 1: Overview of the parameters of the used benchmark instances.  
Figure 5: Hidden states over epochs for the three terms, a, b, and c, for the Friedman-based benchmarks.  
the importance of terms for the current regression task. In a well-behaved GP scenario, the population should converge toward solutions with high impacts, maintaining a steady frequency for variables of high impact while dropping out variables with no importance.  
The interplay between the variable frequency and impacts allows us to analyze whether GP can initially identify useful variables and subsequently adapt to changing scenarios. Adaptation may involve the following:  
* Increasing the frequency of variables gaining importance due to different scenarios;
* Decreasing the frequency of variables no longer relevant due to a change in the scenario.  
Considering the correct usage of variables in terms of the model structure, we hypothesize that variable impacts tend to increase slowly over time during GP convergence but can drop instantly in the face of an abrupt shift in the dynamic problem. In contrast, variable frequencies should change slowly, either gradually increasing for relevant variables or decreasing for variables that are no longer relevant.