# Forgetful Forests: Data Structures for Machine Learning on Streaming Data under Concept Drift
## Appendix B Ongoing Parameter Tuning  
We use the \(AdaptParameters(. )\) function to adaptively change \(currentParams.rSize\), \(maxHeight\), and \(currentParams.iRate\) based on changes in accuracy. \(AdaptParameters(. )\) is called when new data is acquired from the data stream and before function \(UpdateSubTree(. )\) is called. The \(rSize\) and \(maxHeight\) will be applied to the parameters when calling \(UpdateSubTree(.)\). The \(iRate\) and \(rSize\) will also be inputs to the next call to the \(AdaptParameters(. )\) function on this tree. The \(AdaptParameters(. )\) function will first test the accuracy of the model on new incoming data yielding \(newAcc\). The function then recalls the accuracy that was tested last time as \(lastAcc\). Next, because we want \(newAcc\) and \(lastAcc\) to improve upon random guessing, we subtract the accuracy of random uniform guessing from \(newAcc\) (the \(guessAcc\) was already subtracted from \(lastAcc\) in the last update). We posit that the accuracy of random guessing (\(guessAcc\)) to be \(1/nClasses\). The intuitive reason to subtract \(guessAcc\) is that a \(lastAcc\) that is no greater than \(guessAcc\) suggests that the model is no better than guessing just based on the number of classes. That, in turn, suggests that concept drift has likely occurred so old data should be discarded. Following that, \(AdaptParameters(. )\) will calculate the rate of change (\(rChange\)) of \(rSize\) by:  
* When \(newAcc/lastAcc\geq 1\), the \(max\) in the exponent will ensure that \(rChange\) will be \((newAcc/lastAcc)^{2}\). In this way, the \(rChange\) curves **slightly** upward when \(newAcc\) is equal to, or slightly higher than \(lastAcc\), but curves **steeply** upward when \(newAcc\) is much larger than \(lastAcc\). * When \(newAcc/lastAcc<1\), \(rChange\) is equal to \((newAcc/lastAcc)^{3-newAcc/lastAcc}\). In this way, \(rChange\) is flat or curves slightly downward when \(newAcc\) is **slightly** lower than \(lastAcc\) but curves **steeply** downwards when \(newAcc\) is much lower than \(lastAcc\). * Other functions to set \(rChange\) are possible, but this one has the following desirable properties: (i) it is continuous regardless of the values of \(newAcc\) and \(lastAcc\); (ii) \(rChange\) is close to 1 when \(newAcc\) is close to \(lastAcc\); (iii) when \(newAcc\) differs from \(lastAcc\) significantly in either direction, \(rChange\) reacts strongly. \(\big{|}\big{|}X_{n}^{i}\big{|}\big{|}\), because we do not want to forget any new incoming data. When \(\mathit{AdaptParameters}(. )\) is called the first time, we will set \(rSize=||X_{n}^{i}\big{|}+||X_{n}^{\mathit{rfrained}}||\). The two special cases happen when \(\mathit{newAcc}\leq 0\) or \(\mathit{lastAcc}\leq 0\). When \(\mathit{newAcc}\leq 0\), the prediction of the model is no better than random guessing. In that case, we infer that the old data cannot help in predicting new data, so we will forget all of the old data by setting \(rSize=\)\(||X_{n}^{i}\big{|}\). When \(\mathit{lastAcc}\leq 0\) but \(\mathit{newAcc}>0\), then all the old data may be useful. Thus, we set \(rSize=\)\(rSize+||X_{n}^{i}\big{|}\big{|}\). The above adaptation strategy requires a dampening parameter \(iRate\) to limit the increase rate of \(rSize\). When the accuracy is large, the model may be close to its maximum possible accuracy, so we may want a smaller \(iRate\) and in turn to increase \(rSize\) slower. After a drastic concept drift event, when the accuracy has been significantly decreased, we want to increase \(iRate\) to retain more new data after forgetting most of the old data. This will accelerate the creation of an accurate tree after the concept drift. To achieve this, we will adaptively change it as follows: set \(iRate\) (\(\mathit{new}\)) equal to \(iRate(old)*\mathit{lastAcc}/\mathit{newAcc}\) before each update to \(rSize\). \(iRate\) will not be changed if either \(\mathit{lastAcc}\leq 0\) or \(\mathit{newAcc}\leq 0\).