Package: changeforest
Random Forests for Change Point DetectionChange point detection aims to identify structural breaks in the probability
distribution of a time series. Existing methods either assume a parametric model for
within-segment distributions or are based on ranks or distances and thus fail in
scenarios with a reasonably large dimensionality.changeforestimplements a classifier-based algorithm that consistently estimates
change points without any parametric assumptions, even in high-dimensional scenarios.
It uses the out-of-bag probability predictions of a random forest to construct a
classifier log-likelihood ratio that gets optimized using a computationally feasible two-step
method.See [1] for details.InstallationTo install fromconda-forge(recommended), runcondainstall-cconda-forgechangeforestTo install fromPyPI, runpipinstallchangeforestExampleIn the following example, we perform random forest-based change point detection on
a simulated dataset withn=600observations and covariance shifts att=200, 400.In[1]:importnumpyasnp...:...:Sigma=np.full((5,5),0.7)...:np.fill_diagonal(Sigma,1)...:...:rng=np.random.default_rng(12)...:X=np.concatenate(...:(...:rng.normal(0,1,(200,5)),...:rng.multivariate_normal(np.zeros(5),Sigma,200,method="cholesky"),...:rng.normal(0,1,(200,5)),...:),...:axis=0,...:)The simulated datasetXcoincides with thechange in covariance(CIC) setup
described in [1]. Observations in the first and last segments are independently drawn
from a standard multivariate Gaussian distribution. Observations in the second segment
are i.i.d. normal with mean zero and unit variance, but with a covariance of ρ = 0.7
between coordinates. This is a challenging scenario.In[2]:fromchangeforestimportchangeforest...:...:result=changeforest(X,"random_forest","bs")...:resultOut[2]:best_splitmax_gainp_value(0,600]40014.8140.005¦--(0,400]20059.3140.005¦¦--(0,200]6-1.950.67¦°--(200,400]393-8.6680.81°--(400,600]412-9.0470.66In[3]:result.split_points()Out[3]:[200,400]changeforestcorrectly identifies the change points att=200andt=400. Thechangeforestfunction returns aBinarySegmentationResult. We use itsplotmethod
to investigate the gain curves maximized by the change point estimates:In[4]:result.plot().show()Change point estimates are marked in red.Formethod="random_forest"andmethod="knn", thechangeforestalgorithm uses a two-step approach to
find an optimizer of the gain. This fits a classifier for three split candidates
at the segment's 1/4, 1/2 and 3/4 quantiles, computes approximate gain curves using
the resulting classifier log-likelihood ratios and selects the overall optimizer as a second guess.
We can investigate the gain curves from the optimizer using theplotmethod ofOptimizerResult.
The initial guesses are marked in blue.In[5]:result.optimizer_result.plot().show()One can observe that the approximate gain curves are piecewise linear, with maxima
around the true underlying change points.TheBinarySegmentationResultreturned bychangeforestis a tree-like object with attributesstart,stop,best_split,max_gain,p_value,is_significant,optimizer_result,model_selection_result,left,rightandsegments.
These can be interesting to investigate the output of the algorithm further.Thechangeforestalgorithm can be tuned with hyperparameters. Seeherefor their descriptions and default values. In Python, the parameters can
be specified with theControlclass,
which can be passed tochangeforest. The following will build random forests with
50 trees:In[6]:fromchangeforestimportControl...:changeforest(X,"random_forest","bs",Control(random_forest_n_estimators=50))Out[6]:best_splitmax_gainp_value(0,600]4167.4630.01¦--(0,416]20043.9350.005¦¦--(0,200]193-14.9930.945¦°--(200,416]217-9.130.085°--(416,600]591-12.071Thechangeforestalgorithm still detects change points att=200, but is slightly off
witht=416.Due to the nature of the change,method="change_in_mean"is unable to detect any
change points at all:In[7]:changeforest(X,"change_in_mean","bs")Out[7]:best_splitmax_gainp_value(0,600]5898.625References[1] M. Londschien, P. Bühlmann and S. Kovács (2023). "Random Forests for Change Point Detection" Journal of Machine Learning Research