Package: gawseed-tcorex
PurposeThe T-Corex algorithm is designed to identify changes in correlation
in datasets of large signals.  Given windows of signals.This package modifies theoriginal implementationHrayr Harutyunyan to
add documentation and CLI tools.UsageInstallationInstall withpip(orpipx):pip3 install gawseed-tcorexAnalysisTo analyze a temporal time-series of data in a csv file, run tcorex
similar to:tcorex -w 30 -n 100 -w 30 -g .1 -l .5 input_data.csv output.pklDiscussion and help with selecting parameters is discussed below.Displaying resultsTo list the correlations within each window, usetcorex-changepointson the resulting pkl file:tcorex-changepoints output.pklTo show a graph of which time periods had the highest rate of
correlation changes, usetcorex-plot:tcorex-plot -o changes output.pklPicking algorithm parametersNumber of latent factors (-n)Look at the number of signals you have, and if you have a lot there
should be more latent factors.  If
you pick a value too low, then all the signals will appear to be
correlated even when they are not.  If you pick a value which is too
large, only an identity matrix will be produced and all the signals
will appear independent.  Typically start with a roughly small value
and raise it to improve results with respect to signals that are
actually related.The number of latent factors should be roughly 2 order of magnitudes
lower (IEnumber_of_signals/100).Window size (-w)The algorithm is designed to look for changes in correlations between
two windows.The window size indicates how frequently changes will happen to
attempt to find changes between different window sizes.  If the window
is too small, then the accuracy will be reduced and more correlations
will be found as signals always appear similar at some point when
broken into small chunks.  If the window is too large, then no changes
will be easily detected because the difference between varying signals
will appear large.  When windows are large, you also lose track about
where correlation changes actually happen because the change point can
be anywhere in the window.GammaRange: 0.1-10The Gamma value sets the decay rate for sample weights.  Bigger values
will result in more data being included, which is good for smaller
datasets.  If enough data is present, then low values should be used.L1 regularization strengthHigher L1 values are better better for signals with rapid changesL2Better for smooth time series.  There is no current CLI option for this.Correlation Explanation MethodsOfficial implementation of linear correlation explanation (linear CorEx) and temporal correlation explanation (T-CorEx) methods.Linear CorExLinear CorEx searches for independent latent factors that explain all correlations between observed variables, while also biasing
the model selection towards modular latent factor models – directed latent factor graphical models
where each observed variable has a single latent variable as its only parent.
This is useful for covariance estimation, clustering related variables, and dimensionality reduction, especially in the high-dimensional and under-sampled regime.
The complete description of the method is presented in NeurIPS 2019paper"Fast structure learning with modular regularization"by Greg Ver Steeg, Hrayr Harutyunyan, Daniel Moyer, and Aram Galstyan.
If you want to cite this paper, please use the following BibTex entry:@incollection{NIPS2019_9691,
title = {Fast structure learning with modular regularization},
author = {Ver Steeg, Greg and Harutyunyan, Hrayr and Moyer, Daniel and Galstyan, Aram},
booktitle = {Advances in Neural Information Processing Systems 32},
editor = {H. Wallach and H. Larochelle and A. Beygelzimer and F. d\textquotesingle Alch\'{e}-Buc and E. Fox and R. Garnett},
pages = {15567--15577},
year = {2019},
publisher = {Curran Associates, Inc.},
url = {http://papers.nips.cc/paper/9691-fast-structure-learning-with-modular-regularization.pdf}
}Note:Greg Ver Steeg has an alternative implementation of linear CorEx, which is available atgithub.com/gregversteeg/LinearCorex.
That implementation uses a quazi-Newton optimization method for learning the model parameters.
In contrast, the implementation provided in this repository uses ADAM optimizer.
This latter implementation utilizes GPUs better, and can converge to slightly better objective values if the input data is highly non-modular.
Nevertheless, we highly encourage to take a look at the alternative implementation.T-CorExT-CorEx is a method for covariance estimation from temporal data.
It trains a linear CorEx for each time period,
while employing two regularization techniques to enforce temporal consistency of estimates.
The method is introduced in thepaper"Efficient Covariance Estimation from Temporal Data"by Hrayr Harutunyan, Daniel Moyer, Hrant Khachatrian, Greg Ver Steeg, and Aram Galstyan.
If you want to cite this paper, please use the following BibTex entry:@article{tcorex,
  title={Efficient Covariance Estimation from Temporal Data},
  author={Harutyunyan, Hrayr and Moyer, Daniel and Khachatrian, Hrant and Steeg, Greg Ver and Galstyan, Aram},
  journal={arXiv preprint arXiv:1905.13276},
  year={2019}
}Both linear CorEx and T-CorEx have linear time and memory complexity with respect to the number of observed variables and can be applied to high-dimensional datasets.
For example, it takes less than an hour on a moderate PC to estimate the covariance structure for time series with 100K variables using T-CorEx.
Both methods are implemented in PyTorch and can run on CPUs and GPUs.Requirements and InstallationThe code is writen in Python 3, but should run on Python 2 as well.
The dependencies are the following:numpy, scipy, tqdm, PyTorch[optional] nibabel (for fMRI experiments)[optional] nose (for tests)[optional] sklearn, regain, TVGL, linearcorex, pandas (for running comparisions)[optional] matplotlib and nilearn (for visualizations)To install the code, run the following command:python setup.py installDescriptionThe main method for linear CorEx is the classtcorex.Corex, and that of T-CorEx is 'tcorex.TCorex'.
The complete description of parameters of these classes can be found in the corresponding docstrings.
While there are many parameters (especially for T-CorEx), in general only a couple of them need to be tuned (others are set to their "best" values).
Those parameters are:ParameterLinear CorExT-CorExDescriptionm++The number of latent variables. Usually this is much smaller than the number of observed variables.l1-+A non-negative real number specifying the coefficient of l1 temporal regularization.gamma-+A real number in [0,1]. This argument controls the sample weights. The samples of time period t' will have weight w_t(t')=gamma^|t' - t| when estimating quantities for time period t. Smaller values are used for very dynamic time series.UsageRun the following command for a sample run of TCorex.python-mexamples.sample_runThe code is shown below:from__future__importprint_functionfrom__future__importabsolute_importfromtcorex.experiments.dataimportload_modular_sudden_changefromtcorex.experimentsimportbaselinesfromtcoreximportbasefromtcoreximportTCorexfromtcoreximportcovarianceascov_utilsimportnumpyasnpimportmatplotlibmatplotlib.use('agg')frommatplotlibimportpyplotaspltdefmain():nv=32# number of observed variablesm=4# number of hidden variablesnt=10# number of time periodstrain_cnt=16# number of training samples for each time periodval_cnt=4# number of validation samples for each time period# Generate some data with a sudden change in the middle.data,ground_truth_sigma=load_modular_sudden_change(nv=nv,m=m,nt=nt,ns=(train_cnt+val_cnt))# Split it into train and validation.train_data=[X[:train_cnt]forXindata]val_data=[X[train_cnt:]forXindata]# NOTE: the load_modular_sudden_change function above creates data where the time axis# is already divided into time periods. If your data is not divided into time periods# you can use the following procedure to do that:# bucketed_data, index_to_bucket = make_buckets(data, window=train_cnt + val_cnt, stride='full')# where the make_buckets function can be found at tcorex.experiments.data# The core method we have is the tcorex.TCorex class.tc=TCorex(nt=nt,nv=nv,n_hidden=m,max_iter=500,device='cpu',# for GPU set 'cuda',l1=0.3,# coefficient of temporal regularization termgamma=0.3,# parameter that controls sample weightsverbose=1,# 0, 1, 2)# Fit the parameters of T-CorEx.tc.fit(train_data)# We can compute the clusters of observed variables for each time period.t=8clusters=tc.clusters()print("Clusters at time period{}:{}".format(t,clusters[t]))# We can get an estimate of the covariance matrix for each time period.# When normed=True, estimates of the correlation matrices will be returned.covs=tc.get_covariance()# We can visualize the covariance matrices.fig,ax=plt.subplots(1,figsize=(5,5))im=ax.imshow(covs[t])fig.colorbar(im)ax.set_title("Estimated covariance matrix\nat time period{}".format(t))fig.savefig('covariance-matrix.png')# It is usually useful to compute the inverse correlation matrices,# since this matrices can be interpreted as adjacency matrices of# Markov random fields.cors=tc.get_covariance(normed=True)inv_cors=[np.linalg.inv(x)forxincors]# We can visualize the thresholded inverse correlation matrices.fig,ax=plt.subplots(1,figsize=(5,5))thresholded_inv_cor=np.abs(inv_cors[t])>0.05ax.imshow(thresholded_inv_cor)ax.set_title("Thresholded inverse correlation\nmatrix at time period{}".format(t))fig.savefig('thresholded-inverse-correlation-matrix.png')# We can also plot the Frobenius norm of the differences of inverse# correlation matrices of  neighboring time periods. This is helpful# for detecting the sudden change points of the system.diffs=cov_utils.diffs(inv_cors)fig,ax=plt.subplots(1,figsize=(5,5))ax.plot(diffs)ax.set_xlabel('t')ax.set_ylabel('$||\Sigma^{-1}_{t+1} - \Sigma^{-1}_{t}||_2$')ax.set_title("Frobenius norms of differences between\ninverse correlation matrices")fig.savefig('inv-correlation-difference-norms.png')# We can also do grid search on a hyperparameter grid the following way.# NOTE: this can take time!baseline,grid=(baselines.TCorex(tcorex=TCorex,name='T-Corex'),{'nv':nv,'n_hidden':m,'max_iter':500,'device':'cpu','l1':[0.0,0.03,0.3,3.0],'gamma':[1e-6,0.3,0.5,0.8]})best_score,best_params,best_covs,best_method,all_results=baseline.select(train_data,val_data,grid)tc=best_method# this is the model that performed the best on the validation data, you can use it as abovebase.save(tc,'best_method.pkl')if__name__=='__main__':main()