Package: augmented-pca
AugmentedPCAOverviewThis Python package provides implementations of Augmented Principal Component Analysis (AugmentedPCA) - a family of linear factor models that find a set of factors aligned with anaugmenting objectivein addition to the canonical PCA objective of finding factors that represent the data variance. AugmentedPCA can be split into two general families of models: adversarial AugmentedPCA and supervised AugmentedPCA.Supervised AugmentedPCAIn supervised AugmentedPCA (SAPCA), the augmenting objective is to make the factorsalignedwith the data labels, or some outcome, in addition to having the factors explain the variance of the original observed or primary data. SAPCA is useful whenpredictivityof latent components with respects to a set of data labels or outcomes is desired. SAPCA is equivalent to a supervised autoencoder (SAE) with a single hidden layer. Therefore, SAPCA can be applied to situations where the properties of latent representations enforced via deep SAEs are desired, yet where limited data or training inconsistencies are a concern. Below is a diagram depicting the relationship between primary data, supervision data, and the resulting SAPCA factors.Adversarial AugmentedPCAIn adversarial AugmentedPCA (AAPCA), the augmenting objective is to make the factorsorthogonalto a set of concomitant data, in addition to having the factors explain the variance of the original observed or primary data. AAPCA can be used in situations where one wishes to enforceinvarianceof latent components to a set of concomitant data, and is equivalent to an adversarial autoencoder with a single hidden layer. Below is a diagram depicting the relationship between primary data, concomitant data, and the resulting AAPCA factors.DocumentationDocumentation for AugmentedPCA is available on thisdocumentation site.Provided documentation includes:Motivation - Motivation behind AugmentedPCA models and the different approximate inference strategies.Model formulation - Overview of different models and approximate inference strategies as well as more in-depth mathematical descriptions.Tutorials - Step-by-step guide on how to use the different offered AugmentedPCA models.Examples - Use case examples for the different models.DependenciesThe AugmentedPCA package is written in Python, and requires Python >= 3.6 to run. AugmentedPCA relies on the following libraries and version numbers:Python>= 3.6NumPy>= 1.19.2SciPy>= 1.5.2InstallationTo install the latest stable release, usepip. Use the following command to install:$ pip install augmented-pcaIssue Tracking and ReportsPlease use theGitHub issue trackerassociated with the AugmentedPCA repository for issue tracking, filing bug reports, and asking general questions about the package or project.Quick IntroductionA quick guide to using AugmentedPCA is given in this section. For a more in-depth guide, see ourdocumentation.Importing AugmentedPCA ModelsAPCA models can be imported from themodels.pymodule. Below we show an example of importing the AAPCA model.# Import all AugmentedPCA modelsfromapca.modelsimportAAPCAAlternatively, all offered AugmentedPCA models can be imported at once.# Import all AugmentedPCA modelsfromapca.modelsimport*Instantiating AugmentedPCA ModelsAPCA models are instantiated by assigning either an SAPCA or AAPCA object to a variable. During instantiation, one has the option to define parametersn_components,mu, which represent the number of components and the augmenting objective strength, respectively. Additionally, approximate inference strategy can be defined through theinferenceparameter.# Define model parametersn_components=2# factors will have dimensionality of 2mu=1.0# augmenting objective strength equal to 1inference='encoded'# encoded approximate inference strategy# Instantiate adversarial AugmentedPCA modelaapca=AAPCA(n_components=n_components,mu=mu,inference=inference)Fitting AugmentedPCA ModelsAPCA models closely follow the style and implemention ofscikit-learn's PCA implementation, with many of the same methods and functionality. Similar to scikit-learn models, AugmentedPCA models are fit using thefit()method.fit()takes two parameters:Xwhich represents the matrix of primary data andYwhich represents the matrix of augmenting data.# Import numpyimportnumpyasnp# Generate synthetic data# Note: primary and augmenting data must have same number of samples/same first dimension sizen_samp=100X=np.random.randn(n_samp,20)# primary data, 100 samples with dimensionality of 20Y=np.random.randn(n_samp,3)# concomitant data, 100 samples with dimensionality of 3# Fit adversarial AugmentedPCA instanceaapca.fit(X=X,Y=Y)Alternatively, AugmentedPCA models can be fit using thefit_transform()method, which takes the same parameters as thefit()method but also returns a matrix of components or factors.# Fit adversarial AugmentedPCA instance and generate componentsS=aapca.fit_transform(X=X,Y=Y)Approximate Inference StrategiesIn this section, we give a brief overview of the different approximate inference strategies offered for AugmentedPCA. Inference strategy should be chosen based on the data on which the AugmentedPCA model will be used as well as the specific use case. Both SAPCA and AAPCA models use the jointly-encoded approximate inference strategy by default.LocalIn the local approximate inference strategy, the factors (local variables associated with each observation) are included in both the likelihood relating and the augmenting objective. Below is a diagram from our paper depicting the local inference strategy.Because the local variables are included in the augmenting objective, given new data we must have both primaryandaugmenting data to obtain factors. Thus, the local inference strategy should only be used for inference on new data when both primary and augmenting data are available. Below we show an example of how to fit a SAPCA model with local approximate inference strategy to training data and obtain factors for test data.# Import numpyimportnumpyasnp# Import supervised AugmentedPCAfromapca.modelsimportSAPCA# Generate synthetic data and labelsn_samp=100X=np.random.randn(n_samp,20)Y=np.random.randint(low=0,high=1,size=(n_samp,1),dtype=int)# Generate test/train splitstrain_pct=0.7idx=np.arange(start=0,stop=101,step=1,dtype=int)np.random.shuffle(idx)n_train=int(train_pct*len(idx))train_idx=idx[:n_train]test_idx=idx[n_train:]# Split data into test/train setsX_train=X[train_idx,:]X_test=X[test_idx,:]Y_train=Y[train_idx,:]Y_test=Y[test_idx,:]# Instantiate supervised AugmentedPCA model with local approximate inference strategysapca=SAPCA(n_components=3,mu=5.0,inference='local')# Fit supervised AugmentedPCA modelsapca.fit(X=X_train,Y_train)# Generate components for test set# Note: both primary and augmenting data are needed to obtain factorsS_test=sapca.transform(X=X_test,Y=Y_test)Note that when factors are generated for the test set that thetransform()method requires both the primary dataX_testand labelsY_testbe passed as parameters. For a more in-depth description of the local approximate inference strategy, see ourpaperor the correspondingdocumentation section.EncodedIn the encoded approximate inference strategy, a linear encoder is used to transform the data into factors or components. This inference strategy is termed "encoded" because the augmenting objective is enforced via an encoding function. Below is a diagram depicting the encoded inference strategy.In contrast to the local inference strategy, when factors are generated for the test set under the encoded inference strategy thetransform()method only requires the primary dataX_test. Below we show an example of how to fit a SAPCA model with encoded approximate inference strategy to training data and obtain factors for test data.# Instantiate supervised AugmentedPCA model model with encoded approximate inference strategysapca=SAPCA(n_components=3,mu=5.0,inference='encoded')# Fit supervised AugmentedPCA model# Note: both primary and augmenting data are required to fit the modelsapca.fit(X=X_train,Y_train)# Generate components for test set# Note: only primary data are needed to obtain factorsS_test=sapca.transform(X=X_test)For a more in-depth description of the encoded approximate inference strategy, see ourpaperor the correspondingdocumentation section.Jointly-EncodedThe jointly-encoded approximate inference strategy is similar to the encoded in that the augmenting objective is enforced through a linear encoding matrix. However, in the jointly-encoded inference strategy both the primary and augmenting data are required for computing factors, similar to the local inference strategy. Below is a diagram depicting the jointly-encoded inference strategy.Similar to the local inference strategy, when factors are generated for the test set under the jointly-encoded inference strategy thetransform()method requires both the primary dataX_testand augmenting dataY_test. Below we show an example of how to fit a SAPCA model with jointly-encoded approximate inference strategy to training data and obtain factors for test data.# Instantiate supervised AugmentedPCA model model with encoded approximate inference strategysapca=SAPCA(n_components=3,mu=5.0,inference='encoded')# Fit supervised AugmentedPCA model# Note: both primary and augmenting data are required to fit the modelsapca.fit(X=X_train,Y_train)# Generate components for test set# Note: both primary and augmenting data are needed to obtain factorsS_test=sapca.transform(X=X_test)For a more in-depth description of the jointly-encoded approximate inference strategy, see ourpaperor the correspondingdocumentation section.CitationPlease cite our paper if you find this package helpful in your research:@inproceedings{carson2021augmentedpca,
title={{AugmentedPCA}: {A} {P}ython {P}ackage of {S}upervised and {A}dversarial {L}inear {F}actor {M}odels},
author={{Carson IV}, William E. and Talbot, Austin and Carlson, David},
year={2021},
month={December},
booktitle={{P}roceedings of {L}earning {M}eaningful {R}epresentations of {L}ife {W}orkshop at {NeurIPS} 2021}}FundingThis project was supported by the National Institute of Biomedical Imaging and Bioengineering and the National Institute of Mental Health through the National Institutes of Health BRAIN Initiative under Award Number R01EB026937.