Package: gcimpute
gcimputeThis package provides a python implemention to impute missing values by fitting a Gaussian copula model, on incomplete dataset thay may contain continuous, ordinal, binary and truncated variables. The user could either fit a full rank Gaussian copula model [1] or a low rank Gaussian copula model [2]. The package also includes the mini-batch and online implementation of the full rank Gaussian copula model [3], which can accelerate the training process and adapt to a changing distribution in the streaming data. The fitted model can also be used for latent correlation learning and latent correlation change point detection (only for online data).InstallationThe easiest way is to install using pip:pip install gcimputeIf you want to customize the source code, you may install in the editable mode by firstgit clonethis respository, and then dopip install -e .in the cloned directory.Examplesfromgcimpute.gaussian_copulaimportGaussianCopulafromgcimpute.helper_dataimportgenerate_mixed_from_gcfromgcimpute.helper_evaluationimportget_smaefromgcimpute.helper_maskimportmask_MCARimportnumpyasnp# generate and mask 15-dim mixed data (5 continuous variables, 5 ordinal variables (1-5) and 5 boolean variables)X=generate_mixed_from_gc(n=2000)X_mask=mask_MCAR(X,mask_fraction=0.4)# model fittingmodel=GaussianCopula(verbose=1)X_imp=model.fit_transform(X=X_mask)# Evaluation: compute the scaled-MAE (SMAE) for each data type (scaled by MAE of median imputation)smae=get_smae(X_imp,X,X_mask)print(f'The SMAE across continous variables: mean{smae[:5].mean():.3f}and std{smae[:5].std():.3f}')print(f'The SMAE across oridnal variables: mean{smae[5:10].mean():.3f}and std{smae[5:10].std():.3f}')print(f'The SMAE across boolean variables: mean{smae[10:].mean():.3f}and std{smae[10:].std():.3f}')More detailed examples are available under directory Examples. Especially, themain tutorialcovers most functions' usage, and thus is a must-read if you are using our software for the first time.News05-06-2022The up-to-date version (0.0.2 on PyPI) achieves significant speed improvement when there are many ordinal variables. Strongly recommend an update if you are an early user!02-13-2022Our software went through a few name and structural changes. It was previously namedonline_gc_impand thenGaussianCopulaImp. The current namegcimputeis our final pick. The structural changes are substantial: we improved the code quality and the user interface. It now has an interface consistent withsklearn.impute. If you are an early user of our software, you may find your codes using our software no longer work after updating to our current release. We greatly appreciate your interest in our work and sincerely apologize for any inconvenience you may experience. Thenotebookwill help you quickly modify your codes to use our current release.References[1] Zhao, Y. and Udell, M. Missing value imputation for mixed data via Gaussian copula, KDD 2020.[2] Zhao, Y. and Udell, M. Matrix completion with quantified uncertainty through low rank Gaussian copula, NeurIPS 2020.[3] Zhao, Y., Landgrebe, E., Shekhtman E., and Udell, M. Online Missing Value Imputation and Change Point Detection
with the Gaussian Copula, AAAI 2022