Package: pangoro
Project descriptionPangoro: powerful Python data cleaning toolkitWhat is it?Pangorois a Python package that provides fast and flexible methods for cleaning numerical and categorical features in a dataframe. It aims to be a fundamental tool for doing data wrangling in Python.Main Features:For numerical features, pangoro provides the following tools:Handle NA, drop, replace with mean, replace with mode, lamda function, replace with a number, replace with min replace with max, Use KNN classifications. KeepHandle outliers, Keep, PercentileScaling, standard or min maxConvert to nuemericalScan and apply to all numerical or supply a list of featuresFor categorical nominal features, pangoro provides the following tools:Handle NA, Drop, replace with mode, use KNN classification for imputation.Replace with sequence numbers based on supplied dictionary or based on alphabetical order.Scan and apply to all Categorical or supply a list of featuresWhere to get it:The source code is currently hosted onGitHubBinary installers for the latest released version are available at the Python Package IndexPyPipip install --upgrade pangoroUsage1. ImportImport what you need from the pangoro package. The choices are:PangoroDataFrame, a class for cleaning numerical and categorical features in a dataframe in addition to plotting features correlations.For this demonstration, we will import:>>> from pangoro.preprocessing import PangoroDataFrameFor these examples, we'll also use pandas to :>>> import pandas as pd2. Load some DataTypicall data will is read from a tabular formatted file, but for illustration we'll create a simple dataframe from a Python dict:>>> sample_data = pd.DataFrame({'pet':      ['cat', 'dog', 'dog', pd.NA, 'cat', 'dog', 'cat', 'fish'],
                       'color': ['white', 'brown', 'black', 'gold', pd.NA,'black', 'white', 'silver'],
                       'weight':   [50.5, 22.5, 4, pd.NA , 12, 39, 16, pd.NA]})
>>> sample_data['weight'] = pd.to_numeric(sample_data['weight'])
>>> sample_data.dtypes3. Create Instance and Apply Transformation3.1 We will start by creating an instance of PangoroDataFrame using the created sample_data DataFrame above:>>> pdf = PangoroDataFrame(sample_data)3.2 Then, we will automate cleaning of numerical features in PangoroDataFrame by replacing NAs with mean and without performing any scaling:>>> pdf.numerical_transformation(col = ['weight'], na_treat = 'mean', out_treat = True, out_upper = 0.8,
                         out_lower = 0.2, scaling='no', knn_neighbors = 0)Similarly, as we continue with this example and usingpdfas an instancePangoroDataFramewe can apply the other functions to clean and preprocess categorical ordinal by callingpdf.categorical_ordinal_transformationor categorical nominal by callingpdf.categorical_nominal_transformationIn addition, we can quickly plot correlations between features by calling:pdf.plot_all_correlationsNOTEPangoroDataFrame object has to be re-assigned to itself in order to store changes inplace after transforming nominal categories by callingcategorical_nominal_transformation, which will result in adding new features to the original PangoroDataFrame, for example:>>> pdf =pdf.categorical_nominal_transformation(col=['weight'],transform=True )4. Getting HelpTo learn more about we can issue the following command:>>> help(PangoroDataFrame)As a result we get detailed information about the package and each function included.5. Unit Testpangoro package is equipped with test module that contains TestDataFrame class to perform all unit tests required to validate output of the PangoroDataFrame functions. Tests can be called by the follwoing command from the main pangoro package folder:>>> python -m tests.testAs a result, all test functions will be called and performed and the output will be the number of performed tests along with results of the tests (i.e. OK or FAILED(errors))LicenseMITBackgroundWork on pangoro started in 2022 by a group of IE University students and has been under active development since then.Contributing to pangoroAll contributions, bug reports, bug fixes, documentation improvements, enhancements, and ideas are welcome.
Please contact us onGitHub