Package: preprocessy
Preprocessy is a framework that provides data preprocessing pipelines for machine learning. It bundles all the common preprocessing steps that are performed on the data to prepare it for machine learning models. It aims to do so in a manner that is independent of the source and type of dataset. Hence, it provides a set of functions that have been generalised to different types of data.The pipelines themselves are composed of these functions and flexible so that the users can customise them by adding their processing functions or removing pipeline functions according to their needs. The pipelines thus provide an abstract and high-level interface to the users.Pipeline StructureThe pipelines are divided into 3 logical stages -Stage 1 - Pipeline InputInput datasets with the following extensions are supported -.csv, .tsv, .xls, .xlsx, .xlsm, .xlsb, .odf, .ods, .odtStage 2 - ProcessingThis is the major part of the pipeline consisting of processing functions. The following functions are provided out of the box as individual functions as well as a part of the pipelines -Handling Null ValuesHandling OutliersNormalisation and ScalingLabel EncodingCorrelation and Feature ExtractionTraining and Test set splittingStage 3 - Pipeline OutputThe output consists of processed dataset and pipeline parameters depending on the verbosity required.ContributingPlease read ourContributing Guidebefore submitting a Pull Request to the project.SupportFeel free to contact any of the maintainers. We're happy to help!RoadmapCheck out ourroadmapto stay informed of the latest features released and the upcoming ones. Feel free to give us your insights!DocumentationThe documentation can be found athere. Currently, some parts of the documentation are under development. All contributions are welcome! Please see ourContributing Guide.Research Paper and CitationsPreprocessy: A Customisable Data Preprocessing Framework with High-Level APIswas presented at the2022 7th International Conference on Data Science and Machine Learning Applications (CDMA)and is published inIEEE Xplore.Link to full paper:https://ieeexplore.ieee.org/document/9736366If you're using Preprocessy as a part of scientific research, please use the below citations.Plain Text CitationS. Kazi et al., "Preprocessy: A Customisable Data Preprocessing Framework with High-Level APIs," 2022 7th International Conference on Data Science and Machine Learning Applications (CDMA), 2022, pp. 206-211, doi: 10.1109/CDMA54072.2022.00039.BibTeX Citation@INPROCEEDINGS{9736366,
  author={Kazi, Saif and Vakharia, Priyesh and Shah, Parth and Gupta, Riya and Tailor, Yash and Mantry, Palak and Rathod, Jash},
  booktitle={2022 7th International Conference on Data Science and Machine Learning Applications (CDMA)},
  title={Preprocessy: A Customisable Data Preprocessing Framework with High-Level APIs},
  year={2022},
  volume={},
  number={},
  pages={206-211},
  doi={10.1109/CDMA54072.2022.00039}}LicenseSee theLICENSEfile for licensing information.LinksDocumentation:https://preprocessy.readthedocs.io/en/latest/Changes:https://preprocessy.readthedocs.io/en/latest/changes/PyPI Releases:https://pypi.org/project/preprocessy/Source Code:https://github.com/preprocessy/preprocessyDatasets:https://drive.google.com/drive/folders/1MoMHNgd6KR5A_l5PkFIcxeax7lXm72l9?usp=sharingIssue Tracker:https://github.com/preprocessy/preprocessy/issuesChat:https://discord.gg/5q2yCqqU6N