Package: arm-preprocessing
arm-preprocessingFree software:MIT licenseDocumentation:http://arm-preprocessing.readthedocs.ioPython: 3.9.x, 3.10.x, 3.11.x, 3.12xTested OS:Windows, Ubuntu, Fedora, Alpine, Arch, macOS.However, that does not mean it does not work on othersAbout 📋arm-preprocessing is a lightweight Python library supporting several key steps involving data preparation, manipulation, and discretisation for Association Rule Mining (ARM). 🧠 Embrace its minimalistic design that prioritises simplicity. 💡 The framework is intended to be fully extensible and offers seamless integration with related ARM libraries (e.g.,NiaARM). 🔗Why arm-preprocessing?While numerous libraries facilitate data mining preprocessing tasks, this library is designed to integrate seamlessly with association rule mining. It harmonises well with the NiaARM library, a robust numerical association rule mining framework. The primary aim is to bridge the gap between preprocessing and rule mining, simplifying the workflow/pipeline. Additionally, its design allows for the effortless incorporation of new preprocessing methods and fast benchmarking.Key features ✨Loading various formats of datasets (CSV, JSON, TXT, TCX) 📊Converting datasets to different formats 🔄Loading different types of datasets (numerical dataset, discrete dataset, time-series data, text, etc.) 📉Dataset identification (which type of dataset) 🔍Dataset statistics 📈Discretisation methods 📏Data squashing methods 🤏Feature scaling methods ⚖️Feature selection methods 🎯Installation 📦pipTo installarm-preprocessingwith pip, use:pipinstallarm-preprocessingTo installarm-preprocessingon Alpine Linux, please use:$apkaddpy3-arm-preprocessingTo installarm-preprocessingon Arch Linux, please use anAUR helper:$yay-Syyupython-arm-preprocessingUsage 🚀Data loadingThe following example demonstrates how to load a dataset from a file (csv, json, txt). More examples can be found in theexamples/data_loadingdirectory:Loading a dataset from a CSV fileLoading a dataset from a JSON fileLoading a dataset from a TCX fileLoading a time-series datasetfromarm_preprocessing.datasetimportDataset# Initialise dataset with filename (without format) and format (csv, json, txt)dataset=Dataset('path/to/datasets',format='csv')# Load datasetdataset.load_data()df=dataset.dataMissing valuesThe following example demonstrates how to handle missing values in a dataset using imputation. More examples can be found in theexamples/missing_valuesdirectory:Handling missing values in a dataset using row deletionHandling missing values in a dataset using column deletionHandling missing values in a dataset using imputationfromarm_preprocessing.datasetimportDataset# Initialise dataset with filename and formatdataset=Dataset('examples/missing_values/data',format='csv')dataset.load()# Impute missing datadataset.missing_values(method='impute')Data discretisationThe following example demonstrates how to discretise a dataset using the equal width method. More examples can be found in theexamples/discretisationdirectory:Discretising a dataset using the equal width methodDiscretising a dataset using the equal frequency methodDiscretising a dataset using k-means clusteringfromarm_preprocessing.datasetimportDataset# Initialise dataset with filename (without format) and format (csv, json, txt)dataset=Dataset('datasets/sportydatagen',format='csv')dataset.load_data()# Discretise dataset using equal width discretisationdataset.discretise(method='equal_width',num_bins=5,columns=['calories'])Data squashingThe following example demonstrates how to squash a dataset using the euclidean similarity. More examples can be found in theexamples/squashingdirectory:Squashing a dataset using the euclidean similaritySquashing a dataset using the cosine similarityfromarm_preprocessing.datasetimportDataset# Initialise dataset with filename and formatdataset=Dataset('datasets/breast',format='csv')dataset.load()# Squash datasetdataset.squash(threshold=0.75,similarity='euclidean')Feature scalingThe following example demonstrates how to scale the dataset's features. More examples can be found in theexamples/scalingdirectory:Scale features using normalisationScale features using standardisationfromarm_preprocessing.datasetimportDataset# Initialise dataset with filename and formatdataset=Dataset('datasets/Abalone',format='csv')dataset.load()# Scale dataset using normalisationdataset.scale(method='normalisation')Feature selectionThe following example demonstrates how to select features from a dataset. More examples can be found in theexamples/feature_selectiondirectory:Select features using the Kendall Tau correlation coefficientfromarm_preprocessing.datasetimportDataset# Initialise dataset with filename and formatdataset=Dataset('datasets/sportydatagen',format='csv')dataset.load()# Feature selectiondataset.feature_selection(method='kendall',threshold=0.15,class_column='calories')Related frameworks 🔗[1]NiaARM: A minimalistic framework for Numerical Association Rule Mining[2]uARMSolver: universal Association Rule Mining SolverReferences 📚[1] I. Fister, I. Fister Jr., D. Novak and D. Verber,Data squashing as preprocessing in association rule mining, 2022 IEEE Symposium Series on Computational Intelligence (SSCI), Singapore, Singapore, 2022, pp. 1720-1725, doi: 10.1109/SSCI51031.2022.10022240.[2] I. Fister Jr., I. FisterA brief overview of swarm intelligence-based algorithms for numerical association rule mining. arXiv preprint arXiv:2010.15524 (2020).LicenseThis package is distributed under the MIT License. This license can be found online
athttp://www.opensource.org/licenses/MIT.DisclaimerThis framework is provided as-is, and there are no guarantees that it fits your purposes or that it is bug-free. Use it at your own risk!