Package: niaarm
NiaARM - A minimalistic framework for Numerical Association Rule MiningDocumentation:https://niaarm.readthedocs.io/en/latestTested OS:Windows, Ubuntu, Fedora, Alpine, Arch, macOS.However, that does not mean it does not work on othersAbout 📋NiaARM is a framework for Association Rule Mining based on nature-inspired algorithms for optimization. 🌿 The framework is written fully in Python and runs on all platforms. NiaARM allows users to preprocess the data in a transaction database automatically, to search for association rules and provide a pretty output of the rules found. 📊 This framework also supports integral and real-valued types of attributes besides the categorical ones. Mining the association rules is defined as an optimization problem, and solved using the nature-inspired algorithms that come from the related framework calledNiaPy. 🔗Detailed insights 🔍The current version includes (but is not limited to) the following functions:loading datasets in CSV format 📁preprocessing of data 🧹searching for association rules 🔎providing output of mined association rules 📋generating statistics about mined association rules 📊visualization of association rules 📈association rule text mining (experimental) 📄Installation 📦pipInstall NiaARM with pip:pipinstallniaarmTo install NiaARM on Alpine Linux, please enable Community repository and use:$apkaddpy3-niaarmTo install NiaARM on Arch Linux, please use anAUR helper:$yay-Syyupython-niaarmTo install NiaARM on Fedora, use:$dnfinstallpython3-niaarmTo install NiaARM on NixOS, please use:nix-env-iAnixos.python311Packages.niaarmUsage 🚀Loading dataIn NiaARM, data loading is done via theDatasetclass. There are two options for loading data:Option 1: From a pandas DataFrame (recommended)importpandasaspdfromniaarmimportDatasetdf=pd.read_csv('datasets/Abalone.csv')# preprocess data...data=Dataset(df)print(data)# printing the dataset will generate a feature reportOption 2: Directly from a CSV filefromniaarmimportDatasetdata=Dataset('datasets/Abalone.csv')print(data)PreprocessingData SquashingOptionally, a preprocessing technique, called data squashing [5], can be applied. This will significantly reduce the number of transactions, while providing similar results to the original dataset.fromniaarmimportDataset,squashdataset=Dataset('datasets/Abalone.csv')squashed=squash(dataset,threshold=0.9,similarity='euclidean')print(squashed)Mining association rulesThe easy way (recommended)Association rule mining can be easily performed using theget_rulesfunction:fromniaarmimportDataset,get_rulesfromniapy.algorithms.basicimportDifferentialEvolutiondata=Dataset("datasets/Abalone.csv")algo=DifferentialEvolution(population_size=50,differential_weight=0.5,crossover_probability=0.9)metrics=('support','confidence')rules,run_time=get_rules(data,algo,metrics,max_iters=30,logging=True)print(rules)# Prints basic stats about the mined rulesprint(f'Run Time:{run_time}')rules.to_csv('output.csv')The hard wayThe above example can be also be implemented using a more low level interface,
with theNiaARMclass directly:fromniaarmimportNiaARM,Datasetfromniapy.algorithms.basicimportDifferentialEvolutionfromniapy.taskimportTask,OptimizationTypedata=Dataset("datasets/Abalone.csv")# Create a problem# dimension represents the dimension of the problem;# features represent the list of features, while transactions depicts the list of transactions# metrics is a sequence of metrics to be taken into account when computing the fitness;# you can also pass in a dict of the shape {'metric_name': <weight of metric in range [0, 1]>};# when passing a sequence, the weights default to 1.problem=NiaARM(data.dimension,data.features,data.transactions,metrics=('support','confidence'),logging=True)# build niapy tasktask=Task(problem=problem,max_iters=30,optimization_type=OptimizationType.MAXIMIZATION)# use Differential Evolution (DE) algorithm from the NiaPy library# see full list of available algorithms: https://github.com/NiaOrg/NiaPy/blob/master/Algorithms.mdalgo=DifferentialEvolution(population_size=50,differential_weight=0.5,crossover_probability=0.9)# run algorithmbest=algo.run(task=task)# sort rulesproblem.rules.sort()# export all rules to csvproblem.rules.to_csv('output.csv')Interest measuresThe framework implements several popular interest measures, which can be used to compute the fitness function value of rules
and for assessing the quality of the mined rules. A full list of the implemented interest measures along with their descriptions
and equations can be foundhere.VisualizationThe framework currently supports the hill slopes visualization method presented in [4]. More visualization methods are planned
to be implemented in future releases.frommatplotlibimportpyplotaspltfromniaarmimportDataset,get_rulesfromniaarm.visualizeimporthill_slopesdataset=Dataset('datasets/Abalone.csv')metrics=('support','confidence')rules,_=get_rules(dataset,'DifferentialEvolution',metrics,max_evals=1000,seed=1234)some_rule=rules[150]hill_slopes(some_rule,dataset.transactions)plt.show()Text Mining (Experimental)An experimental implementation of association rule text mining using nature-inspired algorithms, based on ideas from [5]
is also provided. Theniaarm.textmodule contains theCorpusandDocumentclasses for loading and preprocessing corpora,
aTextRuleclass, representing a text rule, and theNiaARTMclass, implementing association rule text mining
as a continuous optimization problem. Theget_text_rulesfunction, equivalent toget_rules, but for text mining, was also
added to theniaarm.minemodule.importpandasaspdfromniaarm.textimportCorpusfromniaarm.mineimportget_text_rulesfromniapy.algorithms.basicimportParticleSwarmOptimizationdf=pd.read_json('datasets/text/artm_test_dataset.json',orient='records')documents=df['text'].tolist()corpus=Corpus.from_list(documents)algorithm=ParticleSwarmOptimization(population_size=200,seed=123)metrics=('support','confidence','aws')rules,time=get_text_rules(corpus,max_terms=5,algorithm=algorithm,metrics=metrics,max_evals=10000,logging=True)print(rules)print(f'Run time:{time:.2f}s')rules.to_csv('output.csv')Note:You may need to download stopwords and the punkt tokenizer from nltk by runningimport nltk; nltk.download('stopwords'); nltk.download('punkt').For a full list of examples see theexamples folderin the GitHub repository.Command line interfaceWe provide a simple command line interface, which allows you to easily
mine association rules on any input dataset, output them to a csv file and/or perform
a simple statistical analysis on them. For more details see thedocumentation.niaarm-husage: niaarm [-h] [-v] [-c CONFIG] [-i INPUT_FILE] [-o OUTPUT_FILE] [--squashing-similarity {euclidean,cosine}] [--squashing-threshold SQUASHING_THRESHOLD] [-a ALGORITHM] [-s SEED] [--max-evals MAX_EVALS] [--max-iters MAX_ITERS]
              [--metrics METRICS [METRICS ...]] [--weights WEIGHTS [WEIGHTS ...]] [--log] [--stats]

Perform ARM, output mined rules as csv, get mined rules' statistics

options:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  -c CONFIG, --config CONFIG
                        Path to a TOML config file
  -i INPUT_FILE, --input-file INPUT_FILE
                        Input file containing a csv dataset
  -o OUTPUT_FILE, --output-file OUTPUT_FILE
                        Output file for mined rules
  --squashing-similarity {euclidean,cosine}
                        Similarity measure to use for squashing
  --squashing-threshold SQUASHING_THRESHOLD
                        Threshold to use for squashing
  -a ALGORITHM, --algorithm ALGORITHM
                        Algorithm to use (niapy class name, e.g. DifferentialEvolution)
  -s SEED, --seed SEED  Seed for the algorithm's random number generator
  --max-evals MAX_EVALS
                        Maximum number of fitness function evaluations
  --max-iters MAX_ITERS
                        Maximum number of iterations
  --metrics METRICS [METRICS ...]
                        Metrics to use in the fitness function.
  --weights WEIGHTS [WEIGHTS ...]
                        Weights in range [0, 1] corresponding to --metrics
  --log                 Enable logging of fitness improvements
  --stats               Display stats about mined rulesNote: The CLI script can also run as a python module (python -m niaarm ...)Reference Papers 📚Ideas are based on the following research papers:[1] I. Fister Jr., A. Iglesias, A. Gálvez, J. Del Ser, E. Osaba, I Fister.Differential evolution for association rule mining using categorical and numerical attributesIn: Intelligent data engineering and automated learning - IDEAL 2018, pp. 79-88, 2018.[2] I. Fister Jr., V. Podgorelec, I. Fister.Improved Nature-Inspired Algorithms for Numeric Association Rule Mining. In: Vasant P., Zelinka I., Weber GW. (eds) Intelligent Computing and Optimization. ICO 2020. Advances in Intelligent Systems and Computing, vol 1324. Springer, Cham.[3] I. Fister Jr., I. FisterA brief overview of swarm intelligence-based algorithms for numerical association rule mining. arXiv preprint arXiv:2010.15524 (2020).[4] Fister, I. et al. (2020).Visualization of Numerical Association Rules by Hill Slopes.
In: Analide, C., Novais, P., Camacho, D., Yin, H. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2020.
IDEAL 2020. Lecture Notes in Computer Science(), vol 12489. Springer, Cham.https://doi.org/10.1007/978-3-030-62362-3_10[5] I. Fister, S. Deb, I. Fister,Population-based metaheuristics for Association Rule Text Mining,
In: Proceedings of the 2020 4th International Conference on Intelligent Systems, Metaheuristics & Swarm Intelligence,
New York, NY, USA, mar. 2020, pp. 19–23. doi:10.1145/3396474.3396493.[6] I. Fister, I. Fister Jr., D. Novak and D. Verber,Data squashing as preprocessing in association rule mining, 2022 IEEE Symposium Series on Computational Intelligence (SSCI), Singapore, Singapore, 2022, pp. 1720-1725, doi:10.1109/SSCI51031.2022.10022240.See also[1]NiaARM.jl: Numerical Association Rule Mining in Julia[2]arm-preprocessing: Implementation of several preprocessing techniques for Association Rule Mining (ARM)LicenseThis package is distributed under the MIT License. This license can be found online athttp://www.opensource.org/licenses/MIT.DisclaimerThis framework is provided as-is, and there are no guarantees that it fits your purposes or that it is bug-free. Use it at your own risk!Cite usStupan, Ž., & Fister Jr., I. (2022).NiaARM: A minimalistic framework for Numerical Association Rule Mining. Journal of Open Source Software, 7(77), 4448.