Package: ande
Efficient Method for Optimizing Anomaly Detection with Clustering Algorithmsand for Unifiying in a PackageTo create a common platform for anomaly detection process with some popular clustering algorithms to be an easy solution for data analysis to verify the process data with other clustering algorithms.Table of ContentsAbout The ProjectBuilt WithGetting StartedPrerequisitesInstallationUsageRoadmapContributingLicenseContactAcknowledgmentsAbout The ProjectThe world of data is growing very fast, and it is a new challenge for data analysis to develop new methods to handle this massive amount of data. A large number of data have many hidden factors that need to be identified and used for different algorithms. Clustering is one of the significant parts of data mining. The term comes from the idea of classifying unsupervised data. Now-a-days a lot of algorithms are implemented. Besides that, all those algorithms have some limitations, creating an opportunity to innovate newalgorithms for clustering. The clustering process can be separated in six different ways: partitioning, hierarchical, density, gridmodel, and constraint-based models. The aim of the package is to implement various types of clustering algorithms and helps to determine which one is more accurate on detecting impure data from a large data set. To create a common platform for Some popluar algorithms for anomaly detection are implemented and converged all of them into a package(AnDe). The algorithms which are implemented and combined into the package are: K-means, DBSCAN, HDBSCAN, Isolation Forest, Local Outlier Factor and Agglomerative Hierarchical Clustering. The package reduce the consumption of time by compressing implementation hurdles of each algorithms. The package is also makes the anomaly detection procedure more robust by visualizing in a more precise way along with visualization of comparison in performance(accuracy, runtime and memory consumption) of those algorithmimplemented.Built WithFor using this package, some popular packages are need to be configured in the working environment.NumpyPandaMatplotlibTimeosSklearnHdbscanTracemalloc(back to top)Getting StartedThis is an example of how you set up thie pacage and use in you script.PrerequisitesAt first, need install the package in your working environment for using this package.pipinstallpython=3.8pipinstallnumpypipinstallpandaspipinstallmatplotlibpipinstalltimepipinstallospipinstallsklearnpipinstallHdbscanpipinstallTracemallocInstallationDownload the package from (https://github.com/cbiswascse/AUnifiedPackageForAnomalyDetection)Install the package in you environment.pipinstallcb-clusterImport the pacage in your script.fromEMOADCAUPimportCluster(back to top)UsageCall the cluster function.fromandeimportandeande.ClusterView()Input the Location of CSV file.Please,InputtheLocationofCSV:Select yes(y) If you have Catagorical data in your dataset.DoyouwanttoincludeCatagoricaldata[y/n]:Select yes(y) If you want to scaling your dataset with MinMaxScaler.ScalingdatawithMinMaxScaler[y/n]:Available Clusering Algorithm
Kmeans
Dbscan
Isolation Forest
Local Factor Outlier
Hdbscan
AgglomerativeChooseyourAlgorithm:Kmeans Clusering:Number of ClusterHowmanyclustersyouwant?:Select one of Average Method for Performance Metricsweighted,micro,macro,binaryDbscan:Input a Epsilon valueepsiloninDecimal:Input a Min Samples valueMinSamplesInInteger:Select one of Average Method for Performance Metricsweighted,micro,macro,binary11.Hdbscan:Minimum size of clusterMinimunsizeofclustersyouwant?:Select one of Average Method for Performance Metricsweighted,micro,macro,binary13.Isolation Forest:Contamination valueContaminationvaluebetween[0,0.5]:Select one of Average Method for Performance Metricsweighted,micro,macro,binaryLocal Outlier Factor:Contamination valueContaminationvaluebetween[0,0.5]:Select one of Average Method for Performance Metricsweighted,micro,macro,binary17.Agglomerative:Number of ClusterHowmanyclustersyouwant?:18.Select one of Average Method for Performance Metricsweighted,micro,macro,binary(back to top)LicenseDistributed under the MIT License. SeeLICENSE.txtfor more information.(back to top)ContactChandrima Biswas -cbiswascse26@gmail.comProject Link:https://github.com/cbiswascse/AUnifiedPackageForAnomalyDetection(back to top)AcknowledgmentsI would like to convey my heartfelt appreciation to my supervisor Prof.Dr. Doina Logofatu,for all her feedback, guidance, and evaluations during the work. Without her unique ideas, as well as her unwavering support and  encouragement, I would never have been able to complete this project.  In spite of her hectic schedule, she listened to my problem and gavethe appropriate advice.Furthermore, I express my very profound gratitude Prof. Dr. Peter Nauth for being the second supervisor of this work.