Package: cfof
CFOF (Concentration Free Outlier Factor)🚧Work In Progress..Python implementation of Concentration Free Outlier Factor (CFOF)[1].CFOF propertiesConcentrationfreeDoes not suffer of thehubnessproblemSemi–localityfast-CFOFalgorithm allows to calculate reliably CFOF scores with linear cost both in the dataset size and dimensionalityInstallationTo install the latest release:$ pip install cfofUsageImportCFOFandFastCFOF.>>>fromcfofimportCFOF,FastCFOF>>>importnumpyasnpLoad data.>>>X=np.array([[-1,-1],[-2,-1],[-3,-2],[1,1],[2,1],[3,2]])InstantiateCFOForFastCFOF, then call.compute(X)to calculate the scores..compute(X)returnssc, wheresc[i, l]is score of objectiforϱ_l(rhos[l]).You can also calculate CFOF scores from a precomputed distance matrix using.compute_from_distance_matrix().CFOF (hard-CFOF)Usecomputeto compute CFOF scores directly from data.>>>cfof_clf=CFOF(metric='euclidean',rhos=[0.5,0.6],n_jobs=1)>>>cfof_clf.compute(X)array([[0.5,0.66666667],[0.33333333,0.83333333],[0.5,1.],[0.5,0.66666667],[0.33333333,0.83333333],[0.5,1.]])Usecompute_from_distance_matrixto compute CFOF scores from a precomputed
distance matrix.>>>fromsklearn.metricsimportpairwise_distances>>>distance_matrix=pairwise_distances(X,metric='euclidean')>>>cfof_clf.compute_from_distance_matrix(distance_matrix)array([[0.5,0.66666667],[0.33333333,0.83333333],[0.5,1.],[0.5,0.66666667],[0.33333333,0.83333333],[0.5,1.]])FastCFOF (soft-CFOF)Usecomputeto compute CFOF scores directly from data.>>>np.random.seed(10)>>>X=np.random.randint(0,100,size=(1000,3))>>>>>>fast_cfof_clf=FastCFOF(metric='euclidean',...rhos=[0.001,0.005,0.01,0.05,0.1],...epsilon=0.1,delta=0.1,n_bins=50,n_jobs=1)>>>fast_cfof_clf.compute(X)array([[0.00954095,0.00954095,0.01930698,0.05963623,0.10481131],[0.00954095,0.00954095,0.01930698,0.06866488,0.10481131],[0.00954095,0.00954095,0.02559548,0.06866488,0.10481131],...,[0.00954095,0.00954095,0.01930698,0.05963623,0.10481131],[0.00954095,0.00954095,0.03393222,0.15998587,0.24420531],[0.00954095,0.00954095,0.02559548,0.0390694,0.09102982]])Usecompute_from_distance_matrixto compute CFOF scores from a precomputed
distance matrix.>>>fromsklearn.metricsimportpairwise_distances>>>distance_matrix=pairwise_distances(X,metric='euclidean')>>>fast_cfof_clf.compute_from_distance_matrix(distance_matrix)array([[0.00954095,0.00954095,0.01930698,0.05963623,0.10481131],[0.00954095,0.00954095,0.01930698,0.06866488,0.10481131],[0.00954095,0.00954095,0.02559548,0.06866488,0.10481131],...,[0.00954095,0.00954095,0.01930698,0.05963623,0.10481131],[0.00954095,0.00954095,0.03393222,0.15998587,0.24420531],[0.00954095,0.00954095,0.02559548,0.0390694,0.09102982]])CFOFiSAXThis library provides a wrapper forpyCFOFiSAX[1]>>>fromcfof.cfof_isaximportCFOFiSAXWrapperRefer topyCFOFiSAXdocumentationfor more details.TODOsAdd support forfaiss(GPU).Parallelize FastCFOF.Add unit tests.Add benchmarks.WrappyCFOFiSAX.References[1]ANGIULLI, Fabrizio. CFOF: a concentration free measure for anomaly detection. ACM Transactions on Knowledge Discovery from Data (TKDD), 2020, vol. 14, no 1, p. 1-53.[2]FOULON, Lucas, FENET, Serge, RIGOTTI, Christophe, et al. Scoring Message Stream Anomalies in Railway Communication Systems. In : 2019 International Conference on Data Mining Workshops (ICDMW). IEEE, 2019. p. 769-776.