Package: mixed-naive-bayes
Mixed Naive BayesNaive Bayes classifiers are a set of supervised learning algorithms based on applying Bayes' theorem, but with strong independence assumptions between the features given the value of the class variable (hence naive).This module implementscategorical(multinoulli) andGaussiannaive Bayes algorithms (hencemixed naive Bayes). This means that we are not confined to the assumption that features (given their respectivey's) follow the Gaussian distribution, but also the categorical distribution. Hence it is natural that the continuous data be attributed to the Gaussian and the categorical data (nominal or ordinal) be attributed the the categorical distribution.The motivation for writing this library is thatscikit-learnat the point of writing this (Sep 2019) did not have an implementation for mixed type of naive Bayes.They have one forCategoricalNBherebut it's still in its infancy.scikit-learn now hasCategoricalNB!I like scikit-learn's APIs  😍 so if you use it a lot, you'll find that it's easy to get started started with this library. There'sfit(),predict(),predict_proba()andscore().I've also written a tutorialherefor naive bayes if you need to understand a bit more on the math.ContentsInstallationQuick startsRequirementsPerformance (Accuracy)Performance (Speed)TestsAPI DocumentationTo-DosReferencesRelated workContributing ❤️Installationvia pippipinstallmixed-naive-bayesorpipinstallgit+https://github.com/remykarem/mixed-naive-bayes#egg=mixed-naive-bayesQuick startsExample 1: Discrete and continuous dataBelow is an example of a dataset with discrete (first 2 columns) and continuous data (last 2). We assume that the discrete features follow a categorical distribution and the features with the continuous data follow a Gaussian distribution. Specifycategorical_features=[0,1]then fit and predict as per usual.frommixed_naive_bayesimportMixedNBX=[[0,0,180.9,75.0],[1,1,165.2,61.5],[2,1,166.3,60.3],[1,1,173.0,68.2],[0,2,178.4,71.0]]y=[0,0,1,1,0]clf=MixedNB(categorical_features=[0,1])clf.fit(X,y)clf.predict(X)NOTE: The module expects that the categorical data be label-encoded accordingly. See the following example to see how.Example 2: Discrete and continuous dataBelow is a similar dataset. However, for this dataset we assume a categorical distribution on the first 3 features, and a Gaussian distribution on the last feature. Feature 3 however has not been label-encoded. We can use sklearn'sLabelEncoder()preprocessing module to fix this.importnumpyasnpfromsklearn.preprocessingimportLabelEncoderX=[[0,0,180,75.0],[1,1,165,61.5],[2,1,166,60.3],[1,1,173,68.2],[0,2,178,71.0]]y=[0,0,1,1,0]X=np.array(X)y=np.array(y)label_encoder=LabelEncoder()X[:,2]=label_encoder.fit_transform(X[:,2])print(X)# array([[ 0,  0,  4, 75],#        [ 1,  1,  0, 61],#        [ 2,  1,  1, 60],#        [ 1,  1,  2, 68],#        [ 0,  2,  3, 71]])Then fit and predict as usual, specifyingcategorical_features=[0,1,2]as the indices that we assume categorical distribution.frommixed_naive_bayesimportMixedNBclf=MixedNB(categorical_features=[0,1,2])clf.fit(X,y)clf.predict(X)Example 3: Discrete data onlyIf all columns are to be treated as discrete, specifycategorical_features='all'.frommixed_naive_bayesimportMixedNBX=[[0,0],[1,1],[1,0],[0,1],[1,1]]y=[0,0,1,0,1]clf=MixedNB(categorical_features='all')clf.fit(X,y)clf.predict(X)NOTE: The module expects that the categorical data be label-encoded accordingly. See the previous example to see how.Example 4: Continuous data onlyIf all features are assumed to follow Gaussian distribution, then leave the constructor blank.frommixed_naive_bayesimportMixedNBX=[[0,0],[1,1],[1,0],[0,1],[1,1]]y=[0,0,1,0,1]clf=MixedNB()clf.fit(X,y)clf.predict(X)More examplesSee theexamples/folder for more example notebooks or jump into a notebook hosted at MyBinderhere. Jupyter notebooks are generated usingp2j.RequirementsPython>=3.6numpy>=1.16.1scikit-learn>=0.20.2Thescikit-learnlibrary is used to only import data as seen in the examples. Otherwise, the module itself does not require it.Thepytestlibrary is not needed unless you want to perform testing.Performance (Accuracy)Performance across sklearn's datasets on classification tasks. Runpython benchmarks.py.DatasetGaussianNBMixedNB (G)MixedNB (C)MixedNB (C+G)Iris plants0.9600.960--Handwritten digits0.8580.8580.961-Wine0.9890.989--Breast cancer0.9420.942--Forest covertypes0.6160.616-0.657GaussianNB - sklearn's API for Gaussian Naive BayesMixedNB (G) - our API for Gaussian Naive BayesMixedNB (C) - our API for Categorical Naive BayesMixedNB (C+G) - our API for Naive Bayes where some features follow categorical distribution, and some features follow GaussianPerformance (Speed)The library is written inNumPy, so many operations are vectorised and faster than their for-loop counterparts. Fun fact: my first prototype (with many for-loops) took me 8 times slower than sklearn's 😱.(Still measuring)TestsI'm still writing more test cases, but in the meantime, you can run the following:pytestCorrectnessAccuracy against existing library (sklearn)Input type checkingExample inputsAPI DocumentationFor more information on usage of the API, visithere. This was generated using pdoc3.To-DosPerformance comparisonChange to F-contiguous arrays?Implementpredict_log_proba()Write more test casesSupport refittingRegulariser for categorical distributionVariance smoothing for Gaussian distributionVectorised main operations using NumPyPossible features:Masking in NumPySupport label encodingReferencesscikit-learn's naive bayesRelated WorkCategorical naive Bayes by scikit-learnNaive Bayes classifier for categorical and numerical dataGeneralised naive Bayes classifierContributingPlease submit your pull requests, will appreciate it a lot ❤