Package: machlearn
A Simple Yet Powerful Machine Learning Python LibraryInstallpipinstallmachlearnExample 1: k-Nearest NeighborsfrommachlearnimportkNNkNN.demo("iris")Selected Output:This demo uses a public dataset of Fisher's Iris, which has a total of 150 samples from three species of Iris ('setosa', 'versicolor', 'virginica').
The goal is to use 'the length and the width of the sepals and petals, in centimeters', to predict which species of Iris the sample belongs to.

Using a grid search and a kNN classifier, the best hyperparameters were found as following:
   Step1: scaler: StandardScaler(with_mean=True, with_std=True);
   Step2: classifier: kNN_classifier(n_neighbors=12, weights='uniform', p=2.00, metric='minkowski').Example 2: Naive Bayesfrommachlearnimportnaive_bayesasnbnb.demo(dataset="SMS_spam")Selected Output:This demo uses a public dataset of SMS spam, which has a total of 5574 messages = 4827 ham (legitimate) and 747 spam.
The goal is to use 'term frequency in message' to predict whether the message is ham (class=0) or spam (class=1).

Using a grid search and a multinomial naive bayes classifier, the best hyperparameters were found as following:
   Step1: Tokenizing text: CountVectorizer(analyzer = <_lemmas>, ngram_range = (1, 1));
   Step2: Transforming from occurrences to frequency: TfidfTransformer(use_idf = True).

The top 2 terms with highest probability of a message being a spam (the classification is either spam or ham):
   "claim": 81.28%
   "prize": 80.24%
   "won": 76.29%

Application example:
   - Message: "URGENT! We are trying to contact U. Todays draw shows that you have won a 2000 prize GUARANTEED. Call 090 5809 4507 from a landline. Claim 3030. Valid 12hrs only."
   - Probability of spam (class=1): 95.85%
   - Classification: spamExample 3: Decision Boundary Comparison (Classification with Two Features)frommachlearnimportkNNkNN.demo("Social_Network_Ads")frommachlearnimportnaive_bayesasnbnb.demo("Social_Network_Ads")frommachlearnimportSVMSVM.demo("Social_Network_Ads")frommachlearnimportdecision_treeasDTDT.demo("Social_Network_Ads",classifier_func="DT")frommachlearnimportlogistic_regressionaslog_reglog_reg.demo("Social_Network_Ads")frommachlearnimportneural_networkasNNNN.demo("Social_Network_Ads")frommachlearnimportensembleensemble.demo("Social_Network_Ads")Example 4: Imbalanced Datafrommachlearnimportimbalanced_dataimbalanced_data.demo()Summary of output:To mitigate the problem associated with class imbalance, downsampling the majority class (y=0) to match the minority case (y=1).

These are insensitive to class imbalance:
- Area Under ROC curve
- Geometric mean
- Matthew's Correlation Coefficient
- Recall, TPR
- Specificity, 1-FPR

These are sensitive to class imbalance:
- Area Under PR curve
- Accuracy
- F1 score
- PrecisionExtreme Imbalanced DataMajority Downsampled to Match Minority ClassExample 5: Regularizationfrommachlearnimportlinear_regressionaslinreglinreg.demo_regularization()Summary of output:Issues: (a) high multicollinearity and (b) too many features; these lead to overfitting and poor generalization.
- After L2 Regularization (Ridge regression), reduced variance among the coefficient estimates [more robust/stable estimates], and better R-squared and lower RMSE with the testing set [better generalization]
- After L1 Regularization (Lasso regression), coefficient estimates becoming 0 for relatively trivial features [a simpler model], and better R-squared and lower RMSE with the testing set [better generalization]Example 6: Gradient Descentfrommachlearnimportgradient_descentasGDGD.demo("Gender")Summary of output:This example uses a batch gradient descent (BGD) procedure, a cost function of logistic regression and a learning rate of 0.00025, with Male (1, 0) as the target.
- Theta estimates of [const, Height (inch), Weight (lbs)]: [0.69254314, -0.49262002, 0.19834042]
- Accuracy of prediction:  0.913Descriptive statisticsBatch Gradient Descent Training Loss vs. EpochExample 7: Decision Treefrommachlearnimportdecision_treeasDTDT.demo()DT.demo_from_scratch(question_type="regression")# dataset='boston'DT.demo_from_scratch(question_type="classification")# dataset='Social_Network_Ads', X=not scaled, criterion=entropy, max_depth=2Summary of output:- DT.demo_from_scratch(question_type="regression") uses decision_tree_regressor_from_scratch()
- DT.demo_from_scratch(question_type="classification") provides results essentially identical to the tree graph below.Example 8: Ensemble Methodsfrommachlearnimportensembleensemble.demo()ensemble.demo("Social_Network_Ads")ensemble.demo("boston")Summary of output:- These demos call the following functions developed from scratch and reflect the inner workings of them:
* random_forest_classifier_from_scratch();
* adaptive_boosting_classifier_from_scratch();
* gradient_boosting_regressor_from_scratch() (see training history plot below): R_squared = 0.753, RMSE = 4.419module: model_evaluationfunctiondescriptionplot_ROC_and_PR_curves()plots both the ROC and the precision-recall curves, along with statisticsplot_ROC_curve()plots the ROC (Receiver Operating Characteristic) curve, along with statisticsplot_PR_curve()plots the precision-recall curve, along with statisticsplot_confusion_matrix()plots the confusion matrix, along with key statistics, and returns accuracydemo_CV()provides a demo of cross validation in this moduledemo()provides a demo of the major functions in this modulemodule: datasetsfunctiondescriptionpublic_dataset()returns a public dataset as specified (e.g., iris, SMS_spam, Social_Network_Ads)module: kNNfunctiondescriptiondemo()provides a demo of selected functions in this modulemodule: naive_bayesclass/functiondescriptionnaive_bayes_Gaussian()when X are continuous variablesnaive_bayes_multinomial()when X are independent discrete variables with 3+ levels (e.g., term frequency in the document)naive_bayes_Bernoulli()when X are independent binary variables (e.g., whether a word occurs in a document or not)demo()provides a demo of selected functions in this modulemodule: SVMfunctiondescriptiondemo()provides a demo of selected functions in this modulemodule: decision_treeclass/functiondescriptiondecision_tree_regressor_from_scratch()decision tree regressor developed from scratchdecision_tree_classifier_from_scratch()decision tree classifier developed from scratchdemo_from_scratch()provides a demo of selected functions in this moduledecision_tree_regressor()decision tree regressordecision_tree_classifier()decision tree classifierdemo()provides a demo of selected functions in this modulemodule: neural_networkfunctiondescriptionmulti_layer_perceptron_classifier()multi-layer perceptron (MLP) classifierrnn()recurrent neural networkdemo()provides a demo of selected functions in this modulemodule: logistic_regressionfunctiondescriptionLogisticReg_sklearn()solutions using sklearnLogisticReg_statsmodels()solutions using statsmodelsdemo()provides a demo of selected functions in this modulemodule: linear_regressionfunctiondescriptionLasso_regression()lasso_regressionRidge_regression()ridge_regressiondemo_regularization()provides a demo of selected functions in this moduleLinear_regression_normal_equation()linear_regression_normal_equationLinear_regression()linear_regressiondemo()provides a demo of selected functions in this modulemodule: DSAfunctiondescriptiondemo()provides a demo of selected functions in this modulemodule: imbalanced_datafunctiondescriptiondemo()provides a demo of selected functions in this modulemodule: decompositionfunctiondescriptiondemo()provides a demo of selected functions in this modulemodule: gradient_descentclass/functiondescriptionlogistic_regression_BGD_classifier()logistic_regression_BGD_classifier classbatch_gradient_descent()batch_gradient_descent classdemo()provides a demo of selected functions in this modulemodule: ensembleclass/functiondescriptiongradient_boosting_regressor_from_scratch()gradient boosting regressor developed from scratchadaptive_boosting_classifier_from_scratch()adaptive boosting classifier developed from scratchrandom_forest_classifier_from_scratch()random forest classifier developed from scratchbagging_classifier_from_scratch()bagging classifier developed from scratchgradient_boosting_classifier()gradient boosting classifieradaptive_boosting_classifier()adaptive boosting classifierrandom_forest_classifier()random forest classifierbagging_classifier()bagging classifiervoting_classifier()voting classifierdemo()provides a demo of selected functions in this module