Package: pyDSlib
Python Data Science Library (pyDSlib)Author:John T. LeonardRepo:pyDSlibCustom modules/classes/methods for various data science, computer vision, and machine learning operations in pythonInstalling & ImportingIn your command line interface (CLI):$ pip install --upgrade pyDSlibAfter this, the package can be imported into jupyter notebook or python in general via the comman:import pyDSlibModules:pyDSlib.ML
pyDSlib.plot
pyDSlib.img
pyDSlib.video
pyDSlib.file_utils
pyDSlib.summary_tables
pyDSlib.kaggleModules OverviewBelow, we highlight several of the most interesting modules in more detail.pyDSlib.MLMachine learning module for python focusing on streamlining and wrapping sklearn, xgboost, dask_ml, & tensorflow/keras functionspyDSlib.ML Sub-Modules:pyDSlib.ML.preprocessing 
pyDSlib.ML.model_selection
pyDSlib.ML.NeuralNet
pyDSlib.ML.inspection
pyDSlib.ML.postprocessingThe sub-modules within pyDSlib.ML are summarized below:pyDSlib.ML.preprocessingFunctions related to preprocessing/feature engineering for machine learningThe main class of interest is thepyDSlib.ML.preprocessing.Preprocessing_pipeclass, which iterates through a standard preprocessing sequence and saves the resulting engineered data. The standard sequence is:LabelEncode.categorical_featuresScale.continuous_featuresfor Scaler_ID in Scalers_dict.keys()Impute.categorical_featuresfor Imputer_cat_ID in Imputer_categorical_dict[Imputer_cat_ID].keys():*for Imputer_iter_class_ID in Imputer_categorical_dict[Imputer_cat_ID].keys():Imputer.continuous_featuresfor Imputer_cont_ID in Imputer_continuous_dict.keys():for Imputer_iter_reg_ID in Imputer_continuous_dict[Imputer_cont_ID].keys():OneHotEncodeCorrCoeffThreshold
Finished!pyDSlib.ML.model_selectionFunctions/classes for running hyperparameter searches across multiple types of models & comparing those modelsThe main classes of interest are thepyDSlib.ML.model_selection.GridSearchCVclass and thepyDSlib.ML.model_selection.BayesianSearchCVclass, which run hyperparameter GridSearchCV and BayesianSearchCV optimizations across different types of models & compares the results to allow one to find the best-of-best (BoB) model. The.fitfunctions for both these classes are compatible with evaluating sklearn models, tensorflow/keras models, and xgboost models. Check out the doc-strings for each class for additional notes on implementation.pyDSlib.ML.NeuralNetsub-modules/functions/classes for streamlining common neural-net architectures implemented in tensorflow/keras.The most notetable sub-modules are theDenseNetandConv2Dmodules, which provide a keras implementation of a general dense neural network & 2D convolutional neural network, where the depth & general architecture of the network s are defined by generic hyperparameters, such that one can easily perform a grid search across multiple neural network architectures.pyDSlib.ML.inspectionFunctions to inspect features and/or models after trainingpyDSlib.ML.postprocessingML model outputs postprocessing helper functionspyDSlib.plotThis module contains helper functions related to common plotting operations via matplotlib.The most noteable functions are:pyDSlib.plot.corr_matrix(): Plot a correlation matrix chartpyDSlib.plot.ccorr_pareto(): Plot a pareto bar-chart for 1 label of interest within a correlation dataframepyDSlib.plot.hist_or_bar(): Iterate through each column in a dataframe and plot the histogram or bar chart for the data.pyDSlib.imgThis module contains functions/classes related to image analysis, most of which wrap SciKit image functions in some way.The most noteable functions are:pyDSlib.img.auto_crop.use_edges(): Use skimage.feature.canny method to find edges in the image passed and autocrop on the outermost edgespyDSlib.img.decompose_video_to_img(): Use cv2 to pull out image frames from a video and save them as png filespyDSlib.kaggleThis module contains functions for interacting with kaggle. The simplest and most useful function is:pyDSlib.kaggle.competition_download_files(competition)wherecompetitionis the competition name, such as  "home-credit-default-risk"pyDSlib.file_utilsThis module contains simple but extremely useful helper functions to save and load standard file types including 'hdf', 'csv', 'json', 'dill'. Essentially thesaveandloadfunctions take care of the boiler plate operations related to saving or loading on the file-types specified above.Example NotebooksBasic notebook examples can be found in the (notebooks)[notebooks] folder. Some examples include:example_ML_NeuralNet_Bert_Word2Vecexample_ML_model_selection_BayesianSearchCVexample_Conv2D_AutoEncoderexamples_RCNNThis folder contains various examples related to region-based Conv. Nets., which are typically used for object detections tasks[example-RCNN-mask-pretrained-coco](example-RCNN-mask-pretrained-coco.ipynb