Package: mlpipeline-analyzer
ml-pipeline-analyzerMachine Learning Pipeline Analyzer (MLPA)Machine Learning Pipeline Analyzer (MLPA) is a python package that at its core analyzes, suggests and visualizes machine learning pipelines.One of the primary goals of this package is to provide the user with a self-intuitive visual diagram of the pipeline model that explains the various components of the model and its respective attributes while also suggesting the changes and best pipeline model for the user needs.MotivationAs a machine learning engineer or a Data Science engineer, we often create ML pipelines that perform multiple tasks like:Data extraction -> Data Cleaning -> Data Manipulation -> Feature Selection/Reduction -> Model train and predict -> Cross Validation -> Model load/saveHowever, as the various components of a pipeline increase, creating a manual flowchart is not feasible but rather hard to understand/track. And although, there are certain already existing python packages leveraging DAG to visualize these ML pipelines, yet they can be hard to explore and understand.
Therefore, our goal was to create a package that automates the daunting process of visualizing ML pipelines while also providing a capability to suggest the changes or best pipeline modes for the user inputted dataframes.AcknowledgementsMLPA is an easier and simpler wrapper using the capabilities from the following existing Python libraries:Scikit-LearnEvalMLGraphvizInstalling the packagepip install mlpipeline_analyzerDependenciesInstall the dependencies from the requirements.txt file usingpython -m pip install -r requirements.txtCode Examples:Code example_1: Here in this part, the user uploads a model .pkl file which is then passed as an input to the PipelineDiagram class. The two ML pipeline diagrams are created using .show and .show_params:evalml_pipeline = joblib.load('models/automl_pipeline.pkl')
a = PipelineDiagram(evalml_pipeline)
a.show(title='Evalml ML Pipeline Diagram')
a.show_params(title='Evalml Machine Learning Parameters Pipeline')Code example_2: Here in this part, the suggest function generates the output for the varoius components of the model depending upon what the user specifies:b = PipelineSuggest()
b.fit(data = df, response = 'survived', predictor_list = ['pclass','age','gender'], problem_type='binary', objective='auto', test_size=0.2)
b.suggest(suggest_type='fe')
b.suggest(suggest_type='model')
b.suggest(suggest_type='all')ScreenshotsExamples of outputs generated by the functionsScreenshot of a ML pipeline summary diagram:Image1Screenshot of a ML pipeline hyperparameter diagram:Image2Build StatusAlthough currently MLPA already supports feature engineering to some extent, however, with expandability as one of the project goals, we plan to add more and specific capabilities catered to the following areas:Feature EngineeringFeature Extraction/SelectionFeature ReductionFor project extension, one possible functionality could be the capability for the user to specify the engine that they want to use for their model (example: TPOT, EvalML) and run the MLPA package on top of that engine.Code StyleLanguages used: Python
Coding Style:
- PEP 8
- DocstringsFollowing sofware design principles have been considered while packaging MLPA:Modular design'Somewhat General Purpose'moduleDeep ModulesSeparation of ConcernsReusability/ExtensibilityIntuitableVersion Control using GithubException HandlingSupport for automated CI/CD using TravisUnit testing and coverage for quality assuranceAuthorsAniket FadiaJasmine BhallaSravan HandeTharun Kumar Reddy KarasaniContributeThis project is an open-source project- open to the Python user community for contribution.Change Log0.0.1 (2022-30-01)Initial release0.0.1 (2022-17-03)Added show, show_params and suggest functionalities and updated the description