Package: mlpath
MLPathMLPath is an MLOPs library on Python that makes tracking machine learning experiments and organizing machine learning projects easier. It consists of two subpackages so far, MLQuest for tracking and MLDir for directory structure.Check this fordocumentation. It also has examples and notes that will help you further understand the library.Installationpip install mlpathMLQuestTo get startedThis is your code without mlquest# Preprocessingx_data_p=Preprocessing(x_data=[1,2,3],alpha=1024,beta_param=7,c=12)# Feature Extractionx_data_f=FeatureExtraction(x_data_p,14,510,4)# Model Initializationmodel=RadialBasisNet(x_data_f,12,2,3)# Model Trainingaccuracy=train_model(model)This is your code with mlquestfrommlpathimportmlquestasmlql=mlq.l# Start a new quest, this corresponds to a table where every run of the Python file will be logged.mlq.start_quest('Radial Basis Pipeline',log_defs=False)# Preprocessingx_data_p=l(Preprocessing)(x_data=[1,2,3],alpha=1114,beta_param=2,c=925)# Feature Extractionx_data_f=l(FeatureExtraction)(x_data_p,32,50,4)# x_data_p is an array so it won't be logged.# Model Initializationmodel=l(RadialBasisNet)(x_data_f,99,19,31)# Model Trainingaccuracy=train_model(model)# log the accuracymlq.log_metrics(accuracy)# can also do mlq.log_metric(acc=accuracy) so its logged as accmlq.end_quest()mlq.show_table('Radial Basis Pipeline',last_k=10)# show the table for the last 10 runsAfter the third run, this shows up under the Quests folder (and the notebook itself):infoPreprocessingFeatureExtractionRadialBasisNetmetricstimedatedurationidalphabeta_paramcx_paramy_paramz_paramp_numk_numl_numaccuracy16:31:1602/11/231.01 min1741295135304999935016:32:4002/11/234.91 ms2142951325304991937016:32:5702/11/234.93 ms3111429253250499193170Editors like VSCode support viewing markdown out-of-the-box. You may need to pressCTRL/CMD+Shift+V⦿ Besides of the markdown, you will also find ajsonfolder in that directory with a config file that allows you to customize the columns to show in the markdown table.But you probably prefer a web interfaceIn the same level as theQuestsfolder run the commandmlwebthen openhttp://localhost:5000in your browser. You should see something like this:⦿ You can search for specific runs, an example would bemetrics.accuracy>50(similar syntax to MLFlow)⦿ You can customize the columns to show in the table by clicking oncolumns(in lieu of doing it throughjsonconfig file)⦿ Choose which model (folder containing python files where you run quests) and which pipeline file (quest) using the bar on the leftAn example with Scikit-Learnfrommlpathimportmlquestasmlq# We won't log defaults here but note that being aware of them and their values/impact is important.mlq.start_quest('Fractal-GB',log_defs=False,table_dest='../../')# read the datax_train_i,x_val_i,y_train_i,y_val_i=read_data()# preprocess the datax_train_p,x_val_p=preprocess_data(x_train_i,x_val_i)# extract fractal featuresx_train_f,x_val_f=mlq.l(apply_SFTA)(x_train_p,x_val_p,deviation=10)# initialize a GB modelmodel=mlq.l(GradientBoostingClassifier)(n_estimators=10,learning_rate=220,max_depth=110)# train the modelmodel.fit(x_train_f,y_train_i)# report the accuracyaccuracy=model.score(x_val_f,y_val_i).item()# .item() so its a scalar that can be loggedmlq.log_metrics(acc=accuracy)mlq.end_quest()mlq.show_table('Fractal-GB',last_k=10)# show the table for the last 10 runsinfoapply_SFTAGradientBoostingClassifiermetricstimedatedurationiddeviationn_estimatorslearning_ratemax_depthacc17:26:3402/11/232.33 min1301050120.517:29:0802/11/23344.98 ms2105020100.517:29:1402/11/23251.52 ms3105020100.517:29:2202/11/23266.31 ms410102201100.5An example with PyTorchA real example on a dummy dataset that demonstrates using the library on real models is provided in the MLDir examples mentioned below.MLDirMLDir is a simple CLI that creates a standard directory structure for your machine learning project. It provides a folder structure that is comprehensive, highly scalable (development-wise) and apt for collaboration.Note of caution⦿ Although it integrates well with MLQuest, neither MLQuest nor MLDir require the other to function.⦿ Suppose your project has very few people working on it (only you) or does not require trying many models with many other preprocessing methods and features, then you may not really need MLDir. A notebook and MLQuest should be enough. Otherwise use MLDir to prevent your directory from becoming a spaghetti soup of Python files.📜 The MLDir ManifestoThe directory structure generated by MLDir complies with the MLDir manifesto ( a set of 'soft' standards) which attempts to enforce seperation of concerns among different stages of the machine learning pipeline and among writing code and running experiments (hyperparameter tuning). It specifies:➜ Each stage in the ML pipeline should be a separate directory.➜ If that stage is further broken down into sub-stages, each sub-stage should be a separate directory (in the top level)➜ For any pipeline stage, each alternative that implements that stage should be in a separate directory within that stage's folder.➜ Any implementation of a stage is a set of functions.➜ Functions are defined only in .py files not in notebooks.➜ Notebooks are only for testing or running entire pipelines (e.g., training and hyperparameter tuning). They import the needed functions from pipeline stages.You can also read more about the manifestohere.To get startedMLDir is part of MLPath. So you don't need to install it separately. To create a simple folder structure, run:mldir--name<project_name>⦿ If mldir is ran without a name, it uses the name 'Project'This generates the following folder structure:.
├── DataFiles
│   ├── Dataset 1
│   └── Dataset 2
├── DataPreparation
│   ├── Ingestion
│   └── Preprocessing
├── FeatureExtraction
│   ├── BoW
│   ├── GLCM
│   └── OneHot
├── ModelPipelines
│   ├── GRU
│   ├── GradientBoost
│   └── SVM
├── ModelScoring
│   ├── Pipeline
│   └── Scoring
├── README.md
├── Sandbox.ipynb
│   ├── DataPreparation
│   ├── FeatureExtraction
│   └── ModelsPipelines
└── Sandbox.ipynbThe file in each folder has instructions on how to use it. These are all grouped in the README for a more detailed explanation.Other important optionsmldir--name<project-name>--full⦿ The --full option generates an even more comprehensive folder structure. Including folders such asModelImplementations,Referencesand most importantlyProduction.⦿ TheProductionfolder contains a Flask app that can be used to serve your model as an API. All you need is only to import your final model into app.py and replace the dummy model with it. The Flask app assumes that your model takes a file via path and returns a prediction but it can be easilt extended otherwise to suit your needsA note on the Flask app⦿ Even if you have never used Flask (and need to do more than just plug in your model), notice that the app is composed of a templates folder that stores the HTML of the pages and a static folder that stores CSS/JS and other assets. Lastly, the app.py file contains the code that runs the app by rendering the right HTML page and passing the right parameters to it according to your request (visiting a URL, submitting a file,...)A complete examplemldir--name<project-name>--example⦿ The --example option generates a complete example on a dummy dataset (but real models) that should be helpful for you to understand more about the folder structure and how to use it (or you can use it as a template for your own project). The example also usesCreditsThanks toAbdullahfor all his startling work on the mlweb module and for all the time he spent with me to discuss or test the library.Thanks toJimmyfor all his help in testing the library.CollaboratorsEssam WisamAbdullah Adel