Package: lab-ml
Machine Learning LabA lightweight command line interface for the management of arbitrary machine learning tasks.Documentation is available at:https://bering-ml-lab.readthedocs.io/en/latest/NOTE: Lab is in active development - expect a bumpy ride!InstallationThe latest stable version can be installed directly from PyPi:pipinstalllab-mlDevelopment version can be installed from github.gitclonehttps://github.com/beringresearch/labcdlab
pipinstall--editable.ConceptsLab employs three concepts:reproducible environment,logging, andmodel persistence.
A typical machine learning workflow can be turned into a Lab Experiment by adding a single decorator.Creating a new Lab Projectlabinit--name[NAME]Lab will look for arequirements.txtfile in the working directory to generate a portable virtual environment for ML experiments.Setting up a Lab ExperimentHere's a simple script that trains an SVM classifier on the iris data set:fromsklearnimportsvm,datasetsfromsklearn.model_selectionimporttrain_test_splitfromsklearn.metricsimportaccuracy_score,precision_scoreC=1.0gamma=0.7iris=datasets.load_iris()X=iris.datay=iris.targetX_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.24,random_state=42)clf=svm.SVC(C,'rbf',gamma=gamma,probability=True)clf.fit(X_train,y_train)y_pred=clf.predict(X_test)accuracy=accuracy_score(y_test,y_pred)precision=precision_score(y_test,y_pred,average='macro')It's trivial to create a Lab Experiment using a simple decorator:fromsklearnimportsvm,datasetsfromsklearn.model_selectionimporttrain_test_splitfromsklearn.metricsimportaccuracy_score,precision_scorefromlab.experimentimportExperiment## New Linee=Experiment()## New Line@e.start_run## New Linedeftrain():C=1.0gamma=0.7iris=datasets.load_iris()X=iris.datay=iris.targetX_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.24,random_state=42)clf=svm.SVC(C,'rbf',gamma=gamma,probability=True)clf.fit(X_train,y_train)y_pred=clf.predict(X_test)accuracy=accuracy_score(y_test,y_pred)precision=precision_score(y_test,y_pred,average='macro')e.log_metric('accuracy_score',accuracy)## New Linee.log_metric('precision_score',precision)## New Linee.log_parameter('C',C)## New Linee.log_parameter('gamma',gamma)## New Linee.log_model('svm',clf)## New LineRunning an ExperimentLab Experiments can be run as:labrun<PATH/TO/TRAIN.py>Comparing modelsLab assumes that all Experiments associated with a Project log consistent performance metrics. We can quickly assess performance of each experiment by running:labls

ExperimentSourceDateaccuracy_scoreprecision_score
-------------------------------------------------------------------------
49ffb76etrain_mnist_mlp.py2019-01-150.97:██████████0.97:██████████
261a34e4train_mnist_cnn.py2019-01-150.98:██████████0.98:██████████Pushing models to a centralised repositoryLab experiments can be pushed to a centralised filesystem through integration withminio. Lab assumes that you have setup minio on a private cloud.Lab can be configured once to interface with a remote minio instance:labconfigminio--tagmy-minio--endpoint[URL:PORT]--accesskey[STRING]--secretkey[STRING]To push a local lab experiment to minio:labpush--tagmy-minio--bucket[BUCKETNAME].Copyright 2020, Bering Limited