Package: atm
An open source project from Data to AI Lab at MIT.ATM - Auto Tune ModelsLicense: MITDocumentation:https://HDI-Project.github.io/ATM/Homepage:https://github.com/HDI-Project/ATMOverviewAuto Tune Models (ATM) is an AutoML system designed with ease of use in mind. In short, you give
ATM a classification problem and a dataset as a CSV file, and ATM will try to build the best model
it can. ATM is based on apaperof the same name, and the project is part of theHuman-Data Interaction (HDI) Projectat MIT.InstallRequirementsATMhas been developed and tested onPython 2.7, 3.5, and 3.6Also, although it is not strictly required, the usage of avirtualenvis highly recommended in order to avoid
interfering with other software installed in the system whereATMis run.These are the minimum commands needed to create a virtualenv using python3.6 forATM:pipinstallvirtualenv
virtualenv-p$(whichpython3.6)atm-venvAfterwards, you have to execute this command to have the virtualenv activated:sourceatm-venv/bin/activateRemember about executing it every time you start a new console to work onATM!Install with pipAfter creating the virtualenv and activating it, we recommend usingpipin order to installATM:pipinstallatmThis will pull and install the latest stable release fromPyPi.Install from sourceAlternatively, with your virtualenv activated, you can clone the repository and install it from
source by runningmake installon thestablebranch:gitclonegit@github.com:HDI-Project/ATM.gitcdATM
gitcheckoutstable
makeinstallInstall for DevelopmentIf you want to contribute to the project, a few more steps are required to make the project ready
for development.First, please head tothe GitHub page of the projectand make a fork of the project under you own username by clicking on theforkbutton on the
upper right corner of the page.Afterwards, clone your fork and create a branch from master with a descriptive name that includes
the number of the issue that you are going to work on:gitclonegit@github.com:{yourusername}/ATM.gitcdATM
gitbranchissue-xx-cool-new-featuremaster
gitcheckoutissue-xx-cool-new-featureFinally, install the project with the following command, which will install some additional
dependencies for code linting and testing.makeinstall-developMake sure to use them regularly while developing by running the commandsmake lintandmake test.Data FormatATM input is always a CSV file with the following characteristics:It uses a single comma,,, as the separator.Its first row is a header that contains the names of the columns.There is a column that contains the target variable that will need to be predicted.The rest of the columns are all variables or features that will be used to predict the target column.Each row corresponds to a single, complete, training sample.Here are the first 5 rows of a valid CSV with 4 features and one target column calledclassas an example:feature_01,feature_02,feature_03,feature_04,class
5.1,3.5,1.4,0.2,Iris-setosa
4.9,3.0,1.4,0.2,Iris-setosa
4.7,3.2,1.3,0.2,Iris-setosa
4.6,3.1,1.5,0.2,Iris-setosaThis CSV can be passed to ATM as local filesystem path but also as a complete AWS S3 Bucket and
path specification or as a URL.You can find a collection of demo datasets in theatm-data S3 Bucket in AWS.QuickstartIn this short tutorial we will guide you through a series of steps that will help you getting
started withATMby exploring its Python API.1. Get the demo dataThe first step in order to runATMis to obtain the demo datasets that will be used in during
the rest of the tutorial.For this demo we will be using the pollution csv from the atm-data bucket, which you can download with your browserfrom here, or using the following command:atmdownload_demopollution_1.csv2. Create an ATM instanceThe first thing to do after obtaining the demo dataset is creating an ATM instance.fromatmimportATMatm=ATM()By default, if the ATM instance is without any arguments, it will create an SQLite database
calledatm.dbin your current working directory.If you want to connect to a SQL database instead, or change the location of your SQLite database,
please check theAPI Referencefor the complete list of available options.3. Search for the best modelOnce you have theATMinstance ready, you can use the methodatm.runto start
searching for the model that better predicts the target column of your CSV file.This function has to be given the path to your CSV file, which can be a local filesystem path, an URL to
and HTTP or S3 resource.For example, if we have previously downloaded thepollution_1.csvfile inside our current working directory, we can callrunlike this:results=atm.run(train_path='pollution_1.csv')Alternatively, we can use the HTTPS URL of the file to have ATM download the CSV for us:results=atm.run(train_path='https://atm-data.s3.amazonaws.com/pollution_1.csv')As the last option, if we have the file inside an S3 Bucket, we can download it by passing an URI
in thes3://{bucket}/{key}format:results=atm.run(train_path='s3://atm-data/pollution_1.csv')In order to make this work with a Private S3 Bucket, please make sure to having configured yourAWS credentials file,
or to having created yourATMinstance passing it theaccess_keyandsecret_keyarguments.Thisruncall will start what is called aDatarun, and a progress bar will be displayed
while the different models are tested and tuned.Processingdatasetdemos/pollution_1.csv100%|##########################| 100/100 [00:10<00:00,  6.09it/s]Once this process has ended, a message will print that theDatarunhas ended. Then we can
explore theresultsobject.4. Explore the resultsOnce the Datarun has finished, we can explore theresultsobject in several ways:a. Get a summary of the DatarunThedescribemethod will return us a summary of the Datarun execution:results.describe()This will print a short description of this Datarun similar to this:Datarun1summary:Dataset:'demos/pollution_1.csv'ColumnName:'class'JudgmentMetric:'f1'ClassifiersTested:100ElapsedTime:0:00:07.638668b. Get a summary of the best classifierTheget_best_classifiermethod will print information about the best classifier that was found
during this Datarun, including the method used and the best hyperparameters found:results.get_best_classifier()The output will be similar to this:Classifierid:94Classifiertype:knnParamschosen:n_neighbors:13leaf_size:38weights:uniformalgorithm:kd_treemetric:manhattan_scale:TrueCrossValidationScore:0.858+-0.096TestScore:0.714c. Explore the scoresTheget_scoresmethod will return apandas.DataFramewith information about all the
classifiers tested during the Datarun, including their cross validation scores and
the location of their pickled models.scores=results.get_scores()The contents of the scores dataframe should be similar to these:cv_judgment_metriccv_judgment_metric_stdevidtest_judgment_metricrank00.85841269840.0960095737940.71428571431.010.82222222220.0623609564120.62500000002.020.81476190480.1117618135640.87500000003.030.81393939390.0588721670680.60869565224.040.80677544680.0875180564500.62500000005.0...5. Make predictionsOnce we have found and explored the best classifier, we will want to make predictions with it.In order to do this, we need to follow several steps:a. Export the best classifierTheexport_best_classifiermethod can be used to serialize and save the best classifier model
using pickle in the desired location:results.export_best_classifier('path/to/model.pkl')If the classifier has been saved correctly, a message will be printed indicating so:Classifier94savedaspath/to/model.pklIf the path that you provide already exists, you can ovewrite it by adding the argumentforce=True.b. Load the exported modelOnce it is exported you can load it back by calling theloadmethod from theatm.Modelclass and passing it the path where the model has been saved:fromatmimportModelmodel=Model.load('path/to/model.pkl')Once you have loaded your model, you can pass new data to itspredictmethod to make
predictions:importpandasaspddata=pd.read_csv(demo_datasets['pollution'])predictions=model.predict(data.head())What's next?For more details aboutATMand all its possibilities and features, please check thedocumentation site.There you can learn more about itsCommand Line Interfaceand itsREST API, as well ashow to contribute to ATMin order to help us developing new features or cool ideas.CreditsATM is an open source project from the Data to AI Lab at MIT which has been built and maintained
over the years by the following team:Bennett Cyphersbcyphers@mit.eduThomas Swearingenswearin3@msu.eduCarles Salacsala@csail.mit.eduPlamen Valentinovplamen@pythiac.comKalyan Veeramachanenikalyan@mit.eduMicah Smithmicahjsmith@gmail.comLaura Gustafsonlgustaf@mit.eduKiran Karrakiran.karra@gmail.comMax Kanterkmax12@gmail.comAlfredo Cuesta-Infantealfredo.cuesta@urjc.esFavio André Vázquezfavio.vazquezp@gmail.comMatteo Hochminime@hochweb.comCiting ATMIf you use ATM, please consider citing the following paper:Thomas Swearingen, Will Drevo, Bennett Cyphers, Alfredo Cuesta-Infante, Arun Ross, Kalyan Veeramachaneni.ATM: A distributed, collaborative, scalable system for automated machine learning.IEEE BigData 2017, 151-162BibTeX entry:@inproceedings{DBLP:conf/bigdataconf/SwearingenDCCRV17,author={Thomas Swearingen andWill Drevo andBennett Cyphers andAlfredo Cuesta{-}Infante andArun Ross andKalyan Veeramachaneni},title={{ATM:} {A} distributed, collaborative, scalable system for automatedmachine learning},booktitle={2017 {IEEE} International Conference on Big Data, BigData 2017, Boston,MA, USA, December 11-14, 2017},pages={151--162},year={2017},crossref={DBLP:conf/bigdataconf/2017},url={https://doi.org/10.1109/BigData.2017.8257923},doi={10.1109/BigData.2017.8257923},timestamp={Tue, 23 Jan 2018 12:40:42 +0100},biburl={https://dblp.org/rec/bib/conf/bigdataconf/SwearingenDCCRV17},bibsource={dblp computer science bibliography, https://dblp.org}}Related ProjectsBTBBTB, for Bayesian Tuning and Bandits, is the core AutoML
library in development under the HDI project. BTB exposes several methods for hyperparameter
selection and tuning through a common API. It allows domain experts to extend existing methods
and add new ones easily. BTB is a central part of ATM, and the two projects were developed in
tandem, but it is designed to be implementation-agnostic and should be useful for a wide range
of hyperparameter selection tasks.FeaturetoolsFeaturetoolsis a python library for automated
feature engineering. It can be used to prepare raw transactional and relational datasets for ATM.
It is created and maintained byFeature Labsand is also a part
of theHuman Data Interaction Project.History0.2.2 (2019-07-30)New FeaturesCurate dependencies -Issue #152by @csalaPOST request blocked by CORS policy -Issue #151by @pvk-developer0.2.1 (2019-06-24)New FeaturesRest API Cross-origin resource sharing (CORS) -Issue #146by @pvk-developer0.2.0 (2019-05-29)New Python APINew FeaturesNew API for ATM usage within Python -Issue #142by
@pvk-developer and @csalaImproved Documentation -Issue #142by
@pvk-developer and @csalaCode cleanup -Issue #102by
@csalaEnsure datasets can be downloaded from S3 -Issue #137by @pvk-developerChange to PyMySQL to remove libmysqlclient-dev system dependency -Issue #136by @pvk-developer and @csala0.1.2 (2019-05-07)REST API and Cluster Management.New FeaturesREST API Server - Issues#82and#132by @RogerTangos, @pvk-developer and @csalaAdd Cluster Management commands to start and stop the server and multiple workers
as background processes -Issue #130by
@pvk-developer and @csalaAdd TravisCI and migrate docs to GitHub Pages -Issue #129by @pvk-developer0.1.1 (2019-04-02)First Release on PyPi.New FeaturesUpgrade to latest BTB.New Command Line Interface.0.1.0 (2018-05-04)First Release.