Towards Automated Machine Learning: Evaluation and Comparison of AutoML Approaches and Tools There has been considerable growth and interest in industrial applications of machine learning (ML) in recent years. ML engineers, as a consequence, are in high demand across the industry, yet improving the efficiency of ML engineers remains a fundamental challenge. Automated machine learning (AutoML) has emerged as a way to save time and effort on repetitive tasks in ML pipelines, such as data preprocessing, feature engineering, model selection, hyperparameter optimization, and prediction result analysis. In this paper, we investigate the current state of AutoML tools aiming to automate these tasks. We conduct various evaluations of the tools on many datasets, in different data segments, to examine their performance, and compare their advantages and disadvantages on different test cases.

A Decision Support Framework for AutoML Systems: A MetaLearning Approach In general, the process of building a highquality machine learning model is an iterative, complex and timeconsuming process that involves exploring the performance of various machine learning algorithms in addition to having a good understanding and experience with effectively tuning their hyperparameters. In practice, conducting this process efficiently requires solid knowledge and experience with the various techniques that can be employed. With the continuous and vast increase of the amount of data in our digital world, it has been acknowledged that the number of knowledgeable data scientists can not scale to address these challenges. Thus, there is a crucial need for automating the process of Combined Algorithm Selection and Hyperparameter tuning (CASH) in the machine learning domain. Recently, several systems (e.g., AutoWeka, AutoSKLearn, SmartML) have been introduced to tackle this challenge with the main aim of reducing the role of human in the loop and filling the gap for nonexpert machine learning users by playing the role of the data scientist. MetaLearning is described as the process of learning from previous experience gained during applying various learning algorithms on different types of data, and hence reducing the needed time to learn new tasks. In the context of the Automated Machine Learning (AutoML) process, one main advantage of metalearning techniques is that they allow handengineered algorithms to be replaced with novel automated methods which are designed in a datadriven way. In this paper, we present a methodology and framework for using MetaLearning techniques to develop new methods that serve as an effective decision support for the AutoML process. In particular, we use MetaLearning techniques to answer several crucial questions for the AutoML process including: 1) Which classifiers are expected to be the best performing on a given dataset? 2) Can we predict the training time of a classifier? 3) Which classifiers are worth investing a larger portion of the time budget to improve their performance by tuning them? In our MetaLearning process, we used 200 datasets with different characteristics on a wide set of metafeatures. In addition, we used 30 classifiers from two popular machine learning libraries, namely, Weka and Scikitlearn. Our results and MetaModels have been obtained in a fully automated way. The methodology and results of our framework can be easily embededutilized by any AutoML system.