Automated Model Selection With RapidMiner
By: Yulia Lukashina
Tags: Rapidminer, Data Science, Scikit Learn, Machine Learning

© Yulia Lukashina

Automated Model Selection With RapidMiner

Among other things, the scikit-learn is used to teach algorithms in selecting the best model. RapidMiner enables automated model selection, too.

My First Encounter With RapidMiner

A few years ago, I had a short career stop in a small AI startup. That job had brought me on a new level. After almost four years, I still keep spreading the word about the tools and skills I had learned there.

Most of our algorithms were programmed in R, but we used other data science tools as well. One of them was RapidMiner: previously, a quite expensive one. It is a niche software that offers a drag-and-drop interface alternative for pretty much everything that Python can, and a data scientist may need. Although I always advocate coding over drag-and-dropping, RapidMiner had won my sympathy.

I can think of quite a few other tools that have the same features and are made for either beginners or intermediate level professionals. That’s why I won’t be going into details on such basic features, like data importing, extracting, and cleaning. I previously pointed out that it’s a niche tool. And the niche is machine learning. RapidMiner can go as far as to replace the scikit-learn package, the most popular one for training algorithms, in the toolkits of many data scientists.

The Anatomy of RapidMiner

In a nutshell, RapidMiner is a reincarnation of a flow chart representation of an algorithm. The main difference is that RapidMiner has multiple levels: processes containing subprocess or operators.

Processes are either data pipelines or algorithms with nested processes that can run in parallel. The subprocesses are like brunches of a tree that meet and merge into the tree stump again.

A process in RapidMiner: A few operators connected through their inputs and outputs.

And then there are operators: the smallest units that perform one action. A definition of action is rather broad, though. It can be a whole model, like a decision tree, or it can be one function, one filter, or even one command, like read or write.

Operators pane with the search field

You start a process by adding your first operator. When dragging an operator into the existing process, you need to connect it to the output of the previous operator.

Start: Build and Validate a Model

Import and Polish

I used this nice mall customers open-source dataset from Kaggle to refresh my knowledge of the tool and create screenshots for this article.

Raw data

The data contained demographics of the shopping mall customers and their spending scores: assessments of their buying behavior. The latter I intended to get predicted by the model.

I inserted the data from a .CSV file (the “Read CSV” operator). I connected the output of it directly to the process output to make RapidMiner display descriptive statistics:

Next, I replaced the original score with a new binominal one that had only two values: “high” and “low.” It was necessary to keep this tutorial slim since a classification model would not work on the numeric values without some additional data preparations.

Apply a Model And Do Cross Validation

Then, in the “Set Role” operator, I indicated that I needed the new spending score as “label,” which meant that it was going to be the result of the prediction. I could have directly added the model. Instead, I put the “Cross Validation” operator first.

First steps

As you know, cross validation makes sure that the model remains efficient if applied to a new dataset and not only with the one it was trained on (or developed from).

The cross validation operator iterates over the data multiple times and breaks the same dataset into the training and testing subsets that vary with each iteration. Therefore, the model receives a lot of training until it achieves the best performance parameters.

Inside the cross validation operator, I added the Decision Tree model to train it on the dataset.

Inside the Cross Validation operator

Now the process can run. It produces both a decision tree and a performance table (with awful results, though).

Decision tree in RapidMiner

Performance Vector of the decision tree model

The model is not quite reliable. In such a situation, you want to test several models and maybe train them on several datasets. And you want to make the choice of the best model flexible and self-learning.

Automated Model Selection

A Second Model

There is no universal model that fits with each dataset. Dataset selects the right model, not the other way around.

I added the random forest model. I used the “Multiply” operator to run two models in parallel threads to get the results of both:

Decision tree cross validation on the top right and random forest cross validation on the bottom right

I would need to compare them manually each time I run them. Besides, with each new data, the models would perform differently. And I have only started to go through all available classification models (rule induction, random tree, etc.)

Automation

To automate the selection, all you need is to wrap your cross validation boxes into the “Select subprocess” operator. Just drag-and-drop the new operator, then cut the cross validations with the right-click, and paste them inside the select-subprocess operator. Delete the “Multiply” operator and rename the cross validation boxes to differentiate them. I called them “CV DT” and “CV RF.”

Two models and their cross validations inside the “Select subprocess” operator

The umbrella process will now produce only one model — the most accurate one — as an output.

The highest level of your process

Bonus: Optimization

Additionally, you can optimize both your models and the selection of the best model. To do this, wrap each of your cross validations and the “Select subprocess” operator into an “Optimize parameters” operator. Note: you need three “Optimize parameters” operators!

You can manually select the parameters you want to optimize. For instance, a criterion and a minimal gain for the decision tree model, or number of trees for the random forest. RapidMiner will iterate over these parameters until it finds a perfect match. Or until it reaches the limit on iterations you have imposed.

You are set.

A Short Afterward

In some situations, you want to have your ML processes stored in a tool that does not require an advanced programming experience. For instance, if you want that a colleague can maintain the process when you are on vacations. Or when you want to make it accessible for a bigger audience in the company.

Before I started to work with RapidMiner, I had a long programming gap, and it took me an eternity to finish a line when I had to. But I did understand the logic behind the classifiers. RapidMiner allowed me to do my new data science job without being able to program fluently.

I believe that RapidMiner can be good for people who have learned statistics (like a lot of medicine or economics students) but never had to do much coding before. In a time when ML gradually becomes a routine for many companies, it seems quite natural that more people should be able to work with ML algorithms.