A comprehensive Guide to Scikit-learn Part 1: Overview of the Package
By: Muhammet Bektaş
Tags: Artificial Intelligence, Machine Learning, Data Science, Python, Scikit Learn

Submodules of Scikit-learn:

Now that we saw the basic interfaces of scikit-learn, we can now talk about the modules it contains. In the next articles of this series, we’ll give you hands-on examples of how you can use these modules.

These modules are organized in a way that each module serves only the functionality of its purpose. This clear design of submodules makes understanding and using the library easy and as we’ve touched upon before, the architectural design of the library is what makes it so popular among the machine learning community.

From now on, we’ll discuss the submodules and what kind of classes and functions each one provides. We’ll complete this article with this discussion. Starting from the next article, we’ll examine each module one by one using by demonstrating with code examples.

1-) Datasets : sklearn.datasets

With this module scikit-learn provides various cleaned and built-in datasets so that you can jump start playing with machine learning models right away. These datasets are among the most well-known datasets which you can easily load them with a few lines of codes. The datasets offer complete descriptions of the data itself such as iris, boston house prices, breast_cancer etc. Moreover, this module also provides a dataset fetcher that can be used to load real world datasets that are large in size. Before starting to use a toy dataset(for example boston house price dataset), we should import it like this;

from sklearn.datasets import load_boston

When using a real world datasets, a fetcher for the dataset is built into Scikit-learn. For example “mnist_784” dataset, we should use fetch_openml() function.

from sklearn.datasets import fetch_openml

mnist = fetch_openml('mnist_784', version=1, cache=True)

2-) Preprocessing : sklearn.preprocessing

Before starting to train our machine learning models and make predictions, we usually need to do some preprocessing on our raw data. Among some commonly used preprocessing tasks come OneHotEncoder, StandardScaler, MinMaxScaler, etc. These are respectively for encoding of the categorical features into a one-hot numeric array, standardization of the features and scaling each feature to a given range. Many other preprocessing methods are built-in this module.

We can import this module as follows:

from sklearn import preprocessing

3-) Impute : sklearn.impute

Missing values are common in real world datasets and can be filled easily by using the Pandas library. This module of the scikit-learn also provides some methods to fill in the missing values. For example, SimpleImputer imputes the incomplete columns using statistical values of those columns, KNNImputer uses KNN to impute the missing values. For more on the imputation methods scikit-learn provides, you can look at the documentation.

This module can be imported as shown below:

from sklearn import impute

4-) Feature Selection : sklearn.feature_selection

Feature selection is very crucial in the success of the machine learning models. Scikit-learn provides several feature selection algorithms in this module. For example, one of the feature selectors that is available in this module is RFE(Recursive Feature Elimination). It is essentially a backward selection process of the predictors. This technique starts with building a model on the whole dataset of predictors and computes a score of importance for each predictor. Then the least important predictor(s) are removed, the model is re-built, and scores of importance are again calculated.

We can import this module as the following:

from sklearn import feature_selection

5-) Linear Models : sklearn.linear_model

Linear models are the fundamental machine learning algorithms that is heavily used in supervised learning tasks. This module contains a family of linear methods such that the target value is expected to be a linear combination of the features. Among the models in this module, LinearRegression is the most common algorithm for regression tasks. Ridge, Lasso and ElasticNet are models with regularization to reduce overfitting.

The module can be imported as follows:

from sklearn import linear_model

6-) Ensemble Methods : sklearn.ensemble

Ensemble methods are advanced techniques which are often used to solve complex problems in machine learning using stacking, bagging or boosting methods. These methods allow different models to be trained on the same dataset. Each model makes its own prediction and a meta-model consists of a combination of these estimates. Among the models this module provides, it comes bagging methods like Random Forests, boosting methods like AdaBoost and Gradient Boosting and stacking methods like VotingClassifier.

We can import this module as follows:

from sklearn import ensemble

7-) Clustering : sklearn.cluster

Clustering is a very common unsupervised learning problem. This module provides several clustering algorithms like KMeans, AgglomerativeClustering, DBSCAN, MeanShift and many more.

The module can be imported as the following:

from sklearn import cluster

8-) Matrix Decomposition : sklearn.decomposition

Dimensionality reduction is something we resort to occasionally. This module of scikit-learn provides us several dimension reduction methods. Principal Components Analysis (PCA) is probably the most popular one. Other methods like SparcePCA are also available in this module.

We can import this module as follows:

from sklearn import decomposition

9-) Manifold Learning : sklearn.manifold

Manifold learning is a type of non-linear dimensionality reduction process. This module provides us many useful algorithms that are helpful in tasks like visualization of the high dimensional data. There are various manifold learning methods available in this module such as T-SNE, Isomap etc.

We can import this module as follows:

from sklearn import manifold

10-) Metrics : sklearn.metrics

Before starting to train our models and make predictions, we always consider which performance measure should best suit for our task at hand. Scikit-learn provides access to a variety of these metrics. Accuracy, precision, recall, mean squared errors are among the many metrics that are available in this module.

We can import this module as the following:

from sklearn import metrics

11-) Pipeline : sklearn.pipeline

Machine learning is an applied science and we often repeat some subtasks in a machine learning workflow such as preprocessing, training model, etc. Scikit-learn offers a pipeline utility to help automate these workflows. pipeline.Pipeline() and pipeline.make_pipeline() functions in this module can be used for creating a pipeline.

We can import this module as follows:

from sklearn import pipeline

12-) Model Selection : sklearn.model_selection

The selection process for the best machine learning models is largely an iterative process where data scientists search for the best model and the best hyper-parameters. Scikit-learn offers us many useful utilities that are helpful in both training, testing and model selection phases. In this module, there exists utilities like KFold, train_test_split(), GridSearchCV and RandomizedSearchCV. From splitting our datasets to searching for hyper-parameters, with its offerings, this module is one of the best friends of a data scientist.

We can import this module as the following:

from sklearn import model_selection

Concluding remarks

We’re done with our introduction to scikit-learn. Starting from the next article, we’ll dig deep into the details of this fascinating library. One of the mission of Bootrain is to provide accessible contents for everyone who’d like to jump into data science. So, stay tuned and please follow us in other platforms as well.

LinkedIn: https://www.linkedin.com/company/bootrain

Twitter: https://twitter.com/BootrainSchool

Web: www.bootrain.com