5 Scikit-Learn Must-Know Hidden Gems
By: Andre Ye
Tags: Machine Learning, Towards Data Science, Data Science, AI, Data Analysis

Dataset Generators

Scikit-learn has plenty of dataset generators, which can be used to create artificial datasets with varying complexities and shapes.

For example, the make_blobs function creates “blobs”, or clusters of data, with any amount of samples, centers/clusters, and features/dimensions.

The values of the X and the y are:

When X is graphed and colored according to the labels y , the shape of the data can be visualized:

Scikit-learn has many other dataset creation functions:

make_moons(n_samples=100, noise=0.1)

make_circles(n_samples=100, noise=0.05)

make_regression(n_samples=100, n_features=1, noise=15)

make_classification(n_samples=100)

Pipelines

Pipelines allows for various methods to be combined into one singular model. This is especially the case in natural language processing (NLP) applications which require vectorizers or data that needs to be standardized or normalized. A pipeline can be created by combining several models together, in which data flows sequentially through the aggregate model. It has standard fitting and predicting capabilities, making the training process much more organized.

Various objects can fit into a pipeline:

Imputers. Do you have missing data? Try a Simple Imputer or a KNN Imputer.

Encoders. If your data is non-binary categorical, you may need to use a Label Encoder or a One-Hot Encoder.

NLP Vectorizers. If you are dealing with NLP data, use Count Vectorizers, TD-IDF Vectorizers, or Hash Vectorizers.

Numerical Transformations. Try standardizers, normalizers, and min-max scalers.

Grid-Search

A common task in machine learning is finding the right set of parameters in a model. Usually, one can either guess based on their knowledge of the task and the model or programmatically find the best set. sklearn has a built-in function — GridSearchCV — that automatically finds the best set of parameters for you to optimize model performance.

The GridSearchCV object takes in two parameters: firstly, the model object to be trained (in this case a Support Vector Machine classifier), and secondly, a dictionary describing the parameters of the model. Each key in the dictionary is one parameter from the model, where each value is a corresponding list or tuple of values in which the parameter may take.

After the grid search object is fitted, the best_params_ attribute can be used to output the best-performing parameter values for each of the model’s parameters. Other model parameters include tree depth in decision trees and number of voters in a random forest ensemble.

Validation Curves

To visualize a parameter’s effect on the model’s performance, use sklearn ’s validation_curve . It takes in a few parameters — the model, the parameter to be adjusted, a range of values for the parameter, and the number of folds. It is similar to a Grid-Search for one variable, and can help better visualize the results of a parameter shift.

The output of the validation_curve object is a tuple — one for the scores during training, and another for the testing scores. The number of rows in each represents the value of arrays for each of the parameter values, whereas each element in that array represents the value for each of the k folds.

When the results are plotted, the relationship between the parameter and the accuracy is clear.

This allows us to visualize the impact of tree depths on the accuracy. For instance. note that a tree depth of 5 or at tree depth of 6 performs reasonably well. Specifying a tree depth at anything further would cause overfitting, but one would need to evaluate this on testing accuracy to be sure.

K-Fold Cross Validation

Cross-validation is a method that gives much more accurate results than standard train_test_split methods (and actually requires less code!). With traditional train-test-splits, the data is randomly split into a training set and a testing set (usually a 7:3–8:2 ratio), the model being trained on the training set and evaluated on the testing set to truly measure the model’s ability to measure and not just to memorize. However, since each split is random, splitting the data ten times will yield ten different accuracies.

To address this issue, cross validation using k folds splits the data into k categories, training a model on k-1 folds and testing on the remaining 1 fold. After repeating this process, where each testing fold eventually covers the entire dataset, one arrives at a more complete and honest view of the accuracy. What’s better, there is no need to keep track of x-train, x-test, y-train, and y-test variables. The only downside to cross-validation is that it takes more time — but better results always have higher costs.