4 easy steps to improve your machine learning code performance | by Maksym Balatsko | Towards Data Science

It is really astonishing, how many new machine learning libraries and models are created every day. But it is such a disappointment that most of them are just not properly designed. Actually sometimes great thing with great ideas behind it is just not being used, because it is not well documented, has difficult or not intuitive API or just because the thing doesn’t perform well and is too slow for the production environment. A great example of this would the library that I mentioned in my previous post — normalize. Such a beautiful idea behind it and it produces the results that really can make the difference in some nlp tasks, but there are some unexpected bugs in it and API is not flexible enough. OK, you say, these two things are not so crucial and actually, you can survive it. But the thing you cannot survive is that its performance is awful, it is just SLOW and so can’t be used in a production environment.
I want to encourage everyone, who creates machine learning libraries or just shows his own pieces of reusable code to try to optimize it as much as it is possible. Because everybody, who does it more or less wants to make the difference or to make this world a little bit better, helping other people shorten the way they passed and have better results immediately. I want to show you 4 easy steps, that will help you, without any knowledge about optimization, speed-up your code at least twice and make it more efficient and readable.
Captain Obvious says: “Use math functions of the library you work with or just use numpy and you’ll be always alright”. He is right because in most cases the author of the library optimized all the functions in a way, to make his code as fast as it is possible. But almost every python ML library is based on numpy. Numpy official web page says:
NumPy is the fundamental package for scientific computing with Python. It contains among other things:
a powerful N-dimensional array object
sophisticated (broadcasting) functions
tools for integrating C/C++ and Fortran code
useful linear algebra, Fourier transform, and random number capabilities
Actually, in ML we just making some matrix operations, again and again. Numpy uses all possible state-of-the-art optimization techniques you can imagine to make all these operations efficient. So rule number one:
Use numpy instead of python math, use numpy arrays instead of python arrays.
If you follow just this first simple rule, you’ll soon feel the difference.
How to get rid of annoying for loops that just applies the same function on a set of inputs? Just vectorize this function! Numpy allows you to vectorize a function, which means that newly created function will be applied on list inputs and produce an array of function results. According to this post, it might speed up your computation for at least twice. I highly recommend to read it, because there author shows the beautiful comparison of forloops and numpy vectorize and shows measurements as the proofs that numpy vectorize just works faster! Let’s just see some simple code snippets, to show how it works:
import numpy as npfrom math import sin, cos, gammadef func(x):    if x > 10 * sin(x) * cos(x):        return sin(x) * cos(x)    elif x**100 < (x-1)**101:        return x**89    elif x**100 - gamma(x) < (x-1)**101 * gamma(x):        return gamma(x)    else:        return x**56arr = np.random.randn(10000)# FoorLoop methodres = []for i in arr:    res.append(func(i))# Numpy Vectorizationfunc_vect = np.vectorize(func)res_vet = func_vect(arr)
Numpy vectorization code, in this case, runs 2x faster! Vectorization not only can help to speed-up your code but also make it cleaner. For example, make stemming in nltk to run on all tokens, instead of only one:
import numpy as npfrom nltk.stem import PorterStemmerfrom nltk.tokenize import word_tokenizetokens = word_tokenize(text)porter = PorterStemmer()# vectorizing function to able to call on list of tokensstem_words = np.vectorize(porter.stem)tokens_stems = stem_words(tokens)
It looks fancier than ForLoop alternative:
import numpy as npfrom nltk.stem import PorterStemmerfrom nltk.tokenize import word_tokenizetokens = word_tokenize(text)porter = PorterStemmer()def stem_words(tokens):    stems = []    for t in tokens:        stems.append(porter.stem(t))    return stemstokens_stems = stem_words(tokens)
So to conclude, vectorization is a very powerful mechanism, that will help you not only speed-up your code but also to make it look cleaner.
Almost every computer nowadays has more than one core. Not using all of them, if you have paid for it, as for me is a crime. Actually, you have to take from your computer everything it can give you. For ML developer his computer does not have to be a pretty and fancy thing, first of all, it is a hard-working miner, which have to work in three shifts like Stakhanov in USSR. You just have to take from your computer everything it can give you, if you still don’t understand why, just remember one simple thing: “You have paid for it”. The easiest way for ML developer to use more than core in his pipelines is just not to forget to set n_jobs parameter in every sklearn model he uses. n_jobs tells sklearn how many jobs to run in parallel both to fit and predict on the model. Most sklearn models support parallelized training and transformations. This will help you to save much time while GridSearching the best parameters or training models on large or high dimensional datasets. Beware of setting n_jobs to -1, because parallelizations save your time, not RAM. Yeah, actually parallelization might be memory extensive but using at least 2 jobs instead of 1 will already help you to optimize your model training and prediction performance.
Wikipedia says:
MapReduce is a programming model and an associated implementation for processing and generating big data sets with a parallel, distributed algorithm on a cluster.
Looks like something complex, like something you could use only on a big cluster or on an enormous dataset, right? But it is very easy, I’ll explain it in a couple of sentences and 1 example.
MapReduce is divided into two parts:
Map — function, which somehow modifies an input into an output. This function is called on every sample of data, which have to be processed. This step is parallelized, because for each sample, in fact, you do the same operation.Reduce — function, that aggregates all output of the map function. This step is usually sequential.
Map — function, which somehow modifies an input into an output. This function is called on every sample of data, which have to be processed. This step is parallelized, because for each sample, in fact, you do the same operation.
Reduce — function, that aggregates all output of the map function. This step is usually sequential.
For example, you have to calculate the sum of squares of an array of data. Sequential implementation would look like:
arr = [1, 2, 3, 4, 5]result = 0for i in arr:    result += i**2
Let’s convert this task into MapReduce:
Map — calculate the square of the numberReduce — calculate the sum of map results
Map — calculate the square of the number
Reduce — calculate the sum of map results
Sequential implementation in python:
from functools import reducearr = [1, 2, 3, 4, 5]map_arr = map(lambda x: x**2, arr)result = reduce(lambda x, y: x + y, map_arr)
As was aforementioned the map part will be parallelized, reduce would stay sequential. Let’s take a glance on parallelized implementation:
import multiprocessing as mpfrom functools import reducedef pow2(x):    return x**2arr = [1, 2, 3, 4, 5]# count of avaliable cores, of course you can use less for computationscores = mp.cpu_count()pool = mp.Pool(cores)map_arr = pool.map(pow2, arr)result = reduce(lambda x, y: x + y, map_arr)pool.close()pool.join()
So now for this task, you use all the power of your computer! .map method of pool object assumes that the map function has only one argument, there is an equivalent .starmap , which assumes more than one argument in map function, the full description of these functions you can find here. Good to mention is that both .map and .starmap returns mapped values in the same order as input parameters.
Sometimes it is more efficient to compute map function not only on one sample but on a batch of samples. If the computation task is simple and quick enough, the main overhead of the multiprocessing computation will be transferring data between processes, which could slow down the process a lot. As in the previous part of my post we were talking about n_jobs parameter of sklearn models, let's create universal code snippet of base transformer, which could parallelize the process of its transformations, in this case, batch processing will be the best choice.
Here is a code snippet of pandas apply parallelization inside a base sklearn transformer, which you could use in you further pipelines:
import multiprocessing as mpfrom sklearn.base import TransformerMixin, BaseEstimatorclass ParallelTransformer(BaseEstimator, TransformerMixin):    def __init__(self,                 n_jobs=1):        """        n_jobs - parallel jobs to run        """        self.variety = variety        self.user_abbrevs = user_abbrevs        self.n_jobs = n_jobs    def fit(self, X, y=None):        return self    def transform(self, X, *_):        X_copy = X.copy()        cores = mp.cpu_count()        partitions = 1                if self.n_jobs <= -1:            partitions = cores        elif self.n_jobs <= 0:            partitions = 1        else:            partitions = min(self.n_jobs, cores)                if partitions == 1:            # transform sequentially            return X_copy.apply(self._transform_one)                # splitting data into batches        data_split = np.array_split(X_copy, partitions)                pool = mp.Pool(cores)                # Here reduce function - concationation of transformed batches        data = pd.concat(            pool.map(self._preprocess_part, data_split)        )                pool.close()        pool.join()        return data    def _transform_part(self, df_part):        return df_part.apply(self._transform_one)    def _transform_one(self, line):        # some kind of transformations here        return line
I’ll summarize this code. Determine whether to process dataset in parallel or sequential. If in parallel, divide into n_jobs batches all the data and transform in parallel, then concatenate back transformed batches of the dataset. And that’s all! I hope this code snippet, will help you to boost your transformation pipelines.
I hope you liked my post, if you have further questions, ask in comments I’ll be glad to answer it. I’m looking forward to your feedback!
P.S. If you are an experienced developer, who wants to contribute to open source, one of your possible tasks could be helping good but not optimized libraries become fast and production ready. If you don’t know, where to start, you can help normalize library to optimize its performance. I don’t have any connections with the authors of this library, they don’t pay me and it is not an advertisement. Just another library, which needs your help ;)