What is Vectorization in Machine Learning? | by Jalal Mansoori | Towards Data Science

Make your code execute fast using vectorization
What is Vectorization?How Vectorization is important in Machine learning?Example: Unvectorized Vs Vectorized ImplementationAdvantages of Vectorized ImplementationDemonstration on jupyter notebook
What is Vectorization?
How Vectorization is important in Machine learning?
Example: Unvectorized Vs Vectorized Implementation
Advantages of Vectorized Implementation
Demonstration on jupyter notebook
The first time when I learned about the concept of Vectorization it was when I was learning the most well-known course Machine learning by Prof Andrew Ng on the Coursera platform.
According to Prof Andrew Ng :
“ The ability to perform Vectorization has become a key skill “
So let’s learn this skill and include it into our tools box :)
Vectorization is a technique by which you can make your code execute fast. It is a very interesting and important way to optimize algorithms when you are implementing it from scratch.
Now, with the help of highly optimized numerical linear algebra libraries in C/C++, Octave/Matlab, Python, ...etc. We can make our code run efficiently.
Thanks to people specializing in numerical computing!
In this tutorial, we will play with :
Numpy: Fast Numerical Computing Library in Python
Just like in the real-world we are interested in solving any kind of problem efficiently in such a way that the amount of error is reduced as much as possible.
In machine learning, there’s a concept of an optimization algorithm that tries to reduce the error and computes to get the best parameters for the machine learning model.
So by using a vectorized implementation in an optimization algorithm we can make the process of computation much faster compared to Unvectorized Implementation.
A basic example of Optimization Algorithm: Gradient Descent
Now let’s first play with Numpy to get some idea about using highly optimized computing library
With Numpy: It took 1.52 ms to mean time per loop
Without Numpy: It took 69.9 ms to mean time per loop
# With Python Listsa=list(range(1000000)) #10^6 numbers generated%timeit [val + 5 for val in a] #Computing Element Wise Operation# With Numpy Arraysimport numpy as npa=np.array(a) #Converting into numpy array type%timeit a+5
range(): range function is used to generate a sequence of numbers over time
%timeit: It is an ipython magic function that can be used to time a particular piece of code.
Let’s understand the concept of vectorization using hypothesis function of linear regression in which theta represents weights/coefficients and x represents features
Now there are two ways we can implement the above function :
Using For loopsUsing Vectorization
Using For loops
Using Vectorization
First, let’s generate random numbers for theta and x vectors
n=10000# Generate n random numbers between 0 - 1theta = np.random.rand(n)#Generate n random integers between 0 - 100 x = np.random.randint(0, 100, n)
Now observe the difference between Unvectorized vs Vectorized Code
Both implementations give us the same output but the question is how efficiently one computes?
Using Unvectorized Code: It took 6.46 ms to mean time per loop
Using Vectorized Code: It took 18.8 μs to mean time per loop
# Unvectorizeddef unvectorized(n, theta, x):        prediction= 0.0        for j in range(n):        prediction= prediction + theta[j]*x[j]        return prediction# Vectorizeddef vectorized(theta, x):        prediction= 0.0        prediction = np.dot(theta.transpose() , x)       return prediction#Comparing Performance%timeit unvectorized(n, theta, x)%timeit vectorized(theta, x)
Our code runs efficientlyOur code becomes simpler and easy to debug
Our code runs efficiently
Our code becomes simpler and easy to debug
All the code used in this tutorial is available on my GitHub account Click Here
In this tutorial, you learned about the vectorization technique that is one of the ways to make your code efficient.
See you in the next tutorial :)
Gmail-> jalalmansoori19@gmail.com
Github -> https://github.com/jalalmansoori19
https://speakerdeck.com/jakevdp/losing-your-loops-fast-numerical-computing-with-numpy-pycon-2015Machine Learning Course on Coursera by Prof Andrew Ng
https://speakerdeck.com/jakevdp/losing-your-loops-fast-numerical-computing-with-numpy-pycon-2015
Machine Learning Course on Coursera by Prof Andrew Ng