NumPy — The King of Scientific Computing with Python | by Shubhanshu Singh | Towards Data Science

Python is one of my favourite programming languages ( I love cpp too ). It is super easy to learn because of its beginner-friendly syntax. What makes it stand apart from the other programming language is its awesome developers' community. It comes pre-loaded with so many tools & libraries. And it is one of the most preferred languages in the field of Data Science & Machine Learning.
In this article, I’m going to tell you about one such powerful library in Python known as NumPy.
Enjoy the ride !!
What is NumPy?NumPy vs ListInstallationLet’s get started with NumPy ArraysLet’s check out some built-in methods of NumPya) arangeb) Zeros and Onesc) linspaced) eyee) RandomArray Attributes and Methodsa) Reshapeb) max, min, argmax, argminc) Shaped) dtypeNumPy Indexing and SelectionBroadcastingIndexing a 2D array (matrices)SelectionNumPy Operationsa) Arithematicb) Universal Array FunctionsVectorizationConclusionGithub RepositoryReferencesSuggested Reading
What is NumPy?
NumPy vs List
Installation
Let’s get started with NumPy Arrays
Let’s check out some built-in methods of NumPya) arangeb) Zeros and Onesc) linspaced) eyee) Random
Array Attributes and Methodsa) Reshapeb) max, min, argmax, argminc) Shaped) dtype
NumPy Indexing and Selection
Broadcasting
Indexing a 2D array (matrices)
Selection
NumPy Operationsa) Arithematicb) Universal Array Functions
Vectorization
Conclusion
Github Repository
References
Suggested Reading
NumPy is a Linear Algebra Library for python.
It is the core library for scientific computation in python.It is super fast as it has bindings to C libraries.It provides a high-performance multidimensional array object and tools for working with these arrays.
It is the core library for scientific computation in python.
It is super fast as it has bindings to C libraries.
It provides a high-performance multidimensional array object and tools for working with these arrays.
Who is the winner?
Let’s see the difference
NumPy has a fixed type of nature. NumPy is used for creating homogeneous n-dimensional arrays. Lists can be heterogeneous in nature.
We don’t need to type check while iterating through NumPy arrays. On the other hand, Lists object can be of type int, boolean, string etc.
This code will give an error if we mention the dtype explicitly:
#heterogenous 1D array #gives an error in numpynp_arr = np.array([1,2,"a",3,"b"], dtype=np.int32)
This a valid code for python Lists:
#heterogenous 1D arrayarr = [1,2,"a",3,"b"]
Mathematical operations can’t be performed on heterogeneous data. That’s why it is recommended to have homogenous NumPy arrays
Let’s take one more example:
This code will give an error in python Lists:
x = [1,2,3]y = [4,5,6]#this will give an error,#not possible in Listsx*y
But this a valid code for NumPy:
x = np.array([1,2,3])y = np.array([4,5,6])x*y#Outputnp.array([4,10,18])
We’ll learn more about some awesome features in NumPy in the upcoming sections.
Suppose we have a matrix:
[[1,2,3], [4,5,6], [7,8,9]]
We’ll analyse for a number 7 in the matrix.
Computers read the information in binary.
By default Numpy stores 7 in total memory space of 4 bytes.
On the other hand in Lists, there is a lot of information that needs to be stored for integer 7.
Lists use a built-in int type for python.
The built-in int type consists of four different things:
Depicts the size of that particular integer value.
It tells how many times that particular integer (here 7) has been pointed at.
Every variable in Python is just a reference (a pointer) to an object. It is not the actual value itself.
So for keeping track of references, every object (even integer) has an extra field called reference count that is increased or decreased when a pointer to the object is created or deleted.
It tells whether the object is int, boolean, string etc.
It has its own binary representation
up to 64 bits........00000000 00000111 (8 bytes)
Like this we also have binary representation for size, reference count, object type.
So basically it’s faster to read less bytes of memory.
We can clearly see Numpy takes less memory than Lists for a single integer storage !!
Have look at this illustration
The data on our list will be scattered in the computer’s memory. It doesn't have contiguous blocks of memory. The list will contain just a pointer to those 8 blocks of memory.
On the other hand, NumPy array will have contiguous blocks of memory in the computer. All 8 blocks are right next to each other.
So the takeaway from above analysis is that Lists are very slow and NumPy is superfast !!
NumPy has the speed like cheetah ;)
I’ll recommend you to install Python using the Anaconda distribution to make sure all underlying dependencies (such as Linear Algebra libraries) sync up with the use of conda install.
Refer this link to know more about anaconda installation:
Installation — Anaconda Documentation
If you have Anaconda, install NumPy by going to your terminal or command prompt and start typing:
conda install numpy
If you don’t have Anaconda distribution, you can also use:
pip install numpy
We’ll talk about vectors & matrices.
Vectors are strictly 1-D arrays and Matrices are 2-D arrays
Note: Matrix can still have only one row or one column
First import Numpy
import numpy as np
Let’s create NumPy Arrays using python Lists
Code 1:
#definejust_a_list = [1,2,3]#showjust_a_list
Output:
[1, 2, 3]
Now we will convert the above code 1 snippet variable just_a_list to NumPy array.
Code 2:
np.array(just_a_list)
Output:
array([1, 2, 3])
Code 3:
#definejust_a_matrix = [[1,2,3],[4,5,6],[7,8,9]]#showjust_a_matrix
Output:
[[1, 2, 3], [4, 5, 6], [7, 8, 9]]
Now we will convert the above code 3 snippet variable just_a_matrix to NumPy 2d array
Code 4:
np.array(just_a_matrix)
Output:
array([[1, 2, 3],       [4, 5, 6],       [7, 8, 9]])
It returns evenly spaced values within a given interval.
This is very similar to Python’s own built-in range function.
The upper limit gets excluded in output.
Code 1:
np.arange(0,20)
Output:
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19])
Code 2:
#step size of 2np.arange(0,10,2)
Output:
array([0, 2, 4, 6, 8])
In the above code snippet, 2 is the step size. As you can see in the output, the series is incrementing by the difference of 2.
Code 1:
np.zeros(4)
Output:
array([0., 0., 0., 0.])
So basically np.zeros(4) returns a new array filled with 4 zeros.
Code 2:
np.zeros((3,3))
Output:
array([[0., 0., 0.],       [0., 0., 0.],       [0., 0., 0.]])
np.zeros((3,3)) returns a matrix of shape (3,3) filled with zeros. Shape (3,3) refers to 3 rows & 3 columns.
Code 3:
np.ones(4)
Output:
array([1., 1., 1., 1.])
In the above code snippet, np.ones(4) returns a new array filled with 4 ones.
Code 4:
np.ones((3,3))
Output:
array([[ 1.,  1.,  1.],       [ 1.,  1.,  1.],       [ 1.,  1.,  1.]])
Return evenly spaced numbers over a specified interval.
Code 1:
#generates three numbers#between range 0 to 10np.linspace(0,10,3)
Output:
array([0., 5., 10.])
Code 2:
np.linspace(0,10,20)
Output:
array([ 0.,  0.52631579,  1.05263158,  1.57894737,  2.10526316,         2.63157895,  3.15789474,  3.68421053,  4.21052632,  4.73684211,         5.26315789,  5.78947368,  6.31578947,  6.84210526,  7.36842105,         7.89473684,  8.42105263,  8.94736842,  9.47368421,  10.        ])
Creates an identity matrix.
Code :
np.eye(3)
Output:
array([[1., 0., 0.],       [0., 1., 0.],              [0., 0., 1.]])
Numpy also has a lot of ways to create random number arrays
Creates an array of the given shape and populate it with random samples from a uniform distribution over [0, 1).
Code 1:
np.random.rand(2)
Output:
array([0.01231767, 0.24372946])
Code 2:
np.random.rand(3,3)
Output:
array([[0.75250092, 0.7582111 , 0.16155648],               [0.11020177, 0.51819124, 0.29267394],               [0.70052998, 0.58651773, 0.80589757]])
Return a sample (or samples) from the Standard Normal or Gaussian distribution. Unlike np.random.rand which returns a uniform distribution.
Code 1:
np.random.randn(2)
Output:
array([ 1.5868519 , -2.72817085])
Code 2:
np.random.randn(4,4)
Output:
array([[ 0.24602617, -2.39464955, -0.1382286 ,  1.47872422],           [ 0.70466341, -0.17408009, -2.22898345,  0.28018941],               [ 0.57969196,  0.00664889,  1.20221044,  0.47452959],               [ 0.93939176, -0.07719476, -0.08907384,  1.26465443]])
Return random integers from low (inclusive) to high (exclusive).
Code 1:
np.random.randint(1,100)
Output:
84
Code 2:
#generates 10 random #integers between 1 to 100np.random.randint(1,100,10)
Output:
array([85, 30,  2, 72, 40, 86, 18, 46,  3, 24])
One of the most useful methods you can use on an array is the reshape method.
It returns an array containing the same data with a new shape.
Code 1:
#definearr = np.arange(16)#showarr
Output:
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])
Now let’s reshape the above code 1 snippet variable arr
Code 2:
arr.reshape(4,4)
Output:
array([[ 0,  1,  2,  3],               [ 4,  5,  6,  7],               [ 8,  9, 10, 11],               [12, 13, 14, 15]])
These are useful methods for finding maximum or minimum values. We can also find their index locations using argmin or argmax !!
Let’s generate a random NumPy array of 10 integers
Code 1:
#definerandom_arr = np.random.randint(0,50,10)#showrandom_arr
Output:
array([16,  8, 40,  7, 15, 39,  0, 15, 20, 35])
Code 2:
random_arr.max()
Output:
40
Code 3:
random_arr.argmax()
Output:
2
Code 4:
random_arr.min()
Output:
0
Code 5:
random_arr.argmin()
Output:
6
Shape is an attribute that arrays have (not a method)
We have already defined the variable arr under the code 1 snippet of the Reshape Section.
Code 1:
arr.shape
Output:
(16,)
The above shape indicates that arr is just a one-dimensional vector of rank 1.
Code 2:
arr.reshape(1,16)
Output:
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]])
Notice the two sets of brackets in the output. It indicates the 2-dimensional NumPy array.
Code 3:
arr.reshape(1,16).shape
Output:
(1, 16)
The reshaped arr contains 1 row & 16 columns.
Code 4:
arr.reshape(16,1)
Output:
array([[ 0],        [ 1],        [ 2],        [ 3],        [ 4],        [ 5],        [ 6],        [ 7],        [ 8],        [ 9],        [10],        [11],        [12],        [13],        [14],        [15]])
Code 5:
arr.reshape(16,1).shape
Output:
(16, 1)
The reshaped arr contains 16 rows & 1 column.
Using dtype we can get the data type of the object in the array.
Code 1:
arr.dtype
Output:
dtype(‘int64’)
Now we will learn how to select elements or group of elements from an array. The simplest way of picking up one or more elements from an array looks very similar to python lists
We have already defined the variable arr under the code 1 snippet of the Reshape Section.
Code 1:
#Get a value #at an indexarr[5]
Output:
5
Code 2:
#Get values #in a rangearr[1:5]
Output:
array([1, 2, 3, 4])
The term broadcasting describes how Numpy treats array with different shapes during arithmetic operations.
Numpy arrays differ from a normal Python list because of their ability to broadcast.
Code 1:
#Setting a value with#index rangearr[0:5]=100#Showarr
Output:
array([100, 100, 100, 100, 100, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15])
Code 2:
#Reset array, we'll #see the reason behind it soonarr = np.arange(0,11)#Showarr
Output:
array([ 0,  1,  2,  3,  4,  5,  6,  7,  8,  9, 10])
Code 3:
slice_of_arr = arr[0:6]#Show sliceslice_of_arr
Output:
array([0, 1, 2, 3, 4, 5])
Code 4:
#Change Sliceslice_of_arr[:]=99#Show Slice againslice_of_arr
Output:
array([99, 99, 99, 99, 99, 99])
Code 5:
arr
Output:
array([99, 99, 99, 99, 99, 99, 6, 7, 8, 9, 10])
As you can see in the above output, the changes also occured in our original array !!
This happened because slice_of_arr is just a reference to the first five elements of arr . Assignment operation doesn’t make a separate copy of the data.
To Copy the data, we need to use the copy method !!
Code 6:
arr_copy = arr.copy()arr_copy
Output:
array([99, 99, 99, 99, 99, 99,  6,  7,  8,  9, 10])
Let’s see another example of broadcasting
Code 7:
a = [[1,2,3],[4,5,6]]#Converting to #numpy 2d arraynp_a = np.array(a)#shownp_a
Output:
array([[1, 2, 3],               [4, 5, 6]])
Code 8:
#2 rows, 3 columnsnp_a.shape
Output:
(2, 3)
Code 9:
b = [100,200,300]#converting to #numpy arraynp_b = np.array(b)#shownp_b
Output:
array([100, 200, 300])
Code 10:
np_b.shape
Output:
(3,)
Code 11:
np_b_reshaped = np_b.reshape(1,3)#shownp_b_reshaped
Output:
array([[100, 200, 300]])
Code 12:
#1 row, 3 columnsnp_b_reshaped.shape
Output:
(1, 3)
See the magic of broadcasting
Code 13:
np_a + np_b_reshaped
Output:
array([[101, 202, 303],               [104, 205, 306]])
So basically np_b_reshaped replicates itself for the second row & it’s shape becomes (2,3)
Code 14:
np.array([[100,200,300],[100,200,300]])
Output:
array([[100, 200, 300],               [100, 200, 300]])
The general format is arr_2d[row][col] or arr_2d[row,col].
Code 1 :
arr_2d = np.array(([1,2,3],[4,5,6],[7,8,9]))#Showarr_2d
Output:
array([[1, 2, 3],               [4, 5, 6],               [7, 8, 9]])
Code 2:
#Indexing rowarr_2d[1]
Output:
array([4, 5, 6])
Code 3:
#Getting individual element valuearr_2d[1][0]
Output:
4
Code 4:
#another wayarr_2d[1,0]
Output:
4
Code 5:
#2D array slicing#Shape (2,2) from#top right cornerarr_2d[:2,1:]
Output:
array([[2, 3],               [5, 6]])
Code 6:
#Shape of the #bottom rowarr_2d[2]
Output:
array([7, 8, 9])
Let’s briefly go over how to use brackets for selection based on comparison operators.
Code 1:
arr1 = np.arange(1,11)#showarr1
Output:
array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10])
Code 2:
arr1 > 5
Output:
array([False, False, False, False, False,  True,  True,  True, True,          True])
Code 3:
bool_arr = arr1 > 5#showbool_arr
Output:
array([False, False, False, False, False,  True,  True,  True, True, True])
Code 4:
arr1[bool_arr]
Output:
array([ 6,  7,  8,  9, 10])
Code 5:
arr1[arr1 > 2]
Output:
array([ 3,  4,  5,  6,  7,  8,  9, 10])
You can easily perform array with array arithmetic, or scalar with array arithmetic. Let’s see some examples.
Code 1:
arr2 = np.arange(0,10)
Code 2:
arr2 + arr2
Output:
array([ 0,  2,  4,  6,  8, 10, 12, 14, 16, 18])
Code 3:
arr2 * arr2
Output:
array([ 0,  1,  4,  9, 16, 25, 36, 49, 64, 81])
Code 4:
arr2 - arr2
Output:
array([0, 0, 0, 0, 0, 0, 0, 0, 0, 0])
Warning on division by zero, but not an error!
Numpy will replace it with nan
Code 5:
arr2 / arr2
Output:
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:3: RuntimeWarning: invalid value encountered in true_divide  This is separate from the ipykernel package so we can avoid doing imports untilarray([nan,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.])
Another warning, but not an error. Value will be replaced with infinity.
Code 6:
1 / arr2
Output:
/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:2: RuntimeWarning: divide by zero encountered in true_dividearray([inf, 1., 0.5, 0.33333333, 0.25, 0.2, 0.16666667, 0.14285714, 0.125, 0.11111111])
Cube of numbers
Code 7:
arr2 ** 3
Output:
array([0, 1, 8, 27, 64, 125, 216, 343, 512, 729])
Numpy comes preloaded with many universal array functions, which are essentially just mathematical operations you can use to perform the operation across the array. Let’s see some common ones.
Taking Square Roots
Code 1:
np.sqrt(arr2)
Output:
array([0. , 1., 1.41421356, 1.73205081, 2., 2.23606798, 2.44948974, 2.64575131, 2.82842712, 3.])
Calcualting exponential (e^x)
Code 2:
np.exp(arr2)
Output:
array([1.00000000e+00, 2.71828183e+00, 7.38905610e+00, 2.00855369e+01,        5.45981500e+01, 1.48413159e+02, 4.03428793e+02, 1.09663316e+03,        2.98095799e+03, 8.10308393e+03])
same as arr.max()
Code 3:
np.max(arr2)
Output:
9
Calculating sine value
Code 4:
np.sin(arr2)
Output:
array([ 0.,  0.84147098,  0.90929743,  0.14112001, -0.7568025 ,        -0.95892427, -0.2794155 ,  0.6569866 ,  0.98935825,  0.41211849])
Calculating log
Code 5:
np.log(arr2)
Output:
usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:1: RuntimeWarning: divide by zero encountered in log  """Entry point for launching an IPython kernel.array([ -inf, 0., 0.69314718, 1.09861229, 1.38629436, 1.60943791,  1.79175947, 1.94591015, 2.07944154, 2.19722458])
Vectorization is used to speed up the Python code without using a loop.
Various operations on vectors & matrices can be performed like the dot product, element-wise product etc.
Vectorized version of dot product
Code:
import timea = np.random.rand(1000000)b = np.random.rand(1000000)tic= time.time()c = np.dot(a,b) #dot product of two vectorstoc = time.time()print("Time taken in vectorized version: "+str(1000 * (toc - tic))+" ms")
Output:
Time taken in vectorized version: 1.483917236328125 ms
let’s check the for loop version of dot product
Code:
c = 0tic = time.time()for i in range(1000000):  c += a[i] * b[i]toc = time.time()print("Time taken in for loop version: "+str(1000 *(toc - tic)) + " ms")
Output:
Time taken in for loop version: 579.4703960418701 ms
So basically the takeaway from the above analysis is that Vectorization is way more powerful and efficient than simple for loop. It takes very less time. So whenever possible avoid explicit for loops.
I have tried my best to give you a comprehensive introduction to NumPy.
Linear Algebra becomes a cakewalk with this package.
NumPy is a fundamental package for scientific computation & mathematical operations in python.
NumPy is way more powerful than Lists.
Go ahead and check out my GitHub repository for full code access.
Here is a link to my Medium Articles GitHub repository:
github.com
Here is a direct link to my NumPy Jupyter Notebook containing all the codes snippets used in this article :-
github.com
numpy.org
If you are interested in learning CS Fundamentals. If you are curious about learning Data Structures & Algorithms, then do check out my first blog on Heap Sort Algorithm.
medium.com
Thank you for reading this article.
Leave a comment below if you have any questions.