Numpy Cheat Sheet. A quick guide to the basics of the... | by XuanKhanh Nguyen | Towards Data Science

NumPy is the library that gives Python its ability to work with data at speed. Numpy has several advantages over data cleaning and manipulation. It allows for efficient operations on the data structures often used in machine learning: vectors, matrices, and tensors.
When I first learned NumPy, I had trouble remembering all the functions and methods that needed. So I put together the most frequently used Numpy operations. I sometimes come back to this note to refresh my memory. And I am glad if it helps you on your journey too.
The structure of this note:
N-dimensional arraysArray shape manipulationsNumerical operations on arrayArray manipulations routines ( select and split)Statistical operations
N-dimensional arrays
Array shape manipulations
Numerical operations on array
Array manipulations routines ( select and split)
Statistical operations
This is a long note, make yourself a cup of tea, and let’s get started!
As always, we need to import NumPy library:
import numpy as np
Arrays are a data structure for storing elements of the same type. Each item stored in an array is called an element. Each location of an element in an array has a numerical index, which is used to identify the element.
1D array (i.e., single dimensional array) stores a list of variables of the same data type. It is possible to access each variable using the index.
2D array (i.e, multi-dimensional array) stores data in a format consisting of rows and columns.
Arrays use less memory than lists
Arrays have significantly more functionality
Arrays require data to be homogeneous; lists do not
Arithmetic on arrays operates like matrix multiplication
NumPy is used to work with arrays. The array object in NumPy is called ndarray.
To create a vector, we simply create a one-dimensional array. Just like vectors, these arrays can be represented horizontally (i.e., rows) or vertically (i.e., columns).
# Create 1 dimensional array (vector)vector_row = np.array([1,2,3]) # Create vector as a row>>> array([1, 2, 3])vector_column = np.array([[1],[2],[3]]) #Create vector as a column>>> array([[1],           [2],             [3]])
To create a matrix, we can use a NumPy two-dimensional array. In our solution, the matrix contains three rows and two columns.
matrix = np.array([(1,2,3),(4,5,6)]) # Two dimensional array>>> array([[1, 2, 3],          [4, 5, 6]])
A sparse matrix is a matrix in which most of the elements are zero. Sparse matrices only store nonzero elements and assume all other values will be zero, leading to significant computational savings.
Imagine a matrix where the columns are every article on Medium, the rows are every Medium reader, and the values are how long (minutes) a person has read that particular article. This matrix would have tens of thousands of columns and millions of rows! However, since most readers do not read all articles, the vast majority of elements would be zero.
Let’ say, we want to create a NumPy array with two nonzero values, then converted it into a sparse matrix. If we view the sparse matrix, we can see that only the nonzero values are stored:
from scipy import sparsematrix_large = np.array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0],                         [0, 1, 0, 0, 0, 0, 0, 0, 0, 0],                         [3, 0, 0, 0, 0, 0, 0, 0, 0, 0]])# Create compressed sparse row (CSR) matrixmatrix_large_sparse = sparse.csr_matrix(matrix_large)print(matrix_large_sparse)>>>   (1, 1)	1      (2, 0)	3
In the example above, (1, 1) and (2, 0) represent the indices of the non-zero values 1 and 3, respectively. For example, the element 1 is in the second row and second column.
np.zeros() function returns a new array of given shape and type, filled with zero.
# Create  1d array of zeros, length 3np.zeros(3) >>> array([0., 0., 0.])# 2x3 array of zerosnp.zeros((2,3))>>>array([[0., 0., 0.],         [0., 0., 0.]])
np.ones() function returns a new array of given shape and type, filled with one.
# Create 3x4 array of onesnp.ones((3,4)) >>> array([[1., 1., 1., 1.],           [1., 1., 1., 1.],           [1., 1., 1., 1.]])
np.eye() function returns a matrix having 1’s on the diagonal and 0’s elsewhere.
# Create 5x5 array of 0 with 1 on diagonal (Identity matrix)np.eye(5) >>> array([[1., 0., 0., 0., 0.],           [0., 1., 0., 0., 0.],           [0., 0., 1., 0., 0.],           [0., 0., 0., 1., 0.],           [0., 0., 0., 0., 1.]])
np.linspace() function returns an evenly spaced sequence in a specified interval.
# Create an array of 6 evenly divided values from 0 to 100np.linspace(0,100,6) >>> array([  0.,  20.,  40.,  60.,  80., 100.])
np.arange(start, stop, step) function returns the ndarray object containing evenly spaced values within the given range.
The parameters determine the range of values:
start defines the first value in the array.stop defines the end of the array and isn’t included in the array.step is the number that defines the spacing (difference) between every two consecutive values in the array and defaults to 1.
start defines the first value in the array.
stop defines the end of the array and isn’t included in the array.
step is the number that defines the spacing (difference) between every two consecutive values in the array and defaults to 1.
Note:step can’t be zero. Otherwise, we will get a ZeroDivisionError. We can’t move away anywhere from start if the increment or decrement is 0.
# Array of values from 0 to less than 10 with step 3 np.arange(0,10,3)>>> array([0, 3, 6, 9])
np.full(shape, fill_value) function returns a new array of a specified shape, fills with fill_value.
# 2x3 array with all values 5np.full((2,3),5)>>> array([[5, 5, 5],           [5, 5, 5]])
np.random.rand() function returns an array of specified shape and fills it with random values.
# 2x3 array of random floats between 0–1np.random.rand(2,3)>>> array([[0.37174775, 0.59954596, 0.50488967],           [0.22703386, 0.59914441, 0.68547572]])# 2x3 array of random floats between 0–100np.random.rand(2,3)*100>>> array([[23.17345972, 98.62743214, 21.40831291],           [87.08603104, 84.23376262, 63.90231179]])# 2x3 array with random ints between 0–4np.random.randint(5,size=(2,3))>>> array([[2, 3, 4],           [4, 4, 0]])
It will be valuable to check the shape and size of an array both for further calculations and simply as a gut check after some operation.
NumPy arrays have an attribute called shape that returns a tuple with each index having the number of corresponding elements.
arr = np.array([(1,2,3),(4,5,6)])arr.shape # Returns dimensions of arr (rows,columns)>>> (2, 3)
In the example above, (2, 3) means that the array has 2 dimensions, and each dimension has 3 elements.
It is important to know how to reshape the NumPy arrays so that our data meets the expectation of specific Python libraries. For example, Scikit- learn requires a one-dimensional array of output variables yto be shaped like a two-dimensional array with one column and outcomes for each row.
Some algorithms, like the Long Short-Term Memory recurrent neural network in Keras, require input to be specified as a three-dimensional array comprised of samples, timesteps, and features.
reshape() allows us to restructure an array so that we maintain the same data but it is organized as a different number of rows and columns.
Note: The shape of the original and new matrix contains the same number of elements (i.e, same size)
arr1 = np.arange(1, 11)  # numbers 1 to 10>>> array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10])print(arr1.shape) # Prints a tuple for the one dimension.>>> (10,)
We can usereshape() method to reshape our array to a 2 by 5 dimensional array.
arr1_2d = arr1.reshape(2, 5)print(arr1_2d)>>> array([[ 1,  2,  3,  4,  5],           [ 6,  7,  8,  9, 10]])
If we want NumPy to automatically determine what size/length a particular dimension should be, specify the dimension as -1 which effectively means “as many as needed.” For example, reshape(2, -1) means two rows and as many columns as needed.
arr1.reshape(2, 5)arr1.reshape(-1, 5)  # same as above: arr1.reshape(2, 5)arr1.reshape(2, -1)  # same as above: arr1.reshape(2, 5)
Transposing is a common operation in linear algebra where the column and row indices of each element are swapped.
From the last example, arr1_2d is a 2 by 5 dimensional array, we want to switch its rows with its columns.
arr1_2d.T>>> array([[ 1,  6],          [ 2,  7],          [ 3,  8],          [ 4,  9],          [ 5, 10]])
flatten() is a simple method to transform a matrix into a one-dimensional array.
arr1_2d.flatten()>>> array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10])
resize(arr, new_shape) function returns a new array with the specified shape.If the new array is larger than the original array, then the new array is filled with repeated copies of arr.
# Resize arr1_2d to 3 rows, 4 columns resize_arr = np.resize(arr1_2d, (3,4))resize_arr>>> array([[ 1,  2,  3,  4],           [ 5,  6,  7,  8],           [ 9, 10,  1,  2]])
The inverse of a matrix A is a matrix that, when multiplied by A results in the identity. A good example is in finding the vector of coefficient values in linear regression.
Use NumPy’s linear algebra inv method:
matrix = np.array([[1, 2],                   [3, 4]])# Calculate inverse of matrixnp.linalg.inv(matrix)>>> array([[-2. ,  1. ],          [ 1.5, -0.5]])
When I was first learning Python, one of the errors that I ran into quite often — and sometimes still run into now — looked like this:
Arrays need to be declared whereas lists do not need declaration because they are a part of Python’s syntax. This is the reason lists are more often used than arrays. But in case of performing some arithmetic function to our list, we should go with arrays instead.
In case we want to store a large amount of data, we should consider arrays because they can store data very compactly and efficiently.
arr = np.array([(1,2,3),(4,5,6)])>>> array([[1, 2, 3],          [4, 5, 6]])arr_to_list = arr.tolist() # Convert arr to a Python list>>> [[1, 2, 3], [4, 5, 6]]np.array(arr_to_list) # Convert list to array >>> array([[1, 2, 3],          [4, 5, 6]])
Other useful functions to describe the array:
arr.size # Return number of elements in arrlen(arr) #Length of arrayarr.ndim # Number of array dimensionarr.dtype # Return type of elements in arrarr.dtype.name # Name of data typearr.astype(int) # Convert an array to a different typearr.astype(dtype) # Convert arr elements to type dtypenp.info(np.eye) # View documentation for np.eye
The trace is the sum of all the diagonal elements of a square matrix.
arr = np.array([[2, 0, 0], [0, 2, 0], [0, 0, 2]])np.trace(arr)>>> 6
Determinants a matrix is a special number that can be calculated from a square matrix. It can sometimes be useful to calculate the determinant of a matrix. NumPy makes this easy with det().
matrix = np.array([[1, 2, 3],                   [2, 4, 6],                   [3, 8, 9]])# Return determinant of matrixnp.linalg.det(matrix)>>> 0.0
The rank of a matrix is the estimate of the number of linearly independent rows or columns in a matrix. Knowing the rank of a matrix is important. While solving systems of linear equations, the rank can tell us whether Ax = 0has a single solution or multiple solutions.
matrix = np.array([[1, 1, 3],                   [1, 2, 4],                   [1, 3, 0]])# Return matrix ranknp.linalg.matrix_rank(matrix)>>> 3
Many machine learning problems can be modeled with linear algebra with solutions derived from eigenvalues and eigenvectors.
In NumPy’s linear algebra toolset, eig lets us calculate the eigenvalues, and eigenvectors of any square matrix.
matrix = np.array([[0, 1, 2],                   [3, 4, 5],                   [6, 7, 8]])# Calculate eigenvalues and eigenvectorseigenvalues, eigenvectors = np.linalg.eig(matrix)eigenvalues>>> array([ 1.33484692e+01, -1.34846923e+00, -2.48477279e-16])eigenvectors>>> array([[ 0.16476382,  0.79969966,  0.40824829],           [ 0.50577448,  0.10420579, -0.81649658],           [ 0.84678513, -0.59128809,  0.40824829]])
When we add, subtract, multiply or divide a matrix by a number, this is called the scalar operation. During scalar operations, the scalar value is applied to each element in the array, therefore, the function returns a new matrix with the same number of rows and columns.
new_arr = np.arange(1,10)>>> array([1, 2, 3, 4, 5, 6, 7, 8, 9])# Add 1 to each array elementnp.add(new_arr,1)>>> array([ 2,  3,  4,  5,  6,  7,  8,  9, 10])
Similarly, we can subtract, multiply, or divide a matrix by a number using functions below:
np.subtract(arr,2) # Subtract 2 from each array elementnp.multiply(arr,3) # Multiply each array element by 3np.divide(arr,4) # Divide each array element by 4 (returns np.nan for division by zero)np.power(arr,5) # Raise each array element to the 5th power
A matrix can only be added to (or subtracted from) another matrix if the two matrices have the same dimensions, that is, they must have the same number of rows and columns.
When multiplying matrices, we take rows of the first matrix and multiply them by the corresponding columns of the second matrix.
Note: Remember “rows first, columns second.”
It is important to know the shape of matrics. Then the matrics operations are simple using the NumPy library.
np.add(arr1,arr2) # Elementwise add arr2 to arr1np.subtract(arr1,arr2) # Elementwise subtract arr2 from arr1np.multiply(arr1,arr2) # Elementwise multiply arr1 by arr2np.divide(arr1,arr2) # Elementwise divide arr1 by arr2np.power(arr1,arr2) # Elementwise raise arr1 raised to the power of arr2np.array_equal(arr1,arr2) # Returns True if the arrays have the same elements and shape
Other math operations:
np.sqrt(arr) # Square root of each element in the arraynp.sin(arr) # Sine of each element in the arraynp.log(arr) # Natural log of each element in the arraynp.abs(arr) # Absolute value of each element in the arraynp.ceil(arr) # Rounds up to the nearest intnp.floor(arr) # Rounds down to the nearest intnp.round(arr) # Rounds to the nearest int
append() function is used to append values to the end of a given array.
np.append ([0, 1, 2], [[3, 4, 5], [6, 7, 8]])>>> array([0, 1, 2, 3, 4, 5, 6, 7, 8])np.append([[0, 1, 2], [3, 4, 5]],[[6, 7, 8]], axis=0)>>> array([[0, 1, 2],           [3, 4, 5],           [6, 7, 8]])
The axis along which values are appended. If the axis is not given, both array and values are flattened before use.
insert(): is used to insert the element before the given index of the array.
arr = np.arange(1,6)np.insert(arr,2,10) # Inserts 10 into arr before index 2>>>array([ 1,  2, 10,  3,  4,  5])
delete()we can delete any row and column from the ndarray
arr = np.arange(12).reshape(3, 4)>>> [[ 0  1  2  3]     [ 4  5  6  7]     [ 8  9 10 11]]np.delete(arr,2,axis=0) # Deletes row on index 2 of arr>>> array([[0, 1, 2, 3],           [4, 5, 6, 7]])np.delete(arr,3,axis=1) # Deletes column on index 3 of arr>>> array([[ 0,  1,  2],           [ 4,  5,  6],           [ 8,  9, 10]])
sort()function can be used to sort the list in both ascending and descending order.
oned_arr = np.array([3,8,5,1])np.sort(oned_arr)>>> array([1, 3, 5, 8])arr = np.array([[5, 4, 6, 8],                [1, 2, 4, 8],                [1, 5, 2, 4]])# sort each column of arrnp.sort(arr, axis=0)>>> array([[1, 2, 2, 4],           [1, 4, 4, 8],          [5, 5, 6, 8]])# sort each row of Xnp.sort(arr, axis=1)>>> array([[4, 5, 6, 8],          [1, 2, 4, 8],          [1, 2, 4, 5]])
Joining means putting contents of two or more arrays in a single array. In NumPy, we join arrays by axes. We pass a sequence of arrays that we want to join to the concatenate() function, along with the axis. If the axis is not explicitly passed, it is taken as 0.
# Adds arr2 as rows to the end of arr1arr1 = np.array([1, 2, 3])arr2 = np.array([4, 5, 6])arr = np.concatenate((arr1, arr2), axis=0)>>> array([1, 2, 3, 4, 5, 6])# Adds arr2 as columns to end of arr1arr1 = np.array([[1, 2, 3],[4, 5, 6]])arr2 = np.array([[7, 8, 9],[10, 11, 12]])arr = np.concatenate((arr1,arr2),axis=1)>>> array([[ 1,  2,  3,  7,  8,  9],           [ 4,  5,  6, 10, 11, 12]])
Cool, now we know how to merges multiple arrays into one. How to break one array into multiple? We use array_split() for splitting arrays, we pass it the array we want to split and the number of splits.
Note: If the array has fewer elements than required, it will adjust from the end accordingly.
# Splits arr into 4 sub-arraysarr = np.array([1, 2, 3, 4, 5, 6])new_arr = np.array_split(arr, 4) >>> [array([1, 2]), array([3, 4]), array([5]), array([6])]# Splits arr horizontally on the 2nd indexarr = np.array([1, 2, 3, 4, 5, 6])new_arr = np.hsplit(arr, 2)>>> [array([1, 2, 3]), array([4, 5, 6])]
NumPy offers a wide variety of methods for indexing and slicing elements or groups of elements in arrays.
Note: NumPy arrays are zero-indexed, meaning that the index of the first element is 0, not 1.
Suppose we have two arrays, one contains user_name, and the other stores the number of articles that the person has read.
user_name = np.array(['Katie','Bob','Scott','Liz','Sam'])articles = np.array([100, 38, 91, 7, 25])user_name[4] # Return the element at index 4>>> 'Sam'articles[3] = 17 # Assign array element on index 1 the value 4>>>array([100,  38,  91,  17,  25])user_name[0:3] # Return the elements at indices 0,1,2>>> array(['Katie', 'Bob', 'Scott'], dtype='<U5')user_name[:2] # Return the elements at indices 0,1>>> array(['Katie', 'Bob'], dtype='<U5')articles<50 # Return an array with boolean values>>> array([False,  True, False,  True,  True])articles[articles < 50] # Return the element valuesarray([38,  7, 25])# Return the user_name that read more than 50 articles but less than 100 articlesuser_name[(articles < 100 ) & (articles >50)]>>> array(['Scott'], dtype='<U5')
We use similar methods for selecting elements in multi-dimensional arrays:
arr[2,5] # Returns the 2D array element on index [2][5]arr[1,3]=10 # Assigns array element on index [1][3] the value 10arr[0:3] # Returns rows 0,1,2arr[0:3,4] # Returns the elements on rows 0,1,2 at column 4arr[:2] # Returns returns rows 0,1arr[:,1] # Returns the elements at index 1 on all rows
Often we want to know the maximum and minimum value in an array or subset of an array. This can be accomplished with the max and min methods. Using the axis parameter we can also apply the operation along a certain axis:
Suppose we store the number of articles a person per month in an array.
articles = np.array([[10, 23, 17],                   [41, 54, 65],                   [71, 18, 89]])# Return maximum elementnp.max(articles)>>> 89np.max(articles, axis=0) # Find maximum element in each column>>> array([71, 54, 89])np.max(articles, axis=1) # Find maximum element in each row>>> array([23, 65, 89])
We can use similar methods to find the minimum elements:
np.min(arr) # Return minimum elementnp.min(arr,exis=0)# Find minimum element in each columnnp.min(arr,axis=1)# Find minimum element in each row
Just like with max() and min(), we can easily get descriptive statistics about the whole matrix or do calculations along a single axis.
np.mean(arr,axis=0) # Return mean along specific axisarr.sum() # Return sum of arrb.cumsum(axis=1) # Cumulative sum of the elementsnp.var(arr) # Return the variance of arraynp.std(arr,axis=1) # Return the standard deviation of specific axisarr.corrcoef() # Return correlation coefficient of array
The code in this note is available on Github.
I consider this note as the basics of NumPy. You probably come across these functions repeatedly when reading existing code at work or doing tutorials online. I will try to continuously update this as I find more useful Numpy functions.
All learning activities are undertaken throughout time and experience. It is impossible to learn Python in a couple of hours. Remember that the hardest part of any endeavor is the beginning, and you have passed that, keep on, keeping on!!!
Numpy is a very important library on which almost every data science or machine learning Python packages such as SciPy, Matplotlib, Scikit-learn depends on to a reasonable extent. It is important to have a strong understanding of the fundamentals. Conveniently, there are some great resources to help with this task. I have listed some of my favorites below, some of which get deeper into aspects of linear algebra; check them out if you are eager to learn more!
Linear Algebra, MITBasic Linear Algebra for Deep LearningThe difference between NumPy array and matricsSparse matrics, SciPy documentnp.reshape() steps by stepsMachine Learning — Eigenvalue and eigenvector
Linear Algebra, MIT
Basic Linear Algebra for Deep Learning
The difference between NumPy array and matrics
Sparse matrics, SciPy document
np.reshape() steps by steps
Machine Learning — Eigenvalue and eigenvector