Top 10 Matrix Operations in Numpy with Examples
By: Rukshan Pramoditha
Tags: Matrix, Arrays, Numpy, Python, Linear Algebra

About 30–40% of the mathematical knowledge required for Data Science and Machine Learning comes from linear algebra. Matrix operations play a significant role in linear algebra. Today, we discuss 10 of such matrix operations with the help of the powerful numpy library. Numpy is generally used to perform numerical calculations in Python. It also has special classes and sub-packages for matrix operations. The use of vectorization allows numpy to perform matrix operations more efficiently by avoiding many for loops.

I will include the meaning, background description and code examples for each matrix operation discussing in this article. The “Key Takeaways” section at the end of this article will provide you with some more specific facts and a brief summary of matrix operations. So, make sure to read that section as well.

I will discuss each matrix operation in the following order. Here is the list of the top 10 matrix operations I have chosen for you carefully.

Inner product Dot product Transpose Trace Rank Determinant True inverse Pseudo inverse Flatten Eigenvalues and eigenvectors

Prerequisites

To get the full advantage of this article, you should know the numpy basics and array creation methods. If you don’t have that knowledge, read the following article written by me.

Let’s get started with the first one, the inner product.

Inner product

The inner product takes two vectors of equal size and returns a single number (scalar). This is calculated by multiplying the corresponding elements in each vector and adding up all of those products. In numpy, vectors are defined as one-dimensional numpy arrays.

To get the inner product, we can use either np.inner() or np.dot(). Both give the same results. The inputs for these functions are two vectors and they should be the same size.

Wait till loading the Python code!

The inner product of two vectors (Image by author)

Dot product

The dot product is defined for matrices. It is the sum of the products of the corresponding elements in the two matrices. To get the dot product, the number of columns in the first matrix should be equal to the number of rows in the second matrix.

There are two ways to create matrices in numpy. The most common one is to use the numpy ndarray class. Here we create two-dimensional numpy arrays (ndarray objects). The other one is to use the numpy matrix class. Here we create matrix objects.

The dot product of both ndarray and matrix objects can be obtained using np.dot().

Wait till loading the Python code!

The dot product of two matrices (Image by author)

When multiplying two ndarray objects using the * operator, the result is the element-by-element multiplication. On the other hand, when multiplying two matrix objects using the * operator, the result is the dot (matrix) product which is equivalent to the np.dot() as previous.

Wait till loading the Python code!

Different behaviour of * operator on matrix and ndarray objects (Image by author)

Transpose

The transpose of a matrix is found by switching its rows with its columns. We can use np.transpose() function or NumPy ndarray.transpose() method or ndarray.T (a special method which does not require parentheses) to get the transpose. All give the same output.

Wait till loading the Python code!

Getting transpose of a matrix (Image by author)

The transpose can also be applied to a vector. However, technically, a one-dimensional numpy array cannot be transposed.

import numpy as np a = np.array([1, 2, 3])

print("a = ")

print(a)

print("

a.T = ")

print(a.T)

Image by author

If you really want to transpose a vector, it should be defined as a two-dimensional numpy array with double square brackets.

import numpy as np a = np.array([[1, 2, 3]])

print("a = ")

print(a)

print("

a.T = ")

print(a.T)

Image by author

Trace

The trace is the sum of diagonal elements in a square matrix. There are two methods to calculate the trace. We can simply use the trace() method of an ndarray object or get the diagonal elements first and then get the sum.

import numpy as np a = np.array([[2, 2, 1],

[1, 3, 1],

[1, 2, 2]])

print("a = ")

print(a)

print("

Trace:", a.trace())

print("Trace:", sum(a.diagonal()))

Image by author

Rank

The rank of a matrix is the dimensions of the vector space spanned (generated) by its columns or rows. In other words, it can be defined as the maximum number of linearly independent column vectors or row vectors.

The rank of a matrix can be found using the matrix_rank() function which comes from the numpy linalg package.

import numpy as np a = np.arange(1, 10)

a.shape = (3, 3)

print("a = ")

print(a)

rank = np.linalg.matrix_rank(a)

print("

Rank:", rank)

Image by author

Determinant

The determinant of a square matrix can be calculated det() function which also comes from the numpy linalg package. If the determinant is 0, that matrix is not invertible. It is known as a singular matrix in algebra terms.

import numpy as np a = np.array([[2, 2, 1],

[1, 3, 1],

[1, 2, 2]])

print("a = ")

print(a)

det = np.linalg.det(a)

print("

Determinant:", np.round(det))

Image by author

True inverse

The true inverse of a square matrix can be found using the inv() function of the numpy linalg package. If the determinant of a square matrix is not 0, it has a true inverse.

import numpy as np a = np.array([[2, 2, 1],

[1, 3, 1],

[1, 2, 2]])

print("a = ")

print(a)

det = np.linalg.det(a)

print("

Determinant:", np.round(det))

inv = np.linalg.inv(a)

print("

Inverse of a = ")

print(inv)

Image by author

If you try to compute the true inverse of a singular matrix (a square matrix whose determinant is 0), you will get an error.

import numpy as np a = np.array([[2, 8],

[1, 4]])

print("a = ")

print(a)

det = np.linalg.det(a)

print("

Determinant:", np.round(det))

inv = np.linalg.inv(a)

print("

Inverse of a = ")

print(inv)

Image by author

Pseudo inverse

The pseudo (not genuine) inverse can be calculated even for a singular matrix (a square matrix whose determinant is 0) using the pinv() function of the numpy linalg package.

import numpy as np a = np.array([[2, 8],

[1, 4]])

print("a = ")

print(a)

det = np.linalg.det(a)

print("

Determinant:", np.round(det))

pinv = np.linalg.pinv(a)

print("

Pseudo Inverse of a = ")

print(pinv)

Image by author

There is no difference between true inverse and pseudo-inverse if a square matrix is non-singular (determinant is not 0).

Flatten

Flatten is a simple method to transform a matrix into a one-dimensional numpy array. For this, we can use the flatten() method of an ndarray object.

import numpy as np a = np.arange(1, 10)

a.shape = (3, 3)

print("a = ")

print(a)

print("

After flattening")

print("------------------")

print(a.flatten())

Image by author

Eigenvalues and eigenvectors

Let A be an n x n matrix. A scalar λ is called an eigenvalue of A if there is a non-zero vector x satisfying the following equation.

Image by author

The vector x is called the eigenvector of A corresponding to λ.

In numpy, both eigenvalues and eigenvectors can be calculated simultaneously using the eig() function.

import numpy as np a = np.array([[2, 2, 1],

[1, 3, 1],

[1, 2, 2]])

print("a = ")

print(a)

w, v = np.linalg.eig(a)

print("

Eigenvalues:")

print(w)

print("

Eigenvectors:")

print(v)

Image by author

The sum of eigenvalues (1+5+1=7) is equal to the trace (2+3+2=7) of the same matrix! The product of the eigenvalues (1x5x1=5) is equal to the determinant (5) of the same matrix!

Eigenvalues and eigenvectors are extremely useful in the Principal Component Analysis (PCA). In PCA, the eigenvectors of the correlation or covariance matrix represent the principal components (the directions of maximum variance) and the corresponding eigenvalues represent the amount of variation explained by each principal component. If you want to learn more about them, read the following articles written by me.

Key Takeaways

Thanks to the numpy library, you can perform matrix operations easily with just one or 2 lines of code. Today, we have performed 10 matrix operations in numpy. Numpy has common functions as well as special functions dedicated to linear algebra, for example, the linalg package has some special functions dedicated to linear algebra.

In numpy, matrices and ndarrays are two different things. The best way to get familiar with them is by experimenting with the codes by yourself. It is always better to check the dimensions of matrices and ndarrays.

In Scikit-learn machine learning libraries, most of the matrix operations discussed today work behind the scenes when we create and fit a model. For example, when we use the Scikit-learn PCA() function, the eigenvalues and eigenvectors are calculated behind the scenes. Scikit-learn and many other libraries such as pandas, seaborn, matplotlib were built on top of numpy. So, numpy is a powerful Python library.

We can also combine some matrix operations together to perform complex calculations. For example, if you want to multiply 3 matrices called A, B and C in that order, we can use np.dot(np.dot(A, B), C). The dimensions of A, B and C should be matched accordingly.

Thanks for reading!

This tutorial was designed and created by Rukshan Pramoditha, the Author of Data Science 365 Blog.

Read my other articles at https://rukshanpramoditha.medium.com

2021–03–24