Linear Algebra for Deep Learning. The Math behind every deep learning... | by Vihar Kurama | Towards Data Science

The Math behind every deep learning program.
Deep Learning is a subdomain of machine learning, concerned with the algorithm which imitates the function and structure of the brain called the artificial neural network.
Linear algebra is a form of continuous rather than discrete mathematics, many computer scientists have little experience with it. A good understanding of linear algebra is essential for understanding and working with many machine learning algorithms, especially deep learning algorithms.
Linear algebra, probability and calculus are the ‘languages’ in which machine learning is formulated. Learning these topics will contribute a deeper understanding of the underlying algorithmic mechanics and allow development of new algorithms.
When confined to smaller levels, everything is math behind deep learning. So it is essential to understand basic linear algebra before getting started with deep learning and programming it.
The core data structures behind Deep-Learning are Scalars, Vectors, Matrices and Tensors. Programmatically, let’s solve all the basic linear algebra problems using these.
Scalars are single numbers and are an example of a 0th-order tensor. The notation x ∈ R states that x is a scalar belonging to a set of real-values numbers, R.
There are different sets of numbers of interest in deep learning. N represents the set of positive integers (1,2,3,...). Z designates the integers, which combine positive, negative and zero values. Q represents the set of rational numbers that may be expressed as a fraction of two integers.
Few built-in scalar types are int, float, complex, bytes, Unicode in Python. In In NumPy a python library, there are 24 new fundamental data types to describe different types of scalars. For information regarding datatypes refer documentation here.
Defining Scalars and Few Operations in Python:
The following code snippet explains few arithmetic operations on Scalars.
<class 'int'><class 'float'>12.5-2.537.50.6666666666666666
The following code snippet checks if the given variable is scalar or not.
TrueFalseTrue
Vectors are ordered arrays of single numbers and are an example of 1st-order tensor. Vectors are fragments of objects known as vector spaces. A vector space can be considered of as the entire collection of all possible vectors of a particular length (or dimension). The three-dimensional real-valued vector space, denoted by R^3 is often used to represent our real-world notion of three-dimensional space mathematically.
To identify the necessary component of a vector explicitly, the ith scalar element of a vector is written as x[i].
In deep learning vectors usually represent feature vectors, with their original components defining how relevant a particular feature is. Such elements could include the related importance of the intensity of a set of pixels in a two-dimensional image or historical price values for a cross-section of financial instruments.
Defining Vectors and Few Operations in Python:
<class 'list'>[1, 2, 3, 4, 5, 6][5 7 9]<class 'numpy.ndarray'>[-3  6 -3]
Matrices are rectangular arrays consisting of numbers and are an example of 2nd-order tensors. If m and n are positive integers, that is m, n ∈ N then the m×n matrix contains m*n numbers, with m rows and n columns.
The full m×n matrix can be written as:
It is often useful to abbreviate the full matrix component display into the following expression:
In Python, We use numpy library which helps us in creating ndimensional arrays. Which are basically matrices, we use matrix method and pass in the lists and thereby defining a matrix.
$python
>>> import numpy as np>>> x = np.matrix([[1,2],[2,3]])>>> xmatrix([[1, 2],        [2, 3]])>>> a = x.mean(0)>>> amatrix([[1.5, 2.5]])>>> # Finding the mean with 1 with the matrix x.>>> z = x.mean(1)>>> zmatrix([[ 1.5],        [ 2.5]])>>> z.shape(2, 1)>>> y = x - zmatrix([[-0.5,  0.5],        [-0.5,  0.5]])>>> print(type(z))<class 'numpy.matrixlib.defmatrix.matrix'>
Defining Matrices and Few Operations in Python:
Matrices can be added to scalars, vectors and other matrices. Each of these operations has a precise definition. These techniques are used frequently in machine learning and deep learning so it is worth familiarising yourself with them.
C = A + B (Shape of A and B should be equal)
The methods shape return the shape of the matrix, and add takes in two arguments and returns the sum of those matrices. If the shape of the matrices is not same it throws an error saying, addition not possible.
Adds the given scalar to all the elements in the given matrix.
Multiplies the given scalar to all the elements in the given matrix.
A of shape (m x n) and B of shape (n x p) multiplied gives C of shape (m x p)
With transposition you can convert a row vector to a column vector and vice versa:
A=[aij]mxn
AT=[aji]n×m
The more general entity of a tensor encapsulates the scalar, vector and the matrix. It is sometimes necessary — both in the physical sciences and machine learning — to make use of tensors with order that exceeds two.
We use Python libraries like tensorflow or PyTorch in order to declare tensors, rather than nesting matrices.
To define a simple tensor in PyTorch
Few Arithmetic Operations on Tensors in Python
For more documentation regarding tensors and PyTorch click here.
Important Links
To get started with deep learning in python:
towardsdatascience.com
towardsdatascience.com
Thanks for reading. If you found this story helpful, please click the below 👏 to spread the love.
Special Thanks to Samhita Alla for her contributions towards the article.