ML Fundamentals in Javascript
Tags: javascript, machinelearning, datascience, node
Published: 2021-07-26 22:39:27

![maxresdefault](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ei15xbx4iqip1yns4621.jpeg)

I recently have been exploring the field of [Machine  Learning](https://www.coursera.org/learn/machine-learning); in all honesty, I have had to relearn almost all of my math fundamentals. It has been a while since college and ML is based on a lot of Linear Algebra. In this blog, I plan to curate the fundamentals alongside my implementation of them in Javascript. I know Python's [NumPy](https://numpy.org/) library is the industry standard, but I have always been curious about how the basic operations would translate in Javascript. Full disclosure, I am just learning this myself, so if there are any mistakes along the way, please help me out! 

Topics include:
 - Basics of Linear Algebra 
 - Matrix Operations
 - Types of Matrices
 - Complex Mathematical Expressions

I have also compiled a NodeJS App that contains a working version of all the examples in this blog: [ML Fundamentals 1](https://github.com/mrinasugosh/ml-fundamentals-1)

Let's get started~!

## Basics of Linear Algebra

Linear Algebra is a subset of algebra that deals with scalars, vectors, and matrices. In the simplest terms here's what they are:

### Matrix
I like to think of a matrix as an array or an array of arrays in programming. Where `m` is the number of rows and `n` is the number of columns of a `matrix[m][n]`. If we were coding, it would look something like this:
```javascript
const matrix = [
  [0, 1],
  [2, 3],
  [3, 4]
];
```
### Vector
A vector is simply a type of matrix and specifically has only one column. Therefore, it would look something like this:
```javascript
const vector = [
  [0],
  [1],
  [2],
];
```

### Scalar
Probably, the simplest mathematical object in all of Linear Algebra is a scalar. It is just a number that's often used as a multiplier.
```javascript
const scalar = 2;
```

Most matrices and vectors can be expressed with arrays in Javascript or any language. But what about matrices that are 3D or 4D or XD? Typically, most Linear Algebra courses state that a matrix can have `x` dimensions where `x` is a number greater than 0. This is where we begin to use the idea of tensors in programming where vectors are essentially ranked to correspond to the various dimensions. In fact, Javascript has a framework called [Tensorflow.js](https://www.tensorflow.org/js/guide/tensors_operations) that defines and runs computations using tensors. I will dive more into that in a future blog. For now, let's get back to the basics.

## Matrix Operations
When you think about Matrix Operations, typically my mind jumps to loops. But using loops to iterate a matrix can start to become real ugly real quick. That's when I discovered the features of the [Math.js](https://mathjs.org/) library that provides JS and Node.js projects with powerful, optimized computations. This means that, instead of defining a matrix as an array of arrays, you could simply define a `matrix` and it would look something like this with Math.js:
```javascript
const matrix = math.matrix([[0,1], [2,3], [4,5]])
```
Some useful methods include `size()` and `valueOf()`:
```javascript
math.size(matrix) //returns dimensions of matrix
//[3,2]
matrix.valueOf() //returns string representation of the matrix
//[[0,1], [2,3], [4,5]]
```

Let's explore examples of four main matrix operations such as addition, subtraction, multiplication, and division: 
### Matrix Addition
```javascript
matA = math.matrix([[0, 1], [2, 3], [4, -5]]);
matB = math.matrix([[1, -1], [-2, 4], [-7, 4]]);

const matAdd = math.add(matA, matB);

console.log(matAdd.valueOf());
//[ [ 1, 0 ], [ 0, 7 ], [ -3, -1 ] ]
```
### Matrix Subtraction
```javascript
matA = math.matrix([[0, 1], [2, 3], [4, -5]]);
matB = math.matrix([[1, -1], [-2, 4], [-7, 4]]);

const matSub = math.subtract(matA, matB);

console.log(matSub.valueOf());
//[ [ -1, 2 ], [ 4, -1 ], [ 11, -9 ] ]
```

### Matrix Multiplication
This is when things start to get interesting. Before I jump to coding examples, its important to understand these two properties of Matrix Multiplication: commutative and associative.

#### Commutative
Matrix multiplication is not commutative which simply means A x B != B x A. Let's test this out and check with MathJS's built in `equal` comparator:
```javascript
matA = math.matrix([[0, 1], [2, 3]]);
matB = math.matrix([[2, 4], [6, 2]]);
 
const matAxB = math.multiply(matA, matB);
const matBxA = math.multiply(matB, matA);
 
console.log(math.equal(matAxB.valueOf(), matBxA.valueOf()));
// false
```

#### Associative
Matrix multiplication is associative which simply translates to A x (B x C) == (A x B) x C. Let's test this out as well:
```javascript

const matA = math.matrix([[0, 1], [2, 3], [4, 5]]);
const matB = math.matrix([[2, 4], [6, 2]]);
const matC = math.matrix([[5, 2], [2, -2]]);

const matAxB_C = math.multiply(math.multiply(matA, matB), matC);
const matA_BxC = math.multiply(matA, math.multiply(matB, matC));

console.log(math.equal(matAxB_C.valueOf(), matA_BxC.valueOf()));
// true
```

Its really important to also note that in the case of math.js, the product of a matrix is not just a new matrix containing the product of the individual matrices - element-wise product (or Hardamard product). In fact, the product we see is a matrix product operation. 

An example of a element-wise product is via matrix-scalar multiplication, A x 3, which is performed as such:
```javascript
matA = math.matrix([[0, 1], [2, 3], [4, -5]]);
const scalar = 3;
const matAx3 = math.multiply(matA, scalar);

console.log(matAx3.valueOf());
//[ [ 0, 3 ], [ 6, 9 ], [ 12, -15 ] ]
```

Of course, since a vector is just a matrix it is also possible to perform a matrix-vector multiplication:
```javascript
matA = math.matrix([[0, 1], [2, 3], [4, 5]]);
matB = math.matrix([[2], [1]]);

const matAxvB = math.multiply(matA, matB);

console.log(matAxvB.valueOf());
//[ [ 1 ], [ 7 ], [ 13 ] ]
```

### Matrix Division
Matrix division is also possible to implement in Javascript. Note that in mathematically there is no "Matrix Divsion" as its defined as such:
A/B = AB<sup>-1</sup>
However to save us from thinking about divisions and inverses we can implement matdix.divide() in js:
```javascript
matA = math.matrix([[0, 2], [2, 4], [4, 6]]);
matB = math.matrix([[2, 1], [2, 2]]);

const matAB = math.divide(matA, matB);

console.log(matAB.valueOf());
// [ [ -2, 2 ], [ -2, 3 ], [ -2, 4 ] ]
```

After all, dealing with matrices in math.js isn't that difficult anymore! But you have to know the dimensions of each matrix in your operation, because not every matrix "operates" on another matrix!

## Types of Matrices
There are a couple of matrix types in Linear Algebra that is important to understand as well. 

###Identity Matrix
The Identity (I) Matrix with the dimension i * j is defined as i-dimensional matrix where i == j. They are special and used widely because they are commutative; this means A x I === I x A. Here's an example of an Identity Matrix:
```javascript
const matrix = [
  [1, 0, 0],
  [0, 1, 0],
  [0, 0, 1],
];
```

In math.js you can use the `eye(i)` method to quickly generate an identity matrix with dimension i:
```javascript
const matI_3 = math.eye(3);
console.log(matA.valueOf());
// [ [ 1, 0, 0 ], [ 0, 1, 0 ], [ 0, 0, 1 ] ]
```

###Transpose Matrix
In a transpose matrix the dimensions are flipped. Simply stated, the rows become columns and the columns become rows Here's an example of taking a vector and transposing it. The transposed matrix is referred to as a row vector:
```javascript
const matV = math.matrix([[0], [1], [2]]);
const matV_T = math.transpose(matV);

console.log(matV_T.valueOf());
// [ [ 0, 1, 2 ] ]
```

###Inverse Matrix
Of course, its important to discuss the inverse matrix. What's interesting about this is that matrices can have an inverse A<sup>-1</sup> but not all matrices (specifically singular or degenerate) have one. The formula to find the inverse of a matrix: A(A<sup>-1</sup>) = (A<sup>-1</sup>)A = I.
But Math.js gives us the inverse operation for free as `math.inv()` check it out:
```javascript
matA = math.matrix([[0, 1], [2, 3]]);
const matA_1 = math.inv(matA);

console.log(matA_1.valueOf());
// [ [ -1.5, 0.5 ], [ 1, -0 ] ]
```

## Complex Math Expressions
At some point, using built in math.js the proposed way doesn't scale anymore. Let's be honest, ML gets complicated real quick. Especially when you are trying to perform operations with multiple features and use a multivariate linear regression with gradient descent aka a function with multiple inputs. Take the following function of theta as an example:
```javascript
theta = theta - ALPHA / m * ((X * theta - y)^T * X)^T
```
If you try to represent this out of the box in Javascript you will get a mess like this:
```javascript
theta = math.subtract(
  theta,
  math.multiply(
    (ALPHA / m),
    math.transpose(
      math.multiply(
        math.transpose(
          math.subtract(
            math.multiply(
              X,
              theta
            ),
            y
          )
        ),
        X
      )
    )
  )
);
```
What a mess right? Luckily there is a concise and readable way to evaluate it using the `eval` function:
```javascript
theta = math.eval(`theta - ALPHA / m * ((X * theta - y)' * X)'`, {
  theta,
  ALPHA,
  m,
  X,
  y,
});
```

Shocked that all of this is possible with Javascript? You are not alone! Regardless of the coding language you use today, you will surely find a powerful math library such as MathJS to enable matrix operations and other complex operations to get you started with ML fundamentals! I hope this was helpful.

If you want to experiment with the Math.js library on your own, do checkout the Github repository with all of these examples compiled in a NodeJS app:
https://github.com/mrinasugosh/ml-fundamentals-1

==== Follow me on Social Media(`@mrinasugosh`) ====
Dev.to: @mrinasugosh 
Github: [@mrinasugosh](https://github.com/mrinasugosh)
Twitter: [@mrinasugosh](https://twitter.com/mrinasugosh)
LinkedIn: [@mrinasugosh](https://www.linkedin.com/in/mrinasugosh/)