Data Science: Linear Algebra with Python
Tags: datascience, python, mathematics, beginners
Published: 2022-11-11 11:30:00

---
summary: Summary of the post
tags : datascience, python, mathematics, beginner
alias: [Linear Algebra]
status: Writing
creation_date: Wednesday, 9th November 2022, 18:19:50
Progress: 0
Target: 100
---

## Linear Algebra
Linear algebra is the branch of mathematics that deals with vector spaces. It contains concept of vector, matrix etc. Linear algebra is widely used by data scientists (frequently implicitly, and not infrequently by people who don’t understand it). It wouldn’t be a bad idea to read a textbook.


### Vector 
Vectors, in general, are objects that can be added together (to form new vectors) and that can be multiplied by scalars (numbers) to create new vectors Vectors are concretely (for us) points in some finite-dimensional space. Although you may not think of your data as vectors, they are a good way to represent numerical data. For instance, if you have a large number of people's heights, weights, and ages, You can think of your data as three-dimensional vectors (height, weight, age). If If you're teaching a class with four exams, you can treat student grades as four-point scale. vectors with dimensions (exam1, exam2, exam3, exam4). The most basic method is to represent vectors as lists of numbers. A list of three numbers corresponds to a vector in three-dimensional space, and vice versa:

```python
weight = [
		  10, # kg
		  20, # kg
		  30  # Kg
	  ]

length = [
		  15, # meter
		  25  # Meter
	]
```

One issue with this approach is that we will want to perform arithmetic on vectors. Because Python lists aren't vectors (and thus don't support vector operations), arithmetic), we'll have to create our own arithmetic tools. So let's start there. To begin, we'll frequently need to combine two vectors. Vectors are added component by component. This means that if two vectors v and w have the same length, their sum is the vector itself. whose first element is {% katex inline %}v[0] + w[0]{% endkatex %}, second element is {% katex inline %}v[1] + w[1]{% endkatex %}, and so on on. (If they're not the same length, we can't combine them.) 

Adding {% katex inline %}[a, b]{% endkatex %} and {% katex inline %}[c,d]{% endkatex %} will result in {% katex inline %}[a+c, b+d]{% endkatex %}.


![Vector in Graph](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/vmi3m6md3wh37dq3m6tb.png)



We can easily implement this by zip-ing the vectors together and using a list comprehension to add the corresponding elements:  
```python
def vector_add(v, w):  
	"""adds corresponding elements"""  
	return [v_i + w_i  
		for v_i, w_i in zip(v, w)]
```

Similarly, to subtract two vectors we just subtract corresponding elements:
```python
def vector_subtract(v, w):  
	"""subtracts corresponding elements"""  
	return [v_i - w_i  
		for v_i, w_i in zip(v, w)]
```

We’ll also need to be able to multiply a vector by a scalar, which we do simply by multiplying each element of the vector by that number:

```python
def scalar_multiply(c, v):  
	"""c is a number, v is a vector"""  
	return [c * v_i for v_i in v]
```

We also need ***dot product*** of two vectors. The dot product of two vectors is the sum of their component wise product:

{% katex %}V{\cdot}W = Vx {\cdot} Wx + Vy{\cdot}Wy
{% endkatex %}

```python
def dot(v, w):
	return sum(v_i * w_i for v_i, w_i in zip(v, w))
```

Another value which can be useful is the ***Magnitude (or length)*** of the vector which is square root of sum of square of values.

{% katex %} mag(V) = \sqrt{(Vx)^2+(Vy)^2} 
{% endkatex %}

Distance between two vectors can be calculated by


{% katex %} distance(V,W) = \sqrt{(vx-Wx)^2 + (Vy-Wy)^2} 
{% endkatex %}
```python
import math

def sum_of_squares(v):  
	"""v_1 * v_1 + ... + v_n * v_n"""  
	return dot(v, v)

def magnitude(v):  
	return math.sqrt(sum_of_squares(v))

def squared_distance(v, w):  
	"""(v_1 - w_1) ** 2 + ... + (v_n - w_n) ** 2"""  
	return sum_of_squares(vector_subtract(v, w))

def distance(v, w):  
	return math.sqrt(squared_distance(v, w))
```


### Matrices

A matrix is a two-dimensional number collection. Matrixes will be represented as lists of lists, with each inner list the same size and representing a row of the matrix. If A is a matrix, then {% katex inline %}A[i][j]{% endkatex %} is the element in the *ith* row and the *jth* column. Per mathematical convention, we will typically use capital letters to represent matrices.

![Matrix](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0qys7kzkwdbp1ptrvd5r.png)


Same as vectors we can represent matrix in terms of 2D list. 
```python
A = [
	 [1,2,3],
	 [4,5,6]
]
```


![Note](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a9yfwudw7iwb5c773j8d.png)


The matrix A contains `len(A)` rows and `len(A[0])` columns, which we take to be its shape given this list-of-lists representation:
```python
def shape(A):
	n_rows = len(A)
	n_columns = len(A[0])
	return n_rows, n_columns
```

We shall refer to a matrix as a {% katex inline %}n \times k{% endkatex %} matrix if it contains n rows and k columns. Each row of a n by k matrix can be thought of as a vector of length k, and each column as a vector of length n:
```python
def get_row(A, i):  
	return A[i] # A[i] is already the ith row  
	
def get_column(A, j):  
	return [A_i[j] # jth element of row A_i  
			for A_i in A] # for each row A_i
```

If a diagonal of a matrix contains only 1s then its called *Diagonal Matrix*.

![Diagonal Matrix](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/b3ws9j4h6vnbv063v4me.png)

We can use matrix to represent relations between different entities ex:

```python
friendships = [(0, 1), (0, 2), (1, 2), (1, 3), (2, 3), (3, 4),  (4, 5), (5, 6), (5, 7), (6, 8), (7, 8), (8, 9)]
############# Which Can Also be Represnted as ###############

friendships = [ [0, 1, 1, 0, 0, 0, 0, 0, 0, 0], # user 0  
				[1, 0, 1, 1, 0, 0, 0, 0, 0, 0], # user 1  
				[1, 1, 0, 1, 0, 0, 0, 0, 0, 0], # user 2  
				[0, 1, 1, 0, 1, 0, 0, 0, 0, 0], # user 3  
				[0, 0, 0, 1, 0, 1, 0, 0, 0, 0], # user 4  
				[0, 0, 0, 0, 1, 0, 1, 1, 0, 0], # user 5  
				[0, 0, 0, 0, 0, 1, 0, 0, 1, 0], # user 6  
				[0, 0, 0, 0, 0, 1, 0, 0, 1, 0], # user 7  
				[0, 0, 0, 0, 0, 0, 1, 1, 0, 1], # user 8  
				[0, 0, 0, 0, 0, 0, 0, 0, 1, 0]] # user 9

```

If there are very few connections, this is a much more inefficient representation, since you end up having to store a lot of zeroes. However, with the matrix representation it is  much  quicker  to  check  whether two nodes are connected, you just have to do a matrix lookup instead of (potentially) inspecting every edge:
```python
friendships[0][2] == 1 # True, 0 and 2 are friends  
friendships[0][8] == 1 # False, 0 and 8 are not friends

friends_of_five = [i # only need  
					for i, is_friend in enumerate(friendships[5]) # to look at  
					if is_friend] # one row
```

![Note1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ru5ofmsuqf3kke5m3fuy.png)


#### Resource 
- https://www.khanacademy.org/math/linear-algebra
- https://www.youtube.com/playlist?list=PLZHQObOWTQDPD3MizzM2xVFitgF8hE_ab
- https://web.stanford.edu/~boyd/vmls/
- http://mitran-lab.amath.unc.edu/courses/MATH347DS/textbook.pdf
- https://fong.cs.wmich.edu/modules/LinearAlgebraPrimerConcepts.pdf

## References

This article is highly influenced by the book Data Science from Scratch by Joel Grus, this is a fantastic book to read for people getting started with data science. If you don't like reading book then save the series as there is more to come.