Linear Algebra Essentials with Numpy (part 1) | by Dario Radečić | Towards Data Science

Ah, math. You can’t avoid it forever. You can try, and then try harder, but sooner or later some basic intuition behind it will be needed, provided that you are serious in your endeavors to advance your career in data science.
When it comes to linear algebra, I really like this quote:
If Data Science was Batman, Linear Algebra would be Robin.[1]
It neatly captures the essence in a non-technical way.
Why should I study linear algebra?
Great question. Simply put, if you understand it, you will be able to develop a better intuition for machine learning and deep learning algorithms and you won’t treat them as black boxes. Furthermore, you would also be capable of developing algorithms from scratch and making your own variations of them.
Not only that, but you will also be considered as a cool kid, like these two:
So awesome.
Why should I read this post?
This post certainly won't teach you every nitty-gritty detail about the topic, there are a plethora of books which cover those. The post also won’t dive into proofs, again, read a book from some math or tech faculty if you are into that. This post will, however, provide you with an essential intuition behind linear algebra (and some light calculations) in 14 different areas.
Yes, you’ve read that right. I’m planning to cover 14 different topics in two articles. It will take you some time to read it (and for me to write it), but if you are new to this I would strongly advise reading 2–3 topics per day at most, and then searching the web for more practice problems.
How is the article structured?
So each of the topics is divided into 3 parts:
Theoretical explanationExample (by hand calculations)Implementation in Python (with Numpy)
Theoretical explanation
Example (by hand calculations)
Implementation in Python (with Numpy)
I won’t lie to you — it will be a lot of work. This first part will cover Vectors, and here’s the topic list:
AdditionScalar MultiplicationDot ProductNormUnit VectorAngle Between Vectors
Addition
Scalar Multiplication
Dot Product
Norm
Unit Vector
Angle Between Vectors
Now when the introduction is over, let’s not postpone the inevitable. Grab a cup of coffee (or something stronger) and come with me to the amazing world of linear algebra.
Yeah, no point in starting with vector addition if you have no idea what a vector is. For starters, think of a vector as an arrow in space. You need to keep in mind two quantities here:
Direction
Magnitude
By direction, I mean to where in space does ‘arrow’ point, and the magnitude tells you how far you should go in that direction. If you only have magnitude, but not direction, then you are talking about scalars. Once you give the scalar some direction, it becomes a vector.
I said that you should try to imagine vectors as arrows — that’s a perfect terminology because arrow has both a clear direction and a clear magnitude (length).
Vectors are most commonly denoted with a lowercase letter and arrow above, pointing to the right:
Simple, right?Let’s jump into some vector operations now, I know how impatient you must be (*laughs*).
Let’s dive into the first vector operation, which is vector addition (subtraction works the same way, only you would use minus sign instead of the plus, obviously).
There really isn’t much to say about this basic operation, only that it is performed by adding the corresponding components. It helps to drive the point home if you think about what happens to the vector, and not just in term of pure number addition.
Take a look at this photo from Wikipedia Commons:
In a nutshell, to add vector b to the vector a (to get a+b), draw the vector a from the origin, and then draw the vector b from the tip of vector a. Now to get a+b, you only need to connect the origin to the tip of vector b.
Might seem like overkill if you hear this for the first time, but just take a pen and paper and draw this, you’ll immediately get the gist of it.
And here’s the general formula for vector addition:
Now I will do a simple example by hand and later will implement this same example in Python.
Once again, take a piece of paper and draw this — it’s pretty intuitive.
Implementation in Python:
v = np.array([3, 7])u = np.array([2, 2])print(v + u)
As you can see, Numpy allows you to use the addition operator, pretty neat!
Let me quote myself from earlier in this article:
If you only have magnitude, but not direction, then you are talking about scalars.
Which essentially means that a scalar is a single number which can only change the magnitude of the vector, but not it’s direction. What this actually means is that the vector multiplied by any scalar will remain on the same ‘line’, it will have the same direction, only its length will be altered.
Awesome, let’s see the formula for scalar multiplication:
Above, n represents any number. Let’s see an example. Scalar n will be 2, which means that every component of the vector will be multiplied by 2.
Implementation in Python:
v = np.array([3, 7])print(2 * v)
Awesome, let’s go to the next topic!
To calculate the dot product of two vectors, you first need to multiply the corresponding elements (x1 by y1, x2 by y2, and so on) and then sum the product terms.
This concept is actually much easier to under when you see the general formula, so here is it:
Now I will write down a basic example with two vectors in 2-dimensional space:
Implementation in Python:
v = np.array([3, 7])u = np.array([2, 2])print(v.dot(u))
Yeah, that’s pretty much it about the dot product — not really much to talk about.
Norm is just another term for length or magnitude of a vector and is denoted with double pipes (||) on each side. It is defined as a square root of the sum of squares for each component of a vector, as you will see in the formula below.
The computation is a 3 step process:
Square each componentSum all squaresTake the square root
Square each component
Sum all squares
Take the square root
As you can imagine, the formula is stupidly simple:
And here’s a practical example of a vector in 2-dimensional space:
Implementation in Python:
v = np.array([3, 2, 7])print(np.linalg.norm(v))
The main reason you are calculating unit vectors is that you care only about the direction, not about the magnitude. This process of normalization involves stripping away the magnitude so it doesn’t skew other calculations.[2]
A unit vector is most often denoted with a hat symbol (^) and is calculated by computing the norm and then dividing each component of the vector with the norm.
Without further ado, here’s the formula:
As an example, I’ll calculate the unit vector from an arbitrary 3-dimensional vector:
You don’t have to do the last step — to divide by square root — but it makes the final result much more appealing.
Implementation in Python will be a bit different here. There is no built-in function in Numpy for calculating unit vectors (at least I don’t know about it), but because you know the formula, the computation process is trivial. I’ve declared a function which will take a vector as an input, and then return that vector divided by its norm:
def unit_vector(v):    return v / np.linalg.norm(v)u = np.array([3, 6, 4])print(unit_vector(u))
This is probably the topic in linear algebra I’m most excited about. And the reason is that I use it here and there in my daily job for some cool stuff.
Angle calculation between vectors has a lot of practical applications. It’s widely used in NLP (Natural Language Processing) when looking for strings that are very similar. For example, when TF-IDF is used on some textual data, each input (try thinking of input as an individual email text) is converted into a vector.
By doing so you will be sure that all vectors are of the same dimensions, and if the input contains a certain word at a certain position, the value at the corresponding component of the vector will be 1, and 0 otherwise. If you then calculate the angle between vectors, the ones where angle in the smallest will be more similar (in the email example think of it like two individual emails are on the same topic).
Hope you can see the potential use-cases in your project, if not don’t worry about it, you will see it eventually.
Here’s the formula for calculating the angle. Nothing fancy, you take the dot product of two vectors and divide that by the product of norms:
And here’s a really simple example of two arbitrary 3-dimensional vectors:
As with the unit vectors, Numpy doesn’t have a built-in function for angle calculation. But hey, you know the formula, so how hard can it be?The only ‘trick’ here is to convert radians to degrees because Numpy will return the result in radians by default. Everything else is as simple as you would expect it to be.
Here’s the code:
def angle_between(v1, v2):    dot_pr = v1.dot(v2)    norms = np.linalg.norm(v1) * np.linalg.norm(v2)     return np.rad2deg(np.arccos(dot_pr / norms))v = np.array([1, 4, 5])u = np.array([2, 1, 5])print(angle_between(v, u))
And that pretty much ends the talk about vectors. Wasn’t that bad, was it?
That’s it for some basic essence of linear algebra for data science, at least when it comes to vectors. The next article will covert matrices, and will be a bit longer than this one.
While you are waiting I would highly advise you to check this amazing channel on YouTube, this playlist in particular. It goes really deep into building intuition behind basic and advanced concepts in linear algebra. This channel is my honest recommendation if you know how to calculate stuff, but don’t really know why you are doing the calculations.
Thanks for reading.
towardsdatascience.com
Loved the article? Become a Medium member to continue learning without limits. I’ll receive a portion of your membership fee if you use the following link, with no extra cost to you.