Why is Linear Algebra Taught So Badly? | by Callum Ballard | Towards Data Science

Here’s a quick maths question for you. Raise your hand if you can multiply these two matrices together:
Congratulations if you said:
Now keep your hand up if you know why. And by ‘why’, I don’t mean because:
Whilst mathematically true, this formula is more descriptive of ‘how’ than it is of ‘why’. In and of itself, this formula is pretty much devoid of intuition.
And yet, this is how matrix multiplication is nearly always taught. Memorise the formula. Use it in an exam. Profit? This was certainly my experience, both when first learning linear algebra as a 16 year old, and then while attending an ostensibly world-leading university to study for my mathematics bachelors.
Here’s another question for you: what is the determinant of the following matrix?
Congratulations if you said 2. But you might be able to guess where I’m going with this. We know that for a 2x2 matrix, the determinant is given by the following formula:
But why? And for that matter, what even is the determinant? We are taught that it has a couple of useful properties (for example, a determinant of 0 is a red flag if you’re trying to solve systems of linear equations with row reduction). But in the two compulsory modules of linear algebra I took at university (an institution whose reputation, I suspect, relies on its excellence in research, rather than teaching), not once was the determinant of a matrix contextualised or explained at anything beyond a surface level.
This slightly utilitarian attitude to teaching linear algebra is clearly problematic. Mathematics is a discipline that relies on an ‘incremental’ learning — gaining new knowledge often requires you to build upon what you already know. If your theoretical foundations are predicated on rote learning and plugging numbers into formulae, without a deeper appreciation and understanding of what’s actually going on, then they tend to fall down under the weight of something as heavy as, let’s say, machine learning.
At this point, I’ll mention that this blog was heavily inspired by a series of videos made by Grant Sanderson (a.k.a. 3Blue1Brown). For those unfamiliar with his work, Sanderson creates really nicely animated videos in a way that makes complicated mathematical subjects accessible to the educated layman (his videos explaining neural networks and cryptocurrency are well worth your time).
At its core, Sanderson’s ‘Essence of Linear Algebra’ series seeks to introduce, motivate, and conceptualise many of the basic ideas around linear algebra in terms of linear transformations and their associated visualisations. As it turns out, this is a really helpful way to get your head around many of the core fundamentals.
“The goal here is not to try to teach everything, it’s that you come away with a strong intuition... and that those intuitions make any future learning that you do more fruitful...” — Grant Sanderson
Before answering this question, let’s take a step back and think about what a linear transformation is. To keep things simple, let’s keep things in two dimensions (though the following applies to higher dimensions as well).
A linear transformation is a way of changing the shape of a ‘space’ (in this case, the 2D plane), in such a way that:
Keeps parallel lines parallel
Maintains an equal distance between parallel lines that were equally spaced to begin with
Leaves the origin at the origin
Broadly speaking, this gives us three different types of linear transformations that we could do:
Rotations
Scaling (reducing or increasing the space between parallel lines). Note — this also accounts for reflections in either of the x or y axes, these simply have negative scale factors.
And sheers (note how this preserves equal distance between parallel lines)
Any combination of these three types of operation would also be a linear transformation on its own terms (more on this idea later).
Whilst these illustrations above are created to demonstrate the fact that a linear transformation affects the entirety of 2D space, it transpires that we can describe them in terms of what they do to the two ‘unit vectors’, called î (i-hat) and ĵ (j-hat) respectively.
There’s more detail that one can go into, but in essence, this is driven by the fact that you can reach any point on the 2D plane with a linear combination of î and ĵ (for example, a vector v [3, -2] will simply be equivalent to 3 lots of î plus -2 lots of ĵ).
Suppose we want to think about a linear transformation that rotates everything counter-clockwise by a quarter turn. What becomes of our vector, v? It turns out we can describe what happens to v purely in terms of what happens to î and ĵ.
Recall that v, [3, -2], was given as 3 lots of î plus -2 lots of ĵ. Well, it turns out that transformed v is equivalent to 3 lots of transformed î plus -2 lots of transformed ĵ.
In Sanderson’s words, this line:
transformed_v = 3*[0,1] + (-2)*[-1,0]
is “where all the intuition is”.
In particular, we can take the vectors of ‘transformed î’ and ‘transformed ĵ’, put them together to form a 2x2 matrix, refer back to this more ‘intuitive’ view of what happens to v, and all of a sudden we’ve justified vector multiplication.
So what about the multiplication of two 2x2 matrices that we examined earlier?
We’ve just demonstrated that a 2x2 matrix will necessarily represent some kind of linear transformation in 2D space. In particular, for a given matrix [[a, b], [c, d]], the vectors [a, c] and [b, d] represent the coordinates of ‘transformed î’ and ‘transformed ĵ’ respectively.
Suppose we want to make two linear transformations one after the other. For illustration, let’s suppose we perform the counter-clockwise quarter-turn that we looked at before, and follow this with a reflection in the x-axis. These two transformations can be both be represented by 2x2 matrices. We already know the matrix that represents the rotation, so what about the reflection? We can use the same technique as before — watch what happens to î and ĵ.
Of course, î remains the same, and ĵ becomes negative. We’ve previously shown that we can put these ‘transformed î’ and ‘transformed ĵ’ vectors together to form the matrix representative of the overall transformation.
So how can we think about the case when two transformations are performed one after another; first the rotation, then the reflection? We can approach this in the same way as before — watch what happens to î and ĵ.
We know from before that the rotation takes î from [1, 0] to [0, 1]. If we then want to apply the reflection to this ‘transformed î’, we simply need to multiply the matrix representing this reflection by the vector representing ‘transformed î’, [0, 1] (recall — we’ve already shown that multiplying a transformation matrix by a vector describes what happens to that vector when transformed).
Of course, we now need to observe what happens to ĵ, using the same reasoning.
Now that we know what happens to î and ĵ after they go through the rotation and reflection transformations one after another, we can put these two vectors together to describe the cumulative effect as a single matrix.
Which looks an awful lot like a representation of our standard formula for matrix multiplication. Of course, you could try this thought experiment with any sequence of linear transformations. By following what happens to î and ĵ, you can effectively.
It’s worth noting that, by thinking about matrix multiplication in terms of sequential linear transformations, it becomes quite easy to justify our standard rules of matrix multiplication. For three different matrices A, B, and C, think about why the following properties hold:
A*B ≠ B*A
A*(B*C) = (A*B)*C
A*(B+C) = A*B + A*C
Towards the start of the blog, I showed how to mechanically calculate the determinant. I then asked why the formula holds (and, for that matter, what the determinant even is). I cover this in another blog, but, spoiler alert, the determinant of a 2x2 matrix simply represents the scale by which a given area in 2D space increases or decreases following the transformation given by that matrix.
Not unreasonably, the YouTube comments of Sanderson’s video on the determinant are filled with people who are baffled as to why this isn’t typically mentioned when taught, since it’s such an intuitive concept. I can’t blame them.
Thanks for reading all the way to the end of the blog! I’d love to hear any comments about the above analysis, or any of the concepts that the piece touches on. Feel free to leave a message below, or reach out to me through LinkedIn.