Lets Look At Magic LookAt Matrices
Tags: javascript, webdev, graphics, tutorial
Published: 2022-04-25 13:12:03

## The Magic of the LookAt Matrix

I find math to sometimes be hard, sometimes fun, sometimes magical, and sometimes hard-fun-magical. **Linear Algebra**¹  is the mathematics behind a lot of fun tech, such as VR, AR, graphics, machine learning, data science buzz words etc.

I’ve been on a WebGPU 3D computer graphics kick lately ([last week I wrote about making a triangle in WebGPU](https://dev.to/carmencincotti/drawing-a-triangle-with-webgpu-1mk3)). When it came time to implement a camera, I figured I could just instantiate some sort of camera object and move on.

I quickly learned that **the camera in 3D graphics does not exist.** _It’s all smoke and mirrors_. We give the illusion that it does exist through the magic of linear algebra. Let’s see what I mean by taking _a look at_ (ha!) **the LookAt matrix.**

**The** **LookAt Matrix** is a great exercise in linear algebra. It encompasses the usage of the **dot product** and **cross product**. It involves **vectors**. It involves **matrices.**

Anyway, it turned out to be a fun learning opportunity to really drive some key math concepts down. I’d like to share this knowledge with you.

## Some Theory

**The LookAt matrix** is a matrix that transforms something to look at a point in space. Let’s keep our discussion limited to the application of **the LookAt Matrix** to _cameras_.

Namely, we can use **the LookAt matrix** to transform the positions of the objects within the 3D scene to _give the illusion_ that they are being viewed from the lens of the camera.

Let’s take as an example a 3D scene containing a camera and a 3D ball — and we apply a LookAt matrix to the camera **that transforms it** to view a red ball from a certain position in 3D, we might expect to see something like this:

![A camera looking at a red ball](https://cdn-images-1.medium.com/max/800/1*O_19UJVrvcz4lLoFzZNv2Q.png)

A camera looking at a red ball after applying **the LookAt matrix**

Additionally, we might _instead_ want to view the world  **from the lens of our camera** _(which is a much more common application)_… so we’ll transform the ball / plane instead _to give the illusion_ that we’re viewing it from the perspective of the camera:

![A red ball from the perspective of a camera](https://cdn-images-1.medium.com/max/800/1*-UUltrOesmv8XMPZyBGEYQ.png)

What we might see if we looked through the camera at our scene.

Over the next few sections, we’ll see how we can calculate both of these views. One where we move the camera (and the world remains constant), and where the camera remains static (at origin, looking down the negative z-axis) and the world moves instead.

## Some Code

_Poof!_ Here’s the magic trick in full (I prototyped it for a WebGPU app, so it’s in Javascript).

{% embed https://gist.github.com/ccincotti3/a21f2bbcbd9e9226595eda15b0a91ee1.js %}

⚠️ _Take note that I am actually calculating the LookAt Matrix that we would use to move scene objects in relation to the camera (explanation at the end of the article)._

## Some Calculations

Solving for this matrix is really attempting to model this camera through the careful calculation of it’s coordinate system in relation to world space. Or more simply put, we need to find the vectors `forwardVector` , `upVector` , and `rightVector` of the camera in relation to the ball’s coordinate system.

_This feels a bit like sorcery._ We can take _very_ minimal information and, in the end, come up with an entire matrix representing an orthonormal coordinate system like shown in the image:

![A 3D scene showing a camera looking at a red ball. There is a 3D gizmo illustrating the up, right, and forward vectors.](https://cdn-images-1.medium.com/max/800/1*4EY6XWVxm1-Kia0lSVygAQ.png)

A representation of a 3D scene where a camera is looking at a red ball. Notice how the forward vector points towards the camera.

We’ll start our journey through this _Magical Math forest,_ knowing that our camera here has a certain position in space, as does our red ball. _We’ll also assume the conventions of a right-handed coordinate system,_ so we’re all on the same page.

### Step One: Calculate the Forward Axis direction

This is actually very doable. Given our camera position and red ball position, we can calculate the direction of the `forwardVector` through vector subtraction:

_forwardVector = normalize(cameraPosition — redBallPosition)_

⚠️ _Remember to_ **_normalize_** _the result of the vector subtraction, since we want the directional vector, which is a unit vector._

Step one, done. Let’s hop on our broomsticks to Step Two.

### Step Two: Calculate the Right Axis direction

This step involves _quite a bit of_ _witchcraft_, and I’m not a huge fan of the handwavy-ness of the steps that follow but, hey, _math is magic._

Let’s list out some things we know to be true first before arriving at our next step:

-   We are re-constructing an orthonormal 3D coordinate system based on the position and rotation of our camera
-   The right axis that we are looking for is orthogonal to the forward axis and the up axis (_which is still unknown, but disregard that for now_).

That was a lot to say, but given all that, we can conclude that we need to find the **cross product** of the forward and up axes. Wikipedia describes the **cross product** as:

> Given two linearly independent vectors **a** and **b**, the cross product, **a** × **b** (read “a cross b”), is a vector that is perpendicular to both **a** and **b**

![A visual representation of a cross product](https://cdn-images-1.medium.com/max/800/1*Uom4SbyxSqVxhsDsN_9YNg.png)

An example of the cross product — using the up and forward axes, we can find a vector perpendicular to both of them — the right axis.

OK, well _we know the forward vector_ (since we solved for it in Step One).

BUT, we still lack the up vector. Are we doomed? Nope… **we just need magic.**

_Here’s the trick_: We just need any-old vector that falls within the same plane formed by the forward and up vector… _and not necessarily_ the actual up vector of the camera itself. A common convention in this case is to use (0, 1, 0) as the up vector.

![A visual representation of the cross product, with the temporary up vector](https://cdn-images-1.medium.com/max/800/1*hOcKK0nIeIAEYqm8sp2wUw.png)

The trick — we can define a tempUpVector in place of the actual up axis, since it will still lie within the plane formed by the up and forward axes.

Finally, we can solve for the `rightVector` by doing the following calculation:

_tempUpVector = (0, 1, 0)  
rightVector = normalize(cross(tempUpVector, forwardVector))_

### Step Three: Calculate the Up Axis direction

It’s all smooth flying from here. We now know the `rightVector` and the `forwardVector`… so given our earlier assumptions (these three vectors are orthonormal), we know that to find the `upVector`_,_ we need to find the **cross product** of the `forwardVector` and `rightVector`!

_upVector = normalize(cross(forwardVector, rightVector))_

And that’s that. We have successfully calculated our 3 directional vectors.

Now we could start building our **LookAt matrix**, but we’re still missing a key component: _the translation component_.

### Step Four: Calculate the Camera Translation Vector

Now we need to put on our _Thinking Witch Hats_ to figure out how we will translate our camera so it gets to the position that we want it to be in our coordinate system.

> Why not just translate it by the camera position we used to calculate the `_forwardVector_` ?

That’s a great question. Let’s look at an example.

Let’s assume that our camera was originally at the 3D position in world coordinates **(0,4,4)** and our red ball is at origin **(0,0,0).**

![Camera looking at red ball](https://cdn-images-1.medium.com/max/800/1*EpEBqR1eQCK3IGnYzFIgEw.png)

A naive approach where we assume that the camera is translating along the axes defined on the bottom left (the blue/green arrows).

On the face of it, this looks like we should be able to just translate the camera by **(0,4,4)**… but it’s missing a key understanding of the order of matrix operations that I neglected to clarify earlier but will do now to drive home a point:

**We are rotating our camera first, then translating.**

⚠️ _This is mainly convention, and it’s up to you to decide the order in which you want to perform matrix operations… but if you don’t follow the way I present it to you here, you will not arrive at the same result._

Basically, we need to think about how to translate the camera **after having rotated it.**

![Visual of how to calculate the camera translation vector](https://cdn-images-1.medium.com/max/800/1*RLQu6lGE4Pe0jtqrfVe5UQ.png)

The correct approach where we rotate first, then translate along our new axes.

**The real translation vector of the camera would be (0, 0, 5.65).** since we are changing the coordinate basis (while maintaining its orthonormal properties).

The operation that we need to use to figure this out can be simplified by using **the dot product**. Read more about it [here.](https://mathinsight.org/dot_product)

In short: the dot product will return the magnitude in which two vectors project over each other. Therefore, it will return zero for orthogonal vectors since they are perfectly not-overlapping each other. Not even a little bit. Here’s what the calculation may look like:

_translationX = dot(positionOfCamera, rightVector);  
translationY = dot(positionOfCamera, upVector);  
translationZ = dot(positionOfCamera, forwardVector);_

⚠️_Remember to define_ `_rightVector_` _,_ `_upVector_` _and_ `_forwardVector_` _as unit vectors! The correct-ness of these calculations rely on it._

💡**_Thought experiment — _**_Starting with an object at (0,0,0),_  _can you imagine what translating first then rotating (around the origin) will do? Can you imagine rotating (around the origin) then translating will do?_

### Step Five: Build the Matrix!

Now just fill in the data into this matrix:

![Lookat matrix calculation](https://cdn-images-1.medium.com/max/800/1*T-SAObQaoHUTBlw5jcrVFg.png)

The Magic LookAt Matrix

That’s it, we’re done… but if you’re looking for how you can use this for a 3D application that you’re developing… well keep on reading!

Recall that I said at the very beginner that we would like to view this world from the lens of the camera. The typical name for a transformation matrix that does this operation is a **ViewMatrix.**

The key thing to understand is that the camera _stays at origin_ and looks down the negative z-axis. **It does not move**. To simulate camera movement, _we need to move the objects in the scene instead._

**Therefore, we need to apply to the _inverse_ of the LookAt matrix to the objects in the scene rather than the camera.**

In the code, we calculate just that… the translation is inversed (turned negative), as is the rotation component. However, **take note of a very important point** that made me bang my head at work while trying to debug a rotation issue… **the inverse of a rotation is it’s transpose.**

So, our final inverse LookAt Matrix… or View Matrix, would be:

![View matrix calculation](https://cdn-images-1.medium.com/max/800/1*GMLVIbbWXGE291FTG10toQ.png)

The inverse LookAt matrix is the ViewMatrix

You’re a wizard, reader, be proud of yourself for getting through all of this material. Until next time!

----------

_¹ for the uninitiated — it’s math for arrays and matrices. For the initiated — don’t crucify me for that gross vulgarization._

### Resources

-   [WebGPU — Cameras (theory) | Carmen Cincotti](https://carmencincotti.com/2022-04-25/cameras-theory-webgpu/)
-   [Brown37 — Introduction to Cameras](http://learnwebgl.brown37.net/07_cameras/camera_introduction.html)
-   [Wellesley — Viewports, Aspect Ratio, Depth, Unprojection](http://cs.wellesley.edu/~cs307/readings/07-viewports.shtml)
-   [w3 — WebGPU — Coordinate Systems](https://www.w3.org/TR/webgpu/#coordinate-systems)