The Ultimate Guide to Dynamic Programming
By: Aleks Mitrovic
Tags: Leetcode, Data Structures, Algorithms, Programming, Dynamic Programming

Step 3: Iteration + Tabulation

Step 3 is almost always a viable next option. Key word almost. If you don’t see a way of improving upon your previous step using the information in this section, and/or your interviewer is more than glad with how you’ve set up your recursion and memoization, you can call it a day and move onto whatever’s next to tackle on your list. Otherwise, buckle up because we’re shifting gears once again.

This third step involves solving the problem using iteration instead of recursion.

We will also use tabulation instead of memoization. Fundamentally, both accomplish the same goal — they store calculations we’ve previously done in order to reuse them for future iterations. However, tabulation will involve storing calculations in an array (most often a 2-dimensional one), while memoization most commonly refers to storing calculations in a set, object, dictionary, etc. Tabulation is what will allow us to iteratively walk through an array without relying on recursive calls.

Note that these two changes may not necessarily bring upon any improvements with respect to asymptotic analysis (i.e. time and space complexities). However, the benefit to modifying to an iterative solution lays in the fact that your program will be able to run larger inputs as it is no longer limited to the maximum size of the call stack.

We often forget to note the limitation of call stack size; that is, the call stack does actually have a maximum size that it can reach before it overflows. For example, if you’re coding in Python, this maximum is 997 calls (the maximum stack size is relatively low in Python because the stack frames themselves tend to be quite thick.) So just because switching to iteration will not necessarily fix your Big O time and space complexities, it could allow you to break free of the bottleneck imposed by maximum call stack size (as well as to set you up for potential space complexity improvement in Step 4.)

When you visit YouTube and watch a video of somebody filling out an m x n table one value at a time, they’ve basically jumped to Step 3. And some of those videos are class; top drawer stuff that really does provide a quality solution. However, many of them fail to sequentially walk through the logic on how they got to that solution.

This is the main reason why I wanted to write this article.

Most often, these teachers have only implicitly realized what the brute force recursive solution would be, including what the recursive relationship is. They have also concluded that there is a clear case of overlapping subproblems and an optimal substructure (i.e. Step 1.) If you’re lucky, they did the latter out loud.

They have then most likely identified that the solution could be cleaned up using some simple memoization, usually in the form of storing calculations in an array / hash table / dictionary / set (i.e. Step 2.)

Typically, it’s only after all this has been done that it’s appropriate to jump in and identify that it can be done iteratively in a way that will help us eliminate the call stack size constraint, and potentially even set us up for optimizing space complexity in the next step.

My personal qualm with many of these resources that jump straight into Step 3 deceptively (albeit unknowingly) don’t take the time to explain how or why they got there. And quite often we end up looking like chimps as we‘re sat staring at code with Victorian-era elegance, wondering how on God’s green earth we’d come up with this in an interview setting.

The secret to getting there, once again, lays precisely in Steps 1 and 2.

Now, let’s put some context around this point now with another look at Unique Paths. (For a more comprehensive walkthrough of this solution, you can checkout my YouTube video of it here.)

We’ve made a slight adjustment to our dp table here which I think reads more intuitively for this solution. In particular, we’ve gone ahead and made it of size m x n, with all initial values set to 1.

In lines 9–11, we walk through this table backwards from the cell in the second last row and second last column. (This may not be naturally intuitive, but it is a great trick to consider when filling out a table in problems similar to this one; I use this quite often for problems that require figuring out just about anything regarding getting from point A to point B.)

Notice that every cell in the bottom row and rightmost column will both be filled with 1 and only 1. As noted in Step 1, the reason for this is because from any one of those cells there is exactly 1 way to get to the end. By walking backwards, this allows us to access the number of paths from the cells to our right and below in constant time. It’s an incredibly clean way of filling the table that ends with our entry dp[0][0] representing the number of unique paths from the start of the grid.

Once more, notice that the time and space complexities are identical in Steps 2 and 3. However, we’re finally free of that pesky maximum call stack size — and boy does freedom ever taste good.

And just to score some extra brownie points, I’d like to think that this code looks far sexier than what we’ve previously done. It’s tidy, compact, and quite succinct as well.