A Systematic Approach to Dynamic Programming
By: Fabian Robaina
Tags: Programming, Computer Science, Python, Dynamic Programming

Approaches to DP

The two main approaches to dynamic programming are memoization (the top-down approach) and tabulation (the bottom-up approach).

So far we’ve seen that recursion and backtracking are important when applying the DP premise of breaking a complex problem into smaller instances of itself. However, none of the code snippets above classify as DP solutions even though they use recursion and backtracking.

For a naive recursive solution to apply as a DP solution it should optimize to caching the results of computed sub-problems. In the short definition of DP above, the emphasis is on solving smaller instances only once — with a strong emphasis on “only once.”

Memoization

Memoization = Recursion + Caching

Our framework for a dynamic-programming-worthy problem said that it usually contains overlapping sub-problems. Remember the Fibonacci code above? If we create a recursive tree to compute the seventh Fibonacci number we get this:

Notice how many times we solve the same sub-problem. For example, Fib(3) is computed five times, and every fib(3) call recursively calls two more fib. That’s ten function calls solving the same fib(3) sub-problem.

Now we start talking DP! Instead of solving the same problem multiple times, why don’t we solve it just once and store it on some data structure in case we need it later? That is memoization!

Fib code optimized to caching.

This approach is the easiest of the two DP approaches presented here. Once you can get a recursive solution to the problem, just make sure you cache the solutions to the sub-problems. Before you make recursive calls to solve a sub-problem, check if it was already solved. Notice that here we do a trade: To archive time efficiency, we’re willing to give up memory space to allocate all computed sub-problems’ solutions.

Dynamic programming usually trades memory space for time efficiency.

When caching your solved sub-problems you can use an array if the solution to the problem depends only on one state. For example, in the fib code above, the solution to a sub-problem is the ‘nth Fibonacci’ number. We can use n as an index on a 1D array, where its value represents the solution to the fib(n) sub-problem.

Sometimes the solution to the problem may depend on two states. In this case, you can cache the results using a 2D array, where columns represent one state and rows represent the other. For example, in the famous Knapsack problem (which we’ll explore later) we want to optimize for total value, given a maximum weight constraint and a list of items. A knapsack sub-problem may look like this: KS(W, i) → (Max value), where we interpret it as: “What is the maximum value I can get with a weight ‘W’ and considering the ‘ith’ item?.” Therefore if we want to cache this solution, we need to take both states into account, and that can be accomplished using a 2D array.

Memoization is great — we have the elegance of a problem described recursively, and we’re solving overlapping sub-problems only once. Well, not everything is that great. We’re still making a bunch of recursive calls. Recursion is expensive both on processor time and memory space. Most recursive functions will consume call stack memory linearly with the number of recursive calls needed to complete the task.

There are special types of recursive functions, known as tail-recursion, that don’t necessarily increase the call stack linearly if optimized correctly. These can execute on a constant call stack space. Without going into many details, tail-recursive functions perform the recursive call at the end of its execution, meaning that its stack frame is useless thereafter. The same stack memory space can be reused to hold the state for the next recursive call. The problem that arises is in dealing with the return address. We want to make sure that after the recursive tree ends, you return to the instruction that started the series of recursive calls. Feel free to do some research on this topic.

Recursive functions always carry the weight of potential stack overflows issues. The following is a Python command to check the recursion depth limit. If I try to use Python with recursion to solve a problem whose solution involves a recursive depth of more than 1000 calls, I’ll get a stack overflow exception. That quantity can be increased, but we get into language-specific topics.

Recursion depth limit in Python

In defense of recursive programming, we must say that recursive functions are often easier to formally prove. Recursive functions provide you with the same repetitive behavior of raw loops but without in-block state changes, which is a common source of bugs. States in recursive programming are updated by passing new parameters to new recursive calls, instead of being modified as the loop progresses.

Tabulation

Tabulation aims to solve the same kind of problems but completely removes recursion. Removing recursion means we don’t have to worry about stack overflow issues, as well as the common overhead of recursive functions. In the tabulation approach to DP (also known as the table-filling method) we solve all sub-problems and store their results on a matrix. these results are then used to solve larger problems that depend on the previously computed results. Because of this, the tabulation approach is also known as a bottom-up approach. It’s like starting at the lower level of your recursive tree and moving your way up.

Tabulation can be much more counterintuitive than recursive-plus-cached memoization solutions. But it’s also much more efficient in terms of time complexity and space complexity if we take into account the call stack memory which increases linearly with the number of recursive calls — again, assuming it’s not tail-recursion optimized.

If you go back to the steps initially presented in this piece, you’ll find that tabulation is the last step in the systematic approach to DP. This is because it’s easier to get to a tabulation solution by first solving the problem with recursion and backtracking, then optimizing it to caching with memoization techniques, if necessary, and finally making a few adjustments to update it to a final bottom-up solution. Later you will see a few tricks to achieve that. But first, let’s see how tabulation works.

We’ve been talking about states a lot, but we still do not have a formal definition of what we mean by states on our DP context. What I understand by states are parameters that affect the outcome of a recursive call. States are what differentiate one call from another and allow us to explore different choices and get an optimal result. We’ll get some practice defining states at the end of this article.

Since tabulation proposes a bottom-up approach, we must first solve all sub-problems that a larger problem may depend on. If we don’t solve the smaller problem, we can’t move on to solve the larger one. In tabulation, we use one for-loop for every state of the problem. But where do I make it start? Where do I make it end? To answer that, let’s explore the following recurrence relation.

Let’s say we have a function that solves some optimization problem — call it optimal or OP for short. And let’s assume that the nature of the problem makes OP(n) depend on OP(n-1), so OP(n) = OP(n-1).

This recurrence relation is telling you that you cannot know what OP(n) is if you don’t know what OP(n-1) is. That means we need to start at the lowest value of n, say 0, and solve every sub-problem all the way to n. That’s the trick: If your recurrence relation shows that your states are decreasing, then your loops should be increased so you compute every sub-problem that larger problems depend on. Remember bottom-up.

This will become clear when we apply all the strategies learned to real problems. And guess what? That will start now.