Dynamic programming

**Dynamic programming** is an optimization technique that is used to optimize recursion problems. When we have a large number of recursion calls, in which each subproblem can be called many times with the same result, we can memorize these results so that we can avoid re-computing them when we subsequently solve problems of larger sizes. As a result, we can reduce the time complexity from exponential to polynomial.

## Example: Fibonacci sequence

Recursion vs. Dynamic programming
```
int fib(int n) {
 if (n <= 1) {
 return n;
 }
 return fib(n-1) + fib(n-2);
}

```

 Dynamic programming
```
int fib(int n) {
 if (n <= 1) {
 return n;
 }

 std::vector<int> results(n + 1);
 results[0] = 0;
 results[1] = 1;
 for (int i = 2; i <= n; ++i) {
 results[i] = results[i-1] + results[i-2];
 }
 return results[n];
}

```

Time complexity: **O(n)**

## Basic concepts

### Tabulation vs. memorization

There are two basic approaches to store the values corresponding to subproblems, so that values for the subproblems can be reused:

* **Tabulation**: bottom up.
* **Memorization**: top down.

#### Tabulation

**Tabulation** means to solve the problem in a *bottom-up* manner.

Example: Fibonacci sequence
For an input of value *n*:
1. Create an array of size *n*+1.
2. Set `array[0] = 0`, `array[1] = 1`.
3. For `i = 2, 3, …, n`, fill in the table from left to right.
4. Return `array[n]`.

```
int fib(int n) {
 if (n == 0) {
 return 0;
 }
 else if (n == 1) {
 return 1;
 }

 std::vector<int> table(n + 1);
 table[0] = 0;
 table[1] = 1;
 for (int i = 2; i <= n; ++i) {
 table[i] = table[i-2] + table[i-1];
 }

 return table[i];
}

```

Here we start from the smallest subproblem (i = 0), and then iteratively fill in the result table. Every state can be computed directly from the previously computed value. So we call this the **tabulation method**.

#### Memorization

Another approach is **memorization**, which means solving the problem in a top-down manner.

This approach involves recursive calls to use values stored from subproblems.

Example: Fibonacci sequence
For an input of value *n*:
1. Create an array of size *n*+1.
2. For `i = n, n-1, n-2, …, 1, 0`, use the recursive call: `fib[i] =`
	* `fib[i-1] + fib[i-2]` if `i ≥ 2`
	* 1 if `i == 1`
	* 0 if `i == 0`
3. Return `fib[n]`.

```
int fib(int i, const std::vector<int>& table) {
 if (i == 0) {
 table[i] = 0;
 }
 else if (i == 1) {
 table[i] = 1;
 }
 else {
 table[i] = fib(i-2, table) + fib(i-1, table);
 }
 return table[i];
}

int fib(int n) {
 std::vector<int> table(n + 1);
 return fib(n, table);
}

```

#### Comparison: tabulation vs. memorization

 Code
 Speed
 Solving subproblems
 Table entries

### Optimal substructure property

The **optimal substructure property** means that a problem's optimal solution can be obtained using the optimal solutions of its subproblems.

Example: Fibonacci sequence
The **Fibonacci sequence** problem satisfies the optimal substructure property, since the final solution and all intermediate subproblem solutions can be obtained using previous subproblem solutions, and the solution to each subproblem is an optimal solution.

### Solving a dynamic programming problem

There are four major steps to solve a dynamic programming problem:

1. Identify whether the problem is a dynamic programming problem.
2. Determine a state expression with least parameters.
3. Establish the transitional relationship between states.
4. Perform tabulation (or memorization). Fill in the state table.

#### Step 1: Identify the DP problem

* Typically, problems that asks us to maximize or minimize certain properties, or problems that say to count the arrangements under certain conditions, can be solved using dynamic programming.
* All dynamic programming problems satisfy the overlapping subproblems property. Some problems also satisfy the optimal substructure property. Once we observe these patterns, we can attempt to solve it using dynamic programming.

#### Step 2: Determine the states of a problem

The essential part of solving dynamic programming problems is establishing **states** and transition relationships.

A **state** is a step such that certain *parameters* of a subproblem (inputs) and *values* (outputs) are stored, so that it can be referred again on later computations for solving subproblems of larger sizes. We want to use *as least parameters as possible* so that we can reduce the state space size.

**For example:** in the Edit distance problem, we can use `dist[i][j]` to store the minimum edit distance between the two substrings `str1[0..i-1]` and `str2[0..j-1]`.

* Indices i, j are both *exclusive*.
* For edit distance between an empty substring and a non-empty substring, define:
	+ `dist[0][0] = 0`
	+ `dist[i][0] = i` (i > 0)
	+ `dist[0][j] = j` (j > 0)

#### Step 3: Defining transition relationships

After states are defined, we then need to define the **transition relationship**, which bridges the state of a subproblem and the states of its pertinent subproblems.

Still consider the Edit distance problem:

Text transclusionThis section is transcluded from Edit distance#Defining transitions.
* View or edit this page.

* If either `str1` or `str2` is empty, then the edit distance is the length of the other non-empty string.
* In the general case, consider substrings `str1[0..i]` and `str2[0..j]` (i, j > 0):
	+ If `str1[i-1] == str2[j-1]` (i.e. the last characters of the two substrings are the same), then obviously `dist[i][j] = dist[i-1][j-1]`.
	+ Otherwise, `dist[i][j]` may be given by referring to these three smaller subproblems, and picking the *minimum* of these three results:
		1. **`dist[i-1][j-1] + 1`**: This means that after editing substrings `str1[0..i-2]` to `str2[0..j-2]`, we replace `str1[i-1]` into `str2[j-1]` as the last edit.
		2. **`dist[i][j-1] + 1`**: This means that after editing substrings `str1[0..i-1]` to `str2[0..j-2]`, we append `str2[j-1]` as the last edit.
		3. **`dist[i-1][j] + 1`**: This means that after editing substrings `str1[0..i-2]` to `str2[0..j-1]`, we append `str1[i-1]` as the last edit.
* So the transition formula is: `dist[i][j] =`
	+ If `str1[i-1] == str2[j-1]`: `dist[i-1][j-1]`
	+ Otherwise, minimum of:
		1. `dist[i-1][j-1] + 1`
		2. `dist[i][j-1] + 1`
		3. `dist[i-1][j] + 1`

#### Step 4: Fill in the table

This is the easiest part. We just fill in the table step-by-step and get the result.

But there are additional things to notice:

* Filling in the table may take multiple passes, as in Largest plus sign.
* Depending on the problem, the result may be stored separately instead of as intermediate states in the table, as in Largest plus sign.

## Sample problems

* Edit distance
* Largest plus sign

## References

1. ↑ Tabulation vs Memoization (Archive)
2. ↑ How to solve a dynamic programming problem? (Archive)

## External links

* **Dynamic programming** on GeeksForGeeks.org (Archive)