Learn Decision making under uncertainty — Part 2: Solve the stochastic problem | by Ramakrishna Thiruveedhi | Towards Data Science

In Part 1 we helped a farmer explore 3 different planting options. Even though our decision based on an average yield works reasonably well, we found that planning for an optimistic yield actually works better. But we have to make sure we are not ignoring another option that might work even better. One way to do that is to brute force approach of generating all possible planting options and computing the expected value (EV) of profit for each option just like we computed the EV for the 3 options we have proposed. If we allow real values for acres allocated, the possible options are infinite and hence this approach is not possible. If limit the choices to integer options between 0 and 500, we may be able to enumerate the choices but it is a painstaking approach that cannot easily be modified if the farmer wants to add a constraint for planting the crops.
We can actually enumerate all possible options smartly without using a brute-force approach. We will force the Linear programming solver to do that for us.
This notebook has good explanations that you can step through. I would recommend reading the methodology below before using the notebook.
Let us rewrite the equation for profit.
Profit = — Cost of planting crops + Revenue for crop sales at end of season — Cost of buying Wheat and Corn deficits
Please note that the cost of planting crops only depends on the acres allocated to each crop and does not depend on the yield realized. However, the revenue of sales and the cost of buying crops depends on the yield.
So far we have tried to make a decision on amounts to buy and amounts to sell assuming a yield and then computed expected value over three scenarios. However, we had to solve three separate linear programs. What if we combine three linear programs into one? That is the essence is stochastic programming — we dynamically find the optimal first-stage decision by evaluating the expected value of profit for all possible scenarios.
This formulation is called Deterministic Equivalent Problem (DEP).
Please note that we are allowed to make three different second-stage decisions for buying and selling for each scenario (yields are realized by the time we have to make these decisions). However, we could make only one possible first-stage decision for acres planted (If we are allowed to make three acres decisions, it means we have perfect information). In other words, we are hedging the impact of our first stage decision under various scenarios. But instead of manually computing the EV of our decision we are letting the linear program pick the values for our acres that would maximize the expected profit over all possible scenarios.
We can divide the decision variables into two stages.
One set of first stage variables that link to each set of the second stage variables. These stay same as our first linear program we solved with mean values for yield.
var_acres_planted = LpVariable.dicts("plant_acres", items, lowBound=0, cat='Continuous')
Several sets of second stage variables for each possible scenario. Please note that when we are done solving we do not have one answer for second stage variables; we have several answers — one for each scenario. Our code can be reused by adding scenarios as keys for each set of variables.
Both first stage an second stages contribute towards profit. However second stage contribution depends on each scenario and hence we have to take expected value over all scenarios.
Profit = — Cost of planting crops +
1/3 (Revenue for crop sales at end of season — Cost of buying Wheat and Corn deficits) for bad yield +
1/3 (Revenue for crop sales at end of season — Cost of buying Wheat and Corn deficits) for average yield +
1/3 (Revenue for crop sales at end of season — Cost of buying Wheat and Corn deficits) for good yield
We can think of constraints as two groups.
These constraints only affect decisions in the first stage. We only have total acres constraint today but the model can handle any other constraints. These stay same as our first linear program we solved with mean values for yield.
Farm Acres Available
Acres Planted for all crops ≤ 500
M += lpSum([var_acres_planted[i] for i in items]) <= total_acres
These constraints link each set of second stage variables to first stage variables. Please note that we know the scenario for each of these constraints and hence all the parameters that change with the scenario — for example, we know the yield is bad when we are writing a constraint for variables related to the bad-yield scenario)
For Wheat and Corn:
Crops produced by planting + Crops purchased = Crops used for consumption + Crops sold to market
For Beans:
Crops produced by planting — Crops sold to the market at regular price — Crops sold to the market at a lower price = Consumption for beans ( 0 for our farmer)
Crops sold to the market at regular price ≤ 6000 T
The system of equations can be solved in any linear programming solver. For purpose of learning, I have chosen the open-source Python package PuLP to allow anyone to run the code with minimal installation.
The notebook can be found here. Please install PuLP using conda or pip.
pip install pulp
We can run the solver and obtain an optimal solution and use python to compute the following table to compute costs, revenue, and profit. The solution is the following. The acres allocated to beans is intuitive given the very low secondary selling price of beans but the choices for corn and wheat are not. In general, it is very difficult to find this solution with intuition.
Plant beans to make sure we never sell beans at a lower price — 250 acres at high yield give 6000 tons.
Plant corn to meet farm needs under average yield.
Plant wheat in the remaining acres.
For any individual scenario yield, this solution obviously cannot beat the optimal solutions for that scenario (by definition of optimal solution). But its expected value over the three possible scenarios is better than EV for each of the three solutions ($86600, $107240, $107683 we computed in part 1) and any possible first stage solution for that matter. This is made possible by the stochastic linear program we have solved.
Our competitor knows only Linear programming and makes a reasonable recommendation based on average yield. By knowing this stochastic approach we added additional profit to the farmer. That is the Value of the Stochastic Approach (VSS). In this case, it is $108390–$107240 = $1150.
VSS = EV of Stochastic Solution (DEP) — EV of Mean value solution
What value is the total profit we added to the farmer? It depends on how the farmer would have planted his crops using his intuition. We could compute the expected value of that solution and compute the additional profit we have added. It can easily be done in our notebook by setting the variables to the farmer’s manual solution. Please note that a significant portion of that value is delivered by the linear program giving a profit of $107240.
Exercise: If the farmer typically plants two most profitable crops — beans and wheat by planting 300 acres for beans and 200 acres for wheat, what is the expected value of profit? How much value have we added in total (Ans: $10190? What value is delivered by the Linear program (Answer: $9040)? What percentage is VSS compared to the total value (Answer: 12.7%)?
A team of machine learning experts has proposed that they would be able to provide prediction for yield every year with 100% accuracy. What is the maximum the farmer afford to pay for this information when he already has access to our stochastic solution? This is called Expected Value for Perfect Information (EVPI).
EVPI = EV of using optimal solution for each scenario — EV of stochastic solution
EVPI = 1/3($59950+$118600+$167667) — $108390= $7016
If the probability distribution is discrete, we can repeat this notebook by generating yields dictionary for each discrete value.
If the probability distribution is continuous, we can make it discrete by generating slices of the distribution. How reliable is this method? I will not cover the details here but the theory states that if we generate enough samples this is very accurate. Please note that to get more accuracy we will have hundreds or thousands of scenarios. For this problem the problem should solve easily but if the linear program for a given scenario (mean value problem from part 1) is itself complex, we will have computational challenges. We could use a powerful commercial packages but we might even test their limits. There is a technique called Benders decomposition (L-shaped method) which can solve the problem by decomposing it into two stages.
We learnt how to modify our linear program to account for all possible scenarios by creating multiple copies for every variable that would depend on scenario.
The solution gives us an optimal solution for first stage. The second stage variables have several answers. The farmer can compute them every year after yield is realized and act accordingly (buy the deficits and sell the remaining crops).
Solving stochastic version of the problem gets us better expected value for profit by hedging for all possible scenarios.
In this problem we solved for maximizing expected value. This is the most popular option. We could also solve for other objectives; some of them are complex to model. I would recommend solving them.
Solve multi-stage problem. In the class notes referenced below there is a financial planning problem that is a good example.
I will limit my references to resources that explain basics and refer to this problem.
[1] J. Linderoth, Class Notes covering this problem. Additional notes
[2] J. Birge, F. Louveaux, Textbook: Introduction to Stochastic Programming — this problem is covered in chapter 1.
[3] Official site for PuLP solver in Python