JuMPing into Puzzle Solving with Linear Programming | by Sarah Eade | Towards Data Science

Linear programming (LP) is a powerful technique to solve optimization problems, and you will often see it applied to problems from airline scheduling, vehicle routing, to revenue management. In this post, I’d like to explore a more whimsical application — how can linear programming be used to solve a puzzle?
The puzzle is similar to Sudoku, so I’m going to extend the well-known Sudoku LP formulation for this new brain teaser. I recommend that you understand that Sudoku solution first before proceeding since it will have some similar logic (here’s a medium post about it).
This puzzle was introduced to me by a friend who speculated that it could be solved with LP. The problem was presented on Jane Street Puzzles, and you can read the description here. From just reading about the constraints, it seems pretty similar to Sudoku — there are constraints on number of occurrences, but also some other constraints regarding first/last appearance, the sum of values, etc. Can it be solved with LP? The answer is Yes, and let’s see how.
After reading your problem and getting a strong grasp of the goal, you should start with defining the variables. What would the decision look like? Is it continuous, integer, binary? Does it have bounds? Technically this is a constraint but it may be helpful to think about this in the beginning so you don’t forget to add it later.
For this puzzle, the decision is simply: what number should go into a given cell?
My variable name is going to be x, it’s going to be binary, and it is going to be indexed by three values:
v is the value of the number from 1 to 7 that we are considering
r is the row index from 1 to 7
c is the column index from 1 to 7
In math notation, this is
The resulting solution will then tell us which number should go into which cell. For instance, if the solution specifies x_{1, 1, 1}=1, then that means number 1 goes into cell at location (1, 1). If x_{3, 4, 7}=0, then that means number 3 does not go into cell at location (4, 7). We will have a solution when we know all of these 1s and 0s for each number and cell combination.
This is going to create 7•7•7 = 343 binary variables. Binary variables will always be more difficult to solve than continuous-valued variables for LP problems. Fortunately this is a small number of variables nowadays, and much larger problems with hundreds of thousands of variables can be solved by solvers nowadays. You should have no problem solving this problem locally.
This problem is unusual — there is no objective! Set it to a constant, such as 0. Our goal is just to find a solution that meets all constraints; there is no measure of one solution being better than another. As long as the result is feasible, we will have achieved the goal. Thus we don’t need an objective function.
Here’s where the heavy-lifting comes in. Let’s tackle them one at a time.
Each cell can only contain one number or it can be blank.
For a given cell, if the sum of the binary variables for v=1,2,...7 equals zero, then I’ll interpret it as the cell is blank. We should use an inequality in the constraint to allow for these null cells, and set the upper threshold to 1 because we don’t want more than one number per cell.
Number of occurrences of each value v (not including the null) must equal the value v. This ensures that the number of fives in the whole grid is five, the number of sixes is six, etc. We know the value of the number since it is equal to the index v, so that will be the right-hand-side of the constraint.
Each row has 4 numbers. Also, each column has 4 numbers. So just sum over all the columns (rows) and all the possible values. This must be less 4 because there must be 4 numbers.
The sum of numbers in each row should be 20. Also each column’s sum should be 20. This is similar to the constraint above except we need to multiply by the value of the number, v.
Every 2x2 sub-square must contain at least one empty cell. Thus not every cell can be filled, so there should be 3 or less numbers in a sub-square.
The result should have a connected region. This also means that a number should not be surrounded by nulls.
This is achieved if we look at at the cells adjacent to a given location (think of drawing a box around a number). If the region is entirely full of nulls, then the sum of the binary variables for that border will be zero. In that case, we then want to force the center number to be null too in order to satisfy the rules. If the count of non-null values in the border is greater than zero, then a number in the center is allowed.
The constraints need to be written depending on where they appear in the grid. For the points where we can draw a 3x3 box (see image below) around the cell, the following equation will apply
This equation is modified to accommodate the remaining areas of the grid. For instance, for the cells in the top row, the equation will be modified for the rows to range from 1 to 2 (since there is no 0 row).
The rest of the equations can be similarly modified for the first and last columns and the cells in the corner.
The start of the row/column should start/end with a particular number. For the case of row 2 needs to start with 3, then it means that the first column in row 2 can be null or 3. After applying that, then if we move one-by-one across row 2, then a cell can only have a number if the cells to its left had a 3.
The remaining start/end constraints follow a similar formulation.
That’s it! Those are all the steps needed in order to define the problem. Great work and glad you’re still with me. Now, the next step is to code it up.
I learned optimization in grad school where the Julia language was enthusiastically used. The packages you’ll need are GLPK and Julia for Mathematical Programming (JuMP) — hence the title of this article. It’s a terrific package; it’s very fast, intuitive to write LP problems, and trivial to switch between a variety of solvers. If you’re not familiar with Julia and JuMP, then I urge you to check it out. Even without understanding the language, the code should be understandable. I used Julia 1.1.1, JuMP 0.19.2, and GLPK 0.10.0
using GLPKusing JuMP# Set variable for number of rows/cols/numbersN = 7function solve_puzzle(start_grid::Array{Int64,2}, edge_constraints::Array{Int64,2})    model = Model(with_optimizer(GLPK.Optimizer, Presolve=0, OutputFlag=0));    # Since we don't care about the objective just set it to something arbitrary    @objective(model, Min, 0);# Define variables which will be 7x7x7 matrix    # Use integer instead of binary because I was having trouble fixing Binary values in step below    @variable(model, 0 <= x[1:N, 1:N, 1:N] <= 1, integer=true);# Initialize values    for idx in 1:size(start_grid)[1]        v = start_grid[idx, 1]        r = start_grid[idx, 2]        c = start_grid[idx, 3]        # Fix value to 1        fix(x[v, r, c], 1, force=true)    end# Create the constraints    # At most one value per cell    for c in 1:N        for r in 1:N            @constraint(model, sum(x[v, r, c] for v in 1:N) <= 1)        end    end# The number of occurrences of a number should equal the value of that number    for v in 1:N      @constraint(model, sum(x[v, r, c] for r in 1:N for c in 1:N) == v)    end# Each row has 4 non-null numbers    for r in 1:N      @constraint(model, sum(x[v, r, c] for c in 1:N for v in 1:N) == 4)    end# Each column has 4 non-null numbers    for c in 1:N      @constraint(model, sum(x[v, r, c] for r in 1:N for v in 1:N) == 4)    end# Each row numbers sums to 20    for r in 1:N      @constraint(model, sum(v*x[v, r, c] for c in 1:N for v in 1:N) == 20)    end         # Each column numbers sums to 20    for c in 1:N      @constraint(model, sum(v*x[v, r, c] for r in 1:N for v in 1:N) == 20)    end# Every 2x2 subsquare must contain one or more empty square    for p in 1:(N-1)        for q in 1:(N-1)            @constraint(model, sum(x[v, r, c] for r in p:(p+1) for c in q:(q+1) for v in 1:N) <= 3)        end    end# Connected region = no islands    for c in 1:N        for r in 1:N            @constraint(model, 2*sum(x[v, r, c] for v in 1:N) <= sum(x[v, i, j] for v in 1:N for i in (r-1):(r+1) if i in 1:7 for j in (c-1):(c+1) if j in 1:7  ))        end    end# Edge constraints                                                                                                         val_set = Set(1:N)  # define all possible values    for idx in 1:size(edge_constraints)[1]        v = edge_constraints[idx, 1]  # value        k = edge_constraints[idx, 2]  # affected row/col        is_col = Bool(edge_constraints[idx, 3])  # row/col        is_last = Bool(edge_constraints[idx, 4])v_prime = setdiff(Set(1:7), v)for j in 1:(N-1)            for not_v in v_prime                if is_col                    if is_last                        # last in column k must be v or null                        fix(x[not_v, 7, k], 0; force=true)                        # cell is non-null if the values after cell in column have number                        @constraint(model, x[not_v, j, k] <= sum(x[v, r, k] for r in (j+1):N ))                    else                        # first in column k must be v or null                        fix(x[not_v, 1, k], 0; force=true)                        # cell is non-null if values before cell in column have number                        @constraint(model, x[not_v, j+1, k] <= sum(x[v, r, k] for r in 1:j) )                    end                else                    if is_last                        # last in row k must be v or null                        fix(x[not_v, k, 7], 0; force=true)                        # cell is non-null if values after cell in row have number                        @constraint(model, x[not_v, k, j] <= sum(x[v, k, c] for c in (j+1):N ))                    else                        # first col in row k must be v or null                        fix(x[not_v, k, 1], 0; force=true)                        # cell is non-null if values before cell in row have number                        @constraint(model, x[not_v, k, j+1] <= sum(x[v, k, c] for c in 1:j) )                    end                end            end        end    endoptimize!(model)    # want to see the code 1                                                                                                                                    println(termination_status(model))     primal_status(model)# Create results    results = zeros(N,N)    for i in 1:N        vals = reshape(value.(x[i,:,:]), N, N)        multiplier = ones(N,N)*i        results += multiplier.*vals    end    return results                                                                                                                            end
To run it, just specify the edge constraints and starting grid values, and call the function.
# Initialize starting values# column 1 specifies number, column 2 is the row number, column 3 is the column numbervalues = [7 1 2;          6 1 3;          6 2 4;          6 2 5;          5 3 1;          6 4 2;          4 4 6;          6 5 7;          4 6 3;          7 6 4;          7 7 5;          7 7 6]# Encode the first/end row/col constraints (counter-clockwise)# column 1 specifies the number# column 2 specifies the affected row/col# column 3 has values 0 for row and 1 for col# column 4 has values 0 for start and 1 for endstart_end_constraints = [3 2 0 0;                         5 5 0 0;                         1 7 0 0;                         7 1 1 1;                         2 2 1 1;                         2 6 0 1;                         4 3 0 1;                         4 1 0 1;                         7 7 1 0;                         5 6 1 0];results = solve_puzzle(values, start_end_constraints)
For those wanting the solution:
Thanks for reading! I’ll try to monitor the comments for questions.