The power of linear programming, a real life case study | by Tristan Bilot | Towards Data Science

Linear programming is a powerful tool widely used to solve optimization problems, but this method remains relatively unknown from the point of view of developers.
In this article, I’m going to explain the fundamental principals of linear programming and then apply these principals to a concrete daily life problem.
Linear programming consists to apply mathematical models to linear problems in order to maximize or minimize an objective function respecting some constraints. So it can used to solve any problems that can be represented as a linear function with some parameters and constraints. For instance, the famous knapsack and traveling salesman problems are optimization problems that can be solved using linear programming.
Basically, the goal of the algorithm will be to determine the optimal values for all the parameters of a function f in order to get the best fit for our problem. Hopefully, we already have algorithms to solve this (eg: simplex algorithm) and as developers we even can use solvers that are specially optimized to solve these kind of problems. Most programming languages offers libraries with easy-to-use solvers.
The hard part is not use the algorithm which will solve our problem but to find a way to translate our initial problem in a function with parameters and constraints.
After some theory, let’s apply linear programming to a real life case study.
Let’s pretend that we are a famous e-commerce platform which want to introduce relay points in a city for the shipping of its products. The only data we have is the position (coordinates) of all the clients of the company in that city. Our goal is to optimally place some relay points based on the position of the clients. In other words, we want to minimize the distance between each relay point and each client in the city.
First of all, let’s convert this specification in a usable mathematical expression. We know that in the case of coordinates in a city, we can use the Manhattan distance to approximate the distance of two points.
This distance between two points A and B can be written like the sum of the distance between each coordinate as absolute values:
So now that we know how to compute the distance between two points, let’s generalize it for for the clients and all the relay points:
Where p is the number of relay points we want to place, l the total number of clients in the city, (xi, yi) the p coordinates of the wanted relay points, (al, bl) the m coordinates of all the clients in the city.
As we want to minimize this expression, we add a minimization function min() which means that we want to find the optimal (xi, yi) pairs for which the result of the expression will approach the minimum:
The next step is to define the good constraints regarding our equation. The constraints will be written as inequations like suggested by the slack form. These constraints will avoid the solving infeasibility of the equation by giving some rules to respect. In our case, the only rules we want to respect for the given variables is the absolute value used in the calculus of the distance. In linear programming, we cannot use an absolute function directly on the equation to minimize, so a way to deal with absolute values is through constraints. Based on the following rule, we can transform an absolute value as a pair of inequalities:
When applying these rules on the Manhattan distance between two clients (A1 and A2) and one single relay point (x, y), we get:
Where:
It is important to note that we will need to write the constraints for each pair of coordinates received in input. Hopefully we are going to automate the generation of the inequations for each points in the next part.
For ease of purpose, I chose Python to implement the solution.
Well, now let’s forget the maths for an instant and think about the implementation. We have the equation to minimize, we have all the constraints to apply, now what? As said at the beginning, we generally use solvers to solve linear programs, so no exception here, let’s use a solver in Python:
scipy.optimize.linprog(C, A_ub, b_ub, A_eq, b_eq)
Where C is our equation, A_ub is the left hand side of the inequations, b_ub the right hand side of the inequations, and respectively the same for A_eq and b_eq but with the equations.
Why only 4 variables to define all the constraints? Because the constraints will be represented as huge matrices of coefficients. Each column of a matrix represents a variable and each line an (in)equation.
For instance, 2x1 + 3x2 ≥ 100 and 4x1 + 6x2 ≥ 0 are transformed as:
A_ub = [[-2, -3], [-4, -6]] ; b_ub = [-100, 0]
In the slack form, inequations should use ≤ so we inverse the inequalities.
For each pair of coordinates, the number of constraints (rows) will constantly remain 5, as see in the step 2. But the number of variables (columns) will grow with the number of clients/relays.
As an example, these are the variables used for 1 relay (x, y) and 2 clients (a1, b1), (a2, b2):
We’ll use this example during the following next steps.
1  var x >= 0;2  var y >= 0;3  var a1 >= 0;4  var b1 >= 0;5  var a2 >= 0;6  var b2 >= 0;7  var d1 >= 0;8  var d2 >= 0;9  var dx1 >= 0;10 var dy1 >= 0;11 var dx2 >= 0;12 var dy2 >= 0;
If we consider all theses variables as a vector, we can define the first parameter C, the equation:
C = [0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0]
As we are looking to minimize d1+d2, we are telling that the column which designed d1 and d2 should be activated with the coefficient preceding d1 and d2 (1 here). As referred to the list of variables above, the indexes 6 and 7 are effectively d1 and d2.
Then, let’s compute the left hand side on the inequation matrix representing the constraints defined at the step 2:
A_ub = [[0, 0, 0, 0, 0, 0, -1, 0, 1, 1, 0, 0],[1, 0, -1, 0, 0, 0, 0, 0, -1, 0, 0, 0], [-1, 0, 1, 0, 0, 0, 0, 0, -1, 0, 0, 0], [0, 1, 0, -1, 0, 0, 0, 0, 0, -1, 0, 0], [0, -1, 0, 1, 0, 0, 0, 0, 0, -1, 0, 0], [0, 0, 0, 0, 0, 0, 0, -1, 0, 0, 1, 1], [1, 0, 0, 0, -1, 0, 0, 0, 0, 0, -1, 0], [-1, 0, 0, 0, 1, 0, 0, 0, 0, 0, -1, 0], [0, 1, 0, 0, 0, -1, 0, 0, 0, 0, 0, -1], [0, -1, 0, 0, 0, 1, 0, 0, 0, 0, 0, -1]]
Referring to the vector C, we have to replace the variables by their indexes in the vector. In the above example, I transformed the inequalities in the form x1+x2≤0 in order to always have only a 0 on the right hand side. This is an implementation detail but I found it useful in my case because b_ub will thus be a vector of zeros.
Note that A_ub has 10 rows because there are only 2 clients for one relay point, 2*5 constraints = 10.
Now, the easy part, the right hand side of the inequations should be a vector of zeros as said above:
b_ub = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
To be honest, the hardest part is behind us ;), we’ll now fill the equation matrix. Basically, the inequation was for constraints, now the equation matrix represents the given data, in our case this data is the list of clients’ coordinates. So the left hand side of the matrix enumerates at each row a coordinate (ai, bi) for all the clients, and then define the value in the right hand side.
A_eq = [[0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0], [0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0],[0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0],[0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0]]
As expected, a diagonal appears in the matrix at the position of all the clients coordinates (a1, b1), (a2, b2).
b_eq = [10.0, 20.0, 15.0, 45.0]
These values are examples of clients’ coordinates.
It’s time to start coding! (finally)
In order to compute the 5 previous matrices/vectors, it is preferable to split that in 5 methods: 1 for the equation, 2 for the inequations rhs/lhs and 2 for the equations rhs/lhs. The following methods take two parameters:
M: the list of pairs of coordinates (ai, bi) of the clients.
p: the number of relay points to place.
Finally, call the solver using all the above methods.
Let’s test with the example given previously:
Input:  M = [(10, 20.), (15, 45.)]  p = 1Output:  [(12, 28)]
Given only two points for one relay, there is an infinite number of optimal solutions on the straight line between the two points (eg: (13, 44) is also an optimal solution in this case), but it returns a point around at the middle of the two clients’ coordinates, which is a coherent solution.
What if we use this current implementation on multiple clients? The algorithm will return always the best approximation which is one point. But if we set p > 1 we need to place more than one relay. So we need to find out a solution to divide the population M of clients in some groups on which apply the linear program. Different approaches can be used, however, I’ll show an example using the k-means clustering algorithm which allows to divide our population of clients in p clusters based on their coordinates (the nearest clients will be together in the cluster).
Input:  M=[(0, 50.), (100, 25.), (100, 40.), (50, 85.)]  p=3Output: [[(0, 50)], [(100, 25), (100, 40)], [50, 85]]
In the above example, we divide the population M in three sub-arrays containing the nearest points. Using Python, it can be done using numpy and scipy like so:
Finally, by joining all the pieces, let’s write the solve(M, p) method.
Now enjoy the algorithm on big datasets :)
Input:  n = 10000  M = [(float(random.randrange(0, n)), float(random.randrange(0, n))) for i in range(1000)]  p = 5Output:  [(3521, 8323), (1923, 5113), (2545, 1486), (7495, 2267), (7600, 7528)]
Based on randomly generated coordinates, it could be interesting to simulate the algorithm on a city map.
We can observe that the repartition of the relays is pretty well distributed. It’s now free to you to use your own implementation.
Thank you for reading, do not hesitate to reach me for further questions or for mistakes made in this article, it will be a joy to debate about that!
You can also leave a clap if you learned something in this article :)
Full source code of the project: my GitHub repo.