The Binary Multidimensional Knapsack Problem (MKP) | by Mohammed Agha | Towards Data Science

There are numerous posts addressing the knapsack problem as both an integer programming problem, and as a simple example to explain dynamic programming. Not enough content on the multidimensional knapsack problem though. In this post, I will go through the Multidimensional Knapsack Problem ‘MKP’, point to where we can find benchmark instances, provide my used code file to read these instances (in Python), then proceed with how to model an MKP instance on Python and solve it with IBM CPLEX. This article is intended to serve as a gentle introduction to the MKP and a simple tutorial on how to code it for interested optimization enthusiasts and young practitioners.
The MKP is an NP-hard extension to the standard binary knapsack selection problem. The goal is the same; to find a subset of items that maximizes the total profit/gain (objective function), however, the difference is that instead of having a single knapsack or resource, there are multiple knapsacks/resources (each is a separate constraint) and the subset of items should not violate the capacity of any of these knapsacks. Avid readers interested in the various versions of knapsack problems are referred to the book Knapsack Problems by Kellerer, Pferschy and Pisinger for more details. Here we will only limit our scope to the binary MKP.
The mathematical formulation of the MKP is:
(MKP) maximize z = ∑ c. x
Subject to:
A.x ≤ b, ∀ i = 1,...., n
x ∈ {0,1},∀ j = 1,....,m
where c is the profit-per-item vector, b is the right hand side vector or the capacity of each knapsack, x is a vector of binary variables indicating whether an item is selected, and A is the constraints’ coefficients matrix. The number of variables is m while the number of constraints is n.
The widely used benchmark instances in the literature can be found at the online OR-Library with a complete explanation about the format and content of each benchmark set. These instances are either test problems collected from the literature or test instances solved in Chu & Beasley (1998) [1]. Benchmark sets vary in number of variables (columns) and number of constraints (rows) with the hardest benchmark set containing 30 instances, each comprising 500 columns and 30 rows (columns and variables are used interchangeaby in this post, similarly, rows and constraints).
Benchmark sets found at the online OR-Library are text files containing the:
number of test instances in each set,
size of each instance (i.e., number of columns and rows),
objective function coefficients (profits),
constraints coefficients (resource consumption of each variable from each knapsack/resource) and,
right hand side (capacity of each knapsack).
To simplify things, we will consider a text file containing data of only one instance. At this GitHub repository you can find text files, each with data pertaining to a single instance, in addition to a function that reads the text files, populates the instance and prepares the input to be used in the mathematical model (c, A and b).
There are multiple linear programming open-source or commercial solvers that can be used to model an optimization problem. This informative post is a very good start to learn how to use some solvers.
Optimization Modeling in Python: PuLP, Gurobi, and CPLEX
I will be using IBM CPLEX since I have more experience working with it. CPLEX should be installed in advance and has a free edition for students and academics.
We start by importing relevant libraries:
import cplexfrom docplex.mp.model import Model
we will also import the file containing the function that reads and populates the MKP instances (refered to in the previous section).
# Import the reading function import MKP_populate_function as rdmkp
next, we will call the function on an instance and get our parameters to be used in creating the model:
# Call the function on a given instanceinstance = 'mknapcb1_1.txt'c, A, b = rdmkp.MKPpopulate(instance)# Define the ranges for variables and constraintsnCols, nRows = range(len(c)), range(len(b))
I usually use the last additional step in the previous code block to define ranges for the number of variables and number of constraints. They mainly represent the sets of variables and constraints. It’ll prove handy when we define variables and constraints in the mathematical model.
As we have our parameters ready, we will proceed with creating an empty model:
# Create an empty model mkp = Model('MKP')
Now we will augment the empty model by defining the decision variables. In MKP, variables are binary so we will declare them accordingly. You will notice that I added lower and upper bounds which are not needed for binary variables, however, sometimes I want to check the linear relaxation bound, so I keep bounds defined in all my models but change the binary type to continuous.
# Define decision variables      x = mkp.binary_var_list(nCols, lb = 0, ub = 1, name = 'x')
Each constraint represents a different knapsack with its own capacity. The consumption of each item from this knapsack is given by the elements in the A matrix.
# Declare constraintsconstraints = mkp.add_constraints(sum(A[i][j] * x[j] for j in nCols) <= b[i] for i in nRows)
The next step is to create the objective function, which is to maxmimize the sum of the profits of the chosen subset of items. Adding the objective function as a KPI is a good practice that makes reporting results better. I learned this from more seasoned modelers.
# Declare the objective functionprofit = mkp.sum(c[j] * x[j] for j in nCols)# Add Obj. Function as a kpi for better reporting of results     mkp.add_kpi(profit, 'profit')        # Add objective function to the model as a maximization typeobj = mkp.maximize(profit)
What remains now is to solve the created model and report the results.
# Solving the modelmkp.solve()# Reporting resultsmkp.report()
The following method, when called, provides information about the number of variables and their types, the number of constraints and the type of the problem along with the sense of the objective function(maximization or minimization).
mkp.print_information()
It should be noted that CPLEX finds the exact solution to the MKP. While it seems fast in solving small instances, it takes longer time to solve larger size integer/binary instances.
As Prof. J. E. Beasly puts it: “What makes solving the problem easy when it is small is precisely what makes it become very hard very quickly as the problem size increases”.
This is simply because when the problem size is relatively small, we can enumerate all possible solutions and examine their feasibility and find the optimum one. However, when the problem size grows, the number of solutions grows faster and we cannot check all of them in feasible time. Other methods can be used such as heursitics/metaheuristics but maybe we can discuss them in another post.
If you liked this content you might be interested in this post about a very interesting application in networks optimization where we used social network analysis to measure and minimize disruption to teams.
[1] PC Chu, JE Beasley, A genetic algorithm for the multidimensional knapsack problem (1998), Journal of heuristics 4 (1), 63–86