INFORMS WORKSHOPGUROBI 10.0

April 2023

© 2023 Gurobi Optimization, LLC., All Rights Reserved

AgendaIntroduction
Xavier Nodet

10.0 Overview
Dan Jeffrey

Open Source: Pandas
Zed DeanOpen Source: Gurobi Machine
Learning
Xavier Nodet

Jupyter Notebook Modeling
ExamplesJerry Yurchisin

Q&A
Xavier Nodet

Wrapping up: Games,
Competitions, and Giveaways
Alison Cozad, Lindsay Montanari, Colum Devine© 2023 Gurobi Optimization, LLC., All Rights Reserved | 2

Explainer Video
Telling the Gurobi Story• Written with a non-technical audiencein mind• Covers the basics of how Gurobi works,
in just 60 seconds• Available on Gurobi YouTube Channel:
https://youtu.be/DqjE-P0VyoQ© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 3

Agenda
Introduction
Xavier Nodet

10.0 Overview <-Dan Jeffrey

Open Source: Gurobi Machine
Learning
Xavier Nodet

Open Source: Pandas
Zed Dean

Jupyter Notebook Modeling
ExamplesJerry Yurchisin

Q&A
Xavier Nodet

Wrapping up: Games,
Competitions, and Giveaways
Alison Cozad, Lindsay Montanari, Colum Devine© 2023 Gurobi Optimization, LLC., All Rights Reserved | 5

Gurobi 10 Overview

Performance and API Improvements
Major advances in the underlying algorithmic framework

Enterprise Development & Deployment
Experience
New tools for model deployment, monitoring and advanced diagnosis

Innovative Open-Source Projects
Pandas, Machine Learning, and more.

© 2023 Gurobi Optimization, LLC., All Rights Reserved | 6

Note: Version & Support Policy Has Changed
Recent Major/Minor Releases:• Previously

10.0: 2022-November• Annual major or minor release• Two major versions supported

9.5: 2021-November• Along with related minor ones
• Latest technical release only

9.1: 2020-October
9.0: 2019-November• Going Forward• Aim for annual major release• Quarterly technical release• 3 years support for major version• Along with related minor ones
• Latest technical release only

• More predictable for customers

https://support.gurobi.com/hc/en-us/articles/360048138771-Gurobi-release-and-support-history© 2023 Gurobi Optimization, LLC., All Rights Reserved | 7

Linux on ARM Now Supported• Red Hat® Enterprise Linux 7, 8, 9
• CentOS 7• SUSE® Enterprise Linux 12, 15• Ubuntu® 20.04, 22.04• Amazon Linux 2

Other Supported OS’es• Linux® x86-64 64-bit• Windows 64-bit• MacOS 64-bit Universal• Native support for bothM- and -x64 chips• AIX® 64-bit© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 8

Performance Improvements
In Gurobi 10• Concurrent solver• MIP• Mixed Integer Quadratics• Non-Convex MIQCP© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 9

Poll: Gurobi Versions© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 10

The Value of Performance

What is the immediate business advantage of fastersolve times in your solution?

What would happen if you were able to consider more
optimal outcomes in the same time frame?

How could your solution benefit from increased model
complexity or accuracy?

© 2023 Gurobi Optimization, LLC., All Rights Reserved | 11

Performance
Summary

Compare to Gurobi 9.5

Algorithm

Overall speed-up

On >100sec models

LP – default

10%

25%

• New network simplex algorithm

LP – primal simplex

3%

10%

• Concurrent LP improvements:

LP – dual simplex

3%

10%

• Concurrent only on the final presolved
model

MILP

13%

24%

• Threads used to duplicate work

Convex MIQP

57%

2.4x*

• Crossover improvements

Convex MIQCP

28%

88%*

• New and improved presolve reductions

Non-convex MIQCP

51%

2.6x

* MIQP and MIQCP hard model test sets too small to give reliable benchmark results© 2023 Gurobi Optimization, LLC., All Rights Reserved | 12

MIP Performance

Compare to Gurobi 9.5

Algorithm

Overall speed-up

On >100sec models

LP – default

10%

25%

• Strong branching

LP – primal simplex

3%

10%

• Symmetry improvements

LP – dual simplex

3%

10%

• Aggressive solving of sub-MIPs

MILP

13%

24%

• Presolve reductions

Convex MIQP

57%

2.4x*

• Optimization-based bound tightening

Convex MIQCP

28%

88%*

• Models from machine learning

Non-convex MIQCP

51%

2.6x

• Will be discussed in a later
presentation* MIQP and MIQCP hard model test sets too small to give reliable benchmark results© 2023 Gurobi Optimization, LLC., All Rights Reserved | 13

Comparison of Gurobi Versions (PAR-10)

MIP Performance

1600

90
1446

1400

80

1298

1200

1134

70
1065

60

944

1000

50

776

800

40
600

30

454
400

294

20

244

200

135

0

10
0

v1.1

v2.0

v3.0

v4.0

v5.0

unsolved

v6.0

v7.0

v8.0

v9.0

v10.0

speed-up© 2023 Gurobi Optimization, LLC., All Rights Reserved | 14

MIQ Performance

Compare to Gurobi 9.5

Algorithm

Overall speed-up

On >100sec models

LP – default

10%

25%

• New QUBO heuristic

LP – primal simplex

3%

10%

• Perspective strengthening

LP – dual simplex

3%

10%

• Move Q objective terms to constraints

MILP

13%

24%

• Work limit adjustment for QC fixing
heuristics

Convex MIQP

57%

2.4x*

• Fix binary in certain order for heuristics

Convex MIQCP

28%

88%*

Non-convex MIQCP

51%

2.6x

• Solve set covering problem to selectlinearization• Remove common variables* MIQP and MIQCP hard model test sets too small to give reliable benchmark results• Optimization-based bound tightening• Many MIP improvements also apply© 2023 Gurobi Optimization, LLC., All Rights Reserved | 15

Non-Convex MIQCP

Algorithm

Overall speed-up

On >100sec models

LP – default

10%

25%

• Optimization-based bound tightening

LP – primal simplex

3%

10%

• Dealing explicitly with bipartite graphs in the
product term covering

LP – dual simplex

3%

10%

MILP

13%

24%

Convex MIQP

57%

2.4x*

Convex MIQCP

28%

88%*

Non-convex MIQCP

51%

2.6x

• Improvement on NLP heuristic termination• NLP heuristic multi-start• Many MIP and convex MIQCP
improvements also apply* MIQP and MIQCP hard model test sets too small to give reliable benchmark results© 2023 Gurobi Optimization, LLC., All Rights Reserved | 16

Comparison of Gurobi Versions (PAR-10)
120

Non-Convex MIQCP

100

80

70

99

60
80
50

70
60

40

30
40

Time limit: 10,000 sec.
Intel Xeon CPU E3-1240 v5 @ 3.50GHz
4 cores, 8 hyper-threads
32 GB RAM
Test set has 874 models:- 38 discarded due to inconsistent answers- 308 discarded that none of the versions can solve- speed-up measured on >100s bracket: 205 models

29

20

20
10
0

10

0
v9.0

v9.1
unsolved

v9.5

v10.0

speed-up© 2023 Gurobi Optimization, LLC., All Rights Reserved | 17

Can you use these performance increases in new ways?
© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 18

API and Engine Improvements
In Gurobi 10• New logistic general constraint• New Memory limit parameter• Python matrix API improvements• NuGet package for .NET
• Automate Gurobi downloads using NuGet© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 19

Logistic Constraint• Background: Function Constraints in Gurobi• 𝒚 = 𝒇(𝒙)• Gurobi generates piecewise-linear approximation• Examples: sine, power, exponential

1𝑝 𝑥 =1 + 𝑒 '(

• Logistic function added in v10:
• Applications:•

ecology, statistics, medicine, chemistry, …

• Python example:
y = 1 / (1 + exp(-x))
gc = model.addGenConstrLogistic(x, y, params)© 2023 Gurobi Optimization, LLC., All Rights Reserved | 20

Memory Limit Parameter• SoftMemLimit• Memory limit parameter allows graceful exit
• Set a memory limit in GB• Get best solution• Resume optimization• Contrast with existing MemLimitmodel.SoftMemLimit=28• Memory limit is “soft”• Not as strict as MemLimit• Might overshoot• Both are non-deterministic© 2023 Gurobi Optimization, LLC., All Rights Reserved | 21

Matrix API Improvements
In Gurobi 10• Improve usability• Works well with numpy.ndarray’s• Name construction• NumPy functions• Broadcasting• Modeling Performance© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 22

List-Based vs Matrix-Based𝐦𝐢𝐧 𝒄𝑻 % 𝒙
𝒔. 𝒕. 𝑨 % 𝒙 = 𝒃

List-Based API:

Gurobi 10 Matrix API:x1 = [1.0,2.0,3.0]
x2 = [1.0,2.0,0.0]x3 = [3.0,2.0,0.0]b = np.array([4,1,2])c = np.array([1,2,3])
model2 = gp.Model("Pythonic model")x = model2.addVars(len(b))
model2.setObjective(c[0]*x[0]+ c[1]*x[1] + c[2]*x[2])for m in range(b.size):model2.addConstr(x1[m]*x[0]+ x2[m]*x[1]+ x3[m]*x[2] >= b[m])model.optimize()
for var in model.getVars():
print(var.X)

A = np.array([[1.0,2.0,3.0],[1.0,2.0,0.0],[3.0,2.0,0.0]])b = np.array([4,1,2])c = np.array([1,2,3])
model = gp.Model()x = model.addMVar(3, name="x")model.setObjective(c @ x)
model.addConstr(A @ x <= b)model.optimize()print(f"Result values: {x.X}")© 2023 Gurobi Optimization, LLC., All Rights Reserved | 23

Multi-Dimensional Variables, Expressions• MVar, MLinExpr, MQuadExpr support arbitrary dimensions• 9.5 had limited multi-dimensional modeling support
A = np.array([[1.0,2.0,3.0],[1.0,2.0,0.0],[3.0,2.0,0.0]])
print(A)[[1. 2. 3.][1. 2. 0.][3. 2. 0.]]x = model.addMVar((3,1))
print(x)<MVar (3, 1)>
array([[<gurobi.Var C9>],[<gurobi.Var C10>],[<gurobi.Var C11>]])mLinExpr = A @ x
print(mLinExpr)<MLinExpr (3, 1) >array([[ C9 + 2.0 C10 + 3.0 C11],[ C9 + 2.0 C10],[ 3.0 C9 + 2.0 C10]])© 2023 Gurobi Optimization, LLC., All Rights Reserved | 24

Multi-Dimensional Modeling
Adding constraints à multidimensional MConstr or MQConstr
A = np.array([[0.3, 0.6, 0.3],[0.3, 0.3, 0.2],[0.8, 0.8, 0.8],[0.6, 0.3, 0.3]])
B = np.array( [[0.2, 0.4, 0.5, 0.4],[0.9, 0.3, 0.8, 0.6],[0.8, 0.8, 0.1, 0.2],[0.3, 0.0, 0.1, 0.6],[0.2, 0.4, 0.3, 0.0],[0.2, 0.4, 0.7, 0.3]])
X = model.addMVar(3,4)
model.setObjective(X.sum())
mc = model.addConstr(A @ X <= B)<MConstr (6, 4)>© 2023 Gurobi Optimization, LLC., All Rights Reserved | 25

NumPy Functions for Gurobi Matrix Objects• gurobipy v10 implements the most-used ones
• “sum”, “diagonal”, “reshape”• Extract a diagonal from an MVar X:• X.diagonal(offset).

• Convert a list of Var objects to an MVar:• x = MVar.fromlist(varlist)• Sum along an axis of an MVar X:• X.sum(axis=...)• Elementwise squaring of an Mvar X:• pow(X, 2), X**2

© 2023 Gurobi Optimization, LLC., All Rights Reserved | 26

Name Construction• Names now show n-dimensional index valuesz = model.addMVar((3,1), name="Long Name")model.update()
print(z.VarName)[['Long Var Name[0,0]’]['Long Var Name[1,0]’]['Long Var Name[2,0]']]© 2023 Gurobi Optimization, LLC., All Rights Reserved | 27

Broadcasting 1• Operands of different shape:• What should the result shape be?

• NumPy Broadcasting Convention
model = gp.Model()a = model.addMVar(3, name = "a" )b = model.addMVar(1, name ="b")model.update()
result = a * b
print(result)<MQuadExpr (3,)>
array([ 0.0 + [ a[0] * b[0] ], 0.0 + [ a[1] * b[0] ], 0.0 + [ a[2] * b[0] ]])© 2023 Gurobi Optimization, LLC., All Rights Reserved | 28

Broadcasting 2• When a binary operation is applied to operands of different shape, what should
the result shape be?
• Using NumPy Broadcasting Convention
model = gp.Model()X = model.addMVar((4,3),name="X")y = model.addMVar((3,),name="y")model.update()
qe = X + yprint(qe)<MLinExpr (4, 3)> array([[ X[0,0] + y[0], X[0,1] + y[1], X[0,2] + y[2]],[ X[1,0] + y[0], X[1,1] + y[1], X[1,2] + y[2]],[ X[2,0] + y[0], X[2,1] + y[1], X[2,2] + y[2]],[
X[3,0] + y[0], X[3,1] + y[1], X[3,2] + y[2]]])© 2023 Gurobi Optimization, LLC., All Rights Reserved | 29

Matrix API Performance Comparison
m = 1024; n = 8192; d = 0.2
A = sp.random(m, n, d, format='csr')
b = np.random.rand(m)# Using MVarx = model.addMVar(n)model.addConstr(A@x == b)# Iterate One List
x = model.addVars(n).values()
for i in range(m):(_, colidx, colcoef) = sp.find(A[i, :])le = gp.LinExpr(colcoef, [x[j] for j in colidx])
model.addConstr(le == b[i])# Iterate Two Lists(rowidx, colidx, coef) = sp.find(A)x = model.addVars(n).values()
le = [gp.LinExpr() for i in range(m)]for k in range(len(coef)):le[rowidx[k]] += coef[k] * x[colidx[k]]for i in range(m):model.addConstr(le[i] == b[i])Results

Using MVar: 0.083sec.
One List: 1.729sec.
Two Lists: 53.77sec.

© 2023 Gurobi Optimization, LLC., All Rights Reserved | 30

Enterprise Development& Deployment Experience• Compute Server Improvements• New Licensing Options© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 31

New Platform Features• Compute Server• Background• Gurobi in a service architecture• Web UI, security, queuing, more

• Cluster Manager Improvements• Job Dashboard• Node Dashboard• Web License Service• Now available on all environments© 2023 Gurobi Optimization, LLC., All Rights Reserved | 32

Job Dashboard 1• Global Metrics• number of jobs, execution time, active application,
active users• Predefined filters for last 24h, 7 days or 30 days
• More filtering available• Distributions• Shown in several dimensions• Job and solve statuses, applications, users,
runtimes• Drill down to job list

© 2023 Gurobi Optimization, LLC., All Rights Reserved | 33

Job Dashboard 2• Timeline by several dimensions• Applications, Job/Solve statuses, Users,
Runtime and solve times• Zoom and pan over time• Legend and colors to differentiate
values• Drilldown• Go to the job list of the selected period• Filter the dashboard with the selectedperiod© 2023 Gurobi Optimization, LLC., All Rights Reserved | 34

Node Dashboard• Global Overview• CPU, Memory,• Job in queue and running• Predefined filters for last 24h, 7 days or 30 days

© 2022 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 35

Node Dashboard Detail• Timeline• CPU usage• Memory usage• Job in queue• Running jobs

• Drilldown• Zoom, pan• Node selection© 2023 Gurobi Optimization, LLC., All Rights Reserved | 36

New WLS Licensing Options• WLS = Web License Server• Gurobi-hosted license server• Limits or counts:• Cores• Concurrent solves• Use from anywhere• Limited communication from client• Hostname, IP Address, Core count, OS, Version• No model data

• New options for physical machines & VM’s
• Physical Machines (Linux, Windows, Mac)
• Virtual Machines• Containers© 2023 Gurobi Optimization, LLC., All Rights Reserved | 37

WLS Sessions• Reports now show sessions instead of containers• Sessions = Continuous usage of Gurobi in a container or a machine© 2023 Gurobi Optimization, LLC., All Rights Reserved | 38

WLS Metrics• Reports of versions and platforms© 2023 Gurobi Optimization, LLC., All Rights Reserved | 39

Innovative Open-Source Projects© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 40

Gurobi 10.0 – Open-Source GitHub Repositories• Gurobi Machine Learning• Use machine learning model as MIP constraint• Special session later in this workshop• Gurobipy pandas• Gurobipy model building patterns with pandas• Separate session later in this workshop• Coming soon...
• Gurobi OptiMods• Optimization modules for specific applications• Numerical issues assessment tool
• Analyze root cause of numerical issues© 2023 Gurobi Optimization, LLC., All Rights Reserved | 41

Summary© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 42

Summary

Performance and API Improvements
Major advances in the underlying algorithmic framework

Enterprise Development & Deployment
Experience
New tools for model deployment, monitoring and advanced diagnosis

Innovative Open-Source Projects
Pandas, Machine Learning, and more.

© 2023 Gurobi Optimization, LLC., All Rights Reserved | 43

Gurobipy-pandas
PandasEnables convenient gurobipy model building patterns
with pandas
Zed DeanTechnical Account Manager© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved

AgendaRationale
What is gurobipy-pandas ?
Who is it for ?
Installation & dependencies
Usage Jupyter Notebook
Free Function vers Pandas
accessor
Arithmetic Expressions
Performance© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 45

How do I read and write tabular data with Gurobi ?

Source:How do I read and write tabular data? — pandas 2.0.0 documentation (pydata.org)© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 46

Our Goal• Simplify the process of importing Data Tables into optimization
model.

© 2022 ©
Gurobi
LLC. Confidential,
All Rights| 47
2023 Optimization,
Gurobi Optimization,
LLC. Confidential,
AllReserved
Rights Reserved| 47

What is gurobipy-pandas
gurobipy-pandas is a convenient (optional) wrapper toconnect pandas with gurobipy..

© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 48

Audience
gurobipy-pandas is aimed at experienced pandas users who
are familiar with methods to transform, group, andaggregate data stored in dataframes. It expects somefamiliarity with optimization modelling, but does not require
deep experience with gurobipy© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 49

Gurobipy and pandas
Documentation and examples for users of the PyData stack• Open source, with documentation available on readthedocs.com• Github repository: https://github.com/gurobi/gurobipy-pandas• Documentation: https://gurobi-optimization-gurobipy-pandas.readthedocs-hosted.com• Complete model building examples as Jupyter notebooks• Guidance for writing performant gurobipy-pandas code

© 2022 ©
Gurobi
LLC. Confidential,
All Rights| 50
2023 Optimization,
Gurobi Optimization,
LLC. Confidential,
AllReserved
Rights Reserved| 50

gurobipy-pandas Dependencies & Installation

Dependencies

Standard Imports• gurobipy: Python modellinginterface for the Gurobi Optimizer• pandas: powerful Python data
analysis toolkit• Most gurobipy-pandas
applications will start with the following
imports.

• !pip install gurobipy-pandas© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 51

UsageA mathematical optimization
model has five components,
namely:• Sets and indices.
• Parameters.
• Decision variables.
• Constraints• Objective function(s).

• Optimization models define all data,
variables, and constraints overindexes:• These mathematical indices provide aclear way to structure data in code.
© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 52

Gurobipy and pandas
Easier model building with the popular Python data analytics package• Create pandas Series and DataFrames of Gurobi variables• Use pandas operations to combine variables and data into constraints• Extract solution data as pandas Series• No need to manually translate between pandas and gurobipy!

© 2022 ©
Gurobi
LLC. Confidential,
All Rights| 53
2023 Optimization,
Gurobi Optimization,
LLC. Confidential,
AllReserved
Rights Reserved| 53

Usage Jupyter Notebook Demo
• Create Model• Create variables• Use Pandas’s built-in Expressions• Create Constraints• Set Objective• Solve• Extract solutions© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 54

Two different ways to build the model with gppd
Using Free functions• varsname=gppd.add_vars(model, pandas_obj, *, name=None, lb=0.0, ub=1e+100, obj=0.0,
vtype='C', index_formatter='default’)¶• contraintsname=gppd.add_constrs(model, lhs, sense, rhs, *, name=None, index_formatter='default')
Using Methods• Dataframe.gppd.add_vars(model, *, name, lb=0.0, ub=1e+100, obj=0.0, vtype='C', index_f
ormatter='default')• add_constrs(model, lhs, sense=None, rhs=None, *, name, index_formatter='default')© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 55

Arithmetic Expressions• - Pandas handles this for us• - We always leverage pandas-native functions• - Common operations:•- summation•- arithmetic operations•- groupby (split-apply-combine) operations• - Let's explore some common mathematical expressions© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 57

Performance`gurobipy-pandas` won't magically make your model building code fast•

Best used where your inputs and outputs *already* use pandas•

Data *must* be properly-organized and prepared•

Prescribed style aims to *avoid performance pitfalls*• Performance — gurobipy-pandas 1.0.0 documentation (readthedocs-hosted.com)© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 58

Thank You
For more information: gurobi.com
Zed DeanTechnical Account Manager© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved

Gurobi
Machine Learning
Using trained machine learning predictors in Gurobipy
Xavier Nodet
Development Manager© 2023 Gurobi Optimization, LLC., All Rights Reserved

AgendaMotivating example
Gurobi Machine Learning
Other use-cases
Gurobi 10.0

© 2023 Gurobi Optimization, LLC., All Rights Reserved | 61

Motivating example
Optimizing avocado shipments and prices© 2023 Gurobi Optimization, LLC. All
Confidential,
Rights Reserved
All Rights| 62Reserved | 62

Motivating Example: Price Optimization• Selling avocados in the US• Market is split in regions 𝑅• Total supply 𝑆• Maximize profit:• (sales – shipping costs – unsold penalty)
with given• shipping costs 𝑐! and waste penalty 𝑤• We decide about the prices 𝑝!

https://youtu.be/AJRP9pPBx6s• and estimate the demands d" from the prices© 2023 Gurobi Optimization, LLC., All Rights Reserved | 63

Motivating Example: Estimating Demand• Historical data of avocado sales from
Hass Avocado Board (HAB) available
on Kaggle and HAB website• Features correlated to demand:
year, peak season, region, price• Linear regression gives reasonably
good prediction of demand with those:𝑑 = 𝑔(𝑦𝑒𝑎𝑟, 𝑝𝑒𝑎𝑘, 𝑟, 𝑝)• In the case of linear regression, 𝑔 is anaffine function𝑑 = 𝜙 # 𝑦𝑒𝑎𝑟, 𝑝𝑒𝑎𝑘, 𝑟, 𝑝 + 𝜙$

© 2023 Gurobi Optimization, LLC., All Rights Reserved | 64

Motivating Example: Retrieving ML results• Retrieve results from linear regression model
coef_dict = model.fit().params.to_dict()• Build linear expression defining expected demand from priced = {r: (coef_dict['Intercept’] +coef_dict['price’] * p[r] +coef_dict['C(region)[T.%s]'%r] +coef_dict['year_index’] * (year - 2015) +coef_dict['peak’] * peak_or_not)
for r in R}• Upper bound on sales is expected demandm.addConstrs((s[r] <= d[r] for r in R))© 2023 Gurobi Optimization, LLC., All Rights Reserved | 65

Motivating Example: Optimization Model• Indices• r ∈ 𝑅, the set of regions• Indices• r ∈ 𝑅, the set of regions• Constants• 𝑐! transportation costs• 𝑤 penalty for waste• 𝑆 total supply• Year and season of interest• Variables• p" selling price per unit
• d" demand• u total unsold products

max ∑) 𝑝) − 𝑐) 𝑑) − 𝑤 ∗ 𝑢𝑠. 𝑡.
∑) 𝑑) + 𝑢 = 𝑆𝑑) = 𝑔 𝑦𝑒𝑎𝑟, 𝑝𝑒𝑎𝑘, 𝑟, 𝑝) for r ∈ 𝑅(maximize revenue)(allocate shipments)(define demand with regression model)© 2023 Gurobi Optimization, LLC., All Rights Reserved | 66

Motivating Example: What if?
• With 𝑔 an affine function, the resulting model is a non-convex QP• Solved fast with Gurobi• But what if we need a more accurate prediction with a more complex regression:• Decision tree, Neural network, …

max ∑) 𝑝) − 𝑐) 𝑑) − 𝑤 ∗ 𝑢𝑠. 𝑡.
∑) 𝑑) + 𝑢 = 𝑆𝑑) = 𝑔 𝑦𝑒𝑎𝑟, 𝑝𝑒𝑎𝑘, 𝑟, 𝑝) for r ∈ 𝑅(maximize revenue)(allocate shipments)(define demand with regression model)© 2023 Gurobi Optimization, LLC., All Rights Reserved | 67

Motivating Example: Goals

1. Simplify the process of importing a trained machine learning model built with apopular ML package into an optimization model.
2. Improve algorithmic performance to enable the optimization model to explore asizable space of solutions that satisfy the relationships between variables captured
with the ML model.

© 2023 Gurobi Optimization, LLC., All Rights Reserved | 68

Gurobi Machine Learning
An open-source Python package© 2023 Gurobi Optimization, LLC. All
Confidential,
Rights Reserved
All Rights| 69Reserved | 69

Gurobi Machine Learning• Open source python package• https://github.com/Gurobi/gurobi-machinelearning• https://gurobi-machinelearning.readthedocs.io/• Apache License 2.0• Initial release 1.0.0 last November• Version 1.1.1 recently two days ago• Experimental package• Not supported through usual Gurobi processes© 2023 Gurobi Optimization, LLC., All Rights Reserved | 70

Known regression models• Linear/Logistic/PLS regression• Decision trees• Neural network with ReLU activation• Random Forests• Gradient Boosting• Preprocessing:• Simple scaling• Polynomial features of degree 2• Column transformers• pipelines to combine them

• Dense layers• ReLU layers• Object Oriented, functional orsequential• Dense layers• ReLU layers• Only torch.nn.Sequential models© 2023 Gurobi Optimization, LLC., All Rights Reserved | 71

Example: Define and train ML model...
lin_reg = make_pipeline(columntransformer, LinearRegression())
lin_reg.fit(X_train, y_train)© 2023 Gurobi Optimization, LLC., All Rights Reserved | 72

Example: Creating the variables
import gurobipy_pandas as gppd
data = pd.concat(...)
m = gp.Model("Avocado_Price_Allocation")p = gppd.add_vars(m, data, lb=0.0, ub=2.0)
d = gppd.add_vars(m, data)u = m.addVar()• Variables• 𝑝! selling price per unit
• 𝑑! demand• 𝑢 total unsold products

m.setObjective(((p – c) * d).sum() - w * u, GRB.MAXIMIZE)
m.addConstr(d.sum() + u == S)© 2023 Gurobi Optimization, LLC., All Rights Reserved | 73

Example: Adding regression constraints
𝑑! = 𝑔 𝑦𝑒𝑎𝑟, 𝑝𝑒𝑎𝑘, 𝑟, 𝑝! for r ∈ 𝑅.
fixed = pd.DataFrame(
data={"year": 2020,"peak": 1,"region": regions,},index=regions)feats = pd.concat([fixed, p],axis=1)© 2023 Gurobi Optimization, LLC., All Rights Reserved | 74

Example: Adding Regression Constraints
from gurobi_ml import add_predictor_constr
pred_constr = add_predictor_constr(m, lin_reg, feats, d)
pred_constr.print_stats()© 2023 Gurobi Optimization, LLC., All Rights Reserved | 75

Example: Optimizing
m.Params.NonConvex = 2
m.optimize()© 2023 Gurobi Optimization, LLC., All Rights Reserved | 76

Example: Solution© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 77

Remarks• Function add_predictor_constr creates the formulation for regression
model and returns a modeling object.
• If input of add_predictor_constr has several rows, it introduces one submodel for each row• Query statistics about modeling object, remove it, query solution after solve anderror in solutions.
• Models for logistic regression use a piecewise linear approximation and can havemodeling error (controlled by parameters).
• Models for decision tree can also introduce small errors at threshold values ofnode splitting (can be controlled).

© 2023 Gurobi Optimization, LLC., All Rights Reserved | 78

Other use-cases
More reasons to use
gurobi machine learning© 2023 Gurobi Optimization, LLC. All
Confidential,
Rights Reserved
All Rights| 79Reserved | 79

Student enrollment• Train a ML model to predict student enrollment in college from scholarshipoffered (“merit”), SAT and GPA scores.
• Build an optimization model to decide:• Scholarship offered to each student• To maximize• The (predicted) number of students enrolled• Subject to:• Budget constraint and bounds on individual merit.

• Use gurobi machine learning to set the value of the enroll variable, given
the merit offered and the SAT and GPA scores of each student© 2023 Gurobi Optimization, LLC., All Rights Reserved | 80

Surrogate models• When the modeling of a complex process requires
highly non-linear functions, or simulation• Approximate this function using e.g. a Neural Network with ReLU activation• Use gurobi machine learning to create a ‘surrogate constraint’• For a network with polynomial features, the resulting model is a non-convex QCP© 2023 Gurobi Optimization, LLC., All Rights Reserved | 81

Adversarial Machine Learning• Suppose we have a trained neural network,• And a well-classified example 𝑥• Create another example 𝑥̅ close to 𝑥 that is classified with a different label• Example from the MNIST handwritten digits database

Classified as a 4Classified as a 9© 2023 Gurobi Optimization, LLC., All Rights Reserved | 82

Gurobi 10.0
Performance improvements for models with ML
predictor constraints© 2023 Gurobi Optimization, LLC. All
Confidential,
Rights Reserved
All Rights| 83Reserved | 83

Gurobi 9.5 vs Gurobi 10.0
• Models incorporating Neural
Networks• Performance gains come mostly
from Optimization Based Bound
Tightening (OBBT)• 478 instances• 10.000 seconds time limit• Intel(R) Xeon(R) CPU E3-1240 CPUs, 4 cores, 4
threads• Solved means 0.01% gap reached• Models not solved by any version were
excluded© 2023 Gurobi Optimization, LLC., All Rights Reserved | 84

Conclusion© 2023 Gurobi Optimization, LLC. All
Confidential,
Rights Reserved
All Rights| 85Reserved | 85

Conclusion• Gurobi Machine Learning
https://github.com/Gurobi/gurobi-machinelearning• This is experimental• We seek your input• Performance for models with Neural Networks in Gurobi 10
Dangers and Pitfalls• ML models we can hope to handle is still limited (e.g. no categorical features)• Methodological issues:• How to decide which prediction model to use?
• How to make sure that optimization doesn’t misuse results of the predictor?
© 2023 Gurobi Optimization, LLC., All Rights Reserved | 86

Notebook Example:
Avocado Price Optimization
www.gurobi.com/jupyter_models/avocado-price-optimization/
Jerry Yurchisin
Data Science Strategist© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved

INFORMS GUROBI TUTORIALS:
Gurobi's Newest Educational Resources: Where Data Meets
Decisions - An Overview of Our Free Jupyter Notebook Data
Science Example Library
Monday, April 17, 2022 from 10:30 – 11:20AM – Cottonwood 10
Gurobi Machine Learning: Incorporate your Machine Learning
Models into Optimization
Tuesday, April 18, 2022 from 9:10 – 10:00AM – Cottonwood 11

Thank YouFor more on our Notebook Example Library:www.gurobi.com/jupyter_models/
Jerry Yurchisin

Data Science Strategist© 2023 Gurobi Optimization, LLC. Confidential, All Rights Reserved

QUESTIONS?

Thank YouFor more information: gurobi.com

VISIT US AT BOOTH #217 TO LEARN MORE!

© 2023 Gurobi Optimization, LLC., All Rights Reserved

Gurobipy Card Game
Download the complete game at gurobi.com/cardgame

This game asks learners to learn to write code by first
stepping away from the computer to match the piecesof a math model to optimization code.
© 2023 Gurobi Optimization, LLC., All Rights Reserved | 90

GIVEAWAY

Thank YouFor more information: gurobi.com

VISIT US AT BOOTH #217 TO LEARN MORE!

© 2023 Gurobi Optimization, LLC., All Rights Reserved

Examples of New Capabilities© 2023 Gurobi Optimization, LLC., All Rights Reserved | 92

Numpy Copies & Views• Create View through indexing• Create Copy through advance indexingCopies & Views• In case of doubt, always copy yourslice...

Improved matrix-friendly gurobipy
Gurobi 10.0

• More modeling capabilities for users familiar with NumPy conventions• Enable numpy-style axis sums
• >>> x = m.addMVar((2,4), name='x', vtype=GRB.BINARY)• >>> x.sum(axis=1)• <MLinExpr (3,)>•([<gurobi.LinExpr: x[0,0] + x[0,1] + x[0,2] + x[0,3]>,•<gurobi.LinExpr: x[1,0] + x[1,1] + x[1,2] + x[1,3]>• >>> m.addConstr(x.sum(axis=1) <= 1)

4 𝑥!" ≤ 1 ∀𝑖 ∈ 𝐼"• Enable broadcasting operations for constraint building• >>> x = m.addMVar((3,4), name='x', vtype=gp.GRB.BINARY)• >>> y = m.addMVar(4, name='y', vtype=gp.GRB.BINARY)• >>> m.addConstr(x <= y) # 12 constraints added𝑥!" ≤ 𝑦" ∀ 𝑖 ∈ 𝐼, 𝑗 ∈ 𝐽© 2022 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 95
© 2023 Gurobi Optimization, LLC., All Rights Reserved | 95

• Shape of operation now consistent with analogous ndarray operations• Elementwise multiplication works across all matrix-friendly objects and ndarray• Mvar• Extract a diagonal from an MVar X:• X.diagonal(offset).

• Convert a list of Var objects to an MVar:• x = MVar.fromlist(varlist)• Sum along an axis of an MVar X:• X.sum(axis=...)• Elementwise squaring of an Mvar X:• pow(X, 2), X**2

• MLinExpr• All-zero expression: MLinExpr.zeros(shape)• Sum along an axis of an MLinExpr mle:
• mle.sum(axis=..
© 2023 Gurobi Optimization, LLC., All Rights Reserved | 96

© 2022 Gurobi Optimization, LLC. Confidential, All Rights Reserved | 97
© 2023 Gurobi Optimization, LLC., All Rights Reserved | 97