Controlling variability in power systems
Daniel Bienstock
Columbia University

APAM Nov 17 2017

Bienstock
Controlling variability in power systems

Columbia University

A simple example:

100
100

Bienstock
Controlling variability in power systems

Columbia University

A simple example:

100
100

Only one solution:
200

100

100
200

Bienstock
Controlling variability in power systems

100
100

Columbia University

A simple example:

100
100

Only one solution:
200

100

100
200

100
100

But what if the red node suddenly injects power?
Bienstock
Controlling variability in power systems

Columbia University

100
100

Only one solution:
200

100

100
200

Bienstock
Controlling variability in power systems

100
100

Columbia University

100
100

Only one solution:
200

100

100
100

200

100

If red node suddenly injects power, offset using blue node:
200

100
50
50
50

200

Bienstock
Controlling variability in power systems

100

100
100
Columbia University

Better example: red node is cheap but unreliable source

100

50 + error
100

Base case solution:
150

50

50

100
150

50

100
100

If the red node suddenly injects power, offset using blue node:
150

50

50 + D
D
100

150

100

50 + D
100

Bienstock
Controlling variability in power systems

Columbia University

Examples:30 − 15
160

Y

X

100

160

75 + X75

0Y
10 − 10100100

capacity = 15

Any combination with X + Y = 25 “works” so long as Y ≤ 15.
30 − 20
70

160
160

100

70 + X

18 + Y

X

18
Y

100

10 + 8100

Any combination with X + Y = 12 “works” so long as
Y ≤ −3.
Bienstock
Controlling variability in power systems

Columbia UniversityPower engineering for non-engineers
conductor
rotor

steam

statormagneticfieldωenergysourceBienstock
Controlling variability in power systems

current, voltage

Columbia UniversityPower engineering for non-power engineers

Bienstock
Controlling variability in power systems

Columbia UniversityPower engineering for non-power engineers

Bienstock
Controlling variability in power systems

Columbia UniversityPower engineering for non-power engineers

Bienstock
Controlling variability in power systems

Columbia UniversityPower engineering for non-power engineers

Bienstock
Controlling variability in power systems

Columbia University

AC Power Flows

Bienstock
Controlling variability in power systems

Columbia University

AC Power Flows
Real-time:

Bienstock
Controlling variability in power systems

Columbia University

AC Power Flows
Real-time:
I(t) km
k
V(t)

m
k

Voltage at bus k: vk (t) = Vkmax cos(ωt + θkV )max cos(ωt + θ I ).
Current injected at k into km: ikm (t) = Ikm
km

Bienstock
Controlling variability in power systems

Columbia University

AC Power Flows
Real-time:
I(t) km
k
V(t)

m
k

Voltage at bus k: vk (t) = Vkmax cos(ωt + θkV )max cos(ωt + θ I ).
Current injected at k into km: ikm (t) = Ikm
km

Power injected at k into km: pkm (t) = vk (t)ikm (t).
Averaged over period T :RT.
max cos(θ V − θ I ).
pkm = T1 0 p(t) = 12 Vkmax Ikm
kkm

Bienstock
Controlling variability in power systems

Columbia University

I(t) km
k
V(t)

.
pkm =

1
T

RT
0

Bienstock
Controlling variability in power systems

p(t) =

mk

1 max maxI )Ikm cos(θkV − θkm
2 Vk

Columbia University

I(t) km
k
V(t)

.
pkm =

1
T

RT
0

p(t) =

mk

1 max maxI )Ikm cos(θkV − θkm
2 Vk
Vvk (t) = Vkmax Re e j(ωt+θk ) ,Bienstock
Controlling variability in power systems

I

max Re e j(ωt+θkm )
ikm (t) = Ikm

Columbia University

I(t) km
k
V(t)

.
pkm =

1
T

RT
0

p(t) =

mk

1 max maxI )Ikm cos(θkV − θkm
2 Vk
Vvk (t) = Vkmax Re e j(ωt+θk ) ,. Vkmax jθkV
Vk = √e ,2

Bienstock
Controlling variability in power systems

I

max Re e j(ωt+θkm )
ikm (t) = Ikm
max
I
. Ikm
Ikm = √
e jθmk2

Columbia University

I(t) km
k
V(t)

.
pkm =

1
T

RT
0

p(t) =

mk

1 max maxI )Ikm cos(θkV − θkm
2 Vk
Vvk (t) = Vkmax Re e j(ωt+θk ) ,. Vkmax jθkV
Vk = √e ,2I

max Re e j(ωt+θkm )
ikm (t) = Ikm
max
I
. Ikm
Ikm = √
e jθmk2I ) = Re(V I ∗ )pkm = |Vk ||Ikm | cos(θkV − θkm
k km

Bienstock
Controlling variability in power systems

Columbia University

I(t) km
k
V(t)

.
pkm =

1
T

RT
0

p(t) =

mk

1 max maxI )Ikm cos(θkV − θkm
2 Vk
Vvk (t) = Vkmax Re e j(ωt+θk ) ,. Vkmax jθkV
Vk = √e ,2I

max Re e j(ωt+θkm )
ikm (t) = Ikm
max
I
. Ikm
Ikm = √
e jθmk2I ) = Re(V I ∗ )pkm = |Vk ||Ikm | cos(θkV − θkm
k km

.
∗ )
qkm = Im(Vkm Ikm
Bienstock
Controlling variability in power systems.
and Skm = pkm + jqkm
Columbia University

. V max V
Vk = √k 2 e jθk ,

max
I
. Ikm
e jθmk (voltage, current)
Ikm = √
2∗
pkm = Re(Vk Ikm),

Bienstock
Controlling variability in power systems∗
qkm = Im(Vkm Ikm)(1)

Columbia University

. V max V
Vk = √k 2 e jθk ,

max
I
. Ikm
e jθmk (voltage, current)
Ikm = √
2∗
pkm = Re(Vk Ikm),∗
qkm = Im(Vkm Ikm)(1)

Ikm = y{k,m} (Vk − Vm ),Bienstock
Controlling variability in power systems

Columbia University

. V max V
Vk = √k 2 e jθk ,

max
I
. Ikm
e jθmk (voltage, current)
Ikm = √
2∗
pkm = Re(Vk Ikm),∗
qkm = Im(Vkm Ikm)(1)

Ikm = y{k,m} (Vk − Vm ), y{k,m} = admittance of km.

Bienstock
Controlling variability in power systems(2)

Columbia University

. V max V
Vk = √k 2 e jθk ,

max
I
. Ikm
e jθmk (voltage, current)
Ikm = √
2∗
pkm = Re(Vk Ikm),∗
qkm = Im(Vkm Ikm)(1)

Ikm = y{k,m} (Vk − Vm ), y{k,m} = admittance of km.

(2)

Network EquationskBienstock
Controlling variability in power systems

Columbia University

. V max V
Vk = √k 2 e jθk ,

max
I
. Ikm
e jθmk (voltage, current)
Ikm = √
2∗
pkm = Re(Vk Ikm),∗
qkm = Im(Vkm Ikm)(3)

Ikm = y{k,m} (Vk − Vm ), y{k,m} = admittance of km.

(4)

Network EquationsX
km∈δ(k)Bienstock
Controlling variability in power systems

pkm = P̂k ,X

qkm = Q̂k∀k

(5)km∈δ(k)Columbia University

. V max V
Vk = √k 2 e jθk ,

max
I
. Ikm
e jθmk (voltage, current)
Ikm = √
2∗
pkm = Re(Vk Ikm),∗
qkm = Im(Vkm Ikm)(3)

Ikm = y{k,m} (Vk − Vm ), y{k,m} = admittance of km.

(4)

Network EquationsX
km∈δ(k)pkm = P̂k ,X

qkm = Q̂k∀k

(5)km∈δ(k)Generator: P̂k , |Vk | given. Other buses: P̂k , Q̂k given.

Bienstock
Controlling variability in power systems

Columbia University

Optimal power flow (economic dispatch, tertiary control)

Used periodically to handle the next time window(e.g. 15 minutes, one hour)
Choose generator outputs
Minimize cost (quadratic)
Satisfy demands, meet generator and network constraints
Constant load (demand) estimates for the time window
Bienstock
Controlling variability in power systems

Columbia University

DC-OPF:
min c(p)(convex piecewise-linear or quadratic)s.t.
Bθ = p − d|yij (θi − θj )| ≤ uij
Pgmin ≤ pg ≤ Pgmax(6)for each line ij(7)for each bus g(8)Notation:p = vector of generations ∈ Rn ,
n×n

B ∈R
∀i, j :

d = vector of loads ∈ Rn,(bus susceptance matrix)ij ∈ E (set of lines)
 P −yij ,
y
,
i =j
Bij =
kj
k;{k,j}∈E0,otherwise

Bienstock
Controlling variability in power systems

Columbia University

Managing changing demands

Bienstock
Controlling variability in power systems

Columbia University

Bienstock
Controlling variability in power systems

Columbia University

Bienstock
Controlling variability in power systems

Columbia University

What happens when there is a generation/load mismatch
conductor
rotor

steam

statormagneticfieldωenergysourceBienstock
Controlling variability in power systems

current, voltage

Columbia University

What happens when there is a generation/load mismatch
conductor
rotor

steam

statormagneticfieldωenergysourcecurrent, voltage

Frequency response:

Bienstock
Controlling variability in power systems

Columbia University

What happens when there is a generation/load mismatch
conductor
rotor

steam

statormagneticfieldωenergysourcecurrent, voltage

Frequency response:
mismatch ∆P
Bienstock
Controlling variability in power systems

Columbia University

What happens when there is a generation/load mismatch
conductor
rotor

steam

statormagneticfieldωenergysourcecurrent, voltage

Frequency response:
mismatch ∆P ⇒ frequency change ∆ω ≈ −c ∆P
Bienstock
Controlling variability in power systems

Columbia University

The swing equation

H ω̇ = pm (t) − pe (t) − Dω
ω = ω(t) = frequency
pm (t) = mechanical power supplied by motorpe (t) = electrical power supplied by motor
D > 0 (tamping)

Bienstock
Controlling variability in power systems

Columbia University

Managing changing demands
1

Primary frequency control. Handles instantaneous (small)changes.

Bienstock
Controlling variability in power systems

Columbia University

Managing changing demands
1

Primary frequency control. Handles instantaneous (small)changes. Agent: physics.

Bienstock
Controlling variability in power systems

Columbia University

Managing changing demands
1

Primary frequency control. Handles instantaneous (small)changes. Agent: physics.

2

Secondary control. Handles changes that span more than afew seconds.

Bienstock
Controlling variability in power systems

Columbia University

Managing changing demands
1

Primary frequency control. Handles instantaneous (small)changes. Agent: physics.

2

Secondary control. Handles changes that span more than afew seconds. Agent: algorithms, pre-set controls.

Bienstock
Controlling variability in power systems

Columbia University

Managing changing demands
1

Primary frequency control. Handles instantaneous (small)changes. Agent: physics.

2

Secondary control. Handles changes that span more than afew seconds. Agent: algorithms, pre-set controls.

3“Tertiary” control: OPF (Optimal power flow). Manages
longer lasting changes. Run every few minutes. Goal:
economic generation that meets demands while maintaining
feasibility (stability).

Bienstock
Controlling variability in power systems

Columbia University

Managing changing demands
1

Primary frequency control. Handles instantaneous (small)changes. Agent: physics.

2

Secondary control. Handles changes that span more than afew seconds. Agent: algorithms, pre-set controls.

3“Tertiary” control: OPF (Optimal power flow). Manages
longer lasting changes. Run every few minutes. Goal:
economic generation that meets demands while maintaining
feasibility (stability). Agent: algorithmic computations,humans.

Bienstock
Controlling variability in power systems

Columbia University

Managing changing demands
1

Primary frequency control. Handles instantaneous (small)changes. Agent: physics.

2

Secondary control. Handles changes that span more than afew seconds. Agent: algorithms, pre-set controls.

3“Tertiary” control: OPF (Optimal power flow). Manages
longer lasting changes. Run every few minutes. Goal:
economic generation that meets demands while maintaining
feasibility (stability). Agent: algorithmic computations,humans.

4

Once (?) a day: unit commitment problem. Chooses which
generators will operate in the next day or half-day.

Bienstock
Controlling variability in power systems

Columbia University

Managing changing demands
1

Primary frequency control. Handles instantaneous (small)changes. Agent: physics.

2

Secondary control. Handles changes that span more than afew seconds. Agent: algorithms, pre-set controls.

3“Tertiary” control: OPF (Optimal power flow). Manages
longer lasting changes. Run every few minutes. Goal:
economic generation that meets demands while maintaining
feasibility (stability). Agent: algorithmic computations,humans.

4

Once (?) a day: unit commitment problem. Chooses which
generators will operate in the next day or half-day. Agent:algorithms, humans.

Bienstock
Controlling variability in power systems

Columbia University

Bienstock
Controlling variability in power systems

Columbia University

CIGRE -International Conference on Large High Voltage
Electric Systems ’09
Large unexpected fluctuations in wind power can cause
additional flows through the transmission system (grid)
Large power deviations in renewables must be balanced byother sources, which may be far away
Flow reversals may be observed – control difficult
A solution – expand transmission capacity! Difficult(expensive), takes a long time
Problems already observed when renewable penetration high

Bienstock
Controlling variability in power systems

Columbia University

CIGRE -International Conference on Large High Voltage
Electric Systems ’09“Fluctuations” – 15-minute timespanDue to turbulence (“storm cut-off”)
Variation of the same order of magnitude as mean
Most problematic when renewable penetration starts toexceed 20 − 30%
Many countries are getting into this regime

Bienstock
Controlling variability in power systems

Columbia University

Control model1w̄i + wi = output of renewable at bus i .
w̄i = forecast, wi = error (uncertain).

2δj = response at bus j .

Generic linear control:δj = −

X

αji wii

A: matrix of all values αji ;Bienstock
Controlling variability in power systems(A, . . . , A) ∈ K

Columbia University

Control model1w̄i + wi = output of renewable at bus i .
w̄i = forecast, wi = error (uncertain).

2δj = response at bus j .

Generic linear control:δj = −

X

αji wii

A: matrix of all values αji ;(A, . . . , A) ∈ K
Pe.g.
j (1 − αji ) = 0 ∀i for “full-dimensional”
uncertainty set.
Bienstock
Controlling variability in power systems

Columbia University

Chance-constrained problems (one period)
Optimization Problem
min
g

P ,A

X

ck (Pkg )ks.t. the following system is feasible:→ Flow balance:
linear control
renewablesz }| {z }| {Aw− Pd
B θ = P + w̄ + w −g→ Line limits:max
P(|fkm | > fkm) < km∀ km(ignoring e.g. generator constraints)
Bienstock
Controlling variability in power systems

Columbia University

Safety-constrained problems (one period)
Optimization Problem

min
g

P ,A

X

ck (Pkg )ks.t. the following system is feasible:→ Flow balance:
linear control
renewablesz }| {z }| {
B θ = P + w̄ + w −
Aw
− Pd
g

→ Line limits:max|E(fkm )| + νkm Std(fkm ) ≤ fkm

Bienstock
Controlling variability in power systems∀ km
Columbia University

Optimization Problem

min
g

P ,A

X

ck (Pkg )ks.t. the following system is feasible:→ Flow balance:
linear control
renewablesz }| {z }| {Aw− Pd
B θ = P + w̄ + w −g→ Line limits,

∀ km

max
T y |E(P g + w̄ + w − A w − P d )| + νπkm
kmkm Std(fkm ) ≤ fkm ,T = “shift factors” → from row-differences of pseudo-inverse of B
πkm

Optimization Problem

min
g

P ,A

X

ck (Pkg )ks.t. the following system is feasible:→ Flow balance:
linear control
renewablesz }| {z }| {Aw− Pd
B θ = P + w̄ + w −g→ Line limits,
T y |E(P g
πkm
km

∀ km

max
+ w̄ + w − A w − P d )| + νkm Std(fkm ) ≤ fkm,→ from row-differences of pseudo-inverse of B

Optimization Problem

min
g

P ,A

X

ck (Pkg )ks.t. the following system is feasible:→ Flow balance:
linear control
renewablesz }| {z }| {Aw− Pd
B θ = P + w̄ + w −g→ Line limits,

∀ km

max
T y |E(P g + w̄ + w − A w − P d )| + νπkm
kmkm Std(fkm ) ≤ fkm ,T = “shift factors” → from row-differences of pseudo-inverse of B
πkm
Bienstock
Controlling variability in power systems

Columbia University

Optimization Problem

min
g

P ,A

X

ck (Pkg )ks.t.
→ Flow balance:X(1 − αji ) = 0 ∀i;
ji→ Line limits,Tπkm
ykm |E(P g

X g(Pi + w̄i − Pid ) = 0

∀ km

max
+ w̄ + w − A w − P d )| + νkm Std(fkm ) ≤ fkm

Bienstock
Controlling variability in power systems

Columbia University

E(P g + w̄ + w − A w − P d ) = P g + w̄ − P dVar(fij ) = yij2 πijT (I − A)Ω(I − AT )πij ;Bienstock
Controlling variability in power systems

Columbia University

E(P g + w̄ + w − A w − P d ) = P g + w̄ − P dVar(fij ) = yij2 πijT (I − A)Ω(I − AT )πij ;Ω = covariance of w .

Bienstock
Controlling variability in power systems

Columbia University

E(P g + w̄ + w − A w − P d ) = P g + w̄ − P dVar(fij ) = yij2 πijT (I − A)Ω(I − AT )πij ;Ω = covariance of w .
Yields SOCP formulation for safety-constrained problem.

Bienstock
Controlling variability in power systems

Columbia University

E(P g + w̄ + w − A w − P d ) = P g + w̄ − P dVar(fij ) = yij2 πijT (I − A)Ω(I − AT )πij ;Ω = covariance of w .
Yields SOCP formulation for safety-constrained problem.
Caution!

Bienstock
Controlling variability in power systems

Columbia University

E(P g + w̄ + w − A w − P d ) = P g + w̄ − P dVar(fij ) = yij2 πijT (I − A)Ω(I − AT )πij ;Ω = covariance of w .
Yields SOCP formulation for safety-constrained problem.
Caution! SOCP, but not easy in larger cases.
Should use sparse formulation.
Should use first-order or outer-envelope method.

Bienstock
Controlling variability in power systems

Columbia University

Previous work on chance-constrained OPF (review)
Bienstock, Chertkov, Harnett
Roald, Andersson, several coauthors

Bienstock
Controlling variability in power systems

Columbia University

Previous work on chance-constrained OPF (review)
Bienstock, Chertkov, Harnett
Roald, Andersson, several coauthors
1

Chance-constrained DC-OPF with linear control
Xδj = −
αji wii

can be implemented as convex optimization problem undersuitable assumptions

Bienstock
Controlling variability in power systems

Columbia University

Previous work on chance-constrained OPF (review)
Bienstock, Chertkov, Harnett
Roald, Andersson, several coauthors
1

Chance-constrained DC-OPF with linear control
Xδj = −
αji wii

can be implemented as convex optimization problem undersuitable assumptions
2

However such convex problems (SOCPs) beyond solvers
But first-order methods fast and accurate

Bienstock
Controlling variability in power systems

Columbia University

An extreme example of variability

Lµ, σ 2b0,1,...,k+1 = generators
1, ...,k+1 = participating
generators

k+D

a
k+2

0

1

k

b = loadb = stochastic node

k+1

Quantity k is large. Bus b has a load of L units.
Stochastic injection at bus b = ω, E(ω) = µ, Var(ω) = σ 2 .

Bienstock
Controlling variability in power systems

Columbia University

An extreme example of variability

Lµ, σ 2b0,1,...,k+1 = generators
1, ...,k+1 = participating
generators

k+D

a
k+2

0

1

k

b = loadb = stochastic node

k+1

Quantity k is large. Bus b has a load of L units.
Stochastic injection at bus b = ω, E(ω) = µ, Var(ω) = σ 2 .
2σ > µ.
Linear generation cost function at i (0 ≤ i ≤ k + 1): ci pi .
c0 < c1 = c2 . . . = ck < ck+1 .
Safety parameters equal to 3.
(specify line limits later)
Bienstock
Controlling variability in power systems

Columbia University

Lµ, σ 2b0,1,...,k+1 = generators
1, ...,k+1 = participating
generators

k+D

a
k+2

0

1

k

b = loadb = stochastic node

k+1

Unique optimal safety-constrained solution:
P0g = L − µ − 3σ.
For 1 ≤ i ≤ k: αi = 1/k and Pig = 3σ/k.
g
αk+1 = Pk+1
= 0.

Bienstock
Controlling variability in power systems

Columbia University

Lµ, σ 2b0,1,...,k+1 = generators
1, ...,k+1 = participating
generators

k+D

a
k+2

0

1

k

b = loadb = stochastic node

k+1

Unique optimal safety-constrained solution:
P0g = L − µ − 3σ.
For 1 ≤ i ≤ k: αi = 1/k and Pig = 3σ/k.
g
αk+1 = Pk+1
= 0.

Stochastic flow on ab = 3σ − µ − ω, with variance σ 2 .

Bienstock
Controlling variability in power systems

Columbia University

Lµ, σ 2b0,1,...,k+1 = generators
1, ...,k+1 = participating
generators

k+D

a
k+2

0

1

k

b = loadb = stochastic node

k+1

Unique optimal safety-constrained solution:
P0g = L − µ − 3σ.
For 1 ≤ i ≤ k: αi = 1/k and Pig = 3σ/k.
g
αk+1 = Pk+1
= 0.

Stochastic flow on ab = 3σ − µ − ω, with variance σ 2 .

Bienstock
Controlling variability in power systems

Columbia University

Lµ, σ 2b0,1,...,k+1 = generators
1, ...,k+1 = participating
generators

k+D

a
k+2

0

1

k

b = loadb = stochastic node

k+1

Unique optimal safety-constrained solution:
P0g = L − µ − 3σ.
For 1 ≤ i ≤ k: αi = 1/k and Pig = 3σ/k.
g
αk+1 = Pk+1
= 0.

Stochastic flow on ab = 3σ − µ − ω, with variance σ 2 .
Suppose we want to √reduce variance on ab by 50%.
Then αk+1 = 1 − .5 ≈ .293

Bienstock
Controlling variability in power systems

Columbia University

Lµ, σ 2b0,1,...,k+1 = generators
1, ...,k+1 = participating
generators

k+D

a
k+2

0

1

k

b = loadb = stochastic node

k+1

Unique optimal safety-constrained solution:
P0g = L − µ − 3σ.
For 1 ≤ i ≤ k: αi = 1/k and Pig = 3σ/k.
g
αk+1 = Pk+1
= 0.

Stochastic flow on ab = 3σ − µ − ω, with variance σ 2 .
Suppose we want to √reduce variance on ab by 50%.
Then αk+1 = 1 − .5 ≈ .293
sum of line flow variances > (.5 + (.293)2 (D + 1))σ 2 .
Bienstock
Controlling variability in power systems

Columbia University

Lµ, σ 2b0,1,...,k+1 = generators
1, ...,k+1 = participating
generators

k+D

a
k+2

0

1

k

b = loadb = stochastic node

k+1

Unique optimal safety-constrained solution:
P0g = L − µ − 3σ.
For 1 ≤ i ≤ k: αi = 1/k and Pig = 3σ/k.
g
αk+1 = Pk+1
= 0.

Stochastic flow on ab = 3σ − µ − ω, with variance σ 2 .
Suppose we want to √reduce variance on ab by 50%.
Then αk+1 = 1 − .5 ≈ .293
D = 10, → sum of line flow variances ≈ 1.35 σ 2 .
Bienstock
Controlling variability in power systems

Columbia University

Variance-aware SCOPF
X

min
g

P ,A

s.t.

ck (Pkg )+∆(v 2 )kX

(1 − αji ) = 0

j

∀i;

X

(Pig + w̄i − Pid ) = 0

i

T
max
πkm
ykm |E(fkm )| + νkm Std(fkm ) ≤ fkm

∀ km

(9)

.
v 2 = vector with entries Var(fij ).
∆ = “variance metric”.

Bienstock
Controlling variability in power systems

Columbia University

Variance-aware SCOPF
X

min
g

P ,A

s.t.

ck (Pkg )∆(v 2 )+kX

(1 − αji ) = 0

∀i;

X

j

(Pig + w̄i − Pid ) = 0

i

T
max
πkm
ykm |E(fkm )| + νkm Std(fkm ) ≤ fkm

∀ km

(9)

.
v 2 = vector with entries Var(fij ).
∆ = “variance metric”.
Special case: ∆(Var(f )) =P

ij ∈F ∆ij (Var(fij ))∆ij convex nondecreasing,
Bienstock
Controlling variability in power systems

Columbia University

Variance-aware SCOPF
X

min
g

P ,A

s.t.

ck (Pkg )∆(v 2 )+kX

(1 − αji ) = 0

∀i;

X

j

(Pig + w̄i − Pid ) = 0

i

T
max
πkm
ykm |E(fkm )| + νkm Std(fkm ) ≤ fkm

∀ km

(9)

.
v 2 = vector with entries Var(fij ).
∆ = “variance metric”.
Special case: ∆(Var(f )) =P

ij ∈F ∆ij (Var(fij ))∆ij convex nondecreasing, but F could depend on solution
Bienstock
Controlling variability in power systems

Columbia University

Variance metric
1

P

ij∈F ∆ij (Var(fij ))F = all lines. Var-aware SCOPF is a convex optimization
problem.

Bienstock
Controlling variability in power systems

Columbia University

Variance metricP

ij∈F ∆ij (Var(fij ))1

F = all lines. Var-aware SCOPF is a convex optimization
problem.

2

F = set of N lines with largest flow variance, N fixed.
E.g. N = 50.

Bienstock
Controlling variability in power systems

Columbia University

Variance metricP

ij∈F ∆ij (Var(fij ))1

F = all lines. Var-aware SCOPF is a convex optimization
problem.

2

F = set of N lines with largest flow variance, N fixed.
E.g. N = 50. Convex problem?

Bienstock
Controlling variability in power systems

Columbia University

Variance metricP

ij∈F ∆ij (Var(fij ))1

F = all lines. Var-aware SCOPF is a convex optimization
problem.

2

F = set of N lines with largest flow variance, N fixed.
E.g. N = 50. Convex problem? Convex.

Bienstock
Controlling variability in power systems

Columbia University

Variance metricP

ij∈F ∆ij (Var(fij ))1

F = all lines. Var-aware SCOPF is a convex optimization
problem.

2

F = set of N lines with largest flow variance, N fixed.
E.g. N = 50. Convex problem? Convex.

3

F = set of N lines with largest flow magnitude, N fixed.

Bienstock
Controlling variability in power systems

Columbia University

Variance metricP

ij∈F ∆ij (Var(fij ))1

F = all lines. Var-aware SCOPF is a convex optimization
problem.

2

F = set of N lines with largest flow variance, N fixed.
E.g. N = 50. Convex problem? Convex.

3

F = set of N lines with largest flow magnitude, N fixed.
Convex?

Bienstock
Controlling variability in power systems

Columbia University

Variance metricP

ij∈F ∆ij (Var(fij ))1

F = all lines. Var-aware SCOPF is a convex optimization
problem.

2

F = set of N lines with largest flow variance, N fixed.
E.g. N = 50. Convex problem? Convex.

3

F = set of N lines with largest flow magnitude, N fixed.
Convex?

4

Logarithmic barrier function.
max∆ij = −ρ log(fkm− |E(fkm )| − νkm Std(fkm ))where ρ > 0
Bienstock
Controlling variability in power systems

Columbia University

Variance-aware SCOPF
X

min
g

P ,A

s.t.

ck (Pkg )+P

ij ∈F ∆ij (Var(fij ))kX(1 − αji ) = 0

X

∀i;

j

i

T
πkm
ykm |E(fkm )| +

X

min
g

P ,A

(Pig + w̄i − Pid ) = 0

max
νkm Std(fkm ) ≤ fkm∀ km (10)max
ck (Pkg ) − ρ log(fkm− |E(fkm )| − νkm Std(fkm ))

k

X
s.t.
(1 − αji ) = 0
jBienstock
Controlling variability in power systems

∀i;

X g
(Pi + w̄i − Pid ) = 0iColumbia University

Variance-aware SCOPF

X

min
g

P ,A

s.t.

ck (Pkg )+∆(v 2 )kX

(1 − αji ) = 0

jTπkm
ykm |E(fkm )| +Bienstock
Controlling variability in power systems∀i;X(Pig + w̄i − Pid ) = 0imaxνkm Std(fkm ) ≤ fkm∀ km

Columbia University

Correction vs Formal Variance-Aware Optimization

Several groups of authors:
1

Outer-approximation algorithm for chance-constrained
DC-OPF converges in few iterations.

2

Only a few lines are “at risk” in realistic cases.

Bienstock
Controlling variability in power systems

Columbia University

Correction vs Formal Variance-Aware Optimization

Several groups of authors:
1

Outer-approximation algorithm for chance-constrained
DC-OPF converges in few iterations.

2

Only a few lines are “at risk” in realistic cases.

3

Low-hanging fruit:

Bienstock
Controlling variability in power systems

Columbia University

Correction vs Formal Variance-Aware Optimization

Several groups of authors:
1

Outer-approximation algorithm for chance-constrained
DC-OPF converges in few iterations.

2

Only a few lines are “at risk” in realistic cases.

3

Low-hanging fruit: instead of variance-aware optimization,
first solve SC-OPF problem, and then correct or adjust to
reduce variability without increasing cost (by much).

Bienstock
Controlling variability in power systems

Columbia University

GENERIC CORRECTION TEMPLATE
Input: an instance of the safety-constrained problem and avariance metric.
Step I. Solve safety-constrained OPF problem,
solution (P̄ g , Ā).

with

Step II. Perform a small number of adjustment iterations which shift (P̄ g , Ā) to a new feasible solution thatattains an improved value of the variance metric, while at thesame time increasing generation cost in a moderate manner.

Bienstock
Controlling variability in power systems

Columbia University

Reroute(τ, Â)Â : a given control matrix

Bienstock
Controlling variability in power systems

Columbia University

Reroute(τ, Â)Â : a given control matrix.
2 π T (I − Â)Ω(I − ÂT )π
So Var(fkm ) = ykm
km = V̂km
km
is fixed for every km
0 < τ < 1.

Bienstock
Controlling variability in power systems

Columbia University

Reroute(τ, Â)Â : a given control matrix.
2 π T (I − Â)Ω(I − ÂT )π
So Var(fkm ) = ykm
km = V̂km
km
is fixed for every km
0 < τ < 1.
min
g

X

s.t.

X

P

ck (Pkg )k(Pig + w̄i − Pid ) = 0iTπkm
ykm |E(fkm )| + νkm

q
V̂km

max
≤ (1 − τ )fkm

∀ kmHere, fkm is an implicit function of P g and Â
Bienstock
Controlling variability in power systems

Columbia University

VShift(P̄ g )P̄ g : generation vector. f¯ : corresponding flow vector.

min
A

X



2
T
∆km ykmπkm(I − A)Ω(I − AT )πkm

km∈F (f¯)

constraints on A:

X(1 − αji ) = 0∀ij→ Keep power flows fixed, improve on control

Bienstock
Controlling variability in power systems

Columbia University

Correction procedure
Input: Feasible solution (P g ,0 , A0 ),0 < τ < 1.

For k = 1, 2, . . . , K perform iteration k:
1. Run Reroute(Ak−1 , τ ). If infeasible STOP.
Else let P g ,k be optimal.
2. Solve VShift(P g ,k ), with solution Âk .
3. Set Ak ← (1 − λ)Ak−1 + λÂk .
0 < λ < 1 chosen so that P g ,k , Ak feasible
4. If ∆(Ak ) ≥ ∆(Ak−1 ), STOP.

Bienstock
Controlling variability in power systems

Columbia University

Correction procedure
Input: Feasible solution (P g ,0 , A0 ),0 < τ < 1.

For k = 1, 2, . . . , K perform iteration k:
1. Run Reroute(Ak−1 , τ ). If infeasible STOP.
Else let P g ,k be optimal.
2. Solve VShift(P g ,k ), with solution Âk .
3. Set Ak ← (1 − λ)Ak−1 + λÂk .
0 < λ < 1 chosen so that P g ,k , Ak feasible
4. If ∆(Ak ) ≥ ∆(Ak−1 ), STOP.

Thm.: In convex case if we stop in Step 4, then ∆ minimum.
Bienstock
Controlling variability in power systems

Columbia University

Example on case2746wp

2746 buses, 3514 branches, 520 generators
22 stochastic injection sites, sum of mean injections 4611.57(penetration 18.5% penetration)
average ratio of standard deviation to mean ≈ 30%.

Bienstock
Controlling variability in power systems

Columbia University

Example on case2746wp

2746 buses, 3514 branches, 520 generators
22 stochastic injection sites, sum of mean injections 4611.57(penetration 18.5% penetration)
average ratio of standard deviation to mean ≈ 30%.
P
Variance metric:ij ∈F Var(fij ), where F is the union of

Bienstock
Controlling variability in power systems

Columbia University

Example on case2746wp

2746 buses, 3514 branches, 520 generators
22 stochastic injection sites, sum of mean injections 4611.57(penetration 18.5% penetration)
average ratio of standard deviation to mean ≈ 30%.
P
Variance metric:ij ∈F Var(fij ), where F is the union of
The 100 lines with largest flow magnitude
Lines ij where |f¯ij | + νij Std(fij ) ≥ (1 − τ )fijmax

Bienstock
Controlling variability in power systems

Columbia University

Two iterations
Initial variance metric: ≈ 6.3 × 1004 .

Bienstock
Controlling variability in power systems

Columbia University

Two iterations
Initial variance metric: ≈ 6.3 × 1004 .
After one iteration: ≈ 4.65 × 1004 .

Bienstock
Controlling variability in power systems

Columbia University

Two iterations
Initial variance metric: ≈ 6.3 × 1004 .
After one iteration: ≈ 4.65 × 1004 .
After two iterations: ≈ 4.50 × 1004 .
Approx. 40% reduction relative to original.

Bienstock
Controlling variability in power systems

Columbia University

Two iterations
Initial variance metric: ≈ 6.3 × 1004 .
After one iteration: ≈ 4.65 × 1004 .
After two iterations: ≈ 4.50 × 1004 .
Approx. 40% reduction relative to original.
Cost nearly constant after two iterations

Bienstock
Controlling variability in power systems

Columbia University

Two iterations
Initial variance metric: ≈ 6.3 × 1004 .
After one iteration: ≈ 4.65 × 1004 .
After two iterations: ≈ 4.50 × 1004 .
Approx. 40% reduction relative to original.
Cost nearly constant after two iterations
Variance-shifting SOCP has approximately 1.4 × 1005 variables
and constraints and 1 × 1006 nonzeros
Solution times of a few seconds

ThThu.Nov.16.140404.2017@blacknwhite
Bienstock
Controlling variability in power systems

Columbia University