Package: leap-net
leap_netThis repository implements the necessary algorithm and data generation process to reproduce the results published around LEAP Net.What is the leap netBrief introductionSuppose you have a "system"Sthat generates datayfrom input datax. Suppose also that the responseyof this
system can be modulated depending on some known setpointτ.In our experiments,Swas a powergrid, we were interested in predictingythe vector representing the flows
on each powerline of this grid. These flows are determined by the power injected at each "bus" (a bus is the
word in the power system community close to meaning "nodes") of thex(these injections can be both positive if power
is injected, typically when there is a production unit, or negative when there power is consumed). The vectorτencodes for variation of the topology of the powergrid, typically "is a powerline connected or disconnected" and
"is this powerline connected to this other powerline".In summary we suppose a generation processy = S(x, τ). We also suppose that we have some dataset{(x_i, τ_i, y_i)}that was generated using this model with input data coming from a distributionD = Dist(x, τ).
The LEAP net is a "novel" neural network architecture that is
able to predict some responseŷ_ifromx_iandτ_iwith the following propertiesit is fully trainable by stochastic gradient descent like any neural networkits implementation (given here in keras) is really simpleon data(x, τ)drawn from the same distributionDthan the one use for trainingŷis good approximationmost importantly, under some circumstances, even if(x, τ)isNOTdrawn from the distribution used to train it.We call this last property "super generalization". This is somewhat related to transfer learning andzero shot/few shotslearning. We explored this super-generalization properties with discrete modulationτin the case
where, for example, the neural network is learned when the systemShaszerooronedisconnected powerline but it's still able to perform accurate prediction even whentwopowerlines are disconnected
at the same time.Internally, we also made some experiments on load forecast, where the inputxincluded the past realized loads and
the weather forecast for example. The modulating
variableτincluded the properties of the day to predictegis it a monday or a sunday ? It is during bank holiday ?
Is there lots of road traffic (indicating possible start of holidays) this day ? etc. On another topic we also studied
this model in the context of generative models (cVAE or GANs) wherexwas noise,yMNIST images and the modulatorτincluded the color or rotation of the generated digits.LEAP Net gave also pretty good results on all these tasks that we didn't had time to polish for publishing. This makes
us believe that the leap net model is suited in different context than powergrid related application and usable
for modulatorτboth discrete and continuous.ReferencesTo know more about the leap Net, you can have a look at theLEAP nets for power grid perturbationspaper available on arxiv publish at the
ESANN conference.It has been my main focus during my PhD titledDeep learning methods for predicting flows in power grids : novel architectures and algorithmsalso available online.More recently, some analytical proofs and further development where published in the paperLEAP Nets for System Identification and Application to Power SystemsUse the leap netReproducing results of the neuro computing paper.The repositoryneurocomputing_papercontains the necessary material to reproduce the figures
presented in the paper.NBas of writing, a commercial solver was used to compute the powerflows. We are trying to
port the code to use theGrid2Opframework instead.Use the LEAP NetSetting upQuick and dirty wayOf course, this way of doing is absolutely not recommended. By doing it you need to make sure the license of your
own code is compatible with the license of this specific package etc. You have more information on this topic in theLICENSEfile.The most simple way to use the LEAP Net, and especially the Ltau module is to define this class in your project:# Copyright (c) 2019-2020, RTE (https://www.rte-france.com)# See AUTHORS.txt# This Source Code Form is subject to the terms of the Mozilla Public License, version 2.0.# If a copy of the Mozilla Public License, version 2.0 was not distributed with this file,# you can obtain one at http://mozilla.org/MPL/2.0/.# SPDX-License-Identifier: MPL-2.0# This file is part of leap_net, leap_net a keras implementation of the LEAP Net model.fromkeras.layersimport(Layer,Dense,addaskeras_add,multiplyaskeras_multiply)classLtau(Layer):"""This layer implements the Ltau layer.This kind of leap net layer computes, from their input `x`: `d.(e.x * tau)` where `.` denotes thematrix multiplication and `*` the elementwise multiplication."""def__init__(self,initializer='glorot_uniform',use_bias=True,trainable=True,name=None,**kwargs):super(Ltau,self).__init__(trainable=trainable,name=name,**kwargs)self.initializer=initializerself.use_bias=use_biasself.e=Noneself.d=Nonedefbuild(self,input_shape):is_x,is_tau=input_shapenm_e=Nonenm_d=Noneifself.nameisnotNone:nm_e='{}_e'.format(self.name)nm_d='{}_d'.format(self.name)self.e=Dense(is_tau[-1],kernel_initializer=self.initializer,use_bias=self.use_bias,trainable=self.trainable,name=nm_e)self.d=Dense(is_x[-1],kernel_initializer=self.initializer,use_bias=False,trainable=self.trainable,name=nm_d)defcall(self,inputs):x,tau=inputstmp=self.e(x)tmp=keras_multiply([tau,tmp])# element wise multiplicationtmp=self.d(tmp)res=keras_add([x,tmp])returnresThis is the complete code of the Ltau module that you can use as any keras layer.Clean installation (from source)We also provide a simple implement of the LEAP Net that can be use as a anykeraslayer. First you have to
download this github repository:gitclonehttps://github.com/BDonnot/leap_net.gitcdleap_netThen you need to install it (we strongly encourage to install it in a virtual envrionment):pipinstall-U-e.Then,as all python packages installed from sourceyou need to change the current working directory to use this
module:cd..
rm-rfleap_net# optionnally you can also delete the repositoryIn the future, to ease the installation process, we might provide a version of this package on pypi soon,
but haven't done that at the moment. If you would like this feature, write us an issue on github.LeapNet usageOnce installed, this package provide a keras-compatible of theLtaublock defined in the cited papers. Supposes you
have at your disposal:aXmatrix of dimension (nb_row, dim_x)aTmatrix of dimension (nb_row, dim_tau)aYmatrix of dimentsion (nb_row, dim_x)fromkeras.layersimportInputfromkeras.modelsimportModelfromleap_netimportLtau# this import might change if you use the "quick and dirty way".# create the keras modelx=Input(shape=(dim_x,),name="x")tau=Input(shape=(dim_tau,),name="tau")res_Ltau=Ltau()((x,tau))model=Model(inputs=[x,tau],outputs=[res_Ltau])# "compile" the model with a given optimizeradam_=keras.optimizers.Adam(lr=1e-3)model.compile(optimizer=adam_,loss='mse')# train itmodel.fit(x=[X,T],y=[Y],epochs=200,batch_size=32,verbose=False)# make prediction out of ity_hat=model.predict([X,T])Of course, it is more than recommended to first encode your input dataXwith an encore denoted byEon the paper
and then decode them with a "decoder" denoted byDin the papers. An example of such a model is:fromkeras.layersimportInput,Activation,Densefromkeras.modelsimportModelfromleap_netimportLtau# this import might change if you use the "quick and dirty way".# create the keras modelx=Input(shape=(dim_x,),name="x")tau=Input(shape=(dim_tau,),name="tau")## create E, for example with 2 layers of size "layer_size"layer1=Dense(layer_size)(x)layer1=Activation("relu")(layer1)layer2=Dense(layer_size)(x)layer2=Activation("relu")(layer1)# layer2 is the output of E.## this is Ltaures_Ltau=Ltau()((layer2,tau))## now create D, in this case hidden layer, for examplelayer4=Dense(layer_size)(res_Ltau)layer4=Activation("relu")(layer4)# and make the standard (if you do a regression) linear layer for the outputoutput=Dense(dim_y)(layer4)model=Model(inputs=[x,tau],outputs=[output])# "compile" the model with a given optimizeradam_=keras.optimizers.Adam(lr=1e-3)model.compile(optimizer=adam_,loss='mse')# train itmodel.fit(x=[X,T],y=[Y],epochs=200,batch_size=32,verbose=False)# make prediction out of ity_hat=model.predict([X,T])NBWe think the variable we use above are transparent, and we let the user of this work fine tune the learning
rate, the optimizer, the number of epochs the even the size of the batch to suit their purpose.NBTo use this model easily, we suppose you already format your dataset to have the shape{(x_i, τ_i, y_i)}and
in particular that you have a pre-defined encoding of your modulatorτin the form of a vector. The performance of
the LEAP Net can vary depending on the encoding you choose forτ. More information will be provided in the near
future when we will release a port of the code we used to get our results for the neurcomputing paper. We remind
that this port of the code will not be strictly equivalent to the original implementation of the paper that uses a
proprietary powerflow as this code will use the open sourceGrid2Opframework,
that as not available when the paper was first submitted.Cite this workIf you use this work please cite:@article{DONON2020,
title = "LEAP nets for system identification and application to power systems",
journal = "Neurocomputing",
year = "2020",
issn = "0925-2312",
doi = "https://doi.org/10.1016/j.neucom.2019.12.135",
url = "http://www.sciencedirect.com/science/article/pii/S0925231220305051",
author = "B. Donon and B. Donnot and I. Guyon and Z. Liu and A. Marot and P. Panciatici and M. Schoenauer",
keywords = "System identification, Latent space, Residual networks, LEAP Net, Power systems",
abstract = "Using neural network modeling, we address the problem of system identification for continuous multivariate systems, whose structures vary around an operating point. Structural changes in the system are of combinatorial nature, and some of them may be very rare; they may be actionable for the purpose of controlling the system. Although our ultimate goal is both system identification and control, we only address the problem of identification in this paper. We propose and study a novel neural network architecture called LEAP net, for Latent Encoding of Atypical Perturbation. Our method maps system structure changes to neural net structure changes, using structural actionable variables. We demonstrate empirically that LEAP nets can be trained with a natural observational distribution, very concentrated around a “reference” operating point of the system, and yet generalize to rare (or unseen) structural changes. We validate the generalization properties of LEAP nets theoretically in particular cases. We apply our technique to power transmission grids, in which high voltage lines are disconnected and re-connected with one-another from time to time, either accidentally or willfully. We discuss extensions of our approach to actionable variables, which are continuous (instead of discrete, in the case of our application) and make connections between our problem setting, transfer learning, causal inference, and reinforcement learning."
}