Package: esnpy
esnpyesnpyis an out-of-the-box framework to experiment around ESN and DeepESN.Models have been implemented in pure NumPy/SciPy, so there is no need for a powerful GPU, or any esoteric requirements.Right now, the focus is on batch training, and feedback loops have not been taken into account.But feel free to open a ticket a discuss about anything you need, features you want, or even help !The documentation for the latest release is available onthe github page.Note from the author:esnpyis a small projet I initiated during my master intership, and have recently cleaned up. I might keep working on it for fun, but If you want/need a more robust framework,ReservoirPymight be the one you're searching for ;)Getting StartedInstallationFrom PyPIpipinstallesnpyFrom sourcepipinstallgit+https://github.com/NiziL/esnpy#egg=esnpyUsegithub.com/NiziL/esnpy@<tag or branch>#egg=esnpyto install from a specific branch or tag instead of main.Quickstartimportesnpyreservoir_builder=createBuilder()trainer=createTrainer()warmup,data,target=loadData()# create the echo state networkesn=esnpy.ESN(reservoir_builder.build(),trainer)# train itesn.fit(warmup,data,target)# test itpredictions=esnpy.transform(data)print(f"error:{compute_err(target,predictions)}")ESNandDeepESNYou can create your ESN withesnpy.ESN.
The constructor needs aesnpy.Reservoirand an implementation ofesnpy.train.Trainer.esnpy.DeepESNdoesn't differ a lot, it just expect a list ofReservoirand have an optional parametersmaskto specify from which reservoirs theTrainershould learn. The size ofmaskandreservoirsmust be the same.Then, simply callfitfunction by passing some warm up and training data with the related targets.Once trained, run predictions usingtransform.ReservoirandReservoirBuilderAReservoircan easily be initialized using theReservoirBuilderdataclass.For convenience, the configuration class is also a builder, exposing abuild()method.
This method has an optionalseedparameter used to make deterministic initialization, and so to ease the comparaison of two identical reservoirs.ParametersTypeDescriptionDefaultinput_sizeintSize of input vectorssizeintNumber of units in the reservoirleakyfloatLeaky parameter of the reservoirfnCallableActivation function of the reservoirnp.tanhinput_biasboolEnable the usage of a bias in the inputTrueinput_initesnpy.init.InitializerDefine how to initialize the input weightsinput_tunerslist[esnpy.tune.Tuner]Define how to tune the input weights[]intern_initesnpy.init.InitializerDefine how to intialize the internal weightsintern_tunerslist[esnpy.init.Tuner]Define how to tune the internal weights[]InitializerandTuneresnpy.init.Initializerandesnpy.tune.Tunerare the abstract base classes used to setup the input and internal weights of a reservoir.Initializeris defined by ainit() -> Matrixfunction.esnpyprovides implementations of initializer for both uniform and gaussian distribution of weights, and for both dense and sparse matrix.Tuneris defined by ainit(matrix : Matrix) -> Matrixfunction, which can be used to modify the weights after initialization.
For example,esnpyprovides aSpectralRadiusTunerto change the spectral radius of a weights matrix.TrainerandReaderesnpy.train.Traineris responsible to create the output weights matrix from the training data and targets.It is defined by atrain(inputs: Matrix, data: Matrix, target: Matrix) -> Readerfunction.TheReaderis then responsible for computing the final result from the reservoir activations through__call__.esnpyprovides aRidgeTrainerto compute the output weights using a ridge regression, and aSklearnTrainer(more on this one later).
This trainer has three parameters : one float, the regularization parameter's weightalpha, and two optionals boolean (default to true)use_biasanduse_inputto control if we should use a bias and the input to compute the readout weights.SklearnTraineris an adapter wrapping scikit-learn model, relying on methodsfitandpredict. The feature is still experimental, chaos might ensure.Code ExamplesWant to see some code in action ? Take a look at theexamples/directory:mackey_glass.pydemonstrates how to learn to predict a time series,trajectory_classification/demonstrates how to learn to classify 2D trajectories.BibliographyBased on:The "echo state" approach to analysing and training recurrent neural networksby Herbert Jaeger (pdf),A pratical guide to applying Echo State Networksby Mantas Lukoševičius (pdf),Design of deep echo state networksby Claudio Gallicchio and al (link),Deep echo state network (DeepESN): A brief surveyby Claudio Gallicchio and Alessio Micheli (pdf).Special thanks to Mantas Lukoševičius for hisminimal ESN example, which greatly helped me to get started with reservoir computing.