# Effective Behavioural Dynamic Coupling through Echo State Networks
## 2 Material and Methods  
### Echo State Network  
Echo State Networks (ESN), as seen in Figure 1, provide an architecture for efficient training of Recurrent Neural Networks (RNN) in a supervised manner [30; 31]. One can distinguish two main components in an ESN. Firstly, the Dynamic Reservoir (DR), a large, random, recurrent neural network with fixed weights. These weights get initialised once and are not adapted through the training procedure. The DR is activated by the input and the feedback from the output providing a non-linear response to the input signal. The neurons of the DR usually have sigmoidal activation functions, with hyperbolic tangents to be the prevailing choice. The second part of the ESN is the output, resulting from a linear combination of the reservoir's activations. Only these weights connecting the reservoir with the output are adapted through the training procedure. For an ESN to function properly, the _echo state property_ (ESP) is essential. ESP states that the dynamics of the DR will asymptotically washout, from the initial conditions. It has been observed, that this can be achieved by scaling the _spectral radius_ of the DR weights \(W\) to be less than unity [33]. That is the largest eigenvalue of the weight matrix for the DR weights should be less that unity. This condition states that the dynamics of the ESN is uniquely controlled by the input, and the effect of the initial states vanishes. The setting of spectral radius is also associated with the _memory_ of the DR [33; 34]. That is the time steps it takes for the dynamics of the reservoir to washout and thus the past time steps for which information is incorporated to produce the output. Echo State Network's Dynamics Formalisation  
Assuming an ESN consisting of \(N\) units in the DR, \(K\) input units and \(L\) output units. A matrix \(W_{in}\) of size \([K\times N]\) connecting the input to the DR, a matrix \(W\) of size \([N\times N]\) describing the connections amongst the DR units and a matrix \(W_{outF}\) of size \([N\times L]\) connecting the DR to the output, and finally a matrix \(W_{outFb}\) of size \([L\times N]\) connecting the output to the DR establishing the feedback connections from the output to the DR. Assuming time \(n\), the input signal driving the reservoir is \(u(n)=[u_{1}(n)\cdots u_{K}(n)]\), the state of the DR neurons is \(x(n)=[x_{1}(n)\cdots x_{N}(n)]\) and the output signal is \(y(n)=[y_{1}(n)\cdots y_{L}(n)]\). The state of the reservoir is updated according to  
\[ \begin{split}\mathbf{x}(n+1)=(1-\alpha)x(n)+\\ \alpha f\big{(}\mathbf{W}\mathbf{x}(n)+\mathbf{W}_{in}\mathbf{u}( n+1)+\mathbf{W}_{outFb}\mathbf{y}(n)\big{)}\end{split}  \tag{1}\]  
where \(f\) is a sigmoid function usually the logistic sigmoid or the _tanh_ function, in our case selected to be a hyperbolic tangent. The parameter \(\alpha\) (referred to as leaking rate) regulates the percentage of the contribution of the state's previous time step to the current one. For small \(\alpha\) continuous time dynamics can be approximated [35]. Setting the leaking rate to small values forces the reservoir's dynamics to a slower adaptation, in cases increasing the short term memory of the reservoir. Generally the \(\alpha\) parameter can be understood as the speed of the reservoir's update dynamics discretised in time. Thus, it provides an approximation of the time interval between to consecutive -discrete- samples in the continuous -real- world. In our case the leaking rate allows for and explicit control over the memory of the reservoir, by effectively \(\mathbf{r}\)-sampling part of the reservoir's state \(x\) every time step. The extended system state \(\mathbf{z}(n+1)=[\mathbf{x}(n+1);\mathbf{u}(n+1)]\) at time \(n\) is the concatenation of the reservoir and input states. The extended system state, depending on the particulars of the implementation can also include the output of the reservoir, if the output connections of the reservoir are recurrent. Here there is no recurrency in the output and thus the extended system state is as shown above.