## 4 Artificial Neural Networks
### Hopfield Neural Network  
The Hopfield neural network consists of a set of \(N\) interconnected neurons which update their activation values asynchronously and independently of other neurons. All neurons are both input and output neurons. The Hopfield neural network is classified as a recurrent neural network and could be represented as content-addressable memory systems with binary threshold units [80]. The conventional Hopfield model is the most commonly used model for auto-association and optimization. Hopfield networks are auto-associators in which node values are iteratively updated based on local computation principle: the new state of each node depends only on its net weighted input at a given time [81]. This network is a fully connected network and the weight matrix determination is one of the important tasks while using it for any application. It is a recurrent single layer and unsupervised network. The architecture of Hopfield neural network is shown in Figure 10.  
Figure 9: Routing table stored in network’s node. Each neuron has two states similar to those of McCulloch and Pitts [82], 1 as activation and 0 as deactivation. Each neuron is connected to other neuron by the connectionist weight \(W_{i,j}\). The model proposed in [83], assumes that there is no self connection, \(W_{i,i}=0\), and weights are symmetrical, \(W_{i,j}=W_{j,i}\). Possible realization of basic Hopfield neural network model is shown in Figure 11.  
Each neuron is realized as a separate block with nonlinear transfer function \(g(u_{i})=v_{i}=u_{i}\), called also an activation function. The most used activation function is the sigmoid one, Figure 12.  
\[v_{i}=g_{i}(u_{i})=\frac{1}{1+e^{-u_{i}u_{i}}} \tag{1}\]  
The outputs of each neuron will be fed back to all other neurons through the connection weight matrix \(\mathbf{W}=[W_{ij}]\). In hardware realization weight \(W_{ij}\) is related to conductances. One possible hardware realization of Hopfield neural network is shown in Figure 11. Weight \(W_{ij}\) are realized as resistances \(R_{ij}\), where \(W_{ij}=1/R_{ij}\). The basic HNN model has two types of inputs: external inputs \(I_{i}\) and outputs of other neuron in network [75]. The total input to the neuron \(i\) is given by:  
\[\sum_{j\
eq i}\frac{v_{j}}{R_{ij}}+I_{i} \tag{2}\]  
where \(R_{ij}\) is interconnection weight form neuron \(i\) to neuron \(j\). Figure 10: Hopfield neural network architecture. Figure 11: One possible hardware realization of Hopfield neural network. Output has two states threshold neurons. In that way, output \(v_{i}\) of neuron \(i\) is defined by values \(v^{0}_{i}\) and \(v^{1}_{i}\). These two values are noted as 0 and 1. The term \(u_{i}\) represents the threshold of neuron \(i\), and output of this neuron is in related to \(u_{i}\) by:  
\[ \begin{split}& if\bigg{(}\sum_{j=i}\frac{v_{j}}{R_{ij}}+I_{i}<u_{i} \bigg{)}\bigg{\{}v_{i}=v^{0}_{i}\bigg{\}}\\ &\text{else}\Big{\{}v_{i}=v^{1}_{i}\Big{\}}\end{split}  \tag{3}\]  
State equations of the circuit in Figure 11 are described by:  
\[C_{i}\frac{du_{i}}{dt}=\sum_{ \begin{subarray}{c}j=i\\ j\
eq i\end{subarray} }^{N}\frac{v_{j}}{R_{ij}}-\frac{u_{i}}{R_{i}}+I_{i},\ \ \ i=1,2,...,N \tag{4}\]  
where \(R_{i}\) is an equivalent resistance connected to the cell's capacitor \(C_{i}\). If we assume constants \(a_{i}\) sufficiently large, the stability of the network, in Liapunov sense, may be verified by observing the energy function, \(E\), describing the state of the network:  
\[E=-\frac{1}{2}\sum_{ \begin{subarray}{c}i=i\\ i\
eq j\end{subarray}