## 4 Closed-Loop Control Algorithms
### Single-Neuron-Based Self-Pre-Optimization Algorithm
A significant feature of the unsupervised Hebbian learning rule is that the weighting difference is positively proportional to the product of the input and output, which is expressed as follows:  
\[\Delta w_{ij}(t)=\eta\times x_{j}(t)\times y_{i}(t), \tag{11}\]  
where \(x_{j}\) represents the input from the previous neuron to the current neuron, and \(y_{i}\) represents the output of the current neuron, which is also the input from the current neuron to the next neuron. The unsupervised Hebbian learning rule has the disadvantage that it cannot weaken the neuronal connections. In response to this issue, supervised learning rules are employed. A supervised learning rule must introduce an expectation value \(m_{i}\) during learning. In the supervised delta learning rule, the learning effect can be evaluated using the following error function \(E\):  
\[E=\frac{1}{2}(m_{i}-y_{i})^{2}. \tag{12}\]The delta rule is derived by minimizing the error between the neuron's output and the expectation value through the gradient descent method. The partial derivative of the error concerning the \(i\)-th weight by using the chain rule can be written as:  
\[\frac{\partial E}{\partial w_{ij}}=\frac{\partial E}{\partial y_{i}}\frac{ \partial y_{i}}{\partial w_{ij}}=-(m_{i}-y_{i})\frac{\partial y_{i}}{\partial w _{ij}}. \tag{13}\]  
After replacing \(y_{i}\) using Equation (8), the gradient of the function \(E\) is derived as follows:  
\[\frac{\partial E}{\partial w_{ij}}=-(m_{i}-y_{i})\frac{\partial f\Big{(} \Sigma_{j}w_{ij}\Big{)}}{\partial w_{ij}}\ x_{j}. \tag{14}\]  
According to the gradient descent law, the change in each weight should be proportional to the gradient. By removing the negative sign and multiplying by a scale factor \(\eta\), the weights are moved in the negative direction of the gradient to minimize the error, so the supervised delta learning rule is expressed as follows:  
\[\Delta w_{ij}(t)=\eta\times x_{j}(t)\times(m_{i}-y_{i}(t)). \tag{15}\]  
From Equation (15), the weight difference in the delta learning rule is proportional to the product of the input and the difference between the output value and the expectation value. Compared to the unsupervised Hebbian learning rule, the delta learning rule can weaken the neuronal connections through negative feedback and correct the weight coefficients to make the output value close to the given expectation. By incorporating the unsupervised Hebbian learning rule of Equation (11) with the supervised delta learning rule of Equation (15), the supervised Hebbian learning rule is obtained as follows:  
\[\Delta w_{ij}(t)=\eta\times x_{j}(t)\times(m_{i}-y_{i}(t))\times y_{i}(t). \tag{16}\]  
The supervised Hebbian learning rule can strengthen or weaken the connection between two neurons activated simultaneously for a given expectation, which depends on the correlation between the inputs of the respective neurons. In Section 4.1, the conventional positional PID algorithm is introduced. It has three terms: proportional, integral, and derivative, with the corresponding parameters \(K_{p}\), \(K_{i}\), and \(K_{d}\). To conceptualize the neuronal PID algorithm, consider the PID controller as a single neuron where its three components function as dendrites, and the parameters \(K_{p}\), \(K_{i}\), and \(K_{d}\) act as the weighting coefficients of the input signals to these dendrites. The output of the neuronal PID controller represents the axon of the neuron. Hence, the neuron PID algorithm under the supervised Hebbian learning rule is shown as the following formulas:  
\[u(k)=K\left(\sum_{j=1}^{3}\hat{w}_{j}(k)x_{j}(k)\right), \tag{17}\]  
where \(\hat{w}_{j}\) (\(j=1,\ 2,\ 3\)) denotes the weights of the neuron's dendrites, which are expressed as follows:  
\[\hat{w}_{j}(k)=\frac{w_{j}(k)}{\sum_{j=1}^{3}\left|w_{j}(k)\right|}. \tag{18}\]According to the supervised Hebbian learning rule given in Equation (16), the weighting coefficients for the three components of the neuronal PID controller are expressed as follows:  
\[