# Adaptive Neural-Network-Based Lossless Image Coder with Preprocessed Input Data
## 3 Implemented Artificial Neural Network  
The Artificial Neural Network used, named the Adaptive Neural Network (AdNN) [13; 14; 15; 16], is a rather basic one; see Figure 2. Its inputs are neighbor pixels \(P(1),P(2),...,P(r-1)\), and signal 1 at the \(r\)-th position (hence, \(r-1=5\) in Figure 3); the output is an estimator of coded pixel \(P(0)\). The numbering of pixels is as in Figure 1; a single hidden layer contains \(n_{h}\) nodes. The neural network learning set is defined by a training window; its shape for parameter \(W=3\) is shown in Figure 3 (rectangle \(W\) pixels high and \(2W+1\) wide above \(P(0)\) plus \(W\) pixels to the left from \(P(0)\)). \(P(0)\) estimation is improved in consecutive learning epochs; one epoch consists of learning the network for each position of \(P_{off}(0)\) in window \(Q\); see Figure 3. Final network weights are used as initial ones for estimating the next image pixel. At the beginning, the AdNN is not trained; the initial network weights are randomly chosen from the range \(-0.5\) to \(+0.5\) and are the same in both the coder and the decoder. The suggested numbers of epochs, \(t_{e}\), are 20 [13] or 25 [14; 15; 16].  
Neural network learning formulas are as follows: Output value \(S_{h}(i)\) of the \(i\)-th hidden layer neuron is calculated using a sigmoid function:  
\[S_{h}(i)=\frac{1}{1+e^{-S_{h}(i)}}\text{ for each }i=1,...,n_{h}, \tag{3}\]  
Figure 1: Sample indexing in the neighborhood of coded pixels.  
Figure 3: Training window \(Q\) around pixel \(P(0),r-1=5,W=3\).  
Figure 2: Structure of adaptive neural network.  
where \(S_{in}(i)\) is the weighted average value of outputs from input neurons (note that signal values inside AdNN have a range from \(-0.5\) to \(+0.5\)):  
\[S_{in}(i)=\sum_{j=1}^{r}w_{in}^{(i)}(j)\cdot\bigg{[}\frac{P(j)}{255}-0.5\bigg{]} \text{ for each }i=1,...,n_{h}, \tag{4}\]  
and \(w_{in}^{(i)}(j)\) denotes the input weight of the \(i\)-th neuron from the hidden layer connected to the \(j\)-th input neuron (pixel \(P(j)\)). The output neuron output is computed as follows:  
\[y_{out}=\frac{1}{1+e^{-S_{out}}}, \tag{5}\]  
where \(S_{out}\) is the weighted mean of outputs from hidden layer neurons \(S_{h}(i)\):  
\[S_{out}=\sum_{i=1}^{n_{h}}w_{out}(i)\cdot S_{h}(i), \tag{6}\]  
and \(w_{out}(i)\) denotes the \(i\)-th hidden layer neuron output weight. The neural network settings are corrected using the error of its output:  
\[\delta_{out}=y_{out}\cdot(1-y_{out})\cdot\bigg{[}\frac{P(0)}{255}-y_{out} \bigg{]}, \tag{7}\]  
as well as errors of hidden layer signals:  
\[\delta_{h}(i)=S_{h}(i)\cdot[1-S_{h}(i)]\cdot w_{out}(i)\cdot\delta_{out}. \tag{8}\]  
Errors determine increments in the hidden layer input and output weights for each \(i=1,...,n_{h},j=1,...,r\):  
\[\Delta w_{in}^{(i)}(j)\leftarrow\eta\cdot\delta_{h}(i)\cdot[P(i)/255-0.5]+ \alpha\cdot\Delta w_{in}^{(i)}(j),\]  
\[\Delta w_{out}(i)\leftarrow\delta_{out}\cdot S_{h}(i)+\alpha\cdot\Delta w_{out }(i), \tag{9}\]  
then the weights are corrected; the input and output ones are:  
\[w_{in}^{(i)}(j)\gets w_{in}^{(i)}(j)+\Delta w_{in}^{(i)}(j),\]  
\[w_{out}(i)\gets w_{out}(i)+\Delta w_{out}(i). \tag{10}\]  
In [14], it is suggested to use in the first three epochs learning parameters \(\eta=0.9\) and \(\alpha=0\), then switch to \(\eta=0.1\) and \(\alpha=0.9\). At the end of the learning process, the value of the estimated pixel should be converted from the range \(-0.5\) to \(+0.5\) to the range \(0\) to \(255\).