# Independent Random Recurrent Neural Networks for Infrared Spatial Point Targets Classification
## 3 Classification of IR Radiation Intensity Sequence Based on IRRNN
### Classification Algorithm Based on IRRNN
#### 3.3.1 IRRNN Overall Structure and Algorithm
Through the above analysis of the IRRNN network structure, we use cross entropy as the loss function for the classification problem of the infrared radiation intensity time series of the spatial point target and use the gradient descent with momentum optimization method to update the network parameters during the training process. The specific training process of the IRRNN algorithm is shown in Algorithm 1. It is shown as follows:  
Figure 5: Structure of IRRNN model unfolded by time. ```
1. Determine network parameters:
* Number of neurons in the input layer of the IRRNN model network I, number of neurons in the hidden layer and backward transmission hidden layer H;
* Determine the error threshold for stopping training \(\epsilon>0\);
* The total number of target time series in the sample set N.
2. Sample data preprocessing and parameter settings:

The training set, validation set, and testing set are divided according to 2:1:1, and generate random weighted matrix \(\mathbf{W}_{1}\), \(\mathbf{W}_{2}\),..., \(\mathbf{W}_{\text{T-1}}\). 3. Initialize network weights and offset parameters:

Determine the initial value of \(\mathbf{W}^{\text{HI}}\), \(\mathbf{w}\), \(\mathbf{W}^{\text{HO}}\), \(\mathbf{B}^{\text{H}}\), \(\mathbf{B}^{\text{O}}\), etc. 4. Training process:

Assume that currently all N sequences in the training sample set for the kth pass, take sequence X(n) = [x1, x2,..., xT] as example

(1) For training sample \(\mathbf{x}_{\text{t}}\) at time step t, calculate the value of the network output value and loss function l\({}_{\text{t}}\), etc. \(\mathbf{x}_{\text{t}}^{\prime}=\mathbf{\sigma}\big{(}\mathbf{x}_{\text{t}}+ \mathbf{\theta}\sum_{1}^{\text{t-1}}\mathbf{W}_{\text{i}}\mathbf{y}_{\text{i}} \big{)}\),

\(\mathbf{a}_{\text{t}}=\mathbf{W}^{\text{HI}}\),\(\mathbf{x}_{\text{t}}^{\prime}+\mathbf{w}\odot\mathbf{h}_{\text{t-1}}+\mathbf{B}^{ \text{H}}\), \(\mathbf{h}_{\text{t}}=f_{\text{h}}(\mathbf{a}_{\text{t}})\),

\(\mathbf{b}_{\text{t}}=\mathbf{W}^{\text{OH}}\):\(\mathbf{h}_{\text{t}}+\mathbf{B}^{\text{O}}\), \(\mathbf{y}_{\text{t}}=f_{\text{o}}(\mathbf{b}_{\text{t}})\),

\(\ell_{\text{t}}=-\mathbf{d}_{\text{t}}\cdot\ln\mathbf{y}_{\text{t}}\). (2) Calculate the loss function for all time steps from 1 to T

\(\mathbf{L}_{\text{train}}=\sum_{\text{t=1}}^{\text{T}}\ell_{\text{t}}\)

(3) Calculate the gradient of loss function E\({}_{\text{train}}\) to parameters \(\
abla\mathbf{W}_{\text{ii}}^{\text{HI}}\), \(\
abla\mathbf{w}_{\text{ii}}\), \(\
abla\mathbf{W}_{\text{ii}}^{\text{OH}}\), etc. (4) Update the network parameters by gradient descent with momentum optimization method,

and obtain \(\mathbf{W}_{\text{ii}}^{\text{HI}}(\mathbf{k}+1)\), \(\mathbf{W}_{\text{ii}}(\mathbf{k}+1)\), \(\mathbf{W}_{\text{ii}}^{\text{OH}}(\mathbf{k}+1)\), etc. \(\mathbf{W}_{\text{ii}}^{\text{HI}}(\mathbf{k}+1)=\mathbf{W}_{\text{ii}}^{\text{ HI}}(\mathbf{k})+\text{m}\Big{(}\mathbf{W}_{\text{ii}}^{\text{HI}}(\mathbf{k} )-\mathbf{W}_{\text{ii}}^{\text{HI}}(\mathbf{k}-1)\Big{)}-\lambda\
abla\mathbf{W }_{\text{ii}}^{\text{HI}}\)

\(\mathbf{W}_{\text{ii}}^{\text{OH}}(\mathbf{k}+1)=\mathbf{W}_{\text{ii}}^{\text{ OH}}(\mathbf{k})+\text{m}\Big{(}\mathbf{W}_{\text{ii}}^{\text{OH}}(\mathbf{k} )-\mathbf{W}_{\text{ii}}^{\text{OH}}(\mathbf{k}-1)\Big{)}-\lambda\
abla\mathbf{W }_{\text{ii}}^{\text{OH}}\)

where m is momentum parameters, m [0, 1]; and \(\lambda\) is the learning rate, \(\alpha\in\) [0, 1]. (5) Enter the next sequence X(n') in the training sample set and repeat the training process in steps (1) to (4) until all N sample sequences are processed through the network and proceed to the next step. 5. Stop the training, when the training error reaches the threshold. 6. Save the trained network parameters. ```  
**Algorithm 1** Training process of time series classification algorithm based on IRRNN.