# Independent Random Recurrent Neural Networks for Infrared Spatial Point Targets Classification
## 3 Classification of IR Radiation Intensity Sequence Based on IRRNN
### Structure of RRNN  
The RRNN algorithm adds the historical information before current state in the input space through the random weight matrix, as shown in Figure 4. The input layer in the figure consists of two parts, one is the data input at time step t, and the other is the weighted mapping of all output data information before time t. The rest of the network structure is consistent with the traditional RNN model. Figure 3: MSE of LSTM and IndRNN with different sequence lengths. In the input layer of RRNN, the historical output information is transmitted as a storage unit together with the input data at time step t to the hidden layer, and the storage memory of the historical information is enhanced. Then the input of the hidden layer at time step t is as follows:  
\[\mathbf{x^{\prime}}_{t}=o\Bigg{(}\mathbf{x}_{t}+\beta\sum_{i=1}^{t-1}\mathbf{W }_{i}\mathbf{y}_{i}\Bigg{)}, \tag{15}\]  
where \(\beta\) is the weight that determines the proportion of historical information in the input space. The larger the value is, the larger the weight of the historical information is and the smaller the proportion of input information at the current time is, and \(\beta\) can be determined empirically. \(\sigma(\cdot)\) is a saturated nonlinear function used by traditional RNNs to avoid degradation of the model into a simple linear model and to define the range of values of \(\mathbf{x_{t}}^{\prime}\). Input \(\mathbf{x^{\prime}}_{t}\in\mathbb{R}^{k\times 1}\), and \(\mathbf{W}_{i}\in\mathbb{R}^{l\times O}\) is the weight matrix of historical information \(\mathbf{y}_{i}\) mapped to the input layer. The historical output information \(\mathbf{y}_{i}\), i = 1, 2,... t is analyzed. Since the output of the network is not very reliable at the first few states and the noise is high, it may cause deviations after mapping to the input space. Therefore, it is considered whether to use a randomly generated weight matrix \(\mathbf{W}_{i}\) to cancel the noise, that is, each column element of \(\mathbf{W}_{i}\) is subject to a random distribution of N (0, 1). The advantage of using the random weight matrix \(\mathbf{W}_{i}\) is that in the high-dimensional space, the historical information \(\mathbf{y}_{i}\) can make the input information \(\mathbf{x}_{t}\) at time step t tend to different directions in space by random weighting, so that the combined \(\mathbf{x_{t}}^{\prime}\) data is more separable. According to the pseudo-orthogonal property of the high-dimensional space [27], when the number of rows of the random weight matrix \(\mathbf{W}_{i}\) is large, the column vectors of \(\mathbf{W}_{i}\) are approximately orthogonal. In this paper, the length of the input sequence satisfies the requirement of higher dimension, and the column vectors of \(\mathbf{W}_{i}\) can be regarded as orthogonal. The network output \(\mathbf{y}_{i}\) in this paper is the classification result, and the orthogonal column vector of the random weight matrix \(\mathbf{W}_{i}\) will weight the corresponding \(\mathbf{y}_{i}\), which will make the combined \(\mathbf{x_{t}}^{\prime}\) tend to different directions, thereby facilitating the subsequent classification processing. Since the RRNN structure only improves the input layer based on the traditional RNN's structure, the network structure of other layers remains unchanged. Therefore, the calculation model of the RRNN network becomes:  
\[\mathbf{x^{\prime}}_{t}=o\Big{(}\mathbf{x}_{t}+\beta\sum_{i=1}^{t-1}\mathbf{W }_{i}\mathbf{y}_{i}\Big{)}; \tag{16}\] \[\mathbf{a}_{t}=\mathbf{W}^{\mathrm{H.I.}}\mathbf{x^{\prime}}_{t}+ \mathbf{W}^{\mathrm{H.I.}}\mathbf{h}_{t-1}+\mathbf{B}^{\mathrm{H}};\] \[\mathbf{h}_{t}=f_{h}(\mathbf{a}_{t});\] \[\mathbf{b}_{t}=\mathbf{W}^{\mathrm{OH.}}\mathbf{h}_{t}+\mathbf{B} ^{\mathrm{O}};\] \[\mathbf{y}_{t}=f_{o}(\mathbf{b}_{t}),\]  
Figure 4: The structure of RRNN unfolded by time.