# Feature-Attended Federated LSTM for Anomaly Detection in the Financial Internet of Things +
## 4 Detail of Feature-Attended LSTM
### Model Training  
As shown in Figure 2, the dataset is noted as \(D\), which consists of \(L(L\in N_{+})\) flows with \(K(K\in N_{+})\) dimension features. Let \(f_{i}(i\in N_{+},i\leq K)\) represent the \(i\)-th feature, and the feature set could be noted as \(F=\{f_{1},f_{2},\cdots,f_{k}\}\). Then, according to the feature grouping algorithm, \(K\) data features will be divided into \(m(m\in N_{+},m\leq K)\) groups. Given time step parameter is \(v(v\in N_{+})\) dataset \(D\) will be transformed into \(s(s\in N_{+})\) samples \(D_{i}(i\in N_{+},i\leq s)\) each sample \(X_{i}\) has a length of \(m\). As \(s\) is the largest integer not greater than \(\frac{l}{v}\), the remaining data will be ignored. Thus, the processed dataset is represented as \(D^{{}^{\prime}}=\{D_{1},D_{2},\cdots,D_{s}\}\). After processing the dataset, the feature-attended LSTM model will construct \(m\) training channels, and each training channel contains \(v\) IEU to extract the feature information. These channels will focus on extracting feature information from the corresponding group. Then, the extracted information will be contributed to the reconstruction layer for fusion so as to achieve delicate perceptual extraction from local to global. IEU is a neural network with the same structure as LSTM, consisting of an input gate (Ig), a forget gate (Eg), and an output gate (Og). Let \(G=\{g_{1},g_{2},\cdots,g_{m}\}(m\in N_{+},m\leq K)\) be the group set and \(x_{j}(t)\) be the input data at the \(t\)-th time step of the \(j\)-th group \(g_{i}\), and the input gate, output gate and forget gate be Ig, Fg and Og, respectively. Then, the calculation formulas of the input gate, forget gate and output gate of the \(j\)-th flow at the \(t\)-th time step are Formulas (1)-(3):  
\[Ig_{j}(t) =\sigma\Big{(}W_{(Ig,xj)}x_{j}(t)+W_{(Ig,hj)}h_{j}(t-1)+b_{( Ig,j)}\Big{)} \tag{1}\] \[Fg_{j}(t) =\sigma\Big{(}W_{(Fg,xj)}x_{j}(t)+W_{(Fg,hj)}h_{j}(t-1)+b_{(Fg,j)} \Big{)}\] (2) \[Og_{j}(t) =\sigma\Big{(}W_{(Og,xj)}x_{j}(t)+W_{(Og,hj)}h_{j}(t-1)+b_{(Og,j)} \Big{)} \tag{3}\]  
where \(W_{(Ig,xj)}\), \(W_{(Ig,hj)}\) and \(b_{(Ig,j)}\) represent the input weight, the hidden weight and the bias of input gate, respectively. \(W_{(Fg,xj)}\), \(W_{(Fg,hj)}\) and \(b_{(Fg,j)}\) represent the input weight, the hidden weight and the bias of forget gate, respectively. \(W_{(Og,xj)}\), \(W_{(Og,hj)}\) and \(b_{(Og,j)}\) represent the input weight, the hidden weight and the bias of output gate, respectively. Then, the output \(h_{j}(t)\) of the \(j\)-th flow at the \(t\)-th time step is calculated as Formulas (4)-(6):  
\[z_{j}(t) =tanh(W_{xj}x(t)+W_{hj}h(t-1)+b_{j}) \tag{4}\] \[c_{j}(t) =Fg_{i}(t)\odot c_{j}(t-1)+Ig_{j}(t)\odot z_{j}(t)\] (5) \[h_{j}(t) =Og_{j}(t)\odot tanh(c_{j}(t)) \tag{6}\]  
Figure 2: Structure of feature-attended LSTM. where \(z\) is the input node with a tanh activation function and \(c\) stands for the memory cell's state. As shown in Figure 2, the \(j\)-th IEU output \(h_{j}^{v}(j\leq m)\) will serve as the input of the reconstruction layer, and the reconstruction layer will be calculated based on the output data of each channel as Formula (7):  
\[h_{out}=\sum_{j=1}^{m}a_{j}h_{j}^{v} \tag{7}\]  
where \(a_{j}\) is the impact factor of \(h_{j}^{v}\) and the coefficient used to adjust the output weight of the channel. It can be obtained by the back-propagation algorithm based on the final result of the model. The influence factor can evaluate the influence of each feature on the overall training result from a global perspective so that the model can focus on extracting the internal time correlation of each feature while considering the correlation between various features and accelerating the convergence of the model. After the \(h_{out}\) is obtained, the model parameters can be adjusted by comparing them with the label information of the training data.