# Hyperband-Optimized CNN-BiLSTM with Attention Mechanism for Corporate Financial Distress Prediction
## 3 Methodology
### BiLSTM  
The BiLSTM (bidirectional long short-term memory) network was proposed by [PERSON], [PERSON], and [PERSON] in 1997 as an extension of the long short-term memory (LSTM) time network. BiLSTM combines two LSTM networks that process sequences in a forward manner and a reverse manner, respectively, and this bidirectional architecture is capable of capturing the backward and forward dependencies of time series data well. BiLSTM demonstrates improved performance in many tasks by learning from past and future data, rather than solely from past data [18].  
The LSTM unit is composed of input gates, output gates, forgetting gates, and memory units that can process and preserve the long-term dependencies of the time series [19].  
The forgetting gate, \(f_{t}\), controls the effect of the previous moment memory cell, \(C_{t-1}\), on the current moment, i.e., the information that is unimportant in the previous and current moments can be selectively forgotten; the input gate, \(i_{t}\), controls the effect of the input information, \(x_{t}\), in the current moment on the memory cell, deciding which information will be stored in the memory cell, and the candidate memory cell, \(\hat{C}_{t}\), combines the output, \(h_{t-1}\), of the hidden layer in the previous moment, and the input information, \(x_{t}\), in the current moment to update the new information in the current moment, and the output gate controls how much information is to be output to the hidden layer. Updating the information of the current moment memory cell, \(C_{t}\), combines the forgetting gate and input gate, combining the memory cell, \(C_{t-1}\), of the previous moment and the new information, \(\hat{C}_{t}\), of the current moment; the output gate, \(o_{t}\), controls how much information in the memory cell state, C, needs to be output to the hidden layer, H. The specific formulae are as follows.  
\[h_{t}=\sigma(w_{f}\cdot[h_{t-1},x_{t}]+b_{f}), \tag{4}\]  
\[i_{t}=\sigma(w_{i}\cdot[h_{t-1},x_{t}]+b_{i}), \tag{5}\]  
\[\hat{C}_{t}=tanh(w_{c}\cdot[h_{t-1},x_{t}]+b_{c}), \tag{6}\]  
\[C_{t}=f_{t}\cdot C_{t-1}+i_{t}\cdot\bar{C}_{t}, \tag{7}\]  
\[o_{t}=\sigma(w_{o}\cdot[h_{t-1},x_{t}]+b_{o}), \tag{8}\]  
\[h_{t}=o_{t}\cdot tanh(C_{t}) \tag{9}\]  
where \(w_{f}\), \(w_{i}\), \(w_{c}\), \(w_{o}\) represent the weight matrices of the forgetting gate, the input gate, the candidate memory unit, and the output gate, respectively, \(b_{f}\), \(b_{i}\), \(b_{c}\), \(b_{o}\) represent the biases of the forgetting gate, the input gate, the candidate memory unit, and the output gate, respectively, and \(\sigma\) are sigmoid activation functions with outputs ranging between \(0\) and \(1\), and are hyperbolic tangent functions with outputs ranging between \(-1\) and \(1\).  
BiLSTM combines two LSTM networks to comprehensively capture the dependencies of the sequence data by processing the sequences in both forward and reverse directions.  
1. Positive \(LSTM\) processing sequence. The input sequence is processed step by step from the first to the last time step. For each time step, \(t\), the positive hidden layer state, \(\overrightarrow{h_{t}}\), is computed according to the positive \(LSTM\) formula. \[\overrightarrow{h_{t}}=LSTM(x_{t},\overrightarrow{h_{t-1}},\overbrace{C_{t-1}}^ {\longrightarrow})\] (10)2. The reverse \(LSTM\) processes the sequence. The input sequence is processed step by step from the last time step to the first-time step. For each time step, \(t\), the reverse hidden layer state, \(\overleftarrow{h_{t}}\), is computed according to the reverse \(LSTM\) formula. \[\overleftarrow{h_{t}}=LSTM(x_{t},\overleftarrow{h_{t-1}},\overleftarrow{C_{t -1}}))\] (11) (3) Connect the forward and backward hidden states. For each time step \(t\), the forward and backward hidden states are concatenated. The concatenated hidden state \(h_{t}\) is then passed through the fully connected layer to produce the final output \(y_{t}\). \[h_{t}=[\overrightarrow{h_{t}},\overleftarrow{h_{t}}]\] (12) \[y_{t}=\sigma(w\cdot h_{t}+b)\] (13)