# BRNN-LSTM for Initial Access in Millimeter Wave Communications
## 3 Beam Prediction Access Scheme
### BRNN-LSTM Deep Learning Model  
The beam prediction algorithm relies on the BRNN-LSTM network in two stages. First, the outer processing stage that is between the layers, and the input state inside the LSTM  
Figure 3: LSTM cell structure. cell. For the outer stage between the layers, first the input data is processed by both the bidirectional forward and backward layers to obtain the hidden states. These data reflect contextual information about the beam indices and affiliated power levels. Following this step, the hidden states are fused to obtain the output layer. Here the bidirectional property enables LSTM to retrieve additional beam index contextual information as compared to conventional LSTM, i.e., obtaining the current and future time steps by the backward and forward states. The process at the backward and forward states are similar at each of the bidirectional LSTM units, where each LSTM cell is composed of a state consisting of input \(g_{t}^{in}\), input modulation \(g_{t}^{mod}\), forget \(g_{t}^{f}\) and output \(g_{t}^{out}\) gates that determine information entering the cell state. Consider the details inside the LSTM cell. For the inner stage inside the LSTM unit, first, the cell state \(c_{t}\) at time \(t\) which specifies information carried to the next sequence is modified by \(g_{t}^{f}\) in the sigmoid layer placed underneath it, which is in turn adjusted by \(g_{t}^{mod}\) that delivers the new candidate cell state. The forget gate \(g_{t}^{f}\) receives hidden-state vector \(\eta_{t-1}\) (output vector of the LSTM unit) at time step \(t-1\), and input vector at time step \(t\), \(x_{t}\), as its inputs. This gate then produces an output number between 0 and 1 for each number in the previous cell state at time step \(t-1\), \(c_{t-1}\). Namely, the output of the \(g_{t}^{f}\) instructs the cell state on which information to forget or discard by multiplying 0 by a position in the matrix. Meanwhile, if the output of \(g_{t}^{f}\) is 1, then the information is kept in the cell state, where a _sigmoid_ function, \(\sigma_{g}\), is applied to the weighted input and previous hidden state. Equations (7)-(11) represent the \(c_{t}\), \(g_{t}^{f},g_{t}^{in},g_{t}^{mod}\), and \(g_{t}^{out}\) formulations at time step \(t\), respectively, expressed as [22]  
\[c_{t} =g_{t}^{f}c_{t1}+g_{t}^{in}g_{t}^{mod} \tag{7}\] \[g_{t}^{f} =\sigma_{g}\Big{(}W_{f}[\eta_{t-1},\ x_{t}]+\beta_{f}\Big{)}\] (8) \[g_{t}^{in} =\sigma_{g}(W_{in}[\eta_{t-1},\ x_{t}]+\beta_{in})\] (9) \[g_{t}^{mod} =\tan h(W_{c}[\eta_{t-1},\ x_{t}]+\beta_{c})\] (10) \[g_{t}^{out} =\sigma_{g}(W_{out}[\eta_{t-1},\ x_{t}]+\beta_{out}) \tag{11}\]  
The parameters \(W_{f}\), \(W_{in}\), \(W_{out}\) and \(W_{c}\) are the weight matrices and \(b_{f}\), \(\beta_{in}\), \(\beta_{out}\), \(\beta_{c}\) are the bias vectors for the \(g_{t}^{f}\), \(g_{t}^{in}\), \(g_{t}^{out}\), and \(c_{t}\), respectively, i.e., learnt during the training mode. Finally, the hidden-state layer output \(\eta_{t}\) (working memory) is modeled as \(\eta_{t}\)= \(g_{t}^{out}\).tan\(h(c_{t})\). Note that the key parameters here for the model are the logistic _sigmoid_ and the hyperbolic _tanh_ nonlinear activation function for each gate to predict probability of the output. Foremost, \(g_{t}^{in}\) is a _sigmoid_ function with range \(\in\) [0,1] that can only add memory. Note that the _sigmoid_ function is unable to forget/clear memory, since the cell state equation is a summation between the previous cell state. Therefore, \(g_{t}^{mod}\) is activated with a _tanh_ activation function with a [\(-1\), 1] range that allows the cell state to forget memory. Overall, the training settings for the model include four layers and one dense layer, as presented in Figure 2.  
The dropout layer is used to control the weight of the hidden layers, where the drop-out regularization rate in each layer used as a regularizer is set at 0.2. The model is trained with 350 epoches over a period of two weeks.