## 4 Methodology
### Query-Guided Dual Attention (Qgda)  
At the sentence level of the proposed hierarchical structure, individual sentence representations are combined to derive the overall sentiment score of the document. Modeling the sequential nature of sentences within a document requires an effective sentence encoder. We selected bidirectional gated recurrent unit (GRU) [7] as the fundamental block for three primary reasons. First, GRUs are a standard and proven architecture for sequence modeling in hierarchical NLP tasks, demonstrating strong performance in capturing contextual dependencies. Second, they offer greater computational efficiency and faster training times compared to alternatives such as LSTMs and often achieve comparable performance, which is a significant advantage for practical applications. Finally, our preliminary experiments confirmed that Bi-GRU provided a robust performance-to-complexity tradeoff for our specific dataset and task. Therefore, we utilize Bi-GRU to process the sequence of sentence representations generated by the BERT encoder. For the \(t\)-th sentence representation, the hidden state \(h_{t}\) is updated based on the previous state \(h_{t-1}\) as follows:  
Figure 1: The model architecture of our hierarchical BERT-GRU model with Query-Guided Dual Attention (Qgda). \[r_{t} =\sigma(U_{t}S_{t}+W_{t}h_{t-1}+b_{r}) \tag{1}\] \[z_{t} =\sigma(U_{2}S_{t}+W_{2}h_{t-1}+b_{z})\] (2) \[\widetilde{h_{t}} =tanh(U_{h}S_{t}+W_{h}(r_{t}\odot h_{t-1})+b_{h})\] (3) \[h_{t} =z_{t}\odot h_{t-1}+(1-z_{t})\odot\widetilde{h_{t}} \tag{4}\]  
where \(r_{t}\) and \(z_{t}\) represent the reset and update gates, respectively. The sentence representation sequence is processed bidirectionally, i.e., in both the forward and backward directions. We then concatenate the forward hidden states \(\overrightarrow{h}_{t}=GR\widetilde{U}(S_{t})\) and backward hidden states \(\overleftarrow{h}_{t}=\overleftarrow{GRU}(S_{t})\), where \(t\in[1,m]\), obtaining the final representation of sentence \(t\) as \(h_{t}=[\overrightarrow{h}_{t},\overleftarrow{h}_{t}]\). To effectively integrate financial domain knowledge and guide the model's focus, we employ our Query-Guided Dual Attention (QGDA) mechanism. This mechanism directs the model to prioritize those sentence vectors that are most relevant to the financial concepts embedded in our query set. The QGDA mechanism includes two levels. The first is the sentence level, where we calculate the attention weight between sentence and query using all queries in the query set \(\mathbb{Q}\). The representation of sentence in the document is expressed as follows:  
\[u_{j} =tanh\big{(}W_{j}h+b_{j}\big{)},\;\;j\in[1,k] \tag{5}\] \[a_{j} =softmax\left(\frac{u_{j}Q_{j}^{T}}{\sqrt{d_{\mathbb{Q}}}}\right),\;\;j\in[1,k] \tag{6}\]  
where \(h=(h_{1},h_{2},\cdots,h_{m})^{T}\) denotes the concatenation of all sentence hidden states in document \(d\) and \(a_{j}\) denotes the attention weight for carrying out the calculation using \(h\) and the query representation \(Q_{j}\). This first level identifies sentences relevant to each specific financial query/concept. In the second level, we calculate attention weights over the queries themselves in order to determine the relative importance of each query's perspective for the given document:  
\[u_{c} =tanh(W_{c}h+b_{c}) \tag{7}\] \[v =softmax\left(\frac{u_{c}a_{c}^{T}}{\sqrt{a_{c}}}\right) \tag{8}\]  
where \(a_{c}=[a_{1},a_{2},\ldots,a_{j}]\), \(j\in[1,k]\) represents the concatenation of all query attention weights and \(v\) determines which of the query representations should play a more importance role in the second level of the attention mechanism. Based on \(v\), we obtain the final attention weight \(\widetilde{a}=v^{T}a_{c}\) that is combined with all of the query attention weight \(a_{c}\).