# An Open-Domain Event Extraction Method Incorporating Semantic and Dependent Syntactic Information
## 3 Our Approach
### Semantic Enhancement Dependency Syntax Representation Layer Based on Middle Layer Features  
After obtaining the semantic information of the final feature, this section obtains the dependent syntactic features of semantic enhancement from another dimension. Firstly, to avoid the mutual misdirection of some syntactic structural information contained in the final layer of BERT and the dependency syntax information, the hidden state of the BERT non-adjacent intermediate layer is obtained as the input of the Bi-LSTM to implement coding, providing word representation to model the dependency syntax and avoid capturing information similarity. Previous studies have shown that the selection of the middle layer can affect model performance. Considering the difference in the performances of different downstream tasks set in the middle layer, this paper conducted experiments on open-domain event extraction tasks. After experimental verification, layers 1, 4, 7, and 10 showed the best performance. Therefore, based on the first section, features \(\left[H_{x}^{1},H_{x}^{4},H_{x}^{2},H_{x}^{10}\right]\) were input into the Bi-LSTM to enhance information representation. In the input process, since BERT encodes tokens not as the original words but as word pieces, the first token embedding is used as the word embedding, and we assume that word \(x\) is divided into four subwords \(x_{1}\), \(x_{2}\), \(x_{3}\), and \(x_{4}\); then, \(x_{1}\) is the first token embedding. Equation (5) is the embedding vector representation of the word \(x\). Equation (6) shows the specific approach: the location of the head word is recorded using the \(head_{indices}\) matrix; subsequently, the corresponding location is embedded based on the token sequence. Figure 3: Bidirectional LSTM diagram. \[\mathrm{p_{x}}=h_{\mathrm{x_{1}}} \tag{5}\]  
\[\mathrm{P_{w}}=[h_{\mathrm{w1}},\ldots,h_{\mathrm{wn}}]=\mathrm{head}_{\mathrm{ indices}(h_{\mathrm{x_{1}}},\ldots,h_{\mathrm{xm}})} \tag{6}\]  
The coding process of the Bi-LSTM is shown as follows, where Equation (7) is the result of the forward LSTM, and Equation (8) is the result of the backward LSTM. Together, the results are concatenated to obtain \(M_{2}=\left[\overrightarrow{\mathrm{H}}_{\mathrm{Bi-LSTM}}\middle| \overrightarrow{\mathrm{H}}_{\mathrm{Bi-LSTM}}\right]\). \[\overrightarrow{\mathrm{H}}_{\mathrm{Bi-LSTM}}=\left[\overrightarrow{ \mathrm{P}}_{\mathrm{w1}},\ldots,\overrightarrow{\mathrm{P}}_{\mathrm{wn}} \right]=\overrightarrow{\mathrm{LSTM}}(\mathrm{p_{w1}},\ldots,\mathrm{p_{wn}}) \tag{7}\]  
\[\overleftarrow{\mathrm{H}}_{\mathrm{Bi-LSTM}}=\left[\overleftarrow{\mathrm{P }}_{\mathrm{w1}},\ldots,\overleftarrow{\mathrm{P}}_{\mathrm{w}}\right]= \overleftarrow{\mathrm{LSTM}}(\mathrm{p_{w1}},\ldots,\mathrm{p_{wn}}) \tag{8}\]  
After obtaining the semantic enhancement information based on the middle layer feature, this method also introduces syntactic structural information. The feature representation is input into the DAGCN to realize the direct use of dependency syntactic information and focus more on critical node information to avoid the noise problem. Finally, the dependency syntactic information representation based on the feature semantic enhancement of the middle layer is obtained. Specifically, a matrix \(\widetilde{\mathrm{A}}\) is first defined, corresponding to the dependency label for the dependency graph (A refers to the graph adjacency matrix). Then, all dependency tags are trained to generate the corresponding transition matrix TAG on the basis of this matrix, and \(\widehat{\mathrm{A}}\) is transformed to obtain \(\widehat{\mathrm{A}}\). Finally, a weighted transformation is performed on the adjacent node values of elements in the input matrix \(\mathrm{H}^{\mathrm{I}}\) to update the node values, and the output of the GCN at layer \(1+1\) can be obtained through the activation function.