# OEQA: Knowledge- and Intention-Driven Intelligent Ocean Engineering Question-Answering Framework
## 3 The Method of OEQA
### Marine Text Knowledge Extraction with Data Augmentation  
#### 3.2.1 Named Entity Recognition  
Entities that are either nested inside of or connected to another entity are called nested entities. Compared to other fields, the challenge of the NER work in ocean engineering is that there are numerous nested entities and ambiguous boundaries of named entities. We propose a data augmentation approach to balance the samples and adopt the RoBERTa [25] model based on MRC to enhance the context features.  
The data augmentation approach is based on biological genetic variation, inspired by [PERSON] et al. [26], to avoid being affected by the imbalance of positive and negative samples and to learn entity boundaries more effectively. Here, \(n\) represents the length of the input sequence and \(Y\) represents all the named entity label types (e.g., Project, Coordinates, Organization, etc.) in the dataset given an input sequence \(X=\{x_{1},x_{2},\ldots,x_{n}\}\) and named entity label \(y\in Y\) that corresponds to it.  
As shown in Figure 3, each input sequence and named entity in the training set is regarded as a chromosome and a gene inside the chromosome, respectively. The named entity \(e_{y}\) randomly chosen in the sequence \(X\) is replaced with a marker \([GENE]\). Then, we regard \(X\) as \(X_{y}\) and store it in the chromosome library \(C_{y}\). Similarly, the eliminated named entity \(e_{y}\) is stored in the gene bank \(G_{y}\).  
The process of balancing samples is regarded as a heritable variation in biology. For example, when lacking a positive sample of the Project label, a chromosome \(X_{Project}\) is randomly selected from the chromosome library \(C_{Project}\), which is regarded as the self-replication of the chromosome. A gene \(e_{Project}\) is extracted from the gene bank \(G_{Project}\) and inserted into the corresponding \([GENE]\) in \(X_{Project}\) to form a new positive sample, which is regarded as a chromosomal genetic mutation. The method assures that the dataset can be expanded without destroying the original entity boundary and balances positive and negative samples to enhance the learning of contextual features.  
We adopt the RoBERTa-MRC model to recognize named entities based on the augmented dataset. Firstly, construct a natural label description \(p_{y}\) for each label \(y\); then, each sample is converted into a triplet \((p_{y},e_{y},X)\). We concatenate \(p_{y}\) and \(X\) into a sequence as input to the RoBERTa model:  
\[[CLS]p_{y}[SEP]X[SEP] \tag{1}\]  
Then, the model predicts the probability \(p_{start}\) of each token as a start index according to the matrix \(E\in\mathbb{R}^{n\times d}\) output from RoBERTa, and \(d\) represents the dimension of the last layer of RoBERTa. The start and end index predictions are as follows:  
\[p_{start}=softmax_{each\ row}(E\cdot W_{start})\in\mathbb{R}^{n\times 2} \tag{2}\]  
\[p_{end}=softmax_{each\ row}(E\cdot W_{end})\in\mathbb{R}^{n\times 2} \tag{3}\]  
where \(W_{start}\in\mathbb{R}^{d\times 2}\) and \(W_{end}\in\mathbb{R}^{d\times 2}\) are the weights to learn. By applying \(argmax()\) to each row of \(p_{start}\) and \(p_{end}\), we will obtain the predicted indexes that might be the start or end positions:  
\[I_{start}=\{i|argmax(p_{start}^{(i)})=1,i=1,2,\ldots,n\} \tag{4}\]  
\[I_{end}=\{i|argmax(p_{end}^{(i)})=1,i=1,2,\ldots,n\} \tag{5}\]  
Figure 3: A sample of the chromosome containing several gene types.  
Given any start index \(si\in I_{start}\) and end index \(ei\in I_{end}\), the model predicts the probability \(p_{s_{i},e_{i}}\) of matching through the binary matrices \(E_{s_{i}}\) and \(E_{e_{i}}\):  
\[p_{s_{i},e_{i}}=sigmoid(w\cdot concat(E_{s_{i}},E_{e_{i}})) \tag{6}\]  
Here, \(w\in\mathbb{R}^{1\times 2d}\) is the weight to learn.