# Joint Extraction of Multiple Relations and Entities from Building Code Clauses
## 2 Methodology
### Joint Extraction Model
#### 2.2.4 Subject Extractor
\[head_{i}=\text{Attention}(\mathbf{Q}\mathbf{W}_{i}^{Q},\mathbf{K}\mathbf{W}_{i}^{ K},\mathbf{V}\mathbf{W}_{i}^{V}) \tag{4}\] \[\text{MultiHead}(\mathbf{Q},\mathbf{K},\mathbf{V})=(head_{1}\oplus\dots \oplus head_{m})\mathbf{W}_{O}, \tag{5}\]  
where \(\mathbf{W}_{i}^{Q}\in\mathbb{R}^{2d_{h}\times d_{k}}\), \(\mathbf{W}_{i}^{K}\in\mathbb{R}^{2d_{h}\times d_{k}}\), and \(\mathbf{W}_{i}^{V}\in\mathbb{R}^{2d_{h}\times d_{k}}\) are trainable projection parameters. \(d_{k}=2d_{h}/m\), and \(\mathbf{W}_{O}\in\mathbb{R}^{2d_{h}\times 2d_{h}}\) is also a trainable parameter. \(\text{MultiHead}(\mathbf{Q},\mathbf{K},\mathbf{V})\) is the final multihead attention value. The function of \(\text{MultiHead}(\mathbf{Q},\mathbf{K},\mathbf{V})\) is implemented based on the TensorFlow library according to the work of [PERSON] et al. [34]. Subsequently, the output of the self-attention layer \(\mathbf{h}^{se-sta}=\left[\mathbf{h}_{1}^{se-sta},\mathbf{h}_{2}^{se-sta},\ldots,\mathbf{h}_{n}^{se-sta}\right]\) is fed to a dense layer with softmax activation to produce a probability distribution over all labels. The label of character \(c_{t}\) when it is tagged as a start position is predicted as shown in Equation (7):  
\[P\left(y_{t}^{se-sta}\right)=\text{Softmax}\left(\mathbf{W}^{se-sta} \mathbf{h}_{t}^{se-sta}+\mathbf{b}^{se-sta}\right) \tag{6}\] \[\text{SE-TAG}_{\text{sta}}\left(\mathbf{x}_{t}\right)=\underset{k}{ \arg\max}P\left(y_{t}^{se-sta}=k\right), \tag{7}\]  
where \(\mathbf{W}^{se-sta}\in\mathbb{R}^{|T|\times 2d_{h}}\) and \(\mathbf{b}^{se-sta}\in\mathbb{R}^{|T|}\) are trainable parameters. \(|T|\) denotes the number of output labels. Likewise, the multihead self-attention model can be adopted to capture the features for subject-entity end position identification. Considering that the prediction of the end positions may benefit from the prediction results for the start positions, the output of the first self-attention layer, \(\mathbf{h}^{se-sta}\), is concatenated with the task-specific features \(\mathbf{h}^{se}\) to initialize \(\mathbf{Q}\), \(\mathbf{K}\) and \(\mathbf{V}\) in the second self-attention layer. The output of the second layer is denoted by \(\mathbf{h}^{se-end}=\left[\mathbf{h}_{1}^{se-end},\mathbf{h}_{2}^{se-end},\ldots,\mathbf{h}_{n} ^{se-end}\right]\). Then, the label of character \(c_{t}\) when it is tagged as an end position is predicted as shown in Equation (9):  
\[P\left(y_{t}^{se-end}\right)=\text{Softmax}\left(\mathbf{W}^{se-end} \mathbf{h}_{t}^{se-end}+\mathbf{b}^{se-end}\right) \tag{8}\] \[\text{SE-TAG}_{\text{end}}\left(\mathbf{x}_{t}\right)=\underset{k}{ \arg\max}P\left(y_{t}^{se-end}=k\right), \tag{9}\]  
where \(\mathbf{W}^{se-end}\in\mathbb{R}^{|T|\times 4d_{h}}\) and \(\mathbf{b}^{se-end}\in\mathbb{R}^{|T|}\) are trainable parameters. Figure 3 shows an example of subject-entity extraction. The candidate subject entities are each tagged with the corresponding entity type label (\"E3\" for \"construction element\" and \"E5\" for \"property\") in the start and end tag sequences. Accordingly, a set of subject-entities (i.e., \"\"\" \" \" (stair segment) and \" \" \" (clear width)) are extracted. In addition, the training loss of the subject extractor (which is to be minimized) is defined as the sum of the negative log probabilities of the true start and end tags based on the predicted distributions:  
\[\mathcal{L}_{se}=-\frac{1}{n}\sum_{i=1}^{n}\left(\log P\left(y_{i}^{se-sta}= \tilde{y}_{i}^{se-sta}\right)+\log P\left(y_{i}^{se-end}=\tilde{y}_{i}^{se- end}\right)\right), \tag{10}\]  
where \(\tilde{y}_{i}^{se-sta}\) and \(\tilde{y}_{i}^{se-end}\) are the true start and end tags, respectively, of the \(i\)-th character, and \(n\) is the length of the input clause.