# Joint Extraction of Multiple Relations and Entities from Building Code Clauses
## 2 Methodology
### Joint Extraction Model
#### 2.2.5 Object and Predicate Extractor  
Following the extraction of a set of candidate subject entities, their corresponding object entities and the predicates used to link a subject and an object are simultaneously derived by the object and predicate extractor (see Figure 4). The semantic information of a subject entity is helpful for predicting the corresponding object entities and predicate relations. For a given subject entity composed of a sequence of characters, its semantic information can be encoded using an LSTM model, character by character, to further enrich the task-shared feature vectors acquired from the shared semantic encoder. Specifically, the task-shared features for each character in a subject entity are sequentially input into the LSTM model. Finally, the last hidden states of the LSTM are used as the semantic information of the given subject entity and are concatenated with the task-shared features. The resulting concatenated vector contains both context-enriched character information and the associated subject-entity information. Subsequently, two tagging processes are sequentially performed to identify the start and end positions of each object entity. Moreover, whenever a character is tagged as a start position, the associated predicate relation type is simultaneously labeled. The identification of the start and end positions for the object entities is very similar to that for the subject entities, except that the tags for both the start and end positions of an object entity indicate the type of the predicate relation rather than the type of the object entity. For example, in Figure 4, when the given subject entity is \"\(\bigcirc\)\" (stair segment)\", the start and end positions of the entity \"\(\bigcirc\)\" (clear width) are tagged with the label of the predicate type \"R2\" (i.e., \"hasProperty\") in the start and end tag sequences, respectively. Accordingly, one candidate triplet for the given subject entity \"\(\bigcirc\)\" (stair segment)\" is extracted, that is, [\"\(\bigcirc\)\" (stair segment)\", hasProperty, \"\(\bigcirc\)\" (stair segment, hasProperty, clear width]). In this way, for each identified subject entity, its associated object entities and predicate relations are simultaneously extracted. Formally, given an identified subject entity, the label of character \(c_{t}\) when it is tagged as a start or end position is predicted as shown in Equation (12) and Equation (14), respectively:  
\[P\left(y_{t}^{ope-sta}\right)=\text{Softmax}\left(\mathbf{W}^{ope-sta}\mathbf{h}_{t}^{ ope-sta}+\mathbf{b}^{ope-sta}\right) \tag{11}\]  
\[\text{OPE-TAG}_{\text{sta}}\left(\mathbf{x}_{t}\right)=\underset{k}{\arg\max}P \left(y_{t}^{ope-sta}=k\right) \tag{12}\]  
\[P\left(y_{t}^{ope-end}\right)=\text{Softmax}\left(\mathbf{W}^{ope-end}\mathbf{h}_{t}^{ ope-end}+\mathbf{b}^{ope-end}\right) \tag{13}\]  
\[\text{OPE-TAG}_{\text{end}}\left(\mathbf{x}_{t}\right)=\underset{k}{\arg\max}P \left(y_{t}^{ope-end}=k\right), \tag{14}\]  
where \(\mathbf{h}_{t}^{ope-sta}\) and \(\mathbf{h}_{t}^{ope-end}\) are the outputs of the first and second self-attention layers, respectively, at position \(t\) and \(\mathbf{W}^{ope-sta}\in\mathbb{R}^{|T^{\prime}|\times 2d_{h}}\), \(\mathbf{W}^{ope-end}\in\mathbb{R}^{|T^{\prime}|\times dd_{h}}\), and \(\mathbf{b}^{ope-sta},\mathbf{b}^{ope-end}\in\mathbb{R}^{|T^{\prime}|}\) are trainable parameters. \(|T^{\prime}|\) denotes the number of output labels. Finally, the training loss of the object and  
Figure 4: The extraction of object entities and predicate relations. In this figure, the identified subject entity “\(\bigcirc\)\" (stair segment) is taken as an example.