# Joint Extraction of Multiple Relations and Entities from Building Code Clauses
## 3 Experiment
### Experimental Results and Analysis  
Figure 6 illustrates the test results of the trained joint extraction model. Specifically, the extracted triplets with green backgrounds are correct, whereas a red background indicates an incorrect prediction, and a yellow background indicates an omitted triplet. The test results imply that the model has the capacity to discover multiple relations residing in code clauses, especially overlapping relations. Moreover, the number of extracted triplets varies for many test sentences that cannot be processed using the extraction model proposed in Reference [21]. \begin{table}
\begin{tabular}{l l} \hline \hline
**Standard No.** & **Name of Chinese Building Codes** \\ \hline GB50368-2005 & (Residential building code) \\ GB50096-2011 & (Design code for residential buildings) \\ GB50352-2019 & (Uniform standard for the design of civil buildings) \\ GB50763-2012 & (Code for accessibility design) \\ GB50016-2014 & (Code for the fire protection design of buildings) \\ GB50099-2011 & (Code for the design of schools) \\ GB51039-2014 & (Code for the design of general hospitals) \\ GB50067-2014 & (Design standard for the energy efficiency of public buildings) \\ GB50038-2005 & (Code for the design of civil air defense basements) \\ JGJ100-2015 & (Code for the design of parking garage buildings) \\ JGJ450-2018 & (Standard for the design of care facilities for the aged) \\ JGJ39-2016 & (Code for the design of nursery and kindergarten buildings) \\ JGJ48-2014 & (Code for the design of store buildings) \\ JGJ62-2014 & (Code for the design of hotel buildings) \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Chinese building codes used for dataset preparation. Three metrics, that is, the recall, precision, and F-measure, were calculated to evaluate the model performance. The precision is defined as the percentage of correctly extracted triplets among the total number of extracted triplets (Equation (17)). The recall is the fraction of correctly extracted triplets among the total triplets existing in the source text (Equation (18)). The F-measure combines the recall and precision into a single measure, where \(\beta\) is a parameter that is used to assign relative weights to the recall and precision values (Equation (19)). \[P=\frac{\text{number of correct triplets extracted}}{\text{total number of triplets extracted}} \tag{17}\]  
\[R=\frac{\text{number of correct triplets extracted}}{\text{total number of triplets existing}} \tag{18}\]  
\[F-measure=\frac{(\beta^{2}+1)PR}{\beta^{2}P+R}. \tag{19}\]  
Here, the value of \(\beta\) was set to 1 to assign equal weights to the precision and recall. In this experiment, a triplet was considered correct only if its subject entity, object entity, and predicate relation were all correct. The precision, recall, and F-measure scores achieved by the model in this test for each of the 14 predicate types are listed in Table 4. On average, a precision, recall, and F-measure of 0.8808, 0.8519, and 0.8661, respectively, were achieved. These results validate the effectiveness of the proposed model in extracting multiple relations and entities from building code sentences. In addition, for 7 of the 14 predicate types, the model achieved relatively high F-measure scores of more than 0.8. For the other types (e.g., \"LessThan\" and \"GreaterThan\"), the scores of less than 0.8 may be attributable to the small sample sizes for these types among all clauses. Figure 6: Results of model testing (partial). Table 5 compares the test results of the proposed model with those of the five classic extraction models, from which it can be seen that the former achieved the best performance. The data in Table 5 indicate poor performance of the models proposed by [PERSON] et al. [36] and [PERSON] et al. [37] since they lack the ability to cope with complex overlapping relations.