# Metamorphic Testing of Relation Extraction Models
## 3 Approach
### Metamorphic Relations for RE
#### 3.2.1 MRs with the Replacement Operation
**  
Due to the varying ways of expressing a natural language sentence, different types of entities may be able to convey similar meanings in the context of the sentence. Such entities, although they have different entity types, are co-related in terms of their semantic meanings. Hence, replacing a head or tail entity with its co-related entity should not affect the prediction result of RE. Suppose that there is an entity \(e^{\prime}_{h}\) (\(e^{\prime}_{t}\)) which is co-related with \(e^{s}_{h}\) (\(e^{s}_{t}\)). \(t_{f}=(s^{f},e^{\prime}_{h},e^{\prime}_{t})\) or \((s^{f},e^{s}_{h},e^{\prime}_{t})\), where \(s^{f}\) is constructed from \(s^{s}\) by replacing \(e^{s}_{h}\) (\(e^{s}_{t}\)) with \(e^{\prime}_{h}\) (\(e^{\prime}_{t}\)). Then, \(\mathcal{R}(t_{s})\) and \(\mathcal{R}(t_{f})\) are expected to be identical. An example MG of MR-R\({}_{3}\) is reported in the third row of Table 2. As can be observed, by replacing the entity \"46\" of the type NUMBER (the tail entity in \(t_{s}\)) with an entity \"46-years-old\" of the type DURATION (the tail entity in \(t_{f}\)), the relations between the head entity and the tail entity in \(t_{s}\) and \(t_{f}\) are identical. Implementing MR-R\({}_{3}\) requires constructing a collection of candidate co-related entities for different types of entities. For this, we searched for different entity types that were semantically co-related across all sentences, where the same entity mentions may be labeled with different entity types as co-related entities. Then, for a given input, we randomly selected a candidate co-related entity of a different type from the original entity as a replacement. **MR-R\({}_{4}\): Replacement with coreferential entities. **  
In a natural language sentence, different entity mentions may refer to the same real-world entity information. This phenomenon is known as coreference resolution, which is a common task in NLP and critical to the success of RE [30]. Naturally, replacing a head or tail entity with its coreferential entity should not affect the prediction result of RE. Suppose that there is an entity \(e^{\prime}_{h}\) (\(e^{\prime}_{t}\)) in the given sentence \(s^{s}\) that refers to the same information as \(e^{s}_{h}\) (\(e^{s}_{t}\)). Then, \(t_{f}\) is constructed as \((s^{s},e^{\prime}_{h},e^{s}_{t})\) or \((s^{s},e^{s}_{h^{\prime}},e^{\prime}_{t})\), which takes the entity \(e^{\prime}_{h}\) (\(e^{\prime}_{t}\)) as the head (tail) entity. Thus, \(\mathcal{R}(t_{s})\) and \(\mathcal{R}(t_{f})\) are expected to be identical. It is common that a noun and a relevant pronoun are coreferential. For example, consider the example MG shown in the fourth row of Table 2, where \"[PERSON]\" and \"he\"both refer to the same person in the given sentence. Therefore, after the replacement, the prediction result should remain the same. To implement MR-R4, we applied NeuralCoref ([[https://github.com/huggingface/neuralcoref](https://github.com/huggingface/neuralcoref)]([https://github.com/huggingface/neuralcoref](https://github.com/huggingface/neuralcoref)), accessed on 2 July 2022), a widely used neural network-based coreference resolution tool, to find all coreferential entities (if any) of a given entity in a sentence. An entity may have multiple coreferential entities in a sentence, and the one with the highest coreference resolution score (correlation coefficient) was selected as a candidate replacement in our experiments.