# Uncertainty in Automated Ontology Matching: Lessons from an Empirical Evaluation
## 4 Evaluation Metrics
### Advanced Evaluation Metrics  
Advanced evaluation metrics assume the following definitions [20; 27; 43; 44]:  
* **Precision** measures the \"correctness\" of an alignment. It reflects the share of correct correspondences among all the ones found. It is defined as the ratio of the number of correctly found correspondences (TP) over the total number of found correspondences (TP + FP). Given a _reference_ alignment \(\mathcal{R}\), the _Precision_ of an alignment \(\mathcal{A}\) is a function \(P:\wedge\times\wedge\rightarrow[0,1]\) such that: \[Precision=P(\mathcal{A},\mathcal{R})=\frac{|\mathcal{A}\cap\mathcal{R}|}{| \mathcal{A}|}=\frac{|TP|}{|TP|+|FP|}\]
* **Recall** measures the \"completeness\" of an alignment. It reflects the share of correct correspondences among all the expected correspondences. It is the ratio of the number of correctly found correspondences (TP) over the total number of expected correspondences to be found (TP + TN). Given a _reference_ alignment \(\mathcal{R}\), the _Recall_ of an alignment \(\mathcal{A}\) is a function \(R:\wedge\times\wedge\rightarrow[0,1]\) such that: \[Recall=R(\mathcal{A},\mathcal{R})=\frac{|\mathcal{A}\cap\mathcal{R}|}{| \mathcal{R}|}=\frac{|TP|}{|TP|+|TN|}\]In the ideal case, _Precision_ and _Recall_ reach their highest value of 1.0. * **Noise and Silence** are the complement measures of _Precision_ and _Recall_. Given a _reference_ alignment \(\mathcal{R}\), the _Noise_ and the _Silence_ of an alignment \(\mathcal{A}\) are, respectively, functions \(N\) and \(S:\wedge\times\wedge\rightarrow[0,1]\) such that: \[Noise=N(\mathcal{A},\mathcal{R})=1-\textit{Precision}\] \[Silence=S(\mathcal{A},\mathcal{R})=1-\textit{Recall}\]
* **F-measure** combines _Precision_ and _Recall_, as they cannot accurately assess the matching quality alone. Indeed, the ontology matching tool can have a high _Precision_ and a low _Recall_ and vice versa. _F-measure_ is a combined metric that attaches different importance to _Precision_ and _Recall_. Given a _reference_ alignment \(\mathcal{R}\) and a number \(\alpha\) between 0 and 1 (0 \(\leqslant\alpha\leqslant 1\)), the _F-measure_ of an alignment \(\mathcal{A}\) is a function \(F_{\alpha}:\wedge\times\wedge\rightarrow[0,1]\) such that: \[\textit{F-measure}\big{(}\alpha\big{)}=F_{\alpha}(\mathcal{A},\mathcal{R})= \frac{\textit{Precision}\times\textit{Recall}}{(1-\alpha)\times\textit{ Precision}+\alpha\times\textit{Recall}}\] The \(\alpha\) parameter defines the relative balance between _Precision_ and _Recall_, as higher \(\alpha\) values give more importance to _Precision_ with respect to _Recall_. When \(\alpha=1\), _F-measure_ is equal to _Precision_, and when \(\alpha=0\), _F-measure_ is equal to _Recall_. A value of \(\alpha=0.5\) defines the equal importance of both core metrics. Therefore, when \(\alpha=0.5\), _F-measure_ becomes the harmonic mean of _Precision_ and _Recall_ as follows: \[F1=\textit{F-measure}\big{(}0.5\big{)}=F_{0.5}(\mathcal{A},\mathcal{R})=2\times \frac{\textit{Precision}\times\textit{Recall}}{\textit{Precision}+\textit{Recall}}\] \(\textit{F-measure}(0.5)\), also called \(F1\), is the most commonly used variant of _F-measure_(\(\alpha\)) in IR since it balances the importance of _Precision_ and _Recall_ so that neither is compensated by the other. Matching tools may need parameter tuning. In this case, _F-measure_ is adopted as a driving factor to perform parameter tuning because values that maximize _F-measure_ are considered the optimal ones. Hence, this metric is not only helpful in evaluating the quality of alignments but also in selecting input parameters for matching systems, such as the confidence threshold that returns the best _F-measure_ value. * **Overall** (or _matching accuracy_[45]) is explicitly developed for schema matching purposes, unlike the previously mentioned metrics. It measures the manual error correction effort. That is, it reflects the post-matching effort needed to fix the alignment by adding missing correspondences (FN) and removing false correspondences (FP).