SlowGuess/NAACL_2021_OCR huggingface dataset:
where  $P$  are the predicted relation clusters. After symmetrically calculating taxonomy recall, we can get taxonomy F1 by their harmonic average.
(3) Overall Evaluation Metric. To give a global evaluation of open hierarchical relation extraction problem, we propose the Overall Evaluation Metric. It simply combines the matching metric and taxonomy metric by arithmetic mean and harmonic mean, to give an overall score that considers both cluster-level performance and taxonomy-level performance.