Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judge
Authors: Md Tahmid Rahman Laskar, Israt Jahan, Elham Dolatabadi, Enamul Hoque, Chun Peng, Jimmy Huang
Accepted at: acl25

Improving Automatic Evaluation of Large Language Models (LLMs) in Biomedical Relation Extraction via LLMs-as-the-Judge

Md Tahmid Rahman Laskar*‡, Israt Jahan‡,
Elham Dolatabadi†§, Chun Peng‡, Enamul Hoque‡, Jimmy Xiangji Huang*‡
†York University, §Vector Institute
Toronto, Ontario, Canada

Abstract

Large Language Models (LLMs) have demonstrated impressive performance in biomedical relation extraction, even in zero-shot scenarios. However, evaluating LLMs in this task remains challenging due to their ability to generate human-like text, often producing synonyms or abbreviations of gold-standard answers, making traditional automatic evaluation metrics unreliable. On the other hand, while human evaluation is more reliable, it is costly and time-consuming, making it impractical for real-world applications. This paper investigates the use of LLMs-as-the-Judge as an alternative evaluation method for biomedical relation extraction. We benchmark 8 LLMs as judges to evaluate the responses generated by 5 other LLMs across 3 biomedical relation extraction datasets. Unlike other text-generation tasks, we observe that LLM-based judges perform quite poorly (usually below 50% accuracy) in the biomedical relation extraction task. Our findings reveal that it happens mainly because relations extracted by LLMs do not adhere to any standard format. To address this, we propose structured output formatting for LLM-generated responses that helps *LLM-Judges* to improve their performance by about 15% (on average). We also introduce a domain adaptation technique to further enhance *LLM-Judge* performance by effectively transferring knowledge between datasets. We release both our human-annotated and LLM-annotated judgment data (36k samples in total) here: https://github.com/tahmedge/llm_judge_biomedical_re.

1 Introduction

Relation extraction, the task of identifying meaningful associations between biomedical entities such as drugs, diseases, and genes from vast amounts of unstructured text (Bassignana and Plank, 2022), is a cornerstone of biomedicine. As the volume of unstructured biomedical text grows exponentially, efficient and accurate relation extraction is no longer a convenience but a necessity for advancing medical research, drug discovery, and improving patient outcomes (Luo et al., 2022). Recent research has shown that Large Language Models (LLMs) can achieve strong performance in biomedical relation extraction, even in zero-shot scenarios (Jahan et al., 2024). This capability makes LLMs particularly valuable for real-world biomedical applications where manually annotated datasets are scarce or costly to obtain.

However, LLM’s ability to generate human-like text introduces a major challenge in evaluation (Laskar et al., 2024a; Jahan et al., 2024). Traditional automatic evaluation methods, such as string matching and token overlap, fail to capture semantic equivalence, as LLMs frequently produce synonyms, abbreviations, or paraphrased responses that are meaningfully correct but not exact matches. For example, an LLM may generate “Hepatic carcinoma” instead of “Liver cancer”, leading conventional metrics to misclassify correct extractions as incorrect. Due to these limitations, human evaluation has been the predominant method (see Figure 1 for an example) for assessing LLM performance in biomedical relation extraction (Jahan et al., 2024). However, human evaluation is slow and costly,

![An example of human evaluation of LLM-generated outputs in comparison to the gold labels. The drug "dex" in the gold relation is predicted as "dexamethasone" in LLM response. While exact match will fail in this case, human evaluators can recognize.](page_728_370_393_180.png)

Figure 1: An example of human evaluation of LLM-generated outputs in comparison to the gold labels. The drug “dex” in the gold relation is predicted as “dexamethasone” in LLM response. While exact match will fail in this case, human evaluators can recognize.
Figure 2: Average judgment accuracy based on our evaluation of different LLMs-as-the-judge across 3 Relation Extraction datasets (KD-DTI, DDI, BC5CDR) to evaluate the LLM-generated responses in Jahan et al. (2024).

making it impractical for real-world applications.

To address this, LLMs-as-the-Judge (Zheng et al., 2023) have gained attention as a potential alternative to human evaluation. While this paradigm has been explored in general NLP tasks (Li et al., 2024a; Gu et al., 2024), there is currently no biomedical-specific \( LLM\text{-}Judge \) designed to evaluate relation extraction tasks. Unlike open-domain text generation, biomedical relation extraction requires precise domain knowledge, standardized terminology, and strict adherence to extract relationships between entities. This complexity raises concerns about whether existing \( LLM\text{-}Judges \) can reliably assess biomedical extractions.

To investigate this, we examined the capability of several LLMs-as-Judges in evaluating the responses generated by different LLMs (\( LLM\text{-}Generators \)) across multiple biomedical relation extraction datasets (Jahan et al., 2024). Surprisingly, despite prior success in general NLP evaluation tasks, \( LLM\text{-}Judges \) performed very poorly in biomedical relation extraction in comparison to human evaluators (Figure 2). These findings suggest that domain specificity may significantly impact the effectiveness of LLMs-as-Judge, underscoring the need for adaptation in biomedicine.

To address the shortcomings of LLMs-as-Judge in the biomedical relation task, we propose structured output format (Xia et al., 2024; Li et al., 2024b) for response generation by \( LLM\text{-}Generators \). Based on extensive experiments, we find that structured output format in the response generated by \( LLM\text{-}Generators \) consistently helps the \( LLM\text{-}Judges \) to improve their performance across various relation extraction datasets.

Moreover, we also find that there is a lack of human-annotated judgment data that prohibits the training of \( LLM\text{-}Judges \) for relation extraction. Therefore, we propose a domain adaptation (Laskar et al., 2022) technique to address the lack of human-annotated judgment data by effectively transferring knowledge from out-of-domain data to improve the performance of \( LLM\text{-}Judges \) in the target domain. Our major contributions are summarized below:

• We provide the first comprehensive study of \( LLM\text{-}Judges \) in biomedical relation extraction, benchmarking 8 \( LLM\text{-}Judges \) on responses generated by 5 \( LLM\text{-}Generators \) across 3 biomedical relation extraction datasets. Our findings demonstrate that LLMs are not reliable to serve as evaluators in biomedicine, highlighting their significant performance gap compared to human evaluators.

• To address the above limitation, we propose structured output formatting in LLM-generated responses for biomedical relation extraction to improve the performance of \( LLM\text{-}Judge \). We also propose a domain adaptation technique to effectively transfer knowledge from one domain to another to further improve \( LLM\text{-}Judge \) performance in biomedical relation extraction.

• We conduct over 100 experiments, analyzing the impact of structured output format, domain adaptation, and model scaling on \( LLM\text{-}Judge \) performance. These experiments reveal critical insights into why LLM-based evaluation fails in biomedical relation extraction, establishing the need for task-specific evaluation frameworks.

• We make our judgment data (4k human-annotated and 32k LLM-annotated samples) consisting of 3 relation extraction datasets publicly available1.

2 Background

2.1 Relation Extraction Task Description

The relation extraction task aims to extract relations between named entities in a given text (Zhong and Chen, 2021). The biomedical relation extraction
Figure 3: An Overview of LLM-based Relation Extraction System. After constructing the input prompt for the *LLM-Generator* for a given dataset, the *LLM-Generator* generates a response. The evaluation requires the identification of *the number of correctly predicted relations* and *the total number of predicted relations* in the response to evaluate the performance of the *LLM-Generator* for relation extraction in terms of metrics like Precision, Recall, and F1.

task aims to identify relationships between biomedical named entities like genes, drugs, and diseases (Chen et al., 2023). More specifically, in the context of biomedicine, the goal is to analyze textual data to identify which gene/variant is responsible or which treatment/drug is effective for which disease, as well as identifying drug-drug interactions, and etc. An example for disease-treatment relation extraction is given below.

*Example Text:* *The patient has been given chemotherapy for their rare form of cancer.*

*Expected Relation:* *"Chemotherapy" is a treatment for "rare form of cancer".*

2.2 Related Work

Traditional approaches for biomedical relation extraction relied on supervised learning techniques that required large, manually annotated datasets (Luo et al., 2022). However, the construction of such datasets is expensive and time-consuming. Recently, LLMs have demonstrated impressive zero-shot performance across a wide range of NLP tasks (Laskar et al., 2023a; Bang et al., 2023; Qin et al., 2023). This has inspired researchers to successfully apply LLMs for biomedical relation extraction, particularly in zero-shot scenarios where annotated data is scarce (Jahan et al., 2023, 2024; Tian et al., 2024). Despite LLMs achieving impressive zero-shot performance in the biomedical relation extraction task, evaluating the performance of LLM in this task remains challenging (Jahan et al., 2024). This is because LLMs may generate valid outputs that are semantically equivalent to the gold standard but differ in surface form (e.g., synonyms, abbreviations). This issue is further exacerbated in biomedicine due to the nuanced and complex nature of biomedical language. Therefore, existing automatic metrics like string matching or n-gram overlap often fall short in assessing the semantic correctness of LLM-generated free-form responses (Laskar et al., 2024a).

In the context of biomedical relation extraction, while human experts can assess the relevance of LLM extracted relations (Jahan et al., 2024), human evaluation is inherently time-consuming, expensive, and lacks scalability. This makes it impractical for rapid iteration cycles in research and real-world deployment scenarios. The paradigm of using LLMs-as-judges (Zheng et al., 2023) for evaluating free-form text generated by other LLMs has recently gained a lot of attention as a potential alternative to human evaluations (Laskar et al., 2024a). While prior research suggests that LLM-based evaluators can capture linguistic nuances often overlooked by traditional metrics (Li et al., 2024a; Gu et al., 2024), most of these studies have focused on general-domain tasks, without exploring their effectiveness in specialized domains such as biomedical relation extraction. To this end, this paper explores the potential of LLMs-as-the-Judge for evaluating the response generated by other LLMs in the biomedical relation extraction task to mitigate the dependence on expensive, time-consuming human evaluation, alongside ensuring reliability in LLM evaluation by capturing domain-specific nuances that traditional metrics may overlook.

3 Methodology

3.1 LLM-based Relation Extraction Systems

We show the overall pipeline to develop an LLM-based end-to-end relation extraction system in Figure 3. Here, at first, for a given relation extraction dataset, pre-processing steps are first applied to construct a prompt. This prompt is then provided to an LLM (i.e., *LLM-Generator*), which extracts relations. Afterwards, we demonstrate two evaluation paradigms which we describe below:
(i) Existing Approaches based on Human Evaluation: Where an annotation guideline is first constructed to evaluate the performance of the *LLM-Generator* (i.e., relation extraction LLM) by human annotators. The LLM-generated response is then sent to the human annotators for evaluation.

(ii) Our Proposed Approach based on *LLM-Judge*: Where the prompt for the *LLM-Judge* is first constructed to evaluate the performance of the *LLM-Generator*. The response generated by the *LLM-Generator* is then evaluated by the *LLM-Judge*. This is our proposed approach where human evaluator(s) are replaced with the *LLM-Judge*. In the following, we describe our proposed *LLM-Judge* to evaluate relation extraction models.

3.2 *LLM-Judge* for *LLM*-based Relation Extraction Model Evaluation

Prompting: To utilize *LLM-Judge* for the evaluation of LLM-generated responses (i.e., *LLM-Generators*) in relation extraction, we first design the prompt for the judge, as described below.

<table>
  <tr>
    <th>Prompt to the LLM-Judge</th>
  </tr>
  <tr>
    <td>You are required to annotate the response generated by an AI model for biomedical relation extraction. You are given the relation extraction task description, followed by the biomedical text, then the human annotated gold relations, and finally the AI model predicted relations. Now, identify how many of the AI predicted relations are correct in comparison to the gold relations. Also, identify how many relations in total are predicted by AI. Generate your response in the JSON format with the following keys:

1. correctly_predicted_relations
2. total_predicted_relations

[TASK DESCRIPTION]
[BIOMEDICAL TEXT]
[GOLD RELATIONS]
[AI PREDICTED RELATIONS]</td>
  </tr>
</table>

We designed this prompt based on extensive prompt engineering using several LLMs (e.g., Chat-GPT, Gemini, Claude, etc.) on the outputs generated by different *LLM-Generators* in Jahan et al. (2024) across various relation extraction datasets.

Structured Output Formatting: During prompt engineering in our previous step, we notice that *LLM-Judges* often find it difficult to properly understand the response generated by various *LLM-Generators* since the generated responses are mostly unstructured text. To address this issue, we propose *Structured Output Formatting* for the *LLM-*

<table>
  <tr>
    <th>Unstructured Output</th>
    <th>Structured Output</th>
  </tr>
  <tr>
    <td>- Salbutamol: tachyphylaxis<br>- High dose inhaled salbutamol: tremor, palpitations</td>
    <td>{<br>"drug": "salbutamol",<br>"side-effect": "tremor"<br>}</td>
  </tr>
</table>

Figure 4: An example of the LLM-generated outputs for *drug* and *drug-induced side-effects* relation extraction in the *unstructured* format used by Jahan et al. (2024) and our proposed *structured* format.

Generators. In our proposed *Structured Output Formatting* approach, we require the *LLM-Generators* to generate their response in a structured format, i.e., JSON format (see Figure 4 for an example). JSON format was selected since recent research has demonstrated that LLMs are more reliable in generating responses in “JSON” instead of other formats like “YAML” (Laskar et al., 2024b).

Domain Adaptation via Transfer Learning: Leveraging closed-source LLMs-as-the-judge have several limitations. For instance, closed-source LLMs are continuously updated. Therefore, the release of an updated version of a closed-source model often makes their respective earlier versions obsolete (Biderman et al., 2024; Laskar et al., 2024a). This creates a huge reproducibility concern. While open-source LLMs could be a suitable alternative, they do not have the superior zero-shot ability of their closed-source counterparts (Jahan et al., 2024). Moreover, we find that existing open-source *LLM-Judges* like Prometheus-2 (Kim et al., 2024) are restricted to certain evaluation dimensions (e.g., pairwise ranking or pointwise scoring for specific criteria: helpfulness, factual correctness, etc.) since they are fine-tuned only for such qualitative metrics. This prohibits the customization of the judging criteria based on user needs, making them inapplicable in our case to evaluate the relation extraction performance of other LLMs.

Furthermore, due to the absence of human evaluation data in existing relation extraction datasets for LLM-generated responses, it is difficult to train a reliable *LLM-Judge* model to evaluate the response generated by different relation extraction models. To address this low-resource problem that prohibits the training of *LLM-Judge* for relation extraction, we propose a domain adaptation technique (Garg et al., 2020; Laskar et al., 2022). In our proposed approach, the *LLM-Judge* is fine-tuned on a limited number of human annotated out-of-domain relation extraction data to make it more specialized for
relation extraction evaluation. More specifically, suppose we have two datasets, \( X \) and \( Y \), where the dataset \( X \) may focus on drug-drug-interaction extraction and the dataset \( Y \) may focus on disease-treatment-relation extraction. In our domain adaptation technique, if the target dataset for the judge model to evaluate is the dataset \( X \), then we first fine-tune the judge model on the dataset \( Y \) if we have some human evaluation data (e.g., human-annotated judgment labels containing the number of predicted relations and total predicted relations by the \( LLM\text{-}Generator \)) available for the dataset \( Y \). In this way, we specialize the \( LLM\text{-}Judge \) for biomedical relation extraction evaluation.

4 Experiments

To evaluate our proposed solution of replacing human evaluators with \( LLM\text{-}Judges \), we conduct extensive experiments with various LLMs-as-the-Judge on the responses generated by different \( LLM\text{-}Generators \) for biomedical relation extraction. Below, we first describe the datasets used in our experiments. Then, we describe the LLMs used as the Judge and the Generator. Afterwards, we present our evaluation metrics to measure the performance of the \( LLM\text{-}Judge \). Finally, we briefly describe the implementation details.

4.1 Datasets

To evaluate the \( LLM\text{-}Judges \), we use the responses generated by the \( LLM\text{-}Generators \) for chemical-disease-relation extraction (500 test samples) in the BC5CDR dataset (Li et al., 2016), drug-target-interaction extraction (1159 test samples) in the KD-DTI dataset (Hou et al., 2022), and drug-drug-interaction extraction (191 test samples) in the DDI dataset (Herrero-Zazo et al., 2013).

4.2 Models

LLM-Generators: To evaluate the performance of different \( LLM\text{-}Judges \), at first we benchmark their performance on different LLM-generated responses collected from Jahan et al. (2024). The responses from the following \( LLM\text{-}Generators \) are used: (i) GPT-3.5 (Achiam et al., 2023), (ii) Claude-2 (Anthropic, 2023), (iii) PaLM-2 (Anil et al., 2023), (iv) LLaMA-2-13B (Touvron et al., 2023b). Moreover, to investigate the effectiveness of Structured Output Format, we regenerate the response using GPT-4-Turbo (Achiam et al., 2023) in both unstructured and structured format, and compare the performance of the \( LLM\text{-}Judge \) when the response is generated in different formats by the same \( LLM\text{-}Generator \). For this case, we did not use the \( LLM\text{-}Generators \) used in Jahan et al. (2023) since they are early generation LLMs and we observe that they fail to generate the response in structured format.

LLM-Judges: For the \( LLM\text{-}Judges \), we select the cheapest versions of the currently available closed-source LLMs (GPT-4, Gemini, and Claude-3) to minimize their usage cost. For the open-source \( LLM\text{-}Judges \) (LLaMA, Phi, Qwen, and DeepSeek), we select the models with less than 10B parameters since they can be used in a machine with just 1 L4 GPU (Fu et al., 2024), making them cost-effective for real-world deployment. More specifically, for the \( LLM\text{-}Judge \), we primarily use the following LLMs: (i) GPT-4o-Mini (Achiam et al., 2023), (ii) Gemini-1.5-Flash (Team et al., 2023), (iii) Claude-3-Haiku (Anthropic, 2024), (iv) LLaMA-3.1-8B-Instruct\( ^2 \) (Grattafiori et al., 2024), (v) Qwen-2.5-7B-Instruct\( ^3 \) (Yang et al., 2024b), and (vi) Phi-3.5-Mini-3.8B-Instruct\( ^4 \) (Abdin et al., 2024). With the recent success of reasoning-based LLMs like DeepSeek-R1 (Guo et al., 2025), we also use its distilled versions based on Qwen and LLaMA, (vii) DeepSeek-R1-Distill-Qwen-7B\( ^5 \) and (viii) DeepSeek-R1-Distill-Llama-8B\( ^6 \), respectively. Appendix A.2 contains detailed model descriptions.

While we also tried the biomedical domain-focused BioMistral-7B model (Labrak et al., 2024) as a judge, we observed that it failed to follow the judging instruction and could not evaluate any \( LLM\text{-}Generators \) response (see Appendix A.4.1 for details). Therefore, we did not report its results.

4.3 Evaluation Metrics

We use the following metrics to evaluate the effectiveness of the \( LLM\text{-}Judge \).

Exact Match Accuracy: Measures the Exact Match Accuracy of the \( LLM\text{-}Judge \) annotation in

\( ^2 \)https://huggingface.co/meta-llama/Llama-3-1-8B-Instruct
\( ^3 \)https://huggingface.co/Qwen/Qwen2.5-7B-Instruct
\( ^4 \)https://huggingface.co/microsoft/Phi-3.5-mini-instruct
\( ^5 \)https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
\( ^6 \)https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Llama-8B
comparison to the human-annotated gold label. For exact match calculation, if there is a match for both the number of correctly predicted relations and total predicted relations between the gold human annotation and the LLM-Judge annotation, then we consider it as an exact match, and the score will be 1. Otherwise, the score will be 0. Thus, a higher exact match score denotes better performance.

Root Mean Squared Error: Measures the Root Mean Squared Error (RMSE) distance between the LLM-Judge annotation and the human-annotated gold label. Mean Squared Error (MSE) is defined as the average of the squared differences between the predictions (in this case, the LLM-Judge annotations) and the actual values (the Human-Judge annotated gold labels). RMSE, which is the square root of the MSE, helps penalize larger errors more severely while being in the same units as the target variable. Suppose the correctly predicted relations and the total predicted relations annotated by humans are 0 and 2, respectively, and by LLM-Judge are 1 and 2, respectively, then the RMSE distance:

\[
\sqrt{\frac{1}{2} \left( (1 - 0)^2 + (2 - 2)^2 \right)} = \sqrt{0.5} \approx 0.71.
\]

Contrary to the exact match score, a lower RMSE distance denotes better performance.

4.4 Implementation

For the inference of LLM-Generators and LLM-Judges, we use the temperature value of 1.0, with other decoding parameters being set to the default values: as given in the respective API providers (OpenAI7, Google8, Anthropic9) for the closed-source models, and in HuggingFace (Wolf et al., 2020) for the open-source models. We select the temperature value of 1.0 to allow more diversity in LLM-generated responses such that it allows us to ensure a more robust evaluation of the LLM-Judges in diverse output scenarios. Since both the LLM-Generators and the LLM-Judges do not need to produce outputs of longer sequence length, the maximum output tokens was set to 300 tokens except for the reasoning models (DeepSeek-R1). For the DeepSeek-R1-based models, we increased the output token limit to 1000 tokens to allow the model enough tokens for thinking. For domain adaptation via transfer learning, we fine-tune for 3 epochs with the batch size = 1, learning rate = \(2e-5\), and sequence length = \(3k\). All experiments were run using 1 A100 GPU.

5 Results and Discussion

5.1 Performance in Existing LLM-based Relation Extraction Benchmarks

In this section, we benchmark the performance of different LLM-Judges to evaluate the responses generated by different LLM-Generators used by Jahan et al. (2024). In their work, Jahan et al. (2024) used GPT-3.5 (Achiam et al., 2023), PaLM-2 (Anil et al., 2023), Claude-2 (Anthropic, 2023), and LLaMA-2-13B-Instruct (Touvron et al., 2023b) as the LLM-Generators. We collected their human evaluator annotated labels consisting of the number of correctly predicted relations and total predicted relations for each LLM-Generator. Then we measure the Exact Match Accuracy and the RMSE Distance between the LLM-Judge annotation (the number of correctly predicted relations and the total predicted relations annotated by the LLM-Judge) and the human annotation.

For each dataset, we then compute the average Exact Match Accuracy and RMSE Distance across all LLM-Generators, as demonstrated in Table 1. From Table 1, we find that none of the LLM-Judges could reach accuracy above 50%, except GPT-4o-Mini. Nonetheless, GPT-4o-Mini still fails to achieve more than 60% accuracy in any datasets. Our hypothesis is that the responses generated by the LLM-Generators used by Jahan et al. (2024) were quite unstructured, which could be difficult for the LLM-Judges to evaluate. In the following section, we investigate whether Structured Output Formatting could alleviate this issue.

5.2 Performance of LLM-Judges based on Structured vs Unstructured Output

To improve the LLM-Judge accuracy, we generate the responses in all three datasets using the GPT-4-Turbo (Achiam et al., 2023) model as the generator, with specifically prompting GPT-4-Turbo to extract the relations between the named entities in a more structured way, i.e., JSON format. To compare whether this structured output format could improve the performance of the LLM-Judges, we also regenerate the responses without any structured output format instruction by using the same prompt as used by Jahan et al. (2024). Afterward, two human annotators having backgrounds in NLP
<table>
  <tr>
    <th rowspan="2">LLM-Judge</th>
    <th colspan="2">BC5CDR</th>
    <th colspan="2">DDI</th>
    <th colspan="2">KD-DTI</th>
  </tr>
  <tr>
    <th>EM (\( \Delta \))</th>
    <th>RMSE (\( \nabla \))</th>
    <th>EM (\( \Delta \))</th>
    <th>RMSE (\( \nabla \))</th>
    <th>EM (\( \Delta \))</th>
    <th>RMSE (\( \nabla \))</th>
  </tr>
  <tr>
    <td>Phi-3.5-Mini-3.8B-Instruct</td>
    <td>33.80</td>
    <td>2.40</td>
    <td>43.06</td>
    <td>2.19</td>
    <td>35.55</td>
    <td>2.11</td>
  </tr>
  <tr>
    <td>Qwen-2.5-7B-Instruct</td>
    <td>45.25</td>
    <td>2.42</td>
    <td>46.60</td>
    <td>2.15</td>
    <td>49.98</td>
    <td>1.82</td>
  </tr>
  <tr>
    <td>LLaMA-3.1-8B-Instruct</td>
    <td>29.45</td>
    <td>2.40</td>
    <td>29.32</td>
    <td>2.95</td>
    <td>36.73</td>
    <td>2.10</td>
  </tr>
  <tr>
    <td>Deepseek-R1-Qwen-7B</td>
    <td>30.60</td>
    <td>2.76</td>
    <td>42.67</td>
    <td>3.07</td>
    <td>42.45</td>
    <td>2.51</td>
  </tr>
  <tr>
    <td>Deepseek-R1-LLaMA-8B</td>
    <td>30.50</td>
    <td>3.37</td>
    <td>42.15</td>
    <td>4.16</td>
    <td>33.48</td>
    <td>3.25</td>
  </tr>
  <tr>
    <td>Claude-3-Haiku</td>
    <td>29.50</td>
    <td>2.26</td>
    <td>31.15</td>
    <td>2.70</td>
    <td>40.27</td>
    <td>1.83</td>
  </tr>
  <tr>
    <td>Gemini-Flash</td>
    <td>42.55</td>
    <td>2.09</td>
    <td>47.12</td>
    <td>2.11</td>
    <td>40.68</td>
    <td>1.98</td>
  </tr>
  <tr>
    <td>GPT-4o-Mini</td>
    <td><b>48.35</b></td>
    <td>2.33</td>
    <td><b>59.03</b></td>
    <td><b>1.84</b></td>
    <td><b>53.11</b></td>
    <td><b>1.81</b></td>
  </tr>
</table>

Table 1: Performance of different *LLM-Judges* on the responses generated by the *LLM-Generators* in Jahan et al. (2024) across three datasets: BC5CDR, DDI, and KD-DTI. The Exact Match (EM) Accuracy (higher is better, indicated by \( \Delta \)) and the Root Mean Squared Error (RMSE) (lower is better, indicated by \( \nabla \)) are reported. The reported score for each *LLM-Judge* is the average of their evaluations for all *LLM-Generators* within each dataset.

<table>
  <tr>
    <th rowspan="2">LLM-Judge</th>
    <th colspan="2">BC5CDR</th>
    <th colspan="2">DDI</th>
    <th colspan="2">KD-DTI</th>
  </tr>
  <tr>
    <th>Structured</th>
    <th>Unstructured</th>
    <th>Structured</th>
    <th>Unstructured</th>
    <th>Structured</th>
    <th>Unstructured</th>
  </tr>
  <tr>
    <td>Phi-3.5-Mini-3.8B-Instruct</td>
    <td>53.80</td>
    <td>0.94</td>
    <td>35.60</td>
    <td>2.74</td>
    <td>32.89</td>
    <td>1.92</td>
    <td>32.51</td>
    <td>2.42</td>
    <td>36.94</td>
    <td>2.02</td>
    <td>36.74</td>
    <td>2.55</td>
  </tr>
  <tr>
    <td>Qwen-2.5-7B-Instruct</td>
    <td>67.80</td>
    <td>0.93</td>
    <td>44.60</td>
    <td>2.84</td>
    <td>43.93</td>
    <td>1.64</td>
    <td>43.46</td>
    <td>2.08</td>
    <td>61.78</td>
    <td>1.19</td>
    <td>55.82</td>
    <td>2.18</td>
  </tr>
  <tr>
    <td>LLaMA-3.1-8B-Instruct</td>
    <td>59.20</td>
    <td>0.85</td>
    <td>33.00</td>
    <td>2.67</td>
    <td>37.70</td>
    <td>1.99</td>
    <td>30.37</td>
    <td>2.43</td>
    <td>40.38</td>
    <td>2.44</td>
    <td>38.22</td>
    <td>2.46</td>
  </tr>
  <tr>
    <td>Deepseek-R1-Qwen-7B</td>
    <td>57.00</td>
    <td>1.28</td>
    <td>31.60</td>
    <td>3.32</td>
    <td>41.88</td>
    <td>2.55</td>
    <td>35.60</td>
    <td>2.85</td>
    <td>47.71</td>
    <td>1.83</td>
    <td>46.16</td>
    <td>2.65</td>
  </tr>
  <tr>
    <td>Deepseek-R1-LLaMA-8B</td>
    <td>54.40</td>
    <td>2.25</td>
    <td>37.40</td>
    <td>3.64</td>
    <td>37.70</td>
    <td>4.14</td>
    <td>35.60</td>
    <td>4.62</td>
    <td>48.40</td>
    <td>2.68</td>
    <td>42.36</td>
    <td>3.31</td>
  </tr>
  <tr>
    <td>Claude-3-Haiku</td>
    <td>67.80</td>
    <td><b>0.60</b></td>
    <td>36.80</td>
    <td>2.67</td>
    <td>32.46</td>
    <td>3.01</td>
    <td>25.13</td>
    <td>2.80</td>
    <td>59.28</td>
    <td>1.22</td>
    <td>50.04</td>
    <td>2.22</td>
  </tr>
  <tr>
    <td>Gemini-1.5-flash</td>
    <td><b>70.20</b></td>
    <td>0.66</td>
    <td>48.40</td>
    <td>2.75</td>
    <td>43.98</td>
    <td>1.75</td>
    <td>39.27</td>
    <td>2.29</td>
    <td>41.59</td>
    <td>1.68</td>
    <td>41.52</td>
    <td>2.47</td>
  </tr>
  <tr>
    <td>GPT-4o-mini</td>
    <td>67.20</td>
    <td>0.73</td>
    <td>43.20</td>
    <td>2.72</td>
    <td><b>53.93</b></td>
    <td><b>1.45</b></td>
    <td>47.12</td>
    <td>2.14</td>
    <td><b>72.39</b></td>
    <td><b>1.08</b></td>
    <td>66.35</td>
    <td>2.13</td>
  </tr>
</table>

Table 2: Performance Comparison between *Structured* (our proposed) and *Unstructured* (baseline) output format. All the responses are generated using GPT-4-Turbo as the *LLM-Generator*.

<table>
  <tr>
    <th rowspan="2">LLM-Judge</th>
    <th colspan="2">Data</th>
    <th colspan="2">Fine-Tuning</th>
    <th colspan="2">Evaluation</th>
    <th colspan="2">EM Accuracy</th>
  </tr>
  <tr>
    <th></th>
    <th></th>
    <th>BC5CDR</th>
    <th>KD-DTI</th>
    <th>BC5CDR</th>
    <th>KD-DTI</th>
    <th>BC5CDR</th>
    <th>KD-DTI</th>
  </tr>
  <tr>
    <td>Qwen-2.5-7B-Instruct</td>
    <td></td>
    <td>BC5CDR</td>
    <td>KD-DTI</td>
    <td>75.75 (+13.97)</td>
    <td>71.40 (+3.60)</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Phi-3.5-Mini-3.8B-Instruct</td>
    <td></td>
    <td>BC5CDR</td>
    <td>KD-DTI</td>
    <td>69.54 (+32.60)</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
  <tr>
    <td>Phi-3.5-Mini-3.8B-Instruct</td>
    <td></td>
    <td>KD-DTI</td>
    <td>BC5CDR</td>
    <td>64.80 (+11.00)</td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
    <td></td>
  </tr>
</table>

Table 3: Effectiveness of Domain Adaptation via Transfer Learning. The value in brackets indicates the performance gain relative to the Zero-Shot results for *Structured* output in respective evaluation datasets in Table 2.

and biomedicine annotated the *correctly predicted relations* and the *total predicted relations* in the GPT-4-Turbo generated responses for both structured and unstructured cases. When discrepancies arise between annotations from different annotators, they are resolved through discussions.

We then evaluate the performance of *LLM-Judges* in these structured responses as well as the unstructured responses generated by GPT-4-Turbo. Based on the results presented in Table 2, we find that structured formatting consistently improves the performance for all models. While in the BC5CDR dataset, Gemini-1.5-Flash achieves the best performance in terms of accuracy with Claude-3-Haiku achieving the best result in terms of the RMSE metric, in other two datasets (DDI and KD-DTI), GPT-4o-Mini achieves the best performance in terms of both metrics. For open-source LLMs, in the BC5CDR dataset, Qwen-2.5-7B has the best accuracy and LLaMA-3.1-8B performs the best in terms of RMSE distance. However, the accuracy is quite low for LLaMA-3.1-8B while being larger than Qwen-2.5-7B. In the other two datasets (DDI and KD-DTI), Qwen-2.5-7B-Instruct achieves the best result in terms of both metrics among open-source models, even outperforming reasoning-based DeepSeek-Distilled models.

For all three datasets, we find based on paired t-test that the performance difference between the *structured* and *unstructured* approach is statistically significant (\( p < 0.05 \)) for both metrics.

5.3 Effectiveness of Domain Adaptation via Transfer Learning

In our prior experiments, we demonstrate that proprietary LLMs demonstrate better performance as the judge in zero-shot scenarios. However, issues such as transparency, reproducibility, and cost highlight the need for open-source *LLM-Judge*. While
early work (Kim et al., 2023, 2024) demonstrates the effectiveness of training an open-source *LLM Judge*, to our best knowledge, there is no open-source *LLM-Judge* trained on biomedical data currently available. Since there is also a lack of human-annotated judgment data that prohibits the training of *LLM-Judges* for relation extraction, we investigate a domain adaptation technique via transfer learning to address this low-resource problem.

Recent research has demonstrated that language models can effectively transfer knowledge from one dataset to another (Garg et al., 2020; Laskar et al., 2022). Inspired by the idea, in this work, we also propose transfer learning from one relation extraction dataset to the other. Based on our human-annotated labels for the *Structured* output in Section 5.2, we investigate two scenarios: (i) Fine-Tuning on the BC5CDR dataset (500 samples) and Evaluation on the KD-DTI dataset (1159 samples), and (ii) Fine-Tuning on the KD-DTI dataset (1159 samples) and Evaluation on the BC5CDR dataset (500 samples). For hyperparameter tuning, we use the DDI dataset as the validation set. We fine-tune the Qwen-2.5-7B-Instruct and Phi-3.5-3.8B-Instruct models because our previous experiments demonstrated that Qwen with 7 billion parameters outperforms the larger 8-billion parameter LLaMA model, while Phi-3.5-3.8B achieves comparable performance with the 2x larger LLaMA model.

Based on the results presented in Table 3, we find that our proposed domain adaptation strategy by transferring knowledge via fine-tuning on a limited amount of labeled data could significantly improve the accuracy. The performance gains in BC5CDR and KD-DTI by Qwen-2.5-7B-Instruct even outperform the closed-source models in Table 2. This demonstrates the effectiveness of our domain adaptation technique in low-resource scenarios.

5.4 Impact of Model Scaling

Since one of our motivations behind *LLM-Judge* is to reduce cost and improve efficiency to ensure their real-world utilization, we primarily selected open-source models having less than 10B parameters or the most cost-efficient version of different closed-source LLM providers. In this section, we investigate two scenarios:

(i) What is the impact of reducing the size of the best-performing open-source model?
(ii) Can domain adaptation via fine-tuning help even smaller models outperform larger models?

Below, we demonstrate our findings.

![Bar chart comparing EM Accuracy between Qwen-2.5-1.5B-Instruct and Qwen-2.5-7B-Instruct on BC5CDR, DDI Model, and KD-DTI datasets](page_1012_104_388_246.png)

Figure 5: Performance comparison between Qwen models based on size: 1.5B and 7B, based on evaluating on the GPT-4-Turbo generated *Structured* responses.

![Bar chart comparing EM Accuracy for Qwen-2.5-1.5B-Fine-tuned, Qwen-2.5-3B-Fine-tuned, Qwen-2.5-7B-Zero-Shot, and Qwen-2.5-7B-Fine-tuned](page_1012_420_388_246.png)

Figure 6: Comparing fine-tuned smaller models, Qwen-2.5B-Instruct (1.5B and 3B) against the larger model: Qwen-2.5B-7B-Instruct (both zero-shot and fine-tuned). The KD-DTI dataset with *Structured* response is used.

Reducing the Model Size: In Figure 5, we show the performance difference between the 1.5B and 7B Qwen models in zero-shot and find that reducing the model size significantly drops the accuracy. More specifically, the performance drops by 58.70%, 45.12%, and 74.44%, in BC5CDR, DDI, and KD-DTI, respectively.

Fine-tuning Smaller Models: We investigate the impact of fine-tuning smaller Qwen-2.5-Instruct models: 1.5B and 3B (we did not show their zero-shot result as they could not generate the response in the required format in most cases, leading to very poor accuracy) in comparison to the larger zero-shot model (Qwen-2.5-7B-Instruct). Based on the results shown in Figure 6, we find that the 3B model, fine-tuned on 500 samples in the BC5CDR dataset (the evaluation is conducted on KD-DTI) can outperform the zero-shot 7B model, while achieving performance comparable to the fine-tuned 7B model. This makes it possible to use smaller models in resource-constrained scenarios.
5.5 Ablation Test
In our experiments, we include human-annotated gold relations in the prompt to maintain consistency with the work of (Jahan et al., 2024), where human annotators relied on the gold reference relations to assess the correctness of LLM-generated outputs. We further conduct an ablation experiment using GPT-4o as the judge on the BC5CDR dataset without providing the gold reference relations in the prompt and find that our structured approach again outperforms the unstructured baseline in the reference-free setting. The RMSE distance for the structured approach is 1.53, while for the unstructured approach it is 3.19.

6 Conclusion and Future Work
In this paper, we evaluated the effectiveness of LLMs-as-the-Judge for assessing biomedical relation extraction models, revealing significant performance gaps between \( LLM\text{-}Judges \) and human evaluators. While previous work has demonstrated the potential of LLM-based evaluation for general NLP tasks, our findings indicate that existing \( LLM\text{-}Judges \) struggle to reliably assess biomedical relation extraction due to the nuanced and domain-specific nature of biomedical text. To improve \( LLM\text{-}Judge \) performance, we proposed structured output for LLM-generated responses, which led to substantial accuracy gains across multiple datasets. Additionally, we introduced a domain adaptation technique that effectively transfers knowledge from one biomedical relation extraction dataset to another to enhance the reliability of \( LLM\text{-}Judges \).

In the future, we aim to explore instruction-tuning (Ouyang et al., 2022; Zhang et al., 2023) and chain-of-thought (Wei et al., 2022) prompting techniques to improve the performance of \( LLM\text{-}Judges \) in biomedical relation extraction. Moreover, conducting a more fine-grained analysis by constructing a detailed error taxonomy could also be considered as a future extension of this paper.

Acknowledgements
We would like to thank all the anonymous reviewers and the area chair for their excellent review comments, which helped us improve the overall quality of the paper. This research was supported by the Natural Sciences and Engineering Research Council (NSERC) of Canada, NSERC Discovery Grant, the York Research Chairs (YRC) program, and Compute Canada.

Limitations
Despite the effectiveness of structured output formatting and domain adaptation, \( LLM\text{-}Judges \) still fall short of human evaluators. Biomedical relation extraction requires deep domain-specific reasoning, and the foundation LLMs are trained on general-domain corpora (Zhao et al., 2023; Lu et al., 2024; Minaee et al., 2024). Therefore, they still struggle with complex relationships, ambiguous terms, and implicit entity connections that human experts can recognize (some examples in Appendix A.4.2 are provided demonstrating the common errors made by \( LLM\text{-}Judges \) while evaluating biomedical relations extracted by different \( LLM\text{-}Generators \)). Nonetheless, since our human-annotated judgment dataset will be released, future work may focus on improving the performance of the LLM-Judge.

In our study, we followed a strategy similar to prior work (Laskar et al., 2024b) by first experimenting with prompts on a small subset of the data to identify the optimal prompt that reliably produced outputs by following the intended instructions. While evaluating many prompting strategies is valuable, this process is computationally expensive. Therefore, we set a reasonably effective prompt and compared the LLM judge performance.

Moreover, our study primarily benchmarks a limited set of LLMs, and results may vary with the release of newer or more specialized biomedical models (Singhal et al., 2023; Lu et al., 2024; Saab et al., 2024). Furthermore, cost constraints prevented the exploration of larger, more computationally expensive models (e.g., reasoning models like OpenAI O3), which may have improved results but are impractical in the real world. Nonetheless, due to choosing cheaper closed-source models\footnote{https://docsbot.ai/models/compare}, Gemini-1.5-Flash saves cost approximately 13 times in comparison to Gemini-1.5-Pro, Claude-3-Haiku saves cost approximately 12 times and 60 times in comparison to Claude-3.5-Sonnet and Claude-3-Opus, respectively, and GPT-4o-Mini is approximately 17 times cheaper than GPT-4o. In terms of open-source LLMs, models having less than 10B parameters can be run using just 1 L4 or A100 GPU machine (Laskar et al., 2023b, 2025). This significantly saves the deployment cost (Fu et al., 2024). Moreover, using an LLM judge for evaluation instead of humans can also address the time-consuming and costly human annotation, which is another key motivation of this paper.
Ethical Considerations

Our proposed LLM-Judge is designed solely for the evaluation of LLM-generated responses in biomedical relation extraction and is not intended for direct use by end users in clinical applications. The accuracy and reliability of the relation extraction system depend on the *LLM-Generator*, which produces the relation extraction outputs. Since our proposed model only acts as an evaluator, the ethical risks associated with direct application in biomedical decision-making are minimized. By providing a scalable and efficient evaluation framework, our solution enables researchers and practitioners to quickly assess the quality of their biomedical relation extraction LLMs without relying on costly and time-consuming human evaluations. This can accelerate advancements in biomedical NLP while ensuring that models are assessed using standardized criteria. To further enhance reliability, a human-in-the-loop approach can be implemented where expert annotators verify the outputs of the models that achieve better performance based on *LLM-Judge* evaluation. Moreover, in this paper, additional compensations are not needed for the annotators since two of the authors conducted the human annotation.

References

Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harikrat Behl, et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. *arXiv preprint arXiv:2404.14219*.

Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. *arXiv preprint arXiv:2303.08774*.

Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. Palm 2 technical report. *arXiv preprint arXiv:2305.10403*.

Anthropic. 2023. Model card and evaluations for claude models.

Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku.

Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. 2023. A multitask, multilingual, multimodal evaluation of ChatGPT on reasoning, hallucination, and interactivity. In *Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)*, pages 675–718, Nusa Dua, Bali. Association for Computational Linguistics.

Elisa Bassignana and Barbara Plank. 2022. What do you mean by relation extraction? a survey on datasets and study on scientific relation classification. In *Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: Student Research Workshop*, pages 67–83, Dublin, Ireland. Association for Computational Linguistics.

Stella Biderman, Hailey Schoelkopf, Lintang Sutawika, Leo Gao, Jonathan Tow, Baber Abbasi, Alham Fikri Aji, Pawan Sasanka Ammanamanchi, Sidney Black, Jordan Clive, et al. 2024. Lessons from the trenches on reproducible evaluation of language models. *arXiv preprint arXiv:2405.14782*.

Qijie Chen, Haotong Sun, Haoyang Liu, Yinghui Jiang, Ting Ran, Xurui Jin, Xianglu Xiao, Zhimin Lin, Hongming Chen, and Zhangmin Niu. 2023. An extensive benchmark study on biomedical text generation and mining with chatgpt. *Bioinformatics*, 39(9):btad557.

Xue-Yong Fu, Md Tahmid Rahman Laskar, Elena Khasanova, Cheng Chen, and Shashi Bhushan TN. 2024. Tiny titans: Can smaller large language models punch above their weight in the real world for meeting summarization? *arXiv preprint arXiv:2402.00841*.

Siddhant Garg, Thuy Vu, and Alessandro Moschitti. 2020. Tanda: Transfer and adapt pre-trained transformer models for answer sentence selection. In *Proceedings of the AAAI conference on artificial intelligence*, volume 34, pages 7780–7788.

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhari, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. The llama 3 herd of models. *arXiv preprint arXiv:2407.21783*.

Jiawei Gu, Xuhui Jiang, Zhichao Shi, Hexiang Tan, Xuehao Zhai, Chengjin Xu, Wei Li, Yinghan Shen, Shengjie Ma, Honghao Liu, Yuanzhuo Wang, and Jian Guo. 2024. A survey on llm-as-a-judge. *Preprint*, arXiv:2411.15594.

Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, et al. 2023. Textbooks are all you need. *arXiv preprint arXiv:2306.11644*.

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.
Maria Herrero-Zazo, Isabel Segura-Bedmar, Paloma Martínez, and Thierry Declerck. 2013. The ddi corpus: An annotated corpus with pharmacological substances and drug–drug interactions. Journal of biomedical informatics, 46(5):914–920.

Yutai Hou, Yingce Xia, Lijun Wu, Shufang Xie, Yang Fan, Jinhua Zhu, Tao Qin, and Tie-Yan Liu. 2022. Discovering drug–target interaction knowledge from biomedical literature. Bioinformatics, 38(22):5100–5107.

Israt Jahan, Md Tahmid Rahman Laskar, Chun Peng, and Jimmy Huang. 2023. Evaluation of ChatGPT on biomedical tasks: A zero-shot comparison with fine-tuned generative transformers. In The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, pages 326–336, Toronto, Canada. Association for Computational Linguistics.

Israt Jahan, Md Tahmid Rahman Laskar, Chun Peng, and Jimmy Xiangji Huang. 2024. A comprehensive evaluation of large language models on benchmark biomedical text processing tasks. Computers in Biology and Medicine, page 108189.

AQ Jiang, A Sablayrolles, A Mensch, C Bamford, DS Chaplot, D de las Casas, F Bressand, G Lengyel, G Lample, L Saulnier, et al. 2023. Mistral 7b (2023). arXiv preprint arXiv:2310.06825.

Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, et al. 2023. Prometheus: Inducing fine-grained evaluation capability in language models. In The Twelfth International Conference on Learning Representations.

Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moonae Lee, Kyungjae Lee, and Minjoon Seo. 2024. Prometheus 2: An open source language model specialized in evaluating other language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 4334–4353.

Yanis Labrak, Adrien Bazoge, Emmanuel Morin, Pierre-Antoine Gourraud, Mickael Rouvier, and Richard Dufour. 2024. Biomistral: A collection of open-source pretrained large language models for medical domains. In Findings of the Association for Computational Linguistics ACL 2024, pages 5848–5864.

Md Tahmid Rahman Laskar, Sawsan Alqahtani, M Saiful Bari, Mizanur Rahman, Mohammad Abdul-Ilah Matin Khan, Haidar Khan, Israt Jahan, Amran Bhuiyan, Chee Wei Tan, Md Rizwan Parvez, Enamul Hoque, Shafiq Joty, and Jimmy Huang. 2024a. A systematic survey and critical review on evaluating large language models: Challenges, limitations, and recommendations. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13785–13816, Miami, Florida, USA. Association for Computational Linguistics.

Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, Md Amran Hossen Bhuiyan, Shafiq Joty, and Jimmy Huang. 2023a. A systematic study and comprehensive evaluation of ChatGPT on benchmark datasets. In Findings of the Association for Computational Linguistics: ACL 2023, pages 431–469, Toronto, Canada. Association for Computational Linguistics.

Md Tahmid Rahman Laskar, Xue-Yong Fu, Cheng Chen, and Shashi Bhushan TN. 2023b. Building real-world meeting summarization systems using large language models: A practical perspective. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 343–352, Singapore. Association for Computational Linguistics.

Md Tahmid Rahman Laskar, Enamul Hoque, and Jimmy Xiangji Huang. 2022. Domain adaptation with pre-trained transformers for query-focused abstractive text summarization. Computational Linguistics, 48(2):279–320.

Md Tahmid Rahman Laskar, Mohammed Saidul Islam, Ridwan Mahbub, Ahmed Masry, Mizanur Rahman, Amran Bhuiyan, Mir Tafseer Nayeeem, Shafiq Joty, Enamul Hoque, and Jimmy Huang. 2025. Judging the judges: Can large vision-language models fairly evaluate chart comprehension and reasoning? arXiv preprint arXiv:2505.08468.

Md Tahmid Rahman Laskar, Elena Khasanova, Xue-Yong Fu, Cheng Chen, and Shashi Bhushan Tn. 2024b. Query-OPT: Optimizing inference of large language models via multi-query instructions in meeting summarization. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1140–1151, Miami, Florida, US. Association for Computational Linguistics.

Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiquan Liu. 2024a. Lms-as-judges: a comprehensive survey on llm-based evaluation methods. arXiv preprint arXiv:2412.05579.

Jiao Li, Yueping Sun, Robin J Johnson, et al. 2016. Biocreative v cdr task corpus: a resource for chemical disease relation extraction. Database, 2016.

Yinghao Li, Rampi Ramprasad, and Chao Zhang. 2024b. A simple but effective approach to improve structured language model output for information extraction. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5133–5148, Miami, Florida, USA. Association for Computational Linguistics.

Zhiyong Lu, Yifan Peng, Trevor Cohen, Marzyeh Ghassemi, Chunhua Weng, and Shubo Tian. 2024. Large language models in biomedicine and health: current research landscape and future directions. Journal of the American Medical Informatics Association, 31(9):1801–1811.
Renqian Luo, Liai Sun, Yingce Xia, et al. 2022. BioGPT: generative pre-trained transformer for biomedical text generation and mining. Briefings in Bioinformatics, 23(6).

Shervin Minaee, Tomas Mikolov, Narjes Nikzad, Meysam Chenaghlou, Richard Socher, Xavier Amatriain, and Jianfeng Gao. 2024. Large language models: A survey. arXiv preprint arXiv:2402.06196.

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27750–27744.

Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. 2023. Is chatgpt a general-purpose natural language processing task solver? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1339–1384.

Khaled Saab, Tao Tu, Wei-Hung Weng, Ryutaro Tanno, David Stutz, Ellery Wulczyn, Fan Zhang, Tim Strother, Chunjong Park, Elahe Vedadi, et al. 2024. Capabilities of gemini models in medicine. arXiv preprint arXiv:2404.18416.

Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2023. Large language models encode clinical knowledge. Nature, pages 1–9.

Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.

Shubo Tian, Qiao Jin, Lana Yeganova, Po-Ting Lai, Qingqing Zhu, Xiuying Chen, Yifan Yang, Qingyu Chen, Won Kim, Donald C Comeau, et al. 2024. Opportunities and challenges for chatgpt and large language models in biomedicine and health. Briefings in Bioinformatics, 25(1):bbad493.

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023a. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971.

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajwal Bhargava, Shruti Bhosale, et al. 2023b. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288.

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837.

Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierre Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pages 38–45.

Congying Xia, Chen Xing, Jiangshu Du, Xinyi Yang, Yihao Feng, Ran Xu, Wenpeng Yin, and Caiming Xiong. 2024. FOFO: A benchmark to evaluate LLMs’ format-following capability. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 680–699, Bangkok, Thailand. Association for Computational Linguistics.

An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024a. Qwen2 technical report. arXiv preprint arXiv:2407.10671.

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024b. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115.

An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengyuan Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. 2024c. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122.

Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuie Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. 2023. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792.

Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223.

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging IIm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595–46623.

Zexuan Zhong and Danqi Chen. 2021. A frustratingly easy approach for entity and relation extraction. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 50–61.
A Appendix

A.1 Structured Prompt for the LLM-Generator

Prompt: KD-DTI Dataset
Identify the drug-target interactions in the biomedical text given below (along with the interaction type among the following: 'inhibitor', 'agonist', 'modulator', 'activator', 'blocker', 'inducer', 'antagonist', 'cleavage', 'disruption', 'intercalation', 'inactivator', 'bind', 'binder', 'partial agonist', 'cofactor', 'substrate', 'ligand', 'chelator', 'downregulator', 'other', 'antibody', 'other/unknown'):

[BIOMEDICAL TEXT]

Prompt: BC5CDR Dataset
Identify each pair of drugs (e.g., chemicals) and the drug-induced side-effects (e.g., diseases) in the following passage:

[BIOMEDICAL TEXT]

Prompt: DDI Dataset
Identify the pairs of drug-target interactions in a given passage based on the following four interaction types:
(i) mechanism: This type is used to identify drug-drug interactions that are described by their pharmacokinetic mechanism.
(ii) effect: This type is used to identify drug-drug interactions describing an effect.
(iii) advice: This type is used when a recommendation or advice regarding a drug-drug interaction is given.
(iv) int: This type is used when a drug-drug interaction appears in the text without providing any additional information.

[BIOMEDICAL TEXT]

A.2 Models

• GPT-4-Turbo: It is an advanced version of OpenAI’s original GPT-4 (Achiam et al., 2023). The GPT-4-Turbo11 model offers enhanced performance and efficiency, making it suitable for a wide range of applications requiring natural language understanding and generation.

• GPT-4o-Mini: The GPT-4o-Mini12 is another optimized version of GPT-4. More specifically, it is a more optimized version of the recently released GPT-4o. It balances robust language understanding with efficiency. It’s designed to handle complex tasks while significantly reducing operational costs.

• Gemini-1.5-Flash: Part of Google’s Gemini-1.5 family, this model emphasizes rapid inference and the ability to handle extremely long contexts, up to one million tokens (Team et al., 2023). Gemini-1.5-Flash13 is ideal for real-time applications where speed and processing of large amounts of data are crucial.

• Claude-3-Haiku Similar to GPT-4o-Mini and Gemini-1.5-Flash, it is the most cost-optimized version of the Claude-3 series (Anthropic, 2024). The Claude-3-Haiku14 model is tailored for succinct, creative outputs. It excels at producing elegant, brief responses, while still managing complex instructions and reasoning tasks.

• LLaMA-3.1-8B-Instruct: This 8 billion parameter variant from Meta’s LLaMA-3 (Grattafiori et al., 2024) series has been fine-tuned for instruction following. It strikes a balance between computational efficiency and performance, outperforming its earlier versions (Touvron et al., 2023a,b) and making it suitable for a wide range of tasks.

• Phi-3.5-Mini-8B-Instruct: A compact, instruction-optimized model from Microsoft’s Phi series (Gunasekar et al., 2023). Despite its smaller size, it’s designed to understand and execute diverse tasks in resource-constrained environments while maintaining strong performance (Abdin et al., 2024).

• Qwen-2.5-7B-Instruct Developed by Alibaba, this 7-billion-parameter model is tuned for following instructions. It offers a good balance between efficiency and output quality (Yang et al., 2024a,b).

• DeepSeek-R1-Distilled Models: DeepSeek-R1 (Guo et al., 2025) is a reasoning model developed by DeepSeek-AI, which is mostly trained via large-scale reinforcement learning. DeepSeek-R1-Distill-Qwen-7B and

11https://platform.openai.com/docs/models/gpt-4-and-gpt-4-turbo
12https://openai.com/index/gpt-4o-mini-advancing-cost-efficient-intelligence/
13https://ai.google.dev/gemini-api/docs/models/gemini#gemini-1.5-flash
14https://www.anthropic.com/news/clause-3-haiku
a. Human-Annotated Gold Relation
drug: dex, bort
side-effect: peripheral neuropathy
drug: bort,
side-effect: peripheral neuropathy

b. LLM Predicted Relation
Pair: Rituximab and dexamethasone
Side-effect: Peripheral neuropathy

c. Human Evaluation
{
"correctly_predicted_relations": 1,
"total_predicted_relations": 2
}

d. LLM-Judge Evaluation
{
"correctly_predicted_relations": 0,
"total_predicted_relations": 2
}

Figure 7: Our Evaluation Process where both the (c) Human Judge and the (d) LLM-Judge annotate the correctly predicted relations and the total predicted relations by comparing between (a) Human-Annotated Gold Relation and (b) LLM Predicted Relation. The biomedical text from where the relations are extracted is also provided as a context (not shown here).

DeepSeek-R1-Distill-LLaMA-8B are the distilled version of Qwen-2.5-7B-Math (Yang et al., 2024c) and LLaMA-3.1-8B-Instruct (Grattafiori et al., 2024), respectively, fine-tuned on 800k synthetic reasoning data generated from DeepSeek-R1.

A.3 Evaluation Process Demonstration

In Figure 7, we show our evaluation process. Since there is a mismatch between the gold human annotation and the LLM-Judge annotation for the correctly predicted relations and total predicted relations, the exact match score is 0. The RMSE distance is calculated as follows:

\[
\sqrt{\frac{1}{2} \left( (1 - 0)^2 + (2 - 2)^2 \right)} = \sqrt{0.5} \approx 0.71.
\]

A.4 Error Analysis

A.4.1 BioMistral-7B-as-Judge Outputs

We provide some sample responses generated by BioMistral-7B model (Labrak et al., 2024), based on Mistral-7B (Jiang et al., 2023), as-the-Judge in Table 4, demonstrating its ineffectiveness as the judge by generating improper responses.

A.4.2 Error Outputs by LLM-Judges

We show some of the common error outputs of the LLM-Judges in Table 5. For the error output analysis, we use the overall best-performing zero-shot LLM, the GPT-4o-Mini model and demonstrate how complex biomedical terms could make it difficult for LLMs to evaluate relations extracted in an unstructured format. For instance:

• Example 1 in Table 5 demonstrates a case when 2 drugs and 6 corresponding side effects are extracted by an LLM-Generator, but the GPT-4o-Mini based LLM-Judge only considers total relations as 6 instead of 12. Moreover, it only extracted 5 as correct while the correct should be 4 side-effects for each drug (in total 8 side-effects for the 2 drugs are correct).

• Example 2 in Table 5 demonstrates a case when 2 drugs and 1 corresponding side effect for each of them have been extracted by an LLM-Generator. While drug: thiopentone and disease (side-effects): delirium is one of the correct answers, the LLM-Judge based on GPT-4o-Mini considers only 1 side effect and 1 drug. It is highly likely that GPT-4o-Mini considers Thiopentone and propofol as a single drug even though they should be considered different. This could be the reason it considers no correct relation is extracted. This demonstrates GPT-4o-Mini having limitations in understanding complex biomedical terms.

• Example 3 in Table 5 demonstrates a case when 2 drugs and 6 side effects in total are extracted by an LLM-Generator. While Pair 1 and Pair 3 have 1 side effect each, Pair 2 has 4 different side-effects. However, GPT-4o-Mini based LLM-Judge may have considered all these 4 side-effects as just 1 side-effect. This demonstrates its lack of biomedical text understanding capability.
<table>
  <tr>
    <th>Dataset</th>
    <th>Output Format</th>
    <th>Gold Relations</th>
    <th>Predicted Relations</th>
    <th>BioMistral-7B-as-Judge-Annotation</th>
  </tr>
  <tr>
    <td>BC5CDR</td>
    <td>Unstructured</td>
    <td>drug: famotidine disease (side-effects): delirium</td>
    <td>Famotidine - delirium</td>
    <td>{"correctly_predicted_relations": [" "chemical" : "famotidine", "disease": "delirium" ], "total_predicted_relations": 2}</td>
  </tr>
  <tr>
    <td>KD-DT1</td>
    <td>Unstructured</td>
    <td>drug: PNU156804 target: Janus kinase 3 (JAK-3) interaction: inhibitor</td>
    <td>PNU156804 - Janus tyrosine kinase 3 (Jak3) - inhibitor<br>PNU156804 - IL-2-mediated Jak3 auto phosphorylation/ block<br>PNU156804 - Jak3 substrates signal transducers and activators of transcription (Stat) 5a and Stat5b - blocker<br>PNU156804 - extracellular regulated kinase 1 (ERK1/2) (p Elk2 (p44/p42)) - blocker</td>
    <td>{"correctly_predicted_relations": [" "drug": "PNU156804", "target": "Janus kinase 3 (JAK-3)", "interaction": "inhibitor" ]}</td>
  </tr>
  <tr>
    <td>DDI</td>
    <td>Unstructured</td>
    <td>drug: gemfibrozil target: rosiglitazone interaction: mechanism<br>drug: rifampin target: niacin interaction: mechanism</td>
    <td>gemfibrozil-rosiglitazone, mechanism<br>rifampin-rosiglitazone, mechanism<br>CYP2C9 inhibitor/rosiglitazone, advice<br>CYP2C9 inducer/rosiglitazone, advice</td>
    <td>{"correctly_predicted_relations": [1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}</td>
  </tr>
  <tr>
    <td>BC5CDR</td>
    <td>Structured</td>
    <td>drug: cyclosporine disease (side-effects): thrombotic microangiopathy<br>drug: corticosteroids disease (side-effects): ssc<br>drug: tacrolimus disease (side-effects): ssc</td>
    <td>[{"chemical": "tacrolimus", "disease": "scleroderma renal crisis"}, {"chemical": "prednisolone", "disease": "scleroderma renal crisis"}]</td>
    <td>{"correctly_predicted_relations": ["chemical": "tacrolimus", "disease": "scleroderma renal crisis"], "total_predicted_relations": 2}</td>
  </tr>
  <tr>
    <td>KD-DT1</td>
    <td>Structured</td>
    <td>1. [ "drug": "thiorphan", "target": "neutral endopeptidase (NEP)", "interaction": "inhibitor" ]</td>
    <td>Drug: Thiorphan<br>target: Neutral endopeptidase (MME)<br>interaction: inhibitor</td>
    <td>{"correctly_predicted_relations": [" "drug": "Thiorphan", "target": "Neutral endopeptidase (MME)", "interaction": "inhibitor" ], "total_gold_relations": 1, "percentage_correctly_predicted_relations": 100.0, "total_predicted_relations": 1}</td>
  </tr>
  <tr>
    <td>DDI</td>
    <td>Structured</td>
    <td>1. [ "drug": "CYLERT", "target": "other drugs", "interaction": "inhibitor" ].<br>2. [ "drug": "CYLERT", "target": "antiepileptic medications", "interaction": "effect" ]</td>
    <td>It is important to note that these are drug-induced side-effects or outcomes observed in rats during the study, and further investigation is needed to understand the mechanisms and its relevance to human health.</td>
    <td>{"correctly_predicted_relations": [" "drug": "CYLERT", "target": "antiepileptic medications", "interaction": "effect" ], "total_predicted_relations": 2}</td>
  </tr>
</table>

Table 4: Sample judgment outputs generated by BioMistral-7B in different datasets.

<table>
  <tr>
    <th># Example</th>
    <th>Gold Relations</th>
    <th>Predicted Relations</th>
    <th>Human Annotated Correct Relations</th>
    <th>Human Annotated Total Relations</th>
    <th>LLM Predicted Correct Relations</th>
    <th>LLM Predicted Total Relations</th>
  </tr>
  <tr>
    <td>1.</td>
    <td>drug: s-53482<br>disease (side-effects): growth retardation<br>drug: s-23121<br>disease (side-effects): growth retardation<br>disease (side-effects): teratogenic<br>disease (side-effects): embryonic death<br>drug: s-53482<br>disease (side-effects): embryonic death<br>drug: s-53482<br>disease (side-effects): teratogenic<br>drug: s-23121<br>disease (side-effects): ventricular septal defect<br>drug: s-53482<br>disease (side-effects): ventricular septal defect</td>
    <td>Drugs:<br>1. S-53482<br>2. S-23121<br>Drug-induced side-effects:<br>1. Embryolethality<br>2. Teratogenicity<br>3. Ventricular septal defects<br>4. Wavy ribs<br>5. Growth retardation<br>6. Increased incidence of embryonic death<br>It is important to note that these are drug-induced side-effects or outcomes observed in rats during the study, and further investigation is needed to understand the mechanisms and its relevance to human health.</td>
    <td>8</td>
    <td>12</td>
    <td>5</td>
    <td>6</td>
  </tr>
  <tr>
    <td>2.</td>
    <td>drug: thiopentone<br>disease (side-effects): postoperative delirium<br>drug: thiopentone<br>disease (side-effects): delirium</td>
    <td>Pair of drugs: Thiopentone and propofol<br>Drug-induced side-effect: Delirium</td>
    <td>1</td>
    <td>2</td>
    <td>0</td>
    <td>1</td>
  </tr>
  <tr>
    <td>3.</td>
    <td>drug: dox<br>disease (side-effects): cardiotoxicity<br>drug: dox<br>disease (side-effects): necrosis</td>
    <td>Pair 1:<br>Drug - Doxorubicin<br>Drug-induced side-effects - Cardiotoxicity<br>Pair 2:<br>Drug - Doxorubicin<br>Drug-induced side-effects - Cardiac disarrangement, necrosis, DNA damage<br>(strand breaks and oxidized pyrimidines), decreased total antioxidant performance (TAP)<br>Pair 3:<br>Drug - Doxorubicin<br>Drug-induced side-effects - Increased resistance to oxidative stress</td>
    <td>2</td>
    <td>6</td>
    <td>2</td>
    <td>3</td>
  </tr>
</table>

Table 5: Example of annotation errors by GPT-4o-Mini-as-the-Judge on some sampled data in the BC5CDR dataset.

Reviewer paper summaries: This paper investigates the use of Large Language Models (LLMs) as judges for evaluating other LLMs in biomedical relation extraction tasks. The authors identify a critical challenge: traditional metrics fail to capture semantically equivalent but lexically different outputs, while human evaluation is costly and time-consuming. Through extensive experimentation with 8 LLM judges evaluating 5 LLM generators across 3 biomedical datasets, they discover that LLM judges perform poorly in biomedical domains (typically below 50% accuracy) despite success in general domains. To address this limitation, they propose two effective solutions: structured output formatting, which improves performance by approximately 15%, and a domain adaptation technique that transfers knowledge between datasets to further enhance evaluation capabilities. The paper includes comprehensive error analysis and will release a valuable dataset of 36k human and LLM judgments to support future research in this area.

In the generative AI era, LLMs are utilized to extract relations from texts in diverse domains. As both human evaluation and having gold data for evaluation is costly and time consuming, LLMs are also being used as a judge. This paper focuses on the use of LLMs as the judge for evaluating LLMs for biomedical relation extraction. It demonstrates that LLM-based judges perform poorly in this domain, with a significant gap between human evaluators and LLM judges. The study highlights that LLMs struggle to adhere to the required relation extraction formats, leading to unreliable evaluations. To address these issues, the authors propose structured output formats and domain adaptation to enhance LLM performance as a judge. Extensive experiments underscore the limitations of LLM evaluations in biomedical relation extraction and show the effectiveness of the proposed approach.
Summary of strengths: The paper identifies a significant gap in current evaluation methods for biomedical relation extraction, demonstrating that LLMs-as-judges perform poorly in specialized biomedical tasks (often below 50% accuracy) despite success in general domains.

This finding challenges assumptions about the generalizability of LLM evaluation capabilities and highlights domain-specific challenges in biomedical text processing.

The authors propose structured output formatting as a concrete solution, requiring LLM-Generators to produce responses in a structured JSON format, which improves LLM-Judge performance by approximately 15% on average.

Their domain adaptation technique effectively addresses the lack of human-annotated judgment data by transferring knowledge between datasets, further enhancing LLM-Judge performance.

The paper includes extensive experimentation, benchmarking 8 different LLMs as judges and evaluating responses from 5 different LLM-Generators across 3 biomedical relation extraction datasets.

The authors conducted over 100 experiments to analyze various factors affecting LLM-Judge performance, providing robust evidence for their claims.

They plan to release a large dataset of both human-annotated and LLM-annotated judgments (36k samples in total), which will be a valuable resource for the research community.

The paper considers practical implementation concerns like computational efficiency and cost, specifically testing smaller models (under 10B parameters) that can run on a single GPU.

Detailed error analysis and ablation studies provide insights into why LLM-Judges struggle with biomedical relation extraction and how different factors affect performance.

The work addresses a practical challenge in biomedical NLP evaluation, contributing to making biomedical relation extraction more scalable and practical for real-world applications.

1. A very relevant problem, clearly stated problem statement, clear demonstration of issues, and demonstration of the effectiveness of the proposed solution
2. well-structured and comprehensive set of experiments incorporating a diverse set of open-source LLMs (Phi, Qwen, Llama, DeepSeek) along with proprietary models (Claude, Gemini)
3. Model size-based evaluations provide insights into how different parameter scales impact structured response generation, highlighting whether smaller models can match larger ones in performance.
Summary of weaknesses: The paper mentions that the prompt for LLM judges was designed based on "extensive prompt engineering," yet provides little detail about this process or exploration of alternative prompting strategies. Given that prompt design can significantly impact LLM performance, a more systematic investigation of different prompting approaches would strengthen the paper.

Although the paper demonstrates the effectiveness of JSON formatting, it doesn't explore variations in structured output (e.g., different JSON schemas, XML, or other formats) to determine the optimal structure for biomedical relation extraction evaluation.

While the authors mention trying BioMistral-7B but found it ineffective as a judge, there are other biomedical-specialized LLMs (like BioGPT, BioMedLM, etc.) that could have been evaluated. A more comprehensive comparison with domain-specific models would provide better insights into whether general-purpose or domain-specific models are more effective for this task.

The paper evaluates individual LLM judges but doesn't explore whether combining judgments from multiple LLMs could improve reliability, which has been shown effective in other evaluation contexts. Such ensemble approaches might mitigate individual model weaknesses.

The paper includes some error analysis in the appendix, but lacks a systematic categorization of error types and their frequencies. This would help readers better understand the specific challenges LLM judges face in biomedical relation extraction and potentially guide future improvements.

The paper focuses on comparing LLM judges with human evaluation but doesn't thoroughly compare with simpler automated metrics that could be enhanced for biomedical text (e.g., BERTScore with biomedical embeddings), which might provide more efficient alternatives in some cases.

1. The study uses exact match accuracy and root mean squared error (RMSE) as evaluation metrics, which may not fully capture aspects like semantic correctness, partial correctness, or robustness to variations in structured responses. In my view, it is a major weakness of the study restricting its broader adaptation.

2. No justification for the choice of datasets, despite the availability of other datasets with larger instance counts and more diverse relations (https://academic.oup.com/bib/article/25/3/bbae132/7644532). The omission limits the scope of the evaluation, as broader datasets could better reflect LLMs' relational understanding and challenge their generalization capabilities.
Comments, suggestions and typos: The authors could explore chain-of-thought prompting, which they mention as future work. Incorporating some preliminary experiments with this approach could strengthen the current paper, especially since reasoning-based models like DeepSeek-R1 were included in the study.

A more detailed analysis of how biomedical domain knowledge affects judgment accuracy would enhance the paper. Examining whether errors correlate with the rarity or complexity of biomedical terms could provide insights into the knowledge gaps of current LLMs.

While the paper explores zero-shot and fine-tuning approaches, few-shot prompting with examples could be a middle ground worth exploring, potentially offering improvements without the computational cost of full fine-tuning.

The paper doesn't discuss how the length or complexity of the biomedical text affects judgment accuracy, which could be an important factor in understanding when LLM judges are most reliable.

A more granular analysis of exactly how structured output helps would provide deeper insights. For instance, examining whether it helps more with certain types of relations or entities could guide future improvements in prompt design.

Suggestions -
1. A more comprehensive analysis could be conducted by comparing the LLMs beyond just their sizes. Incorporating both quantitative and qualitative analyses of the generated data, as well as how LLMs judge it, could provide deeper insights and potentially lead to new findings.
Overall assessment scores: 4

3
Metareview score: 3.5
Metareview: The paper explores the use of large language models (LLMs) as evaluators of outputs generated by other LLMs in biomedical relation extraction tasks. Since LLM-generated biomedical relations often appear in diverse formats, direct comparison with gold-standard answers can be challenging. To address this, the authors propose using separate LLMs as judges to assess whether the generated outputs align with the gold answers. Extensive experiments were conducted using 8 LLM judges and 5 LLM generators across 3 biomedical datasets. The findings reveal that, despite their success in general domains, LLM judges struggle in biomedical settings. To mitigate this, the authors propose two strategies: enforcing structured output formats and applying a domain adaptation technique that transfers knowledge across datasets to improve evaluation performance.
Summary of reasons to publish: - "LLM-as-a-judge" is a timely research.
- Extensive experiments were conducted and the results support the effectiveness of the presented method.