prompt:
  template: |-
    Your mission is to judge the response from an AI model, the *test* response, calibrating your judgement using a *baseline* response.
    Please use the following rubric criteria to judge the responses:

    <START OF RUBRICS>
    Your task is to analyze the test response based on the criterion of "Instruction Following". Start your analysis with "Analysis".

    **Instruction Following**
    Please first list the instructions in the user query.
    In general, an instruction is VERY important if it is specifically asked for in the prompt and deviates from the norm. Please highlight such specific keywords.
    You should also derive the task type from the user query and include the task-specific implied instructions.
    Sometimes, no instruction is available in the user query.
    It is your job to infer if the instruction is to autocomplete the user query or is asking the LLM for follow-ups.
    After listing the instructions, you should rank them in order of importance.
    After that, INDEPENDENTLY check if the test response and the baseline response meet each of the instructions.
    You should itemize, for each instruction, whether the response meets, partially meets, or does not meet the requirement, using reasoning.
    You should start reasoning first before reaching a conclusion about whether the response satisfies the requirement.
    Citing examples while reasoning is preferred.

    Reflect on your answer and consider the possibility that you are wrong.
    If you are wrong, explain clearly what needs to be clarified, improved, or changed in the rubric criteria and guidelines.

    In the end, express your final verdict as one of the following three json objects:

    ```json
    {
      "Instruction Following": "No Issues"
    }
    ```

    ```json
    {
      "Instruction Following": "Minor Issue(s)"
    }
    ```

    ```json
    {
      "Instruction Following": "Major Issue(s)"
    }
    ```

    <END OF RUBRICS>

    # Your task
    ## User query
    <|begin_of_query|>
    {{full_prompt}}
    <|end_of_query|>

    ## Test Response:
    <|begin_of_test_response|>
    {{response_a}}
    <|end_of_test_response|>

    ## Baseline Response:
    <|begin_of_baseline_response|>
    {{response_b}}
    <|end_of_baseline_response|>

    Please write your analysis and final verdict for the test response.
  template_variables:
    - full_prompt
    - response_a
    - response_b
  metadata:
    description: "An evaluation prompt from the paper 'The FACTS Grounding Leaderboard: Benchmarking LLMs’ Ability to Ground
      Responses to Long-Form Input' by Google DeepMind.\n    The prompt was copied from the evaluation_prompts.csv file from
      Kaggle.\n    This specific prompt elicits a three class classifier to detect issues linked to instruction following
      with context.\n    Note that the double {{}} around the json blocks was simplified to a single {}."
    evaluation_method: ineligible_responses_filter_with_context
    tags:
      - fact-checking
    version: 1.0.0
    author: Google DeepMind
    source: https://www.kaggle.com/datasets/deepmind/FACTS-grounding-examples?resource=download&select=evaluation_prompts.csv
  client_parameters: {}
  custom_data: {}