Repository: prometheus-eval/prometheus-eval
Issue: How to evaluate HHH, MT_Bench_human? Where to get human scores for other val sets?
Root cause: Documentation Deficiency

**1. Can we say Preference data is an extension of Feedback data? There is a huge difference in numbers.**
(I know I'm making set of strings so not a good indicator but still sticking to my question to confirm whether Old and New rubrics are same or not)

Hi, while testing very vaguely, I found that there is difference in data. Not an issue but just asking why didn't you use the extension of Old data to generate new data `(Answer B)`? Seems like very few rubrics are same.

<img width="852" alt="Screenshot 2024-05-06 at 10 12 34 AM" src="https://github.com/prometheus-eval/prometheus/assets/50293852/4b79b046-440a-4ed6-81cb-fbf088e2bee7">


# ---
Another thing is that, how have you evaluate the `HHH` data? It is `tie:0/1`. Can you please explain your criteria? Similarly the `mt_human` data is `selected/rejected` answer.

Apart from `Flask`, there is no human scores but just `GPT-4`. How can we get human scores as well? Are the scores joined on Mean/ Median/ Top / Min / Max etc?


@scottsuk0306  @SeungoneKim  @eltociear 

Hi @deshwalmahesh! Thank you for the interest in our work!

Regarding the first question, I'm not fully sure what `old` and `new` are. Can you provide me the full code so that I can inspect on it?

For the second question, I think you wanted to ask how we evaluated `mt_bench_human_judgement` because it includes tied responses (`hhh_alignment` does not include tied responses). 

MT Bench Human Judgment and AutoJ test set includes a ‘tie’ option assessed by human evaluators. We evaluate in two ways: by excluding all ‘tie’ options for pairwise ranking (denoted as ‘w/o tie’), or by using direct assessment where responses scored as ‘ties’ are grouped, and pairwise rankings are applied to the remaining responses with differing scores (denoted as ‘w/ tie’).

For the last part, unfortunately we don't have human scores for other direct assessment benchmarks. The results in our paper are calculated based on the mean of the scores.

> Hi @deshwalmahesh! Thank you for the interest in our work!
> 
> Regarding the first question, I'm not fully sure what `old` and `new` are. Can you provide me the full code so that I can inspect on it?

New = `prometheus-eval/Preference-Collection`
Old =  `prometheus-eval/Feedback-Collection`
@scottsuk0306 

> For the last part, unfortunately we don't have human scores for other direct assessment benchmarks. The results in our paper are calculated based on the mean of the scores.

Please correct me if I'm wrong:
So in the Prometheus (1) paper, you had said that scores are better than GPT-4 when evaluated by humans, you must have got the scores per query from humans. I'm asking about those scores. 

In other words, how do I evaluate `Vicuna, MT, Feedback Collection`? Correlation( Pearson, Spearman, Kendall) with respect to the mean of GPT-4 scores? What if GPT itself was wrong there? Maybe I'm missing something here. Please help.

That's a very good question! As far as I know, the correlation between human & {GPT-4, Prometheus} in Prometheus 1 paper is calculated based on the small samples (45 instances) from Vicuna, MT-Bench, and Feedback Collecion. (You can refer to section 5.1). 

These scores are currently not available in our repository - this is because we haven't utilized these data in Prometheus 2 project. I need to check with the authors of Prometheus 1 for this.

> Correlation( Pearson, Spearman, Kendall) with respect to the mean of GPT-4 scores? What if GPT itself was wrong there? Maybe I'm missing something here. Please help.

You're right. GPT-4 can be wrong in the first place, and that's why we presented the correlation with `claude-opus` in our paper. As you know, the process of collecting scores from human annotators is extremely costly and time-consuming. Although there are limitations, we believe that high correlation with both proprietary models is a good indicator for evaluating evaluator LMs.

We're currently working on organizing the claude scores in our repository. It will be updated before next week so please stay tuned! Let me know if you need more clarifications on this 🙂

> 1. Can we say Preference data is an extension of Feedback data? There is a huge difference in numbers.
(I know I'm making set of strings so not a good indicator but still sticking to my question to confirm whether Old and New rubrics are same or not) Hi, while testing very vaguely, I found that there is difference in data. Not an issue but just asking why didn't you use the extension of Old data to generate new data (Answer B)? Seems like very few rubrics are same.

I'm still not sure what you mean by this question here. We used Feedback Collection to generate Preference Collection! It's just that we needed feedbacks for comparing the two responses - not feedbacks for grading a response in a sclae of 1-5. You can refer to section 3.3 of our paper.

<img width="478" alt="Screenshot 2024-05-06 at 3 59 01 PM" src="https://github.com/prometheus-eval/prometheus-eval/assets/39263767/33f90175-64f0-4c8d-83e6-6d2fab8a5848">

The reason for slight difference in the numbers is because of GPT-4 generation error.

Thanks a lot @scottsuk0306  for being so helpful. I think I got the answer but how come the new data has double the rows from the old data?

TL;DR: In other words, I wanted to know whether merging Preference and Feedback will give me duplicated Question `AND/OR` Rubric `AND/OR` Reference Answer `AND/OR` Answer?


Regarding `Preference Data`, I wanted to know whether:

1. **EACH** and every Question  + Rubric + Feedback which is in Feedback is present in Preference Data  as well 
2.  **EACH** of the `Answer-A, Answer-B` is present in `Preference Data`  are present in `Feedback Data`? If yes, how? As much as I know, `Feedback Data` has only 1 Answer per question while `Preference Data` has 2.
3. If One of the Answers are same, which of them it is? Answer-A or Answer-B?


# ---
Just to be sure, you have used `Feedback-OOD` and `Feedback-Test` [given in this repo right](https://github.com/prometheus-eval/prometheus/tree/main/evaluation/benchmark/data)? And your highest Spearman-R reached 90%, right? 

Thanks for the clarifications. Let me try my best to answer your questions 😃 

The new `Preference Collection` has double the rows compared to the `Feedback Collection` because for each instruction (question) in the Feedback Collection, they create ten pairs of responses. The original Feedback Collection has five responses per instruction, and by pairing them, they generate ten combinations (5 choose 2 = 10).

So the dataset statistics are the following:
- Number of instructions: `20K`
- Feedback Collection: 5 responses (which scores 1, 2, 3, 4, 5) per instruction, so `100K` rows
- Preference Collection: Make parings from the Feedback Collection 5 responses - 5C2 = 10 response pairs, so `200K` rows.

For example let's say there is one instruction `I` and the responses for that feedback collection are `[r1, r2, r3, r4, r5]`.
Then the pairings for Preference Collection are
```
[(r1, r2), (r1, r3), (r1, r4), (r1, r5), (r2, r3), (r2, r4), (r2, r5), (r3, r4), (r3, r5), (r4, r5)]
```

However, note that if we use the data in this way, then the score is always  `response A < response B`, so the chosen response is always `response B`. We randomly shuffled to ensure to have balanced labels.

---
### Regarding the Preference Collection:
a. Each Question, Rubric, and Reference Answer from the Feedback Collection should be present in the Preference Collection, as they are creating pairs from the existing responses in the Feedback Collection. However, they do not share feedbacks.
b. Each response (`response_A` and `response_B`) in the Preference Collection is present in the Feedback Collection. The difference is that in the Feedback Collection, each question has five separate responses, while in the Preference Collection, these responses are paired up, resulting in two answers (A and B) per row.
c. I think the above information answers your question already.


> Just to be sure, you have used Feedback-OOD and Feedback-Test [given in this repo right](https://github.com/prometheus-eval/prometheus/tree/main/evaluation/benchmark/data)? And your highest Spearman-R reached 90%, right?

Please use the data [here](https://github.com/prometheus-eval/prometheus-eval/tree/main/eval/benchmark/data) as we are not actively maintaining the [Prometheus repository](https://github.com/prometheus-eval/prometheus). We only utilized the `feedback_collection_ood_test` (you can think this as Feedback Bench) and yes - it reached spearman correlations (with GPT-4 0613) of 0.909 for `prometheus-7b-v2.0` and 0.912 for `prometheus-8x7b-v2.0`.



Thank you very much for your patience and such a detailed info. Appreciated that. Congratulations to you and the team for a great paper. I've been following Prometheus for quite some time and working in house for the same for the production grade system. This new paper opens up doors for relative scoring too.