loubnabnl HF staff commited on
Commit
f6fae6e
1 Parent(s): 9196092
Files changed (1) hide show
  1. evaluation/intro.txt +1 -1
evaluation/intro.txt CHANGED
@@ -72,4 +72,4 @@ scores
72
 
73
  ````
74
 
75
- If we take a closer look at the unit test results for each candidate solution in the three tasks, we find that only 3 passed the test which corresponds to `1/30 = 0.333`, our pass@1, the scores pass@5 and pass@10 are higher, because the more samples we select from the candidate solutions, the more likely we are to include the correct solution. Without surprise pass@10 is '2/3=0.73': if we select all candidates two tasks out of three get solved.
 
72
 
73
  ````
74
 
75
+ If we take a closer look at the unit test results for each candidate solution in the three tasks, we find that only 3 passed the test which corresponds to `1/30 = 0.333`, our pass@1, the scores pass@5 and pass@10 are higher, because the more samples we select from the candidate solutions, the more likely we are to include the correct solution. Without surprise pass@10 is '2/3=0.73': if we select all candidates two tasks out of three get solved.