lvwerra HF staff commited on
Commit
589fd5f
1 Parent(s): e8002b4

add coding question question

Browse files
Files changed (1) hide show
  1. app.py +9 -0
app.py CHANGED
@@ -64,6 +64,15 @@ References
64
  - [5] - `joblib`: https://joblib.readthedocs.io/en/latest/
65
  """
66
 
 
 
 
 
 
 
 
 
 
67
  internships = {
68
  'Accelerate': default_question,
69
  'Diffusion distillation': default_question,
64
  - [5] - `joblib`: https://joblib.readthedocs.io/en/latest/
65
  """
66
 
67
+ code_question = """
68
+ You are probing your code generation model on a program synthesis benchmark and
69
+ 1 out of 4 the candidate solutions produced by your model pass the unit tests of a coding challenge.
70
+ <br/>
71
+ <br/>
72
+ What’s the pass@2 metric (in percent) as introduced in the
73
+ <a href="https://arxiv.org/abs/2107.03374" target="_blank"><code>Codex paper</a></code> (see section 2.1)?
74
+ """
75
+
76
  internships = {
77
  'Accelerate': default_question,
78
  'Diffusion distillation': default_question,