Spaces:

---

FINAL-Bench

/

leadboard

Like 42

Running

Hello. I hit a wall submitting to the Phenotype board; here is a reproduction.

1. Reproduction

POST /api/submit with board: "Phenotype", putting 16 numbers per compound in predictions ({"LB-…": [x1, …, x16], …}, 2,001 entries). Response:

HTTP 400
{
 "detail": "이 부문은 화합물마다 숫자 하나가 필요한데 배열이 왔다 (2001건, 예: LB-F615EE7B2FF1)"
}

(verbatim. The server responded in Korean; it reads: "This board requires a single number per compound, but an array was received (2001 entries, e.g. LB-F615EE7B2FF1)".)

2. But this board's ranking metric requires an axis vector

• GET /api/board/Phenotype/testset → "axes": 16
• GET /api/leaders?board=Phenotype → "metric": "profile_corr", "higher_better": true
• The three baselines in that same response all carry a profile_corr value — constant 0.09156, nearest-neighbour 0.07926, Morgan+LightGBM 0.15912.

A correlation is defined between two vectors; it is not defined for a single number per compound. Yet the baselines have one. So it looks like the scoring path accepts axis vectors while only the submit-validation path requires a scalar.

3. The on-screen guidance is also split (you may want to look at this too)

/i18n.js contains an axis-board string fPredAx ("This board has N axes — upload JSON with an array of N numbers per compound"), which appears intended to override the generic string on boards that declare axes. However, the submit tab does not show it — it shows the generic "JSON or CSV (compound_id, prediction)". The starter snippet in the participation guide is also scalar-based (%.4f).

4. Impact

Right now an array is rejected by the validator, and a scalar cannot produce the ranking metric. This does not look specific to us — any participant trying this board would hit the same thing.

5. Note (not a claim)

In #1 you identified boards() overwriting each board's declared metric with mae, and replied that "Each board now shows its own metric". The card, the guide and the baseline evaluation do appear to have been updated — GET /api/leaders now returns "metric": "profile_corr" correctly for this board. The submit-validation path, however, may simply not have been on that list — it looks like the last remaining branch of that same one-line cause. We cannot see your code, so this is a guess; just flagging it for a check.

Thanks for looking.