gaia-benchmark/leaderboard · Clarifying Apple Stock Question

Hi - Just wanted to ask a clarifying question (keeping it a bit vague to avoid any risk of further contaminating others' validation results).

Task ID: 08cae58d-4084-4616-b6dd-dd6534e4825b

Question: "According to Google Finance, when was the first year the Apple stock went above $50 (without adjusting for stock split)?"

Annotator Steps: '''

typed in 'Google finance apple' on browser
clicked first link
clicked 'max' to display entire history of apple stock
hovered mouse around the area that line crosses over $50
noted the date'''

This was confusing - "why is my agent getting such a different result?"
The reason is that, like most finance websites, Google Finance adjusts for stock splits by default.
So the provided response is the first year Apple's stock went above $50 (adjusting for stock splits).
The date Apple's stock first crossed $50 (without adjusting for stock splits) is > 10 years earlier.

So I think the question should be updated to remove the word "without".

Happy to elaborate further if helpful.

Thanks,