Together, we are collecting the hardest and broadest set of questions ever. Can you think of something you know that would stump current artificial intelligence (AI) systems? This will help us better evaluate the capabilities of AI systems in the years to come. If you submit a challenging question that passes review, your name will be associated with the question and you will be invited as an author of the paper corresponding to this dataset.
Simply think of a hard question, and see if AIs get it right. If it's hard for the AIs it is likely good to submit. If there is an exceptionally difficult questions you've given in an exam, or a niche result you've encountered in your research, feel free to make this the basis for a question. We accept challenging questions from all fields, including mathematics, rocket engineering, analytic philsophy, and more.

Process
1. Write Question. You will first write a valid, very difficult question that AIs and average humans cannot answer. We aim to get questions that only exceptional individuals can answer correctly, so try and create questions that would impress you if an AI solved them.

2. AIs Assess Question Difficulty. Upon creating a question, we will ask state-of-the-art AI models to answer the question to help determine whether the question is too easy.

3. Write Answer Explanation. If the question is not too easy for the AIs, then we will ask you to write a thorough but concise solution for your question.

4. Peer Review. After submitting your question, answer, and rationale, your response will be saved, and we will conduct another round of manual review to maintain the quality of the benchmark. The answer rationale will help experts and AIs determine whether the answer you provided is correct. Authors can make changes or remove their submitted questions any time at the dashboard.

5. Publish. If your question is selected for the dataset, your contribution will be highlighted alongside the question. People who write more accepted questions will appear earlier in the author list of the paper, and the top question writers will also be emphasized in the appendix, should they choose to be on the paper. The top 50 questions will get $5000 in prizes, and the next top 500 questions will get $500, for a total of $500,000 in awards. (A small fraction of these questions will be kept private to detect if an AI is cheating and memorizing answers. You will still be credited in the paper if your question is put in the private cheating detection dataset. Coauthorship on the paper is optional.) More Details

Guidelines
1. Original. Questions must be original, not copy and pasted from someone else. You may possibly include questions from your exams that you have created, provided the answers are not publicly available and the questions are extremely challenging. Do not submit multiple questions that are minor variants of each other.

2. Challenging. Questions should be difficult for non-experts and not easily answerable by everyday people. Questions should not be simple trick questions. Questions should not be straightforward calculation/computation questions. The questions for this dataset can be so challenging that they are impractical for real-world human exams or problem sets. The answers should not be easily found through a Google search. It should impress you if an AI correctly answers your question. We found questions written by undergraduates tend to be too easy for the models. As a rule of thumb, if a randomly selected undergraduate can understand what is being asked, it is likely too easy for the frontier LLMs of today and tomorrow. Avoid using images if the question can easily be expressed in text.

3. Objective and Self-Contained. Questions should have answers accepted by other experts with relevant expertise. No highly disputed, "personal taste," ambiguous, or subjective questions. All necessary context and definitions should be included within the question (do not link to outside sources). It is acceptable to use technical jargon or notation without definition as long as it is standard, unambiguous, and would be widely recognized within your specialty. Answers should not have have many decimals or be imprecise.

4. No weapon questions. No questions about virology. No questions highly related to chemical, biological, radiological, nuclear weapons, or cyberweapons used for attacking critical infrastructure.


given the stated, provide 10 questions and answers to the respective questions