reviews: 
Comments: This study proposes 1,227 human-written and human-verified questions to asses cultural knowledge of LLMs covering 45 global regions, and 17 diverse topics. Each question is verified by five independent annotators.
Strengths: - Coverage of 45 regions associated with different culture
- Evaluated using 30 current LLMs. 
- Details experimental results with different analysis
Weaknesses: - Number of questions are relatively low compare to any other recent datasets. While it covers more countries, but number of questions per country is low (Appendix B, Table 2). For example, how do you justify the performance by just asking 7 questions (e.g., Canada)? This performance evaluation can have both effects, either it is underestimating of overestimating the performance. As a rule of thumb, a sample size of at least 30 is considered the minimum for statistical analysis due to the Central Limit Theorem. 
- L161: "but did not verify the final questions by humans" --> Is this necessary when question is already written by human? Do you mean verifying by other annotators. 
- L203: "These non-human written benchmarks" --> this is a contradictory statement. This study also used LLMs in the process, which makes the authors' criticism of prior work for doing the same seems contradictory. In addition, this study criticizes prior work "Over-reliance on Web Sources -- scraped web sources may be used during models pretraining". This whole question formulation process is based on LLMs knowledge, which is inherently knowledge from web sources. 
- L206-208: Low performance does not necessary mean challenging. 
- L329: GPT-4o-mini is used for the classification of questions into topics. It could be have done manually. This categorization is subjective in nature and requires a guideline to follow. For example, one can argue that "how to greet people" is a daily life activity. One of the major claims/contributions is the breadth of topic coverage, which has been addressed inadequately. 
- I did not find any prompt in Appendix C. 
- What was the reason to limit to "most 3-7 questions"? Is there any reference that culture can be defined by 5-7 questions? 
- L333-336: The discussion seems arbitrary. Annotators ask question based on their knowledge. If number of annotators are less, meaning we are limiting the background knowledge, also limiting the diversity of the questions. 
- This study is highlighting "discovery approach" very much. It is important to justify why it is beneficial. 
- While it is an study about cultural knowledge, therefore, one would expect that dataset will be in native language, however, this study is focused on English. 

- By looking at the resource, it seems that question is generic, can be applicable to different countries by replacing country. See first example here. For the second example, answer varies location to location within the culture.  

23,23,"In Bangladesh, which festival celebration style is most likely appropriate for expressing the collective cultural identity?",Hosting a potluck where each individual brings a dish from their own ethnic background.,Celebrating only the most widely recognized national holiday.,Selecting one culture each year to highlight and exclusively celebrate its festivals.,Ignoring all cultural festivals to maintain neutrality.,B,Bangladesh

94,94,"In Bangladesh, what is a customary action before stepping into someone's home?",Light incense at the entrance,Remove your shoes,Offer to cook a meal,Clap your hands twice,B,Bangladesh
Recommendation: 5
Reviewer confidence: 5
Soundness/correctness: 2
Clarity: 3