Evaluation Sheet for Deep Research: A Use Case for Academic Survey Writing (2025)

Abstract: Large Language Models (LLMs) powered with argentic capabilities are able to do knowledge-intensive tasks without human involvement. A prime example of this tool is Deep research with the capability to browse the web, extract information and generate multi-page reports.In this work, we introduce an evaluation sheet that can be used for assessing the capability of Deep Research tools. In addition, we selected academic survey writing as a use case task and evaluated output reports based on the evaluation sheet we introduced. Our findings show the need to have carefully crafted evaluation standards. The evaluation done on OpenAI‘s Deep Search and Google’s Deep Search in generating an academic survey showed the huge gap between search engines and standalone Deep Research tools, as well as the shortcomings in representing the targeted area.

## introduction deep research tools are designed to create comprehensive, long-form reports that dive deep into complex topics @xcite . their defining characteristics include unassisted web browsing, compilation of several sources, long waiting time, and results that resemble reports, not chat responses @xcite . deep research improves traditional search capabilities from keyword-based searching to more exhaustive search incorporating reasoning, inference synthesis, and response generation. this profound research feature transcends basic question-answering; it enables llms to navigate the internet, process extensive datasets, synthesize insights, and create structured reports with appropriate citations @xcite . llm providers such as google 1 , openai 2 , per-plexity foot_0 , xai 4 , and others are making available their deep research agent-based applications. deep research tools are increasingly used to assist academic tasks like literature reviews, offering draft summaries in minutes and aggregating data from numerous sources. however, they still require oversight, as they may hallucinate, cite unreliable sources, or prioritize outdated content. even though, deep research tools are powerful for scaling up our research capabilities, users must understand their strengths and limitations to choose the right tool. in this work: 1) we introduce evaluation sheet as a road-map for evaluating the performance of deep research tools. 2)as a use case (intended only as an example), we selected three recent nlp survey papers focused on african countries and languages: an ethiopian language survey (tonja et al., 2023), a nigerian language survey (inuwa-dutse, 2025), and a kenyan language survey (amol et al., 2024) to assess the applicability of the introduced evaluation sheet in order to evaluate the generated deep research report. is determining whether these tools can effectively identify the most important and impactful research, even when such research papers do not appear in the top search results. the primary issue we aim to address is how the growing popularity of these tools and their increasing role in replacing traditional searche engines affects the visibility and accessibility of significant research. ## conclusion in this work, we developed an evaluation sheet to help researchers identify the most critical evaluation criteria for assessing deep research tools for different use cases. this evaluation sheet seeks to standardize benchmarking datasets by highlighting key focus areas. to demonstrate its applicability, we conducted a proof-of-concept study on "deep research for survey paper generation" and used it to evaluate two well-known deep research tools. we hope researchers will adopt this evaluation sheet to create benchmarking datasets in their respective domains, ultimately improving the effectiveness of agentic tools that require minimal human interaction. by ensuring these tools generate reliable and informative outputs comparable to those found through independent searches, we aim to enhance their practical utility and trustworthiness. when such research papers do not appear in the top search results. the primary issue we aim to address is how the growing popularity of these tools and their increasing role in replacing traditional searche engines affects the visibility and accessibility of significant research. to access the usage of llms & deep research in survey report writing in low-resource languages, we crafted the following question: