Rethinking Search: A Study of University Students’ Perspectives on Using LLM s and Traditional Search Engines in Academic Problem Solving (2025)

Abstract: With the increasing integration of Artificial Intelligence (AI) in academic problem solving, university students frequently alternate between traditional search engines like Google and large language models (LLMs) for information retrieval. This study explores students’ perceptions of both tools, emphasizing usability, efficiency, and their integration into academic workflows. Employing a mixed-methods approach, we surveyed 109 students from diverse disciplines and conducted in-depth interviews with 12 participants. Quantitative analyses, including ANOVA and chi-square tests, were used to assess differences in efficiency, satisfaction, and tool preference. Qualitative insights revealed that students commonly switch between GPT and Google: using Google for credible, multi-source information and GPT for summarization, explanation, and drafting. While neither tool proved sufficient on its own, there was a strong demand for a hybrid solution. In response, we developed a prototype, a chatbot embedded within the search interface, that combines GPT’s conversational capabilities with Google’s reliability to enhance academic research and reduce cognitive load.

## introduction the rapid advancement of artificial intelligence has significantly reshaped the ways in which university students seek academic information and engage in research activities @xcite . traditionally, search engines like google have served as the dominant tool for retrieving scholarly content due to their accessibility, breadth of indexed materials, and access to verified sources. however, the emergence of llms, such as openai's chatgpt, has introduced a new paradigm-offering students direct, conversational responses and contextualized summaries that can streamline information consumption @xcite . this evolution in digital research tools raises important questions about how students perceive and utilize these systems, particularly in academic settings where accuracy, credibility, and efficiency are critical. prior research suggests that while llms facilitate rapid content summarization and task-specific assistance, their reliability varies depending on context and task complexity @xcite . conversely, search engines provide access to a wide range of authoritative sources but often require users to sift through multiple links and evaluate conflicting information independently. several studies have documented this complementary behavior-students tend to use llms for explanation and drafting while relying on search engines for fact-checking and source validation @xcite . despite their respective strengths, both tools also have well-documented limitations. llms can generate confident yet incorrect outputs, potentially misleading users @xcite , while traditional search engines can lead to information overload and inefficiency in time-sensitive academic contexts. as a result, students are increasingly adopting a hybrid approach-strategically switching between llms and search engines to balance speed with credibility @xcite . however, this constant toggling between tools introduces cognitive overhead and fragmented workflows, especially when performing complex academic tasks. to investigate these dynamics systematically, this study addresses the following research questions: while traditional search engines like google have long been the primary tool for academic information seeking, recent research increasingly explores how llms compare in terms of usability, task performance, and user trust. divekar et al. (2024) examined how university students use llms like chatgpt alongside traditional search engines for learning new topics. their findings indicate that while llms support rapid summarization and ease of understanding, their effectiveness varies depending on the complexity and nature of the task. in a similar vein, kumar et al. (2024) analyzed how students use llms to generate sql queries. they observed that llm assistance improved query formulation and contributed positively to the learning experience. several studies have also investigated task completion performance. @xcite conducted a randomized experiment and found that llm users completed decision-making tasks more quickly and with fewer queries. however, the authors warned of a major drawback: users often overtrust llm outputs, especially when incorrect answers are presented confidently. they suggested the inclusion of confidence indicators to mitigate this issue. @xcite echoed this concern, emphasizing the need for rigorous fact-checking when relying on llm responses. in terms of task preference, caramancion (2024) evaluated 20 types of information-seeking scenarios and concluded that users favored traditional search engines for fact-based queries, while preferring llms for creative or complex tasks. supporting this, @xcite found that nearly 70% of participants preferred chatgpt-style conversational interfaces due to their personalization, perceived efficiency, and convenience. extending these findings, @xcite studied image geolocation tasks and observed that tool selection often influenced user strategy: llms were used more intuitively, while traditional tools required structured navigation. the issue of credibility remains central. @xcite argued that despite the convenience and rapidity of ai tools, traditional search methods remain more reliable for academic research. in contrast, llms often lack source transparency, which can be problematic in scholarly settings. to address this trade-off, researchers have proposed hybrid models. @xcite explored metasearch engines that aggregate content from various sources to improve accuracy, and caramancion (2024) advocated for systems that combine the contextual depth of llms with the source validation strengths of search engines. however, existing studies have primarily evaluated llms and traditional search engines in isolation or through task-specific comparisons, without fully exploring how students naturally combine both tools in academic workflows @xcite . there is a lack of empirical research that integrates both performance metrics and user perspectives to understand this hybrid usage behavior @xcite . while tools like perplexity ai @xcite attempt to bridge this gap by combining ai-generated responses with source links, and google has introduced ai summaries through its search generative experience (sge), these systems remain largely static, lacking personalization, real-time adaptation, and task-specific reasoning. this study addresses these limitations through a mixed-methods approach and the design of a userinformed, context-aware prototype. ## methodology to explore university students' preferences and usage behaviors regarding llms and traditional search engines for academic tasks, we employed a mixed-methods approach that combined quantitative and qualitative data collection and analysis, shown in figure 1 . this design allowed us to examine both broad patterns and deeper user experiences in a complementary manner. we first conducted an online survey that collected responses from 109 university students across a range of academic disciplines. while the survey primarily targeted students in technologyrelated fields such as computer science and engineering (cse), electrical and electronics engineering (eee), and data science, it was also distributed to students from other areas, including business administration and medicine, to ensure diversity. the questionnaire included both closed-and open- survey interview themes qualitative analy sis @xmath0: thematic -effectiveness -usability and cognitive load -trust and credibility -contextual tool selection ended questions designed to assess tool usage frequency, satisfaction, efficiency, and perceived ease of use when using gpt-based llms and traditional search engines like google. descriptive statistics were used to summarize the data, and inferential statistical tests, one-way anova and chi-square tests, were employed to evaluate differences in user perceptions and the influence of demographic variables on tool preference. to enrich and validate the survey findings, we conducted in-depth in-person interviews with 12 students from cse, eee, and bba backgrounds, primarily recruited from united international university (uiu), dhaka, bangladesh. each participant was asked to complete six academic tasks: summarizing a research article, solving a coding problem (only for cse students), addressing a circuit-related issue (only for eee students), analyzing business data (only for bba students), drafting a formal academic email, and comparing two popular academic concepts. participants were divided into four groups based on their tool usage behavior: gpt-only users, google-only users, balanced users who alternated between both tools, and random-choice users who freely switched between gpt and google depending on preference. the qualitative analysis synthesized insights from both the open-ended survey responses and interview transcripts. thematic coding focused on perceived effectiveness, usability, trustworthiness, and the contextual factors that influenced tool selection. this analysis provided a comprehensive understanding of how students navigate the strengths and limitations of both tools and how their choices are shaped by the nature of the academic task, fa-miliarity with the subject, and perceived cognitive effort. ## qualitative analysis the qualitative analysis draws on open-ended survey responses and in-person interview transcripts to explore participants' perceptions, experiences, and decision-making strategies when using gpt and google for academic tasks. we employed a thematic analysis approach to identify recurring patterns and categories in the qualitative data. initial coding was conducted independently by two researchers who reviewed all textual responses lineby-line. codes were then grouped into broader themes through iterative comparison and refinement until consensus was reached. four major themes emerged from the data: (1) task suitability and tool preference, (2) perceptions of reliability and accuracy, (3) workflow efficiency and cognitive load, and (4) usability and interaction experience. task suitability and tool preference. participants frequently distinguished between tools based on the academic task. gpt was consistently described as effective for quick answers, summarization, and writing support. one participant noted, "i use chatgpt whenever i need to summarize something quickly or generate a draft; it saves a lot of time." in contrast, google was preferred for tasks requiring deeper exploration and source triangulation. for example, a bba student shared "google helps me see what different sources are saying, especially when i need to analyze business trends from multiple angles." ## discussion the findings from both the survey and in-person interviews reveal a nuanced interplay between llms and traditional search engines in academic information-seeking behavior. gpt-based systems were consistently valued for their ability to provide structured, coherent, and contextually relevant responses. their strengths were particularly evident in tasks requiring rapid summarization, coding support, or written content generation, where participants appreciated the speed and reduced cognitive effort offered by conversational interfaces. however, while llms excelled in usability and perceived efficiency, their limitations, such as occasionally outdated or overly generalized content, prompted students to cross-reference with more authoritative sources. in contrast, traditional search engines like google remained the preferred tool for in-depth research, source validation, and academic rigor. students highlighted google's extensive access to peer-reviewed literature, academic websites, and multiple viewpoints as vital for tasks requiring critical evaluation or citation. nonetheless, participants also reported experiencing information overload and inefficiencies due to the need to manually sift through links, assess credibility, and synthesize fragmented content. these trade-offs suggest that tool preference is not static but shaped by the academic task's complexity, time constraints, and the student's familiarity with the subject matter. a recurring theme across both quantitative and qualitative data was the strong interest in a hybrid model that seamlessly integrates the complementary strengths of gpt and google. as illustrated in figure 3 , a significant portion of participants expressed a desire for an academic support tool that combines gpt's conversational and summarization capabilities with google's multi-source, real-time information retrieval. such a system would enable users to receive concise responses with embedded citations and links to original sources, streamlining the verification process without sacrificing depth or credibility. participants viewed this hybrid approach as a way to reduce cognitive load, eliminate repetitive tool-switching, and enhance learning out-comes through more fluid academic workflows. while llms and traditional search engines serve distinct purposes, students view them as complementary rather than competing tools. the integration of their respective advantages, llms for generation and synthesis, and search engines for depth and verification, represents a promising direction for the future of academic information retrieval. these findings underscore the importance of designing intelligent, context-aware tools that adapt to students' diverse needs while upholding standards of reliability and academic integrity. these findings also raise a key question: what happens when students use gpt and google together: does it help or hurt? the evidence from our study suggests a compelling answer. participants who used both tools, using gpt to quickly summarize and clarify complex topics, and google to verify facts and consult authoritative sources, consistently outperformed those who relied on either tool alone. these hybrid users achieved the highest task accuracy (90%), demonstrating that the strategic integration of llms and traditional search engines not only complements their respective strengths but also minimizes their individual weaknesses. although this dual-tool approach required more time, students perceived it as a worthwhile trade-off for greater confidence, deeper understanding, and higher-quality outcomes. this finding underscores the potential of thoughtfully designed hybrid systems to support academic workflows, reducing cognitive load while maintaining rigor and trustworthiness in the learning process. ## proposed prototype drawing on user insights from both survey responses and interviews, we propose a conceptual prototype that integrates gpt-based assistance directly into the traditional search engine interface. 1 the goal is to address the cognitive and logistical burden of switching between tools by creating a unified platform that combines the conversational utility of llms with the source-rich infrastructure of search engines. the prototype is designed as an embedded chatbot, positioned unobtrusively in the corner of the search interface, allowing users to engage in interactive, context-aware dialogue without disrupting their familiar browsing workflow. unlike standalone llm interfaces, the proposed assistant does not replace standard search results. the prototype's novelty lies in its hybrid architecture that allows toggling between raw search content and ai-enhanced interpretation. crucially, each ai-generated insight is accompanied by links to the original source, promoting transparency and reducing the risk of hallucinated or unverifiable responses. this feature directly addresses the concern, voiced by multiple participants, regarding llm trustworthiness in academic work. although conceptual in its current form, the prototype was informed by empirical findings from this study and inspired by real-world user preferences. its contribution lies in reimagining academic information retrieval as an interactive and adaptive process. over time, the assistant could learn user preferences, discipline-specific language, and search habits to deliver more relevant and personalized guidance. by embedding this intelligent layer into the search experience, the prototype aims to reduce cognitive load, increase search efficiency, and promote evidence-based academic practices, ultimately bridging the current gap between generation and verification in digital research tools. ## conclusion our study demonstrates that university students adopt a complementary approach to academic information retrieval, using llms for quick explanations and drafting, and traditional search engines for verification and accessing credible sources. our mixed-methods findings underscore the taskdependent nature of tool preference and reveal strong interest in a hybrid model. while the proposed prototype remains conceptual and the interview sample was limited, the results offer practical insights for designing ai-assisted academic tools. future work will focus on expanding participant diversity, validating qualitative themes, and implementing a functional prototype to assess real-world usability and impact.