**Catherine Gracey**

University of New Brunswick, Fredericton, NB, Canada

**Julie Morris**

University of New Brunswick, Fredericton, NB, Canada

**Richelle Witherspoon**

University of New Brunswick, Fredericton, NB, Canada

**Erik Moore**

University of New Brunswick, Fredericton, NB, Canada

# A STUDY OF GRADUATE STUDENTS' EXPERIENCES OF ARTIFICIAL INTELLIGENCE AT THE UNIVERSITY OF NEW BRUNSWICK

## Abstract

Artificial intelligence (AI) is increasingly being used by students in higher education for a wide range of tasks, such as brainstorming, finding information, or drafting papers. While we understand the general use cases for AI in the classroom, there is a gap in the research about students' processes for learning, evaluating and implementing new tools into their learning workflows. This talk summarizes the initial findings of focus groups conducted with graduate students at the University of New Brunswick that explored student perceptions of—and experiences with—AI technology.

## Introduction

AI has altered the way graduate students approach their education, using it to form ideas, validate results, edit their work, and more. The Digital Education Council (2024) found that 86% of students are using Generative AI (GenAI) in their academic pursuits, with searching for information being the most popular use case.

Much of the existing learner-focused Artificial Intelligence in Education (AIEd) research has focused on either (1) students' perceptions of GenAI broadly, or (2) those tasks students are already using AI to support (Digital Education Council, 2024; Johnston et al., 2024). However, little remains known about the integration of AI into academic workflows, including methods of selecting and evaluating tools. Such skills are critical, given that there are well-documented issues with artificial intelligence tools, including bias (Dancy & Saucier, 2022; Lewis et al., 2024), lack of transparency (Siau & Wang, 2020), and hallucinations (Walters & Wilder, 2023).
In this paper, we describe the results of two focus group sessions with graduate students at the University of New Brunswick (UNB), held in February and March of 2025. The study explores participants' use of AI, methods for finding and evaluating tools, perspectives on its impact on learning, and barriers to appropriate use.

To guide our research, we wished to investigate how graduate students find, appraise and evaluate AI tools for use in their academic pursuits. We were equally interested in how graduate students perceive the barriers to, and benefits of using these tools.

## Literature Review

Zawacki-Richter et al. (2019) described the four areas of study within the field of Artificial Intelligence in Education as: (1) profiling and prediction, (2) assessment and evaluation, (3) adaptive systems and personalization, and (4) intelligent tutoring systems. The latter two areas can be characterized as ‘learner-focused’ areas, as they focus on applications for student use, rather than for teachers or administrators (Baker et al., 2019). As new Generative AI tools emerge, the use cases for learners are multiplying. However, little AIED research has focused on identifying the specific tools students utilize beyond ChatGPT. Emphasis has been placed on capturing student perspectives on the utility and limitations of GenAI generally (Chan & Hu, 2023; Johnston et al., 2024; Wang et al., 2024), or ChatGPT specifically (Shoufan, 2023; Strzelecki, 2024), and less so on specific use cases or incorporation into research workflow.

Library and Information Science (LIS) researchers and practitioners have explored AI literacy as an information skill (Dott & Charlton, 2024), noting that GenAI tools have potential to accompany or replace aspects of the information search students engage in to find sources for assignments (Chaudhuri & Terrones, 2024; Chowdhury & Chowdhury, 2024; Deschenes & McMahon, 2024; Lo, 2023). Students appear to be aware of the limitations of GenAI (Gruenhagen et al., 2024) and feel that there is a lack clarity about whether specific uses of GenAI violate academic integrity (Chan, 2023; Gruenhagen et al., 2024). It is not clear how these perceptions impact usage; however, this study addresses a gap in how students discover and evaluate information about AI tools and their outputs.

## Background

We chose to focus this study on graduate students, rather than undergraduate students. Graduate students typically exhibit greater motivation in pursuing academic advancement and use more diverse learning strategies (Dong et al., 2024, p. 1). They are also likely seeking credentials to further their career (Hegarty, 2011), and approach scholarship in pursuit of professional skills development and mastery rather than grades (McCollum & Kajs, 2007).

## Methodology
Librarian facilitated focus groups were held on both of UNB's campuses, in Fredericton (17 participants) and Saint John (7 participants). Graduate students from any academic discipline and level were eligible, and their faculty/department was noted.

Each focus group event began with a brief presentation outlining the scope of the study and technologies. We used the 'think, pair, share' method that had students individually reflect on their answers to the below questions, then discuss with a partner, and then share their thoughts with the rest of the group seated at their table. The librarian facilitator took notes during these conversations, and both these notes and the text from the question cards were used for analysis.

Fredericton Focus group questions:

1. What AI tools have you used to improve your education and success in your graduate program? What did each of these tools help you with, and are there any costs or disadvantages associated with using them?

2. How did you learn about the tools you're using? What is it about these tools that made you choose them over other tools you may have encountered?

3. How did you test / evaluate these tools prior to implementing them in your workflow? How do you decide to trust a tool to support your education?

4. What do you find are the barriers (ethical or structural) to using AI to support your education, and how do you manage them?

Saint John Focus group questions:

1. Have you used AI tools in your academic work? If so, how and why? If not, why not?

2. What potential does AI have to improve learning and education?

3. How could AI tools be integrated into education without compromising student learning? Give specific examples?

4. How does AI affect academic integrity, and what policies and guidelines should be in place to guide its use?

Data analysis was performed according to Braun and Clarke's six phases of thematic analysis (2006):

1. The **principal investigator (PI)** familiarized herself with the data by reading and re-reading the transcripts, making initial notes, and identifying early patterns and potential themes.

2. The PI then conducted **systematic coding**, assigning initial codes to meaningful segments of the data and grouping similar codes together to identify patterns.

3. These codes were analyzed for overarching **themes**, with related codes collated under broader thematic categories.

4. To ensure **rigor and consistency**, the PI and two co-investigators (Co-Is) collaboratively reviewed the themes, assessing their coherence in relation to both the initial codes and the dataset.
5. The research team then refined and defined the themes, ensuring clarity, distinctiveness, and alignment with the underlying data, ultimately shaping a coherent narrative.

6. The findings presented in this paper represent the final synthesis of our thematic analysis.

## Results

Despite having two distinct sets of questions, the nature of the student responses and the subsequent discussion were similar, so a common set of themes was extracted from across both groups. The researchers also opted to code only students' written responses in data coding, using facilitator notes solely to provide clarification where student responses were unclear.

The researchers launched their analysis with an exploratory count of responses regarding the specific AI tools being used (Appendix A) and the sources they were learning about these tools from (Appendix B). Unsurprisingly, the most frequently mentioned tool was ChatGPT; however, students also used other tools including Grammarly, Quilbot, and Copilot. Students primarily reported learning about AI tools from online sources, like social media, or through word of mouth within UNB, such as from other students or professors.

Utilizing the thematic analysis process described in the methods section, the researchers identified five emergent themes: AI as Assistant, AI as Private Tutor, AI Appraisal, AI as Threat, AI as Practice Gap.

### AI as Assistant

This theme was the first to emerge in most discussions; students described AI tools as assistants that could support their brainstorming or revision, but did not consider it an agent that could do their work for them. However, they also strongly emphasized their mistrust of its accuracy and responses, indicating they still experience “some hallucinations,” and “expert supervision is always needed.”

### AI as Threat

Concerns over plagiarism and over-reliance on AI among the student body, particularly among the undergraduate students they taught, were also common. Participants worried some students would “abuse AI and copy paste without learning.” Another major concern was about the ethics and sustainability of AI companies, and whether they could “trust the tool to not misuse the data I provide it or if censorship will make results biased.”

### AI as Private Tutor
Another common theme was the utility of AI as a personalized tutoring tool. Students noted that AI tools could be “used as a resource to train, tutor, or explain things in different ways.” Participants liked that it gave “direct answers to your questions,” and that interactions could be iterative, whereas they felt interacting with instructors or fellow students often led to less focused and efficient responses.

## AI Appraisal

After indicating that AI generated content cannot always be trusted, students described their approaches to evaluating tools, which can generally be categorized into three methods.

1. *Comparing the tools' responses to their own knowledge:* the most frequent way in which students evaluated AI tools was “by asking questions that I knew the right answer for.”

2. *Comparing the tool's responses to those of an authoritative source:* students described evaluating tools by “cross referencing with accepted websites,” or would “cross-check it with the professor's materials.”

3. *Comparing one tool's responses to those of another:* students often appraised tools “by using multiple platforms initially,” to determine which provided the best or most accurate results.

In their evaluations of AI tools, students reaffirmed the importance of not trusting GenAI fully, recognizing the need to validate results against subject area expertise to use AI effectively.

## AI as Practice Gap

This theme revealed the lack of clarity and support that students were experiencing when it came to AI use in their education. Many students expressed fear of accidentally misusing AI and voiced frustration with the variability between professors on what constituted acceptable use.

Two key student needs emerged from these issues. First, participants highlighted the lack of clear and consistent policies regarding acceptable AI use and acknowledgement. Second, students indicated the need for effective education about AI use including the benefits, issues, ethical considerations, and use cases.

## Discussion

This study confirms the findings of preceding research: students consider AI to be useful for certain tasks, like revision and brainstorming, and less so for others, such as replacing research and writing skills (Chan, 2023; Gruenhagen et al., 2024; Johnston et al., 2024). AI can be used as a tool to facilitate academic and scholarly efforts but should not be used to replace them. This paper adds an understanding of how students evaluate AI for their own use, and what they need from their academic institutions to continue to pursue reasonable and ethical AI use.
Across all key themes, graduate students identified gaps in institutional policy that would inform ethical uses and best practices related to AI use within the research lifecycle. A common thread through the themes was a general confusion or misunderstanding about what constituted ethical, fair, and best use of AI.

There are numerous opportunities for using AI in graduate student research, however some key issues from the study that are worthy of attention. First, respondents reported trying to balance the facility of use of AI while vetting the accuracy of the data outputs. Second, concern over misconduct, whether intentional or unintentional, was a primary fear of graduate students. Most students cited a lack of standardization across the institution, department or faculties related to fair use of AI. Thus, students were cautious to use AI in a substantive manner in their assignments or theses. Many used AI to test or validate against their own knowledge and expertise, rather than attempting to use AI for innovative purposes. Finally, ethical concerns, like sustainability issues, made some respondents feel that using AI was not worth it in every situation.

There is an opportunity for graduate students to integrate AI into their research lifecycles. There also exists an opportunity for librarians and administrators to contribute meaningfully to training and guidance on the responsible use of AI tools within the institution. If educators and administrators are to encourage innovative use of AI, it must be made clear to graduate students where the boundaries of ethical applications lie. Clear guidelines and training around best uses for validation, prompt engineering, and applications will reassure students that the institution recognizes the delineation between intentional and unintentional misconduct, and thus create a safe environment for students to explore the use of AI in their research.
## References

Baker, T., Smith, L., & Anissa, N. (2019). *Educ-AI-tion Rebooted? Exploring the future of artificial intelligence in schools and colleges*. Nesta.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. *Qualitative Research in Psychology*, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Chan, C. K. Y. (2023). A comprehensive AI policy education framework for university teaching and learning. *International Journal of Educational Technology in Higher Education*, 20(1), 38. https://doi.org/10.1186/s41239-023-00408-3

Chan, C. K. Y., & Hu, W. (2023). Students’ voices on generative AI: Perceptions, benefits, and challenges in higher education. *International Journal of Educational Technology in Higher Education*, 20(1). Scopus. https://doi.org/10.1186/s41239-023-00411-8

Chaudhuri, J., & Terrones, L. (2024). Reshaping Academic Library Information Literacy Programs in the Advent of ChatGPT and Other Generative AI Technologies. *Internet Reference Services Quarterly*, 0(0), 1–25.
https://doi.org/10.1080/10875301.2024.2400132

Chowdhury, G., & Chowdhury, S. (2024). AI- and LLM-driven search tools: A paradigm shift in information access for education and research. *Journal of Information Science*. Scopus.
https://doi.org/10.1177/01655515241284046

Dancy, C. L., & Saucier, P. K. (2022). AI and Blackness: Towards moving beyond bias and representation. *IEEE Transactions on Technology and Society*, 3(1), 31–40.
https://doi.org/10.1109/TTS.2021.3125998

Deschenes, A., & McMahon, M. (2024). A Survey on Student Use of Generative AI Chatbots for Academic Research. *Evidence Based Library and Information Practice*, 19(2), Article 2.
https://doi.org/10.18438/eblip30512

Digital Education Council. (2024). *Digital Education Council Global AI Student Survey 2024*. https://www.digitaleducationcouncil.com/post/digital-education-council-global-ai-student-survey-2024

Dong, J., Manning, K. Q., & Wang, C.-H. (2024). The influence of adjustment, mental health and motivation on graduate student learning strategies during COVID-19. *Discover Psychology*, 4(1), 195. https://doi.org/10.1007/s44202-024-00314-3

Dott, E., & Charlton, T. (2024, November 20). *Is AI literacy an information skill? THE Campus Learn, Share, Connect*. https://www.timeshighereducation.com/campus/ai-literacy-information-skill

Gruenhagen, J. H., Sinclair, P. M., Carroll, J.-A., Baker, P. R. A., Wilson, A., & Demant, D. (2024). The rapid rise of generative AI and its implications for academic integrity: Students’ perceptions and use of chatbots for assistance with assessments. *Computers and Education: Artificial Intelligence*, 7, 100273. https://doi.org/10.1016/j.caeai.2024.100273

Hegarty, N. (2011). Adult Learners as Graduate Students: Underlying Motivation in Completing Graduate Programs. *The Journal of Continuing Higher Education*, 59(3), 146–151.
https://doi.org/10.1080/07377363.2011.614883

Johnston, H., Wells, R. F., Shanks, E. M., Boey, T., & Parsons, B. N. (2024). Student perspectives on the use of generative artificial intelligence technologies in higher education. *International Journal for Educational Integrity*, 20(1), 2.
https://doi.org/10.1007/s40979-024-00149-4
Lewis, J. E., Whaanga, H., & Yolgörmez, C. (2024). Abundant intelligences: Placing AI within Indigenous knowledge frameworks. *AI & SOCIETY*. https://doi.org/10.1007/s00146-024-02099-4

Lo, L. S. (2023). My new favorite research partner is an AI What roles can librarians play in the future? *College and Research Libraries News*, 84(6), 209–211. Scopus.
https://doi.org/10.5860/crln.84.6.209

McCollum, D. L., & Kajs, L. T. (2007). Applying Goal Orientation Theory in an Exploration of Student Motivations in the Domain of Educational Leadership. *Educational Research Quarterly*, 31(1), 45–59.

Shoufan, A. (2023). Exploring Students’ Perceptions of ChatGPT: Thematic Analysis and Follow-Up Survey. *IEEE Access*, 11, 38805–38818. Scopus.
https://doi.org/10.1109/ACCESS.2023.3268224

Siau, K., & Wang, W. (2020). Artificial Intelligence (AI) Ethics: Ethics of AI and Ethical AI. *Journal of Database Management*, 31(2), 74–87.
https://doi.org/10.4018/JDM.2020040105

Strzelecki, A. (2024). Students’ Acceptance of ChatGPT in Higher Education: An Extended Unified Theory of Acceptance and Use of Technology. *Innovative Higher Education*, 49(2), 223–245. https://doi.org/10.1007/s10755-023-09686-1

Walters, W. H., & Wilder, E. I. (2023). Fabrication and errors in the bibliographic citations generated by ChatGPT. *Scientific Reports*, 13(1), 14045. https://doi.org/10.1038/s41598-023-41032-5

Wang, C., Wang, H., Li, Y., Dai, J., Gu, X., & Yu, T. (2024). Factors Influencing University Students’ Behavioral Intention to Use Generative Artificial Intelligence: Integrating the Theory of Planned Behavior and AI Literacy. *International Journal of Human-Computer Interaction*, 0(0), 1–23. https://doi.org/10.1080/10447318.2024.2383033

Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? *International Journal of Educational Technology in Higher Education*, 16(1), 39. https://doi.org/10.1186/s41239-019-0171-0
# Appendix A

A count of the number of times any AI tools was mentioned by participants in their written responses to the provided questions

<table><thead><tr><td>AI tools being used by participants</td><td>Count of Mentions</td></tr></thead><tbody><tr><td>ChatGPT</td><td>34</td></tr><tr><td>Grammarly</td><td>9</td></tr><tr><td>Copilot (any version - rarely specified)</td><td>6</td></tr><tr><td>Quillbot</td><td>6</td></tr><tr><td>Gamma</td><td>3</td></tr><tr><td>Monica</td><td>3</td></tr><tr><td>Grammar Checker</td><td>3</td></tr><tr><td>Gemini</td><td>3</td></tr><tr><td>DeepSeek</td><td>2</td></tr><tr><td>Poe</td><td>1</td></tr><tr><td>Perplexity</td><td>1</td></tr><tr><td>RW</td><td>1</td></tr><tr><td>Tableau AI</td><td>1</td></tr><tr><td>PopAI</td><td>1</td></tr></tbody></table>
Appendix B

<table><caption>A count of the number of times participants mentioned a source for learning about AI tools.</caption><thead><tr><td>Where participants learned about AI tools</td><td>Count of Mentions</td></tr></thead><tbody><tr><td>Online Sources (searches, social media, etc)</td><td>10</td></tr><tr><td>University (students, colleagues, profs)</td><td>9</td></tr><tr><td>Friends</td><td>5</td></tr><tr><td>Family</td><td>2</td></tr><tr><td>Mainstream media</td><td>1</td></tr><tr><td>Other people</td><td>1</td></tr></tbody></table>