Pharmacy Education (2023) 23(1) 665 - 675
https://doi.org/10.46542/pe.2023.231.665675

RESEARCH ARTICLE

Knowledge and attitudes of pharmacy students
towards artificial intelligence and the ChatGPT
Makiko Iwasawa1, Masahiro Kobayashi2, Katsuya Otori3,4

1 Division of Clinical Pharmacy (Drug Information) and Research and Education Center for Clinical Pharmacy, School of Pharmacy,

Kitasato University, Japan
2 Division of Clinical Pharmacy (Clinical Pharmacokinetics) and Research and Education Center for Clinical Pharmacy, School of
Pharmacy, Kitasato University, Japan
3 Division of Clinical Pharmacy (Pharmacy Practice and Science I) and Research and Education Center for Clinical Pharmacy, Schoolof Pharmacy, Kitasato University, Japan
4 Department of Pharmacy, Kitasato University Hospital, Japan

Keywords
Artificial intelligence
ChatGPT
Curriculum
Education
Pharmacy
Student

Correspondence
Makiko Iwasawa
Division of Drug Information
School of Pharmacy
Kitasato University
Kanagawa
Japan
iwasawam@pharm.kitasato-u.ac.jp

Abstract
Background: This study examined pharmacy students' knowledge and attitudes to
artificial intelligence (AI) and ChatGPT. Methods: A survey was conducted with 258
fourth-year pharmacy students at Kitasato University, Japan, assessing their AI
knowledge, ChatGPT impressions, and literacy regarding its appropriate use. Data wereanalysed descriptively, and participants were divided into two groups based on ChatGPT
awareness for further analysis. Results: A total of 113 students (43.7%) participated,with 19.5% having prior ChatGPT experience, 4.4% were unaware, and 42.5% could
describe it. General AI terms like "machine learning" were more familiar (83.2%) thanspecific ones like "natural language processing" (2.6%). Positive impressions of ChatGPT
included 65.5% finding it efficient and 50.4% mentioning enhanced comprehension.
However, only 3.5% correctly identified all inappropriate purposes, and 24.8% selected
all precautions for its use. The 48 participants (42.5%) who described ChatGPT and had
higher AI knowledge showed more correct answers than the other group (65 participants,
57.5%). Conclusion: Students with AI knowledge and the ability to describe ChatGPT
demonstrated a better understanding of its appropriate usage. Educating students on AI
fundamentals helps them utilise AI tools like ChatGPT effectively. This study offers
valuable insights on ChatGPT among pharmacy students, with implications forresponsible AI usage and education.

Introduction
ChatGPT, a large language model developed by OpenAI(San Francisco, California, USA), helps with tasks suchas answering questions, generating text, and engaging
in diverse conversations. It initially offered free accessto users starting in November 2022 and rapidly
garnered 100 million users worldwide within twomonths of its release (Mori & Hayashi, 2023a).
According to a report published by Nomura ResearchInstitute (NRI) in May 2023, Openai.com received 7.6
million visitors from Japan in mid-April, ranking third interms of traffic share after the United States (US) andIndia (Mori & Hayashi, 2023a).

Pharmacy Education 23(1) 665 - 675
ISSN 1477-2701 online © 2023 FIP

While ChatGPT’s versatility originates from its
extensive language understanding and broad
knowledge base, its application in education has
sparked considerable controversy. Universities
worldwidearedeveloping
guidelines/policies
concerning student utilisation of ChatGPT (Scribbr,
2023). Furthermore, certain institutions have
implemented restrictions, prohibiting the use of AIgenerated text for report writing without obtaining
faculty permission (University of Oxford Academic
Support, 2023; Harvard University Information andTechnology, 2023). As of July 2023, 251 universities inJapan have released official statements regarding their665

Iwasawa et al.

Pharmacy students' AI knowledge & attitudes: ChatGPT

utilisation of ChatGPT. In April 2023, Kitasato University(KU) in Tokyo, Japan, prohibited the use of Artificial
Intelligence (AI) services like ChatGPT for educationaland research reports, and ChatGPT for assignments anddissertations may be considered cheating unless faculty
permission is obtained (Kitasato University, 2023).

Methods

To facilitate digital transformation (DX) in Japan, there
is an urgent need to develop human resources inscience, mathematics, data science, and AI, leading toincreasing pressure for university education reform.
Japanese pharmacy schools adhere to the “Model Core
Curriculum for Pharmaceutical Education” establishedby the Ministry of Education, Culture, Sports, Science,
and Technology (MEXT). The revised core curriculumfor 2022, introduced in February 2023, emphasises theinclusion of AI, big data, and related technologies as
essential qualities for pharmacists (Ministry of
Education, Culture, Sports, Science, and Technology,
2023a). However, the incorporation of AI education
into pharmacy school curricula remains limited.
The rapid pace of developments in AI may significantly
influence healthcare and pharmacy practices. AI isalready being utilised in healthcare to predict adverse
drug reactions (Martin et al., 2022), as well as forpatient education (Görtz et al., 2023). The appropriate
use of digital health technology can have a positive
impact on patient care and treatment outcomes. The
American Society of Health System Pharmacists (ASHP)
highlights, in its “Statement on the Use of Artificial
Intelligence in Pharmacy,” that AI can enhance clinicaldecision support systems and improve patient care;moreover, it can optimise inventory management andproduct verification within pharmacy operations(Schutz et al., 2020). Therefore, the development ofdigital human resources in the field of pharmacy shouldcommence without delay. Given that AI education in
universities is still in its early stages, the integration of
AI education into the curriculum has become crucial, asemphasised in the revised Core Curriculum for
Pharmacy Education.
This study aims to investigate pharmacy students’knowledge of AI and their attitudes toward ChatGPT,
providing valuable insights for the introduction of AI
education. To the best of the author's knowledge, as ofJuly 2023, no previous studies have examined theknowledge and attitudes of pharmacy students
regarding AI and ChatGPT.

Pharmacy Education 23(1) 665 - 675

An online survey was conducted at KU with fourth-year
student pharmacists who had registered for the Drug
Information course during the first term of the2023−2024 academic year. The Drug Information
course is a required one-credit hour course that meetsfor ten consecutive weeks of 1.5 hours each classduring the first term of the fourth year of students’studies of KU’s six-year B.S. Pharmacy programme. The
course covers the fundamentals of drug information
practice, an introduction to various drug information
resources, and internet-based information retrieval, as
well as precautions to take when using medical
information obtained from the internet.
A total of 258 students were invited to participate in thesurvey. Participation was voluntary, and they were
notified on the questionnaire that all data collected
would be kept confidential. By the end of the first term,a document comprising an explanation of the survey
and a QR code to access the web page of the Google
Form was developed, and invitations to access it wereposted on the university’s website. The QR code wasalso printed on the last page of the lecture handouts
given to the students.
A web programme, Google Forms, was used to develop
the online questionnaire, which included a mixture ofmultiple-choice and Likert-scale questions. The survey
consisted of three sections: 1) Assessment of pharmacy
students’ knowledge of AI by asking for their level ofunderstanding of basic terms such as natural language
processing, machine learning, deep learning, andneural networks; 2) Assessment of Pharmacy students’impression of ChatGPT; and 3) Assessment of literacy
regarding the appropriate use of ChatGPT (Appendix A).
The questions were formulated based on the study’s
objectives, referring to AI-related literature (Furukawa& Sakai, 2023), including sources explaining ChatGPT,
as well as questions from previous studies on AI
regarding impressions of ChatGPT (Mehta et al.,2021;
Jha et al.,2022), and OpenAI's usage policy regarding
the proper use of ChatGPT (Open AI, 2023a; Open AI,
2023b). To ensure clarity and relevance, educational
experts reviewed the questions for any ambiguities inwording. Additionally, an AI research expert provided
valuable feedback by answering a prototype version of
the questionnaire, helping us refine and organise thequestions. To examine the psychometric scale's
reliability, the alpha coefficient (Cronbach's alpha) was
utilised for each section, assessing the internalreliability of the questionnaire items. An alphacoefficient of 0.7 or greater was considered sufficientfor internal reliability.

666

Iwasawa et al.

Pharmacy students' AI knowledge & attitudes: ChatGPT

Data were presented in the form of descriptive
statistics and analysed. Additionally, the participants
were divided into two groups based on their responses
to a question regarding their awareness of ChatGPT.
Group 1 comprised participants who indicated thatthey could describe ChatGPT, whereas Group 2
consisted of participants who indicated that they could
not describe ChatGPT or had never heard of ChatGPT.
Their characteristics were compared and analysed interms of their knowledge of AI, their impression of
ChatGPT, and their literacy regarding the appropriate
use of ChatGPT. Section 1 consisted of four ThreePoint-Likert-Scale questions. The scoring criteria wereas follows: -1 = “I have no idea,” 0 = “I have heard of it,
but I cannot explain it,” and 1 = “I know the term andcan explain it.” The median scores were compared
between the two groups. Section 2 comprised nineFive-Point-Likert-Scale questions. The scoring criteria
were as follows: 2 = “Agree,” 1 = “Somewhat Agree,” 0= “Neither Agree nor Disagree,” -1 = “Somewhat
Disagree,” and -2 = “Disagree”. The median scores werecompared between the two groups. Section 3 included
two multiple-choice questions, and the scoring criteria
were as follows: selecting the correct answer earned ascore of 1, while not selecting the correct answer
resulted in a score of -1.

(19.5%). Regarding their experience with ChatGPT, 22participants (19.5%) reported having used it, while 91
participants (80.5%) reported never having used it.
When asked if they had heard of ChatGPT, only five
participants (4.4%) indicated that they had never heardof it. On the other hand, 48 participants (42.5%) statedthey could describe it, whilst 60 participants (53.1%)
mentioned they had heard of it but were unable toprovide a description. Thirty-three students (29.2%)
were able to describe ChatGPT despite having no priorexperience using it. The narrative summary of the three
sections is below.

Data collection was carried out using Google Forms,
and all analyses were conducted using SPSS Statistics
Version 27 software (IBM Corporation, New York, NY,
USA). Quantitative data are presented as medians(minimum, maximum), while categorical data arereported as frequencies (percentages). The MannWhitney U test was used to assess the knowledge of AI,
the impression of ChatGPT, and literacy concerning theappropriate use of ChatGPT among two groups ofpharmacy students. To simplify the data, the total scoreof each section was categorised utilising cut-off values,and Fisher's exact test was employed to assess the twogroups of pharmacy students. Statistical significancewas determined using two-sided tests with an alphalevel of 0.05.

Results
Cronbach's alpha of 0.770 for Section 1 (Knowledge ofAI), 0.832 for Section 2 (Impression of ChatGPT), and
0.818 for Section 3 (Literacy concerning theappropriate use of ChatGPT) confirmed the internal
validity of the scale, all exceeding 0.7. Out of the 258
students invited, a total of 113 students (43.7%)
participated in the survey. Among the participants, themajority were female, accounting for 91 students(80.5%), while males accounted for 22 students

Pharmacy Education 23(1) 665 - 675

Knowledge of AI
In terms of students' knowledge, their familiarity withgeneral terms surpassed that of more specific terms.
Specifically, 83.2% of the participants were familiar
with the term "machine learning," while only 16.8% ofthem could describe it. Similarly, 71.7% of theparticipants were familiar with the term "deeplearning," but only 12.4% of them were able to describe
it. Regarding the terms "natural language processing"and "neural networks," 26.5% of the participants werefamiliar with these terms, while only 2.6% of them wereable to describe them.
Impression of ChatGPT
To commence, the general impressions of ChatGPT
from participants were gathered. When participants
were asked about the reliability of ChatGPT, 24.9%
answered “reliable” or “somewhat reliable.”
Additionally, 65.5% of participants responded “agree”
or “somewhat agree” when asked if ChatGPT helps
themgatherinformation
more
efficiently.
Furthermore, concerning the comprehension of theirresearch findings, 50.4% of participants indicated thatthey “agree” or “somewhat agree” that utilising
ChatGPT enhances their comprehension. In contrast,
54.8% of participants expressed that utilising ChatGPT
improves their learning efficiency, while 43.3%
mentioned that it enhances their knowledge andunderstanding. As for concerns or doubts about using
ChatGPT, 40.7% of participants indicated that they had
such reservations.
Next, the impact of ChatGPT on education and theprofession was explored. When asked if they believed
ChatGPT would have a positive impact on education,
32.7% agreed or somewhat agreed, while 32.7%
remained neutral. When asked if they believed the
responses and information available from ChatGPT
were adequate for gathering information on pharmacyrelated topics, only 15.9% indicated that they wereadequate or somewhat adequate. Moreover, when
asked if they believed ChatGPT would assist

667

Iwasawa et al.

Pharmacy students' AI knowledge & attitudes: ChatGPT

pharmacists in improving their work efficiently, 32.0%
responded that it would be helpful or somewhat
helpful, and 33.6% responded neutrally.

medical information,” at 83.2% and 66.4%,
respectively. The second question was asked to confirm
participants' understanding of general precautions
when using ChatGPT (Open AI, 2023b). Only 28 of 113
participants (24.8%) were able to identify all theappropriate items. Of the three correct responses,
91.2% of the participants were able to select“Sometimes nonsensical,” 72.6% were able to select“Limited capability to provide medical, legal, and otherprofessional advice,” and 66.4% were able to select“Can be used as learning data.”

Literacy concerning the appropriate use of ChatGPT
To check literacy regarding the proper use of ChatGPT,
two types of choice questions were created. The firstquestion was designed to ascertain participants'
understanding of OpenAI's published ChatGPT usage
policies (Open AI, 2023a). Then the participants wereasked to identify all inappropriate purposes for the useof ChatGPT. Only four out of 113 participants (3.5%)
were able to identify all inappropriate purposes. The
item with the fewest number of correct answers, at
18.6%, was "Precise computations involving complex
mathematical formulas." The item with the secondlowest number of correct responses was"Automatically extract big data from websites," at
20.4%. Relatively more correct answers were given for“obtaining personal information” and “providing

In the second part of this study, participants weredivided into two groups based on their responses
regarding their awareness of ChatGPT. Theircharacteristics were compared and analysed in termsof their knowledge of AI, impressions of ChatGPT, andliteracy regarding the appropriate use of ChatGPT.
Table I provides a summary of the second part of thestudy.

Table I: Comparison of the scores of the two groups on each of the questions in the three sections
Group 1 (n = 48)
Questionn(%)

Medianscore[min, max]Group 2 (n = 65)n(%)

Medianscore[min, max]

p-value(1) Assessing knowledge of Artificial Intelligence:
Please tell us about the following termsb

Q1 Natural language processing-1 [ -1, 1 ]-1 [ -1, 0 ]<0.001Q2 Machine learning

0 [ -1, 1 ]0 [ -1, 1 ]0.022

Q3 Deep learning

0 [ -1, 1 ]0 [ -1, 1 ]0.023

Q4 Neural networks-1 [ -1, 1 ]-1 [ -1, 0 ]0.049

Total score of section (1)-1 [ -4, 4 ]-2 [ -4, 1 ]0.002

Number of section (1) score was ≥ 0 (cutoff value)17(35.4%)

7(11.1%)bbbb<0.001c(2) Assessing the impression of ChatGPTQ1 Do you think the information obtained using ChatGPT isreliable?

-1 [ -2, 2 ]0 [ -2, 2 ]0.306

Q2 Do you believe that utilising ChatGPT will enhance theefficiency of your information gathering?

1 [ -2, 2 ]1 [ -2, 2 ]0.061

Q3 Do you believe that utilising ChatGPT will improve yourcomprehension of your research findings?

1 [ -2, 2 ]0 [ -2, 2 ]0.027

Q4 Do you believe that utilising ChatGPT will improve yourefficiency in learning?

1 [ -2, 2 ]0 [ -2, 2 ]0.075

Q5 Do you believe that utilising ChatGPT will enhance yourknowledge and understanding?

0 [ -2, 2 ]0 [ -2, 2 ]0.275

Q6 Do you have any concerns or queries regarding the usageof ChatGPT?

1 [ -1, 1 ]

1 [ -1, 1 ]0.859

Q7 Do you believe that ChatGPT has a positive impact on
education?

0 [ -2, 2 ]0 [ -2, 2 ]0.111

Q8 Do you believe that the responses and information
provided by ChatGPT are suitable for gathering information
on pharmacy-related topics?

-1 [ -2, 2 ]0 [ -2, 1 ]0.328

Pharmacy Education 23(1) 665 - 675

b

b

b

b

b

b

b

b

668

Iwasawa et al.

Pharmacy students' AI knowledge & attitudes: ChatGPT

Table I: Comparison of the scores of the two groups on each of the questions in the three sections (Continued)
Group 1 (n = 48)
QuestionnMedianscore[min, max](%)

Group 2 (n = 65)nMedianscore[min, max](%)

p-value

Q9 Do you believe that ChatGPT can assist pharmacists inimproving their work efficiency?

0 [ -2, 2 ]0 [ -2, 2 ]0.654

Total score of section (2)2 [-16, 17 ]1 [ -16, 11 ]0.384

No cutoff values were detected in section (2)

-

-

-

-

13

(20.0%)

41(63.0%)

8(12.3%)

53(81.5%)

33(50.8%)bb-(3) Assessing literacy regarding the appropriate use of ChatGPTQ1 Please identify all inappropriate purposes for utilizing ChatGPT. (True or False)
A. Using ChatGPT to automatically extract big data fromwebsites

10(20.8%)

B. Using ChatGPT to provide medical information

34(70.8%)

C. Using ChatGPT for precise computations involving complex
mathematical formulas

13(27.1%)

D. Using ChatGPT to obtain personal information

41(85.4%)

E. Elementary school students independently using ChatGPT
for self-study

24(50.0%)

a

a

a

a

a

Total score of section (3) Q1

a

c

1

a

c

0.426

a

c

0.054

a

c

0.622

a

1 [ -3, 5 ]c1-1 [ -3, 5 ]

b

0.219

Q2 Please select all the correct statements regarding ChatGPT (True or False)
45(93.8%)

B. The information entered into ChatGPT may potentially be
used as training data

32

(66.7%)

C. There is no need to worry about copyright when using
ChatGPT

40(83.3%)

D. It is possible to distinguish between AI-generated text andother information obtained from ChatGPT

35(72.9%)

E. By using ChatGPT, you can verify the most up-to-date
information36(75.0%)

F. If you ask the same question to ChatGPT repeatedly, youwill always receive the same answer

43(89.6%)

G. ChatGPT has limited capacity to provide medical, legal,
and other professional advice

40(83.3%)

a

a

a

a

a

a

a

58

(89.2%)

43(66.2%)

62(95.4%)

50(76.9%)

48(73.9%)

54(83.1%)

42(64.6%)

c

0.513

a

c

1

a

c

0.051

a

c

0.664

a

c

1

a

c

0.418

a

c

0.034

b

Total score of section (3) Q2

5 [ -1, 7 ]5 [ -3, 7 ]0.544

Total score of section (3) Q1 + Q2

4 [ -2, 10 ]4 [ -6, 8 ]0.194

Number of section (3) Q1 + Q2 scores were ≥ 8 (cutoff value)aaA. Sometimes, ChatGPT may generate inaccurate information

18b(37.5% )9(13.8%)

b
c

0.007

c

Number (%) of students selected correct answer; Mann - Whitney U test; Fisher's extract test

Group 1 comprised 48 participants who could describe
ChatGPT, while Group 2 consisted of 65 participants
who were unable to provide a description, including
five participants who had never heard of ChatGPT. In
terms of understanding AI, participants in Group 1
displayed higher median scores compared to Group 2 (1 vs. -2). Notably, Group 1 scores were significantly
higher than those of Group 2 across all dimensions ofAI knowledge (p < 0.001). While participants in Group 1
exhibited a greater familiarity with fundamental AI
terminology than their Group 2 counterparts, themedian total score for both groups remained negative,
indicating a knowledge shortfall. Additionally, in

Pharmacy Education 23(1) 665 - 675

Section 1, only 35.4% of Group 1 participants achieved
a cutoff of zero or higher. Figure 1 illustrates thecomparison of scores on basic AI-related terms'
knowledge in both groups, with Group 1 representing alarger proportion of positive responses as the numberof positive responses increases.
Regarding their impression of ChatGPT, both groups
displayed positive median total scores, indicating afavourable impression of ChatGPT. However, no
statistically significant differences were identified in
the total scores during between-group comparisons,and no cutoff values were established where the trend
achieved significance. In Section 2, a notably higher

669

Iwasawa et al.

Pharmacy students' AI knowledge & attitudes: ChatGPT

median for Group 1 was observed for Q3: “Do you
believe that utilising ChatGPT will improve yourcomprehension of your research findings?” Conversely,
within Section 2, significantly higher medians for Group
2 were seen in Q1: “Do you think the information
obtained using ChatGPT reliable?” and Q8: “Do you

believe that the responses and information provided by
ChatGPT are suitable for gathering information on
pharmacy-related topics?” Figure 2 compares thescores on positive impressions of ChatGPT in bothgroups, and no significant differences in the
distribution of scores were observed.

Figure 1: Comparison of scores on knowledge of basic terms related to AI in both groups

Figure 2: Comparison of scores on positive images for ChatGPT in both groups

In Section 3, both groups exhibited positive median
total scores; however, no statistically significant
differences were observed between the two groups.

Pharmacy Education 23(1) 665 - 675

Nevertheless, a notable disparity was evident in thenumber of individuals in Group 1 who demonstrated
high literacy (the total score of Q1 and Q2 was eight or670

Iwasawa et al.

Pharmacy students' AI knowledge & attitudes: ChatGPT

higher). In Group 1, 18 participants (37.5%) scoredabove the cutoff, whereas in Group 2, nine participants(13.8%) scored above the cutoff (p = 0.007). Figure 3
illustrates the comparison of scores between the twogroups regarding their literacy in the appropriate use of

ChatGPT. The score distribution indicates that as thenumber of correct responses increases, Group 1
represents a larger percentage of the total correct
answers.

Figure 3: Comparison of scores on knowledge of the proper use of ChatGPT in both groups

Discussion
To the best of the authors’ knowledge, this is the firststudy aiming to assess pharmacy students' knowledgeof AI and their impressions and literacy of ChatGPT. The
study found that pharmacy students were morefamiliar with general terms such as machine learningand deep learning compared to more specific terms.
While some of the students who were able to describe
ChatGPT demonstrated a high score, the majority
stated that they were unable to define these terms.
Several studies have investigated familiarity with AIrelated terms, and their findings align with the study
mentioned (Mehta et al., 2021; Boillat et al., 2022; Jhaet al., 2022). For instance, Mehta and authors
conducted a survey on medical students' knowledge
and attitudes towards AI, ranking students'
understanding of AI-related terms from highest to
lowest as follows: Understanding the meaning of AI(83.3%), machine learning (65.9%), neural networks(42.3%), and deep learning (18.7%) (Mehta et al.,2021). In another study conducted in the United ArabEmirates, similar scores were observed for AI andmachine learning that is higher scores, while deeplearning and neural networks scored lower (Boillat etal., 2022).

Pharmacy Education 23(1) 665 - 675

The KU School of Pharmacy has not incorporated AI
education into its curriculum, resulting in students
acquiring knowledge about AI mainly through selfteaching, leaving them uninformed about AI. In 2022,
Japan had 790 universities, but only a limited numberof them offered AI education. To expand opportunities
for improving skills in mathematics, data science, andAI, MEXT implemented the "Accreditation System” in
2021. However, as of August 2022, only 96 university
programmes were accredited, including only one
pharmacy school. This shortage of AI education isparticularly evident in pharmacy faculties, where asignificant number of students enrol with the goal of
pursuing a career in pharmacy. As a result, a substantial
portion of their study time is dedicated to mastering
the coursework required for national examinations.
Consequently,
information-related
coursesinpharmacy schools primarily focus on developing
information literacy skills to enhance students'
academic training at universities. Doi and colleagues
conducted a survey on ICT-related courses offered inpharmacy faculties across 77 universities and 79
faculties, revealing a lack of focus on clinicalapplications of ICT (Doi et al., 2022). They proposed
integrating ICT literacy into existing courses, suggesting
the incorporation of ICT-related learning components
into relevant subjects (Doi & Tomizawa, 2022). To

671

Iwasawa et al.

Pharmacy students' AI knowledge & attitudes: ChatGPT

prepare for the digital future in healthcare, it isnecessary to develop a course that fosters thefundamentals of AI knowledge and its clinicalapplications.

but could explain ChatGPT. Some recent reports on theactual use of ChatGPT in Japan have been published(Mori & Hayashi, 2023a; Mori & Hayashi, 2023b; Omori
et al., 2023). In June 2023, an online survey was
conducted among university students nationwide, with4,000 participants enrolled (Omori et al., 2023). The
survey revealed that 89.8% of the participants were
aware of ChatGPT, and 32% had utilised it. Among thestudents, 45.5% in the departments of science,technology, and agriculture used ChatGPT most, while
21.2% in the fields of Medicine, Nursing, Other Medical
Sciences, Dentistry, and Pharmacy used it. The survey
conducted by NRI also reported that ChatGPT exhibitedahighadoptionrate
among
undergraduate/graduate/professional
students(21.6%) and faculty/staff (20.5%) among different
occupational groups (Mori & Hayashi, 2023a).
However, the usage rate was relatively low among
doctors/medical professionals (5.3%). Although thesample size was small, the study results showed that
more than 95 % of participants were aware of ChatGPT
compared to previous studies (Mori & Hayashi, 2023a;
Omori et al., 2023). However, the rate of utilising
ChatGPT was found to be low, consistent with thefindings from those other studies (Mori & Hayashi,
2023a; Omori et al., 2023).

The lack of AI education in the curriculum is a common
problem not only in pharmacy schools but also in
medical schools, and it is prevalent not only in Japanbut also worldwide. Ejaz and authors conducted
research on the state of AI in medical education
globally and reported that the number of students who
had received teaching on AI was limited, highlighting
the importance of incorporating AI education into theircurriculum (Ejaz et al., 2022). Moreover, a study
conducted in the United Kingdom revealed that only
9.2% of surveyed students received some form ofteaching on AI (Sit et al., 2020). A recent systematic
review also showed that most students had low
knowledge and limited skills in working with AI (Baigi etal., 2023).
Mantel-Teeuwisse and colleagues revealed thatstudents exposed to digital health education inpharmacy schools were 2.5 times more likely to pursue
continuous education in this field, highlighting thepivotal role of early introduction (Mantel-Teeuwisse etal., 2021). Therefore, it is essential for universities in
the healthcare domain to incorporate AI knowledge
and skills into their curriculum. The study also
highlighted various challenges in implementing digital
health education within the healthcare professions(Mantel-Teeuwisse et al., 2021). These challenges
include the absence of necessary standards for digital
health education, a dearth of appropriately trained
academics, and a lack of collaborators with expertise inscientific fields like AI and machine learning.
Additionally, there is a scarcity of educational
materials, case studies, and adequate laboratory spacein the digital health field (Mantel-Teeuwisse et al.,2021). In response to the survey results, theInternational Pharmaceutical Federation (FIP) is
offering an educational programme on digital health toits members (International Pharmaceutical Federations(FIP), 2022), although the availability of educational
opportunities remains limited. Despite various issues inhealthcare universities, such as overcrowded curriculaand a lack of human resources capable of providing AI
education, it is crucial to devise creative educational
methods. These methods could include fosteringcollaboration among multiple professions andintegrating the clinical application of AI into eachrelated subject, nurturing future digital healthprofessionals.
Regarding the impression of ChatGPT, approximately
20% of the participants had experience using it, while
30% of the participants stated they had no experience

Pharmacy Education 23(1) 665 - 675

While only a small number of participants reported
having prior experience using ChatGPT, a certain
percentage of them were aware of its existence but
chose not to utilise it. Interestingly, over half of theparticipants expressed their belief that ChatGPT could
help them gather information more efficiently andenhance their learning efficiency. However, it isimportant to note that about 40% of the participants
exhibited concerns or doubts regarding the use of
ChatGPT. Notably, in the study, only 25% of theparticipants selected ChatGPT as reliable or somewhat
reliable. This indicates that doubts about the reliabilityof the information may significantly influence students'
decisions not to use ChatGPT. Conversely, the study
suggests that the group unable to explain ChatGPT
might be more inclined to rely on the accuracy andexpertise of ChatGPT responses. This raises the concernthat students might use ChatGPT without fullycomprehending its capabilities and limitations. This
underscores the importance of promptly providing
literacy education.
Regarding literacy on the proper use of ChatGPT, thecorrect response rate was low for both OpenAI's policyon the use of ChatGPT and the instructions on its
proper use. This indicates a need for awareness andeducation. In the NRI survey, the authors mentionedthat one possible explanation for the relatively low
adoption of ChatGPT among doctors/medical
professionals is the limited scope of its application

672

Iwasawa et al.

Pharmacy students' AI knowledge & attitudes: ChatGPT

within their work, where its usage is still rather
restricted (Mori & Hayashi, 2023a). However, theresults indicate that few students understand OpenAI's
use policy. Instead, the findings suggest that somestudents might have chosen not to use ChatGPT due to
doubts about the reliability of the information or
because they are aware of the risks associated with
applying unreliable information to medical care.

Conclusion

To assess the characteristics of participants who could
describe ChatGPT and those who were unable to do so,they were divided into two groups and compared theirscores in three sections. The results indicated thatparticipants who were able to describe ChatGPT
displayed a greater understanding of AI concepts, withsome of them achieving a higher percentage of correct
answers concerning the appropriate use of ChatGPT.
This suggests that offering education about
fundamental AI knowledge will lead to a better
understanding and utilisation of AI tools like ChatGPT.
On the other hand, in terms of the impression of
ChatGPT, there was no difference between the twogroups, and both indicated having a positive impressionof ChatGPT. In a recent systematic review, many
healthcare students showed positive and promising
attitudes towards AI in medicine (Baigi et al., 2023),
aligning with the mentioned findings.

Students who possessed knowledge of AI and could
explain ChatGPT demonstrated a better understandingof the proper usage of ChatGPT. On the other hand,
many students did not comprehend the appropriate
usage of ChatGPT, highlighting the necessity toincorporate AI education into the curriculum. Given the
anticipation of ChatGPT's rapid growth in usage and thecorresponding increase in opportunities for its
application, there is an urgent need for literacy
education that encompasses knowledge about theproper usage of AI, including ChatGPT. Furthermore,this education should encompass warnings against
relying excessively on the accuracy and expertise of
information transmitted by AI. The study provides
valuable insights into ChatGPT familiarity, impressions,and understanding among pharmacy students, with
implications for responsible AI usage and education.

Conflict of interest
The authors declare no conflict of interest. There are no
financial conflicts of interest to disclose. This study did
not receive any funding.

Limitations

Source of funding

This study had several limitations. Firstly, it was
conducted in a single course at a lone school ofpharmacy in Japan, potentially limiting its
generalisability to students' attitudes at otherinstitutions worldwide. Moreover, the obtained
responses constituted only 43.7% of the total, raising
the possibility of bias in the results. Those who chose toparticipate in the survey may have already possessed
an interest in and knowledge of AI, potentially making
them more inclined to engage in the study.
Furthermore, due to the limited number of ChatGPT
users, group classification was based on participants'familiarity with ChatGPT rather than their experience
using the system. Another limitation arises from theself-reported nature of the data; participants'
responses were rooted in their perceptions and may
not entirely reflect their genuine understanding. To
address these limitations, future investigations will
involve objective knowledge tests focused specificallyon AI and ChatGPT. This approach aims to gain better
insights into students' comprehension and improve theaccuracy and reliability of the analysis.

The authors did not receive any funding.

Pharmacy Education 23(1) 665 - 675

Acknowledgements
We wish to thank the pharmacy students who
participated in the survey.

Ethics approval
This study has been reviewed by Kitasato University’s
Institutional Review Board, and it was determined to be
exempt from full review.

References
Baigi, S., Sarbaz, M., Ghaddaripouri, K., Ghaddaripouri, M.,
Mousavi, A. S., & Kimiafar, K. (2023). Attitudes, knowledge,
and skills towards artificial intelligence among healthcare673

Iwasawa et al.

Pharmacy students' AI knowledge & attitudes: ChatGPT

students: A systematic review. Health Science Reports, 6(3),e1138. https://doi.org/10.1002/hsr2.1138

artificial Intelligence in healthcare: A provincial survey studyof medical students. MedEdPublish, 10(75).
https://doi.org/10.1101/2021.01.14.21249830

Boillat, T., Nawaz, F. A., & Rivas, H. (2022). Readiness toembrace artificial intelligence among medical doctors andstudents: Questionnaire-based study. JMIR Medical
Education, 8(2), e34973. https://doi.org/10.2196/34973
Doi, N., Omi, A., Nakajima, M., & Tomizawa, T. (2022).
Understanding the current status and issues of ICT
education in the faculty of pharmacy. Japanese Journal of
Pharmaceutical Education, 6, 2021‒035.
https://doi.org/10.24489/jjphe.2021-035
Doi, N., & Tomizawa, T. (2022). Current status and future
prospects of ICT literacy education in pharmaceutics toward
Pharma Tech utilization. Japanese Journal of Pharmaceutical
Education, 6, 2022‒005.
https://doi.org/10.24489/jjphe.2022-005
Ejaz, H., McGrath, H., Wong, B. L. H., Guise, A., Vercauteren,T., & Shapey, J. (2022). Artificial intelligence and medical
education: A global mixed-methods study of medical
students’ perspectives. Digital Health, 8,20552076221089099.
https://doi.org/10.1177/20552076221089099
Furukawa, S., & Sakai, M. (2023). IT x Business Lecture
ChatGPT. The Future of Interactive AI. Impress
Görtz, M., Baumgärtner, K., Schmid, T., Muschko, M.,
Woessner, P., Gerlach, A., Byczkowski, M., Sültmann, H.,
Duensing, S., & Hohenfellner, M. (2023). An artificial
intelligence-based chatbot for prostate cancer education:
Design and patient evaluation study. Digital Health, 9,20552076231173304.
https://doi.org/10.1177/20552076231173304
Harvard University Information and Technology. (2023).
Initial guidelines for using ChatGPT and other generative AI
tools at Harvard. https://huit.harvard.edu/news/aiguidelines
International Pharmaceutical Federation (FIP). (2022). FIP
Train the trainer online course for educators on digital
health. https://www.fip.org/fipx-courses

Mantel-Teeuwisse, A. K., Meilianti, S., Khatri, B., Yi, W.,
Azzopardi, L. M., Acosta Gómez, J., Gülpınar, G., Bennara, K.,& Uzman, N. (2021). Digital health in pharmacy education:
Preparedness and responsiveness of pharmacy
programmes. Education Sciences, 11(6), 296.
https://doi.org/10.3390/educsci11060296
Ministry of Education, Culture, Sports, Science andTechnology. (2023). Model core curriculum forpharmaceutical education 2022 revised edition.
https://www.mext.go.jp/content/20230227-mxt_igaku100000058_01.pdf
Ministry of Education, Culture, Sports, Science andTechnology. (2021). Accreditation system for mathematicaland data science and AI education programs.
https://www.mext.go.jp/a_menu/koutou/suuri_datascience_ai/00001.htm
Mori, T., & Hayashi, H. (2023a). Trends in ChatGPT usage inJapan - Positive evaluation by most of the users.
https://www.nri.com/jp/knowledge/report/lst/2023/cc/052
6_1
Mori, T., & Hayashi, H. (2023b). Trends in ChatGPT usage inJapan - Increasing usage, especially among young people.
https://www.nri.com/jp/knowledge/report/lst/2023/cc/062
2_1
Omori, F., Saito, J., Matsuba, R., & Kita, T. (2023).
Preliminary results of a survey on the use of ChatGPT byuniversity students and its impact on their skill
development. https://dber.jp/wpcontent/uploads/2023/06/chatgpt_report.pdf
Open AI. (2023a). Usage policies.
https://openai.com/policies/usage-policies
Open AI. (2023b). Our approach to AI safety.
https://openai.com/blog/our-approach-to-ai-safety

Jha, N., Shankar, P. R., Al-Betar, M., Mukhia, R., Hada, K., &Palaian, S. (2022). Undergraduate medical students' andinterns' knowledge and perception of artificial intelligencein medicine. Advances in Medical Education and Practice,13, 927‒937. https://doi.org/10.2147/AMEP.S368519

Schutz, N., Olsen, C. A., McLaughlin, A. J., Yi, W. M., Nelson,
S. D., Kalichira, A. L., Smith, A. H., Miller, K. A., Le, T.,
Chaffee, B. W., Worthy Woodbury, C. D. R. K., & Patel, H.
(2020). ASHP statement on the use of artificial intelligencein pharmacy. American Journal of Health-System
Pharmacists, 77(23), 2015–2018.
https://doi.org/10.1093/ajhp/zxaa249

Kitasato University. (2023). Use of ChatGPT, etc. in
education and research. https://www.kitasatou.ac.jp/jp/news/20230414-05.html

Scribbr. (2023). University policies on AI writing tools:
Overview & list. https://www.scribbr.com/ai-tools/chatgptuniversity-policies/

Martin, G. L., Jouganous, J., Savidan, R., Bellec, A., Goehrs,C., Benkebil, M., Miremont, G., Micallef, J., Salvo, F.,
Pariente, A., & Létinier, L. (2022). Validation of artificial
intelligence to support the automatic coding of patient
adverse drug reaction reports, using nationwide
pharmacovigilance data. Drug Safety, 45, 535–548.
https://doi.org/10.1007/s40264-022-01153-8

Sit, C., Srinivasan, R., Amlani, A., Muthuswamy, K., Azam, A.,
Monzon, L., & Poon, D. S. (2020). Attitudes and perceptionsof UK medical students towards artificial intelligence andradiology: A multicentre survey. Insights Imaging, 11(1), 14.
https://doi.org/10.1186/s13244-019-0830-7

Mehta, N., Harish, V., Bilimoria, K., Morgado, F., Ginsburg,
S., Law, M., & Das, S. (2021). Knowledge of and attitudes on

Pharmacy Education 23(1) 665 - 675

University of Oxford Academic Support. (2023).
Unauthorised use of AI in exams and assessment.
https://academic.admin.ox.ac.uk/article/unauthorised-useof-ai-in-exams-and-assessment674

Iwasawa et al.

Pharmacy students' AI knowledge & attitudes: ChatGPT

Appendix A: Survey questionnaire
Survey on ChatGPT among pharmacy students
Gender□ Male □ Female

Awareness of ChatGPT□ Yes □ No

Experience with ChatGPT□ Yes □ No

(1) Assessing knowledge of Artificial Intelligence: Please tell us about the following terms
Q1 Natural language processing
Q2 Machine learning
Q3 Deep learning
Q4 Neural networks

I have no idea

I have heard of it, but I
cannot explain it

I know the term andcan explain it

□
□
□
□

□
□
□
□

□
□
□
□

(2) Assessing impression of ChatGPTQ1 Do you think the information obtained using
ChatGPT is reliable?
Q2 Do you believe that utilising ChatGPT will
enhance the efficiency of your information
gathering?
Q3 Do you believe that utilising ChatGPT will
improve your comprehension of your researchfindings?
Q4 Do you believe that utilising ChatGPT will
improve your efficiency in learning?
Q5 Do you believe that utilising ChatGPT will
enhance your knowledge and understanding?
Q6 Do you have any concerns or queries regarding
the usage of ChatGPT?
Q7 Do you believe that ChatGPT has a positive
impact on education?
Q8 Do you believe that the responses andinformation provided by ChatGPT are suitable forgathering information on pharmacy-related topics?
Q9 Do you believe that ChatGPT can assist
pharmacists in improving their work efficiency?

Agree

Somewhat
agree

Neither
agree nordisagreeSomewhatdisagreeDisagree

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

□

(3) Assessing literacy regarding the appropriate use of ChatGPTQ1 Please identify all inappropriate purposes for utilising ChatGPTA. Using ChatGPT to automatically extract big data from websites.
B. Using ChatGPT to provide medical information.
C. Using ChatGPT for precise computations involving complex mathematical formulas.
D. Using ChatGPT to obtain personal information.
E. Elementary school students independently using ChatGPT for self-study.

True

False

□
□
□
□
□

□
□
□
□
□

True

False

□
□
□
□
□
□
□

□
□
□
□
□
□
□

Q2 Please select all the correct statements regarding ChatGPTA. Sometimes, ChatGPT may generate inaccurate information.
B. The information entered into ChatGPT may potentially be used as training data.
C. There is no need to worry about copyright when using ChatGPT.
D. It is possible to distinguish between AI-generated text and other information obtained from ChatGPT.
E. By using ChatGPT, you can verify the most up-to-date information.
F. If you ask the same question to ChatGPT repeatedly, you will always receive the same answer.
G. ChatGPT has limited capacity to provide medical, legal, and other professional advice.

Pharmacy Education 23(1) 665 - 675

675