Development and Assessment of the Multiple Mini-Interview in a School of Pharmacy Admissions Model

Objective. To describe the development, implementation, and evaluation of the multiple mini-interview (MMI) within a PharmD admissions model. Methods. Demographic data and academic indicators were collected for all candidates who participated in Candidates’ Day (n = 253), along with the score for each MMI station criteria (7 stations). A survey was administered to all candidates that completed the MMI and another survey was administered to all interviewers to examine perceptions of the MMI. A three-facet multifaceted Rasch measurement (MFRM) analysis was completed to determine interviewer severity, candidate ability, and MMI station difficulty using data from Chapel Hill and Asheville. Data for the Asheville MFRM included both 2013-2014 and 2014-2015 admission cycles while the Chapel Hill MFRM analyzed data from the 2013-2014 cycle alone. Results. Analyses suggest that MMI stations assessed different attributes as designed, with Cronbach’s alpha for each station ranging from 0.90 to 0.95. All correlations between MMI station scores and academic indicators were negligible (rp < 0.2). No significant differences in average station scores were found based on age, gender, or race. Surveys indicated the MMI was generally considered acceptable by candidates and faculty. The MFRMs found differences in candidate ability explained 36-45% of the variance in MMI scores while differences in interviewer severity explained 9-16% of the variance in MMI scores. Conclusion. This study provides additional support for the use of the MMI as an admissions tool in pharmacy education. INTRODUCTION In addition to cognitive skills or academic ability, non-cognitive attributes are vital to the success of healthcare providers. Non-cognitive attributes, also referred to as professional attributes or skills, generally include non-academic indicators such as empathy, collaboration, leadership, and integrity. A growing body of literature demonstrates the need for healthcare providers to possess these non-cognitive skills, including the ability to work collaboratively on interdisciplinary healthcare teams and communicate effectively. The success of pharmacists amid ongoing healthcare reform is likely to depend on the development of these skills as they take on expanded roles. The evolving healthcare environment and growing importance of non-cognitive skills in the workplace has prompted numerous calls for curricular reform in the health professions. The UNC Eshelman School of Pharmacy is in the midst of redesigning its doctor of pharmacy curriculum in order to transform education, enlighten students, and ultimately to advance health care. Desired outcomes of the new curriculum include: exemplary practitioners who provide high-quality, team-based, patient-centered care; leaders and innovators who recognize the healthcare needs of patients and society and who lead teams toward improvement and change for the betterment of patient care; and lifelong learners who continually strive for positive impact. The new curriculum places a renewed emphasis on non-cognitive attributes that will help student pharmacists prepare for and participate in class, contribute to innovative problem solving, adapt to a changing healthcare system, and work as part of a healthcare team to solve complex problems. As pharmacists take on more responsibility in the care of patients and curricula are redesigned to meet these changes, pharmacy educators must be prepared to design and implement strategies for identifying and developing these professional attributes in student pharmacists. To assess the non-cognitive attributes of prospective student pharmacists, schools have traditionally used the structured interview. This approach typically consists of 1-2 interviewers spending time with a single candidate, asking pre-determined questions and engaging in discussion about the candidate’s experiences, opinions, and/or beliefs. However, research suggests that the subjectivity and bias associated with this approach can compromise the validity and reliability of interview results. In addition, evidence of a correlation between structured interview scores and success as a student or practitioner is lacking. To address the shortcomings of the structured interview, the multiple mini-interview (MMI) was developed and implemented as an admissions tool in Michael DeGroote Medical School at McMaster University. The logistics of the MMI are similar to those of an Objective Structured Clinical Exam (OSCE), which consists of multiple stations, approximately 5-10, in which the candidate has a short amount of time, generally 5-10 minutes, to respond to a situational prompt or case. Unlike structured interviews, the MMI allows the candidate to independently interact with multiple interviewers. Having these multiple, independent and focused interactions can reduce bias in assessing candidates. The MMI was developed to discriminate amongst prospective students based on non-cognitive abilities, and ideally better predict which applicants would be the most successful practitioners. Since its first description in the literature, the MMI has been replicated in many medical schools and, more recently, in pharmacy schools, pharmacy and medical residency programs, and veterinary schools. Across these settings, a number of studies provide evidence that suggests the MMI is a useful admissions tool for measuring professional attributes. In a systematic review of studies exploring the use of the MMI for student selection in health professions training, Pau et al. concluded that candidates and interviewers find the process acceptable and fair. Additionally, MMI performance does not strongly correlate with previous academic performance and appears to successfully evaluate non-academic traits as intended. There is some evidence that the MMI is predictive of success as a practitioner, as performance on the MMI has been shown to be predictive of clerkship performance in medical students. Given the importance of non-cognitive attributes in achieving the outcomes of the proposed new curriculum, the School identified the need to design and implement a new admissions model that would effectively identify and select students that possess characteristics likely to aid their success in the new curriculum and beyond. For the 2013-2014 admissions cycle, the UNC Eshelman School of Pharmacy implemented the MMI model as part of the doctor of pharmacy admissions process. The purpose of this paper is to describe the development, implementation, and evaluation of the MMI as an admissions tool at the UNC Eshelman School of Pharmacy. Evaluations of the MMI included: surveys of faculty and candidate impressions, analyses of correlation between candidate baseline characteristics and MMI performance, factor analysis of MMI stations, and multifaceted Rasch measurement (MFRM) of Chapel Hill and Asheville MMIs. Some of these data have been published and other data have been submitted for publication. This study is unique, as it examines a PharmD program in the United States using the MMI to replace the previous interview process and not just as a pilot program.