MMIRA Webinar
2023.03.06/07

An Outline of Today’s Talk

The Mixed Methods Education
Development Center Project"Development of a Guidebook and e-learning
programs for Mixed Methods for Nursing
Researchers “

I. My Mixed Methods Trajectory(KAKENHI grant number: 20H03966)

II. The Historical Development of Mixed Methods Research in Japan
https://www.mmredu.org/

The Prevalence and Challenges of Mixed Methods Research in Japan:
Focusing on the Field of Nursing

III. The Prevalence of Mixed Methods Research in Japan
IV. The Challenges of Mixed Methods Research Nursing Researchers in Japan Face
V. Concluding Thoughts

Hisako Kakai, Ph.D.
Aoyama Gakuin UniversityTokyo, JAPANhkakai@aoyamagakuin.jp© Hisako Kakai© Hisako Kakai

II. The Historical Development of MMR in Japan

I. My Mixed Methods Trajectory

Original Japanese Books on MMR

Translation of English Books on MMR
Late ‘90s

2000

2005

2007～
2007

Research Design(J. W. Creswell)

2015～

2013～

2010

Designing & Conducting MMR(Creswell & Plano Clark)
Introduction to MMR (2015)(H. Kakai)© Hisako Kakai

2017

2017

A Concise Introduction to MMR(J. W. Creswell)© Hisako Kakai

A Guide to MMR (2021)(Fetters & Kakai (Eds.)

Foundation of MMR(Teddlie & Tashakkori)

Invitation to MMR (2016)(JSMMR)

Membership and Activities of JSMMR

III. The Prevalence of Mixed Methods Research in Japan JSMMR was established in 2015.
JSMMR currently has roughly 240 members:
12% of its members are JSMMR-MMIRA dual membership holders.
The largest group of its members are nursing researchers.
 JSMMR offers the following types of activities:
Conferences
Workshops
Colloquiums
Seminars
Journal Publication (Annals of Mixed Methods Research, since Apr. 2022)

Extracted references using the following keywords in the title or abstract:
混合研究法 OR ミックス法 OR ミックスメソッド OR ミックスドメソッド OR“mixed method” OR “mixed methods”© Hisako Kakai© Hisako Kakai© Hisako Kakai© Hisako Kakain = 418

© Hisako Kakai© Hisako Kakai

IV. The Challenges of Mixed Methods Research
Nursing Researchers in Japan Face

Project title:“Development of a Guidebook and e-learning programs for Mixed Methods for NursingResearchers” (KAKENHI grant number: 20H03966)
Principal Investigator: KAKAI, Hisako
Research Manager: ABE, Michiko
Project members (alphabetical order):Nursing Researchers: FUKUDA, Miwako; KAMEI, Tomoko; MAKABE, Sachiko;
Nozaki, Manami; OKAWARA, Chikako
Researchers in Other Fields: HATTA, Taichi; INABA, Mitsuyuki; INOUE, Michiko;
KAWAMURA, Yoko; NARITA, Keiichi; TAKAGI, Akiko;
TAJIMA, Chihiro© Hisako Kakai© Hisako Kakai

Culturally-Specific Mixed Methods

The socio-ecological framework
for the field of mixed methods research
The socio-ecological framework“acknowledges that mixed methods
research practice, which is broadly defined
as any use of mixed methods byresearchers, scholars, and otherstakeholders, is a dynamic and interactive

process and involves complex relationships
among multiple methodological domains
and contexts that inform this process.”
Creswell & Sinley (2017, p. 102)© Hisako Kakai(Ivankova & Plano Clark, 2018, pp. 412-413)© Hisako Kakai

The Design of Our 5-year Projectbuildingbuilding

Survey Researchwith
Nursing Researchers

A Delphi study
withMM experts

Data Source

The 1st Round

QUAL

Developing an MM
educational guide

QUAN+Integration

Hatta et al.

The 2nd RoundQUAN+qual

QUAL
Focus GroupInterview(FGI)

Data Sources and MM Elements of the Four Integrative Analyses in Our Project

Inaba et al.

group membership datainterview

crossover mixed analysis
text mining + Constructivist GT

Okawara et al.

group membership datainterview

crossover mixed analysis
sentiment analysis + qualitative
content analysis

integration

2024© Hisako Kakai

2021, 2022

convergent design +CL data transformation of QAUAL +Chi-square test or Fisherʼs exact test

Evaluating MM elearning programs

QUAL

2020

demographic data
surveyinterview

Kawamura et al.

The 4th Round

2020

convergent design
CLANOVA + Thematic Analysis

Building MM elearning programs

The 3rd RoundQUAN+qual

Mixed Methods Elements

surveyinterview

2023, 2024
Kakai (2021)© Hisako Kakai

Participant Flow Chart of Phase I

Characteristics of Survey Participants

Online Survey Participants (n=322)

Kakai et al. (2022)(a paper published inJapanese)

StatusCompleted MM Skill Assessment(Guetterman et al., 2017)(n=113)

Awareness of MMR
Only know

Not completed MM Skill Assessment(Guetterman et al., 2017)(n=209)Other

the name

6%9%Graduate
Inaba et al. (2021, Oct.
30-31)
Okawara et al. (2021, Oct.
30-31)

Participated in Focus Group Interviews(n=10)

20%

MMR
12%

NoStudents
Participated in Focus Group Interviews(n=21)

Methodological expertise

17%

Faculty

Yes
74%

74%

QUAL
39%

Integrative Analysis

QUAN
49%

n=21

Hatta et al. (2021, Oct. 30-31)
Kawamura et al. (manuscript underpreparation)n = 262n = 302QUAN
n=21n = 208

QUAL
n=21

Kakai et al. (2022)© Hisako Kakai© Hisako Kakai

Teaching experience and Years of teaching research methods-related courses by position(n = 207)

Years of teachingexperience

Years of teaching
QUAN

Years of teaching
QUAL
Years of teaching
MMR

Professor(n=58)

Kakai et al. (2022)© Hisako Kakai

Associate
Professor(n=41)

Assistant
Professor(n=37)

Associate
Lecturer(n=60)

AssistantLecturer(n=11)

Kakai et al. (2022)© Hisako KakaiA Diagram for Procedures and Products in Phase I Convergent MM Study

Characteristics of Interview Participants

StatusAwareness of MMR

Methodological expertise

Only know
Otherthe name

10%

10%

MMR
5%
QUAL

Graduate
Students

Faculty

Yes

22%

68%

90%

n = 31

33%

QUAN
62%

Procedures- nursing researchers- online survey- demographic
characteristics- Years of experience inlearning, teaching, andconducting QUAN, QUAL,and MMR- MM skill Assessment(Guetterman, et al. 2017)- descriptive stats.
- cluster analysis- Chi-square- ANOVAn = 21n = 31

QUAN DATACOLLECTIONQUAN DATAANALYSIS

Products- n =322- a list for
QUAL datacollection- n=21- typologiesof theparticipants

Procedures

Products- 8 FGIs(+3ind.interviews)- challenges
and learningneeds of MMR- n=31- interview
transcripts- transcribing
data- coding- developing
themes- text-mining

MergedInterpretation

Kakai et al. (2022)© Hisako Kakai© Hisako Kakai

A Self-Rated Mixed Methods Skills Assessment Instrument(Guetterman et al., 2017) The self-rated mixed methods assessment instrument was originally developed to help
participants in the Mixed Methods Research Training Program for Health Sciences(MMRTP) assess the extent to which they had acquired MMR skills (Earley, 2014).
 The instrument exhibited good reliability and validity, thus usable for university
faculty and graduate students (Guetterman et al. 2017).
 This instrument focuses on six areas, including (1) research questions, (2) design,(3) sampling, (4) data collection, (5) analysis, and (6) dissemination.
 The instrument helps individuals assess their ability to define and explain mixed
methods concepts, their ability to apply them to problems, and the extent to which
they require improvement in these six areas.
 The instrument also asks respondents to rate their professional experiences withmixed methods, familiarity with software, and familiarity with mixed methods
references.
© Hisako Kakai© Hisako Kakai

QUAL DATACOLLECTIONQUAL DATAANALYSIS- n=21- quotes- codes- themes- visualpresentations-side-by-side comparison- data transformation- Joint displays

Integrated Findings(merged interpretations) [Challenges & Learning Needs] x [MM Experiences] (Hatta et al., 2021, Oct. 30-31)
The challenges & learning needs of 4 clusters generated by the participants’ MMR
experiences (defined by methodological training backgrounds and achievements inMMR) were compared and contrasted.
 Different patterns in motivations to learn MMR and attributions related to MM
challenges were found across groups.
 [Challenges & Learning Needs] x [Self-evaluation of MM Skills](Kawamura et al., manuscript under preparation)
The challenges & learning needs of participants classified into 3 categories according
to the participants’ self-assessment of MM skills were compared and contrasted.
 Different levels and scopes of needs related to MM learning were exhibited across groups.

Necessary to design e-learning programs considering the different learning needs of the users atdifferent levels.
© Hisako Kakai© Hisako Kakai

Findings from Crossover Mixed Analyses (Hitchcock & Onwuenbuzie, 2019)
 Identifying Challenges of MMR using GTxA (Inaba et al., 2021, Oct. 30-31)
Many participants perceived “qualitative research” as a challenge of MMR. The
use of MMR was often discouraged due to its heavy workload and timeconsuming nature.
＊GTxA=Grounded Text Mining Approach (Inaba & Kakai, 2019; in press) Identifying Challenges of MMR using qualitative content analysis and sentimentanalysis (Okawara et al., 2021, Oct. 30-31)
Many perceived qualitative research as a challenge of MMR. Due to the exploratory
nature of qualitative methods, MMR has problems such as the inability to maintainconsistency in research questions and the difficulty of integrating qualitative andquantitative results.

© Hisako Kakai© Hisako Kakai

Responding to the Identified Challenges of Nursing Researchers in Japan

Challenges of Conducting MMR Identified by Nursing Researchers in Japan - A Summary From FGI

Environmental
challenges（Other Elements）
Lack ofunderstanding/support for
MMR in the workplace

Lack ofinstructors/mentors

Lack of colleagues toDifficulty in getting
work/study together
approval from IRB
Individual Challenges on MMR
Lack of confidence in :
 Qual. methods
 Quan. methods / statistics
 Both Qual. and Quan
methods
 Knowledge and Skills inMMR
Lack of knowledge to negotiatewith IRB
Cannot findtextbooks or
Language (English)
methodologicalbarriers
research articles inJapaneseDifficulty in obtaining“Time”, “Energy”,“Motivation”-Increase thevisibility andthe level ofunderstandingof mixed
researchmethods

Environmental
challenges
Lack of understanding/support
for MMR in the workplace

Lack ofinstructors/mentors
Lack of colleagues towork/study together
on MMR

Difficulty in getting
approval from IRB

Individual Challenges”Self-learning”“Feelings of not
wanting to fail”“Seeking efficiency”- Expanding Japanese
MMR literature- Helping researchers
to disseminate theirworks in English.

© Hisako Kakai

Lack of confidence in :
 Qual. methods
 Quan. methods or statistics
 Both Qual. and Quan methods
 Knowledge and Skills of MMR
Lack of knowledge to negotiatewith IRB-Developing e-learning
programs that are culturally
and linguistically appropriate
for learners.
-Developing e-learning
programs considering thedifferent needs of learners.
- Developing e-learning
programs incorporating
insights and advice of MM
scholars inside/outside thecountry.

Language (English)
barriers
Cannot find textbooks
or methodologicalresearch articles© Hisako Kakai

References

V. Concluding Thoughts (Implications for MMR)An Overall Research Purpose of the Project

Polyphonic Meta‐inferences of the Team Project“Polyphonic”
Integration

 Using a team approach for achieving“polyphonic” integration for investigating acomplex phenomenon.
 Investigating team approach within theJapanese cultural context.
 Paying more attention to language-related
challenges for promoting the global use ofMMR.

Meta‐inference A’
RQ：A’

Meta‐inference B’
RQ：B’

MM Integrative
Analysis A

MM Integrative
Analysis B

Meta‐inference C’
RQ：C’
MM Integrative
Analysis C

An image of the team approach to polyphonic integration© Hisako Kakai-Providing opportunities forresearchers to find their MM
mentors/partners

Kakai et al. (2021, Oct. 30-31)
31

Creswell, J. W. (2003). Research Design ‐ Qualitative, Quantitative, and Mixed Methods Approaches (2nd ed.). Sage.
Creswell, J. W. (2015). A Concise Introduction to Mixed Methods Research (1st ed.). Sage.
Creswell, J. W., & Plano Clark, V. L. (2007). Designing and conducting mixed methods research. Sage.
Creswell, J. W., & Sinley, R. C. (2017). Developing a culturally‐specific mixed methods approach to global research. Kölner
Zeitschrift für Soziologie und Sozialpsychologie, 69(Supplement 2), 87‐105.
Earley, M. A. (2014). A synthesis of the literature on research methods education. Teaching in Higher Education, 19(3), 242‐253.
Fetters, M. D., & Kakai, H. (2021). A guide to mixed methods research: A treasure hunt for learning research design to writing forpublication [in Japanese]. Tomishobo.
Guetterman, T. C., Creswell, J. W., Wittink, M., Barg, F. K., Castro, F. G., Dahlberg, B., Watkins, D. C., Deutsch, C., & Gallo, J. J.
(2017). Development of a self‐rated mixed methods skills assessment: the NIH mixed methods research training program forthe health sciences. The Journal of continuing education in the health professions, 37(2), 76.
Hatta, T., Abe, M., Tajima, C., and Kakai, H. (October 30‐31, 2021). Forms of "integration" in Mixed Methods Research (MMR):
Implications of MMR on educational needs [in Japanese]. Paper presented at the 2021 MMIRA Asia Regional Conference.
Hirose, M., & Creswell, J. W. (2023). Applying core quality criteria of mixed methods research to an empirical study. Journal ofMixed Methods Research, 17(1), 12‐28.
Hitchcock, J. H., & Onwuegbuzie, A. J. (2020). Developing mixed methods crossover analysis approaches. Journal of MixedMethods Research, 14(1), 63‐83.
Inaba, M., & Kakai, H. (2019). Grounded text mining approach: A synergy between grounded theory and text mining
approaches. In A. Bryant & K. Charmaz (Eds.). The Sage handbook of current developments in grounded theory (332‐351).
Sage.
Inaba, M., & Kakai, H. (in press). Grounded text mining approach: An integration strategy of grounded theory and textual datamining. In C. N. Poth (Ed.).The Sage handbook of mixed methods research design. Sage.
Inaba, M., Okawara, C., Makabe, S., Fukuda, Mo, Nozaki, M., Abe, M., and Kakai, H. (October 30‐31, 2021). Hurdles in learningand practicing mixed methods research in nursing research: An analysis of focus group interviews using GTxA [in Japanese].
Paper presented at the 2021 MMIRA Asia Regional Conference.

© Hisako Kakai

Ivankova, N. V., & Kawamura, Y. (2010). Emerging trends in the utilization of integrated designs in the social, behavioral, andhealth sciences. The Sage handbook of mixed methods in social and behavioral research, 2, 581‐611.
Ivankova, N. V., & Plano Clark, V. L. (2018). Teaching mixed methods research: using a socio‐ecological framework as apedagogical approach for addressing the complexity of the field. International Journal of Social Research Methodology, 21(4),409‐424.
Kakai, H. (2015). Introduction to mixed methods research ‐ The art of integration through qualitative and quantitative approaches[in Japanese]. Igaku Shoin.
Kakai, H. (2021). Mixed methods for nursing researchers: Developing an e‐Learning System and its Significance [in Japanese].
Nursing Research, 54(2), 94‐100.
Kakai, H. Hatta, T., Okawara, C., Inaba, M., Abe, M., & Kamei, T. (2021, October 30‐31). Examples of ways to achieve "integration"in mixed methods research ‐ A team approach to polyphonic integration [in Japanese]. Paper presented at the 2021 MMIRA
Asia Regional Conference.
Kakai, H., Hatta, T., Abe, M., Okawara, C., Makabe, S., Fukuda, M., Nozaki, M., & Kamei, T. (2022). Awareness and utilization ofmixed methods research and issues in its research practice: From a survey of nursing researchers [in Japanese]. Aoyama Journalof International Studies no. 9, 93‐121.
Kakai, H., & Narita, K. (2016). Invitation to mixed methods research: A new approach bridging qualitative and quantitative
research [in Japanese]. Tomishobou.
Kawamura, Y., & Kakai, H. (manuscript under preparation). Examining learning needs of nursing researchers based on their self‐assessment of mixed methods skills measured by the Guetterman scale [in Japanese].
Okawara, C., Inaba, M., Makabe, S., Fukuda, Mo, Nozaki, M., Abe, M., and Kakai, H. (October 30‐31, 2021). Examples of achieving“integration” in mixed methods research: A mixed methods analysis using quantitative text mining and qualitative contentanalysis [in Japanese]. Paper presented at the 2021 MMIRA Asia Regional Conference.
Teddlie, C., & Tashakkori, A. (2009). The foundations of mixed methods research: Integrating quantitative and qualitative
approaches in the social and behavioral sciences. Sage

Thank you for yourkind attention!

Any questions?
You can find me at:hkakai@aoyamagakuin.jp

34© Hisako Kakai© Hisako Kakai