How to Enhance Qualitative Research Appraisal: Development of the Methodological Congruence Instrument

Researchers generally think of the research process as falling into three major phases: designing a study, conducting a study, and reporting on the study results. Within any of these three phases, it is likely that some form of appraisal of the research will take place. Following the design of a study, a dissertation committee, funding agency, or Institutional Review Board may conduct manuscript appraisal. In these cases, the appraisal has the purpose of determining one or more of the following: that the researcher has the knowledge to conduct the research, has outlined steps to ensure the research is ethical, and/or is conducting a study that aligns with a particular funding priority. Appraisal may also occur by advisors or reviewers to confirm that the audit trail supports the research findings. The most common form of research appraisal, however, occurs in the form of peer review by editorial boards and journal reviewers. While research appraisal may receive far less attention in the literature than designing, conducting, or reporting research, it is critically important to the dissemination of trustworthy findings that uphold methodological rigor.A variety of high quality instruments exist that support and guide the appraisal process in qualitative inquiry (Cooper, 2011). However, in our review of the literature related to qualitative research appraisal, we did not locate an instrument or rubric that offers detailed guidance on assessing methodological congruence for the five major qualitative research traditions-ethnography, grounded theory, phenomenology, narrative inquiry, and case study research. In this article, we describe our context as authors and the circumstances that led to the development of the appraisal instrument presented below, which we call the Methodological Congruence Instrument (MCI; see appendix). For the purposes of this article, methodological congruence is defined as a "fit" between research purpose, research question, methodology, data sources and types, and data analysis (Creswell, 2013). To provide some context for the elements and content of the MCI, we briefly review some of the characteristics of major qualitative research traditions and methodologies. In addition, we offer some discussion of how this appraisal instrument might be utilized by a variety of stakeholders, including editorial board members, teachers, students, and the researchers themselves.Context and BackgroundThe authors of this paper are qualitative researchers who came together to work on this project within the context of a course on appraising qualitative research offered in Nova Southeastern University's Qualitative Research Graduate Certificate Program (QRGP). The authors include students in the course (Alice, Annette, Bruce, and Cynthia), as well as the course instructor (Robin) and the course Teaching Assistant (Doles). The students are all experienced educators and researchers who enrolled in the QRGP to strengthen their qualitative research knowledge and skills. During the course, as the class discussed how to assess quality in qualitative research reports, one of the traits we identified as indicative of quality was that of congruence-consistency between tradition/model and procedures, as well as between tradition/model and reporting conventions.Annette raised the idea of developing a table to help track the characteristics associated with various methodologies, as a tool to support the appraisal process, and she drafted an initial version. She shared her idea with the class and invited anyone interested to join with her in developing this instrument further. Following the conclusion of the course, the authors of this paper worked together to do just that, and we share the product below (appendix). It was never our intent to create a standardized instrument that would dictate issues of congruence. The intent of the MCI is to assist novice researchers in their learning process, serve as a platform for discussion among mentors and students, and get people thinking about ethics and rigor in methodological congruence. …