Evidence in Management and Organizational Science : Assembling the Field ’ s Full Weight of Scientific Knowledge Through Syntheses

This chapter advocates the good scientific practice of systematic research syntheses in Management and Organizational Science (MOS). A research synthesis is the systematic accumulation, analysis and reflective interpretation of the full body of relevant empirical evidence related to a question. It is the * Corresponding author. Email: rousseau@andrew.cmu.edu 476 • The Academy of Management Annals critical first step in effective use of scientific evidence. Synthesis is not a conventional literature review. Literature reviews are often position papers, cherrypicking studies to advocate a point of view. Instead, syntheses systematically identify where research findings are clear (and where they aren’t), a key first step to establishing the conclusions science supports. Syntheses are also important for identifying contested findings and productive lines for future research. Uses of MOS evidence, that is, the motives for undertaking a research synthesis include scientific discovery and explanation, improved management practice guidelines, and formulating public policy. We identify six criteria for establishing the evidentiary value of a body of primary studies in MOS. We then pinpoint the stumbling blocks currently keeping the field from making effective use of its ever-expanding base of empirical studies. Finally, this chapter outlines (a) an approach to research synthesis suitable to the domain of MOS; and (b) supporting practices to make synthesis a collective MOS project. Evidence in Management and Organizational Science It is the nature of the object that determines the form of its possible science. (Bhaskar, 1998, p. 3) Uncertain knowledge is better than ignorance. (Mitchell, 2000, p. 9) This chapter is motivated by the failure of Management and Organizational Science (MOS) to date to make full effective use of its available research evidence. Failure to make effective use of scientific evidence is a problem both management scholars and practitioners face. Effective use of evidence, as we employ the term here, means to assemble and interpret a body of primary studies relevant to a question of fact, and then take appropriate action based upon the conclusions drawn. For science, appropriate action might be to direct subsequent research efforts elsewhere if the science is clear, or to recommend a new tact if findings are inconclusive. For practice, appropriate action might begin with a summary of key findings to share with educators, thought leaders, consultants, and the broader practice community. Unfortunately, bodies of evidence in MOS are seldom assembled or interpreted in the systematic fashion needed to permit their confident use. A systematic review of the full body of evidence is the key first step in formulating a science-based conclusion. As a consequence at present, neither the MOS scholar nor the practitioner can readily claim to be well-informed. This lapse has many causes. Two are central to our failure to use MOS evidence well: (1) overvaluing novelty to the detriment of accumulating convergent findings; and (2) the general absence of systematic research syntheses. These two causes are intertwined in that, as we shall show, use of research syntheses ties closely with how a field gauges the value of its research. This chapter’s subject, the systematic research synthesis, is not to be confused Evidence in Management and Organizational Science • 477 with a conventional literature review, its less systematic, non-representative counterpart. Systematic research syntheses assemble, analyze and interpret a comprehensive body of evidence in a highly reflective fashion according to six evidentiary criteria we describe. The why, what, and how of research synthesis in MOS is this chapter’s focus. The explosion of management research since World War II has created knowledge products at a rate far outpacing our current capacity for recall, sense-making, and use. In all likelihood, MOS’s knowledge base will continue to expand. We estimate over 200 peer-reviewed journals currently publish MOS research. These diverse outlets reflect the fact that MOS is not a discipline; it is an area of inter-related research activities cutting across numerous disciplines and subfields. The area’s expansion translates into a body of knowledge that is increasingly fragmented (Rousseau, 1997), transdisciplinary (Whitley, 2000), and interdependent with advancements in other social sciences (Tranfield, Denyer & Smart, 2003). The complicated state of MOS research makes it tough to know what we know, especially as specialization spawns research communities that often don’t and sometimes can’t talk with each other.