Evaluating TQM: The Case for a Theory Driven Approach

Speaking to General Accounting Office (GAO) evaluators in 1991, Total Quality Management (TQM) guru Joseph Juran lamented how woefully inadequate our understanding was of his progeny's impact. Wrote Juran (1991, p. 51): "As far as measuring the [TQM] results that have been achieved, there's a big information vacuum out there. Hardly anything useful is going on as far as evaluating results. " Unfortunately, little has changed in the interim to negate Juran's observation substantially. Indeed, a paucity of systematic and rigorous evaluative efforts still exists, thus prompting accusations that TQM is little more than a "religious doctrine" that admonishes the unwashed to "just do it, in the blind hope that results will follow" (Sims et al., 1992, p. 138). Although Juran was largely concerned that flawed or nonexistent evaluations of TQM would limit its application in both the private and public sectors, more pedestrian concerns ought to make systematic and rigorous evaluations of TQM interventions attractive to public managers and scholars serious about improving quality. First, TQM interventions in the public sector can be costly endeavors. For example, in one small city with 700 employees, initial TQM training cost $75,000, with training costs over the ensuing four years running $30,000 annually (Walters, 1992). Second, TQM applications are increasingly viewed with skepticism in the private sector, a skepticism influenced by anecdotal and empirical analyses stressing its decidedly mixed results (Mathews and Katel, 1992; Dumas, 1989; Clemmer, 1991; Mathews, 1993). Despite proponents' warnings not to expect short-term cost savings, many CEOs have grown cynical upon seeing little change in their companies earnings statements. What is more, Mathews (1993, p. H1) reports that private sector consulting firms in the 990s are thriving on the "TQM repair work" spawned by TQM "miscarriages and abortions" crafted in the 1980s. Third, the wholesale adoption of "unreformed" TQM techniques in the public sector has been questioned widely Swiss, 1992; Kronenberg and Loeffler, 1991; Cohen and Brand, 1993; Durant and Wilson, 1993). Programs, for instance, with dense contextual goals (Durant and Wilson, 1993) or with more diffuse decision structures than those typically found in private businesses may inordinately slow the implementation of TQM, thus diminishing the likelihood of success (Juran, 1991). Finally, although TQM advocates typically counsel public agencies to "pick the lowest hanging fruit first" (i.e., to selectively apply TQM interventions in areas most likely to succeed), they tend to offer little advice as to how public managers might recognize these situations in practice or convincingly monitor their progress and assess their results. Arguably, only rigorous evaluation of TQM interventions in public agencies, as well as the testing of hypotheses culled from these experiences by researchers, can address such issues. How can public managers, program analysts, and scholars best approach TQM evaluations? We argue that TQM evaluations and research must become more theory grounded and contingency based than has historically been the case if practice, cumulative knowledge bases, or theory-building are to improve significantly. Explicitly delineated models of change processes, the assumptions underlying them, and the ways in which variables in the models are expected to interact to produce these outcomes must inform any serious attempt at TQM evaluation or theory building in the public sector. Moreover, lest these models, assumptions, and interaction patterns are eventually specified and progress toward their achievement is explicitly assessed, TQM's "half-life" in the public sector could be shortened appreciably. More precisely, ways must be found to identify, assess, and demonstrate short-term, nonmonetary progress toward both intermediate and ultimate TQM goals of cultural change and quality improvement. Otherwise, the private sector's emerging cynicism and disenchantment with TQM's inability to show short-term monetary benefits could spread rapidly to the public sector as well. …