REGULATORY ERRORS UNDER TWO-SIDED UNCERTAINTY: OR, THE POLITICAL ECONOMY OF VIOXX 1

How do the errors of regulators – approving bad products, or rejecting good ones – depend upon the submission strategies and characteristics of submitting private entities or firms? We develop a model of approval regulation in which both firm and regulator are uncertain about the underlying quality of a product, but where the firm is better informed than the regulator. The model predicts that the commission of regulatory error depends heavily upon the induced submission strategies of firms, and in particular on the amount of time that firms take to experiment with their products before submitting them. Specifically, our analysis predicts that when experimentation is short and experimental costs are high, Type I errors (approving bad products) should be disproportionately associated with products submitted by firms with lower experimentation costs (larger firms), and Type II errors (rejecting good products) should be concentrated among products submitted by smaller firms. However, when experimentation is long and experimental costs are low, Type I errors should be concentrated among small firms, while Type II errors will be concentrated among larger firms. Under all conditions Type I errors should be increasing in the cost of regulatory submission, and the inverse should be true for Type II errors. We test hypotheses from the model in a statistical analysis of regulatory errors committed by the FDA in the approval of new pharmaceutical products. Using two different sets of measures for Type I errors, we find consistent support for the predictions of the model. We also find modest evidence that recent predictable reductions in approval times have been associated with a reduced rate of Type I error by the FDA, as our model would predict. The model also sheds new light on more (in)famous cases of regulatory error, including the FDA’s delay in approving beta-blockers in the 1970s and the recent Vioxx episode. Bureaucratic and regulatory organizations commit numerous errors—some trivial, some catastrophic. The standard metaphor used to understand these mistakes is that of “Type I” and “Type II” errors, borrowed from statistical decision theory. An agent commits a Type I error (“commission”) when she rejects a null (default) hypothesis when it is true. She commits a Type II error (“omission”) when she affirms a null hypothesis that is in fact false. So understood, bureaucratic and regulatory errors involve crucial questions of politics, policy and institutions. Welfare agencies may fail to issue a check to a deserving beneficiary (Type II), or pay someone who has not qualified for benefits (Type I). Space agencies may launch rockets that are doomed to destruction (I), or ground launches that would have gone off smoothly (II). Family service agencies may dissolve reasonably healthy families (I), or fail to remove children from abusive homes (II). Safety regulators may approve products that should never have been marketed (I), or reject or delay products that are socially beneficial (II). A grant-making agency may reject an excellent project (II) while funding a mediocre one (I). Despite the salience of bureaucratic and regulatory error, there remain numerous questions for which students of bureaucracy and regulation lack general insight.2 • Why do some of the many decisions made by agencies result in error and others not? What circumstances of a case or features of bureaucratic procedure lead to error? • What causes agencies to commit certain kinds errors rather than others (i.e., Type I versus Type II)? • Do errors depend in part upon the portfolio of cases that the agency receives? Does this portfolio depend in part upon the agency’s own decision making and the way that private entities react to it? In other words, might the relationship between agency decision making and firm submissions be simultaneous, with important consequences for the determination of error? The problems that occupy bureaucrats and regulators only rarely come to them exogenously. Instead, cases are actively brought to the agency by private entities or other public actors through a process of submission (explicit or implicit). Citizens may bring complaints or charges to a prosecutor or enforcement agency. Firms may submit license or new product applications. Researchers may submit patents or new research ideas embedded within grant applications. Our aim in this paper is to consider – theoretically via analysis of a game-theoretic model and empirically via analysis All of these examples presume agreement on the null hypothesis, and that the truth can be known ex post. Ours is far from the first treatment of these questions. Below we discuss two sets of theories – redundancy and bounded rationality – that have attempted to shed light upon these problems.