Centre for the Study of Choice ( CenSoC ) Working Paper No . 04-002 Complex Statistical Choice Models : Are the Assumptions True , and If Not , What Are the Consequences ?

We discuss reasons why most published empirical choice modeling results in health economics and related areas are likely to be content dependent and not generalisable. That discussion leads to logical reasons for pooling various sources of data to improve statistical efficiency, enable identification and enhance the potential to generalise results. We then review several assumptions associated with choice models, such as constant variance assumptions for individuals, withinand between individual attribute scale commensurability and aggregation of utilities. We discuss the consequences of the failure of several key assumptions and we illustrate these consequences with several simple examples. A wide range of empirical results are reviewed that support the contention that many key assumptions are likely to be correct. We suggest three ways forward to try to overcome the problems identified. Introduction The purpose of this paper is to review some current work in complex statistical choice models, raise a number of issues about this stream of research in particular and choice modeling in general and discuss some alternatives to current practice. As my background is in psychology, not economics, the focus of this paper will be on human behaviour, not statistics per se. That is, my perspective is that the overarching objectives of the scientific study of choice behaviour should be to understand and predict choices, not just fit statistical models to choice data from real markets or experiments. That is not to say that statistics is not important or that statistical models are not important, but it is to say that statistics does not provide theories of human behaviour; statistics provides tools that can be used to implement and test theories of human behaviour. So, in my view a primary objective of social science should be to develop and/or apply theory that explains particular aspects of human behaviour and/or develop empirical generalizations that support theory development to broaden our understanding of human behaviour. Thus, statistical models applied in social science research should be derived from behavioural theory instead of “statistical tools” that appear to fit types of data available for analysis (although I recognize that we sometimes need to and will do both). Thus, complex discrete multivariate and related statistical models have been developed in statistics, biostatistics, Bayesian statistics and econometrics, but the fact that they exist does not necessarily mean that we should use them to analyse and interpret choice data from experiments or real markets. For example, the fact that one CAN estimate random effects models does not mean that one SHOULD do this or would even want to do this as we later explain in more detail. All statistical models require assumptions about (human) behavioural processes that may or may not in fact be satisfied in any real data source or for any sample of humans. When we impose a statistical model on a source of data and estimate the parameters of that model from the data, it does not follow that the results are correct, or even “good”. As we discuss later in this paper, all models require maintained assumptions, and as scientists, we should constantly ask whether these assumptions are satisfied in any empirical work we do. For example, we later note that the validity and interpretation of almost all published statistical choice models that posit random effects rely on the truth of assumptions about individual error distributions that are rarely if ever discussed in the literature. The Challenge of Complexity in Real Markets The general problem of developing theory and deriving models of consumer behaviour from theory is challenging due to the complexity of real market behaviour(s). For example, consider Figure 1, which represents a stylized abstraction of the structure of real markets at any point in time reflected in a distribution of consumers in particular product markets. That is, for any good/service at some point in time, there is a distribution of consumers in the states in Figure 1. The states in Figure 1 reflect the reality that consumers must be aware that goods exist before they can be chosen, consumers must have some minimal level of interest in a category of goods before choosing one of them, and consumers must take into account constraints before deciding whether to choose. For consumers who pass through these stages into later stages, this hierarchical process eventuates in decision strategies or utility functions that lead to formation of sets of goods that pass muster called “choice sets”. Once consumers form choice sets they make decisions about if and when to choose, and if choosing what to choose, how many to choose, etc. Following choice, they evaluate what they chose, and then reevaluate their level(s) of interest in the category or in particular offerings in the category. Figure 1: A hierarchy of consumer decisions Virtually all Random Utility Theory (RUT) based choice models in the literature have dealt only with the final stages of such processes, and only recently has dynamics been introduced (eg, Erdem and Keane 1996). Thus, it is worth noting that many factors like prices can enter a consumer’s decision process at several levels in this hierarchy. For example, price may have an effect at pre-awareness for some consumers if they never become aware of certain goods because they are too expensive (or perhaps too cheap); other goods may not create sufficient levels of interest to move consumers into that state because they are too expensive (or too cheap); and yet others may involve income, time or other constraints where price matters. Awareness Interest Constraints Preference formation Choice set formation Choose not to act or choose Choose to delay action or choice Choose to act or choose now Timing of choice