**Optimal Experimental Design for Binary Choice Experiments**

Barbara J. Kanninen

Hubert H. Humphrey Institute of Public Affairs

University of Minnesota

June, 1998

[Please send correspondence to:]{.underline}

2431 N. Nottingham St.

Arlington, VA 22207

Phone: (703) 536-6949

FAX: (703) 536-6905

e-mail: barbkann@aol.com

This research has been funded by NSF grant: NSF/SBR-9613045.

**Optimal Experimental Design for Binary Choice Experiments**

**Abstract**

This paper derives D-optimal designs for binary choice experiments with
multiple attributes using the attribute *levels* as design parameters.
The optimal solutions are shown to closely resemble traditional 2^m^
fractional factorial arrays with all but one attribute being placed at
their extreme points and the final attribute being used to balance the
response probabilities.

**1. Introduction**

To assess the values of market or nonmarket goods, or specific
attributes of those goods, researchers often apply experimental choice
techniques that allow them to explore preferences for real or
hypothetical goods or services (Carson et al. 1994). In a choice
experiment, consumers are presented with a set of scenarios that differ
by the levels of a series of attributes, including price, and are asked
to rank the alternative scenarios, or choose their most preferred.
Researchers can use the consumers' responses to estimate a model of
choice behavior that allows estimation of separate marginal values for
each attribute, or total values for any particular scenarios or goods as
described by a specific set of attribute levels. Researchers can also
estimate the marginal rates of substitution or trade-offs respondents
are willing to make between any two attributes.

Compared to most approaches for collecting microeconomic data, a unique
feature of the experimental choice process is that researchers have the
opportunity to specifically design part of their data by choosing the
attributes and attribute levels to be included in the choice sets.
Traditionally, attribute levels are specified in advance of the design
stage, based on the researcher's best judgment or project
specifications, and choice sets are designed using randomized, cyclical
or foldover procedures, based on the full or fractional factorial
designs generated by the array of specified attribute levels (Louviere
1988).

A number of recent papers have searched for, or derived, *efficient*
experimental choice designs (Bunch, Louviere and Anderson 1994, Kuhfeld,
Tobias and Garratt 1994, Lazari and Anderson 1994, Huber and Zwerina
1996), that is designs that increase the data's statistical information
in a sense defined by a design criterion. The most commonly used design
criterion has been D-optimality, or the maximization of the determinant
of the Fisher information matrix. Generally, however, these papers have
failed to obtain fully optimal designs because of restrictions,
consistent with the traditional experimental choice design literature,
placed on the number of attribute levels and/or the attribute levels
themselves prior to the design process. To obtain full design
optimality, these two factors must be design parameters, free to
optimize the design criterion.

In this paper, D-optimal designs will be derived for binary choice
experiments. An important feature of the approach taken here is that all
attributes are assumed, a priori, to be continuous variables that can be
bounded from above and below. By solving for the optimal designs over a
continuous domain, the number of attribute levels and the levels
themselves become design parameters. This approach is consistent with
the optimal design literature for dose-response models (Abdelbasit and
Plackett 1980, Minkin 1987, Wu 1988) and the more recent literature on
optimal design for contingent valuation experiments (Alberini 1995,
Alberini and Carson 1993, Kanninen 1993a and 1993b).

It turns out that the D-optimal solutions require continuity of only one
attribute. All other attributes are optimized by allocating their levels
according to a 2^m^ main effects factorial design with the two levels
being set at their upper and lower bounds. Even under restrictive
circumstances where attribute levels are fixed in advance of the design
stage, the requirement that one attribute be continuous can be easily
satisfied through the price attribute which should be able to take any
value that some portion of the experimental respondents might consider
reasonable for a particular choice being offered.

It also turns out that two features of the efficient design literature
mentioned above are features of the optimal designs: 1) "utility
balance" where choice sets are generated so that the average consumer is
indifferent among the choices (Huber and Zwerina 1996), and 2) placement
of the design points at "the corners of the design space" consistent
with optimal design for the linear model (Kuhfeld et al. 1994). The
specific solutions depend on the number of attributes in the problem,
with the influence of utility balance increasing as the number of
attributes increase. Interestingly, utility balance provides a complete
solution for the special (but impractical) case where the number of
attributes in the model is infinite.

This paper is organized as follows: first, the model and traditional
experimental design methodology for binary choice experiments are
discussed. Then a progression of optimal design solutions are presented
beginning with the one variable linear model and moving to the
multivariate linear model, one variable binary choice model and
multivariate binary choice model. This progression is presented to
clarify where and how the particular elements of the optimal design
solutions for binary choice experiments emerge. The paper concludes with
final comments and suggestions for further research.

**2. The Logit Model for Binary Choice Experiments**

The utility-theoretic approach to modeling discrete choices was
developed by McFadden (1974) and is discussed in detail by Ben-Akiva and
Lerman (1985). When offered a choice between two scenarios or goods,
*j*={0,1}, that differ only by a particular set of *m* attributes,
designated *x~i~^j^ = {x~1i~^j^ , x~2i~^j^ , \..., x~mi~^j^* }, a
consumer, *i*, will choose the good that offers him or her the greatest
utility. Specifying utility to be linear with a fixed component
^j^+*x~i~ ^j\ ^*and an additive random component ~i~^j^ that follows
an extreme value distribution, the probability that consumer *i* prefers
choice 1 to choice 0 is:

$P(\theta_{i}) = \frac{\text{Exp}(\theta_{i})}{1 + \text{Exp}(\theta_{i})}$
(1)

where:

$\ \theta_{i} = \alpha + \beta_{1}(x_{1i}^{1} - x_{1i}^{0}) + \beta_{2}(x_{2i}^{1} - x_{2i}^{0}) + \ldots + \beta_{m}(x_{\text{mi}}^{1} - x_{\text{mi}}^{0})$
(2)

and  = ^1^-^0^. It should be noted that this model is a "main
effects" model; that is, it assumes that the interactive effects among
the various combinations of attributes are negligible. [^1]

Letting *y~i~* equal 1 when consumer *i* prefers choice 1 and 0 when he
or she prefers choice 0, the log-likelihood function is:

$\text{log} - L(y_{i};\theta_{i}) = \sum_{i = 1}^{n}{y_{i}\text{log}P(\theta_{i})} + (1 - y_{i})\text{log}(1 - P(\theta_{i}))$
(3)

An important aspect of the design problem is that the log-likelihood
function is a function only of the difference between the two attribute
level vectors, *x~i~^1^ - x~i~^0\ ^*. This simplifies the design problem
as it is only this difference that must be specified for each attribute
(Bunch et al., 1994). For notational convenience, in the remainder of
the paper, let *x~i~ = x~i~^1^ - x~i~^0^*. Further, let *x~i~* be
continuous and bounded: *x~i~ * \[-1, 1,\]. These bounds are chosen,
without loss of generality, to allow *x* to correspond with the {-1,1}
notation often used in the experimental design literature.[^2]

Once maximum likelihood estimation is performed on the above model, a
number of analyses may be performed, for example, willingness to pay
(WTP) for choice 1 may be estimated as:

$\ W\hat{T}P_{i} = \frac{- (\hat{\alpha} + {\hat{\beta}}_{2}x_{2i} + {\hat{\beta}}_{3}x_{3i} + \ldots + {\hat{\beta}}_{m}x_{\text{mi}})}{\beta_{1}}$
(4)

where ~1~ is arbitrarily chosen to be the coefficient on the price
attribute. Further, the marginal rate of substitution of attribute *k*
for *l* may be estimated as:

$M\hat{R}S_{\text{kl}} = \frac{{\hat{\beta}}_{l}}{{\hat{\beta}}_{k}}$
(5)

The special case of a binary choice model with one attribute has been
extensively considered in the literature on discrete response contingent
valuation (CV). [^3] With discrete response CV, there is a choice
between a status quo situation and a single scenario with fixed
attribute levels offered at a particular price. Respondents are asked
whether or not they would be willing to pay the offered price for the
described scenario. This approach allows the researcher to estimate
willingness to pay for the alternative scenario but not for the
individual attributes associated with that scenario, as they do not vary
over the sample set. For this case, ~i~ is equal to +*x~i~* where
*x~i~* is the offered price.[^4]

CV experiments are performed principally to estimate willingness to
pay.[^5] Making no further assumptions on the model,[^6] mean or median
WTP can be estimated as:

$\ W\hat{T}P = \frac{- \hat{\alpha}}{\hat{\beta}}$ (6)

**3. Design of Binary Choice Experiments**

Louviere (1988) and Louviere and Woodworth (1983) summarize experimental
design issues for choice experiments.[^7] The principal consideration in
these discussions has been model identification. The approach assumes
the researcher has specified the attribute levels to be used in the
choice experiments in advance of the design stage. For fully specified
models, i.e. those that include all secondary and higher order
interactive effects, design involves the generation of a "full factorial
design" or one that includes all permutations of the specified attribute
levels. The full factorial design for the case of three attributes that
each take two levels, for example, includes 2^3^, or 8, different
observations. As the number of attributes or levels increase, the full
factorial increases exponentially and can quickly deplete budgetary
resources.

Table 1 shows a main effects design for the case of three attributes
that each take two levels. Because this is a reduced design compared to
the full factorial, it is referred to as a "fractional factorial
design." The limitations to using such a design are shown in Table 1:
each of the two-way interactive effects are confounded with a main
effect (e.g. the occurrences of *x~1~x~2~* are equivalent to the
occurrences of *x~3~*) and the three-way effect does not vary. Under the
assumption that these effects are negligible though, the main effects
model is identifiable.

The important distinction between the traditional approach to
experimental design and the one taken in this paper is that in the
traditional literature, the attribute levels are specified in advance,
presumably based on background research or pretest procedures (Louviere
1988). By starting under this premise, approaches to improving the
efficiency of designs are limited to search procedures that start with a
factorial design and make marginal changes to one or more choice sets.
The approach taken here does not specify the attribute levels upfront
but instead allows the optimality criteria to determine the optimal
levels, within reasonability bounds that are imposed a priori. The
approach also does not require model identification as an a priori
condition, but the solution does turn out to have that property.[^8]

Because only attribute level differences matter in the choice model,
optimal design of choice sets for binary choice experiments is
straightforward once design solutions are available: the optimal
attribute difference dictates the alternative attribute level once the
base attribute level is specified.

**4. Optimal Design of Choice Experiments**

In this section, D-optimal designs will be derived for linear and binary
choice experiments. By general consensus in the experimental choice
literature, D-optimality appears to be the preferred optimal design
criterion (Carson et al. 1994). D-optimality refers to the maximization
of the determinant of the Fisher information matrix, which is equivalent
to the minimization of the generalized variance of the parameters, or
the minimization of the joint confidence sphere surrounding the
parameter estimates. It is, in a sense, a criterion that seeks
statistical efficiency for the overall model.

A progession of optimal design problems will be addressed, beginning
with the one variable and multivariate linear models, then moving to
one-variable and multivariate binary choice models. This progression
allows a systematic demonstration of the emergence of the two influences
on D-optimal designs for binary choice models: utility balance and
placement of the design solutions at the corners of the design space.

*D-optimality for Ordinary Least Squares*

[Case 1: One independent variable]{.underline}

Consider the one variable linear model where consumers provide utility
scores or rankings, *y~i~*, for each choice:

$y_{i} = \alpha + {\beta x}_{i} + \varepsilon_{i}$ (7)

for *i* = {1,\...,*n*}. The one independent variable, price for example,
is assumed to be continuous and bounded: *x~i~*  \[-1, 1\]. The Fisher
information matrix is:

$I = \frac{1}{\sigma^{2}}\begin{pmatrix}
n & \sum_{}^{}x_{i} \\
\sum_{}^{}x_{i} & \sum_{}^{}x_{i}^{2}
\end{pmatrix}$ (8)

and the determinant is:

$\text{Det}(I) = \frac{1}{\sigma^{2}}\sum_{i = 1}^{n}{\sum_{j = i + 1}^{n}{(x_{i} - x_{j})^{2}}}$
(9)

The principle behind D-optimality for the linear model is easily
understood from equation (9): the determinant is a linear function of
the squared differences of all pairs of the *x* variable. First, it can
be seen that the maximum of the determinant can be solved for an
arbitrary pair (*i, j)* of the *x* variable; this means that the optimal
design solution is a two-point design that places half of the
observations at the solution for *x~i~* and half at the solution for
*x~j~*. Second, the pair {*x~i~*, *x~j~*}should differ from each other
as much as possible; in other words, one variable (let it be *x~i~*)
should be placed at the maximum possible value for *x* (+1, by
assumption) and the other (*x~j~*) should be placed at the minimum value
(-1, by assumption).

The optimal solution is intuitive in that it takes only two different
design points to draw a regression line and, given that those two points
will be observed with error, the regression line will most closely
approximate the true relationship between the regressor and independent
variable if the two design points are positioned as far apart as
possible. No other point along the domain of *x* is necessary for model
identification or statistically more informative from a D-optimal
perspective.

[Case 2: Multiple independent variables]{.underline}

For the general case of *m* variables:

$y_{i} = \alpha + \beta_{1}x_{1i} + \beta_{2}x_{2i} + \ldots + \beta_{m}x_{\text{mi}} + \varepsilon_{i}$
(10)

the (*m*+1) x (*m*+1) Fisher information matrix is:

$I = \frac{1}{\sigma^{2}}\begin{pmatrix}
n & \sum_{}^{}x_{1i} & \cdots & \sum_{}^{}x_{\text{mi}} \\
 & \sum_{}^{}x_{1i}^{2} & \cdots & \sum_{}^{}{x_{1i}x_{\text{mi}}} \\
 & & \ddots & \vdots \\
 & & & \sum_{}^{}x_{\text{mi}}^{2}
\end{pmatrix}$ (11)

and the determinant is:

$\text{Det}(I) = \ (\frac{1}{\sigma^{2}})\ |X'X|$ (12)

where *X* is a (*n* x *m*+1) matrix containing all vectors, *x~1~* \...
*x~m~* plus a vector of ones. Maximizing Det(*I*) is equivalent to
maximizing the determinant of X*'X*.

To understand the properties of the design solution, it is useful here
to consider the geometric properties of a determinant. In the case of a
matrix consisting of two, two-element vectors, the determinant is
equivalent to the area that results from completing the vectors into a
parallelogram. In the case of a multi-dimensional matrix, the same act
of completing the vectors results in a multi-dimensional
"parallelogram." If the matrix were nonorthogonal, the dimension of the
parallelogram would be less than the dimension of the matrix and
completion of the vectors would result in a partially collapsed cube.
Further, the area of the cube is maximized by maximizing the length of
each vector.

Two conclusions can be drawn from this: first that, to the extent
possible, the D-optimal solution will be *orthogonal* and second that
the design points will be at their extreme values -- on the endpoints of
the domain of *x*. Overall, the design solution will contain points that
are as far apart from each other as possible. For a main effects model,
any orthogonal main effects array that can be drawn from the full
factorial is optimal. Assuming the bounds of \[-1,1\] for all variables,
one optimal design solution for three continuous variables is exactly
the design presented in Table 1. In general, when an orthogonal design
exists for a particular number of attributes, the optimal design will be
that orthogonal design, modified to reflect the assumed upper and lower
bounds on the attribute levels. In a sense, the optimal solution reduces
*x~1~ \... x~m~* to a series of two-level attributes with the two levels
being the upper and lower bounds of each attribute.

*D-optimality for binary choice experiments*

[Case 1: One attribute:]{.underline}

The log-likelihood for the binary logit model is presented in equation
(3). The Fisher information matrix (dropping the  term for simplicity)
is:

$I = \begin{pmatrix}
\sum_{}^{}{P_{i}(1 - P_{i})} & \sum_{}^{}{P_{i}(1 - P_{i})x_{i}} \\
\sum_{}^{}{P_{i}(1 - P_{i})x_{i}} & \sum_{}^{}{P_{i}(1 - P_{i})x_{i}^{2}}
\end{pmatrix}$ (13)

and the determinant of this matrix is:

$\text{Det}(I) = \sum_{i = 1}^{n}{\sum_{j = i + 1}^{n}{P_{i}(1 - P_{i})P_{j}(1 - P_{j})(\theta_{i} - \theta_{j})^{2}}}$
(14)

As in the linear model with one parameter, this determinant is a
function of two design points and is therefore maximized with a
two-point design. The expression for this determinant is similar to that
in equation (9), except that this expression is weighted by the products
*P~i~*(1-*P*~i~) and *P~j~*(1-*P*~j~). Taken alone, these weights would
be maximized at *P~i~* = *P~j~* = .50.[^9] This illustrates the
influence of "utility balance" (Huber and Zwerina, 1996) in optimal
design for binary response models. With probabilities of .50, consumers
are, on average, perfectly indifferent between the two choices offered.
On the other hand, the squared difference term would be maximized by
design points placed at their extreme limits: where *P~i~* and *P~j~*
are closer to 0 or 1. This influence is just the opposite of utility
balance: with probabilities of 0 or 1, consumers prefer one choice over
the other100% of the time. The optimal solution can be derived
numerically and is a compromise between these two influences: {*~i~^\*^
,* *~j~*^\*^} = {-1.5434, +1.5434}, a symmetric design at the 18^th^
and 82^nd^ percentiles of the underlying response function.[^10]

To generate the price offers associated with this design solution, the
researcher can solve for *x~i~* = (*~i~*^\*^,- )/ and *x~j~* =
(*~j~^\*^* - )/, or, more directly, determine the levels of *x~i~*
and *x~j~* that would give *P*(*~i~*) = .18 and *P*(*~j~*) = .82. To
implement this design solution, the research must therefore know, or be
able to approximate, the underlying model. In practice, researchers
generally have some knowledge of the underlying model, based on focus
group or pretest information, before conducting their final version of
the survey. Kanninen (1993b) has shown that a sequential approach to
conducting CV surveys can substantially improve the information
available to the researcher and the efficiency of the ultimate estimates
obtained.[^11]

[Case 2: Multiple attributes]{.underline}

The Fisher information matrix for the case of multiple attributes is:

$I = \begin{pmatrix}
\sum_{}^{}w_{i} & \sum_{}^{}w_{i}x_{1i} & \cdots & \sum_{}^{}{w_{i}x_{\text{mi}}} \\
 & \sum_{}^{}w_{i}x_{1i}^{2} & \cdots & \sum_{}^{}w_{i}x_{1}x_{\text{mi}} \\
 & & \ddots & \vdots \\
 & & & \sum_{}^{}w_{i}x_{\text{mi}}^{2}
\end{pmatrix}$ (15)

where *w~i~* = *P~i~*(1-*P~i~*). The determinant (see Appendix) is:

$\underset{n}{\text{Det}(I) = \sum_{}^{n}}$ (16)

where the sum is over all permutations (labeled *i, j,\... k*) of the
*n* observations taken *m*+1 at a time and *X^\*i^~,j,\...k~* is an
(*m*+1) x (*m*+1) matrix composed only of those observations.

To maximize equation (16) it is useful to construct a reparameterization
of the problem. Without loss of generality, let ~1~ =
+~1~*x~1~+*~2~*x*~2~+\...*+*~m~*x*~m~ and ~j~ = ~j~*x*~j~ for *j*
= {2,\...,*m*}. It can be shown (see Appendix) that equation (16) can be
expressed as:

$\underset{n}{\text{Det}(I) = \left( \frac{1}{\beta_{1}\beta_{2}\text{.}\text{.}\text{.}\beta_{m}} \right)^{2}\sum_{}^{n}}$
(17)

Where, again, the sum is over all permutations (*i, j,\... k*) of the
*n* observations taken *m*+1 at a time and ^\*^*~i,j,\...k~* is an
(*m*+1) x (*m*+1) matrix composed only of those same observations on the
's. Note that, once again, because the determinant is a sum of
functionally equivalent terms, the optimal solution can be derived for
one set of *m* observations, arbitrarily labeled, *i, j,\..., k.*

What is convenient about this formulation is that ~2~ through ~m~
appear only in the determinant. The expressions *w~i~, w~j~, \..., w~k~*
are functions only of ~1~. With this separation, the maximization
problem can be solved in two stages: first, maximizing equation (17)
with respect to ~2~ through ~m~ for an arbitrary ~1;~ then plugging
these solutions into equation (17) and maximizing with respect to ~1~.

The first stage of the problem, maximizing with respect to ~2~ through
~m~, returns us to the same problem found in case 2 of the linear
model. To maximize the determinant, ~2~ \... ~m~ (and *x~2~ \...
x~m~*) should be orthogonal and contain values as large in absolute
value terms as possible. The solution for the design of these *m*-1
attribute vectors is therefore to set them to their extreme limits
according to *m*-1 arbitrarily chosen lines of the familiar, 2^m^
orthogonal main effects design.

Recall that under a choice framework, *x~2~ \... x~m~* refers to
attribute level differences. To maximize these differences, not only is
an attribute level placed at one of its extreme points but the level of
the same attribute in the alternative choice is placed at its opposite
extreme. When the design calls for the level of one attribute in the
base choice to be +1, the level of the same attribute in the alternative
choice should be placed at --1, and vice versa.

An example solution for the case of three attributes can be drawn from
Table 1. Under the assumption that attributes 2 and 3 are bounded by --1
below and +1 above, the optimal solutions for these two attributes are
any two columns of the main effects design.

Once the solutions for *m*-1 attributes have been established, the
second stage of the maximization problem, maximizing with respect to
~1~, is qualitatively similar to the problem of optimal design for a
binary choice model with one variable. The solution must balance the two
competing forces: the determinant alone would be maximized by setting
the design points at their extremes, where probabilities go to 0 or 1;
and the *P~i~(1-P~i~)* are maximized in the middle range, where *P~i~* =
.50.

Taking the first order conditions for an arbitrary design point, ~1j~,
gives:

$\frac{(1 - e^{\theta_{1j}})}{(1 + e^{\theta_{1j}})} = - 2\frac{\left| \Theta_{1j}^{+} \right|}{|\Theta|}$
(18)

where ~1j~^+^ represents the (1,j) cofactor of . The optimal solutions
for ~j~ can be derived numerically.[^12] Table 2 displays the D-optimal
results for three, seven and fifteen attributes in addition to the one
attribute case discussed above. Note that the constant term has
essentially been transformed for each element of ~1.~ Taking the first
two columns of the main effects design in Table 1 as the solution for
attributes 1 and 2 in a three-attribute design problem, the constant
terms for ~1~ are: { -~1~ - ~2~,  -~1~ + ~2~,  +~1~ - ~2~, 
+~1~ + ~2~}.

The designs are symmetrical, two-point designs with half the
observations placed at a single percentile point on each side of the
distribution. More importantly, the designs follow the course of the
standard, orthogonal main effects design, placing ~1~ on the lower part
of the distribution when the standard main effects design calls for a
--1, and vice versa when the design calls for a +1. This feature of the
solution guarantees that all main effects parameters will be
identifiable.

What is particularly pleasing about these design solutions is how
closely they resemble standard 2^m^ fractional factorial designs. The
optimal solutions for all attributes but one follow the 2^m^ main
effects orthogonal design exactly, modified to accommodate the assumed
upper and lower bounds of the attribute levels. The final attribute,
~1~, is used to balance choice sets to achieve certain response rates,
depending on the number of attributes in the experiment.

Again, to use the optimal design results in practice, researchers must
know or guess the underlying model. It is important to point out,
however, that it is not necessary to know or guess the exact values of
all parameters. Instead, once the first *m*-1 attributes are specified
from the factorial array, researchers must be have a sense of how to set
the final attribute so that the appropriate response rates will be
obtained.

**5. Optimal Design for an Infinite Number of Attributes**

It is interesting to note that the placement of the design point
solutions for ~1~ in the previous section appear to converge slowly
toward the center of the distribution as the number of attributes
increase. Although researchers would be unlikely to increase the number
of attributes to this extent, it is of intellectual interest to
investigate the path that the design solutions follow as the number of
attributes increase toward infinity.

First, note that for any *m*, the solutions for ~2~ \... ~m~ follow
the same pattern as above, being drawn from the main effects factorial
design. Summing the first order conditions (equation 18) over all design
points ~1j~ and multiplying each side by ~1j~ gives:

$\sum_{j = 1}^{m}{\theta_{1j}\frac{(1 - e^{\theta_{1j}})}{(1 + e^{\theta_{1j}})}} = - 2\sum_{j = 1}^{m}{\theta_{1j}\frac{\left| \Theta_{1j}^{+} \right|}{|\Theta|}}$
(19)

The sum on the right hand side is equal to 1. The left hand side of this
condition is symmetric and peaks at zero. Using the fact that the
absolute values of the ~j~'s are equal, we can re-write equation (19)
as:

$\ \theta_{j}\frac{(1 - e^{\theta_{j}})}{(1 + e^{\theta_{j}})} = - \frac{2}{m}$
(20)

and as *m* goes toward infinity, the design solution, ~j~, tends toward
zero. Since *P*(0) = .5, the solution as the number of attributes
increases tends toward a solution of complete utility balance.

This result helps clarify the workings of the optimal design problem. As
the number of optimally placed attributes increases, the determinant in
equation (16) increases exponentially. The *w*'s in the expression then
begin to take on more importance for overall maximization.

**6. Concluding Comments**

This paper has derived D-optimal designs for binary choice experiments
with multiple attributes and demonstrated the continuance of certain
optimal design principles across the spectrum of linear and binary
choice models. With the linear model, D-optimality calls for placement
of the design points at the extreme corners of the design space. With
binary choice, the concept of utility balance appears as a competing
force and begins to dominate the optimal design solutions as the number
of attributes increase. In the limit, where the number of attributes is
infinite, the D-optimal solution dictates complete utility balance.

What is most interesting about the design results presented here is how
closely they resemble traditional experimental designs. The optimal
solutions for all attributes but one follow the 2^m^ main effects
orthogonal design exactly, modified only to accommodate the assumed
upper and lower bounds of the attribute levels. The optimal solutions
for these attributes basically transform them into two-level attributes.
The final attribute, arbitrarily chosen but recommended here to be
price, is used to balance choice sets to achieve certain response rates.
The use of this final attribute to balance response probabilities is the
primary innovation of this paper.

There are a number of possible directions for future research in the
area of optimal design for experimental choice. First, the results
presented here should be extended to the cases of larger choice sets and
the presence of higher order, or cross-effects. Both extensions
introduce levels of complexity to the information matrix that appear to
make optimization extremely cumbersome.

Second, the results should be compared to alternative optimal design
criteria, such as C-, A- or E-optimality. D-optimality seems to be
favored as an optimization criterion because it is nonspecific; that is,
it addresses the overall model rather than any particular parameters or
aspects of the model. Researchers, however, generally have specific
goals when conducting experiments and optimality criteria can be
developed that more directly target these specific goals.

**Appendix**

To demonstrate the equivalence of equations (16) and (17), two
properties of determinants are used:

1\) The determinant of a matrix is linear in each row, implying:

i\)

$\left| \begin{matrix}
 - & a_{1} & - \\
 - & a_{2} & - \\
 - & a_{3} & - 
\end{matrix} \right| + \left| \begin{matrix}
 - & b_{1} & - \\
 - & a_{2} & - \\
 - & a_{3} & - 
\end{matrix} \right| = \left| \begin{matrix}
 - & a_{1} + b_{1} & - \\
 - & a_{2} & - \\
 - & a_{3} & - 
\end{matrix} \right|$

ii\)

$\left| \begin{matrix}
 - & \text{ka}_{1} & - \\
 - & a_{2} & - \\
 - & a_{3} & - 
\end{matrix} \right| = k\left| \begin{matrix}
 - & a_{1} & - \\
 - & a_{2} & - \\
 - & a_{3} & - 
\end{matrix} \right|$

2\) Adding a multiple of one row to another row leaves the determinant
unchanged:

$\left| \begin{matrix}
 - & a_{1} & - \\
 - & a_{2} & - \\
 - & a_{3} & - 
\end{matrix} \right| = \left| \begin{matrix}
 - & a_{1} + \text{ka}_{2} & - \\
 - & a_{2} & - \\
 - & a_{3} & - 
\end{matrix} \right|$

[Let:]{.underline}

$I = \begin{pmatrix}
\sum_{}^{}w_{i} & \sum_{}^{}w_{i}x_{1i} & \cdots & \sum_{}^{}{w_{i}x_{\text{mi}}} \\
 & \sum_{}^{}w_{i}x_{1i}^{2} & \cdots & \sum_{}^{}w_{i}x_{1i}x_{\text{mi}} \\
 & & \ddots & \vdots \\
 & & & \sum_{}^{}w_{i}x_{\text{mi}}^{2}
\end{pmatrix}$

where *w~i~* = *P~i~*(1-*P~i~*)

[Show:]{.underline}

$\underset{n}{\text{Det}(I) = \left( \frac{1}{\beta_{1}\beta_{2}\text{.}\text{.}\text{.}\beta_{m}} \right)^{2}\sum_{}^{n}}$

Using property 1i), Det(I) can be separated, one row at a time, into
parts as follows:

$\left| \begin{matrix}
\sum_{}^{}w_{i} & \sum_{}^{}w_{i}x_{1i} & \cdots & \sum_{}^{}{w_{i}x_{\text{mi}}} \\
 & \sum_{}^{}w_{i}x_{1i}^{2} & \cdots & \sum_{}^{}w_{i}x_{1i}x_{\text{im}} \\
 & & \ddots & \vdots \\
 & & & \sum_{}^{}w_{i}x_{\text{mi}}^{2}
\end{matrix} \right| = \left| \begin{matrix}
w_{1} & w_{1}x_{\text{11}} & \cdots & w_{1}x_{m1} \\
 & \sum_{}^{}w_{i}x_{1i}^{2} & \cdots & \sum_{}^{}w_{i}x_{1i}x_{\text{mi}} \\
 & & \ddots & \vdots \\
 & & & \sum_{}^{}w_{i}x_{\text{mi}}^{2}
\end{matrix} \right|$$+ \left| \begin{matrix}
w_{2} & w_{2}x_{\text{12}} & \cdots & w_{2}x_{m2} \\
 & \sum_{}^{}w_{i}x_{1i}^{2} & \cdots & \sum_{}^{}w_{i}x_{1i}x_{\text{mi}} \\
 & & \ddots & \vdots \\
 & & & \sum_{}^{}w_{i}x_{\text{mi}}^{2}
\end{matrix} \right| + \text{.}\text{.}\text{.} + \left| \begin{matrix}
w_{m} & w_{m}x_{1m} & \cdots & w_{m}x_{\text{mm}} \\
 & \sum_{}^{}w_{i}x_{1i}^{2} & \cdots & \sum_{}^{}w_{i}x_{1i}x_{\text{mi}} \\
 & & \ddots & \vdots \\
 & & & \sum_{}^{}w_{i}x_{\text{mi}}^{2}
\end{matrix} \right|$

Taking the first determinant in the equation above and separating the
second row into parts gives:

$\left| \begin{matrix}
w_{1} & w_{1}x_{\text{11}} & \cdots & w_{1}x_{m1} \\
 & \sum_{}^{}w_{i}x_{1i}^{2} & \cdots & \sum_{}^{}w_{i}x_{1i}x_{\text{mi}} \\
 & & \ddots & \vdots \\
 & & & \sum_{}^{}w_{i}x_{\text{mi}}^{2}
\end{matrix} \right|\  = \ \left| \begin{matrix}
w_{1} & w_{1}x_{\text{11}} & \cdots & w_{1}x_{m1} \\
w_{1}x_{\text{11}} & w_{1}x_{\text{11}}^{2} & \cdots & w_{1}x_{\text{11}}x_{m1} \\
 & & \ddots & \vdots \\
 & & & \sum_{}^{}w_{i}x_{\text{mi}}^{2}
\end{matrix} \right|$

$\left| \begin{matrix}
w_{1} & w_{1}x_{\text{11}} & \cdots & w_{1}x_{m1} \\
w_{2}x_{\text{12}} & w_{2}x_{\text{12}}^{2} & \cdots & w_{2}x_{\text{12}}x_{m2} \\
 & & \ddots & \vdots \\
 & & & \sum_{}^{}w_{i}x_{\text{mi}}^{2}
\end{matrix} \right| + \text{.}\text{.}\text{.} + \left| \begin{matrix}
w_{1} & w_{1}x_{\text{11}} & \cdots & w_{1}x_{m1} \\
w_{m}x_{1m} & w_{m}x_{1m}^{2} & \cdots & w_{m}x_{1m}x_{\text{mm}} \\
 & & \ddots & \vdots \\
 & & & \sum_{}^{}w_{i}x_{\text{mi}}^{2}
\end{matrix} \right|$

Note that the first term on the right hand side is equal to zero since
the second row, divided by *x~11~*, is equal to the first row, rendering
the determinant to be of less than full rank. So the separating process
produces nonzero determinants only when each row is drawn from a unique
observation . Continuing the process until all rows are fully separated
gives a set of $\begin{pmatrix}
n \\
m
\end{pmatrix}$! unique, nonzero determinants:

$$

$\underset{n}{\text{Det}(I) = \sum_{}^{n}}$

This sum contains $\begin{pmatrix}
n \\
m
\end{pmatrix}$! different combinations of observations (with each
combination appearing *m*! times). For any combination, the *m*!
determinants can be combined by summing within and across determinants
as follows. For every determinant, divide the second row by *x~1j~* and
add to the first row; divide the third row by *x~2k~* and add to the
first row, continuing until the first row is summed over all
observations contained in the matrix. Now for every determinant, there
is exactly one other determinant that is identical except for the second
row, and each of these pairs can be combined linearly. The second row
can now be completed by adding a linear combination of the remaining
rows as described for the first row. After the second rows are
completed, each remaining determinant has exactly two others that are
identical except for the third row. The process continues until there is
a unique determinant representing each of the $\begin{pmatrix}
n \\
m
\end{pmatrix}$! combinations of the observations. Noting that each of
the rows is multiplied by a constant, *w~i~* and using property 1ii),
gives:

$\underset{n}{\text{Det}(I) = \sum_{}^{n}}$

where

$X_{i,j,\text{.}\text{.}\text{.}k}^{} = \left| \begin{matrix}
1 & 1 & \cdots & 1 \\
x_{1j} & x_{2j} & \cdots & x_{\text{mj}} \\
 & & \ddots & \vdots \\
x_{m1} & & & x_{\text{mk}}
\end{matrix} \right|$

Noting that, because *X~i,j,\...k~*^\*^ is a square matrix:
\|*X~i,j,\...k~*^\*^' *X~i,j,\...k~*^\*^\| = \|*X~i,j,\...k~*^\*^\|^2^.
It is straightforward to convert \|*X~i,j,\...k~*^\*^\| to \[1/(~1~\...
~m~)\] \|*~i,j,\...k~*^\*^\| by multiplying each row of
*X~i,j,\...k~*^\*^ by the appropriate component of the  vector to
convert to the ~j~ (*j*=2,\...,*m*) notation, and multiplying the first
row by  and adding the subsequent rows to it to convert that row to
~1~=+~1~x~1~ + \... + ~m~x~m.~ Because the matrix,
*~i,j,\...k~*^\*^, is square, \|*~i,j,\...k~*^\*^\|^2^ =
\|*~i,j,\...k~*^\*^' *~i,j,\...k~*^\*^\| and equation (17) is
achieved.

**Table 1: Fractional Factorial (Main Effects) Design for Three
Two-Level Attributes**

  --------- --------- --------- -------------- --------- --------- -------------- -----------
  Main                          Two-way                            Three-way      
  Effects                       Interactions                       Interactions   

  x~1~      x~2~      x~3~      x~1~ x~2~      x~1~ x~3~ x~2~ x~3~                x~1~ x~2~
                                                                                  x~3~

  -1        -1        +1        +1             -1        -1                       +1

  -1        +1        -1        -1             +1        -1                       +1

  +1        -1        -1        -1             -1        +1                       +1

  +1        +1        +1        +1             +1        +1                       +1
  --------- --------- --------- -------------- --------- --------- -------------- -----------

**Table 2: D-Optimal Design for Binary Choice Experiment with Three
Attributes**

  --------- ----------------------- -------- -------- ---------- ----------
  *~1~*    *X~1~*                  *x~2~*   *X~3~*   P()       1-P()

  +1.04     (+1.04-+~2~+~3~)/    -1       -1       .74        .26
            ~1~                                                 

  -1.04     (-1.04-+~2~+~3~)/    +1       -1       .26        .74
            ~1~                                                 

  -1.04     (-1.04-+~2~+~3~)/    -1       +1       .26        .74
            ~1~                                                 

  +1.04     (+1.04-+~2~+~3~)/    +1       +1       .74        .26
            ~1~                                                 
  --------- ----------------------- -------- -------- ---------- ----------

**Table 3: D-optimal Designs for ~1~**

  ------------ --------------- ------------ --------- ----------- ---------
  Number of    Total Number of n/2 points   P()      n/2 points  P()
  Attributes   Design points   at ~1\ =~             at ~1~ =   
                               **^\*^**                           

  1            2               1.54         .18       -1.54       .82

  3            4               1.04         .26       -1.04       .74

  7            8               0.72         .33       -0.72       .67

  15           16              0.51         .38       -0.51       .62

                              0            .5        0           .5
  ------------ --------------- ------------ --------- ----------- ---------

\* These results must be placed into the appropriate choices along with
optimally placed ~2~ \... ~m~, according to the main effects
orthogonal design.

**References**

Abdelbasit, K.M. and R.L. Plackett, 1980,"Experimental Design for Binary
Data," *Journal of the American Statistical Association,* 87: 381,
90-98.

Alberini, A., 1995, "Optimal Designs for Discrete Choice Contingent
Valuation Surveys: Single-bound, Double-bound and Bivariate Models,"
*Journal of Environmental Economics and Management,* 28(3), 287-306.

Alberini, A. and R.T. Carson, 1993, "Efficient Threshold Values for
Binary Discrete Choice Contingent Valuation Surveys and Economic
Experiments," Resources for the Future Discussion Paper, Quality of the
Environment Division, Washington, D.C.

Anderson, D.A. and J.B. Wiley, 1992, "Efficient Choice Set Designs for
Estimating Cross-Effects Models," *Marketing Letters,* 3, 357-70.

Ben-Akiva, M. and S. Lerman, 1985, *Discrete Choice Analysis,* MIT
Press, Cambridge.

Bunch, D.S., J.J. Louviere and D. Anderson, 1994, "A Comparison of
Experimental Design Strategies for Multinomial Logit Models: The Case of
Generic Attributes," Working Paper, Graduate School of Management,
University of California, Davis.

Cameron, T.A., 1988, "A New Paradigm for Valuing Non-market Goods Using
Referendum Data: Maximum Likelihood Estimation by Censored Logistic
Regression," *Journal of Environmental Economics and Management,* 15,
355-79.

Carson, R., J. Louviere, D. Anderson, P. Arabie, D. Bunch, D. Hensher,
R. Johnson, W. Huhfeld, D. Steinberg, J. Swait, H. Timmermans, J. Wiley,
1994, "Experimental Analysis of Choice," *Marketing Letters,* 5, 351-67.

Hanemann, W.M., 1984, "Welfare Evaluations in Contingent Valuation
Experiments with Discrete Responses, *American Journal of Agricultural
Economics,* 66, 332-41.

Hanemann, M. and B. Kanninen, 1998, "The Statistical Analysis of
Discrete-Response CV Data," in *Valuing Environmental Preferences:
Theory and Practice of the Contingent Valuation Method in the U.S., E.U.
and Developing Countries,* I.J. Bateman and K.G. Willis, Eds., Oxford
University Press, Oxford.

Huber, J. and K. Zwerina, 1996, "The Importance of Utility Balance in
Efficient Choice Designs," *Journal of Marketing Research*, 307-17.

Kanninen, B.J., 1993a, "Design of Sequential Experiments for Contingent
Valuation Studies," *Journal of Environmental Economics and Management,*
25: S-1-11.

Kanninen, B.J., 1993b, "Optimal Experimental Design for Double-Bounded
Dichotomous Choice Contingent Valuation," *Land Economics,* 69(2):
138-46.

Kuhfeld, W.F., R.D. Tobias and M. Garratt, 1994, "Efficient Experimental
Design with Marketing Research Applications," *Journal of Marketing
Research,* 21, 545-57.

Lazari, A.G. and D.A. Anderson, 1994, "Designs of Discrete Choice
Experiments for Estimating Both Attribute and Availability Cross
Effects," *Journal of Marketing Research,* 31, 375-83.

Louviere, J.J, 1988, *Analyzing Decision Making: Metric Conjoint
Analysis,* Sage Publications, Inc., Newbury Park, CA.

Louviere, J.J. and G. Woodworth, 1983, "Design and Analysis of Simulated
Consumer Choice of Allocation Experiments: A Method Based on Aggregate
Data," *Journal of Marketing Research,* 20, 350-67.

Mathematica 3.0, 1996, Wolfram Research, Inc., Champaign, IL.

McFadden, D., 1974, "Conditional Logit Analysis of Qualitative Choice
Behavior," in *Frontiers in Econometrics,* Zarembka, P., ed., Academic
Press, New York, 105-42.

Minkin, S., 1987, "Optimal Designs for Binary Data," *Journal of the
American Statistical Association,* 82:400, 1098-1103.

Mitchell, R.C. and R.T. Carson, 1989, *Using Surveys to Value Public
Goods: The Contingent Valuation Method,* Resources for the Future,
Washington, D.C.

Winer, B.J., D.R. Brown and K.M. Michels, 1991, *Statistical Principles
in Experimental Design,* Third Edition, McGraw-Hill, Inc., New York.

Wu, C.F.J., 1988, "Optimal Design for Percentile Estimation of a Quantal
Response Curve," in *Optimal Design and Analysis of Experiments,* Y.
Dodge, V. Fedorov and H. Wynn, eds., Elsevier Science Publishers.

[^1]: Anderson and Wiley (1992) and Lazari and Anderson (1994) discuss
    optimal design when cross-effects are present.

[^2]: Obviously, for some attributes these bounds are inappropriate:
    price, for example, cannot be negative. The specific bounds chosen
    do not, however, affect the results that follow.

[^3]: See Mitchell and Carson (1989) for a summary of the CV method and
    Hanemann and Kanninen (1998) for a discussion of statistical issues
    and estimators.

[^4]: In many cases with CV, the status quo scenario describes a public
    good that is provided freely. Of course, if both scenarios involve a
    price, then *x~i~* represents the price difference.

[^5]: Hanemann (1984) developed a utility-theoretic framework for
    analyzing CV data with binary responses. Cameron (1988) developed a
    competing approach that bypasses the utility function and models WTP
    directly. For linear specifications, the two approaches are
    observationally equivalent.

[^6]: See Hanemann and Kanninen (1998) for a wide range of models and
    assumptions that can be made on the WTP distribution.

[^7]: The statistical underpinnings of experimental design are
    summarized thoroughly in Winer et al. (1991).

[^8]: Not all optimal design solutions allow model identification. The
    C-optimal design that minimizes the asymptotic variance of the
    median of a logit or probit model, for example, is a one-point
    design that does not allow identification of the two model
    parameters (Wu, 1988). The D-optimal criterion generally produces an
    information matrix that has full rank and so allows model
    identification.

[^9]: Placing all design points at the median value (where P=.50) is the
    optimal design solution when the criterion is to minimize the
    asymptotic variance of the .5 percentile, or median (Wu, 1988).

[^10]: This solution, first derived by Abdelbasit and Plackett (1980)
    and Minkin (1987), is correct only for the case of an even number of
    observations. Minkin (1987) discusses the case of an odd number of
    observations. In practice, with a large number of observations, the
    difference between the performance of the two types of solutions
    should be negligible.

[^11]: Kanninen (1993a) recommends conducting at least three,
    equal-sized waves of the experiment.

[^12]: The author minimized the negative of equation (17) using the
    "FindMinimum" command in Mathematica 3.0 (1996).