# HYBRID APPROACH TO CHOICE-BASED CONJOINT ANALYSIS

Ondřej Vilikus*

## Introduction

Throughout more than forty years since its beginnings, progress in conjoint analysis has been driven by the effort to satisfy two contradictory requirements of practitioners in market research. To obtain useful models of consumer preferences on the market of our interest, we need to account for heterogeneous preferences in the population by modelling preferences of each individual respondent. Yet we are strongly limited by the amount of information that we are able to collect from each respondent before his fatigue causes diminishing validity of his answers. (It is well known that once a respondent gets bored by the repetitive nature of conjoint tasks, he either develops facilitating techniques in making his choices not reflecting his real-life behaviour or he starts to answer in a completely random fashion just to get past the survey.)

The use of hierarchical Bayesian models has allowed an immense reduction in data per respondent needed for reasonably robust individual-level models. Yet there is still great opportunity to improve the accuracy of our models of consumer preferences by increasing the quality of the data collected.

Adaptive choice-based conjoint (ACBC) is a successful method of conjoint analysis that introduces the use of different types of questions to keep respondent's motivation to answer responsibly high and increases the efficiency of the data collection process by avoiding the use of choice tasks not relevant for the respondent. In this paper, a new hybrid approach to choice-based conjoint (HCBC) is presented. This approach combines direct questions with traditional choice tasks, which are made more efficient by using information acquired directly.

<footnote>* University of Economics, Prague, Faculty of Informatics and Statistics (ondrej.vilikus@vse.cz).</footnote>
It is shown that, under certain conditions, this method can yield more accurate models of choice-behaviour while keeping the length of the questionnaire and fatigue of respondents reasonably low.

# 1. Basics of conjoint analysis

## 1.1 Analysing consumer preferences

To illustrate some basic concepts used in conjoint analysis, let us introduce a simple example. A procurement manager in a large company is responsible for ordering new chairs for the headquarters offices. He strives to keep the budget low but he also wants the new chairs to be accepted well by the staff, so he is interested in their preferences. For obvious reasons, all the chairs in the office need to be the same but each employee has different preferences.

If the producer of the chairs offered a limited number of chair models to choose from, the manager could easily obtain all the necessary information about the staff preferences by asking each employee to rate or rank his preference for each model. However, the producer allows the chairs to be custom-designed for bulk orders by choosing levels of several attributes. That makes the number of chair models available to be in hundreds or even thousands and impossible to be asked one by one. This is where conjoint analysis finds its use to model preference as a function of several parameters that constitute the product or service.

Instead of letting the consumer judge the attractiveness of thousands of chairs, a carefully prepared set of product descriptions (profiles) is shown to the respondent. We assume that the product can be described by $k$ attributes numbered $l = 1, 2, \dots, K$. In the case of chairs, it could be upholstery colour, backrest type, etc. Each of the attributes has $L_k$ levels labelled $l = 1, 2, \dots$ such as different colours of the upholstery or various types of the backrest. The attractiveness (utility) of the product is then modelled as a sum of the utilities of its present levels. If we denote the description of the $j$-th profile shown to the respondent by vector $\mathbf{x}_j$ with $\sum_{k=1}^{K} L_k - 1$ elements (the value of each element is 1 if the given level is chosen and 0 otherwise) and the utilities of the $i$-th respondent from all the levels by vector $\mathbf{\beta}_i$ of the same size, then the utility of the $j$-th profile is estimated to be

$$U_{ij} = \mathbf{x}_j \cdot \mathbf{\beta}_i. \qquad (1)$$

By letting each respondent evaluate a sufficient number of profiles, we can estimate their preferences $\mathbf{\beta}_i$ with the use of regression models. Estimating the attractiveness of any profile that can be created from the test levels is then a matter of using the formula (1). That way, the manager will be able to choose the chair representing the best compromise for all the employees (whatever that means for him).
## 1.2 Choice-based conjoint

While the first applications of conjoint analysis (Green and Rao, 1971) relied on rating or ranking profiles by the respondents, models estimated in such a way were found inappropriate for predicting consumer choices. Neither was rating and ranking a natural way of expressing preferences for the respondents nor was the relationship between estimated utilities and choice among a set of profiles formalised.

Choice-based conjoint as suggested by Louviere and Woodworth (1983) uses discrete choice models to estimate consumer preferences. Respondents do not rank all the profiles shown but they are repeatedly asked to choose the most preferred option among a set of profiles representing goods or services. In each task $t = 1, ..., T$, the $i$-th respondent chooses one of $J$ profiles shown. We then use a logit model to estimate the probability $P_{ijt}$ of this respondent choosing the alternative $j$' as

$$P_{j,t}(\mathbf{X}_{it} | \boldsymbol{\beta}) = \frac{\exp(\mathbf{x}_{jt} | \boldsymbol{\beta}_t)}{\sum_{j=1}^{J} \exp(\mathbf{x}_{jt} | \boldsymbol{\beta}_t)}, \quad (2)$$

where $\mathbf{x}_{jt}$ is the profile description of the $j$-th alternative in the $t$-th choice task and $\mathbf{X}_{it}$ represents a matrix of all profile descriptions in the task $t$ shown to the respondent $i$.

Choice-based conjoint has also allowed us to model minimum utility for the respondent to treat the alternative as acceptable by introducing a “none” option in the survey. The respondent can decide whether he chooses one of the options provided or whether none of them is acceptable. Since choosing the “none” option gives us no information about the preference among the options provided, we can also use a “dual none” option: letting the respondent choose one of the alternatives and asking him whether the chosen alternative is acceptable or not.

Aside from its numerous benefits, choice-based conjoint has at least one severe drawback, which is the low efficiency of the data collection. In each task, respondents have to study and compare several profiles but this gives us only information about which profile is the most attractive. Since answering such tasks is monotonous and rather boring, their number needs to be kept low and it is rarely possible to estimate the preferences at the individual level. Since consumers’ preferences usually differ, models estimated at an aggregate level tend to have lower accuracy.

## 1.3 Use of hierarchical Bayesian models

Modelling individual-level preferences from choice-based conjoint data was not possible until the introduction of hierarchical Bayesian models in conjoint analysis (Allenby and Ginter, 1995; Lenk et al., 1996). A hierarchical model assumes that each respondent has different preferences but that the deviation from the “average respondent” can be described by multivariate distribution.

In the simplest case, the hierarchical model for choice-based conjoint data has three levels. The lowest level is represented by the discrete-choice model in which
each respondent's choices $y_{ij}$ (coded as 1 if the $j$-th alternative is chosen by the $i$-th respondent and 0 otherwise) are explained as a function of his individual preferences

$$P(y_{ij} | \beta_i) = P(y_{ij} = 1) = \frac{\exp(x_j \beta_i)}{\sum_{j=1}^{J} \exp(x_j \beta_i)} \quad (3)$$

The second level specifies the distribution of the vectors $\beta_i$ in the population. We assume these to have multivariate normal distribution with mean $\bar{\beta}$ and covariance matrix

$$P(\beta_i | \bar{\beta}, \Sigma_{\beta}) = N(\bar{\beta}, \Sigma_{\beta}). \quad (4)$$

In the last level, we specify prior distributions for estimated hyperparameters $\bar{\beta}$ and $\Sigma_{\beta}$. Since we usually do not have any prior information about these parameters, we choose some uninformative distributions. In the case of the multivariate normal model, Rossi et al. (2005) suggest using

$$\bar{\beta} \approx N(\bar{\beta}, A^{-1}) \quad \text{and} \quad \Sigma_{\beta} = IW(v, \Sigma), \quad (5)$$

where $\bar{\beta}$ represents overall mean preferences and $A^{-1}$ is a diagonal matrix with diagonal elements of 100. $IW$ is an inverse Wishart matrix with a low number of degrees of freedom and an identity matrix as a mean.

Thanks to the assumption that the vectors of individual preferences $\beta_i$ come from the common distribution and so stronger deviations from the rest of the population are unlikely, estimates from hierarchical models are more robust than estimates that would be obtained (if it were possible) from individual models.

The effect of shifting individual level estimates towards overall mean (called Bayesian shrinkage) does not affect all cases equally. If we do not have enough data from a given respondent or his answers are inconsistent with respect to the model, the shrinkage is strong, making larger deviations impossible. However, if we have enough data to prove that the respondent's preferences differ from the rest of the population, this effect is weaker. It was previously shown (Vilikus, 2012) that hierarchical Bayesian models provide more accurate predictions than other approaches for modelling choice-based conjoint data under a very wide range of situations. Furthermore, the Bayesian approach brings other interpretational benefits as summarised by Hebák (2012).

## 1.4 Adaptive choice-based conjoint

Since the use of hierarchical Bayesian models was agreed as a standard in choice-based conjoint analysis, it has become apparent that users of choice-based conjoint also face other issues that cannot be solved by introduction of more sophisticated data analysis methods since these are associated with the data collection process.

Adaptive choice-based conjoint (ACBC) was introduced by Johnson and Orme (2007) as a modification of the traditional CBC survey in order to eliminate some
of these issues by combining different types of questions in the survey. ACBC is based
on the idea that purely random generation of profiles that are shown to the respondents
is not efficient when few of the profiles shown are actually relevant for the respondent.
This can be because they are too far away from what the respondent would consider an
ideal profile or because some attribute levels are found unacceptable by him. In either
case, the respondent is not motivated to make trade-offs among more attributes and his
answers are not informative regarding his preference.

Let us return to the example with the new chairs being offered for the office.
Assume that a respondent needs a chair without casters. Any chair with casters is
unacceptable for him. If we use traditional CBC to model the respondent's preferences,
it is likely that most of the chairs presented to the respondent in the choice tasks will
have casters. The respondent therefore does not choose any of the chairs in many tasks.
We may find out that he rejects casters but his further preferences will remain unknown.
This is where ACBC helps to enhance the efficiency of the data collection.

In the ACBC survey, the respondent is asked first about his ideal profile and then
he is shown several profiles that are not generated purely at random but rather by
changing the number by some levels in the ideal profile – see Sawtooth (2009) for
more details. For each profile, the respondent is asked whether it is acceptable or not.
If there is a sign that the respondent is screening out profiles with a certain attribute
level, this rule is checked with a direct question so that the given level is not shown in
further profiles. Profiles that are chosen as acceptable are used in the choice tasks in
the following section. To make the choices even more challenging for the respondent,
profiles selected in the choice tasks are used again in the following tasks until the most
attractive profile of all those generated is chosen.

An ACBC survey usually takes longer but respondents often find it more enterta-
ining than the traditional set of choice tasks. More importantly, it has been found that
in many cases using ACBC leads to a model that is more accurate in predicting respon-
dents' choices in holdout tasks (Sawtooth, 2009).

## 2. Hybrid choice-based conjoint approach

2.1 Main idea

As a combination of different types of questions has been proven to increase the efficiency of the data collection, the proposed hybrid approach uses different types of questions to further improve the efficiency of the data collection process by combining direct questions and optimised choice tasks. The main idea of the hybrid choice-based conjoint approach is based on the assumption that the information that we acquire from the respondents during the survey can be split into three parts. Some information can be asked about directly, while some not.

The first part represents the preference order of the levels of all the attributes. This information is of an ordinal scale and is available in most cases from the respondents by direct questioning (for example, the respondent is able to rank colours of chair uphol-
stery according to his preference). For some attributes, the order of preference can be
obvious and it does not have to be asked about or estimated at all.

The second part of the information is the absolute differences in the attractiveness of the attribute levels. Let us assume that chairs are available in three colours: red, black and grey. When the respondent prefers the grey colour over black and black over red, the “distance” between grey and black can be perceived differently from the distance between black and red. More specifically, the respondent may be almost indifferent between grey and black but red upholstery can be completely unacceptable for him. This part of the information is of a cardinal scale and it is virtually impossible to obtain it by direct questions.

Direct questions are also impractical for obtaining the third part of the information acquired in a conjoint analysis survey. This part represents the relative importance of the specific attributes of the product or service tested such as how much more important the type of the chair backrest is compared to the colour of the chair. Direct questions about the importance of given attributes usually give answers with low validity, but if we know which backrest type and which colour are the most and least preferred, it is possible to get at least a ranking of the attributes by importance by comparing specifically designed profiles.

Since some information can be obtained with direct questions, we may already
use this information to further increase the efficiency of the choice tasks used later
in the survey for getting the rest of the information about the respondent's preferences.

## 2.2 Increasing efficiency by optimising the experimental design

One of the approaches historically used for improving the efficiency of the data colle-
ction in conjoint analysis is construction of optimal experiment designs (what profiles
will be shown to each individual respondent). Various algorithms leading to efficient
designs achieving minimum estimation errors of the model parameters have been
proposed by Bunch, Louviere and Anderson (1996) for a model without interactions or
by Lazari and Anderson (1994) for a model with interactions. A more detailed overview
of these algorithms can be found in Chrzan and Orme (2000).

While these algorithms lead to designs that are efficient in situations where we
have no a priori knowledge of the parameters estimated and so the researcher expects
all the model parameters to be zero, Huber and Zwerina (1996) proposed algorithms
that can increase efficiency of the design for situations where we have some knowledge
of the preference of some attribute levels over other and where the strategies assuming
no a priori information about the estimated parameters are too conservative.

The *a priori* information is often based on previous studies or theoretical
knowledge. Alternatively, it can also be based on information acquired from direct
questions as mentioned earlier. According to Huber and Zwerina (1996), the efficiency
of the design using *a priori* knowledge can be improved by 10 to 50 percent, which
means that we can achieve estimates with an accuracy with up to 50 percent fewer of
choice tasks per respondent.
For measuring the efficiency of the experimental design, it is practical to denote as $\mathbf{X}$ the matrix of all matrices $\mathbf{X}_j$ describing profiles shown in task $t$ with $t$ ranging from 1 to the total number of tasks $T$. Furthermore, we denote as $\mathbf{y}$ the vector with $J \cdot T$ elements $y_{jt}$. These are of the value 1 if the respondent has chosen the alternative $j$ in the task $t$ and 0 otherwise. Then the logarithm of the likelihood function of the logit model can be written as

$$ \ln L(\mathbf{y}|\mathbf{X}, \boldsymbol{\beta}) = \sum_{j=1}^{T} \sum_{t=1}^{J} y_{jt} \ln(P_{jt}(\mathbf{X}_j, \boldsymbol{\beta}))+k_L, \quad (6) $$

where $k_L$ is constant with respect to $\mathbf{X}$ and $\boldsymbol{\beta}$.

By maximising function (6), we achieve an estimate $\hat{\boldsymbol{\beta}}$, which according to McFadden (1974) has the expected value $\boldsymbol{\beta}$ and asymptotically normal distribution and covariance matrix

$$ \Omega_p^{-1} = (Z'PZ)^{-1} = \left( \sum_{j=1}^{T} \sum_{t=1}^{J} z'_{jt} p_{jt} z_{jt} \right)^{-1}, \quad (7) $$

where $P$ is a diagonal matrix of the dimension $J \cdot T \times J \cdot T$ with diagonal elements $p_{jt}$ representing the expected probability of choosing the alternative $J$ in task $T$ and $Z$ is a matrix of the dimension $J \cdot T \times \sum_{k=1}^{K} L_k - 1$, whose rows $\mathbf{z}_{jt}$ with $\sum_{k=1}^{K} L_k - 1$ elements (one for each attribute level) are obtained as

$$ \mathbf{z}_{jt} = \mathbf{x}_{jt} - \sum_{j=1}^{J} \mathbf{x}_{jt} p_{jt}. \quad (8) $$

If we have no prior information about $\boldsymbol{\beta}$ so we assume $\boldsymbol{\beta}=\mathbf{0}$, the covariance matrix of $\hat{\boldsymbol{\beta}}$ simplifies to

$$ \Omega_0^{-1} = (Z'PZ)^{-1} = \left( \frac{1}{J} \sum_{j=1}^{T} \sum_{t=1}^{J} z'_{jt} z_{jt} \right)^{-1}, \quad (9) $$

where

$$ \mathbf{z}_{jt} = \mathbf{x}_{jt} - \frac{1}{J} \sum_{j=1}^{J} \mathbf{x}_{jt} = \mathbf{x}_{jt} - \bar{\mathbf{x}}, \quad (10) $$

since the expected probability is for all $J$ alternatives equal $1/J$.

Based on the matrices $\Omega_p^{-1}$ and $\Omega_0^{-1}$, Huber a Zwerina define two measures of design efficiency denoted as

$$ D_p = (\det \Omega_p^{-1})^{1/K}, \quad (11) $$

and

$$ D_0 = (\det \Omega_0^{-1})^{1/K}, \quad (12) $$

where $K$ is the number of parameters estimated.
In the case of choice-based conjoint, the design efficiency depends on four principles. Two of them are common with the traditional conjoint design (orthogonality and level balance). The third principle represents the condition of minimum overlap. It is desirable that the attribute levels shown in one task are not equal among the profiles. On the other hand, strict adherence to this requirement may result in a situation where a respondent's choices are based on a single attribute only and his preference of levels for the other attributes is unknown, so the condition is relaxed to the requirement of balanced overlap.

While these three principles minimise the $D_0$ error, the $D_p$ error can by minimised by focusing on the fourth condition – the utility balance. It can be shown that the $D_p$ error is minimal if the expected probabilities $p_{ji}$ of choosing all are equal $1/J$. Conversely, if the probabilities are closer to 0 or 1, information acquired from such tasks is less valuable and the error of estimates increases.

Huber a Zwerina showed that the $D_p$ error can be lowered by using two methods of design adjustments minimising the $D_0$ error: swapping attribute levels between the profiles within tasks and relabelling attribute levels in the design changing instances of one level with another and vice versa. With relabelling, we can achieve a design with the same $D_0$ error as the original but if the substitution of labels yields more balanced expected utilities, the $D_p$ error of the relabelled design is lower. The increase in efficiency therefore comes at no cost but except for the smallest designs the improvement it very small.

While switching the labels among profiles in one task to reduce the $D_p$ error does not affect level balance, unlike relabeling it affects the orthogonality of the sample as well as the $D_0$ error. An increase in $D_0$ error is acceptable in most cases according to Huber and Zwerina, and a switched design can lead to more accurate estimates even in cases where our prior knowledge of $\beta$ is not very high.

## 2.3 Hybrid choice-based conjoint approach

The hybrid choice-based conjoint approach is based on the assumption that we can ask directly for preference of attribute levels and that we can create a more efficient experiment design if we use that knowledge.

The survey has three phases. In the first phase, the respondent is asked to rank by preference profiles that only differ in levels of one attribute. For each attribute, he or she responds to one set. All the attributes are asked in a random order and the chosen levels are used in all the profiles for the following rankings. If the order is obvious for some attributes, these attributes are skipped.

The goal of the second phase is to obtain rough estimates of the importance of all the attributes. Since direct questions about importance usually do not offer acceptable results, the estimate is based on a single task in which the respondent ranks a set of profiles by preference. The number of the profiles is equal to the number of attributes tested and they are derived from the optimum profile. For each profile, all the attributes are set to the preferred level with exactly one attribute switched for the least preferred level. The attribute for which the switched profile is the least attractive is assumed to be the most important and the preference rating of the other profiles indicates the order of importance of the rest of the attributes.
Information from the first two phases is used for rough estimation of individual part-worth utilities. This rough estimate is then used to adjust the traditional choice-based conjoint tasks shown in the third phase by switching labels to achieve utility balance.

The process of estimation is based on the following simple heuristics, which has been found to be quite robust. Part-worth utilities of all the levels are estimated as

$$b_{kl}^* = I_k \frac{L_k - 2y_{kl} + 1}{L_k - 1}, \qquad (13)$$

where $y_{kl}$ is the ranking by preference of the level $l$ of the attribute $k$ with $L_k$ levels and $I_k$ is the estimated importance for each attribute. The estimated importance is equal to 1 for the most important attribute, 2/3 for the second and third and 1/3 for all the other attributes. This approach leads to relatively conservative estimates respecting the experience that respondents mainly focus on the top two or three attributes in their choices.

Responses from the first two parts of the survey are not used only for increasing efficiency of the choice tasks but they are also coded as discrete choices from the set of profiles and used for the estimation of model parameters.

# 3. Practical study setup

## 3.1 Setup outline

The proposed method of a hybrid choice-based conjoint differs from other approaches mainly in the data collection process. So, to illustrate and compare the HCBC method required setting up a study involving real respondents.

In the study, each respondent was assigned randomly to one of three test groups. One group of respondents was given traditional CBC tasks, the second group responded to an adaptive choice-based conjoint (ACBC) survey such as proposed by Orme (2010) and the third was given HCBC questions as outlined above.

Apart from the main part of the survey, each respondent filled in several demographic questions and evaluated his or her experience completing the questionnaire. For assessing model fit, five holdout tasks that were not used for the estimation were placed at the very end of the survey.

## 3.2 Study parameters

For an illustration of the new method and its benefits or limitations in comparison to other data collection methods, the object of the study was selected the same as in the example mentioned earlier. Office chairs satisfy the following needs: the product should be relevant to a broad audience (so it is not difficult to find respondents), it can be described with several relatively independent attributes (so that conjoint analysis is an appropriate method for analysing preference) and the order of attribute levels by preference is not obvious (so that the use of the hybrid approach is meaningful).
In the study, chairs were described using six attributes:

- backrest type (3 levels: upholstered with and without headrest, mesh),

- armrest type (4 levels: plastic, gel, textile, none),

- frame colour (2 levels: light, dark),

- upholstery colour (7 levels: black, grey, dark blue, light blue, green, beige, dark red),

- upholstery material (4 levels: smooth textile, rough textile, suede, leather),

- casters (4 levels: soft, hard, locking, none).

While the armrest type, casters and upholstery material were described only verbally, the other attributes were depicted using images provided with the courtesy of Humanscale Inc.

## 3.3 Setup details and data collection

The group responding to the traditional CBC was given 12 choice tasks with three alternatives and two none questions. The experimental design was prepared using the balanced overlap method. Respondents assigned to the ACBC questionnaire responded to the build-your-own section followed by 24 profiles shown in sets of 3. Based on their answers, they were asked for up to 5 must-haves and unacceptables. Up to 20 profiles that were rated as acceptable by the respondent were then used in the choice tasks where the respondent was asked to choose the preferred profile out of three. Chosen profiles are shown in the tasks. The HCBC group ranked all the attribute levels and was given 12 choice tasks with balanced utilities.

The first three holdout tasks were the same for all the respondents while the fourth one included three profiles not selected in the previous tasks and the last included the winners of the first three tasks to create more balanced and therefore difficult tasks with both high and low utility of the three profiles overall.

The questionnaire was scripted using Sawtooth Software SSI Web v7.0.26. Subsequently, the link was shared with friends and colleagues to be spread further. While the sample is by no means representative of any target group, it represents a set of respondents with various preferences and is adequate for the purpose of the study. Out of the 421 respondents who completed the survey, 143 took part in the CBC, 142 were assigned to the ACBC survey, and 136 belonged to the HCBC test group.

# 4. Results

## 4.1 Comparison of the estimates

Before comparing the efficiency of the data collection approaches, it was necessary to check that all the approaches lead to similar estimates and that there are no systematic differences in the part-worth utilities obtained, which would make objective assessment of the efficiency impossible.

First, the estimated importances of the tested attributes were compared using the three approaches. In the case of the HCBC group, the estimates were based on direct
questions (phases 1 and 2) and choice tasks (phase 3). Relative importance of attributes has been traditionally expressed based on the difference between the part-worth utilities of the most and least preferred level of each attribute (Green and Wind, 1975). In a situation where we estimate part-worth utilities at the individual level, we use the average of individual relative importances for all the respondents

$$I_{ik'} = \sum_{i=1}^{n} \frac{\max_{l} \beta_{ikl} - \min_{l} \beta_{ikl}}{\sum_{k=1}^{K} (\max_{l} \beta_{ikl} - \min_{l} \beta_{ikl})} \quad (14)$$

as a measure of overall importance $\bar{I}_k$ for each attribute. We can see from the comparison of the importance listed in Table 1 that the ranking of all the attributes is very similar in all the cases. The type of casters was detected as the most important attribute in all the cases except for the CBC, where the importance of the armrests was slightly higher. In all the other cases, armrests scored second in importance. The importance of the backrest type and upholstery material was again quite similar in all the cases and all the methods agreed on upholstery colour and frame colour being the least important in determining consumer choices.

<table><caption>**Table 1**
Overall estimated attribute importance for compared approach</caption><thead><tr><th rowspan="3">Attribute</th><th colspan="5">I&#773;<sub>k</sub> (%)</th></tr><tr><th rowspan="2">(CBC)</th><th rowspan="2">(ACBC)</th><th colspan="3">(HCBC)</th></tr><tr><th>1+2+3</th><th>1+2</th><th>3</th></tr></thead><tbody><tr><td>Casters</td><td>29,3</td><td>31,0</td><td>30,3</td><td>26,3</td><td>32,3</td></tr><tr><td>Armrest type</td><td>30,7</td><td>26,1</td><td>27,1</td><td>24,5</td><td>29,5</td></tr><tr><td>Backrest type</td><td>16,4</td><td>14,7</td><td>16,3</td><td>18,3</td><td>14,1</td></tr><tr><td>Upholstery material</td><td>11,5</td><td>15,4</td><td>13,4</td><td>14,1</td><td>12,4</td></tr><tr><td>Upholstery colour</td><td>9,0</td><td>10,5</td><td>9,8</td><td>11,6</td><td>8,8</td></tr><tr><td>Frame colour</td><td>3,1</td><td>2,3</td><td>3,2</td><td>5,2</td><td>2,9</td></tr></tbody></table>

While all the methods agree on ranking of the attributes by importance, we can see that the relative importances are distributed more evenly in the case of the use of direct questions from the HCBC survey only. This is in agreement with the assumption that respondents tend to focus more on just the most important attributes in the choice task and direct questions motivate respondents to focus more on the least important attributes as well. The hybrid approach could therefore be potentially useful in cases where we expect respondents to be less motivated to study all the parameters of the profiles shown compared to a situation where they are choosing the product in real life and we would like to minimise or at least identify this effect causing measurement error.

To further compare the results provided by all the methods, average part-worth utilities of all the attribute levels are listed in Table 2. Ranking of the levels for the two most important attributes is very close for all the methods and the agreement is
high even for the backrest type and the upholstery material, where the differences in preference are not so strong. The upholstery colour attribute is of low importance but a high number of levels, so for conjoint analysis methods relying on choice tasks only, it is difficult to estimate consumers' preference. While the overall ranking differs, it still shows a relatively similar pattern.

<table><caption>**Table 2**
Comparison of average estimated part-worth utilities</caption><thead><tr><th rowspan="2">Attribute level</th><th rowspan="2">(CBC)</th><th rowspan="2">(ACBC)</th><th colspan="3">(HCBC)</th></tr><tr><th>1+2+3</th><th>1+2</th><th>3</th></tr></thead><tbody><tr><td>Soft casters</td><td>2.3</td><td>3.0</td><td>2.6</td><td>3.6</td><td>2.5</td></tr><tr><td>Hard casters</td><td>1.9</td><td>2.5</td><td>2.5</td><td>3.3</td><td>2.6</td></tr><tr><td>Locking casters</td><td>1.1</td><td>0.9</td><td>1.2</td><td>1.1</td><td>1.7</td></tr><tr><td>No casters</td><td>-5.3</td><td>-6.4</td><td>-6.3</td><td>-8.0</td><td>-6.7</td></tr><tr><td>Gel armrests</td><td>2.2</td><td>2.1</td><td>2.6</td><td>3.6</td><td>2.4</td></tr><tr><td>Textile armrests</td><td>2.0</td><td>1.8</td><td>1.9</td><td>2.4</td><td>2.0</td></tr><tr><td>Plastic armrests</td><td>1.3</td><td>1.0</td><td>1.0</td><td>0.8</td><td>1.7</td></tr><tr><td>No armrests</td><td>-5.5</td><td>-4.9</td><td>-5.4</td><td>-6.8</td><td>-6.2</td></tr><tr><td>Textile with headrest</td><td>1.4</td><td>1.7</td><td>1.9</td><td>3.3</td><td>1.7</td></tr><tr><td>Textile backrest</td><td>0.0</td><td>-0.6</td><td>-0.9</td><td>-2.0</td><td>-0.7</td></tr><tr><td>Mesh backrest</td><td>-1.4</td><td>-1.2</td><td>-1.0</td><td>-1.4</td><td>-1.1</td></tr><tr><td>Rough textile</td><td>0.8</td><td>1.0</td><td>0.9</td><td>1.3</td><td>1.0</td></tr><tr><td>Smooth textile</td><td>0.7</td><td>1.3</td><td>0.8</td><td>1.6</td><td>0.2</td></tr><tr><td>Leather</td><td>-1.3</td><td>-1.8</td><td>-1.0</td><td>-1.3</td><td>-1.1</td></tr><tr><td>Suede</td><td>-0.3</td><td>-0.4</td><td>-0.7</td><td>-1.7</td><td>-0.1</td></tr><tr><td>Black upholstery</td><td>-0.1</td><td>0.8</td><td>0.7</td><td>1.4</td><td>0.2</td></tr><tr><td>Green upholstery</td><td>0.4</td><td>0.0</td><td>-0.3</td><td>-0.5</td><td>-0.1</td></tr><tr><td>Dark blue upholstery</td><td>0.3</td><td>0.4</td><td>0.6</td><td>1.0</td><td>0.0</td></tr><tr><td>Grey upholstery</td><td>-0.1</td><td>0.2</td><td>0.2</td><td>0.5</td><td>-0.2</td></tr><tr><td>Dark red upholstery</td><td>-0.1</td><td>0.0</td><td>0.0</td><td>0.0</td><td>0.0</td></tr><tr><td>Beige upholstery</td><td>-0.2</td><td>-1.0</td><td>-1.1</td><td>-2.0</td><td>-0.4</td></tr><tr><td>Light blue upholstery</td><td>-0.2</td><td>-0.3</td><td>-0.1</td><td>-0.4</td><td>0.4</td></tr><tr><td>Dark frame</td><td>0.2</td><td>0.3</td><td>0.4</td><td>1.0</td><td>-0.1</td></tr><tr><td>Light frame</td><td>-0.2</td><td>-0.3</td><td>-0.4</td><td>-1.0</td><td>0.1</td></tr></tbody></table>

For the two attributes of the lowest importance, it is apparent that when choice task data were used, all the levels seemed to have similar attractiveness. However, when direct questions were used, the differences in preference among the levels of these
attributes were larger. While this extra information about the least important attributes has only a minimal impact on predicting consumers' choices when the profiles are distinct in the more important attributes, it can be valuable in cases where the levels of the important attributes are equal and the respondent is making his choice only based on the remaining attributes.

## 4.2 Comparing accuracy of predictions

Successful identification of important attributes and preferred levels at an aggregate level does not guarantee accurate predictions of individual choices. To compare the predictive accuracy of the models using data collected by the three methods, all the models were used to predict the individual choices in the holdout tasks. A Comparison of the hit rates achieved for the holdout tasks and overall is summarised in Table 3.

<table><caption>**Table 3**
Hit rates achieved for each holdout task</caption><thead><tr><th rowspan="2">Task</th><th rowspan="2">(CBC)</th><th rowspan="2">(ACBC)</th><th colspan="3">(HCBC)</th></tr><tr><th>1+2+3</th><th>1+2</th><th>3</th></tr></thead><tbody><tr><td>1</td><td>0.604</td><td>0.592</td><td>0.629</td><td>0.568</td><td>0.575</td></tr><tr><td>2</td><td>0.664</td><td>0.590</td><td>0.774</td><td>0.729</td><td>0.743</td></tr><tr><td>3</td><td>0.800</td><td>0.772</td><td>0.831</td><td>0.814</td><td>0.803</td></tr><tr><td>4</td><td>0.703</td><td>0.642</td><td>0.715</td><td>0.675</td><td>0.677</td></tr><tr><td>5</td><td>0.593</td><td>0.725</td><td>0.702</td><td>0.694</td><td>0.633</td></tr><tr><td>Total</td><td>0.673</td><td>0.664</td><td>0.730</td><td>0.696</td><td>0.686</td></tr></tbody></table>

It is apparent that when using all available data, the hybrid approach leads to the highest hit rate in four out of five holdout tasks and is a clear winner overall. Only in the case of the last holdout tasks, where the respondents were presented profiles chosen in the first holdout tasks, did the ACBC result in a slightly higher success rate. This is to be expected since the ACBC was designed to predict well choices among more attractive profiles. Interestingly enough, the HCBC hit rate was slightly higher than for the CBC or ACBC even if we only use the 12 choice tasks, which proves that the adjustment of the design did lead to higher efficiency.

It is also of interest that the predictive ability of the model based on direct questions only was higher than that of any other model based solely on choice tasks. This supports the hypothesis that the direct approach can be a valuable source of information in some applications of conjoint analysis.

## 4.3 Respondent fatigue

Experience shows that a high hit rate of the model does not have to be a priority if it comes at the cost of too high respondent fatigue, which can result in low completion rates and low motivation to answer responsibly. It is necessary to take this aspect into
account as well and choose approaches that keep respondents' motivation high during the survey.

Respondent fatigue was measured in multiple ways. The first and most straight-forward was a measurement of time spent on the main part of the questionnaire. The information about time spent on each screen was measured and the times were checked for outliers indicating that the respondent had interrupted answering the questionnaire.

Summaries of the observed times are shown in Table 4. While the respondents spent 4 minutes on average on the 12 CBC tasks, and most of the respondents were able to answer in less than 6 minutes, the time spent on the ACBC tasks was 77% longer and in the case of the HCBC even by 113%.

<table><caption>**Table 4**
Time spent answering the main part of the questionnaire in minutes</caption><thead><tr><th></th><th>CBC</th><th>ACBC</th><th>HCBC</th></tr></thead><tbody><tr><td>Average</td><td>4.37</td><td>7.74</td><td>9.30</td></tr><tr><td>Standard deviation</td><td>1.65</td><td>2.66</td><td>2.75</td></tr><tr><td>25% percentile</td><td>3.13</td><td>6.04</td><td>7.14</td></tr><tr><td>50% percentile</td><td>4.12</td><td>7.38</td><td>8.62</td></tr><tr><td>75% percentile</td><td>5.39</td><td>9.23</td><td>11.02</td></tr></tbody></table>

<table><caption>**Table 5**
Respondents' ratings (averages using 5-point scale, significant differences at the 5% level based on independent sample t-test labelled with an asterisk)</caption><thead><tr><th></th><th>CBC</th><th>ACBC</th><th>HCBC</th></tr></thead><tbody><tr><td>How would you compare your overall experience with this survey compared to other internet surveys you have completed?</td><td>2.92</td><td>3.31*</td><td>3.15</td></tr><tr><td>The survey format made it easy for me to give realistic answers.</td><td>4.09</td><td>4.13</td><td>4.18</td></tr><tr><td>The survey was at times monotonous and boring</td><td>4.08</td><td>3.63*</td><td>4.00</td></tr><tr><td>I'd be very interested in taking another survey just like this in the future.</td><td>2.75</td><td>2.89*</td><td>2.62</td></tr><tr><td>It was easy to make the choices.</td><td>3.99</td><td>4.11</td><td>4.10</td></tr><tr><td>The way the options were presented made me want to slow down and make careful choices.</td><td>3.52</td><td>3.82*</td><td>3.66</td></tr></tbody></table>

While the differences in time spent were quite large, the respondents' evaluations show that it is not the only factor determining their fatigue and motivation to respond to further questions. Table 5 summarises average ratings of the evaluations of several statements using a 5-point scale (1 = strongly disagree, 5 = strongly agree). Even though the ACBC questionnaire took much longer, it was rated as more entertaining
and less monotonous than the traditional CBC questionnaire and even the HCBC rating is at least comparable to the CBC. Thanks to the use of more types of questions, the ACBC and HCBC questionnaires were not found to be more demanding for the respondents. There were no other significant differences in the evaluations and even the share of incomplete questionnaires was comparable for all the techniques (19% for the ACBC and CBC and 22% for the HCBC).

### 4.4 Model fit as a function of number of tasks used

The number of tasks used in both the CBC and HCBC tasks was chosen as more than satisfactory for the given application, which could influence the comparison of the efficiency of the methods. Since the profiles shown in both the methods are, unlike in the ACBC, independent of the choices in previous tasks, the efficiency of the methods as a function of the number of tasks used can be tested by re-estimating the models using data from first few tasks only. Since the time spent on each screen was recorded, we can use the total time with the given number of tasks in the analysis as well.

<figure><img src="image_1.png" /><figcaption>**Figure 1**
Hit rate as a function of survey length</figcaption></figure>

As is apparent from the results presented before, even the use of data from the first two parts of the HCBC survey resulted in better fit than using data from all the CBC tasks. The use of extra tasks increases the fit further but even in the case of 3 to 5 tasks the hit rate was around 70%. In the case of traditional CBC, the fit increases significantly for the first 4 tasks but then the improvements are relatively low.
Not only is the hit rate higher in all the cases using HCBC but so it the survey longer even in the case of using only direct questions. On the other hand, we can expect direct questions to be less monotonous for the respondent and the part with direct questions can be shorter in situations where at least some attributes have an obvious order of level preference so the use of HCBC with fewer choice tasks can be used as an interesting alternative making the questionnaire only slightly longer but more entertaining for the respondent.

## Conclusions

The proposed hybrid method of choice-based conjoint uses direct questions for getting information about level preference and choice tasks for analysis of the trade-off the customers are facing, which is difficult to ask directly.

Although direct questions were used in the estimation of the conjoint model, the estimated parameters did not show any significant bias and the model using the data from the proposed approach resulted in a highest hit rate in holdout tasks compared to the traditional CBC as well as ACBC surveys.

Along with the improvements in model precision, HCBC has been found as beneficial for getting the information about preference in the case of attributes that are of lower importance but are possibly interesting for managerial decisions. While in the case of traditional CBC, these attributes are overlooked in the choices and so we have little information on which levels are preferred, the hybrid approach can give information about preference even in situations where some attributes are dominated by others.

Even if we are not interested in the less important attributes and we only care about predictive accuracy, HCBC brings increased efficiency with the use of balanced profiles in choice tasks and a possibility to increase the hit rate without the need for too high a number of monotonous choice tasks.

While it is not possible to make definite conclusions based on a single study, we can say that the proposed approach is potentially useful in situations where:

- we are afraid that less important attributes will be overshadowed by the key ones (or those with a range of levels too wide),

- some attributes have more levels than others,

- due to the complexity of the profiles, we want to minimise the number of profiles shown.

## References

- ALLENBY, G. M.; GINTER, J. L. Using Extremes to Design Products and Segment Markets. *Journal of Marketing Research*. 1995, Vol. 32, No. 4, p. 392–403.

- BUNCH, D. S.; LOUVIERE, J. J.; ANDERSON D. A Comparison of Experimental Design Strategies for Multinomial Logit Models: The Case of Generic Attributes. [Working paper UCD·GSM·WP# 11.96]. Davis: Graduate School of Management, University of California, 1996. faculty.gsm.ucdavis.edu/~bunch/blaWP_1996.pdf.

- CHRZAN, K.; ORME, B. An Overview and Comparison of Design Strategies for Choice-Based Conjoint Analysis. [Sawtooth Software Research Paper Series]. Sequim : Sawtooth Software, 2000. www.sawtoothsoftware.com/download/techpap/desncbc.pdf.
GREEN, P. E.; RAO, V. R. Conjoint Measurement for Quantifying Judgmental Data. *Journal of Marketing Research*. 1971, Vol. 8, p. 355–363.

GREEN, P. E.; WIND, Y. New Way to Measure Consumers’ Judgements. *Harvard Business Review*. 1975, July-August, p. 107–117.

HEBÁK, P. Srovnání klasické a bayesovské pravděpodobnosti a statistiky (1.). *Acta Oeconomica Pra-gensia*. 2012, No. 1, p. 69–87.

HUBER, J.; ZWERINA, K. The Importance of Utility Balance in Efficient Choice Designs. *Journal of Mar-keting Research*. 1996, Vol. 33, p. 307–317.

JOHNSON, R. M.; ORME, B. K. A New Approach to Adaptive CBC. In Sawtooth Software (ed.) 2007 Sawtooth Software Conference Proceedings. 2007, p. 67–84.

LAZARI, A. G.; ANDERSON D. A. Designs of Discrete Choice Experiments for Estimating Both Attribute and Availability Cross Effects. *Journal of Marketing Research*. 1994, Vol. 31, p. 375–383.

LENK, P. J.; DESARBO, W. S.; GREEN P. E.; YOUNG, M. R. Hierarchical Bayes Conjoint Analysis: Reco-very of Partworth Heterogeneity from Reduced Experimental Designs. *Marketing Science*. 1996, Vol. 15, p. 173–191.

MCFADDEN, D. Conditional Logit Analysis of Qualitative Choice Behavior. *Frontiers in Econometrics*. 1974, Vol. 1, No. 2, p. 105–142.

ORME, B. K. Task Order Effects in Menu-Based Choice. [Sawtooth Software Technical Pa-per Series]. Sequim, WA : Sawtooth Software, 2010. www.sawtoothsoftware.com/download/techpap/mbcorder2010.pdf.

SAWTOOTH SOFTWARE. ACBC Technical Paper. Sequim, WA : Sawtooth Software, 2009. www.sawtoothsoftware.com/download/techpap/acbctech.pdf.

VILIKUS, O. Analýza heterogenních preferencí pomocí hierarchických modelů. In DOUCEK, P.; NEDO-MOVÁ, L. (ed.). Sborník prací účastníků vědeckého semináře doktorského studia FIS VŠE. Praha : Nakladatelství Oeconomica, 2012, s. 259–268.

## HYBRID APPROACH TO CHOICE-BASED CONJOINT ANALYSIS

**Abstract:** Conjoint analysis is a popular tool for analysing consumer preferences in market research which has undergone rapid development throughout history. It is now generally agreed that choice-based conjoint (CBC) has a stronger theoretical background than traditional conjoint methods and that it mimics the real decision-making process of consumers more closely. When hierarchical Bayesian models allowed robust estimation of consumer preferences from sparse data available from choice-based conjoint tasks, formerly popular self-explicated or hybrid approaches lost their popularity. In this article, it is shown that hybrid approaches can be a useful alternative to pure CBC design. A hybrid approach to CBC that combines self-explicated questions on attribute levels with individualised choice tasks is suggested and illustrated on a real example and its efficiency is compared to traditional CBC and adaptive CBC. The results of the study support the hypothesis that this approach can be beneficial under certain circumstances and yield higher model fit while keeping the questionnaire length and respondent fatigue at an acceptable level.

**Keywords:** hybrid approach, choice-based conjoint, Bayesian models, adaptive conjoint

**JEL Classification:** C35, C91