# Combining Determinism and Intuition through
Univariate Decision Strategies for Target Detection from
Multi-sensors

Edwin El-Mahassni

► To cite this version:

Edwin El-Mahassni. Combining Determinism and Intuition through Univariate Decision Strategies for Target Detection from Multi-sensors. Third IFIP TC12 International Conference on Artificial Intelligence (AI) / Held as Part of World Computer Congress (WCC), Sep 2010, Brisbane, Australia. pp.123-132, 10.1007/978-3-642-15286-3_12. hal-01058353

HAL Id: hal-01058353

https://inria.hal.science/hal-01058353

Submitted on 26 Aug 2014

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Combining Determinism and Intuition through
Univariate Decision Strategies for Target
Detection from Multi-Sensors

Edwin El-Mahassni

ISR Division

Defence Science and Technology Organisation
Edinburgh, SA, 5111, Australia

Edwin.El-Mahassni@dsto.defence.gov.au

**Abstract.** In many surveillance systems, an operator will have to make a decision on whether a target is present or not from the outputs of a sensor. Here, we assume that such an operator has at his or her disposal two sensors to give him or her more confidence in his decision-making process. In addition, we propose several univariate decision strategies which combine sensor characteristics, target probabilities and reward schemes together with deterministic and intuitive operator decision-making parameters. Further, using Chebyshev's inequality we develop a method for selecting the most robust strategy that mitigates poor performance. We illustrate through examples how different strategies will be more advantageous over others depending on the reward scheme and sensor parameters.

**Key words:** Decision-making, Target detection, Intuition

# 1 Introduction

The ability to make decisions is ubiquitous in many defence settings. Sometimes, an operator has to make a deterministic decision based on the outputs of a sensor or sensors. On the other hand, an operator's experience should not be discounted and his intuition in events should also be used [1]. To add to the complexity of decision-making, an operator has to be aware of the rewards and costs associated with making the right or wrong decision [2, 3].

In general, there are two distinct philosophical systems toward decision making. These are the rational (or logical or analytical) vs. the naturalistic (or intuitive) [4] techniques. The latter are based upon an expert's background in dealing with similar situations in the past [1] and assume that experience is vital in making the right decision. The solutions are based on finding a satisficing outcome, rather than the optimal. In fact, it was found that over 95% of naval commanders and army officers use their intuition. We also remark that intuitive decision-making is being used to replace the standard military decision-making process [1].
More mathematical approaches can be found in [5], where the reward of making the right decision diminishes the longer it takes for the decision to be made. Other approaches which also claim to be rational-based can be found in [6, 7], where criteria and weights are used; though the fact that these are set by humans themselves makes their claim to objectivity weak [5].

Here, we derive and examine some decision strategies that combine deter-
minism (where the same decision is always made given the same conditions) and
intuition (greater flexibility with no fixed rules) for an operator to make a judge-
ment from two sensors. For target detection we recall the well-known formula for
the expected value of a decision strategy found in [2]. This formula is not only
dependant on detection, false alarm and target probabilities, but also considers
the rewards and costs of correct and incorrect decision-making. The decision
strategies listed here will all consist of one only variable which we define as x;
and it will be used to quantify an operator's probability of deciding when a tar-
get is present for some scenarios. We remark that at all times, the distribution of
x is known and could be predetermined through testing or simulations. Further,
any knowledge, such as sensor parameters, target probability, reward schemes,
etc. that the operator might have is assumed to be accommodated within this
distribution.

Therefore, if an operator has to make the same choice for a particular multi-
sensor reading we deem this to be a deterministic decision. However, as indicated
in [1], the benefits of intuitive decision-making should not be discounted; with
the variable *x* being used to model this intuition. On the other hand, because we
are also relying on personal judgement, we should not rule out the effects that
fatigue or his emotions can have in his decision-making ability [8]. In these cases,
poor performances are modelled by all the expected values of a decision strategy
*E*(*x*) which are themselves given by varying *x* from its optimal value of 0 or 1.
Alternatively, even within experienced operators, there could be different biases
and life experiences which could result in different decisions being made for the
same scenario [9]. Thus, we are interested in analysing decision strategies which
can accomodate some intuition, and then select the most robust strategy when
poor performance ensues. We note, however, that this given freedom to make
decisions will be restricted to only some sensor readings, thereby combining the
intuition or the experience of an operator with the certainty of determinism. The
following two assumptions are made: 1) when an operator has poor or underper-
forming intuitive skills, then the expected value of his decision strategy is given
by the mentioned formula in [2] and 2) when intuition aids an operator, then we
assume this expected value is greater than that given by the same formula.

The paper will be divided as follows. In section 2, we outline 5 decision
strategies and then derive their respective expected value formulae. Section 3
is devoted to analysis and description of the algorithm; in section 4 we provide
some examples, and finally in section 5 we provide some concluding remarks.
# 2 Univariate Decision Strategies

In this section, we recall the original expected value of a decision strategy function and then proceed to define some univariate decision strategies based on the output of 2 sensors.

## 2.1 The Decision Value Function

We note that originally the expected value function was framed in terms of signal detection theory [2] and whether a target was present or not. That is, first, we let $h_0$ and $h_1$ denote the hypotheses indicating the absence or presence of a target respectively. And, we let $H_0$ and $H_1$ be the response of the decision maker accepting the $h_0$ or $h_1$ hypothesis respectively. And, we also define the following:

- Let $V_{00}$ be the reward value associated with a correct choice of $H_0$, which occurs with probability $p(H_0|h_0)$. This is also known as a correct rejection.

- Let $V_{01}$ be the cost value associated with an incorrect choice of $H_1$ (when, in fact, $H_0$ is the correct alternative); that is, the person loses $V_{01}$ when this type of incorrect choice is made which occurs with probability $p(H_1|h_0)$. This is also known as a false alarm.

- Let $V_{11}$ be the reward value associated with a correct choice of $H_1$, which occurs with probability $p(H_1|h_1)$. This is also known as a hit.

- Let $V_{10}$ be the cost value associated with an incorrect choice of $H_0$ (when, in fact, $H_1$ is the correct alternative); that is, the person loses $V_{10}$ when this type of incorrect choice is made, which occurs with probability $p(H_0|h_1)$. This is also known as a miss.

Thus, the expected value of the decision strategy is given by

$$E = V_{00}P(h_0)P(H_0|h_0) + V_{11}P(h_1)P(H_1|h_1) - V_{10}P(h_1)P(H_0|h_1) - V_{01}P(h_0)P(H_1|h_0) \quad (1)$$

noting that of course $\sum_{i=0,1} P(H_i|h_j) = 1$ for $j = 0, 1$.

## 2.2 Decision Strategies

Now, in formulating decision strategies from sensor outputs, we need to make a link between $p(H_i|h_j), i, j \in \{0,1\}$ and $p(H_i|(s^1, s^2))$, where $s^1, s^2 \in \{0,1\}$. The values 1 and 0 for $(s^1, s^2)$ indicate the state where a sensor reports that a target is present or not respectively. But, we also note that the sensors' reports are dependent on whether there is an actual target present or not. Thus, target probability will also influence sensor reporting. These reports then form the basis of any judgement by an analyst or operator. This relationship can be expressed below as:

$$p(H_i|h_j) = \sum_{s^1,s^2 \in \{0,1\}} p(H_i|(s^1, s^2))p((s^1, s^2)|h_j). \quad (2)$$
If we now define $d^i$ and $f^i$ to be the probabilities of detection and false alarm for sensor $i$ respectively, then we have that

$$ 
\begin{align*}
p((1, 1)|h_0) &= f^1 f^2 & , p((1, 1)|h_1) &= d^1 d^2 \\
p((0, 1)|h_0) &= (1 - f^1) f^2 & , p((0, 1)|h_1) &= (1 - d^1) d^2 \\
p((1, 0)|h_0) &= f^1 (1 - f^2) & , p((1, 0)|h_1) &= d^1 (1 - d^2) \\
p((0, 0)|h_0) &= (1 - f^1)(1 - f^2) & , p((0, 0)|h_1) &= (1 - d^1)(1 - d^2).
\end{align*}
$$

Next, we outline 5 decision strategies based on variations of $p(H_i|(s^1, s^2))$ through the use of one variable. But, first, for brevity, we denote by $\tilde{s} = (s^1, s^2)$ as the instantaneous outputs of 2 sensors. Lastly, for all instances we let the variable $0 \le x \le 1$ be the probability that an operator will make a decision that a target is present for some $\tilde{s}$. The 5 decision strategies, remembering that $p(H_1|\tilde{s}) + p(H_0|\tilde{s}) = 1$, now follow, and they are used to model the mean decision value when an operator performs poorly; in the sense that he or she is not using intuition. The first strategies assume that $p(H_1|\tilde{s}) = 0$ if $\tilde{s} = (0, 0)$. This means that we assume that in this instance no target can possibly be present when neither sensor reports. The last two strategies, on the other hand, imply that $p(H_1|\tilde{s}) = 1$ if $\tilde{s} = (1, 1)$ always indicating the presence of a target. The third strategy assigns a deterministic choice when the sensors agree, and leaves an operator to make non-deterministic decisions when only 1 sensor reports. We denote for each strategy the value $x_i, i = 1, \dots, 5$ to stress the fact $x$ is not necessarily the same value for all decision strategies; although some dependencies within different strategies could also be accomodated. Further, the $x_i, i = 1, \dots, 5$ variables might lie between different upper and lower bounds also.

1. The first strategy is given by

$$ p(H_1|\tilde{s}) = \begin{cases} x_1, & \tilde{s} = (1, 1) \\ 0, & \tilde{s} \in \{(0, 0), (0, 1), (1, 0)\} \end{cases} \qquad (3) $$

Here, we label this case as very pessimistic. This is because we consider the sensors to be so unreliable that we do not even consider a target being present unless both sensors report. The operator is told to ignore all the cases where both sensors do not report and only use his discretion when both indicate the presence of a target. Combining (1) and (2), we have that the expected value is:

$$ E(x_1) = x_1(p(h_1)d^1d^2(V_{11} + V_{10}) - p(h_0)f^1f^2(V_{00} + V_{01})) + p(h_0)V_{00} - p(h_1)V_{10}. $$

2. The second strategy is given by

$$ p(H_1|\tilde{s}) = \begin{cases} x_2, & \tilde{s} \in \{(1, 1), (0, 1), (1, 0)\} \\ 0, & \tilde{s} = (0, 0) \end{cases} \qquad (4) $$
Here, we label this case as pessimistic. This strategy stipulates that even if only 1 sensor reports, the operator, to be on the safe side should treat this scenario on the same level as if they both reported. However, just like in the previous strategy, when both sensors do not report, it is assumed that a target is not present. Combining (1) and (2), we have that the expected value is:

$$E(x_2) = x_2 (p(h_0)((1-f^1)(1-f^2)-1)(V_{01} + V_{00}) + p(h_1)(1-(1-d^1)(1-d^2))(V_{10} + V_{11})) + p(h_0)V_{00} + p(h_1)V_{10}$$

3. The third strategy is given by

$$p(H_1|\tilde{s}) = \begin{cases} 1, & \tilde{s} = (1, 1) \\ x_3, & \tilde{s} \in \{(0, 1), (1, 0)\} \\ 0, & \tilde{s} = (0, 0) \end{cases} \qquad (5)$$

Here, we label this case as the unbiased case. The operator makes fully deterministic judgements when both sensors agree and is asked to use his discretion only when they disagree in their outputs. Combining (1) and (2), we have that the expected value is:

$$E(x_3) = x_3 (p(h_0)(V_{00} + V_{01})(2f^1f^2 - f^1 - f^2) + p(h_1)(V_{11} + V_{10})(d^1 + d^2 - 2d^1d^2) + p(h_0)(V_{00} - (V_{00} + V_{01})f^1f^2) + p(h_1)(V_{10} + (V_{11} + V_{10})d^1d^2))$$

4. The fourth strategy is given by

$$p(H_1|\tilde{s}) = \begin{cases} 1, & \tilde{s} = (1, 1) \\ x_4, & \tilde{s} \in \{(0, 0), (0, 1), (1, 0)\} \end{cases} \qquad (6)$$

Here, we label this case as the optimistic case. The operator always assumes that a target is present when both sensors report; all other $\tilde{s}$ are treated equally and left up to the operator's discretion to decide whether there is a target present or not. Combining (1) and (2), we have that the expected value is:

$$E(x_4) = x_4 (p(h_0)(V_{00} + V_{01})(f^1f^2 - 1) + p(h_1)(V_{11} + V_{10})(1 - d^1d^2)) + p(h_0)(V_{00}(1 - f^1f^2) - V_{01}f^1f^2) + p(h_1)(V_{10}(d^1d^2 - 1) + V_{11}d^1d^2)$$

5. The fifth strategy is given by

$$p(H_1|\tilde{s}) = \begin{cases} 1, & \tilde{s} \in \{(1, 1), (0, 1), (1, 0)\} \\ x_5, & \tilde{s} = (0, 0) \end{cases} \qquad (7)$$
Here, we label this case as the very optimistic case. Apart from $\tilde{s} = (0,0)$, the operator in all other instances determines that there is a target present. When both sensors do not report, then it is left up to the operator to decide whether they could have both missed a target or not. Combining (1) and (2), we have that the expected value is:

$$ 
\begin{aligned}
E(x_5) = x_5 & (p(h_0)(V_{00} + V_{01})(f^1 - 1)(1 - f^2) \\
& +p(h_1)(V_{11} + V_{10})(1 - d^1)(1 - d^2)) \\
& +p(h_0)((V_{00} + V_{01})(1 - f^1)(1 - f^2) - V_{01}) \\
& +p(h_1)((V_{11} + V_{10})(d^1 - 1)(1 - d^2) + V_{11})
\end{aligned}
$$

We note of course that for all decision strategies $E(x)$ models poor performances as $x$ varies. Of course, we remark that it is obvious that when all the values, target, detection and false alarm probabilities are fixed then the value of $x$ that maximises $E(x)$ will be either 0 or 1. However, we recall that from the beginning the operator was given some discretion to use his intuition within some constraints to make some decisions. If we next take the worst case scenario and assume that the operator has performed poorly, we then wish to mitigate his poor decision-making. And because we only know his choice of $x$ will be within a given range and with a certain distribution, we have to consider all the possible variable values (or at least a large percentage of them) of $x$ and all the corresponding $E(x)$. Lastly, we note that we assumed that when an operator effectively uses his intuition, then for his choice of $x$ and a given decision strategy, the expected decision value of his decision-making process will be greater than $E(x)$.

### 3 Analysis and Selection

Throughout the rest of the paper, we have assumed the distribution of $x$ to be uniform. However, the focus of this paper is to develop a robust intelligent decision support system, with the techniques developed here applicable to any distribution of $x$. Now, for the uniform distribution, we can obtain a closed form solution for the mean and standard deviation using simple linear regression theory [10]. That is, the centre of mass (average value) is simply given by $\bar{E} = E(x = 0.5)$ and the standard deviation of $E(x)$ is given by $ms$, where $m$ is the slope of the line and $s$ is the standard deviation of the uniform distribution (note that the correlation coefficient in this case is 1). Thus, in this case, the variance of the expected values is given by $m^2/12$. Thus, we can select a strategy by making use of Chebyshev's inequality [10] which we list next.

**Theorem 1.** If the random variable $X$ has mean $\mu$ and variance $\sigma^2$, then for every $k \ge 1$,

$$ P(|X - \mu| \ge k\sigma) \le \frac{1}{k^2} $$
This theorem, which applies to all distributions, in effect states that at least $(1 - 1/k^2) \times 100\%$ of all the variable values lie within $k$ standard deviations of the mean. A weaker implication of this is that at least $(1 - 1/k^2) \times 100\%$ of the variable values are greater than $\mu - \sigma k$. In our case, for the uniform distribution, this is given approximately by $E(x = 0.5) - 0.29mk$. For other distributions, if a closed form solution is not available, then a Montecarlo simulation could be used to determine a set of $x$ values, from which the expected values of each decision strategy can be calculated along with their means and variances.

This methodology ensures that the expected value and variance of the chosen decision strategy is not only analytically tractable, but is also best overall when an operator's performance deteriorates. Note that in making his decision as to whether there is a target or not, the operator is not directly using any information from other sources, and at most only drawing on his previous experience of similar situations or "gut feeling". We end this section by briefly outlining the algorithm.

## 3.1 Algorithm

We now propose to outline a possible way this technique might be employed.

**Step 1** Obtain sensors' performance parameters, target probability and reward and cost values of object(s) of interest

**Step 2** From a range of available decision strategies, plot the graphs of expected value formulae.

**Step 3** Select the strategy which is optimal by using Chebyshev's inequality. That is, suppose we wish to determine the largest lower bound for at least $q\%$ of the values in the decision strategy. Then $k = \sqrt{100/(100-q)}$ and so for each decision strategy, this value, which we call the decision figure, is given by $E(0.5) - 0.29m\sqrt{100/(100-q)}$

An obvious application of the decision strategies listed here would be in offline processing of imagery with reports from sensors. That is, suppose we have an image with some sensor reports, indicating they are either from 1 or both sensors simultaneously. The operator (using for instance strategy VP) is then asked to make judgement on whether there is an actual target there or not, but only when both sensors report. Clearly, we do not know what value of $x$ he will choose with the only information regarding $x$ is that it is uniformly distributed. If we take a risk mitigation approach, then we can model the operator's performance in terms of $E(x)$; a model of poor operator performance as defined here. We extend this process to all the other decision strategies and then apply the methodology described with the aim of choosing the most robust decision strategy when performance deteriorates. That is, we wish to find the decision strategy with the largest lower bound as determined by (1) that corresponds to at least
$q\%$ of the possible $x$ values. Such a strategy is then deemed to be optimal for the purposes of this paper.

## 4 Examples

We now present some examples that illustrate how the reward and cost values can influence which decision strategy to choose, given some fixed sensor parameters and target probabilities. For the purposes of this example, we have assumed that $x$ is uniformly distributed, however, the same methodology can be used for any distribution. For instance, for a normal distribution, we simply perform some Montecarlo sampling to get some values of $x$ and the corresponding values of $E(x)$. Then, we calculate the mean and variance of the expected value samples, thus allowing Chebyshev's inequality to be employed. In all cases we have let $f^1 = f^2 = 0.35$, $d^1 = d^2 = 0.75$ and the probability of a target is 0.175. We present 4 examples with different reward and cost values. For Table 1, we let VP, P, U, O, VO represent the very pessimistic, pessimistic, unbiased, optimistic and very optimistic decision strategies as defined above. Note that the reward and cost values labelled neutral (N), airport (A) and gain (G) were obtained from [3]. In N, both types of errors were given the same value as were both types of correct decision. The second scheme was originally called "airport". Here, this could be the reward scheme of a bomb hidden in a suitcase. The cost of missing such bomb would be prohibitively expensive compared to false alarms that resulted in a relatively quick manual inspection. The last scheme, denoted by D for deterrent, is indicative of what would happen if a security guard was employed at the exit of a shop to inspect bags. Here, the cost of employing such security guard would be perhaps much higher than any potential gains from catching someone who is trying to steal an item. However, anyone who is falsely stopped and accused of stealing an item could be upset enough to never shop in that store again thereby losing a customer and potential business.

<table><caption>**Table 1.** Means (E) and Variances (V) of examples with the same sensor parameters but varying reward and cost values</caption><thead><tr><th rowspan="3">Ex.</th><th colspan="4">Rewards/Costs</th><th colspan="8">Decision Strategy</th></tr><tr><th colspan="4">Values</th><th colspan="2">VP</th><th colspan="2">P</th><th colspan="2">U</th><th colspan="2">O</th><th colspan="2">VO</th></tr><tr><th>V<sub>00</sub></th><th>V<sub>01</sub></th><th>V<sub>10</sub></th><th>V<sub>11</sub></th><th>E</th><th>V</th><th>E</th><th>V</th><th>E</th><th>V</th><th>E</th><th>V</th><th>E</th><th>V</th></tr></thead><tbody><tr><td>N</td><td>1</td><td>50</td><td>50</td><td>1</td><td>-7.99</td><td>0.0015</td><td>1.61</td><td>21.2</td><td>1.54</td><td>21.0</td><td>-24.6</td><td>90.8</td><td>-32.5</td><td>24.7</td></tr><tr><td>A</td><td>1</td><td>50</td><td>900</td><td>100</td><td>-110.0</td><td>725.1</td><td>228.2</td><td>1630</td><td>274.8</td><td>180.0</td><td>-43.6</td><td>131.0</td><td>-20.3</td><td>3.90</td></tr><tr><td>G</td><td>1</td><td>50</td><td>50</td><td>950</td><td>38.7</td><td>725.1</td><td>79.5</td><td>1630</td><td>126.1</td><td>180.0</td><td>105.2</td><td>131.0</td><td>128.4</td><td>3.90</td></tr><tr><td>D</td><td>900</td><td>225</td><td>50</td><td>50</td><td>681.8</td><td>898.8</td><td>491.5</td><td>22500</td><td>439.5</td><td>1440</td><td>226.5</td><td>54200</td><td>18.6</td><td>12700</td></tr></tbody></table>

For instance, for example N, if we want to select the strategy which gives the largest decision figure for at least 75% of the variable values ($k = 2$), then,
through Chebyshev's inequality, the pessimistic strategy should be chosen. But, if instead we want to select the strategy which gives the largest decision figure for at least 97% of the variable values ($k = 6$), then the neutral strategy should be selected.

<figure><img src="image_2.png" /><figcaption>Fig. 1. Example with $f^1 = f^2 = 0.35$, $d^1 = d^2 = 0.75$ and target probability of 0.175 with varying reward/cost values</figcaption></figure>

# 5 Concluding Remarks

In this paper, we have derived decision strategies from multisensor outputs which combine an operator's intuition for some sensors' outputs with deterministic decisions in other scenarios. The strategies listed here are by no means the only ones and combinations using 2 or more of the options listed above are possible. More importantly, we have outlined a method for selecting the most robust decision strategy (which combine determinism and intuition) when intuition fails and the operator's decision-making process thus deteriorates.

For more than 2 sensors, these schemes can be easily adapted. However, the number of possible sensors' outputs grows from 4 to $2^N$ if N sensors are employed. Through the use of examples, we have shown how different reward and cost value schemes can influence the choice for the most desirable strategy. If the variable which represents the operator's intuition is uniformly distributed then this is simply calculated by discretizing the probability range and then selecting
the most robust strategy as explained above. But for other distributions of $x$, a Montecarlo approach might be required to generate the distribution of expected values of the decision strategies. The most robust decision strategy can then be chosen after direct calculation of the appropriate mean and variance measures, and employing Chebyshev's inequality as given above. Of course, this inequality does not provide the tightest bounds for all distributions, but its advantage lies in its universality and ease of implementation. Of course, we note that although our examples were theoretical, they were, however, realistic in terms of the likely reward and cost values for the scenarios mentioned. Other tests such as ANOVA could also be employed to see if two or more distributions, corresponding to different decision strategies, significantly differ from one another. Lastly, the types of decision strategies and algorithm listed here could also be implemented in other critical environments such as hospital and business theatres. Potential future work could lie in examining other types of distributions.

## References

1. Klein, G.: Intuition at Work. Doubleday (2003)

2. Green, D. G. and Swets, J. A.: Signal Detection Theory and Psychophysics. Peninsula Publishing (1988)

3. Navalpakkam, V., Koch, C. and Perona, P.: Homo Economicus in Visual Research. J. Vision, 9(1), No. 36, 1-13 (2009)

4. Azuma, R.; Daily, M. and Furmanski, C.: A Review of Time Critical Decision Making Models and Human Cognitive Processes. Proc. 2006 IEEE Aero. Conf. (2006) (online).

5. Horvitz, E. and Barry, M.: Display of Information for Time-Critical Decision Making. Proc. 11th Conf. on Uncertainty in Artificial Intelligence 296-305 (1995)

6. Keeney, R. L. and Raiffa, H.: Decisions with Multiple Objectives. John Wiley & Sons, Inc. (1976)

7. Anderson, J.: Cognitive Architectures in a Rational Analysis. in K. VanLehn (ed.), Architectures for Intelligence. Lawrence Erlbaum Associates, 1-24 (1991)

8. Straker, D. Changing Minds. Syque Press (2003)

9. Friedman, S.: Learning to Make More Effective Decisions: Changing Beliefs as a Prelude to Action. Qualitative Market Research: The Learning Organisation, 11(2), 110-128 (2004)

10. Christensen, R.: Analysis of Variance, Design and Regression: Applied Statistical Methods, Chapman and Hall (1996).