Evaluating the discriminatory power of EQ-5D, HUI2 and HUI3 in a US general population survey using Shannon’s indices

Objectives To compare quantitatively the discriminatory power of the EQ-5D, HUI2 and HUI3 in terms of absolute and relative informativity, using Shannon’s indices.
Introduction
The need for assessing health-related quality of life (HRQL) has brought forth hundreds of HRQL instruments, both generic and disease-specific [1, 2]. Generic instruments fall into two main categories: (1) preference-based health classification systems, and (2) non-preference based measures, sometimes referred to as health profile or psychometric measures [1, 3, 4]. Preference-based classification systems, also referred to as multi-attribute utility instruments (MAUIs) are standardized health state classifications that can be used to obtain a single summary index (utility score) or so-called preference weight for different health states. At the core of any MAUI is a classification system consisting of multiple attributes (dimensions) with ordered levels for each dimension. Most MAUIs are generic and aim to cover the full spectrum of disease and disability. MAUIs are widely used as measures of health outcome and are applied in clinical and economic evaluation (to calculate QALYs) and in population health surveys. Three widely used MAUIs are the EQ-5D, the Health Utilities Index Mark 2 (HUI2) and the Health Utilities Index Mark 3 (HUI3) [5–7]. All three instruments have shown acceptable psychometric properties as established by conventional measures [8, 9].
Feasibility, reliability, validity and responsiveness are important measurement properties in MAUIs, just as they are in non-preference based HRQL and health status measures such as the SF-36. However, these properties may be operationalized differently in MAUIs compared to non-preference based measures [4, 10, 11]. An underlying property to the concepts of reliability, validity and responsiveness is the ability of an instrument to discriminate between (‘true’) different levels of health. This requires a MAUI to define the full range of potential health states, and to be sensitive over this range. A necessary measurement property for any health status measure (including MAUIs) is the ability to discriminate among people at a single point in time. This property is sometimes referred to as sensitivity or, more accurately: “discriminatory power” [12–14].
Guyatt et al. (1992) proposed a reliability coefficient as a suitable statistic to express discriminatory power [15]. Reliability essentially reflects two different concepts: (1) consistency, e.g. between raters (inter-rater reliability) or over time (test-retest reliability), and (2) discriminatory power: the ability of an instrument to discriminate among people [16]. We propose Shannon’s indices of informativity as suitable measures that solely reflect discriminatory power [17].
Discriminatory power of MAUIs is usually investigated in an informal and partial manner by examining the frequency distributions, e.g. for floor or ceiling effects [12, 13, 18–20]. Shannon’s indices are suitable to assess discriminatory power in MAUIs for two reasons: first, they are theoretically based and second, they incorporate the frequency distribution across all categories of a MAUI’s health status classification system (not just the highest and lowest categories, as is the case with ceiling and floor effects).
Our aim is to investigate the discriminatory power of the EQ-5D, HUI2 and HUI3 in a general population sample, as expressed by Shannon’s indices. Informativity was assessed separately by dimension and by MAUI as a whole.
Methods
Data
A publicly available dataset was used (at http://www.ahrq.gov/rice/), resulting from the US EQ-5D valuation study [21, 22]. Collected data consisted of self-completed EQ-5D and HUI2/3 data from a sample of the general adult US population, with an over-sampling of Hispanics and non-Hispanic Blacks. The HUI2/3 data were collected using a standardized 15-item questionnaire, from which HUI2 and HUI3 health profiles were extracted using available recoding algorithms [23]. Only the responses of 3,691 respondents who had no missing data on any of the three instruments were included in this study (91.2% of the total number of respondents).
Instruments
The EQ-5D descriptive system consists of 5 dimensions (items) with 3 levels each, logically defining 243 unique health states (permutations). The HUI2 was originally developed to assess outcomes in survivors of cancer in childhood and contains 6 dimensions (excluding the original HUI2 dimension of fertility) with 4–5 levels per dimension. The HUI3, originally developed for a general population health survey in Canada, has 8 dimensions with 5–6 levels per dimension. The HUI2 and HUI3 descriptive systems define 8,000 and 972,000 unique health states, respectively [6]. Table 1 compares the 5 dimensions common to at least two of the classification systems: Mobility/Ambulation; Anxiety/Depression/Emotion; Pain/Discomfort (EQ-5D; HUI2; HUI3); Self-Care (EQ-5D; HUI2); and Cognition (HUI2; HUI3).Table 1Level descriptions for common dimensions between EQ-5D, HUI2 and HUI3EQ-5DHUI2HUI3MobilityMobilityAmbulationNo problems in walking aboutAble to walk, bend, lift, jump, and run normally for ageAble to walk around the neighbourhood without difficulty, and without walking equipmentSome problems in walking aboutWalks, bends, lifts, jumps, or runs with some limitations but does not require helpAble to walk around the neighbourhood with difficulty; but does not require walking equipment or the help of another personConfined to bedRequires mechanical equipment (such as canes, crutches, braces, or wheelchair) to walk or get around independentlyAble to walk around the neighbourhood with walking equipment, but without the help of another personRequires the help of another person to walk or get around and requires mechanical equipment as wellAble to walk only short distances with walking equipment, and requires a wheelchair to get around the neighbourhoodUnable to control or use arms and legsUnable to walk alone, even with walking equipment. Able to walk short distances with the help of another person, and requires a wheelchair to get around the neighbourhoodCannot walk at allSelf-careSelf-careNo problems with self-careEats, bathes, dresses, and uses the toilet normally for ageSome problems washing or dressing selfEats, bathes, dresses, or uses the toilet independently with difficultyUnable to wash or dress selfRequires mechanical equipment to eat, bathe, dress, or use the toilet independentlyRequires the help of another person to eat, bathe, dress, or use the toiletPain/DiscomfortPainPainNo pain or discomfortFree of pain and discomfortFree of pain and discomfortModerate pain or discomfortOccasional pain. Discomfort relieved by non-prescription drugs or self-control activity without disruption of normal activitiesMild to moderate pain that prevents no activitiesExtreme pain or discomfortFrequent pain. Discomfort relieved by oral medicines with occasional disruption of normal activitiesModerate pain that prevents a few activitiesFrequent pain; frequent disruption of normalactivities. Discomfort requires prescription narcotics for reliefModerate to severe pain that prevents some activitiesSevere pain. Pain not relieved by drugs and constantly disrupts normal activitiesSevere pain that prevents most activitiesAnxiety/DepressionEmotionEmotionNot anxious or depressedGenerally happy and free from worryHappy and interested in lifeModerately anxious or depressedOccasionally fretful, angry, irritable, anxious, depressed, or suffering "night terrors"Somewhat happyExtremely anxious or depressedOften fretful, angry, irritable, anxious, depressed, or suffering "night terrors"Somewhat unhappyAlmost always fretful, angry, irritable, anxious, depressedVery unhappyExtremely fretful, angry, irritable, anxious, or depressed usually requiring hospitalization or psychiatric institutional careSo unhappy that life is not worthwhile CognitionCognitionLearns and remembers school work normally for ageAble to remember most things, think clearly and solve day to day problemsLearns and remembers school work more slowly than classmates as judged by parents and/or teachersAble to remember most things, but have a little difficulty when trying to think and solve day to day problemsLearns and remembers very slowly and usually requires special educational assistanceSomewhat forgetful, but able to think clearly and solve day to day problemsUnable to learn and rememberSomewhat forgetful, and have a little difficulty when trying to think or solve day to day problemsVery forgetful, and have great difficulty when trying to think or solve day to day problemsUnable to remember anything at all, and unable to think or solve day to day problems
Shannon’s indices: background and properties
The Shannon index, named after Claude Shannon who is considered to be the founder of information theory, was initially developed to separate noise from information carrying signals in telecommunication systems [17]. The Shannon index is also known as the Shannon–Weaver index because of Warren Weaver’s contribution to Shannon’s original paper, and as the Shannon–Wiener index named after Norbert Wiener who independently developed a concept similar to Shannon’s [24, 25]. The Shannon index has been applied in a variety of fields, ranging from ecology (as a measure of biodiversity) to psychology, record linkage and molecular biology (genetic diversity) [26–30].
In information theory, the information of a signal is distinguished from the meaning or the semantic content of a signal. Rather, the information is quantified and is identified with uncertainty. Informativity is dependent on the number of classes (e.g. bits or response options) and the distribution of the observations (the ‘signal’) among classes. For classifications, this implies that if one would want to develop a useful (informative) distinction between, say, European countries, distinguishing between Scandinavian and non-Scandinavian countries would be far less informative than distinguishing between Northern, Western, Eastern and Southern European countries. Note that the latter classification not only contains more categories but the countries are also more evenly distributed among categories.
The Shannon index is defined as:
where H′ represents the absolute amount of informativity captured, C is the number of possible categories (levels or permutations in this study), and pi = ni/N, the proportion of observations in the ith category (i = 1,...,C), where ni is the observed number of scores (responses) in category i and N is the total sample size [17]. Any log base can be used, as long as one is consistent. Using log base 2, as did Shannon, allows the interpretation of the resulting units as bits per individual. The higher the index H′ is, the more information is captured by the system. In case of a homogeneous (rectangular) distribution, i.e. ratings are evenly distributed among categories (pi = p* for all i), the optimal amount of information is captured and H′ has reached its maximum (H′max) which equals log2C. If the number of categories (C) is increased, H′max increases accordingly but H′ will only increase if the newly added categories are actually used. The variance of the Shannon index is defined as [31]:
Accordingly, standard errors and 95% confidence intervals can be calculated.
The Shannon index combines the absolute information content as expressed by the number of categories with the extent to which the information is evenly spread over these categories. Shannon’s Evenness index (J′) exclusively reflects the latter component, i.e. the rectangularity of a distribution. This measure was first proposed by Lloyd and Ghelardi [32]; Shannon already referred to it as relative entropy and Pielou termed the concept ‘evenness’ [17, 33]. Shannon’s Evenness index (J′) is defined as: J′ = H′/H′max, which expresses the use of the system (H′) given its potential (H′max). Shannon’s index H′ can be considered as an expression of the absolute informativity of a system whereas Shannon’s Evenness index J′ expresses the relative informativity of a system or ‘evenness’ of a distribution, regardless the number of categories.
Two alternative measures of (bio)diversity are the Simpson and the Brillouin index. We used the Shannon index, since the Brillouin index is dependent on sample size and the Simpson index gives very little weight to categories that are rarely occupied [26, 34, 35].
Shannon indices applied to MAUIs
The basic characteristics of Shannon’s indices which make them suitable to reflect discriminatory power have been documented and are explained as follows. In an item where a response option has a very high (or low) endorsement, e.g. p is over 0.95 (or under 0.05), one learns very little because one can predict with more than 95% certainty what the answer will be. In other words, there is very little information being transmitted. Conversely, the maximum amount of information (uncertainty) is being transmitted when, in an item with two response options, p is 0.50 for each response option. As described above, this characteristic of an even distribution underlies the Shannon indices. In case of an even distribution, the item (dimension) is being most efficiently used, which means that the discriminant ability of the level descriptors is maximal.
The Shannon indices can be calculated by dimension separately or by MAUI as a whole. To calculate Shannon’s indices by dimension, levels are treated as categories, so C represents the number of levels (L), pi is the proportion of responses of the ith level, and H′max equals log2L. Suppose the EQ-5D Mobility dimension is scored by 10 respondents: no problems (n = 6), some problems (n = 3) and confined to bed (n = 1). Shannon’s index for Mobility is calculated as H′ = –((0.6 log2 0.6) + (0.3 log2 0.3) + (0.1 log2 0.1)) = 1.30 and H′max = log23  =  1.58, so J′ = 1.30/1.58 = 0.82.
Figure 1 illustrates the difference between absolute and relative informativity (H′, evenness J′) relative to the number of levels (L) in a series of hypothetical health classification systems designed to describe the same underlying dimension. For illustrative purposes we consider only one dimension. Figure 1a shows two distributions of responses corresponding to two different classification systems, both of which have 3 levels; one system results in a skewed distribution while the other results in a rectangular distribution. Assuming these responses are obtained within the same population, the system that yields the rectangular distribution is superior in discriminating between patients and the Shannon indices have both reached their maximum values. Figure 1b illustrates the concept of relative informativity. The left panel shows the same skewed distribution as depicted in Figure 1a, the right panel shows the same distribution of responses but now as it results from a 5 level classification system in which levels 2 and 4 are unused. Absolute informativity (Shannon’s H′) remains unchanged but J′ decreases, expressing lower relative informativity. Clearly, adding 2 extra levels that do not represent anyone in the population (no individual shifts from a current level to any of the new levels) does not lead to a gain in absolute informativity (H′) while the potential of a 5 level system is underutilized, compared to a 3 level system, which is expressed by a lower J′. So why not use just the Shannon Evenness index? Figure 1c shows the added value of absolute informativity (the H′ index). If the 3 and 5 level systems both yield rectangular distributions, evenness J′ will be the same but obviously H′ increases since the 5 level system is much more refined in discriminating between patients.Fig. 1Examples of Absolute Informativity (H′) and Relative Informativity (J′) with Skewed and Rectangular Distributions in a 3 Level System and 5 Level System
To calculate Shannon’s indices by instrument as a whole, permutations are treated as unique categories (e.g. 243 categories for EQ-5D), so C is the number of permutations (Pmax), pi is the proportion of the ith permutation, and H′max now equals log2Pmax.
Since the number of observations in our study (N = 3,691) is lower than the number of theoretically possible permutations in HUI2 (8,000) and HUI3 (972,000), maximum informativity (H′max) in HUI2 and HUI3, and consequently maximum relative informativity J′ cannot be reached a priori. Therefore, Shannon’s indices by MAUI as a whole were calculated using an estimation approach. Assuming that the current sample is representative, subsamples of the original set of observed health states were drawn in order to estimate the number of different health states in hypothetical populations of 1, 10, and 100 million respondents, by means of extrapolation. This procedure was repeated for different proportions of the population in relation to the number of health states (e.g. 11 different EQ-5D health states accounted for a 90% proportion of the respondents), in order to estimate the shape of the frequency distribution in the hypothetical populations of 1, 10, and 100 million respondents. Finally, Shannon’s H′ and J′ could be calculated (details can be obtained from the authors).
Results
The mean age of the respondents was 42.9 years (range: 18.0–99.3 years), with 42.2% of the respondents being male. White (non-Hispanic) respondents were 1,435 (38.9%), non-Hispanic blacks were 1,018 (27.6%) and Hispanic were 1,100 (29.8%).
Table 2 shows the frequencies of responses to the EQ-5D, HUI2 and HUI3 dimensions. The dominant response was ‘no problems’ (level 1) for all dimensions in all instruments, with a proportion larger than 90% for 1 out of 5 dimensions in EQ-5D (Self-Care), 1 out of 6 in HUI2 (Self-Care) and 3 out of 8 in HUI3 (Hearing, Speech, Dexterity). In all EQ-5D and HUI2 dimensions, frequencies decreased with increasing level severity. In the HUI3 Cognition dimension however, more respondents reported problems at level 3 (17.9%) and level 4 (7.4%) than at level 2 (4.1%). Although small, these differences also occurred in the HUI3 Vision and HUI3 Hearing dimensions.Table 2Frequency distribution (%) of responses to the EQ-5D, HUI2 and HUI3 instruments (N  =  3,691) Level 1Level 2Level 3Level 4Level 5Level 6EQ-5DMobility82.1717.530.30–––Self care95.584.010.41–––Usual activities84.8813.571.54–––Pain/Discomfort61.2834.714.01–––Anxiety/Depression73.8623.572.57–––HUI2Sensation44.5443.5410.761.16––Mobility87.248.483.600.680.00–Emotion69.2027.851.820.650.49–Cognition68.3629.941.630.08––Self-care96.642.950.190.22––Pain48.1740.947.102.980.81–HUI3Vision48.5047.871.002.470.030.14Hearing94.580.921.521.650.301.03Speech92.684.822.030.430.03–Ambulation87.248.482.551.060.510.16Dexterity92.445.820.790.700.140.11Emotion72.5022.323.741.160.27–Cognition68.364.1517.857.372.190.08Pain49.3433.7311.464.011.46–
Figure 2 shows absolute informativity (Shannon’s H′) and relative informativity (Shannon’s Evenness J′) of the common dimensions among the three instruments. Absolute informativity (H′) was highest for HUI3 in all common dimensions, with largest differences between HUI3 and the other two instruments in the dimensions Pain/Discomfort (0.52 compared to EQ-5D; 0.15 compared to HUI2) and Cognition (0.41 compared to HUI2).Fig. 2The Shannon Index (H′) and the Shannon Evenness Index  for the common dimensions between EQ-5D, HUI2 and HUI3: comparison by dimension. NA = not available;  Confidence intervals had an average range of 0.0012 (H′) and of 0.00081 (J′)
Relative informativity (J′) was highest for EQ-5D in all common dimensions, with largest differences with the other two instruments in the dimensions Mobility/Ambulation (0.14 compared to HUI2; 0.16 compared to HUI3) and Anxiety/Depression/Emotion (0.14 compared to HUI2; 0.13 compared to HUI3).
Table 3 shows Shannon’s indices by classification system as a whole. The EQ-5D, HUI2 and HUI3 descriptive systems distinguished 91, 322, and 694 observed different unique health states, accounting for 37.4%, 4.0%, and 0.07% of all possible permutations, respectively. The estimation procedure indicated that absolute informativity was highest for HUI3 (range 10.96–13.36), followed by HUI2 (range 8.57–9.48), and lowest for EQ-5D (range 6.24–6.41). Relative informativity was highest in EQ-5D (range 0.79–0.81), followed by HUI2 (range 0.66–0.73), and lowest for HUI3 (range 0.55–0.67).Table 3Shannon’s index (H′) and Shannon’s evenness index (J′) for EQ-5D, HUI2, and HUI3: Comparison by instrument EQ-5DHUI2HUI3Pmax (permutations)2438000972,000Observed health states91322694H′max7.9212.9719.89EstimationH′J′H′J′H′J′N = 1,000,0006.240.798.570.6610.960.55N = 10,000,0006.370.809.120.7012.290.62N = 100,000,0006.410.819.480.7313.360.67
Discussion
We compared the discriminatory power of the EQ-5D, HUI2 and HUI3 in the general population, using Shannon’s indices of absolute and relative informativity, for each dimension separately and by MAUI as a whole.
As might be expected in a general population sample, most respondents reported no problems on all dimensions and there were fewer responses with increasing level severity. An exception is HUI3 Cognition, where respondents reported more problems on levels 3 and 4 than on level 2. This is probably due to the fact that this dimension is not unidimensional, and levels 2 and 3 are conceptualized parallel rather than ordinal. That is, HUI3 Cognition level 2 focuses on problems in thinking and problem solving, level 3 addresses problems in remembering, whereas level 4 combines the problems mentioned in levels 2 and 3.
Absolute informativity by dimension was highest for the HUI3 descriptive system. EQ-5D appears to underperform in the Pain/Discomfort dimension. Moreover, EQ-5D appears to miss a considerable ‘amount’ of disability: 61.3% of the population indicated to have no problems on EQ-5D, against 48.2% on HUI2 and 49.3% on HUI3 (Table 2). Shannon’s H′ ‘translated’ this difference adequately (Figure 2). Apparently, for this population, the EQ-5D would benefit from more levels on the Pain/Discomfort dimension. Regarding the Cognition dimension, the difference in absolute informativity between HUI2 and HUI3 might be explained by the 2 extra levels in HUI3, but the higher J′ value in HUI3 suggests an alternative contributive factor. One explanation may be that HUI3 Cognition is not unidimensional and more sensitive to mild problems (levels 2–4) than HUI2 Cognition (level 2). Another explanation could be that the difference is due to currently suboptimal recoding algorithms.
For relative informativity by dimension, the EQ-5D descriptive system showed superior results in Mobility/Ambulation, Self-Care and Anxiety/Depression/Emotion. The large differences in Mobility/Ambulation could be due to a relatively large leap in the grading of the level descriptions in HUI3 Ambulation, where the difference between level 1 (‘without difficulty’) and level 2 (‘with difficulty’) can be considered disproportionately large in a 6 level dimension. The same leap from level 1 (‘normal’) to level 2 (‘with difficulty’) occurs in HUI2 Self-Care. We found that the 3 level EQ-5D Self Care outperformed the 4 level HUI2 Self-Care in both absolute and relative informativity (Fig. 2), which is probably due to the severe grading of level 2 in HUI2. The difference in relative informativity between EQ-5D and the HUI instruments in Anxiety/Depression/Emotion is probably due to the 2 extra levels in HUI2 and HUI3 that are rarely endorsed.
Overall, performance in terms of informativity of EQ-5D, HUI2 and HUI3 of the common dimensions varies over dimensions. The Pain/Discomfort dimension of EQ-5D, but perhaps also other dimensions, might benefit from an extension to 4 or 5 levels. HUI2 and HUI3 might benefit from more sensitive grading terms in their level descriptions, especially the ‘threshold’ level 2, in Ambulation (HUI3) and Self-Care (HUI2).
When assessing informativity by instrument, HUI3 shows the best results on absolute informativity but the lowest on relative informativity while EQ-5D shows highest relative informativity and lowest absolute informativity. HUI2 seems to be the optimal compromise. The importance of differences in the Shannon indices ultimately requires empirical evidence over a wider range of populations, conditions and instruments, including evidence on discriminant validity.
As Shannon’s indices are new in the field of health status measurement, some methodological issues need to be addressed, taking into account that their principal focus is on classifications with mutually exclusive categories, rather than conventional (health status) measures which by design contain multiple partially overlapping items.
The Shannon indices share some properties with reliability coefficients. Like reliability indices, they express discriminatory power. Furthermore, they are also non-dimensional, i.e. they have no relation to the content, meaning or clinical relevance of what the instrument aims to measure, which make them suitable for comparability, between instruments as well as between populations. However, reliability reflects two different concepts: discriminatory power as such, and consistency, e.g. consistency between raters (inter-rater reliability) or consistency over time (test-retest reliability). This requires a repeated measurement (repetition ‘over raters’ or over time) which introduces an error component in case of a difference among the repeated measurements. Shannon’s indices solely reflect discriminatory power, and need only a single measurement. Furthermore, the Shannon indices are non-parametric measures and therefore highly suitable for nominal or ordinal measurement scales.
Since Shannon’s indices have no dimension and are independent of any external standard, a rectangular distribution is always the ideal from the perspective of informativity. When comparing the discriminatory power of similar dimensions of different MAUIs, rectangularity is always optimal as it reflects which MAUI is the most sensitive in discriminating between different persons in that particular population. This implies that one MAUI cannot be superior in varying populations (e.g. a general population and a diseased population sample). Furthermore, MAUIs are bound to score rather low on discriminatory power in a general population sample, as the extreme categories, which have to be included for coverage of the full spectrum of diseases, will not be endorsed frequently.
Previously, the common approach to investigate discriminatory power was examining the frequency distributions of responses, e.g. for ceiling or floor effects. A comprehensive, formal measure to express discriminatory power such as Shannon’s indices seems clearly superior to such a ‘face-value’ method. Furthermore, when the number of categories is large (e.g. when comparing MAUIs as a whole), it becomes very difficult to make a sound comparison by just looking at the distributions.
We have demonstrated the use of the Shannon indices to compare the discriminatory power of different MAUIs, to show which instrument is more sensitive in differentiating between levels of health in the population at hand. But they may also be used to guide the development of new, or optimization of existing MAUIs, by helping determine how many levels are efficient for each dimension. This is a particularly relevant consideration for MAUIs, since adding extra levels in a descriptive system makes it increasingly complex, and the derivation of a robust set of preference weights more challenging.
Apart from MAUIs, the Shannon indices can also be used in a wide range of other classifications in the medical domain (e.g. the Karnofsky scale, the Spitzer QL index) and in the clinical domain (e.g. the APGAR score, the Child-Pugh classification).
A practical weakness of the Shannon approach is that when the sample size is exceeded by the total number of health states described by all permutations across all dimensions of a MAUI, informativity (for the instrument as a whole) has to be estimated. This implies that using the Shannon Evenness index by instrument is not very practical when a health classification system has a large number of permutations as was the case in HUI3 (972,000 permutations). This however is not a disadvantage of the Shannon methodology per se, but also a matter of classification design (overload of dimensions with detailed response options producing an excessive amount of ‘empty’ permutations), or a practical problem (excessive data collection).
From a clinical or psychometric perspective it may seem tempting to extend any MAUI with extra levels or dimensions as it provides more clinically relevant detail generally and improves reliability. But Shannon’s indices reveal that this may not always be a prudent approach. Increasing the number of levels per dimension (or permutations in the entire system) will probably result in higher H′ values but J′ values are likely to drop, as in fact our results for HUI3 indicate. This raises the question where the balance between H′ and J′ is optimal as more categories require more extensive subsequent studies to derive utility functions for the associated classification system.
How the Shannon indices will behave in a different population, such as patient populations, remains to be investigated. So far, Shannon’s indices proved to be useful in showing weaknesses of level gradings used in EQ-5D, HUI2 and HUI3, and offers leads for improvement, establishing their practical psychometric value.

psychometrics
population health
health status
methodology
health-related quality-of-life


Poor accuracy of freehand cup positioning during total hip arthroplasty

Several studies have demonstrated a correlation between the acetabular cup position and the risk of dislocation, wear and range of motion after total hip arthroplasty. The present study was designed to evaluate the accuracy of the surgeon’s estimated position of the cup after freehand placement in total hip replacement. Peroperative estimated abduction and anteversion of 200 acetabular components (placed by three orthopaedic surgeons and nine residents) were compared with measured outcomes (according to Pradhan) on postoperative radiographs. Cups were placed in 49.7° (SD 6.7) of abduction and 16.0° (SD 8.1) of anteversion. Estimation of placement was 46.3° (SD 4.3) of abduction and 14.6° (SD 5.9) of anteversion. Of more interest is the fact that for the orthopaedic surgeons the mean inaccuracy of estimation was 4.1° (SD 3.9) for abduction and 5.2° (SD 4.5) for anteversion and for their residents this was respectively, 6.3° (SD 4.6) and 5.7° (SD 5.0). Significant differences were found between orthopaedic surgeons and residents for inaccuracy of estimation for abduction, not for anteversion. Body mass index, sex, (un)cemented fixation and surgical approach (anterolateral or posterolateral) were not significant factors. Based upon the inaccuracy of estimation, the group’s chance on future cup placement within Lewinnek’s safe zone (5–25° anteversion and 30–50° abduction) is 82.7 and 85.2% for anteversion and abduction separately. When both parameters are combined, the chance of accurate placement is only 70.5%. The chance of placement of the acetabular component within 5° of an intended position, for both abduction and anteversion is 21.5% this percentage decreases to just 2.9% when the tolerated error is 1°. There is a tendency to underestimate both abduction and anteversion. Orthopaedic surgeons are superior to their residents in estimating abduction of the acetabular component. The results of this study indicate that freehand placement of the acetabular component is not a reliable method.
Introduction
Acetabular cup position after total hip arthroplasty is correlated with the risk of dislocation, wear and range of motion [1–6]. Lewinnek et al. [7] described a safe range (5–25° anteversion and 30–50° abduction) to position the cup. Within this range the dislocation rate was 1.5% and outside this range 6.1%. Although the position of the cup is important for the prognosis and function of the hip, most surgeons place the cup without any specific guidance devices. During surgery the surgeon estimates the position of the acetabular component and decides if it meets the desired orientation before securing it.
In this study the accuracy of the surgeons peroperative estimation of the position of the cup is evaluated.
Materials and methods
All patients undergoing primary total hip arthroplasty were enrolled in the study. Patient characteristics as name, sex, age, operated side and the body mass index were recorded. All cooperating surgeons (three orthopaedic surgeons and nine residents) received a list on which the following data concerning the operation were recorded: fixation (cemented or uncemented), cup model, surgical approach, complications and the peroperative estimated anteversion and abduction of the acetabular component.
Two days after the operation two standardised X-rays were made. One anteroposterior X-ray was taken with the beam centered over the hip, the second was a plain AP radiograph centred on the symphisis, showing both hips. Radiographic cup anteversion was measured on the first radiograph, according to Pradhan [8], and cup abduction was assessed on the second by measuring the angle between the teardrop line and the line bisecting the opening of the acetabular cup. The combination of both radiographs was used to determine if a cup was in the ante- or retroverted position, as described by Flabeck et al. [9].
For both anteversion and abduction, the inaccuracy of estimation was determined by calculating the difference between peroperative estimation and postoperative X-ray measurements. Mean values and standard deviations were calculated for measured cup position, estimated cup position and inaccuracy of estimation. A comparison of these values was assessed between orthopaedic surgeons and residents, using student’s t test (P < 0.05 assumed as significant) and a multivariate linear regression analysis was applied on the data in order to investigate which factors influenced the inaccuracy of estimation.
Results
Two hundred cups in 194 patients, placed between June 2003 and May 2005, were included in the study. There were 55 males and 139 females. At time of operation the mean age of the patients was 72.4 years (34–92) with a mean body mass index of 27.5 (16.6–38.1). The cups were placed by 12 different surgeons. Eighty-five cups (42.5%) were placed by three orthopaedic surgeons and 115 (57.5%) by 9 of their residents, always under supervision of one of the surgeons.
One hundred and fifty-seven (78.5%) cups had a cemented and 43 (21.5%) an uncemented fixation; 89 (44.5%) cups were placed in the patients’ right hip and 111 (55.5%) in the left. Two surgical approaches were used. The anterolateral approach according to Mallory was used in 57 cups (28.5%), supervised by one orthopaedic surgeon and the postero-lateral approach by the other two surgeons in 143 cups (71.5%). The residents used the approach of their supervising surgeon. For the anterolateral approach according to Mallory, mean placement of the cup was in 11.8° anteversion and 47.1° abduction. Positioning of the cup differed significantly when a posterolateral approach was applied: 17.7° anteversion and 50.7° abduction (both P < 0.001). Comparing both anterolateral and posterolateral approaches, no significant differences were found for inaccuracy of estimation for either anteversion (4.9° vs. 5.5°, respectively) or abduction (5.3° vs. 6.1°).
Table 1 shows relevant data of estimations and the measurements of the acetabular component by both orthopaedic surgeons and residents. None of the cups was placed in retroversion.
Table 1Mean estimated and measured anteversion and abductionCups NAnteversion (degrees)DifferenceMean inaccuracy of estimationEstimatedMeasuredIa Orthopedic surgeon8514.514.7−0.25.2 (SD 4.5) Residents11514.716.9−2.25.7 (SD 5.0) Total20014.6 (SD 5.9)16.0 (SD 8.1)−1.45.5 Significance––––Cups NAbductionDifferenceMean inaccuracy of estimationEstimatedMeasuredIb Orthopedic surgeon8547.248.6−1.44.09 (SD 3.9) Residents11545.650.5−4.96.28 (SD 4.6) Total20046.3 (SD 4.3)49.7 (SD 6.7)−3.45.4 Significance0.0010.0430.0000.001
From the data from which Fig. 1 was constructed, it was derived that 129 cups (64.5%) were placed within 5° of the estimated abduction, for anteversion this was the case in 122 cups (61%). There is a tendency to underestimate both abduction and anteversion.
Fig. 1Results of the distribution of the difference between the estimated and measured values (degrees) for abduction (a) and anteversion (b) of the acetabular components (N)
Table 2 demonstrates the results of the chance for cup placement according to Lewinnek (5–25° anteversion and 30–50° abduction), based upon inaccuracy of estimation for the orthopaedic surgeons, their residents and the group in total. Virtual intended cup placement was set at 15° anteversion and 40° abduction. Only 56.5% of the cups were placed in this safe zone according to their measured abduction and anteversion. Table 3 shows the percentage for the group and orthopaedic surgeons and residents separately, for cup placement within different ranges (20, 10, 5, 2.5 and 1°) of an intended cup position X.
Table 2Results of the chance for cup placement within the limits according to Lewinnek (5–25° anteversion and 30–50° abduction) for the orthopedic surgeons, their residents and the entire group. Intended cup placement was virtually set on 15° anteversion and 40° abductionAnteversion (%)Abduction (%)Placement according to Lewinnek (%)Orthopedic surgeons85.793.780.3Residents80.578.963.5Group82.785.270.5Table 3Deviation in % of cup placement in between 20°, 10°, 5°, 2.5° and 1° of position X for the entire group (orthopaedic surgeons and residents): (a), the orthopaedic surgeons: (b) and the residents: (c)X ± 20 (%)X ± 10 (%)X ± 5 (%)X ± 2.5 (%)X ± 1 (%)(a) Group Abduction10085.246.82616.4 Anteversion99.982.74626.717.5 Combined99.970.521.56.92.9(b) Surgeon Abduction10093.759.33421.2 Anteversion99.985.748.327.517.6 Combined99.980.328.69.43.7(c) Residents Abduction99.878.93920.712.7 Anteversion99.880.544.526.217.4 Combined99.663.517.45.42.2
A multivariate regression analysis was applied in order to identify any factors that might be responsible for the inaccuracy of the estimation. A significant result was found for abduction and anteversion, concerning age of the patient and if the patient was operated by an orthopaedic surgeon or resident. Other factors like body mass index, sex, operated side, (un)cemented fixation of the acetabular component, model of the cup and the surgical approach did not reveal any significant differences.
Discussion
Malposition of the acetabular cup is probably the most important factor for dislocation of a total hip prosthesis. Therefore it is essential that the surgeon has maximum control over the position of the socket during the operation. Free hand positioning with the patient in a standardized position and a cup positioner with the patient and floor as reference is the routine method. Specific mechanical alignment guides were designed to add precision, as do navigation systems in the concept of CT-guided computer-assisted surgery. Surprisingly, there are very limited data on the precision rate of free hand positioning of the acetabular shell. The aim of our prospective study was to determine the accuracy of the free hand technique. Only then the presently introduced systems can be tested against this standard. The term “free hand” is not used uniformly in the literature. Either it is referred to pure manual positioning or with the aid of the acetabular cup impactor-positioner provided with the implant. In the present study, the latter definition was applied.
One comparable study is that of Saxler et al. [10] who showed, in a retrospective CT-controlled design, that 27 out of their 105 cups (25.7%) were placed within the safe zone of Lewinnek. A second study by DiGioa et al. [11] with a specific mechanical acetabular alignment guide (with A-frame) where the results were controlled peroperatively with a hip navigation system in 74 hips aimed at 45° of abduction and 20° of flexion. With their specific guide only 22% of the cups were placed within Lewinneks safe zone. Compared with these two studies our prospective data demonstrate an accuracy within the safe zone of 70.5%.
There is an elegant in vitro study by Jolles et al. [12] comparing free hand (without the help of any guide), mechanical alignment guide and computer-assisted cup placement. Onehunderd and fifty acetabular implants were placed in ten identical models of the pelvis (covered with artificial soft tissue of soft cast and foam) by ten surgeons. The mean accuracy for anteversion was 8° (5.0–10.5) for free hand with cup positioner and 4° (3.0–5.5) for abduction; with computer navigation this was 1.5° (1.0–2.0) and 2.5° (2.0–3.5), respectively. In the present in vivo study the data were 5.5° for anteversion and 5.4° for abduction with 200 acetabular shells. Jolles et al. state that their in vitro conditions of ideal approach and anatomic relations and perfect placement of the patient would favour the free hand positioning as computer-assisted surgery devices are only slightly sensitive to modifications of these parameters and inaccuracy of the freehand will be enhanced greatly in vivo. Our study proves this statement not to be the case as results are quite comparable.
Without doubt it is essential to actually place what is aimed for. Our present study shows that the 70.5% positioned within the safe zone is remarkably good for free hand placement compared with literature. But when the target is reset from the wide safe zone of Lewinnek to for instance within 5° or even 1° of error from the judged position for both abduction and anteversion, only 21.5 and 2.9% of our sockets are placed within this narrow definition of safe zone. We believe that the power of our study comes from the distinctive set up where the recorded data are a reflection of the surgeon’s perception of cup placement. To our opinion this is superior then to allow for a wide range or set a predefined target. At the actual point of cup placement a surgeon is also committed to or governed by the patient’s anatomy, which will influence the actual position. For instance when less anteversion is accepted in the socket; the stem will be given some additional anteversion.
Another important item as stipulated by DiGioia et al. [11] is the actual position of the patient in the lateral decubitus position. They demonstrated that the mean difference of pelvic orientation on the operating table with the desired position during acetabular alignment was 18° in version and 3° in abduction. This effect is not completely eliminated with the use of computer navigation as these systems are influenced by pelvic tilt [13].
Considering the estimation of abduction, significant results were found in favour of the orthopaedic surgeons compared to their residents. A learning curve as a result of experience is probably responsible for this difference. As to surgical approach, acetabular components placed posterolaterally were measured to have a significantly higher-degree of anteversion and abduction. This is obviously related with the intention to minimize the change of a dislocation for both approaches. Although both approaches take a different anatomical route to the acetabulum, visualization and presentation of the acetabulum do not seem to be important factors. The conclusion is that an anterolateral or posterolateral approach is not a distinguishing factor in the accuracy of cup placement in our series.
An important potential flaw in this investigation is the use of planar radiographs for the evaluation of true abduction and especially anteversion. Variations in pelvic flexion-extension during imaging are responsible of variations in flexion up to −26° to +10° [14]. This study concludes that radiographic measurements are not a reliable method to evaluate cup orientation, especially flexion or anteversion alignment [14]. Pradhan, however, stated that his method, as used in our study, proved to be reliable in an in vitro model in which the calculated anteversion on the X-ray was compared with the true (known) cup anteversion. In our study special attention was given for optimal positioning of the patients pelvis to minimize rotation and tilt before the X-ray was taken. From a practical point of view it is preferable to determine the actual cup position with planar radiographs rather then with CT as in most clinics the availability and the costs of a CT scan combined with the high radiation dose for the patient are reasons for not using this instrument as a routine practice.
From this prospective study it is concluded that an accuracy of 70.5% for placement of the socket with an acetabular shell imactor-positioner within Lewinnek’s safe zone and of only 21.5% within an error of 5° for both abduction and anteversion is unacceptable.
The strength of our research is that literature is extremely scarce on studies as our prospective study on free hand positioning without special aiming devices. The results of the present study appear convincing but limitations are the high-number of participating surgeons, different surgical approaches (anterolateral and posterolateral) and two methods of cup fixation (uncemented and cemented). Multivariate analysis on these factors revealed only differences between the achievements of surgeons and residents.
The consequence of our conclusion that free hand cup positioning is an unreliable method is that attention now should be focussed on the results of a randomized trial comparing the different techniques of cup positioning.

accuracy
freehand cup positioning
total hip arthroplasty


The latency for correcting a movement depends on the visual attribute that defines the target

Neurons in different cortical visual areas respond to different visual attributes with different latencies. How does this affect the on-line control of our actions? We studied hand movements directed toward targets that could be distinguished from other objects by luminance, size, orientation, color, shape or texture. In some trials, the target changed places with one of the other objects at the onset of the hand’s movement. We determined the latency for correcting the movement of the hand in the direction of the new target location. We show that subjects can correct their movements at short latency for all attributes, but that responses for the attributes color, form and texture (that are relevant for recognizing the object) are 50 ms slower than for the attributes luminance, orientation and size. This dichotomy corresponds to both to the distinction between magno-cellular and parvo-cellular pathways and to a dorsal–ventral distinction. The latency also differed systematically between subjects, independent of their reaction time.
Introduction
Neurons in different cortical visual areas respond to different visual attributes with different latencies (Livingstone and Hubel 1987, 1988; Schmolesky et al. 1998; Bullier 2001). How do such differences affect the way in which visual information guides our movements? Two main streams of visual processing have been identified within the brain; each specialized for processing a different kind of information (Trevarthen 1968; Mishkin et al. 1983; Goodale and Milner 1992; Ungerleider and Haxby 1994). A ventral stream in which information travels from V1 to the inferotemporal cortex is crucial for the identification of objects, whereas a dorsal stream passing through the posterior parietal region plays a major role in spatial vision. A widely accepted explanation for this distinction is that the dorsal stream is specialized for quickly processing spatial information to control action, whereas the slower, detailed visual analysis of other attributes that are important for recognizing objects takes place in the ventral pathway (DeYoe and Van Essen 1988; Goodale and Milner 1992; Tanne et al. 1995; Desmurget et al. 1999; Rossetti et al. 2000; Lee and van Donkelaar 2002). This view implies that not all information can be used to quickly adjust hand movements. We tested this prediction.
One of the problems with comparing experiments that investigate whether various kinds of visual information can contribute to fast control of actions is that investigators use different experimental paradigms for different visual attributes, which themselves may give different results for a single attribute (e.g., compare Pisella et al. 1998; Schmidt 2002; Brenner and Smeets 2004; Cressman et al. 2006; White et al. 2006 for the role of color in guiding hand movements). It would therefore be very useful to systematically investigate the latency of responses to various visual attributes with a single paradigm. We therefore compared responses to changes in target location for hand movements directed toward targets defined by attributes that are normally relevant for goal-directed actions (luminance-contrast, size and orientation) with ones toward targets defined by attributes that are normally more relevant for object recognition (color, shape and texture). In all cases, there were three possible target positions. The target was at one of them. Other objects (references) occupied the other two positions. In half of the trials, the target switched locations with one of the references just after the hand started moving (Fig. 1a, b).
Fig. 1Schematic overview of the experiment. a The participant holds a pen at the starting position when a target (dark square in this example) and two other objects (references, the brighter squares in this example) appeared. b In half of the trials, the target switches to one of the other locations just after movement onset. c Ten paths of one participant’s hand (P6) for a target that remained at the leftmost position in the “luminance (390%)” condition. d All ten paths of the same participant’s hand in the condition in which the target jumped from the leftmost to the rightmost position at movement onset
An issue that cannot be ignored is how conspicuous the distinction between target and reference is within each attribute; i.e., how easily the target can be distinguished from the references. When comparing responses to targets defined by the same attribute, the response becomes faster if the relevant contrast is increased (Brenner and Smeets 2003). We therefore have to show that any differences in latency that we find cannot be simply caused by differences in conspicuousness. The question is how to evaluate conspicuousness independently from the response latency.
In previous studies, this issue was resolved by equating a critical stimulus parameter (e.g. cone contrast, White et al. 2006), or by equating the stimuli in terms of an independent perceptual judgment (e.g., how conspicuous the targets are judged to be, Brenner and Smeets 2003). The first approach is not feasible in our current experiment because it is not at all clear how stimulus parameters for different attributes could be matched. The second approach is also problematic, because the visual processing for the perceptual judgment task might differ from that for the real experiment. We will therefore develop a new method to deal with differences in conspicuousness. This method, which will be developed in the results section, is based on examining responses to various targets that are all distinguished on the basis of the same attribute, but that differ in conspicuousness. We will show that beside leading to shorter latencies, more conspicuous targets also lead to shorter reaction times, higher velocities, and a steeper slope of the responses. We will then compare latencies for targets defined by different attributes when they give rise to comparable reaction times, velocities, and response slopes. Differences in latency after such matching cannot be caused by differences in conspicuousness.
Materials and methods
Participants
Twelve subjects took part in this study. Three of them were the authors and the others were colleagues and friends who were unaware of the hypothesis that was being tested. All of them had normal or corrected to normal vision. Two of the participants were left-handed. Each participant made 120 pointing movements in each of 13 different sessions. Six of the participants also made 120 pointing movements in each of 7 additional sessions in which various luminance contrasts were used. Each participant performed the sessions in a different random order. The ethics committee of the Faculty of Human Movement Sciences approved the study.
Apparatus and stimuli
Participants sat behind an A2-sized graphic tablet (Digitizer II, Wacom Ltd, Tokyo, Japan) and viewed a projection surface via a semitransparent mirror that was placed above this graphic tablet. The images were back-projected from above the projection surface. The distance between the mirror and the projection surface was identical to that between the mirror and the surface of the tablet, so that the projected image appeared to be at the surface of the tablet. Lamps underneath the mirror ensured that the participants could see their hands. The resolution of the projected image was 1,024 by 768 pixels, with 1 pixel corresponding to about 0.5 mm. The position of the pen was determined every 5 ms (200 Hz).
At the near edge of the screen was a starting point (a black, 1 cm diameter disk with a luminance of 2.3 cd/m2 as measured with a Minolta LS-110 luminance meter). At a distance of 25 cm from this point in the sagittal direction, three objects were projected next to each other on a white background (35 × 45 cm; 28.0 cd/m2). The distance between the centers of the objects was 3 cm (Fig. 1a). One of the three objects served as the target; this object differed from the other two objects (the references).
We varied the attribute in which the target differed from the references (and the magnitude of this difference) between conditions (Fig. 2). Each condition was tested in a separate session. In the control condition, the reference objects were equal to the background so they were invisible: only the dark gray target square (1 cm sides, 5.1 cd/m2) was visible.
Fig. 2Schematic representations of the stimuli in the 20 conditions (depicted with the target on the left). The luminance of both the target and the references is 5.1 cd/m² and they are both 1 cm2 gray squares unless otherwise mentioned (see “relevant details”). a The 13 conditions that were performed by all participants. Red lettering indicates that the responses depend on attributes that are normally needed to recognize objects, and blue lettering indicates that all items change. b The seven additional luminance conditions that were performed by six participants (numbers between brackets are the percentage that the references are brighter than the target)
In the luminance (390%) condition, the target differed from the references in luminance only (for further details about this and subsequent conditions see Fig. 2). There were seven additional conditions with different luminance contrasts between the target and the references. These were used to determine how responses depend on how conspicuous the distinction is between the target and the references.
In the three orientation conditions, the target and the references were rectangles (0.67 × 1.50 cm, 5.1 cd/m2). The target rectangle was orientated at an angle of 45°, 60° or 90° with respect to the parallel references. In the three size conditions the target squares had different sizes. The reference squares had surface areas of 1 cm2, as did all the objects in the other conditions. In the shape session the target was a circle, and the references were squares. In the color condition, the target was a green square (5.1 cd/m2) and the references were two red squares (5.1 cd/m2). In the texture condition, the target square was filled with ten 1-mm wide stripes, alternating between dark (2.3 cd/m2) and bright (7.9 cd/m2) ones. The average luminance of the stripes was equal to the luminance of the references (5.1 cd/m2), to ensure that participants had to analyze the texture to recognize the target.
During pilot-experiments, some of the participants had the impression (especially in the orientation and the shape conditions) that they were not responding to a difference between the target and the references, but to the change at the new target position. To investigate whether this was really an issue we performed two control conditions in which we masked such effects by also changing aspects of the references when the target changed its position. In a masked orientation condition, the references both changed their orientation by 90° when the target jumped to a new location. In a masked shape condition, the reference squares increased in size at the same moment that the target circle changed position.
Procedure
Participants started each trial by placing the pen on the starting point. After a random time (about 2 s), the target and references appeared (Fig. 1a). The task was to reach the target with the pen and stop there as soon as possible. The sum of the movement time and the reaction time was to be minimized. Participants were shown which object was the target (and which the references) before they started each session.
Within each session of 120 trials, the target appeared 40 times at each of the 3 positions. In half of the trials the target and the references remained at their initial position (unperturbed trials, Fig. 1c). In the other half of the trials the target changed its position to one of the two reference positions about 62 ms after the subject’s finger reached a velocity threshold of 0.02 m/s (perturbed trials, Fig. 1b). At the same moment the reference (that had been at that position) changed its position to where the target had been. Whenever the target changed position participants had to adjust their hand movement to reach the new target location (Fig. 1d). If a movement ended on the target within 1.2 s of the target first appearing, the participant was rewarded with auditory feedback. The 120 trials (three initial positions, change in position or not, two positions to which to change, ten replications) were presented in random order. Since different positions to which to change are irrelevant when there is no change, there were actually 20 trials in which the target remained at each initial position.
Analysis
Velocities were calculated for the interval between every two measurements by dividing the displacement of the tip of the pen by the 5 ms between the measurements. The beginning of the movement was defined as the first position after the tangential velocity reached 0.02 m/s. The end of the movement was defined as the first position after the tangential velocity fell below 0.02 m/s. To evaluate the corrections, we only used the lateral component of the velocity (parallel to the displacement of the target).
In order to isolate the responses to changes in target position, we first synchronized all the measurements relative to the moment that the target changed—or would have changed—position. We then separately averaged the lateral velocity for each combination of initial and final target position. We defined the lateral velocity in the direction of the position change as being positive; when the position did not change, we defined the direction in which it would have changed as being positive. We characterize the response to a change in target position by the additional lateral velocity: the difference in the lateral component of the velocity between the trials in which the target changed from a certain initial position to a certain final position and the ones in which it remained at the same initial position. To obtain a single response per attribute for each participant we averaged the additional lateral velocity across the six combinations of initial and final target positions. We used these curves to determine the latencies of the responses.
To estimate the latencies of the responses we approximated the initial responses by straight lines (dotted lines in Fig. 3). We used a robust method (one that is not too sensitive to the exact shape of the response near its onset) to approximate the initial response: we determined the amplitude of the peak in the average additional lateral velocity and then determined the points at which the additional lateral velocity reached 25 and 75% of this value. We consider a line through these two points as a good and robust approximation of the initial response. We considered the intersection of this line with a line at an additional lateral velocity of zero to give a reliable estimate of the latency of the reaction to the change in target position. We evaluated the influence of the conditions on the slopes of the above-mentioned lines and on their intersection points with the zero-velocity line (across and participants) using paired t-tests.
Fig. 3Mean additional lateral velocity for each attribute as a function of the time after the perturbation. The mean additional lateral velocity is the mean difference between the lateral velocity on the trials in which the target did and did not change position. To estimate the latency of the response, a line through the points at 25 and 75% of the peak velocity (see dotted lines) was extrapolated to intersect the line representing an additional velocity of 0 (black horizontal line). The time at the point of intersection was considered to be the response latency. a Averages of six participants for the luminance contrast conditions. b Averages of 12 participants for the conditions in which they all participated
The slope of the line through the 25 and 75% points of the average additional lateral velocity does not only depend on the intensity of the response on individual trials (lower acceleration results in a shallower slope). If the response does not always occur at the same time, averaging will result in an average response that has a lower peak velocity, shallower slope and longer duration than the responses in individual trials. To evaluate the shape of the response curve without the influence of variability in timing, we used a second way of synchronizing trials before determining the additional lateral velocity.
To estimate the intensity of individual responses for each condition and participant, we synchronized the lateral velocity curves of all perturbed trials at the peak lateral velocity in the direction of the new target position (irrespective of when the target position changed), and produced an additional lateral velocity curve for each subject by averaging across replications and perturbation directions. We averaged these synchronized additional lateral velocity curves across positions and across participants to investigate whether there are systematic differences between the intensity of the responses for different attributes. We averaged the curves across conditions rather than participants to investigate whether there are systematic differences between the intensity of different participants’ responses. In order to determine whether the response was proportional to the size of the perturbation, we also averaged the curves separately for all pairs of positions in which the target jumped 3 cm, and all pairs of positions in which the target jumped 6 cm.
Results
Overview of responses
On average, it took participants 365 ms to react to the appearance of the target (and the two references). The average movement time (time until participants stopped their movement on the target) was another 275 ms. Figure 1 shows one participant’s paths in one condition with the target initially positioned on the left. When the target stayed at the initial target position (Fig. 1c), the hand moved to the target along a slightly curved path (note that the lateral deviations are exaggerated due to the unequal scaling of the figure). When the target jumped at the onset of the movement (Fig. 1d), participants obviously initially moved along similar paths to those seen in Fig. 1c, but approximately half way to the target the path curved toward the new target position.
Figure 3 shows the additional lateral velocity for the various conditions (summarized in Fig. 2), averaged over all subjects. Each curve represents the difference in lateral velocity between trials in which the target did and did not change position (see “materials and methods” for details). The dotted lines are drawn through the points at which the mean additional lateral velocity curves reached 25 and 75% of their peak values. We consider the points at which these lines intersected the line at an additional lateral velocity of zero as the latencies of the responses. We used targets that only differed in luminance from the references to evaluate effects of conspicuousness (for half of our subjects). We found that lower luminance contrast (a less conspicuous target) results in both a longer latency and a shallower slope (Fig. 3a). The conclusion that we can draw from the main conditions depicted in Fig. 3b, is that both the timing and the slopes of the curves differ between conditions. Figure 3 shows an analysis over responses averaged over subjects. The rest of the analysis is done on the basis of determining slopes and latencies for each subject (and condition) separately.
Dealing with conspicuousness
It is clear from Fig. 3a that the latencies are larger for less conspicuous targets. In order to compare the latencies of responses to targets defined by different attributes, we have to take into account differences in conspicuousness. Figure 4a, b and c shows how not only the latency of the response varies with conspicuousness, but also the reaction time of the initial movements toward less conspicuous targets are longer (Fig. 4a), the slope of the response to a perturbation is shallower (Fig. 4b), and both the peak velocity of the initial movement toward the target and that of the correction are slower (Fig. 4c). All three plots yield significant correlations (P < 0.05), and the data are distributed along the fitted line with deviations that can be expected on the basis of the error bars.
Fig. 4Comparing the responses across conditions and subjects. a–c Relationship between various movement parameters for the luminance conditions. The latency and slope refer to the correction (Fig. 3). Points are averages across subjects with standard errors. Dashed lines are linear fits through these points. d–f Similar graphs for each of the conditions in which all subjects participated. Symbols in red are the conditions based on attributes for object recognition; symbols in blue are for conditions in which the change is masked by changing all items. g–i Data for individual subjects, averaged across conditions. The brightness of the symbol indicates the average response latency of the subject (bright is short latency)
Since several measures co-vary with the conspicuousness (i.e. the luminance contrast) of the target, we can use these measures to evaluate whether latency differences can be due to differences in conspicuousness. Except for the latency (the variable we are interested in), the slope is the variable that varies most clearly with conspicuousness. This is therefore the best measure to use to evaluate whether effects of condition on latency are caused by a difference in conspicuousness, followed by the reaction time (Fig. 4a) and response velocity (Fig. 4c). The correlation of the conspicuousness with the response velocity (Fig. 4c) is less clear than its correlation with the slope (Fig. 4b). This suggests that a shallower slope of the average response is not only caused by the response being less vigorous, but is also largely caused by more variability in the latency. To verify the effect of variability in latency on the slope of the response, we removed the effect of variability in the latency by averaging the responses after synchronizing them at the moment of peak lateral velocity. The responses resulting from this averaging method have almost the same slope for all conditions: 282 m/s2 with a coefficient of variation of 0.13 (see Fig. 5a). The slopes of the mean additional lateral velocity curves (Fig. 3) are not only much shallower (on average 187 m/s2), but are also much more variable between conditions (coefficient of variation 0.28). We conclude that the differences between the slopes of the mean additional lateral velocity curves are mainly caused by differences in the variability in timing between trials (rather than by differences in the intensity of the response). Thus a more conspicuous target results in less variable (Fig. 4b) as well as more vigorous (Fig. 4c) responses.
Fig. 5Magnitudes of the responses as determined by averaging all trials after synchronizing at their peak lateral velocity. a Each curve indicates a condition; color coding as in Fig. 4. The similarity between the curves indicates that the intensity of the response did not differ between conditions. b Responses of subjects. The brightness of the curve indicates the subject’s average latency (bright is short latency). c Comparing the responses for the two jump amplitudes. The additional lateral velocity for all trials in which the target jumped 6 cm was twice as large as that for the trials in which the target jumped 3 cm
Different attributes, different latencies?
Figure 4d and e shows the same relationships as Fig. 4a, b for the main experimental conditions (Fig. 2a). The dashed lines reproduce the relationship that we found for the luminance conditions (Fig. 4a, b). Participants responded to the various size and orientation conditions with a relationship between latency and conspicuousness that was similar to that for the responses to luminance-defined targets. The targets defined by the attributes color, shape and texture were not less conspicuous than the other targets (reaction times and slopes are similar), but subjects take exceptionally long to react to changes in their position (t-test, P < 0.05). These responses were about 50 ms longer than the ones indicated by the dashed lines. The movements toward the targets defined by color, shape, or texture had similar peak velocities, both of the movements themselves and of the corrections, as the movements toward other targets (Fig. 4f).
For two attributes (orientation and shape) we constructed a “masked” condition, in which not only the target changed location, but also the references changed. This prevents subjects from using the change itself as an indicator of the new target location. The latency for these two conditions is 210 ms, which is longer than for the two corresponding original conditions (paired t-test, P < 0.05). The masking decreased the velocity of the movements and of the corrections (Fig. 4f), but did not change the conspicuousness of the targets, either in terms of the reaction time (Fig. 4d) or of the slope of the average response (Fig. 4e). So the increase in latency due to the masking is not caused by masking making the targets less conspicuous, but by requiring additional processing, presumably for a renewed selection of the target. Due to the blocked design, subjects could counteract the effects of their later responses by slowing down their movements (low values for the masked targets in Fig. 4f).
Subjects and amplitudes
Before discussing how to interpret the results, we want to answer two additional questions. The first is, how does the variability between participants contribute to the results? To find this out, we plotted the data averaged over the conditions for each of the participants separately (Figs. 4g–i, 5b). We coded subjects according to the latency of their adjustments: bright curves and symbols correspond to subjects with short latencies. The intensity of the response differed much more between subjects (SD = 91 m/s2, Fig. 5b) than between conditions (SD = 37 m/s2, Fig. 5a), with no apparent correlation between the peak velocity and latency (Fig. 4i: brighter symbols are not concentrated in one region of the figure). All subjects show more or less the same pattern of latencies as we found for the average data (not shown). This was so for the different attributes as well as for the different levels of conspicuity. There is quite some variability between the subjects, but subjects that are slow in one condition are generally slow in the other conditions as well. The result is that subjects differ systematically in their latency, without any correlation with their reaction time (Fig. 4g) or with the slope of their response (Fig. 4h). This performance difference cannot easily be explained by factors such as age, visual acuity or body size.
The second question is whether the similarity of the curves in Fig. 5a means that subjects have a pre-programmed response to a perturbation that is used irrespective of the precise visual information. To answer this question, we compared the responses for the two amplitudes of the perturbation. When the target jumped 6 cm from its initial location (from one extreme position to the other), participants responded about twice as strongly as when the target jumped 3 cm (Fig. 5c). Thus, the response is proportional to the position change, and therefore clearly tailored to the visual information rather than that pre-programmed and triggered by the change.
Discussion
By considering the relationship between response latency and measures of conspicuousness (reaction time and variability in the latency), we can conclude that participants reacted with a similar latency to differences in orientation and size than to corresponding differences in luminance. The attributes color, shape and texture gave rise to approximately 50 ms longer latencies than did differences in luminance, orientation and size that were equally conspicuous. Note that these latencies are still 150 ms faster than the original reaction times. We found a similar order of fast and more slowly processed attributes across participants, which supports our idea that we can use this method to measure systematic differences between attributes in terms of the time that it takes to process them, even if participants respond differently (see different velocity profiles in Fig. 5b).
An important question in this kind of research is whether the apparent response to a particular attribute could actually be a response to a small difference in another attribute. For instance, in the color condition, participants could respond to small differences in luminance instead of to differences in color between the target and the references, since the luminance of the target and the references was not equated for individual participants. They clearly do not, because if they had done so, the response would be similar to that for one of the small luminance contrast conditions, which is clearly not the case (the slope of the response is much steeper for color).
As mentioned earlier, it takes about as long to respond to changes in position defined by target size as to those defined by target luminance. We cannot reject the possibility that the reactions to the difference in surface area between the target and the references in the size condition were actually responses to the average luminance within an area larger than that of the target itself. To be sure that this was not the case we would have had to vary the luminance of the targets and references, which we did not do in the present study.
Desmurget et al. (1999), showed that applying transcranial magnetic stimulation over the left posterior parietal cortex disrupted corrections of reaching movements of the right hand after a perturbation in target location in four of their five subjects, thereby demonstrating that the dorsal pathway is necessary for fast corrections of the hand. The fact that participants are able to respond very quickly to changes in target location when the target is defined by the attributes orientation, size and luminance (but not color, texture or shape) is therefore an indication for dorsal processing of orientation, size and luminance (but not of color, texture and shape). This division of attributes is consistent with the distinction between pathways for “what” and “where” (Mishkin et al. 1983; Ungerleider and Haxby 1994). The fact that subjects were still able to correct their movements to targets defined by color, texture or shape (albeit at a longer delay) implies that these attributes also reach the parietal cortex, although at a longer latency. This is in line with studies using electrophysiology (Murata et al. 2000; Sakata et al. 2005), positron emission tomography (Faillenot et al. 1997; Vidnyanszky et al. 2000) and functional magnetic resonance imaging (Oliver and Thompson-Schill 2003) that all suggest that the parietal cortex is involved in at least some aspects of discriminating shape. These studies were, however, not concerned with the exact timing of the activity.
Why are responses for targets defined by shape relatively slow? Assuming that all the responses that we measured are indeed mediated by the posterior parietal cortex, the dichotomy in latencies could be explained in two ways. It could be that the parietal cortex mediates the responses to some targets after initial processing of the attributes that are relevant for the “what” of the object (like color, shape and texture (Mishkin et al. 1983) in the ventral pathway. This indirect pathway is likely to result in longer processing times than the direct dorsal pathway for the “where” attributes.
Alternatively, the dichotomy may arise earlier, with the distinction between the magnocellular retinogeniculate pathway that is faster and more sensitive to luminance contrast, but is color-blind and has a low spatial resolution, and the parvocellular pathway that is color selective and has a higher spatial resolution, but is slower and less sensitive to luminance contrast (Livingstone and Hubel 1987, 1988). This distinction corresponds nicely with the dichotomy in the latencies that we found. Assuming that both the magnocellular and the parvocellular pathways provide input to the dorsal pathway could therefore account for the different latencies that we found for different attributes (Brenner and Smeets 2003), without requiring input from the ventral pathway.
Although we cannot decide between these two lines of explanation, the two masked conditions indicate what is essential for the very fast responses. Being able to identify the target on the basis of “where” attributes (size, orientation, luminance) is necessary for fast responses. However this is not sufficient, because the latency of responses to the change in location of a target defined by orientation increases by more than 50 ms if the references change their orientation. Masking changes prevents responses to the location of the change. Presumably, the target has to be found (identified) again, which requires ventral processing (according to the first explanation).
The conclusion that identifying targets by their color takes 50 ms longer than doing so on the basis of luminance seems to be in conflict with earlier results where we reported that subjects could respond within 120 ms to color information (Brenner and Smeets 2004). In that study, we compared two conditions: one with one reference object of a different color and one with no reference object (similar to the control condition in the present experiment). The main difference between the responses in these conditions was the larger variation in latency for the condition with one reference object (a more shallow slope). It seems as if the distribution of responses consisted of a small peak at short latency, and a larger peak 50 ms later (Fig. 3 in Brenner and Smeets 2004). Perhaps, the few very fast responses in the condition with the reference object were related to the very simple design of that experiment: the position to which the target could change was known in advance, so detecting the change in color at the original target location was enough to start an adequate correction. In the present experiment, the location of the new target position had to be determined on the basis of color information. That directly responding to a transient can lead to exceptional fast responses is evident from the fact that masking the jump of the oriented target by orientation changes of the references caused an increase in latency of 50 ms.
A last issue to discuss is the variability between subjects. Some subjects responded consistently later to the perturbations than others (Fig. 4g, h). The difference between the slowest and fastest subject was 70 ms; larger than the difference between the responses to targets defined by color and luminance. Such a large variation is in line with the remarkable finding that stimulating the posterior parietal cortex directly after movement onset disrupts adjustments in some subjects, but not in others (Desmurget et al. 1999). We can draw two conclusions from these variations in latency. The first is that since none of the subjects had particular complaints of clumsiness, the exact value of the latency is apparently not critical for performing every day tasks. The second is that in situations in which short latencies are important, such as when playing table tennis or boxing, some participants may perform better than others could ever hope to, due to differences between their minimal visuomotor latencies. This latter implication (and in particular whether a shorter latency can be achieved by training) is open for experimental verification.

hand movement
color
shape
visual processing
dorsal stream
target perturbation
double step