Support and stability

Sir, In a recent contribution, Goloboff et al. (2008) discuss different justifications of jackknifing (and by implication, other permutation support measures as well) as a measure of ‘‘stability’’ and as a measure of ‘‘degree of support’’. The interpretation of jackknifing as ‘‘degree of support’’ goes back to Farris et al. (1996) and is extensively discussed in a number of other papers cited by Goloboff et al. (2008). However, the references they give for the ‘‘stability’’ justification (Siddall, 2002; Hovenkamp, 2004) both provide no more than very summary statements on this topic, thus leaving it mainly to the reader to work out how the ‘‘stability’’ justification is based on actual arguments. The purpose of this note is to help the reader to form an opinion on both types of justification, mainly by reiterating a few points made by Hovenkamp (2004) in some greater detail. The point is not devoid of interest, as, ultimately, preference for a particular justification may lead to a preference for a particular method (Goloboff et al., 2003, 2008; Hovenkamp, 2006). Firstly, stability is the prima facie interpretation of any permutation support statistic. These statistics are produced by investigating the persistence of clades under conditions of changes in the dataset: changes in characteror taxon-sampling, or changes in the dataset itself. Thus they directly report stability of results under a chosen regime of change of the primary observational data. Secondly, stability equates to repeatability, a basic requirement for observations in empirical science. ‘‘We do not take even our own observations quite seriously, or accept them as scientific observations, until we have repeated and tested them’’ (Popper, 1959, Ch. 1 par. 8). Should this requirement be extended to cladograms and their constituent nodes? I think it should. Cladograms are uniquely determined by the observed distribution of character states over the terminal taxa, and the position that repeatability is sufficiently guaranteed by having exact algorithms to derive cladograms from the character observations (Lipscomb, 1998, p. 65) is mistaken. Indeed, it is the very exactness of these algorithms that detracts from their usefulness in this context. As they are guaranteed to produce exactly the same results each time the algorithm is applied, their repeated results do not increase support for acceptance (Collins, 1984; Franklin and Howson, 1984). For example, when Fleischmann and Pons (1989) reported their surprising results on cold fusion, the main question that soon arose was not whether the experiment could be replicated exactly, for instance, by asking Fleischmann and Pons themselves to repeat the readings, using exactly the same equipment as the first time and under exactly the same external conditions. Instead, most scientists expected that the results of the crucial experiment could be duplicated under slightly changed circumstances as well: by other scientists working in other laboratories, preferably using slightly different equipment (e.g., containers of different size, shape or brand, or electrodes of different size or shape). Repeating the same cladistic analysis on the same data using the same algorithm is an even more exact repetition, equivalent to copying, for a second time, the notes from Fleischmann and Pons lab journal. The results of such exact repetitions may confirm that the computer is working properly, but they cannot establish the repeatability of the results in any other sense. Thus, a meaningful repetition of a cladistic study necessitates any of a number of possible changes in the basic data, and finds those elements of a cladogram to be repeatable that persist under a specified regime of changes. Although by necessity, permutation supports are limited to the evaluation of changes that can be ‘‘reverse*Corresponding author: E-mail address: hovenkamp@nhn.leidenuniv. nl