Rejoinder: Further considerations for meta‐analysis of transformed quantities such as absolute values

In the broadest, and in my opinion (and others, Scheiner, 2016 in particular, see also Glass, 1976) most useful, definition of meta-analysis, we seek to analyse multiple results from the primary literature to make quantitative inferences about the distributions of biologically interesting parameters in nature. A range of parameters might be of interest in meta-analysis: means, variances, mean absolute values, differences between pairs of estimates, signs (e.g. positive or negative values) and so on. The considerations for inference of each of these might be very different: approaches that are useful for estimation of one sort of metaanalytic quantity (e.g. means) may be misleading when applied to other meta-analytic quantities (e.g. mean absolute values, or ‘average magnitudes’). Calsbeek & McPeek (2016) highlight various quantities that might be of interest, and question that these might not be possible to study via meta-analysis. I agree entirely that such quantities may be of interest. I disagree that they are beyond the reach of meta-analysis. They are only beyond the scope of meta-analysis if we understand meta-analysis to be about estimating overall means. They suggest an example where mean selection is trivial, but where selection is positive or negative in different contexts. Indeed, such a contribution to the primary literature contains information far beyond the mean value of selection coefficients in nature. It contains information about the mean magnitude, the within-study variance in selection and the environmental dependence of selection. I see all these as quantities that meta-analysis can get at, at least in principle. Most meta-analyses of such quantities will need to use tools beyond those that are appropriate for meta-analysis of the mean. I think that Scheiner’s (2016) categorizations of goals of meta-analysis are quite useful. He suggests three classes of question, which I might characterize as: (1) the sign of mean effects, particularly hypothesis tests pertaining to the meta-analytic mean, for example is E[x] greater than zero? , (2) parameter value estimation, for example what is the value of E[x]? and (3) consistency, for example how often is x greater than zero? In my opinion, the future of methods to answer all of these questions lies in model-based approaches: ‘The great advantage of the model-based over the ad hoc approach, it seems to me, is that at any given time we know what we are doing’ (Box, 1979). Given modelbased estimates of the mean, variance and potentially other aspects of the distribution of some quantity x, all three questions can be addressed (using transformation of model-based inferences about x, as necessary). Examples of model-based tests of consistency are Morrissey & Hadfield (2012)’s inference of the frequency of sign changes of natural selection through time, and the frequency of sexually antagonistic selection (Morrissey, 2016). Some of the main proponents of meta-analysis in evolutionary biology (Jennions, 2016; Nakagawa & Lagisz, 2016) express some discomfort with my statements about the place of meta-analysis among approaches to scholarly synthesis. Meta-analytic inference of the distributions of various parameters in nature is undoubtedly a valuable method of contributing to scholarly synthesis. The point of contention is, I think, that I do not believe that meta-analysis should be seen as the foremost mode of synthesis. I do not believe that ‘there is no alternative to quantitatively synthesizing data from multiple studies’. I have invested some months in trying to understand how different meta-analytic approaches behave, and how best to make inferences about quantities such as the mean magnitude of selection and prevalence of sexually antagonistic selection. This reflects my opinion that meta-analysis is an important part of scholarly synthesis! What aspects of scholarly synthesis are poorly handled when meta-analysis dominates? Take, for example, the general phenomenon that viability selection can generate nonrandom patterns in evolutionary parameters among surviving individuals. In principle, this can even reverse the sign of net natural selection, where nonrandom mortality prior to trait expression generates a ‘missing fraction’ (Grafen, 1988; Hadfield, 2008). There are insufficient empirical studies of this topic for meta-analysis. However, one particularly convincing study of the missing fraction used fitness differences among selection lines to show that viability selection prior to expression of a trait (flower size) reversed the sign of net selection of that trait (Mojica & Kelly, 2010). The results of this study are not easily Correspondence: Michael B. Morrissey, School of Biology, University of St Andrews, Dyers Brae House, St Andrews KY16 9TH, Fife, UK. Tel. : +44 (0) 1334 463738; fax: +44 (0) 1334 463366; e-mail: michael.morrissey@st-andrews.ac. uk