Peer review of: Behavioural correlations of the domestication syndrome are decoupled in modern dog (2019)

Reviewers' Comments:
Reviewer #1:
Remarks to the Author:
The MS has undergone a considerable improvement during the revision. The Authors has thoughtfully
considered my comments on the previous version and convincingly addressed the majority of them.
However, I still have some observations concerning two conceptual aspects and the new analytical
design.
1) In their response letter, the Authors argue in detail about the necessity of focusing on the within-
breed patterns and the irrelevance of the among-breed correlations. The reasoning is relevant, but it
regretfully remains non-transparent for the readers who will see the text of the paper only. From their
perspective, based on the information that is currently given in the MS, it might not be that evident
that by domestication syndrome, the Authors mean within-breed (among individual) correlations.
Confusions may arise because artificial selection targets certain breed-specific traits due to some
desired function, and in this context it might be also of interest to test how such selection pressures
among-breed patterns. Accordingly, recent selection regimes affecting the mean level of expression in
certain traits but not in others will break among-breed correlations but not necessarily the within-
breed correlations. More motivation should be given to the general readers about the need of focusing
on the within-breed patterns when testing the working hypothesis of this paper.
2) From line 232 the Authors conclude that “in the context of the domestication syndrome hypothesis,
our results suggest that different genes and/or hormonal pathways are causing the correlations
between these behaviours and that the behavioural correlations associated with domestication may, to
some extent, not share a simple common mechanistic origin”. I think that given that the Authors do
not investigate mechanisms and cannot separate genetic effects, this remains a speculation. The fact
that some correlations have been weakened does not necessarily mean that the corresponding traits
were originally governed by different causative pathways. They may also have had the same
mechanistic origin, but selection might have broken up this shared hormonal/genetic link during the
evolutionary history.
3) I welcome that the Authors adopted the phylogenetic-meta-analytic framework available in
 MCMCglmm, but some details of the analyses need to be revised:
 a) First, using the entire data the Authors calculate the residuals of the behavioural scores and then
estimate the pairwise correlation between these residuals separately for each breed. This can be
misleading for two reasons. First, combining the entire data for the first analysis will lead to models in
which within- and between-breed effects are mixed. For example, if age differently affects a
behavioural trait in one breed than in another, it is not correct to rely on the same general equation
when calculating the residuals. Second, it requires a “stat on stat” approach, while a more
parsimonious solution would be available. My suggestion is to create a single model for each bred and
pair of traits, in which the original scores are used (one as a predictor and one as a response) and the
confounders are included at once. From this model, the relevant slope and its standard error can be
used to estimate the desired effect size (r)
b) I am not sure how the Authors dealt with autocorrelation. The behavioural traits are assessed in a
strict sequence after each other (if I understand correctly). Thus it might be that behaviour in one test
can determine the behaviour in the next test situation (e.g. by the level of stress that individuals
experience). Thus, correlations between these traits might simply reflect these carryover effects over
the test situations. Accordingly, modern and ancient breeds might differ in this autocorrelation
component without any consequence for the domestication syndrome.
c) I do not agree with the construction of the main model. From table 1 and from the description, I
 understand that breed-specific correlations (16 for each breed considering all pairwise combinations)
 were the response variable, and breed category, expected correlation (i.e. within or between prosocial
 and reactive categories) and their interaction were the fixed predictors, while breed, phylogeny and
 correlation type (e.g sociability v. playfulness) were the random effects. I think, this model does not
 appropriately cover the hierarchical structure of the data. Breeds are nested within the two breed
 categories, while breed categories cover deep level phylogenetic effects. Hence, ideally these
 hierarchies should be covered by the random part of the model. The main interest is to estimate the
 amount of variance that is accounted by breed categories on top of the breed and phylogenetic
 effects. Furthermore, the pseudo-replication induced by the fact that same traits are used to obtain
 different correlations should be accounted by including both traits in the correlation as random factors
 (e.g. sociability for trait1 and playfulness for trait2) instead of using correlation type (e.g sociability-
 playfulness). From this model, it might be difficult (technically, it is possible) to obtain grouped
 random effect estimates separately for the two “within or between prosocial and reactive” categories
 (that is equivalent for the interaction between Breed category:Expected correlation), but this could be
 achieved by creating two models relying on either the within- or the between-category correlations.
 I must note in addition that Table 1 is confusing. I am not sure what “Dog” means in this model, since
 the unit of this analysis is the breed-specific correlation, so the lowest hierarchical level in the
 underlying data is the breed level and not individual level. Furthermore, it is supposed to be a result of
 a phylogenetically inferred model, while random effect for phylogeny is not given (Breed ID is different
 than phylogeny effect).
 d) I am confused by the statement from line 441 “We were not able to fit both sampling variance of Zr
 and a phylogenetic effect in these models due to the fact that each model only had one observation
 per breed.” I do not think it is true. Sampling variance of Zr should be 1/N-3, where N is the number
 of individuals used to estimate Zr. So I am not sure how Vz is calculated and accounted for in the
 models.
 Reviewer #2:
 Remarks to the Author:
 Authors have submitted an extensively revised manuscript based upon reviewer comments. By doing
 so they have addressed most but not all key reviewer comments.
 1. A key comment was that authors should use multivariate mixed-effects models that control for
 various biasing effects to estimate the within-breed variances and covariances (hence correlations).
 Authors clarify that this task was not possible computationally, which is surprising. Nevertheless,
 authors have now used univariate approaches to estimate values controlling for bias which they then
 correlated across traits, which seems an appropriate pragmatic solution. My only comment here is that
 in the results this shouldn't be called "residual correlation" but rather within-breed among-individual
 correlation (calculated off residuals or something of that kind).
 2. A second key comment was that changes in correlation structures can be caused by changes in
 variances not covariances: r = covAB/sqrt(varA*varB). The request was to therefore additionally
 estimate changes in variances and covariances (see e.g. Dingemanse et al. 2007 J Anim Ecol for an
 early paper appreciating this problem). The authors have not taken up this task. Consequently, based
 on the analyses presented we cannot ascertain that the main conclusions stand (tighter covariance
 structure in certain types of breed). Possibly, covariances were unchanged between types of brood
 
because variances were breed-specific, leading to changed correlations. In my view these additional
 analyses are required to forcefully interpret the data.
 3. A third key argument is that authors should use a multivariate approach to multivariate data rather
 than presenting sets of pairwise correlations. It is not clear to me whether the authors have taken this
 comment on board. If they must present sets of pairwise correlations, then it would be appropriate to
 control for repeated testing of the same data.
 4. The "a priori" hypothesis of certain correlations being positive and others being negative is
 insufficiently detailed. Authors provide insufficient arguments making their case, leading to the feeling
 that the hypothesis was a postiori rather than a priori formulated.
 Reviewer #3:
 Remarks to the Author:
 All my concerns have been clearly and succinctly addressed. I think the paper should be published in
 the present version.