Cooking Pasta in La Paz : Bilingualism , Bias and the Replication Crisis

Literature on bilingualism and cognition is characterised by a large amount of conflicting evidence. In some studies, bilinguals perform better then monolinguals on executive tasks involving inhibition, monitoring and switching but are slower on tasks of lexical processing. Other studies don’t find any significant effects and challenge the very existence of cognitive differences between monolinguals and bilinguals. In this paper I question the assumption that different studies performed in different parts of the world should yield the same results. I argue that the environment (in the widest sense of the word) in which an experiment is conducted can exert profound influence on its outcome. Against the background of the current debate about the replication crisis in science I propose that conflicting evidence is not a threat to the trustworthiness of scientific research but a sign of the health of a discipline and a welcome opportunity to identify new relevant variables. 1. Controversy, conspiracy or consensus? At first sight, the question whether bilingualism is associated with cognitive benefits, particularly in the area of executive functions is a topic of an intense controversy. The evidence so far is inconsistent. On one hand, a large number of studies show better cognitive functions in bilingual children (Kovács & Mehler, 2009), young adults (Bak, Vega-Mendoza, & Sorace, 2014; Vega-Mendoza, West, Sorace, & Bak, 2015) and elderly (Bak, Nissan, Allerhand, & Deary, 2014; Kavé, Eyal, Shorek, & Cohen-Mansfield, 2008). These findings have been linked to evidence from structural and functional neuroimaging (Abutalebi et al., 2013; Gold, Kim, Johnson, Kryscio, & Smith, 2013). Moreover, a potential clinical relevance of these differences is underlined by studies reporting a bilingual delay in the onset of dementia (Alladi et al., 2013; Bialystok, Craik, & Freedman, 2007; Woumans et al., 2015) and a better cognitive outcome after stroke (Alladi et al., 2015). On the other hand, many of these findings have been called into question, including results in children (Duñabeitia et al., 2014), young adults (Paap & Greenberg, 2013) and in dementia (Zahodne, Schofield, Farrell, Stern, & Manly, 2014). New evidence seems to be growing constantly on both sides, unlikely to provide a satisfactory explanation for the discrepancy in findings. The stakes in this debate have been raised by the suggestion that bilingualism might have a more profound effect on dementia than any currently available drug treatment. With such a potential relevance for public health, any findings in this area need to be carefully scrutinised (Bak, 2016; Bialystok et al, 2016). There are two main strategies with which this problem of conflicting evidence can be approached. The first one assumes that one side is right and, therefore, the other one must be necessarily wrong. Accordingly, either all the positive or all the negative results must be a product of bias, self-deception, unsound statistical analysis, faulty logic or some other type of error. Such an approach has much in common with the adversarial style dominating legal systems and political life in many countries. It has many advantages: it forces the opponents to formulate clear hypothesis, sharp arguments and convincing justifications. It focuses the debates on the most relevant points and can be more entertaining to readers, viewers and listeners than the sometimes rather dull consensus. However, in science, as in politics, the adversarial approach has also its downsides, often just the other side of what we could justly consider as its advantage. The focused nature of the arguments can mean a narrowing of the perspective, leaving out the sheer diversity of the phenomena in question. The confrontational nature of the debate offers a temptation to reduce the complexity of the matter to well-sounding but inadequate slogans. It pushes the adversaries towards a partisan myopia and “selective scepticism”, in which a detailed scrutiny of the other side is combined with an uncritical acceptance of everything that can be used in one’s favour. After all, one would not expect the prosecution to search for arguments that would strengthen the defence and vice versa. And once the arguments are exhausted, an adversarial debate risks shifting the emphasis from the relevant topics to the individuals representing them; taken to extremes it can lead to conspiracy theories. An alternative approach is constructive scepticism. It is constructive in assuming that both positive and negative results are genuine, but reflect possible differences in experimental designs, definitions of the phenomena in question, examined populations and the environment in which the research is being conducted. As true scepticism, it critically examines the evidence and the arguments on both sides. It sees as its main objective not to prove one’s point but to try to understand and explain current puzzles and contradictions. It does not deny opposing points of view but tries to transcend them. It can build on a long tradition of dialectics, from Ancient Greece (and indeed Ancient India), through medieval universities up to the present day. In the dialectical method, the thesis and antithesis is followed by a synthesis, integrating both and, hopefully, coming closer to the truth. Superficially, it is easy to get carried away by the colourful rhetoric of the current “bilingualism debate”. But looking below the surface, we might find a surprising amount of convergence. Admittedly, some studies show significant differences between monoand bilingual groups and others don’t, but if there is a difference, it has a remarkably similar pattern across various studies. It is like a mountain chain submerged under water, hidden and exposed in turn by rising and falling tides but nevertheless maintaining its shape. The highest peak of this chain are executive functions; if a study comparing monoand bilingualism finds only one difference between both groups, it is much more likely to be in the executive domain than in any other cognitive function, such as memory or visuospatial skills (Bak, Nissan, et al., 2014). Even within the executive functions, there is a recognisable profile. The executive functions most frequently reported to be modified by bilingualism are those connected with inhibition, monitoring and switching. In contrast, reasoning tasks such as Tower of London usually do not show any difference between the groups (de Bruin, Bak, & Della Sala, 2015). The same consistency of profile applies to the negative effects of bilingualism. One of the best-established (although at first slightly counter-intuitive) findings in bilingualism is the observation of its detrimental effect on several linguistic measures such as lexical access (Gollan, Montoya, Fennema-Notestine, & Morris, 2005; Ivanova & Costa, 2008). A similar contrast between executive functions and language extends also into the studies of the effects of bilingualism on brain diseases. In stroke, bilingualism is associated with a better outcome in terms of post-stroke dementia and post-stroke mild cognitive impairment but not in terms of post-stroke aphasia (language disorder) (Alladi et al., 2015). Indeed, a similar pattern emerges even in papers cited usually as examples of null-results, reporting no differences between monoand bilinguals. A large longitudinal study of the Hispanic population from the Washington Heights area of New York (Zahodne et al., 2014) found a better initial performance in bilinguals on all four cognitive composite scores they measured. However, the difference was biggest for the executive function score, followed by speed, language and memory. Can we explain satisfactorily such a consistent profile through random variation? Or can it lead us to the possible reason for the conflicting evidence? If we follow the metaphor of a submerged mountain chain, we can predict which parts are likely to appear first when the water levels begin to fall. The task is to identify what determines the changes in water level. This paper argues that the conflicting evidence in the area of bilingualism and cognition and the ensuing debate are neither surprising, nor worrying, nor exceptional compared with other areas of science. I will focus on three areas. Firstly, I will examine some common myths, misconceptions and false dichotomies about bilingualism and cognition. By exaggerating or misinterpreting research results they naturally invite well-justified criticism. Secondly, I will place the current debate into a broader context of the “replication crisis” in modern science. Finally, I will attempt to address the fundamental question behind this debate: should we expect all studies to replicate, regardless of the environment in which they have been conducted? Or can the non-replications turn out to be a source of new insights and a motor of future scientific progress? Although my arguments will naturally revolve around bilingualism, I hope that they will be of some interest also to the topic of the “replication crisis” in general. 2. Are bilinguals better lovers? Myths, misconceptions and false dichotomies The recent findings of bilingualism research have produced an extraordinary interest in the media. Newspapers, radio and television report extensively about studies exploring potential influence of bilingualism on cognitive functions across the lifespan and in brain disease. While the interest of general public in scientific discoveries is to be welcomed, it can lead to a simplification of complex findings to make them understandable to a broader readership or audience. This can easily produce exaggerations going well beyond what is supported by evidence, as exemplified by the bold claim that “People Who Are Bilingual Are Smart, Creative and Better Lovers” (http://elitedaily.com/life/culture/bilingual-people-smartcreative-lovers/1012727/). It would be tempting but inaccurate to