different paradigms -> different results

see: "Will COVID-19 be evidence-based medicine’s nemesis?" (June 30, 2020)

https://journals.plos.org/plosmedicine/article?id=10.1371/jo...

"Once defined in rhetorical but ultimately meaningless terms as “the conscientious, judicious and explicit use of current best evidence in making decisions about the care of individual patients” [1], evidence-based medicine rests on certain philosophical assumptions: a singular truth, ascertainable through empirical enquiry; a linear logic of causality in which interventions have particular effect sizes; rigour defined primarily in methodological terms (especially, a hierarchy of preferred study designs and tools for detecting bias); and a deconstructive approach to problem-solving (the evidence base is built by answering focused questions, typically framed as ‘PICO’—population-intervention-comparison-outcome) [2].

The trouble with pandemics is that these assumptions rarely hold. A pandemic-sized problem can be framed and contested in multiple ways. Some research questions around COVID-19, most notably relating to drugs and vaccines, are amenable to randomised controlled trials (and where such trials were possible, they were established with impressive speed and efficiency [3, 4]). But many knowledge gaps are broader and cannot be reduced to PICO-style questions. Were care home deaths avoidable [5]? Why did the global supply chain for personal protective equipment break down [6]? What role does health system resilience play in controlling the pandemic [7]? And so on."

...

"It is surely time to turn to a more fit-for-purpose scientific paradigm. Complex adaptive systems theory proposes that precise quantification of particular cause-effect relationships is both impossible (because such relationships are not constant and cannot be meaningfully isolated) and unnecessary (because what matters is what emerges in a particular real-world situation). This paradigm proposes that where multiple factors are interacting in dynamic and unpredictable ways, naturalistic methods and rapid-cycle evaluation are the preferred study design. The 20th-century logic of evidence-based medicine, in which scientists pursued the goals of certainty, predictability and linear causality, remains useful in some circumstances (for example, the drug and vaccine trials referred to above). But at a population and system level, we need to embrace 21st-century epistemology and methods to study how best to cope with uncertainty, unpredictability and non-linear causality"

"In a complex system, the question driving scientific inquiry is not “what is the effect size and is it statistically significant once other variables have been controlled for?” but “does this intervention contribute, along with other factors, to a desirable outcome?”. Multiple interventions might each contribute to an overall beneficial effect through heterogeneous effects on disparate causal pathways, even though none would have a statistically significant impact on any predefined variable [11]. To illuminate such influences, we need to apply research designs that foreground dynamic interactions and emergence. These include in-depth, mixed-method case studies (primary research) and narrative reviews (secondary research) that tease out interconnections and highlight generative causality across the system [16, 17]."

....

"In the current fast-moving pandemic, where the cost of inaction is counted in the grim mortality figures announced daily, implementing new policy interventions in the absence of randomized trial evidence has become both a scientific and moral imperative. Whilst it is hard to predict anything in real time, history will one day tell us whether adherence to “evidence-based practice” helped or hindered the public health response to Covid-19—or whether an apparent slackening of standards to accommodate “practice-based evidence” was ultimately a more effective strategy."