abstract: Until the advent of modern neuroscience, free will used to be a theological and a metaphysical concept, debated with little reference to brain function. Today, with ever increasing understanding of neurons, circuits and cognition, this concept has become outdated and any metaphysical account of free will is rightfully rejected. The consequence is not, however, that we become mindless automata responding predictably to external stimuli. On the contrary, accumulating evidence also from brains much smaller than ours points towards a general organization of brain function that incorporates flexible decision-making on the basis of complex computations negotiating internal and external processing. The adaptive value of such an organization consists of being unpredictable for competitors, prey or predators, as well as being able to explore the hidden resource deterministic automats would never find. At the same time, this organization allows all animals to respond efficiently with tried-and-tested behaviours to predictable and reliable stimuli. As has been the case so many times in the history of neuroscience, invertebrate model systems are spearheading these research efforts. This comparatively recent evidence indicates that one common ability of most if not all brains is to choose among different behavioural options even in the absence of differences in the environment and perform genuinely novel acts. Therefore, it seems a reasonable effort for any neurobiologist to join and support a rather illustrious list of scholars who are trying to wrestle the term 'free will' from its metaphysical ancestry. The goal is to arrive at a scientific concept of free will, starting from these recently discovered processes with a strong emphasis on the neurobiological mechanisms underlying them.
author: Björn Brembs\*<email@example.com>
date: 2011-03-22
institute: Freie Universität Berlin, Institute for Biology – Neurobiology, Königin-Luise-Strasse 28/30, 14195 Berlin, Germany
references:
title: Towards a scientific concept of free will as a biological trait: spontaneous actions and decision-making in invertebrates

# Introduction: the rejection of the metaphysical concept of free will

What could possibly get a neurobiologist with no formal training in philosophy beyond a few introductory lectures, to publicly voice his opinion on free will? Even worse, why use empirical, neurobiological evidence mainly from invertebrates to make the case? Surely, the lowly worm, snail or fly cannot even be close to something as philosophical as free will? The main reason is this neurobiologist's opinion that free will is a biological trait and not a metaphysical entity. 'Free will is a biological property, not a gift or a mystery' \[1\]. Today, neurobiology has accumulated sufficient evidence that we can move on from speculating about the existence of free will towards plausible models of how brains have implemented it. On the surface, this statement seems to contradict public statements from many other neurobiologists who fervently deny free will. In fact, it appears that if neurobiologists feel compelled to write about free will, they do so only to declare that it is an illusion \[2–5\]. Of course, all of these neurobiologists are correct in that free will as a metaphysical entity indeed most probably is an illusion. Colloquial and historical use of the term 'free will' has been inextricably linked with one variant or another of dualism. There have been so many and thorough recounts of the free will debate, that I will only reference some, which can serve to introduce the concepts used here \[6–9\]. Psychologists and neurobiologists have rightfully pointed out for decades now that there is no empirical support for any form of dualism. The interactionism proposed by Popper & Eccles was probably one of the last prominent accounts of dualism \[10\]. Since then, these and related positions have largely fallen into irrelevance. Today, the metaphysical concept of free will is largely devoid of any support, empirical or intellectual.

# The rejection of determinism

That said, it is an all too common misconception that the failure of dualism as a valid hypothesis automatically entails that brains are deterministic and all our actions are direct consequences of gene–environment interactions, maybe with some random stochasticity added in here and there for good measure \[2\]. It is tempting to speculate that most, if not all, scholars declaring free will an illusion share this concept. However, our world is not deterministic, not even the macroscopic world. Quantum mechanics provides objective chance as a trace element of reality. In a very clear description of how keenly aware physicists are that Heisenberg's uncertainty principle indeed describes a property of our world rather than a failure of scientists to accurately measure it, Stephen Hawking has postulated that black holes emit the radiation named after him \[11\], a phenomenon based on the well-known formation of virtual particle–antiparticle pairs in the vacuum of space. The process thought to underlie Hawking radiation has recently been observed in a laboratory analogue of the event horizon \[12,13\]. On the 'mesoscopic' scale, fullerenes have famously shown interference in a double-slit experiment \[14\]. Quantum effects have repeatedly been observed directly on the nano-scale \[15,16\], and superconductivity (e.g. \[17\]) or Bose–Einstein condensates (e.g. \[18\]) are well-known phenomena. Quantum events such as radioactive decay or uncertainty in the photoelectric effect are used to create random-number generators for cryptography that cannot be broken into. Thus, quantum effects are being observed also on the macroscopic scale. Therefore, determinism can be rejected with at least as much empirical evidence and intellectual rigor as the metaphysical account of free will. 'The universe has an irreducibly random character. If it is a clockwork, its cogs, springs, and levers are not Swiss-made; they do not follow a predetermined path. Physical indeterminism rules in the world of the very small as well as in the world of the very large' \[9\].

# Behavioural variability as an adaptive trait

If dualism is not an option and determinism is equally untenable, what other options are we left with? Some scholars have resorted to quantum uncertainty in the brain as the solution, providing the necessary discontinuity in the causal chain of events. This is not unrealistic, as there is evidence that biological organisms can evolve to take advantage of quantum effects. For instance, plants use quantum coherence when harvesting light in their photosynthetic complexes \[19–22\]. Until now, however, it has proved difficult to find direct empirical evidence in support of analogous phenomena in brains \[9\]. Moreover, and more importantly, the pure chance of quantum indeterminism alone is not what anyone would call 'freedom'. 'For surely my actions should be caused because I want them to happen for one or more reasons rather that they happen by chance' \[9\]. This is precisely where the biological mechanisms underlying the generation of behavioural variability can provide a viable concept of free will.

Biologists need not resort to quantum mechanics to understand that deterministic behaviour can never be evolutionarily stable. Evolution is a competitive business and predictability is one thing that will make sure that a competitor will be out of business soon. There are many illuminating examples of selection pressures favouring unpredictability, but three recently published reports dealing with one of the most repeatable and hence best-studied class of behaviours are especially telling. These examples concern escape behaviours.

One of the most well-studied escape behaviours is the so-called C-start response in fishes. The response is called C-start because fishes that perceive sudden pressure changes on one side of their body bend in a C-shape away from the perceived stimulus to escape in the opposite direction. One of the largest neurons in vertebrate nervous systems is mediating this response, the Mauthner cell (e.g. \[23\]). Recently, Kenneth Catania and colleagues described the hunting technique of tentacled snakes (*Erpeton tentaculatus*) \[24,25\]. The snakes hunt for fishes by cunningly eliciting a C-start response in the potential prey animal with a more caudal part of their body, prompting the fish to C-start exactly into the mouth of the snake.

Some of the most important predators of earthworms are moles. When moles dig through the ground, they produce a very distinctive sound. Earthworms have evolved to respond to this sound by crawling to the surface, where the moles will not follow them. Kenneth Catania recently reported that the technique of 'worm-grunting', employed in order to catch earthworms as fish bait, exploits this response. The worm grunters use a combination of wooden poles and metal rods to generate the sound and then collect the worms from the surface \[26\].

In the third example, another very well-studied escape response is exploited by birds. Under most circumstances, the highly sophisticated jump response of dipteran flies is perfectly sufficient to catapult the animals out of harm's way (e.g. \[27\]). However, painted redstarts (*Myioborus pictus*) are ground-hunting birds that flush out dipterans by eliciting their jump response with dedicated display behaviours. Once the otherwise well-camouflaged flies have jumped, they are highly visible against the bright sky and can be caught by the birds \[28,29\].

It is not a huge leap to generalize these insights from escape responses to other behaviours. Predictability can never be an evolutionarily stable strategy. Instead, animals need to balance the effectiveness and efficiency of their behaviours with just enough variability to spare them from being predictable. Efficient responses are controlled by the environment and thus vulnerable. Conversely, endogenously controlled variability reduces efficiency but increases vital unpredictability. Thus, in order to survive, every animal has to solve this dilemma. It is no coincidence that ecologists are very familiar with a prominent, analogous situation, the exploration/exploitation dilemma (originally formulated by March \[30\]): every animal, every species continuously faces the choice between staying and efficiently exploiting a well-known, but finite resource and leaving to find a new, undiscovered, potentially much richer, but uncertain resource. Efficiency (or optimality) always has to be traded off with flexibility in evolution, on many, if not all, levels of organization.

A great invertebrate example of the sort of Protean behaviour \[31,32\] selected for by these trade-offs is yet another escape behaviour, that of cockroaches. The cerci of these insects have evolved to detect minute air movements. Once perceived, these air movements trigger an escape response in the cockroach away from the side where the movement was detected. However, which angle with respect to the air movement is taken by the animal cannot be predicted precisely, because this component of the response is highly variable \[33\]. Therefore, in contrast to the three examples above, it is impossible for a predator to predict the trajectory of the escaping animal.

# Brains are in control of variability

Competitive success and evolutionary fitness of all ambulatory organisms depend critically on intact behavioural variability as an adaptive function \[34\]. Behavioural variability is an adaptive trait and not 'noise'. Not only biologists are aware of the fitness advantages provided by unpredictable behaviour, but philosophers also realized the adaptive advantages of behavioural variability and their potential to serve as a model for a scientific account of free will, as long as 25 years ago (e.g. \[6\]). The ultimate causes of behavioural variability are thus well understood. The proximate causes, however, are much less studied. One of the few known properties is that the level of variability also can vary. Faced with novel situations, humans and most animals spontaneously increase their behavioural variability \[35–38\]. Animals even vary their behaviour when a more stereotyped behaviour would be more efficient \[39\].

These observations suggest that there must be mechanisms by which brains control the variability they inject into their motor output. Some components of these mechanisms have been studied. For instance, tethered flies can be trained to reduce the range of the variability in their turning manoeuvres \[40\]. For example, one such stationary flying fly may be trained to cease generating left-turning manoeuvres by heating the fly (with an infrared heat beam) whenever it initiates such actions and by not heating it during right-turning manoeuvres. Before such training, it would generate left- and right-turning manoeuvres in equal measure. Protein kinase C activity is required for such a reduction \[41\]. Interestingly, analogous to the exploration–exploitation dilemma mentioned above, the mechanism by which the animals learn to decrease their behavioural variability ('self-learning') interacts with the learning mechanism by which the animals learn about external stimuli ('world learning'). Part of this interaction balances self- and world learning such that self-learning (i.e. the endogenous reduction in behavioural variability) is slowed down whenever the world-learning mechanism is engaged. This part of the interaction is mediated by a subpopulation of neurons in a part of the insect brain called the mushroom bodies \[42,43\]. This population of neurons ensures that animals preferentially learn from their environment and reduce their endogenous behavioural variability only when there are good reasons for doing so. Such an organization may underlie the need for practice in order to reduce our behavioural variability when learning new skills, e.g. the basketball free-throw or the golf swing. The parallel to the exploration–exploitation dilemma lies in the balance between the endogenous and exogenous processing these interactions bestow upon the animal: learning about the world first allows the animal to keep its behaviour flexible in case the environment changes, while at the same time being able to efficiently solve the experimental task. If, however, it turns out that the environment does not change, then—and only then—is the circuitry controlling the behaviour itself modified, to more permanently alter the behaviour-generating process itself and thereby maximize on efficiency by reducing the endogenous variability.

Animals other than insects also learn to control their variability using feedback from the environment, such that levels of behavioural variability—from highly predictable to random-like—are directly influenced by reinforcement. For instance, consummatory feeding behaviour of the marine snail *Aplysia* is highly variable \[44,45\]. Recent evidence suggests that the seemingly rhythmic cycling of biting, swallowing and rejection movements of the animal's radula (a tongue-like organ) vary in order to be able to adapt to varying food sources \[46\]. In fact, much like the reduced variability in flies trained to avoid heat in the self-learning paradigm explained above, *Aplysia* can be trained to reduce the variability in their feeding behaviour and generate rhythmic, stereotyped movements \[47–52\]. It also takes practice for snails to become efficient and predictable. The default state is to behave variably and unpredictably.

The mechanisms to control behavioural variability are in place also in humans. For instance, depression and autism are characterized by abnormally stereotypic behaviours and a concomitant lack of behavioural variability. Patients suffering from such psychopathologies can learn to vary their behaviours when reinforced for doing so \[53,54\]. Also, the interactions between world- and self-learning seem to be present in vertebrates: extended training often leads to so-called habit formation, repetitive responses, controlled by environmental stimuli (e.g. \[55,56\]). It is intriguing that recent fMRI studies have discovered a so-called default-mode network in humans, the fluctuations in which can explain a large degree of the individual's behavioural variability \[57\], and that abnormalities in this default network are associated with most psychiatric disorders \[58–60\].

# What are the neural mechanisms generating behavioural variability?

It thus appears that behavioural variability is a highly adaptive trait, under constant control of the brain balancing the need for variability with the need for efficiency. How do brains generate and control behavioural variability in this balance? These studies have only just begun. As was the case in much of neuroscience's history, be it ion channels, genes involved in learning and memory, electrical synapses or neurotransmitters, invertebrate model systems are leading the way in the study of the neural mechanisms underlying behavioural variability as well.

Two recent reports, concerned another highly reproducible (and therefore well-studied) behaviour, optomotor responses. Tethered flies respond to a moving grating in front of them with characteristic head movements in the same direction as the moving grating, aimed at stabilizing the image on the retina. By recording from motion-sensitive neurons in fly optic lobes, the authors found that the variability in these neurons did not suffice to explain the variability in the head movements \[61,62\]. Presumably, downstream neurons in the central brain inject additional variability, not present in the sensory input, which is reflected in the behaviour.

A corresponding conclusion can be drawn from two earlier studies, which independently found that the temporal structure of the variability in spontaneous turning manoeuvres both in tethered and in free-flying fruitflies could not be explained by random system noise \[63,64\]. Instead, a nonlinear signature was found, suggesting that fly brains operate at criticality, meaning that they are mathematically unstable, which, in turn, implies an evolved mechanism rendering brains highly susceptible to the smallest differences in initial conditions and amplifying them exponentially \[63\]. Put differently, fly brains have evolved to generate unpredictable turning manoeuvres. The default state also of flies is to behave variably. Ongoing studies are trying to localize the brain circuits giving rise to this nonlinear signature.

Results from studies in walking flies indicate that at least some component of variability in walking activity is under the control of a circuit in the so-called ellipsoid body, deep in the central brain \[65\]. The authors tested the temporal structure in spontaneous bouts of activity in flies walking back and forth individually in small tubes and found that the power law in their data disappeared if a subset of neurons in the ellipsoid body was experimentally silenced. Analogous experiments have recently been taken up independently by another group and the results are currently being evaluated \[66\]. The neurons of the ellipsoid body of the fly also exhibit spontaneous activity in live imaging experiments \[67\], suggesting a default-mode network also might exist in insects.

Even what is often presented to students as 'the simplest behaviour', the spinal stretch reflex in vertebrates, contains adaptive variability. Via the cortico-spinal tract, the motor cortex injects variability into this reflex arc, making it variable enough for operant self-learning \[68–72\]. Jonathan Wolpaw and colleagues can train mice, rats, monkeys and humans to produce reflex magnitudes either larger or smaller than a previously determined baseline precisely because much of the deviations from this baseline are not noise but variability deliberately injected into the reflex. Thus, while invertebrates lead the way in the biological study of behavioural variability, the principles discovered there can be found in vertebrates as well.

One of the common observations of behavioural variability in all animals seems to be that it is not entirely random, yet unpredictable. The principle thought to underlie this observation is nonlinearity. Nonlinear systems are characterized by sensitive dependence on initial conditions. This means such systems can amplify tiny disturbances such that the states of two initially almost identical nonlinear systems can diverge exponentially from each other. Because of this nonlinearity, it does not matter (and it is currently unknown) whether the 'tiny disturbances' are objectively random as in quantum randomness or whether they can be attributed to system, or thermal noise. What can be said is that principled, quantum randomness is always some part of the phenomenon, whether it is necessary or not, simply because quantum fluctuations do occur. Other than that it must be a non-zero contribution, there is currently insufficient data to quantify the contribution of such quantum randomness. In effect, such nonlinearity may be imagined as an amplification system in the brain that can either increase or decrease the variability in behaviour by exploiting small, random fluctuations as a source for generating large-scale variability. A general account of such amplification effects had already been formulated as early as in the 1930s \[73\]. Interestingly, a neuronal amplification process was recently observed directly in the barrel cortex of rodents, opening up the intriguing perspective of a physiological mechanism dedicated to generating neural (and by consequence behavioural) variability \[74\].

# Determinism versus indeterminism is a false dichotomy

Together with Hume, most would probably subscribe to the notion that 'tis impossible to admit of any medium betwixt chance and an absolute necessity' \[75\]. For example, Steven Pinker (1997, p. 54) concurs that 'A random event does not fit the concept of free will any more than a lawful one does, and could not serve as the long-sought locus of moral responsibility' \[76\]. However, to consider chance and lawfulness as the two mutually exclusive sides of our reality is only one way to look at the issue. The unstable nonlinearity, which makes brains exquisitely sensitive to small perturbations, may be the behavioural correlate of amplification mechanisms such as those described for the barrel cortex \[74\]. This nonlinear signature eliminates the two alternatives, which both would run counter to free will, namely complete (or quantum) randomness and pure, Laplacian determinism. These represent opposite and extreme endpoints in discussions of brain functioning, which hamper the scientific discussion of free will. Instead, much like evolution itself, a scientific concept of free will comes to lie between chance and necessity, with mechanisms incorporating both randomness and lawfulness. The Humean dichotomy of chance and necessity is invalid for complex processes such as evolution or brain functioning. Such phenomena incorporate multiple components that are both lawful and indeterminate. This breakdown of the determinism/indeterminism dichotomy has long been appreciated in evolution and it is surprising to observe the lack of such an appreciation with regard to brain function among some thinkers of today (e.g. \[2\]). Stochasticity is not a nuisance, or a side effect of our reality. Evolution has shaped our brains to implement 'stochasticity' in a controlled way, injecting variability 'at will'. Without such an implementation, we would not exist.

A scientific concept of free will cannot be a qualitative concept. The question is not any more 'do we have free will?'; the questions is now: 'how much free will do we have?'; 'how much does this or that animal have?'. Free will becomes a quantitative trait.

# Initiating activity: actions versus responses

Another concept that springs automatically from acknowledging behavioural variability as an adaptive trait is the concept of actions. In contrast to responses, actions are behaviours where it is either impossible to find an eliciting stimulus or where the latency and/or magnitude of the behaviour vary so widely, that the term 'response' becomes useless.

A long history of experiments on flies provides accumulating evidence that the behaviour of these animals is much more variable than it would need to be, given the variability in the neurons mediating the stimulus-response chain (reviewed in \[77\]). For instance, in the study of the temporal dynamics of turning behaviours in tethered flies referenced above \[63\], one situation recorded fly behaviour in constant stimulus conditions, i.e. nothing in the exquisitely controlled environment of the animals changed while the turning movements were recorded. Yet, the flies kept producing turning movements throughout the experiment as if there had been stimuli in their environment. Indeed, the temporal structure in these movements was qualitatively the same compared with when there were stimuli to be perceived. This observation is only one of many demonstrating the endogenous character of behavioural variability. Even though there was nothing in the environment prompting the animals to change their behaviour, they kept initiating turning manoeuvres in all directions. Clearly, each of these manoeuvres was a self-initiated, spontaneous action and not a response to some triggering, external stimulus.

In fact, such self-initiated actions are a necessary prerequisite for the kind of self-learning described above \[41–43\]. At the start of the experiment, the fly cannot know that it is its own turning manoeuvres that cause the switch from cold to hot and vice versa. To find out, the fly has to activate the behavioural modules it has available in this restrained situation and has to register whether one of them might have an influence on the punishing heat beam. There is no appropriate sensory stimulus from outside to elicit the respective behaviour. The fly must have a way to initiate its behaviours itself, in order to correlate these actions with the changes in the environment. Clearly, the brain is built such that under certain circumstances the items of the behavioural repertoire can get released independent of sensory stimuli.

The fly cannot know the solutions to most real-life problems. Beyond behaving unpredictably to evade predators or outcompete a competitor, all animals must explore, must try out different solutions to unforeseen problems. Without behaving variably, without acting rather than passively responding, there can be no success in evolution. Those individuals who have found the best balance between flexible actions and efficient responses are the ones who have succeeded in evolution. It is this potential to behave variably, to initiate actions independently of the stimulus situation, which provides animals with choices.

# Freedom of choice

The neurobiological basis of decision-making can also be studied very well in invertebrate models. For instance, isolated leech nervous systems chose either a swimming motor programme or a crawling motor programme to an invariant electrical stimulus \[78–80\]. Every time the stimulus is applied, a set of neurons in the leech ganglia goes through a so far poorly understood process of decision-making to arrive either at a swimming or at a crawling behaviour. The stimulus situation could not be more perfectly controlled than in an isolated nervous system, excluding any possible spurious stimuli reaching sensory receptors unnoticed by the experimenter. In fact, even hypothetical 'internal stimuli', generated somehow by the animal must in this case be coming from the nervous system itself, rendering the concept of 'stimulus' in this respect rather useless. Yet, under these 'carefully controlled experimental circumstances, the animal behaves as it damned well pleases' (Harvard Law of Animal Behaviour) \[34\].

Seymour Benzer, one of the founders of Neurogenetics, captured this phenomenon in the description of his first phototaxis experiments in 1967: ' … if you put flies at one end of a tube and a light at the other end, the flies will run to the light. But I noticed that not every fly will run every time. If you separate the ones that ran or did not run and test them again, you find, again, the same percentage will run. But an individual fly will make its own decision'. (cited from Brown & Haglund (1994) *J. NIH Res.* **6**, 66–73). Not even 10 years later, Quinn *et al.* separated flies, conditioned to avoid one of two odours, into those that did avoid the odour and those that did not. In a subsequent second test, they found that both the avoiders and the non-avoiders separated along the same percentages as in the first test, prompting the authors to conclude: 'This result suggests that the expression of learning is probabilistic in every fly' \[81\]. Training shifted the initial 50–50 decision of the flies away from the punished odour, but the flies still made the decisions themselves—only with a different probability than before training. Most recently, in the experiments described above, the tethered flies without any feedback made spontaneous decisions to turn one way or another \[63\]. These are only three examples from more than 40 years in which many behavioural manifestations of decision-making in the fly brain have been observed. Like heat, flies can control also odour intensity with their yaw torque \[40\]. They can control the angular velocity of a panorama surrounding them not only by yaw torque but also by forward thrust, body posture or abdomen bending \[82\]. In ambiguous sensory situations, they actively switch between different perceptual hypotheses, they modify their expectations about the consequences of their actions by learning and they can actively shift their focus of attention restricting their behavioural responses to parts of the visual field \[83,84\]. These latest studies prompted further research into the process of the endogenous direction of selective attention in flies \[85–89\]. Martin Heisenberg realized early on \[90\] that such active processes entail the sort of fundamental freedom required for a modern concept of free will and keeps prominently advocating this insight today \[91\].

John Searle has described free will as the belief 'that we could often have done otherwise than we in fact did' \[92\]. Taylor & Dennett cite the maxim 'I could have done otherwise' \[93\]. Clearly, leeches and flies could and can behave differently in identical environments. While some argue that unpredictable (or random) choice does not qualify for their definition of free will \[2\], it is precisely the freedom from the chains of causality that most scholars see as a crucial prerequisite for free will. Importantly, this freedom is a necessary but not a sufficient component of free will. In order for this freedom to have any bearing on moral responsibility and culpability in humans, more than mere randomness is required. Surely, no one would hold a person responsible for any harm done by the random convulsions during an epileptic seizure. Probably because of such considerations, two-stage models of free will have been proposed already many decades ago, first by James \[94\], later also by Henri Poincaré, Arthur Holly Compton, Karl Popper, Henry Margenau, Daniel Dennett, Robert Kane, John Martin Fisher, Alfred Mele, Stephen Kosslyn, Bob Doyle and Martin Heisenberg (cited, reviewed and discussed in \[7\]), as well as Koch \[9\]: one stage generates behavioural options and the other one decides which of those actions will be initiated. Put simply, the first stage is 'free' and the second stage is 'willed'. This implies that not all chance events in the brain must manifest themselves immediately in behaviour. Some may be eliminated by deterministic 'selection' processes before they can exert any effects. Analogous to mutation and selection in evolution, the biological process underlying free will can be conceptualized as a creative, spontaneous, indeterministic process followed by an adequately determined process, selecting from the options generated by the first process. Freedom arises from the creative and indeterministic generation of alternative possibilities, which present themselves to the will for evaluation and selection. The will is adequately determined by our reasons, desires and motives—by our character—but it is not *pre*-determined. John Locke (1689, p. 148) already separated free from 'will', by attributing free to the agent and not the will: 'I think the question is not proper, whether the will be free, but whether a man be free' \[95\]. Despite the long tradition of two-stage models of free will, only now are the first, tangible scientific pieces of evidence being published. For instance, the independent discovery of nonlinear mechanisms in brains from different phyla is compatible with such two-stage models \[63,74\]. Essentially, the existence of neural circuits implementing a two-stage model of free will 'would mean that you can know everything about an organism's genes and environment yet still be unable to anticipate its caprices' \[96\]. Importantly, this inability is not due to inevitable stochasticity beyond control; it is due to dedicated brain processes that have evolved to generate unpredictable, spontaneous actions in the face of pursuit–evasion contests, competition and problem-solving.

# Consciousness and freedom

It thus is no coincidence that we all feel that we possess a certain degree of freedom of choice. It makes sense that depriving humans of such freedom is frequently used as punishment and the deprived do invariably perceive this limited freedom as undesirable. This experience of freedom is an important characteristic of what it is like to be human. It stems in part from our ability to behave variably. Voltaire expressed this intuition in saying 'Liberty then is only and can be only the power to do what one will'. The concept that we can decide to behave differently even under identical circumstances underlies not only our justice systems. Electoral systems, our educational systems, parenting and basically all other social systems also presuppose behavioural variability and at least a certain degree of freedom of choice. Games and sports would be predictable and boring without our ability of constantly changing our behaviour in always the same settings.

The data reviewed above make clear that the special property of our brain that provides us with this freedom surely is independent of consciousness. Consciousness is not a necessary prerequisite for a scientific concept of free will. Clearly, a prisoner is regarded as un-free, irrespective of whether he is aware of it or not. John Austin \[97\] provides another instructive example 'Consider the case where I miss a very short putt and kick myself because I could have holed it'. We sometimes have to work extremely hard to constrain our behavioural variability in order to behave as predictably as possible. Sports commentators often use 'like a machine' to describe very efficient athletes. Like practice, conscious efforts are able to control our freedom up to a certain degree. Compare, for instance, a line that you quickly drew on a piece of paper, with a line that was drawn with the conscious effort of making it as straight as possible. However, the neural principle underlying the process generating the variability is beyond total conscious control, requiring us to use rulers for perfectly straight lines. Therefore, the famous experiments of Benjamin Libet and others since then \[2,4,5,98–100\] only serve to cement the rejection of the metaphysical concept of free will and are not relevant for the concept proposed here. Conscious reflection, meditation or discussion may help with difficult decisions, but this is not even necessarily the case. The degree to which our conscious efforts can affect our decisions is therefore central to any discussion about the degree of responsibility our freedom entails, but not to the freedom itself.

# The self and agency

In contrast to consciousness, an important part of a scientific concept of free will is the concept of 'self'. It is important to realize that the organism generates an action itself, spontaneously. In chemistry, spontaneous reactions occur when there is a chemical imbalance. The system is said to be far from thermodynamic equilibrium. Biological organisms are constantly held far from equilibrium, they are considered open thermodynamic systems. However, in contrast to physical or chemical open systems, some of the spontaneous actions initiated by biological organisms help keep the organism away from equilibrium. Every action that promotes survival or acquires energy sustains the energy flow through the open system, prompting Georg Litsche to define biological organisms as a separate class of open systems (i.e. 'subjects'; \[101\]). Because of this constant supply of energy, it should not be surprising to scientists that actions can be initiated spontaneously and need not be released by external stimuli. In controlled situations where there cannot be sufficient causes outside the organism to make the organism release the particular action, the brain initiates behaviour from within, potentially using a two-stage process as described above. The boy ceases to play and jumps up. This sort of impulsivity is a characteristic of children every parent can attest to. We do not describe the boy's action with 'some hidden stimuli made him jump'—he jumped of his own accord. The jump has all the qualities of a beginning. The inference of agency in ourselves, others and even inanimate objects is a central component of how we think. Assigning agency requires a concept of self. How does a brain know what is self?

One striking characteristic of actions is that an animal normally does not respond to the sensory stimuli it causes by its own actions. The best examples are that it is difficult to tickle oneself and that we do not perceive the motion stimuli caused by our own eye saccades or the darkness caused by our eye blinks. The basic distinction between *self*-induced (re-afferent) and externally generated (ex-afferent) sensory stimuli has been formalized by von Holst & Mittelstaedt \[102\]. The two physiologists studied hoverflies walking on a platform surrounded by a cylinder with black and white vertical stripes. As long as the cylinder was not rotated, the animals seemed to behave as if they were oblivious to the stripes. However, as soon as the cylinder was switched on to rotate around the flies, the animals started to turn in register with the moving stripes, in an attempt to stabilize their orientation with respect to the panorama. Clearly, when the animals turned themselves, their eyes perceived the same motion stimuli as when the cylinder was rotated. The two scientists concluded that the animals detect which of these otherwise very similar motion signals are generated by the flies and which are not and dubbed this the 'principle of reafference'. To test the possibility that the flies just blocked all visual input during self-initiated locomotion, the experimenters glued the heads of the animals rotated by 180° such that the positions of the left and right eye were exchanged and the proboscis pointed upwards. Whenever these 'inverted' animals started walking in the stationary striped cylinder, they ran in constant, uncontrollable circles, showing that they did perceive the *relative* motion of the surround. From this experiment, von Holst and Mittelstaedt concluded that self-generated turning comes with the expectation of a visual motion signal in the opposite direction that is perceived but normally does not elicit a response. If the visual motion signal is not caused by the animal, on the other hand, it most probably requires compensatory action, as this motion was not intended and hence not expected. The principle of reafference is the mechanism by which we realize which portion of the incoming sensory stream is under our own control and which portion is not. This is how we distinguish between those sensory stimuli that are consequences of our own actions and those that are not. Distinguishing self from 'world' is the prerequisite for the evolution of separate learning mechanisms for self- and world learning, respectively \[43\], which is the central principle of how brains balance actions and responses. The self/world distinction is thus the second important function of behavioural variability, besides making the organism harder to predict: by using the sensory feedback from our actions, we are constantly updating our model of how the environment responds to our actions. Animals and humans constantly ask: What happens if I do this? The experience of willing to do something and then successfully doing it is absolutely central to developing a sense of self and that we are in control (and not being controlled).

Thus, in order to understand actions, it is necessary to introduce the term self. The concept of self necessarily follows from the insight that animals and humans initiate behaviour by themselves. It would make no sense to assign a behaviour to an organism if any behavioural activity could, in principle, be traced back by a chain of causations to the origin of the universe. An animal or human being is the agent causing a behaviour, as long as no sufficient causes for this activity to occur are coming from outside the organism. Agency is assigned to entities who initiate actions themselves. Agency is crucial for moral responsibility. Behaviour can have good or bad consequences. It is the agent for whom the consequences matter the most and who can be held responsible for them.

# Why still use the term free will today?

By providing empirical data from invertebrate model systems supporting a materialistic model of free will, I hope to at least start a thought process that abandoning the metaphysical concept of free will does not automatically entail that we are slaves of our genes and our environment, forced to always choose the same option when faced with the same situation. In fact, I am confident I have argued successfully that we would not exist if our brains were not able to make a different choice even in the face of identical circumstances and history. In this article, I suggest re-defining the familiar free will in scientific terms rather than giving it up, only because of the historical baggage all its connotations carry with them. One may argue that 'volition' would be a more suitable term, less fraught with baggage. However, the current connotations of volition as 'willpower' or the forceful, conscious decision to behave against certain motivations render it less useful and less general a term than free will. Finally, there may be a societal value in retaining free will as a valid concept, since encouraging a belief in determinism increases cheating \[103\]. I agree with the criticism that retention of the term may not be ideal, but in the absence of more suitable terms, free will; remains the best option.

I no longer agree that ' ''free will'' is (like ''life'' and ''love'') one of those culturally useful notions that become meaningless when we try to make them ''scientific'' ' \[96\]. The scientific understanding of common concepts enrich our lives, they do not impoverish them, as some have argued \[100\]. This is why scientists have and will continue to try and understand these concepts scientifically or at least see where and how far such attempts will lead them. It is not uncommon in science to use common terms and later realize that the familiar, intuitive understanding of these terms may not be all that accurate. Initially, we thought atoms were indivisible. Today we do not know how far we can divide matter. Initially, we thought species were groups of organisms that could be distinguished from each other by anatomical traits. Today, biologists use a wide variety of species definitions. Initially, we thought free will was a metaphysical entity. Today, I am joining a growing list of colleagues who are suggesting it is a quantitative, biological trait, a natural product of physical laws and biological evolution, a function of brains, maybe their most important one.

## Acknowledgements

Concepts and ideas in several sections of this article have been adapted from a to-be-published presentation of Martin Heisenberg. I am very grateful for his sharing this presentation with me. I am also indebted to Christopher Harris, Bob Doyle, Matt Leifer, Sandeep Gautam, Andrew Lang, Julien Colomb and two anonymous referees for helpful comments on an earlier version of the manuscript.

# References


abstract: Protocol design complexity has increased substantially during the past decade and this in turn has adversely impacted drug development economics and performance. This article reviews the results of two major Tufts Center for the Study of Drug Development studies quantifying the direct cost of conducting less essential and unnecessary protocol procedures and of implementing amendments to protocol designs. Indirect costs including personnel time, work load and cycle time delays associated with complex protocol designs are also discussed. The author concludes with an overview of steps that research sponsors are taking to improve protocol design feasibility.
author: Kenneth Getz
date: 2014-05-12
institute: Center for the Study of Drug Development, School of Medicine, Tufts University, 75 Kneeland Street, Suite 1100, Boston, MA 02111, USA; E-Mail: <email@example.com>; Tel.: +1-617-636-3487; Fax: +1-617-636-2425
references:
title: Improving Protocol Design Feasibility to Drive Drug Development Economics and Performance

# 1. Introduction

Research and Development leaders widely agree that among the many factors that drive the high and rising cost of drug development, one of the most significant is protocol design \[1\]. As the blueprint articulating project strategy and directing project execution performed by both internal and external personnel, protocol design is uniquely positioned to fundamentally and directly impact—positively or negatively—drug development efficiency and economics.

Protocol design has historically been the domain of clinical research scientists. Under a resource-rich R&D operating environment, the regulatory approval of safe and effective medical products was the primary test for assessing study design quality. In the current resource-constrained operating environment, a primary test of quality study design seeks to exclude any protocol that substantially wastes scarce resources, gathers extraneous data, and exposes study subjects to unnecessary risk.

Making matters worse, in the current environment, a rising number of stakeholders influence protocol design practice and quality. Not only are scientists, regulatory agencies, health authorities, operating managers and key opinion leaders informing study design decision making, but also patients/patient organizations, investigative site personnel, health care providers, policymakers and payers are increasingly playing a role.

Interviews conducted among drug development executives suggest a number of considerations and pressures that protocol design decision-making must accommodate: New scientific understanding about chronic disease mechanisms and how to measure their progression and economic impact requires collecting more clinical data. Crowded classes of experimental therapies and the ongoing movement to develop stratified medicines compel research sponsors to collect more genetic and biomarker data.

Clinical scientists and statisticians often add procedures to gather more contextual data to aid in their interpretation of the findings and to guide development decisions. Clinical teams are now collecting biomarker data, genetic material, data on economic and therapeutic outcomes, and companion diagnostic data.

Clinical teams routinely add procedures guided by the belief that the marginal cost of doing so, relative to the entire clinical study budget, is small when the risk of not doing so is high \[1,2\]. Additional clinical data is collected as a precautionary measure in the event that a study fails to meet its primary and key secondary objectives. Data from these procedures may prove valuable in *post-hoc* analyses that reveal new and useful information about the progression of disease, its treatment and new directions for future development activity. Clinical teams add procedures for fear that they may neglect to collect data requested by regulatory agencies and health authorities, purchasers and payers. Failure to collect requested data elements could potentially delay regulatory submission, product launch and product adoption. And medical writers and protocol authors also often permit outdated and unnecessary procedures into new study designs because they are routinely included in legacy protocol authoring templates and operating policies.

Collectively, these factors have all contributed to the dramatic increase in protocol design complexity during the past decade. Research conducted by the Tufts Center for the Study of Drug Development (Tufts CSDD, Boston, MA, USA) documents this alarming trend (see Table 1) and characterizes rising levels of scientific complexity (e.g., number of endpoints; number of procedures performed; number of study volunteer eligibility criteria) and operating complexity (e.g., number of countries where clinical trials are conducted; number of investigative sites activated and monitored; number of patients screened and enrolled).

In 2012, to demonstrate safety and efficacy, a typical phase III protocol had 170 procedures on average performed on each study volunteer during the course of 11 visits across an average 230-day time span. Ten years ago, the typical phase III protocol had an average of 106 procedures, nine visits and an average 187-day time span. For the typical phase III protocol conducted in 2012, study volunteers came from an average of 34 countries and 196 research centers, up from 11 countries and 124 research centers ten years ago. And in 2012, to qualify to participate in a typical phase III protocol, each volunteer had to meet 50 eligibility criteria -- up from an average of 31 inclusion and exclusion criteria ten years ago \[1\].

ijerph-11-05069-t001_Table 1

Comparing Scientific and Logistical Complexity of a Typical Phase III Protocol Across Two Time Periods.

Design Characteristics

(All Values are Means)
2002
2012

Total number of endpoints
7
13

Total number of procedures
106
167

Total number of eligibility criteria
31
50

Total number of countries
11
34

Total number of investigative sites
124
196

Total number of patients randomized
729
597

In an effort to reign in rising drug development costs, a growing number of pharmaceutical and biotechnology companies are looking to identify protocol design practices that can be modified. This article reviews two major Tufts CSDD studies that have quantified the direct cost impact of protocol design practices. This article also summarizes growing knowledge of the impact of protocol design complexity on indirect costs. At the conclusion of the article, steps that drug developers are taking to simplify protocol design and lower overall study costs are discussed.

# 2. Direct Costs Associated with Protocol Design Practices

## 2.1. Protocol Procedures

In a study completed in 2012, Tufts CSDD demonstrated that the cost of adding multiple individual protocol procedures, in the aggregate, is substantial. Fifteen mid-sized and large pharmaceutical and biotechnology companies participated in the study. Each company provided data on their phase II and III protocols targeting diseases across multiple therapeutic areas and executed by investigative sites dispersed globally since 2009. To minimize unusual and atypical designs, pediatric, medical device, orphan drug and extension studies were excluded from the sampling frame. In all, 116 unique phase II and III protocols having at least one procedure tied to a primary endpoint were analyzed \[2\].

Participating companies classified each protocol procedure according to the objective and endpoint it supported as defined by the clinical study report (CSR) and the study's specific statistical analysis plan (SAP) along the following lines:

1.  "Core" procedures—supported primary and/or secondary study objectives or primary or key secondary and safety endpoints.

2.  "Required" procedures—supported screening requirements and compliance-related activity including drug dispensing, informed consent form review, and study drug return.

3.  "Standard" procedures—are commonly performed during initial and routine study participant visits including medical history, height and weight measurement, adverse event assessment, and concomitant medication review.

4.  "Non-Core" procedures—supported supplemental secondary, tertiary and exploratory endpoints, and safety and efficacy procedures not associated with a study endpoint or objective.

Participating companies classified 25,103 individual phase II and III protocol procedures. Direct cost data for 16,607 procedures was analyzed using Medidata Solutions PICAS^®^ database. Overall, half of the total procedures per protocol were classified as "Core" to the study. Wide variability in the incidence of procedures supporting endpoint classifications was observed across therapeutic areas. Protocols targeting endocrine and central nervous system (CNS) disorders contained a higher relative average number of procedures supporting supplementary, tertiary, and exploratory procedures (e.g., "Non-Core") endpoints. Oncology protocols had the highest relative proportion of procedures supporting "Core" endpoints and objectives, and the lowest relative proportion supporting "Non-Core" endpoints.

The distribution of direct costs was similar to that of categorized procedures: Overall, 47.9% of the total study budget on average was spent on the direct costs to administer "Core" procedures in phase II and phase III protocols. The direct cost to administer "Required" (regulatory compliance) and "Standard" procedures for phase III protocols was 22.7% and 12.0% of the study budget respectively.

For phase III protocols alone, approximately half of the total direct cost (46.7%) was spent to administer "Core" procedures; 18.6% on average was spent to administer "Non-Core" procedures; 22.7% to administer procedures supporting screening requirements and regulatory compliance; and 12% supported "Standard" procedures (See Table 2).

ijerph-11-05069-t002_Table 2

Distribution of Procedures and Direct Costs per Procedure by End Point Classification.

| Endpoint Type | Phase II Procedures | Phase III Procedures | Phase II Procedure Costs | Phase III Procedure Costs |
|----|----|----|----|----|
| Core | 54.4% | 47.7% | 55.2% | 46.7% |
| Required | 8.0% | 10.0% | 16.3% | 22.7% |
| Standard | 19.7% | 17.6% | 15.4% | 12.0% |
| Non-Core | 17.9% | 24.7% | 13.1% | 18.6% |
| Total | 100% | 100% | 100% | 100% |

One out of five (22.1%) phase II and III protocol procedures, on average, supported tertiary and exploratory objectives and endpoints. The proportion of procedures collecting non-core data in phase III studies alone was even higher (24.7%). Non-core procedures consumed on average 19% of total direct costs for each phase III study, and 13% of total direct costs for each phase II study \[2\].

The estimated total cost to the pharmaceutical industry each year to perform procedures supporting "Non-Core" objectives and endpoints for all FDA-regulated phase II and III protocols is an estimated \$4–\$6 billion USD. This estimate is very conservative as it excludes all indirect costs for personnel and infrastructure required to capture, monitor, clean, analyze, manage and store extraneous protocol data and it does not include any estimate for the unnecessary risk to which patients may be exposed.

## 2.2. Protocol Amendments

A second Tufts CSDD study showed that the incidence of protocol amendments rises with an increase in protocol design complexity. Amendments that are implemented after protocols have been approved are commonplace, but they are widely viewed as a nuisance, resulting in unplanned study delays and additional costs. Although clinical research professionals view protocol amendments as major problems, it is perhaps more accurate to view amendments as attempts to address underlying protocol design problems and external factors impacting design strategies.

Conducted in 2010, Tufts CSDD convened a group of seventeen mid-sized and large pharmaceutical and biotechnology companies each of whom provided protocol amendment data from 3,410 protocols approved between January 2006 and December 2008. The protocols were representative of multiple therapeutic areas. Protocols approved within the most recent 12-months were excluded from the study because these had not had enough time to accumulate amendments \[3\].

Amendments were defined as any change to a protocol requiring internal approval followed by approval from the Institutional Review Board (IRB), Ethical Review Board (ERB) or regulatory authority. Detailed data on 3,596 amendments containing 19,345 total protocol modifications was analyzed. Companies participating in this study assigned specific changes made per amendment yielding a total of 6,855 changes classified. Companies also provided the top causes for amendments and rated each amendment in terms of whether it was "Completely Avoidable", "Somewhat Avoidable", "Somewhat Unavoidable" or "Completely Unavoidable".

Nearly all protocols required at least one amendment. Completed protocols across all phases had an average of 2.3 amendments, though later-stage phase II and III protocols averaged 2.7 and 3.5 amendments respectively. Each amendment required an average of 6.9 changes to the protocol. Therapeutic areas that had the highest incidence of amendments and changes per amendment included cardiovascular and GI protocols (See Table 3).

ijerph-11-05069-t003_Table 3

Mean Number of Amendments and Changes per Amendment per Protocol by Phase.

| Research Phase | Number of Amendments | Protocol Changes per Amendment |
|----------------|----------------------|--------------------------------|
| Phase I        | 1.9                  | 5.6                            |
| Phase II       | 2.7                  | 6.8                            |
| Phase III      | 3.5                  | 8.5                            |
| Phase IIIb/IV  | 2.6                  | 8.3                            |
| All phases     | 2.3                  | 6.9                            |

Of the 6,855 changes categorized, 16% were modifications made to the description and eligibility criteria of the patient population under investigation; 12% were adjustments made in the number and types of safety assessment procedures; 10% were edits and revisions made to the general information contained in the protocol (e.g., protocol title and study staff contact information).

Nearly 40% of all amendments occurred before the first study volunteer received his or her first dose in the clinical trial. This was most pronounced in phase I studies where 52% of amendments occurred prior to beginning patient enrollment. In phase II, III, and IIIb/IV studies, 37%, 30%, and 38% of amendments occurred before first patient first dose, respectively (see Figure 1).

The most common cause of amendments was the availability of new safety information (20%), followed by requests from regulatory agencies to amend the study (19%) and changes in the study strategy (18%). Protocol design flaws and difficulties recruiting study volunteers were also top cited causes at 11% and 9% of categorized amendments, respectively.

Two-thirds (63%) of amendments had causes that sponsor companies considered unavoidable, including amendments that were the result of new safety information, new regulatory requests, changes in the standard of care or study objectives. Nearly one-out-of-four protocol amendments (37%) was considered partially or completely avoidable.

Based on data provided by participating companies, on average the direct cost to implement (*i.e.*, first patient resubmitted under the revised protocol) a single protocol amendment is approximately \$500,000 (USD) in unplanned expense and adds 61 days to the project timeline. This figure undercounts the full economic impact as it does not include the cost of internal staff time dedicated to implementing each amendment, costs or fees associated with protocol language translation, and costs associated with resubmission to the local authority.

Tufts CSDD estimates that the total cost for sponsors to implement "avoidable" protocol amendments in 2014 was approximately \$2 billion (USD). This estimate is based on the incidence of amendments by phase for all active global FDA regulated clinical trials as reported by the agency; the direct cost to implement each amendment; and the proportion of amendments that is avoidable. This estimate does not include the billions of dollars realized annually in aggregate time-savings and earlier commercialization due to the elimination of each avoidable amendment \[3\].

To date, Tufts CSDD research on the direct cost impact of protocol design complexity has focused on non-core procedures. Tufts CSDD is currently conducting research looking at the direct cost of Core, Required and Standard procedures.

## 2.3. Indirect Costs

Tufts CSDD research is only beginning to quantify the indirect costs associated with protocol design complexity. These costs are no doubt many magnitudes higher than their estimated direct costs and include increasing cycle time and delays; full- and part-time staffing costs associated with project management and oversight, protocol administration and the gathering, monitoring, collecting, cleaning, analyzing, maintaining and storing of clinical data.

## 2.4. Cycle Time

Research published in peer-reviewed and trade literature shows a very clear relationship between study design complexity and performance: Study designs that include a relatively large number of eligibility requirements and unique procedures conducted frequently have lower study volunteer recruitment and retention rates, take longer, and generate lower quality clinical data than designs without such features \[4\].

Tufts CSDD research has demonstrated that the average overall duration of clinical trials—from "Protocol Ready" to the last patient completing her last visit—is 74% longer for complex clinical trials. And whereas three-out-of-four volunteers are randomized following screening and two-thirds of volunteers complete simpler protocols; 59% are randomized and less than half (48%) complete complex protocols \[4\].

Other studies in the literature corroborate these findings and provide insights into causes of cycle time delays \[5,6,7,8,9\]. Clark found, for example, that the collection of excessive and unnecessary clinical data is driving longer study durations. The author warned that data collection and regulatory agency submission delays may ultimately harm regulatory approval rates \[5\].

Ross and colleagues conducted a comprehensive analysis of peer-reviewed academic studies and found that health professionals were less likely to refer, and patients less likely to participate in, more complex clinical trials \[6\]. Madsen showed that patients are significantly less likely to sign the informed consent form when facing a more demanding protocol design \[7\].

In a study conducted by Boericke and Gwinn, the higher the number of study eligibility criteria, the more frequent and longer were the delays in completing clinical studies \[8\]. Andersen and colleagues showed that volunteer drop-out rates are much higher among patients participating in more complex clinical trials. The authors cautioned that when volunteers terminate their participation early and are lost to follow-up, the validity of the study results may be compromised \[9\].

## 2.5. Investigative Site Work Burden

Rising protocol design complexity also places additional execution burden on study staff. Work burden to administer protocol procedures was assessed using an approach developed by Tufts CSDD in 2008 and adapted from Medicare's Relative Value Unit (RVU) methodology \[4\]. The RVU scale was created by the Federal government in 1982 to determine reimbursement payment levels for physicians' relative costs instead of prevailing charges for medical procedures. Tufts CSDD mapped medical procedure RVUs to clinical trial procedures to derive Work Effort Units (WEU) per protocol procedure. For procedures that were not already assigned a Medicare RVU, a panel of 10 physicians at the Tufts University School of Medicine was convened to estimate the time spent per protocol procedure. "Investigative Site Work Burden" is the product of WEUs per procedure and the frequency of the procedures that were conducted over the course of the protocol.

Total investigative site work burden to administer phase II protocol procedures grew the fastest—73%—during the ten year period 2002 to 2012 outpacing growth in the total number of procedures performed per protocol. The work effort required of site personnel to administer procedures supporting phase I, III and IV protocols increased 48%, 56% and 57% during the same ten year period respectively (see Figure 2). Protocols targeting diseases associated with immunomodulation, oncology, CNS, and cardiovascular required the highest work effort from investigative site personnel to administer.

## 2.6. Study Monitoring, Data Management and Logistics Costs

Tufts CSDD studies have only just started to develop methodologies to measure the incremental indirect costs supporting study monitoring activity, data management and study logistics associated with complex protocol designs. In a preliminary analysis of regulatory submissions from nine major pharmaceutical companies, Tufts CSDD found that the average NDA in 1997 was nearly 70,000 pages in length and required 2 gigabytes of memory to store electronically. In 2012, the average NDA was almost 3.5 million pages and required 100 gigabytes of digital memory \[10\]. According to research from Medidata Solutions, a typical Phase III study collected nearly 1 million clinical data points in 2012, up from an estimated half that level 10 years ago \[1\].

Studies in the literature describe the impact of collecting too much data. Friedman and colleagues report that the high volume of data collected in today's protocols distracts research scientists, compromises the data analysis process and ultimately harms data quality \[11\]. As more data is collected during protocol administration, error rates increase, according to Nahm *et al*. \[12\]. Barrett found that more procedures per protocol were associated with a higher incidence of unused data in NDA (New Drug Application) submissions \[13\]. Abrams *et al.* found that unused clinical data compromises the data analysis process \[14\].

Preliminary results from a 2014 Tufts CSDD study suggest that study monitors perform 100% source data verification on clinical data gathered from all non-core procedures and that one-third of all queries per case report form are associated with these less essential procedures.

As mentioned earlier, Tufts CSDD research indicates that the number of countries and investigative sites where clinical trials are conducted has grown considerably. As clinical trials have become more globally dispersed, research sponsors have introduced numerous logistical complexities including delivering clinical supplies and collecting lab data from more remote locations; interacting with multiple regulatory agencies and health authorities; monitoring and overseeing sites in remote locations where infrastructure is more variable.

## 2.7. Steps to Optimize Study Design

During the past several years, a growing number of organizations have taken the initial steps of diagnosing their protocol design practices and comparing them with industry benchmarks. Sponsor organizations have referred to published data in peer-reviewed papers to compare internal study design practices with benchmarks. Commercially available software and consulting services are also available to assist sponsor companies in assessing the problem and implementing new practices to streamline and simplify study design \[1\].

In addition to diagnosing the problem, sponsor organizations are looking for ways to ensure that protocol feasibility is conducted more effectively. Whereas the scientific objectives solely dictated study design elements in the past, in an environment where resources and capital are far more limited, operating objectives now carry substantially more influence over design decisions. As such, the primary objective in optimizing protocol design is to perform great science that can be feasibly and cost-effectively executed.

Many sponsor organizations now solicit feedback from principal investigators, study coordinators and from patients to identify areas where study design feasibility can be improved prior to final approval of the protocol. Some of these feedback mechanisms are conducted as in-person meetings and focus groups while others are conducted online using social and digital media communities \[15\].

Protocol authoring practices received a valuable new reference resource in 2013. A group of international study design stakeholders issued a collection of checklists and guidelines designed to ensure that the purpose of protocol procedures are transparent and tied to core objectives and endpoints. The SPIRIT checklist was developed with input from 115 multi-disciplinary contributors from medical writing, journal editors, regulatory agencies, ethics committees, clinical research and health care professionals.

The SPIRIT 2013 checklist and guidelines call for simple, minimalist designs that are clearly tied to core endpoints and objectives as defined by the clinical study report. Formatting conventions for various protocol elements (e.g., table of contents, glossary of terms, abbreviated terms) are also provided. The SPIRIT checklist has been pilot tested and can be downloaded for free \[16\].

A 2013 Tufts CSDD study documented the creation of internal governance committees charged with challenging clinical teams to improve protocol feasibility \[1\]. Pharmaceutical and biotechnology companies began establishing these committees 24 months ago in an effort to adopt a more systematic and long-term approach to optimizing study design. Committees are positioned within their respective organizations as objective governance and assessment mechanisms, offering guidance and input into the existing protocol review process without requiring organizations to alter legacy study design practices and procedures.

The committees raise clinical team awareness of the impact that design decisions have on study budgets and on study execution feasibility. Committees typically provide input into the study design just prior to final protocol approval and they routinely offer insight into how protocol designs can be streamlined and better "fit to purpose". Long term, internal facilitation committees may assist organizations in fundamentally improving their study design practices.

Nearly all of the facilitation committees are comprised of cross-functional representatives who volunteer their time. Committees representatives come from a variety of functions including clinical development; clinical operations; statistics; data management; medical writing; clinical pharmacology; regulatory; safety; and pharmacovigilance in addition to finance and/or procurement. Although some committees have more expansive goals than do others, all committees are charged with simplifying and streamlining protocols by reducing unnecessary procedures and the number of amendments. To meet these goals, committees assess the direct cost to perform non-core procedures and core procedures that may be conducted more frequently than necessary.

Adaptive trial designs represent an optimization opportunity that has received limited attention from research sponsors to date. Adaptive trial designs are preplanned, typically through the use of trial simulations and scenario planning where one or more specified clinical trial design elements are modified and adjusted—while the trial is underway—based on an analysis of interim data.

Tufts CSDD estimates that approximately one out of five (20%) late stage clinical trials are using a simple adaptive design approach. A much lower percentage—5%—is using more sophisticated adaptations (adaptive dose range studies and seamless phase II-III studies). Sponsor companies report that they expect the adoption of adaptive trial designs in earlier exploratory phase clinical trials to increase significantly over the next several years \[17\].

Study terminations due to safety and/or efficacy futility are the most common simple adaptive design used and they are more widely used at this time. Sponsor companies have found that early terminations due to futility are relatively easy to implement and a growing number of companies similarly view studies employing sample size re-estimation adaptations.

Although the concept of adaptive trial designs has been widely discussed for more than ten years, adoption has been slow for a variety of reasons. Internal organizational resistance appears to be the primary factor limiting more widespread adoption \[17\]. Based on interviews with company executives, regulatory agency receptivity to the use of adaptive trial designs does not appear to be as significant a barrier to adoption though agency clarity with regard to its position on the use of adaptive designs appears to be lacking.

Clinical teams and operating functions perceive enrollment and logistical factors—specifically delays and disruptions in trial execution, patient participation and distribution of clinical supplies—as major barriers to adoption. Sponsors are also concerned about introducing bias following interim analyses; the lack of adaptive trial design experience among both internal development teams and external contract research organizations; gaps in infrastructure and technology to implement more sophisticated adaptive designs; and the limited capacity of independent data monitoring committees.

Adaptive trial designs hold promise in optimizing study design. Early study terminations due to futility and sample size re-estimation could save up to a hundred million dollars (USD) in direct and indirect costs annually per pharmaceutical company depending on clinical trial scope, when the trial is actually terminated, and on the sponsor's overall implementation of this adaptive approach across the development portfolio. Perhaps the greatest impact from adaptive trial designs will come from improvements in late stage success rates. Even modest improvements in success rates for new molecular entities (NME) and new biologic entities (BME) represent billions of dollars in increased revenue potential for research sponsors.

In the immediate term, adaptive trial designs are offering cross-functional teams direct insights into study design through scenario planning and trial simulation prior to finalizing the protocol. Rigorous upfront planning—similar to optimization practices for traditional study designs—is forcing organizations to challenge protocol feasibility prior to placing the protocol in the clinic.

# 3. Conclusions

Protocol design holds the key to fundamentally and sustainably transforming drug development performance, cost and success. Research conducted by Tufts CSDD and others contributes to our growing understanding of specific opportunities that can be implemented to optimize protocol design. There is no question that pharmaceutical and biotechnology companies will adopt and implement new approaches to test and modify study design feasibility with the goal of best executing excellent science. More flexible and adaptive trial designs are expected to play a growing role in helping to optimize study design by compelling sponsor companies to perform more rigorous upfront planning and simulation and to implement preplanned adaptions that may lower fixed operating costs and ultimately improve program success rates.

## Acknowledgments

The author wishes to thank his colleagues at the Tufts Center for the Study of Drug Development and participating companies for their support of the Center's research studies and Medidata Solutions for providing an unrestricted study grant to quantify the direct cost of procedures by endpoint classification.

## Conflicts of Interest

The author declares no conflict of interest.

# References


author: James E Ferrell
date: 2009
institute: 1Department of Chemical and Systems Biology, Stanford University School of Medicine, Stanford, CA 94305-5174, USA
title: Q&A: Systems biology

# What is systems biology?

Systems biology is the study of complex gene networks, protein networks, metabolic networks and so on. The goal is to understand the design principles of living systems.

# How complex are the systems that systems biologists study?

That depends. Some people focus on networks at the 'omics'-scale: whole genomes, proteomes, or metabolomes. These systems can be represented by graphs with thousands of nodes and edges (see Figure 1). Others focus on small subcircuits of the network; say a circuit composed of a few proteins that functions as an amplifier, a switch or a logic gate. Typically, the graphs of these systems possess fewer than a dozen (or so) nodes. Both the large-scale and small-scale approaches have been fruitful.

# Why is systems biology important?

Stas Shvartsman at Princeton tells a story that provides a good answer to this question. He likens biology's current status to that of planetary astronomy in the pre-Keplerian era. For millennia people had watched planets wander through the nighttime sky. They named them, gave them symbols, and charted their complicated comings and goings. This era of descriptive planetary astronomy culminated in Tycho Brahe's careful quantitative studies of planetary motion at the end of the 16th century. At this point planetary motion had been described but not understood.

Then came Johannes Kepler, who came up with simple theories (elliptical heliocentric orbits; equal areas in equal times) that empirically accounted for Brahe's data. Fifty years later, Newton's law of universal gravitation provided a further abstraction and simplification, with Kepler's laws following as simple consequences. At that point one could argue that the motions of the planets were understood.

Systems biology begins with complex biological phenomena and aims to provide a simpler and more abstract framework that explains why these events occur the way they do. Systems biology can be carried out in a 'Keplerian' fashion – look for correlations and empirical relationships that account for data – but the ultimate hope is to arrive at a 'Newtonian' understanding of the simple principles that give rise to the complicated behaviors of complex biological systems.

Note that Kepler postulated other less-enduring mathematical models of planetary dynamics. His *Mysterium Cosmographicum* showed that if you nest spheres and Platonic polyhedra in the right order (sphere-octahedron-sphere-icosahedron-sphere-dodecahedron-sphere-tetrahedron-sphere-cube-sphere), the sizes of the spheres correspond to the relative sizes of the first six planets' orbits. This simple, abstract way of accounting for empirical data was probably just a happy coincidence. Happy coincidences are a potential danger in systems biology as well.

# Is systems biology the antithesis of reductionism?

In a limited sense, yes. Some 'emerging properties', as discussed below, disappear when you reduce a system to its individual components.

However, systems biology stands to gain a lot from reductionism, and in this sense systems biology is anything but the antithesis of reductionism. Just as you can build up to an understanding of complex digital circuits by studying individual electronic components, then modular logic gates, and then higher-order combinations of gates, one may well be able to achieve an understanding of complex biological systems by studying proteins and genes, then motifs (see below), and then higher-order combinations of motifs.

# What are emergent properties?

Systems of two proteins or genes can do things that individual proteins/genes cannot. Systems of ten proteins or genes can do things that systems of two proteins/genes cannot. Those things that become possible once a system reaches some level of complexity are termed emergent properties.

# Can you give a concrete example of an emergent property?

Three proteins connected in a simple negative-feedback loop (A → B → C -\| A) can function as an oscillator; two proteins (A → B-\|A)can not. Two proteins connected in a simple negative-feedback loop can convert constant inputs into pulsatile outputs; a one-protein loop (A -\| A) cannot. So pulse generation emerges at the level of a two-protein system and oscillations emerge at the level of a three-protein system.

# In systems biology there is a lot of talk about nodes and edges. What is a node? An edge?

Biological networks are often depicted graphically: for example, you could draw a circle for protein A, a circle for protein B, and a line between them if A regulates B or vice versa. The circles are the nodes in the graph of the A/B system. Nodes can represent genes, proteins, protein complexes, individual states of a protein, and so on.

A line connecting two nodes is an edge. The edge can be directed: for example, if A regulates B, we write an arrow – a directed edge – from A to B, whereas if B regulates A we write an arrow from B to A. Or the edge can be undirected; for example, it represents a physical interaction between A and B.

# Staying with graphs, what's a motif?

As defined by Uri Alon, a motif is a statistically over-represented subgraph of a graphical representation of a network. Motifs include things like negative feedback loops, positive feedback loops, and feed-forward systems.

# Isn't positive feedback the same thing as feed-forward regulation?

No. They are completely different. In a positive-feedback system, A activates B and B turns around to activate A. A transitory stimulus that activates A could lock the system into a self-perpetuating state where both A and B are active. In this way, the positive-feedback loop can act like a toggle switch or a flip-flop. A positive-feedback loop behaves much like a double-negative feedback loop, where A and B mutually inhibit each other. That system can act like a toggle switch too, except that it toggles between A on/B off and A off/B on states, rather than between A off/B off and A on/B on states. Good examples of this type of system include the famous lambda phage lysis/lysogeny toggle switch, and the CDK1/Cdc25/Wee1 mitotic trigger.

In a feed-forward system, A impinges upon C directly, but A also regulates B, which regulates C. A feed-forward system can be either 'coherent' or 'incoherent', depending upon whether the route through B does the same thing to C as the direct route does. There is no feedback – A affects C, but C does not affect A – and the system cannot function as a toggle switch. A good example of feed-forward regulation is the activation of the protein kinase Akt by the lipid second messanger PIP3 (PIP3 binds Akt, which promotes Akt activation, and PIP3 also stimulates the kinase PDK1, which phosphorylates Akt and further contributes to Akt activation). Since both routes contribute to Akt activation, this is an example of coherent feed-forward regulation. Uri Alon's classic analysis of motifs in *Escherichia coli* gene regulation identified numerous coherent feed-forward circuits in that system.

# In high school I hated physics and math, but I loved biology. Should I go into systems biology?

No.

# What kind of physics and math is most useful for understanding biological systems?

Some level of comfort in doing simple algebra and calculus is a must. Beyond that, probably the most useful math is nonlinear dynamics. The Strogatz textbook mentioned below is a great introduction to nonlinear dynamics.

# Do I need to understand differential equations?

Systems biologists often model biological processes with ordinary differential equations (ODEs), but the fact is that almost none of them can be solved exactly. (The one that can be solved exactly describes exponential approach to a steady state, and it's something every biologist should work out at some point in his or her training.) Most often, systems biologists solve their ODEs numerically, often with canned software packages like Matlab or Mathematica.

Ideally, a model should not only reproduce known biology and predict unknown biology, it should also be 'robust' in important respects.

# What is robustness, and why is it important to systems biologists?

Robustness is the imperviousness of some performance characteristic of a system in the face of some sort of insult – such as stochastic fluctuations, environmental insults, or deletion of nodes from the system. For example, the period of the circadian oscillator is robust with respect to changes in the temperature of the environment. Robustness can be quantitatively defined as the inverse of sensitivity, which itself can be defined a few ways – often sensitivity is taken to be:

$$\frac{d\ln Response}{d\ln Pertubation}$$

so that robustness becomes

$$\frac{d\ln Pertubation}{d\ln Response}$$

Robustness is important to systems biologists because of the attractiveness of the idea that a biological system must function reliably in the face of myriad uncertainties. Maybe robustness, more than efficiency or speed, is what evolution must optimize to create successful biological systems.

Modeling can provide some insight into the robustness of particular networks and circuits. Just as a biological system must be robust with respect to insults the system is likely to encounter, a successful model should also be robust with respect to parameter choice. If a model 'works', but only for a precisely chosen set of parameters, the system it depicts may be too finicky to be biologically useful, or to have been 'found' in evolution.

# What other types of models are useful in systems biology?

ODE models assume that each dynamical species in the model – each protein, protein complex, RNA, or whatever – is present in large numbers. This is sometimes true in biological systems. For example, regulatory proteins are often present at concentrations of 10 to 1,000 nM. For a four picoliter eukaryotic cell, this corresponds to 24,000 to 2,400,000 molecules per cell. This is probably large enough to warrant ODE modeling. However, genes and some mRNAs are present at concentrations of one or two molecules per cell. At such low numbers, each individual transcriptional event or mRNA degradation event becomes a big deal, and the appropriate type of modeling is stochastic modeling.

Sometimes systems are too complicated, or have too many unknown parameters to warrant ODE modeling. In these cases, Boolean models and probabilistic Bayesian models can be particularly useful.

Sometimes it is important to see how dynamical behaviors propagate through space, in which case either partial differential equation (PDE) models or stochastic reaction/diffusion models may be just the ticket.

# Where can I go for more information?

## Review articles

Hartwell LH, Hopfield JJ, Leibler S, Murray AW: **From molecular to modular biology**. *Nature* 2005, **402(Suppl):**C47–C52.

Kirschner M: **The meaning of systems biology**. *Cell* 2005, **121:**503–504.

Kitano H: **Systems biology: a brief overview**. *Science* 2002, **295:**1662–1664.

## Textbooks

Alon U: *An Introduction to Systems Biology: Design Principles of Biological Circuits*. Boca Raton, FL: Chapman & Hall/CRC; 2006.

Heinrich R, Schuster S: *The Regulation of Cellular Systems*. Berlin: Springer; 1996.

Klipp E, Herwig R, Kowald A, Wierling C, Lehrach H: *Systems Biology in Practice: Concepts, Implementation and Application*. Weinheim, Germany: Wiley-VCH; 2005.

Palsson B: *Systems Biology: Properties of Reconstructed Networks*. Cambridge University Press; 2006.

Strogatz SH: *Nonlinear Dynamics and Chaos: With Applications to Physics, Biology, Chemistry and Engineering*. Boulder, CO: Westview Press; 2001.