# Cracking the neural code of how the brain represents time may make the dualistic stance obsolete

Virginie van Wassenhove

► To cite this version:

Virginie van Wassenhove. Cracking the neural code of how the brain represents time may make the dualistic stance obsolete. Timing & Time Perception Reviews, 2023, pp.1-9. 10.1163/22134468-bja10099. hal-04336571

HAL Id: hal-04336571

https://hal.science/hal-04336571v1

Submitted on 11 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

# Cracking the Neural Code of How the Brain Represents Time
## May Make the Dualistic Stance Obsolete

Virginie van Wassenhove

CEA, DRF/Joliot, NeuroSpin, INSERM, CNRS, Cognitive Neuroimaging Unit,

Université Paris- Saclay, 91191 Gif/Yvette, France

ORCID iD: 0000-0002-2569-5502

E-mail: virginie.van.wassenhove@gmail.com

Received 20 April 2023; accepted 30 October 2023

## Abstract

This is an invited commentary on the position paper of Gruber, Block and Montemayor (2022). First, I oppose the premise in Gruber *et al.*'s proposal according to which there exists a 'real' and an 'illusory' time. Second, our knowledge about the universe is hypothesized, tested, and verified by the most complex physical systems known to date (brains) hence postulating the coexistence of an 'absolute' and a 'real', or an 'illusory' and a 'nonillusory' time might be unnecessary. Instead, and parsimoniously, (organic or atomic) measuring devices for duration, simultaneity, order, and so forth, have variable precision. The most exquisite clock is atomic, a good enough one for survival on Earth is at the scale of neural networks. A difference between measuring devices and brains is that the latter compute on stored temporal information. Last, I suggest additional gadgets for the Information Gathering and Utilizing Systems (IGUs) proposal. In sum, I oppose some of the theoretical premises while lending support to the idea that representing temporal statistics is a useful heuristics for some aspects of temporal cognition. From the cognitive neuroscience standpoint, IGUs are compatible with information-theoretic approaches, and with current Bayesian and predictive models implemented in neural systems, in which brains code information and compute on symbolic representations.

## Keywords

absolute time, cognitive gadget, illusion, reality, relative time
# 1. Real (Physical) and Illusory (Manifest) Time: An Ontological Question or a Theoretical Stance?

> *The Brain — is wider than the Sky —*
> *For — put them side by side —*
> *The one the other will contain*
> *With ease — and you — beside —*
> *Emily Dickinson*

Gruber and colleagues (Gruber et al., 2022) depart from the strong dichotomy of what is ('manifest time') and what is not ('physical time') an illusion. Taking such a dualistic stance to the study of nature according to which, speaking loosely, 'physics describes reality and the rest is illusion' is scientifically untenable, intellectually disagreeable, and places the human condition in quite an absurd position. On the one hand, we, human beings, would stand as knowledgeable observers outside the realm of reality since our entire psychology is deemed remote from the described object under scrutiny (the site of illusion, namely the mind embodied in the brain). On the other hand, human beings (and their mind/brain) are very real since they, after all, elaborated the very equations and formalisms that grant physics access to a so-called reality.

The status of illusions in Gruber et al.'s position paper and in perceptual sciences (Rogers, 2022) is discussed because it creates epistemological issues that bear concrete relevance in experimental sciences, well beyond the study of time. For instance, the authors state (p. 3) 'The term illusion refers to a perception that has no basis in reality, which in turn defers the problem to what the currently accepted laws of physics suggest.' First, the observer (the locus of the illusion) is removed from the phenomenon under study since it has 'no basis in reality': this is a dualist stance despite the authors' claiming otherwise. Second, it implies that the knowledge of the observers is unreal or bears no relevance to 'physical reality' yet foundational work in cognitive science grounded the issue of 'psychological reality' (Chomsky, 1980), with its strongest form suggesting that linguistic rules and representations in the human mind implement truth value about the structure of the world. While arguments put forward in the psychological reality debate make strong philosophical statements, the pragmatic implication for neurosciences is that the brain hosts internal models of the world that can predict and evaluate the veracity of communicated signals, whether sensory or verbal. Denying psychological reality falls into a reductionist bias that cannot provide satisfactory explanations for the existence of minds. It may even lay roots for dangerous ideologies by abnegating the very existence of reason and its impact on political and socioeconomic systems. Third, the dichotomy between real and illusory assumes that the animate observer's sole goal is to represent veridically a reality 'out there'. The
basic requirement of mapping the world from exogenous distal sensory sources to compressed representations that our cognitive systems compute on (Gallistel & King, 2009) relies on a transduction process from physical energies to neural codes that are non-isomorphic: neurons code information in a manner that is not a facsimile of the 'reality out there'. If it were, there would be no mind, no use for language, music, or mathematics, no use for symbolic thought, no memory, no lived temporalities that appear to be illusory and we would be stuck in feeling and thinking the present. For instance, no energies in the visible spectrum are physically blue or red or purple: our brains assign a color to a particular frequency in the visible spectrum thanks to the receptors and neural machinery that convert electromagnetic energies of different wavelengths into perceptual qualities and linguistic conventions shared by our species.

Similarly, 'space' and 'time' are coded in a non-facsimile fashion so as to map the environment and communicate it, even in fascinating invertebrates like bees (Frisch, 2013; Moser et al., 2017). The translation of information into a neural code implies imprecision, and abstraction of the object to be computed (e.g., distance, spatial position of landmarks, duration, boundaries, events, sequences of temporal landmarks). A neural representation of this object — ultimately supporting not only its perception but, most importantly, the knowledge of that fact that it exists and is not hallucinated — is not simultaneous with the object: it is predicted ahead of its very existence through internal models of the brain, which comply with or falsify the sensory inputs, after it has existed. Hence, there is a major difference between postulated 'physical' and 'psychological' realities: in physics, the observer observes; in cognitive neuroscience, the observer observes recursively, that is, the brain uses internal representation of increasing abstraction on which it computes hierarchically (i.e., the outcome of one process becomes the input to the next). The study of brain and mind is self-contained: there is no other observer to one's psychological reality than awareness itself. This needs not be mystified further than what it is as this very reality is fascinating of its own right. What does that have to do with time?

If physicists can appeal to an observer, who assigns a 't' variable to any function describing the universe, cognitive neuroscientists cannot afford to do so unless they want to stand outside the realm of an explanatory science to the mind and to consciousness (van Wassenhove, 2017, 2023). In other words, the psychology of time (like spatial navigation, vision, language, memory, ...) needs to be explainable as symbolic rules and representations in organic matter, whose processes owe to be compatible with the laws of physics. Entropy and the laws of thermodynamics apply to aging: our brains age and the knowledge that 'I exist' through time (persistence) is encrypted in our minds. The authors (Gruber et al., 2022) mention that "A human needs to feel that she persists and is not simply a conglomerate of impermanent events as spacetime cosmologies suggest" (p. 202). This is only correct to the extent that humans do not 'feel persistence' but infer persistence of self.
In agreement with the authors' viewpoint, the duality of aging and self (change and persistence, respectively) is familiar to all biologists: none of the cells we are born with will be the same when we die, except for most neurons. In cognition, the emergent 'self' is not reducible to the dynamics of complex systems themselves but must appeal to a symbolic representation of a set of properties that define the 'self' within these dynamics. The bodily self evolves in real time; the self as identity we casually refer to in daily language is a complex construct made of past memories and projections of one's own finitude into the future. The simulations brains do in real time (brain activity we record) make up 'timelines' that are generated as coordinate systems enabling the representations of meaningful events at a given moment in time (Gauthier *et al.*, 2018; van Wassenhove, 2017, 2022).

This moment lasts a few seconds at most and it is compatible with the proposal of Information Gathering and Utilizing Systems (IGUs): this is because the information that populates what we call ‘a thought’ is highly compressed, discrete, and brief. The ranking of these moments is enabled by neural operations ordering information in past, present, and future: mental chronologies do not map onto physical time in a linear fashion (Dennett & Kinsbourne, 1992; Gallistel, 1990; Hayek, 1952; Lashley, 1951; van Wassenhove, 2023). In sum, thoughts are as real as black holes may be, and need not be displaced away from the very physical reality they belong to. From my experimentalist point of view, perceptions are as precise as life on Earth requires them to be for the organism to survive, live, evolve and compute.

## 2. Absolute and Real Time

*Psychological reality — that is, truth of a certain theory*
Chomsky, 1980: 191.

The expression that ‘time is passing (fast or slow)’ in our block universe is perhaps the logical outcome of coincidence detectors and egocentric observers from neurons to whole systems, and possibly beyond (social networks). Brains have to rule on their status (as complex dynamical systems) relative to their surrounding environments (full of other clocks) and orient in space–time (for thought and action). Our brains incorporate temporal statistics from the environment and the body (e.g., the rate of changes or their regularity in a given time lapse) and these quantifications contribute to the sense that time is passing (Hicks *et al.*, 1977) aka the ‘flow of time’. The neuronal calculations and operations are constitutive of representations, which feed the phenomenology of the ‘passing of time’.

Temporal scales in the living are also hierarchical: we have sensitive access only to the becoming of some of them (minutes) and a rational access to others (centuries). The lived temporal phenomenologies that accompany aging are
partial, malleable, and subjective; they may be comparable to IGUs' decisions on the fly. The temporal landmarks and episodes we live through aging are stored and time-stamped in episodic memory; this process requires sophisticated storage and selective retrieval mechanisms that appear unaccounted for by the currently proposed IGU's capacities. The present moment for brains is as thick as Husserl predicted it to be, to the exception that it is even more layered. Now this intuition has clear neuroscientific explanations. The analysis of information at different time-scales coexists hierarchically along cortical gradients of integration and segregation in the cortex (Buzsáki, 2006; Hasson et al., 2008; Murray et al., 2014; La Rocca et al., 2018) but the underlying processing does not directly or necessarily speak to the experienced temporality. A majority of temporal processes are cognitively impenetrable, that is, inaccessible to consciousness. Processing windows package information from different sources of energies into basic constituents of meaning such as syllables with integrative times of 200–300 ms (Poeppel, 2003; van Wassenhove et al., 2007). Perceiving a syllable is not accompanied by the perception of these 200–300 ms time chunks. This is because ‘perceiving’ time is the outcome of a trade-off with perceiving information (Polti et al., 2018). Assessing these ‘unconscious’ temporalities requires the use of dedicated and sophisticated experimental paradigms: verbal reports are insufficient.

I agree with the IGU approach that representing time does not require the sophistication of human brains and many species can estimate durations with relative precision from a few seconds to a few minutes relative to the incidence of events in the sensory world (the ‘stimuli’). We speak of subjective time in reference to numerous statistical temporal measures (simultaneity, duration, order, rhythm, ...) reported by an individual (animal or human) with a certain precision, and with some perceptual and decisional biases (psychological measures) that are all quantifiable by subtle and hard experimentations. Objective time is reserved for measurements made by instruments such as clocks, whose atomic precision is far superior to the orders of magnitude characteristic (and useful) to the individuals’ biology.

## 3. IGU’s Gadgets and Beyond

Many perceptual illusions reveal evolutionary adaptive processes of neural circuits so that the likelihood of what happens given what is registered provides an efficient approximation. This stands as a generic principle of mental models (Craik, 1952; Helmholtz, 1867): ‘illusions’ reveal the sophistication of neural principles for experienced and remembered temporalities.

Combining the ‘tau’ effect (Helson & King, 1931) and the ‘kappa’ effect (Cohen et al., 1953), the rabbit illusion (Geldard & Sherrick, 1972) shows that isochronous stimuli in a sequence are perceived as spatially equidistant even when the spatial arrangement is not ‘physically’ so. This illusion has been elegantly argued
to be purely temporal (Dennett & Kinsbourne, 1992) and postdictive (Shimojo,
2014) although recent work rather shows the implication of a predictive analysis
of sequences (Grabot et al., 2021). The nonisomorphic representations of time
in neural systems cause fundamental issues when grasping the link between
serial ordering of observable events, sequences of neural events, and conscious
chronologies (Friston & Buzsáki, 2016; Gauthier et al., 2018, 2020; Grabot & van
Wassenhove, 2017; Lashley, 1951; van Wassenhove, 2023). The same case can
be entertained for duration itself (Efron, 1970a, b; van Wassenhove, 2009; van
Wassenhove & Lecoutre, 2015).

As for the coexistence of objective and subjective time, or ‘dual times’ in the
IGU framework, recent observations show that individuals can estimate the tem-
poral errors made in their estimates and productions of durations (Akdoğan &
Balcı, 2017; Balci et al., 2009; Kononowicz et al., 2019, 2022). This implies that
within brains, relative and absolute representations of durations co-exist: the rela-
tive time is the distribution of an individual’s duration production (accuracy or
distance relative to objective time, and precision or width around that value). The
absolute time resides in the individual’s ability to estimate their duration produc-
tion, which requires access to the variance of the distribution (knowledge of the
precision). Work from my lab has shown that the precision of timing is reflected
in the strength of neural oscillatory coupling (Grabot et al., 2019), and that meta-
cognitive inference can be found in the distance between neural trajectories rela-
tive to the actually produced duration (Kononowicz & Van Wassenhove, 2019;
Kononowicz et al., 2019).

4. Conclusions

To summarize, I support the core notion that IGUs represent temporal statistics
with additional gadgets to account for an observer's distinct temporal phenom-
enologies. However, the advantage of substituting or trading well-established cog-
nitive operations for IGU's gadgets is unclear. IGUs do not address higher-order
temporal phenomenologies such as perspectival or (dis)orientation or mental
time travel. IGUs may provide an essential bridge between the physical and bio-
logical studies of natural phenomena but should incorporate recent perceptual,
cognitive, and neuroscientific evidence. A tangible gain could also be to acknowl-
edge that relative to neurosciences, the observer cannot be set aside.

Acknowledgements

This work was supported in part by the European Commission H2020 Framework
Programme under Grant 101017727 of the Project EXPERIENCE.
## References

Akdoğan, B., & Balci, F. (2017). Are you early or late?: Temporal error monitoring. *J. Exp. Psychol. Gen.*, **146**, 347–361. doi: 10.1037/xge0000265.

Balci, F., Freestone, D., & Gallistel, C. R. (2009). Risk assessment in man and mouse. *Proc. Natl Acad. Sci. U.S.A.*, **106**, 2459–2463. doi: 10.1073/pnas.0812709106.

Buzsáki, G. (2006). *Rhythms of the Brain*. New York, NY, USA, Oxford Academic. doi: 10.1093/acprof:oso/9780195301069.001.0001.

Chomsky, N. (1980). Rules and representations. *Behav. Brain Sci.*, **3**, 1–15. doi: 10.1017/S0140525X00001515.

Cohen, J., Hansel, C. E. M., & Sylvester, J. D. (1953). A new phenomenon in time judgment. *Nature*, **172**, 901. doi: 10.1038/172901a0.

Craik, K. J. W. (1952). *The Nature of Explanation* (Vol. 445). Cambridge, United Kingdom, Cambridge University Press.

Dennett, D. C., & Kinsbourne, M. (1992). Time and the observer: The where and when of consciousness in the brain. *Behav. Brain Sci.*, **15**, 183–201. doi: 10.1017/S0140525X00068229.

Efron, R. (1970a). The minimum duration of a perception. *Neuropsychologia*, **8**, 57–63. doi: 10.1016/0028-3932(70)90025-4.

Efron, R. (1970b). The relationship between the duration of a stimulus and the duration of a perception. *Neuropsychologia*, **8**, 37–55. doi: 10.1016/0028-3932(70)90024-2.

Frisch, K. von. (2013). The dance language and orientation of bees. In *The Dance Language and Orientation of Bees*. Cambridge, MA, USA, Harvard University Press. doi: 10.4159/harvard.9780674418776.c44.

Friston, K., & Buzsáki, G. (2016). The functional anatomy of time: what and when in the brain. *Trends Cogn. Sci.*, **20**, 500–511. doi: 10.1016/j.tics.2016.05.001.

Gallistel, C. R. (1990). *The Organization of Learning*. Cambridge, MA, USA, The MIT Press.

Gallistel, C. R., & King, A. P. (2009). *Memory and the Computational Brain: Why Cognitive Science Will Transform Neuroscience*. Chichester, United Kingdom, Wiley-Blackwell.

Gauthier, B., Pestke, K., & van Wassenhove, V. (2018). Building the arrow of time ... over time: a sequence of brain activity mapping imagined events in time and space. *Cereb. Cortex*. **29**, 4398–4414. doi: 10.1093/cercor/bhy320.

Gauthier, B., Prabhu, P., Kotegar, K. A., & van Wassenhove, V. (2020). Hippocampal contribution to ordinal psychological time in the human brain. *J. Cogn. Neurosci.*, **32**(11), 2071–2086. doi: 10.1162/jocn_a_01586.

Geldard, F. A., & Sherrick, C. E. (1972). The cutaneous "rabbit": a perceptual illusion. *Science*, **178**, 178–179. doi: 10.1126/science.178.4057.178.

Grabot, L., & van Wassenhove, V. (2017). Time order as psychological bias. *Psychol. Sci.*, **28**, 670–678. doi: 10.1177/0956797616689369.

Grabot, L., Kononowicz, T. W., La Tour, T. D., Gramfort, A., Doyère, V., & Van Wassenhove, V. (2019). The strength of alpha-beta oscillatory coupling predicts motor timing precision. *J. Neurosci.*, **39**, 3277–3291. doi: 10.1523/JNEUROSCI.2473-18.2018.

Grabot, L., Kayser, C., & van Wassenhove, V. (2021). Postdiction: when temporal regularity drives space perception through prestimulus alpha oscillations. *Eneuro*, **8**, ENEURO.0030-21.2021. doi: 10.1523/ENEURO.0030-21.2021.
Gruber, R. P., Block, R. A., & Montemayor, C. (2022). Physical time within human time. *Front. Psychol.*, **13**, 718505. doi: 10.3389/fpsyg.2022.718505.

Hasson, U., Yang, E., Vallines, I., Heeger, D. J., & Rubin, N. (2008). A hierarchy of temporal receptive windows in human cortex. *J. Neurosci.*, **28**, 2539–2550. doi: 10.1523/JNEUROSCI.5487-07.2008.

Hayek, F. A. (1952). *The Sensory Order: An Inquiry into the Foundations of Theoretical Psychology*. Chicago, IL, USA, University of Chicago Press. doi: 10.7208/9780226321301.

Helmholtz, H. (1867). *Handbuch der physiologischen Optik*. Leipzig, Germany, L. Voss.

Helson, H., & King, S. M. (1931). The tau effect: an example of psychological relativity. *J. Exp. Psychol.*, **14**, 202–217. doi: 10.1037/h0071164.

Hicks, R. E., Miller, G. W., Gaes, G., & Bierman, K. (1977). Concurrent processing demands and the experience of time-in-passing. *Am. J. Psychol.*, **90**, 431–446. doi: 10.2307/1421874.

Kononowicz, T. W., & Van Wassenhove, V. (2019). Evaluation of self-generated behavior: untangling metacognitive readout and error detection. *J. Cogn. Neurosci.*, **31**, 1641–1657. doi: 10.1162/jocn_a_01442.

Kononowicz, T. W., Roger, C., & van Wassenhove, V. (2019). Temporal metacognition as the decoding of self-generated brain dynamics. *Cereb. Cortex*, **29**, 4366–4380. doi: 10.1093/cercor/bhy318.

Kononowicz, T. W., van Wassenhove, V., & Doyère, V. (2022). Rodents monitor their error in self-generated duration on a single trial basis. *Proc. Natl Acad. Sci. U.S.A.*, **119**, e2108850119. doi: 10.1073/pnas.2108850119

La Rocca, D., Zilber, N., Abry, P., van Wassenhove, V., & Ciuciu, P. (2018). Self-similarity and multifractality in human brain activity: A wavelet-based analysis of scale-free brain dynamics. *J. Neurosci. Methods*, **309**, 175–187. doi: 10.1016/j.jneumeth.2018.09.010.

Lashley, K. S. (1951). The problem of serial order in behavior: the Hixon Symposium. In L. A. Jeffress (Ed.), *Cerebral mechanisms in behavior* (pp. 112–136). New York, NY, USA, John Wiley & Sons.

Moser, E. I., Moser, M.-B., & McNaughton, B. L. (2017). Spatial representation in the hippocampal formation: a history. *Nat. Neurosci.*, **20**, 1448–1464. doi: 10.1038/nn.4653.

Murray, J. D., Bernacchia, A., Freedman, D. J., Romo, R., Wallis, J. D., Cai, X., Padoa-Schioppa, C., Pasternak, T., Seo, H., Lee, D., & Wang, X.-J. (2014). A hierarchy of intrinsic timescales across primate cortex. *Nat. Neurosci.*, **17**, 1661–1663. doi: 10.1038/nn.3862.

Poeppel, D. (2003). The analysis of speech in different temporal integration windows: cerebral lateralization as 'asymmetric sampling in time'. *Speech Commun*, **41**, 245–255. doi: 10.1016/S0167-6393(02)00107-3.

Polti, I., Martin, B., & van Wassenhove, V. (2018). The effect of attention and working memory on the estimation of elapsed time. *Sci. Rep.*, **8**, 6690. doi: 10.1038/s41598-018-25119-y.

Rogers, B. (2022). When is an illusion not an illusion? An alternative view of the illusion concept. *Front. Hum. Neurosci.*, **16**, 957740. doi: 10.3389/fnhum.2022.957740.

Shimojo, S. (2014). Postdiction: Its implications on visual awareness, hindsight, and sense of agency. *Front. Psychol.*, **5**, 196. doi: 10.3389/fpsyg.2014.00196.

van Wassenhove, V. (2009). Minding time in an amodal representational space. *Philos. Trans. R. Soc. Lond. B Biol. Sci.*, **364**, 1815–1830. doi: 10.1098/rstb.2009.0023.

van Wassenhove, V. (2017). Time consciousness in a computational mind/brain. *J. Consc. Stud.*, **24**, 177–202.

van Wassenhove, V. (2023). Cognitive neuroscience of time: nows, timelines, and chronologies. In: R. Lestienne and P. A Harris (Eds), *Time and science, Vol. 2, Life sciences* (pp. 97–129). World Scientific Publishing, London, United Kingdom. doi: 10.1142/9781800613751_0005.
van Wassenhove, V., & Lecoutre, L. (2015). Duration estimation entails predicting *when*. *Neuroimage*, *106*, 272–283. doi: 10.1016/j.neuroimage.2014.11.005.

van Wassenhove, V., Grant, K. W., & Poeppel, D. (2007). Temporal window of integration in auditory-visual speech perception. *Neuropsychologia*, *45*, 598–607. doi: 10.1016/j.neuropsychologia.2006.01.001.