# Aligning Statistical Models with Inference Goals in the Neuroscience of Language: A Dual-Dependency-Taxonomy

Sophie Bouton, Valerian Chambon, Narly Golestani, Elia Formisano,
Timothée Proix, Anne-Lise Giraud

<figure><img src="image_1.png" /></figure>

To cite this version:

Sophie Bouton, Valerian Chambon, Narly Golestani, Elia Formisano, Timothée Proix, et al. Aligning Statistical Models with Inference Goals in the Neuroscience of Language: A Dual-Dependency-Taxonomy. 2025. pasteur-05304225

HAL Id: pasteur-05304225

https://pasteur.hal.science/pasteur-05304225v1

Preprint submitted on 8 Oct 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Aligning Statistical Models with Inference Goals in the
Neuroscience of Language : A Dual-Dependency-Taxonomy

Sophie Bouton¹,², Valérian Chambon³, Narly Golestani⁴,⁵,⁶, Elia Formisano⁷,⁸,⁹, Timothée Proix¹⁰, Anne-Lise Giraud¹,¹⁰

1. Université Paris Cité, Institut Pasteur, AP-HP, Inserm, Fondation Pour l'Audition, Institut de l'Audition, IHU reConnect, F-75012 Paris, France

2. Laboratoire de Sciences Cognitives et Psycholinguistique, Département d'Etudes Cognitives, Ecole Normale Supérieure, EHESS, CNRS, PSL University, F-75005 Paris, France

3. Institut Jean Nicod, CNRS/École Normale Supérieure UMR 8129, PSL University, 75005 Paris, France

4. Section of Psychology, University of Geneva – Campus Biotech, 9 chemin des Mines, 1202 Geneva, Switzerland

5. Brain and Language Lab, Vienna Cognitive Science Hub, University of Vienna, Austria

6. Department of Behavioral and Cognitive Biology, Faculty of Life Sciences, University of Vienna, Austria

7. Department of Cognitive Neuroscience, Faculty of Psychology and Neuroscience, Maastricht University, Maastricht, the Netherlands

8. Maastricht Brain Imaging Center, Maastricht, the Netherlands

9. Maastricht Centre for Systems Biology, Maastricht University, Maastricht, the Netherlands

10. Department of Basic Neurosciences, University of Geneva – Campus Biotech, 9 chemin des Mines, 1202 Geneva, Switzerland

Timothée Proix and Anne-Lise Giraud contributed equally to this work.

Corresponding author:

Sophie Bouton

E-Mail address: sophie.bouton@pasteur.fr

ORCID: https://orcid.org/0000-0001-5496-4583

Keywords. data analysis, interpretation, dependency, features, language.
Abstract

The rise of large language models challenges neuroscience to reconsider how language is represented in the human brain and what aspects are uniquely human. Modeling language in the brain is complex, involving time-varying signals, structured symbolic representations, and interdependent features. Current models make trade-offs between variable independence, temporal continuity, and interpretability, shaping the inferences they support. The Dual-Dependency-Taxonomy (2D-taxonomy) addresses this by classifying models based on how they handle covariance and temporal dependencies. It distinguishes four classes—multivariable, multivariate, dynamical, and latent dynamical—each aligned with specific scientific goals. The 2D-taxonomy offers a principled framework for evaluating modeling strategies and helps clarify how different statistical choices influence what we can learn about language in the brain.
## Interpretability in Neural Modeling

Understanding how the human brain processes language is increasingly important—not only for neuroscience, but also for the development of artificial intelligence. AI systems, such as large language models, offer new perspectives on how complex language behaviors might be implemented, while insights from neuroscience could inspire more biologically grounded and energy-efficient designs. This growing convergence raises new challenges for analysing brain activity during language processing. In particular, it calls for a clearer understanding of the assumptions that underlie the statistical models we use. This article aims to review these challenges and address them through a simple two-dimension taxonomy, designed to clarify how different modeling approaches shape the kinds of inference we can draw from neural data.

Traditional approaches in the neuroscience of language (EEG, MEG, fMRI, or intracranial EEG) rely on cycles of inductive-deductive reasoning [1,2], where models are refined over time through a cycle of pattern identification in neural or neurological data and hypothesis testing in controlled experiments (deduction) (Figure 1). This approach allows observed changes in neural activity to be reliably linked to specific causal factors. For example, incidental damage to Broca's area led to hypotheses about its role in speech production [3–5], which were then tested via targeted perturbations, such as electrical stimulation [6] or focal cortical cooling [7]. These studies relied on methods that offer interpretability by design: the deliberate variation of experimental parameters enables direct attribution of neural activity to specific processes.

<figure><img src="image_3.png" /><figcaption>Figure 1. The inductive/deductive cycle illustrates the iterative logic of scientific inference: theoretical hypotheses guide experimental design (deduction), while observed patterns motivate the model refinement and the formulation of new hypotheses (induction). This cycle supports both controlled experiments and data-driven approaches.</figcaption></figure>
In this deductive framework, multivariable models have played a central role. These models, such as classical General Linear Models (GLM), assume that each predictor contributes independently to neural or linguistic signals. Their inferential strength lies in their ability to separate the contribution of distinct variables—especially when they are well-controlled and minimally correlated. These models are often applied in tightly structured paradigms, such as factorial designs, and the key assumption is that each variable can be treated as functionally independent. This logic enables precise, interpretable inference, and has led to foundational insights into how linguistic features such as phoneme identity, word frequency, syntactic complexity, or semantic ambiguity are represented across auditory and language-selective brain regions [8–11], as well as into the hierarchical organization of speech and language processing in the brain [12–15]. Although highly interpretable, this approach has been criticized for its limited ecological validity. The reliance on simplified, highly constrained stimuli fails to capture the interactive and intricate nature of real-world language use.

In response to these limitations, recent years have seen a growing shift toward inductive, data-driven approaches. Advances in statistical methods, deep neural networks (DNNs), and dimensionality reduction techniques have enabled researchers to model neural responses recorded during naturalistic language experiences—such as conversation, storytelling, or spontaneous listening [16]. These models uncover latent structure in high-dimensional data but often do so at the cost of interpretability. Because they rely on latent features or high-capacity architectures, it becomes difficult to map internal model states back to cognitive operations or brain mechanisms [17].

To navigate the growing methodological landscape in speech and language neuroscience, we propose a taxonomy that links model interpretability to the statistical dependencies it explicitly or implicitly involves. This framework is grounded in two fundamental properties of language-related brain activity: it reflects the structured co-variation of linguistic features at a given time point, and it evolves continuously over time in response to unfolding input (Figure 2). These properties give rise to two statistical dependencies—covariance and temporal—whose joint modeling is essential for capturing the richness of language processing. In the proposed taxonomy, we conceptualize them independently in order to highlight the specific modeling challenges each type of dependency introduces. This separation helps clarify how different modeling strategies align with distinct inferential goals and sets the stage for understanding the promises and limitations of their integration. Covariance and temporal dependencies structure both the linguistic input and the neural
response, making them jointly relevant for any model that seeks to map language to brain activity.

Covariance dependencies arise when multiple linguistic or neural features are statistically
correlated—either due to co-occurrence in natural speech or spatial overlap in brain activity.
For example, acoustic features like intensity and spectral content may correlate with
higher-level linguistic properties such as word frequency or semantic predictability. Likewise,
neural signals recorded from adjacent voxels or electrodes often exhibit spatial covariance.
Ignoring these dependencies can lead to misattributed effects. Multivariable models
minimize such risks by requiring that predictors be independent—either by design or
through careful control—while multivariate models are designed to capture and exploit
covariance structures, albeit at the cost of interpretability.

Temporal dependencies refer to the influence of past states—linguistic or neural—on
current responses. For instance, when listening to speech, instantaneous neural activity
reflects not only the current acoustic input but also preceding phonemes, words, or
syntactic structures [18,19]. Similarly, ongoing neural states often exhibit temporal
autocorrelation, reflecting predictive coding, working memory, or feedback processes [20].
Dynamical models explicitly capture this temporal structure using observable variables,
while latent dynamical models infer unobserved, time-evolving latent states to account for
complex neural trajectories.

<figure><img src="image_4.png" /><figcaption>Figure 2. Illustration of the two core statistical dependencies in speech and language processing: covariance dependencies (red curved arrows) reflect correlations between co-occurring features—such as acoustic properties linked to lexical or semantic content, or spatial correlation in neural signals; temporal dependencies (blue loops), reflect how past states—phonemes, words, or neural activity—shape present responses through predictive and memory-related mechanisms. Linguistic and neural data are represented as vertically stacked variables (black dots) sampled over</figcaption></figure>
time (transparent layers), with mapping arrows representing statistical models linking stimulus and response.

The Dual-Dependency-Taxonomy (2D-taxonomy) organizes modeling approaches along two orthogonal axes: covariance dependencies and temporal dependencies. Each quadrant of the 2D-taxonomy corresponds to a class of statistical models defined by how it addresses these two dependencies (Figure 3):

* **Multivariable models** minimize covariance dependencies by assuming that each variable contributes independently to the modeled signal. This class includes classical linear models such as GLMs, ANOVAs or voxel-wise encoding models, which are often used with orthogonalized variables or low-dimensional design matrices. They are preferred for hypothesis-driven, deductive inference as they can be relatively unambiguously interpreted.

* **Multivariate models** exploit covariance among variables to uncover interactions, co-activations, or distributed representations—typical of naturalistic language data. These include Representational Similarity Analysis (RSA), Principal Component Analysis (PCA), and (static) feed-forward DNNs. Their strength lies in pattern discovery, but they often obscure the unique contribution of individual variables, and are therefore best aligned with inductive reasoning and exploratory inference.

* **Dynamical models** account for temporal dependencies by explicitly modeling how neural activity evolves over time. This includes autoregressive (AR) or vector autoregressive (VAR) models, as well as other time-series methods based on observable variables. They enable analysis of temporally structured processes, though they require a shift in interpretive logic. If hypotheses are framed in terms of dynamical properties, such as variations, transitions or trajectories over time, rather than static contrasts, they remain relatively well interpretable.

* **Latent dynamical models** integrate both covariance and temporal dependencies through latent variables that summarize high-dimensional, correlated, and time-varying data. This class includes hidden Markov models (HMMs), state-space models, and deep recurrent architectures trained to model brain responses to structured language input. They are powerful for uncovering latent structure, capturing hidden dynamics that may not be directly observable, but often at the cost of interpretability. As such, they are more appropriate for exploration.

The 2D-taxonomy thus provides a structural framework to guide the selection of statistical models based on the inferential goals of a study. However, the taxonomy outlines four
idealized modeling logics, whereas many real-world approaches combine elements across quadrants—integrating, for instance, interpretable predictors within multivariate or dynamical frameworks. Rather than enforcing rigid boundaries, the 2D-taxonomy is intended as a conceptual space for articulating the assumptions and affordances of each modeling strategy.

<figure><img src="image_5.png" /><figcaption>Figure 3. The 2D-taxonomy organizes modeling approaches along two independent axes: covariance (horizontal) and temporal (vertical) dependencies, defining four model classes: multivariable models (e.g., regression, ANOVAs) assume independent predictors and support highly interpretable hypothesis-driven inference; multivariate models (e.g., representational similarity analysis, principal component analysis) capture covariance across variables but omit temporal structure; dynamical models (e.g., autoregressive models, Granger Causality) model temporal evolution using observable variables; latent dynamical models (e.g., hidden Markov models, state-space models) infer both covariance and temporal dependencies via latent states.</figcaption></figure>

## Multivariable Models: Specifying Functional Contributions through Independent Variables

Multivariable models have been foundational tools for hypothesis-driven research in speech and language neuroscience. Their central strength lies in their capacity to disentangle the contributions of distinct linguistic variables by assuming that each predictor contributes independently to the observed neural response. This logic enables interpretability: neural effects are directly attributed to specific linguistic properties, as long as those properties are
sufficiently controlled and minimally correlated. Although multivariable models are often employed within tightly structured experimental paradigms—such as factorial designs—they can also be extended to naturalistic data, as long as predictor interdependence is kept in check. Their defining characteristic within the 2D-taxonomy is statistical: feature covariance dependencies are minimized by experimental design and the models treat each dimension as functionally independent.

This approach has yielded some of the most robust insights into the cortical organization of speech and language. Studies that manipulate low-level acoustic features—such as spectral complexity or modulation rate—have shown consistent activation in early auditory areas, particularly in Heschl's gyrus [21–23]. When experiments target higher-order linguistic features—such as lexical frequency, semantic ambiguity, or syntactic embedding—activation shifts to more anterior and inferior regions, including the middle temporal gyrus and inferior frontal cortex [24,25]. These findings have supported for instance the view of a functional posterior-to-anterior and dorsal-to-ventral organization, associated with increasing linguistic abstraction across cortical regions [26].

Multivariable models have also been instrumental in demonstrating functional dissociations across the dual-stream model of language processing. For instance, when comprehension and production are modeled separately, distinct activation patterns emerge: a dorsal stream involving posterior superior temporal and premotor areas for articulatory and phonological encoding, and a ventral stream involving the anterior temporal lobe and inferior frontal gyrus for semantic interpretation [27–29].

The same logic has been applied at the level of low-level auditory encoding, particularly through the use of spectro-temporal receptive fields (STRFs). STRFs represent the estimated mapping between time-frequency structure in the auditory signal and the corresponding neural responses, typically derived using linear regression models with time-lagged acoustic predictors. Applied to continuous stimuli, STRFs assume no internal dynamics or feature interactions; instead, they assess whether each acoustic component contributes independently to the neural response. This makes them a paradigmatic example of multivariable modeling, where covariance between predictors is not modeled but treated as a potential confound. Although real-world stimuli exhibit correlated spectro-temporal features, STRFs do not capture those dependencies. Yet despite this constraint, STRF studies have revealed systematic progression in temporal integration—from neurons in primary auditory areas responding to transient features, to higher-order regions integrating over longer time windows to align with syllabic, lexical, or even phrasal structures [30–33].
Across these dimensions of language processing—hierarchy of abstraction, stream dissociation, and graded expansion of processing windows—multivariable models provide a rigorous, interpretable framework for linking linguistic features to neural responses. Their inferential clarity stems from what they exclude: by imposing minimal covariance by design and rejecting temporal dependencies, they enable strong attribution of function[34]. But this clarity comes at a cost. In naturalistic language, phonetic, syntactic, semantic, and prosodic dimensions rarely vary independently. Coarticulation blurs phoneme boundaries; prosody modulates syntax; and meaning emerges from distributed context. On the neural side, spatial covariance is equally pervasive: adjacent voxels or electrodes often exhibit shared activity patterns, reflecting both local processing and network-level organization. Multivariable models thus underfit the complexity of language processing. In the 2D-taxonomy, they occupy the quadrant of low covariance, low temporal dependency, and high interpretability. They are suitable tools for confirmatory inference under tightly controlled experimental and analysis conditions.

## Multivariate Models: Uncovering Distributed Representations through Covariance Structure

Multivariate models have become essential tools to understand how speech and language operate in natural conditions despite the many redundant and interacting features allowing language to be processed and deployed on the fly. The core strength of these models lies in their ability to capture shared variance across linguistic features and neural dimensions, enabling the analysis of how multiple, potentially interacting representations are encoded simultaneously in distributed brain activity [35]. This shift in logic—from isolating variables to embracing their covariation—has opened the door to a more holistic view of neural encoding, in which the unit of inference is no longer a single predictor but the structure of relationships across many [36].

A key contribution of multivariate models has been to reveal how linguistic representations are shared across overlapping neural populations. Techniques such as (static) feedforward DNNs, Representational Similarity Analysis (RSA), and multivoxel encoding models have shown that features once considered independently—such as acoustic pattern, phonetic structure, word identity, and syntactic complexity—are reflected simultaneously within shared cortical substrates [37–40]. Notably, these include regions such as the superior temporal sulcus, middle temporal gyrus, and inferior frontal cortex [41–43]. These findings suggest that the brain does not segregate linguistic features into discrete, modular
pathways, but instead encodes them through intersecting, distributed patterns of activation
[44–47]. Multivariate models make this convergence visible by exploiting the covariance
structure of naturalistic language and neural activity, rather than treating such dependencies
as noise [48]. By modeling co-activations across features and across space, multivariate
models bridge multiple levels of linguistic representation within a single analytic framework,
providing a more ecologically valid account of real-world language processing [49–51].

This strength, however, inevitably comes with a cost. Because multivariate models share
variance across predictors, they do not isolate the unique contribution of each variable. This
results in a difficulty to determine whether observed effects reflect specific linguistic features
or their interactions—reducing causal specificity and complicating interpretation [46]. This
limitation becomes particularly salient in studies using representative spaces—such as PCA,
factor analysis or deep neural network embeddings—to uncover both the semantic structure
in linguistic data and their representation in neural semantic spaces. These methods have
revealed continuous, low-dimensional gradients of semantic similarity across
temporo-parietal and anterior temporal regions [52,53]. Yet the dimensions extracted often
lack clear cognitive interpretability and may be shaped more by statistical optimization than
by representational logic. The same challenge extends to neural modeling: dimensionality
reduction of brain responses—whether through PCA, manifold learning, or DNN-based
mappings—yields low-dimensional neural representations that are difficult to map precisely
onto explicit cognitive constructs [54]. Unlike multivariable models, which reveal direct
associations between controlled predictors and localized brain responses, multivariate
approaches identify emergent structures whose interpretation require independent
validations to ground their interpretations [55–57]. This challenge has been partially
addressed in studies that reintroduce experimental control into multivariate frameworks—for
example, by manipulating the word order in spoken stories to test whether neural responses
to changes in word meaning depend on whether the listener understands the overall
narrative [58].

Addressing representational convergence and semantic space modeling, multivariate
models have reshaped the questions that can be asked in speech and language
neuroscience. They shift the inferential focus from variable attribution to representational
mapping—capturing how linguistic features interact and co-occur, and how these
relationships are reflected in distributed neural responses. In the 2D-taxonomy, multivariate
models occupy the high-covariance, low temporal-dependency quadrant. Their value lies in
modeling the richness of real-world language, where complexity is not noise to be removed,
but structure to be revealed, at the cost of precise feature/function mapping.

Dynamical Models: Modeling Recurrent Processes through Temporal
Dependencies

Speech and language unfold continuously over time, engaging multiple parallel processing
loops where past events condition neural activity. Unlike multivariable and multivariate
approaches, dynamical models take temporal dependencies into account by being explicitly
recurrent: they model how present states emerge from the history of prior states. Their
defining characteristic within the 2D-taxonomy does not lie in the number of variables or
dimensionality, but in their ability to model inherent temporal structure: the sequential
dependencies that govern stimulus processing, neural dynamics, and the continuous
transformation of information over time. One recent extension of this logic involves the use
of dynamic temporal response functions, in which canonical response profiles are adjusted
in latency and amplitude based on the predictability of each word and its context, allowing
for finer-grained modeling of EEG responses to naturalistic speech [59]

A foundational contribution of dynamical modeling is to provide tools for estimating
directed dependencies across time. Autoregressive (AR) models predict current neural
responses as a function of their own past values. In multivariate settings, vector
autoregressive (VAR) models extend this principle by characterizing how temporal
interactions are established across multiple variables simultaneously. That is, each signal is
modeled as a function of its own past and the past of other signals, allowing for the explicit
modeling of cross-regional dynamics over time. These approaches have uncovered
structured temporal interactions linking sensory and association cortices during speech
perception [60,61]. Granger causality builds on this framework to test whether including the
past of one signal improves the prediction of another—enabling inference of directed
influence among observable neural sources. For instance, in natural sentence processing,
slow oscillations from association auditory regions were found to control gamma-band
activity in primary auditory areas, reflecting contextual influence on early acoustic encoding
[62]. Similarly, beta-band activity in frontal cortices was found to modulate gamma
responses in the superior temporal gyrus during phoneme categorization [63,64],
supporting theories of top-down predictive feedback in speech analysis [65].

Such findings are typically derived from controlled experimental paradigms with limited
variable sets. In high-dimensional or naturalistic contexts, standard VAR-based Granger
models face major challenges, including overfitting and lack of interpretability. To address these limitations, sparse VAR (SVAR) models introduce regularization constraints that retain only the most informative dependencies [66,67]. Further refinements, such as conditional Granger causality (CGC) controls for common inputs and indirect effects, disambiguating direct from mediated influences across distributed regions [68,69]. These methods open powerful avenues for dissecting how temporal coordination among brain regions shapes speech and language comprehension or production.

While autoregressive models (VAR, SVAR) capture how neural responses evolve with time, they do not account for the structure of the input itself. Speech is acoustically continuous, but the linguistic system it encodes is composed of discrete symbolic units—phonemes, morphemes, words—organized by rules that govern how elements follow, constrain, and transform one another. Bridging this representational gap requires models that infer symbolic regularities from continuous input. Models such as Markov chains and Recurrent Neural Networks are particularly suited to this task: they formalize the sequential structure of symbolic systems by modeling transition probabilities between discrete units. Markov models have been used to stimulate phoneme and word transitions in natural speech, predicting how local conditional probabilities modulate neural responses—consistent with probabilistic inference and predictive coding theories [70,71]. Extending beyond short-range transitions, RNNs trained on large corpora capture hierarchical linguistic dependencies, such as syntactic embeddings and long-range semantic relationships, maintaining dynamic contextual representations across time [72]. By implementing recurrence as an explicit computational mechanism, these models emulate how the brain accumulates and updates contextual information—offering a dynamic framework for understanding how meaning unfolds over time.

Dynamical models therefore shift the focus of inference from structure to process: rather than identifying where linguistic features are encoded, they ask how such encoding unfolds through time. Central to this shift is the notion of recurrence—not only as a mathematical property, but as a functional principle whereby past neural or linguistic states shape present ones, via feedback processes. These models formalize such dependencies explicitly, offering a direct account of temporal sequencing, rhythm, and predictive feedback. Their interpretability stems from their observability: no latent states are inferred, and the model's variables correspond to experimentally defined inputs and outputs. Within the 2D-taxonomy, they occupy the quadrant of low-covariance and high-temporal dependency.
Dynamical models are best suited for studying domains where temporality is not a confound
to be removed, but the very mechanism to be explained.

Latent Dynamical Models: Inferring Evolving Representations through
Hidden States

Latent dynamical models embody the most abstract, least interpretable, quadrant of the 2D-taxonomy. They extend dynamical models by introducing latent variables—unobserved/hidden internal states—that evolve over time and generate the observed neural data. In doing so, they aim to reconstruct the evolving internal representations that shape language perception and comprehension across time. Rather than simply describing statistical dependencies among observed features, these models infer the structure and geometry of the internal space where representations unfold. The central underlying assumption is that high-dimensional linguistic and neural signals reflect the projection of a lower-dimensional, temporally structured latent process [73]. In this framework, linguistic and neural dynamics are interpreted not as sequences of observable states, but as trajectories through a latent space shaped by both internal priors and external input. While these models offer an interesting way to capture complex temporal regularities, the inferred latent representations are often difficult to interpret directly, and require strong theoretical and empirical constraints to support meaningful scientific inference. Methods such as Hidden Markov Models (HMMs), State-Space Models, and Gaussian Process Latent Variable Models formalize this intuition, providing tools for inferring and interpreting latent trajectories from complex temporal data.

Latent dynamical models have made critical contributions by revealing the hidden temporal
architecture best explaining how speech and language representations are sequentially
deployed. Latent models have been used to recover internal state sequences aligned with
phonemic or lexical units [74], to estimate shifts in attentional state during multi-speaker
processing [75], and to identify continuous latent trajectories reflecting hierarchical linguistic
structure [71]. Importantly, these models have revealed that speech-evoked neural activity
reflects nested temporal scales, with fast-changing dynamics capturing phoneme-level
information and slower trajectories encoding lexical or conceptual integration [76]. In
regions such as the anterior superior temporal gyrus, latent trajectories jointly capture
bottom-up and top-down influences, reflecting the integration of acoustic and semantic
features in real time [77]. They have also revealed wave-like propagations across the
auditory cortex preceding word onsets, consistent with mechanisms of anticipatory tuning
and predictive pre-activation [78–81]. Such latent dynamics not only scaffold anticipatory processing, but also give rise to the categorical units of language themselves. Phonemes and words emerge as transient condensations within a continuously evolving representational space, reflecting the brain’s ability to carve discrete percepts out of low-dimensional, dynamic neural activity [51,82].

Yet, since latent spaces are not directly observed, it is difficult to determine whether the resulting dimensions correspond to meaningful neural computations or to artifacts of the model’s architecture. The structure of the latent space—its dimensionality, smoothness, and geometry—depends on model assumptions, and can bias interpretation [83]. Too sparse models may collapse distinct processes into a single trajectory, whereas larger ones may overfit noise or fail to generalize. These concerns are particularly relevant when latent trajectories are used to infer specific cognitive operations, such as syntactic parsing, prediction, or attentional control [84]. Emerging techniques such as variational inference and deep latent models open the possibility of recovering richer, nonlinear latent structures—including manifold-based trajectories reflecting the evolution of representation over time [85–89]. Realizing this potential will require integrating these techniques with biologically grounded constraints, ensuring that latent spaces reflect meaningful neural computations rather than abstract statistical structures.

Latent dynamical models are thus best viewed as tools for hypothesis generation and representational inference. Within the 2D-taxonomy, they occupy the quadrant of high-covariance and high-temporal-dependency. They are most appropriate when the research goal is to uncover the internal, time-evolving structure of neural computation, rather than localizing specific functions or attributing effects to explicitly defined features.

## Concluding Remarks

The evolving challenges of language neuroscience call for a deeper understanding of dynamic neural mechanisms and of the dependencies that shape both the data and the models used to interpret them. The Dual-Dependency Taxonomy (2D-taxonomy) systematizes the investigation model choice given inferential goals and data structure. By organizing modeling approaches along two independent dimensions—instantaneous feature covariance and temporal dependency—it identifies four cardinal modeling logics. More than a classification tool, the 2D-taxonomy helps clarify how different model types foreground distinct aspects of neural language processing—from functional specificity and representational overlap to temporal prediction and latent structure. As the field
increasingly relies on large-scale datasets and high-capacity models, the tension between
predictability and interpretability becomes a central concern. Researchers everywhere are
grappling with the growing divide between algorithm discovery and human understanding.
Two decades ago, science-fiction writer Ted Chiang imagined a world in which genetically
enhanced "metahuman" scientists produced knowledge too complex for unmodified
humans to comprehend [90]. This vision seems increasingly prescient in the age of AI:
today's model generates remarkably fine-grained predictions, but their internal logic often
remains opaque, even to experts. Yet Chiang's vision also carries a hopeful inversion in
which the scientist's role remains intact—not to outpace the machine, but to ask the right
questions, define the space of relevance, and frame the purpose of knowledge itself.
Beyond its purpose as a decisional guide, the 2D-taxonomy is intended as a tool for
reflection. It supports a sustained engagement with the inductive-deductive cycle that
underpins scientific reasoning, and promotes clarity of inference in the context of
increasingly complex, model-driven approaches to language neuroscience.
<table><caption>Table 1.</caption><thead><tr><td>Model Category</td><td>Key Characteristics</td><td>Mathematical Approach</td><td>Strengths</td><td>Limitations</td><td>Examples of Use in Speech & Language Neuroscience</td></tr></thead><tbody><tr><td>Multivariable Models</td><td>Models effects of individual predictors assumed to act independently</td><td>General Linear Models (GLMs), regression with orthogonalized predictors, univariate contrasts, multivariable encoding/decoding models</td><td>High interpretability, strong attribution of neural responses to specific linguistic features; ideal for hypothesis testing</td><td>Assumes minimal covariance; limited ecological validity; underfits complexity in naturalistic language</td><td>Hierarchical encoding (e.g., Heschl's gyrus to IFG); dual-stream dissociations; STRFs in controlled settings</td></tr><tr><td>Multivariable Models</td><td>Models shared variance between predictors; captures distributed, overlapping representations across brain regions</td><td>Multiple regression with correlated predictors, Partial Least Squares (PLS), Representational Similarity Analysis (RSA), Principal Component Analysis (PCA), multivariable encoding/decoding models</td><td>Sensitive to co-activation and interaction effects; suited for high-dimensional, naturalistic data; reveals representational structure</td><td>Limited causal attribution; emergent dimensions may lack interpretability; dependent on structure of predictor space</td><td>Phonetic-semantic convergence in STG and STS; nested representations of linguistic levels; semantic gradients across cortex</td></tr><tr><td>Dynamical Models</td><td>Models temporal dependencies explicitly; tracks how neural responses evolve as a function of past inputs or activity</td><td>Autoregressive (AR) models, Granger Causality, Vector Autoregressive (VAR), Sparse VAR, Conditional Granger Causality, Markov models</td><td>Captures sequential processing and feedback; transparent structure; well-suited for testing predictive coding hypotheses</td><td>Limited spatial dimensionality; does not infer latent states; can be sensitive to noise and overfitting in multivariate settings</td><td>Temporal coordination in speech perception; top-down modulation; phoneme prediction from past context</td></tr><tr><td>Latent Dynamical Models</td><td>Infers unobserved internal states that evolve over time; reconstructs representational dynamics in latent space</td><td>Hidden Markov Models (HMMs), State-Space Models, Gaussian Process Latent Variable Models, Variational Autoencoders, deep latent dynamical models</td><td>Captures nested timescales and abstract structure; reveals hidden trajectories; suited for exploratory modeling and hypothesis generation</td><td>Lower interpretability; outcomes depend on model assumptions; risk of overfitting or conflating latent dimensions</td><td>Phoneme-to-word transitions as continuous trajectories; anticipatory wave propagation; nested temporal trajectories</td></tr></tbody></table>
# Acknowledgements

We gratefully acknowledge support by the Fyssen Foundation (PI: S.B.), by the Fondation pour l'Audition (RD-2016-R to S.B. and FPA IDA11 to A.L.G), by Agence Nationale de la Recherche (ANR-21-CE28-0028 to S.B., ANR-19-CE37-0014-01, ANR-21-CE37-0020-02, ANR-17-EURE-0017 and ANR-10-IDEX-0001-02 PSL to V.C.), by Swiss National Science Foundation (career grant 193542 to T.P.; and Grant # 100014_182381 to N.G.), by Evolving Language, Swiss National Science Foundation Agreement #51NF40_180888 (to A-L. G. and N.G.). This work has also benefited from a French government grant managed by the Agence Nationale de la Recherche under the France 2030 program, reference ANR-23-IAHU-0003.

# Conflicts of Interest

The authors declare no Conflicts of Interest

# References

1. Popper, K.R. (1935) *The Logic of Scientific Discovery*, Routledge

2. Sun, R. (2023) *The Cambridge Handbook of Computational Cognitive Sciences*, Cambridge University Press

3. Broca, P. (1861) Remarques sur le siège de la faculté du langage articulé, suivies d'une observation d'aphémie (perte de la parole). *Bull Mem Soc Anthropol Paris* 6, 0-357

4. Flinker, A. et al. (2015) Redefining the role of Broca's area in speech. *Proceedings of the National Academy of Sciences* 112, 2871-2875

5. Pracar, A.L. et al. (2025) The neuroanatomy of Broca's aphasia. *Front. Lang. Sci.* 4, 1496209

6. Lu, J. et al. (2021) Functional maps of direct electrical stimulation-induced speech arrest and anomia: a multicentre retrospective study. *Brain* 144, 2541-2553

7. Long, M.A. et al. (2016) Functional Segregation of Cortical Regions Underlying Speech Timing and Articulation. *Neuron* 89, 1187-1193

8. Miceli, G. et al. (2002) The Neural Correlates of Grammatical Gender: An fMRI Investigation. *Journal of Cognitive Neuroscience* 14, 618-628

9. Keller, T.A. (2001) The Neural Bases of Sentence Comprehension: a fMRI Examination of Syntactic and Lexical Processing. *Cerebral Cortex* 11, 223-237

10. Giraud, A.-L. et al. (2007) Endogenous cortical rhythms determine cerebral specialization for speech perception and production. *Neuron* 56, 1127-34

11. Overath, T. and Paik, J.H. (2021) From acoustic to linguistic analysis of temporal speech structure: Acousto-linguistic transformation during speech perception using speech quilts. *NeuroImage* 235, 117887

12. Okada, K. et al. (2010) Hierarchical Organization of Human Auditory Cortex: Evidence from Acoustic Invariance in the Response to Intelligible Speech. *Cerebral Cortex* 20, 2486-2495

13. Humphries, C. et al. (2014) Hierarchical organization of speech perception in human auditory cortex. *Frontiers in Neuroscience* 8, 1-12

14. Snijders, T.M. et al. (2009) Retrieval and Unification of Syntactic Structure in Sentence Comprehension: an fMRI Study Using Word-Category Ambiguity. *Cerebral Cortex* 19, 1493-1503

15. Makuuchi, M. et al. (2009) Segregating the core computational faculty of human language from working memory. *Proc. Natl. Acad. Sci. U.S.A.* 106, 8362-8367

16. Hamilton, L.S. and Huth, A.G. (2018) The revolution will not be controlled: natural stimuli in speech neuroscience. *Language, Cognition and Neuroscience* 35, 573-582

17. Whiteway, M.R. and Butts, D.A. (2019) The quest for interpretable models of neural population activity. *Current Opinion in Neurobiology* 58, 86-93

18. Heilbron, M. et al. (2022) A hierarchy of linguistic predictions during natural language comprehension. *Proc. Natl. Acad. Sci. U.S.A.* 119, e2201968119
19. Hovsepyan, S. et al. (2020) Combining predictive coding and neural oscillations enables online syllable recognition in natural speech. *Nature Communications* 11, 1–12

20. Gow, D.W. and Caplan, D.N. (2012) New Levels of Language Processing Complexity and Organization Revealed by Granger Causation. *Front. Psychology* 3

21. Giroud, J. et al. (2020) Asymmetric sampling in human auditory cortex reveals spectral processing hierarchy. *PLoS Biol* 18, e3000207

22. Giroud, J. et al. (2024) The human auditory cortex concurrently tracks syllabic and phonemic timescales via acoustic spectral flux. *Sci. Adv.* 10, eado8915

23. Overath, T. et al. (2012) Sensitivity to temporal modulation rate and spectral bandwidth in the human auditory system: fMRI evidence. *Journal of Neurophysiology* 107, 2042–2056

24. Friederici, A.D. et al. (2000) Segregating Semantic and Syntactic Aspects of Processing in the Human Brain: an fMRI Investigation of Different Word Types. *Cerebral Cortex* 10, 698–705

25. Hauk, O. et al. (2006) The time course of visual word recognition as revealed by linear regression analysis of ERP data. *NeuroImage* 30, 1383–1400

26. Hagoort, P. and Indefrey, P. (2014) The Neurobiology of Language Beyond Single Words. *Annu. Rev. Neurosci.* 37, 347–362

27. Saur, D. et al. (2008) Ventral and dorsal pathways for language. *Proc. Natl. Acad. Sci. U.S.A.* 105, 18035–18040

28. Visser, M. et al. (2012) Both the Middle Temporal Gyrus and the Ventral Anterior Temporal Area Are Crucial for Multimodal Semantic Processing: Distortion-corrected fMRI Evidence for a Double Gradient of Information Convergence in the Temporal Lobes. *Journal of Cognitive Neuroscience* 24, 1766–1778

29. Poldrack, R.A. et al. (1999) Functional Specialization for Semantic and Phonological Processing in the Left Inferior Prefrontal Cortex. *NeuroImage* 10, 15–35

30. Bai, F. et al. (2022) Neural dynamics differentially encode phrases and sentences during spoken language comprehension. *PLoS Biol* 20, e3001713

31. Barchet, A.V. et al. (2025) Attentional engagement with target and distractor streams predicts speech comprehension in multitalker environments. *bioRxiv* DOI: https://doi.org/10.1101/2025.04.04.647157

32. Holdgraf, C.R. et al. (2016) Rapid tuning shifts in human auditory cortex enhance speech intelligibility. *Nat Commun* 7, 13654

33. Venezia, J.H. et al. (2019) Hierarchy of speech-driven spectrotemporal receptive fields in human auditory cortex. *NeuroImage* 186, 647–666

34. Kraus, F. et al. (2024) Neurophysiology of Effortful Listening: Decoupling Motivational Modulation from Task Demands. *J. Neurosci.* 44, e0589242024

35. Mitchell, T.M. et al. (2008) Predicting Human Brain Activity Associated with the Meanings of Nouns. *Science* 320, 1191:1195

36. Kriegeskorte, N. and Kievit, R.A. (2013) Representational geometry: Integrating cognition, computation, and the brain. *Trends in Cognitive Sciences* 17, 401–412

37. Carota, F. et al. (2021) Distinct fronto-temporal substrates of distributional and taxonomic similarity among words: evidence from RSA of BOLD signals. *NeuroImage* 224, 117408

38. Cope, T.E. et al. (2023) Temporal lobe perceptual predictions for speech are instantiated in motor cortex and reconciled by inferior frontal cortex. *Cell Reports* 42, 112422

39. De Angelis, V. et al. (2018) Cortical processing of pitch: Model-based encoding and decoding of auditory fMRI responses to real-life sounds. *NeuroImage* 180, 291–300

40. Zheng, Y. et al. (2025) Neural representation of sensorimotor features in language-motor areas during auditory and visual perception. *Commun Biol* 8, 41

41. Chalas, N. et al. (2022) Multivariate analysis of speech envelope tracking reveals coupling beyond auditory cortex. *NeuroImage* 258, 119395

42. Daube, C. et al. (2019) Simple Acoustic Features Can Explain Phoneme-Based Predictions of Cortical Responses to Speech. *Current Biology* 29, 1924–1937.e9

43. Caucheteux, C. and King, J.-R. (2022) Brains and algorithms partially converge in natural language processing. *Commun Biol* 5, 134

44. Degano, G. et al. (2024) Speech prosody enhances the neural processing of syntax. *Commun Biol* 7, 748

45. Gwilliams, L. et al. (2022) Neural dynamics of phoneme sequences reveal position-invariant code for content and order. *Nat Commun* 13, 6606

46. Huth, A.G. et al. (2016) Natural speech reveals the semantic maps that tile human cerebral
cortex. *Nature* 532, 453–458

47. Keshishian, M. et al. (2023) Joint, distributed and hierarchically organized encoding of linguistic features in the human auditory cortex. *Nat Hum Behav* DOI: 10.1038/s41562-023-01520-0

48. Sohoglu, E. et al. (2024) Convergent neural signatures of speech prediction error are a biological marker for spoken word recognition. *Nat Commun* 15, 9984

49. de Heer, W.A. et al. (2017) The Hierarchical Cortical Organization of Human Speech Processing. *J. Neurosci.* 37, 6539–6557

50. Rutten, S. et al. (2019) Cortical encoding of speech enhances task-relevant acoustic information. *Nature Human Behaviour* 3, 974–987

51. Orepic, P. et al. (2023) Neural manifolds carry reactivation of phonetic representations during semantic processing

52. Caucheteux, C. et al. (2022) Deep language algorithms predict semantic comprehension from brain activity. *Sci Rep* 12, 16327

53. Zhang, Y. et al. (2020) Connecting concepts in the brain by mapping cortical representations of semantic relations. *Nat Commun* 11, 1877

54. Guest, O. and Love, B.C. (2017) What the success of brain imaging implies about the neural code. *eLife* 6, e21397

55. Brendel, W. et al. (2011) Demixed Principal Component Analysis. *Neural Information Processing Systems* 24, 2654–2662

56. Halai, A.D. et al. (2020) Investigating the effect of changing parameters when building prediction models for post-stroke aphasia. *Nat Hum Behav* 4, 725–735

57. Naselaris, T. and Kay, K.N. (2015) Resolving Ambiguities of MVPA Using Explicit Models of Representation. *Trends in Cognitive Sciences* 19, 551–554

58. Broderick, M.P. et al. (2022) More than words: Neurophysiological correlates of semantic dissimilarity depend on comprehension of the speech narrative. *Eur J of Neuroscience* 56, 5201–5214

59. Dou, J. et al. (2025) Dynamic modeling of EEG responses to natural speech reveals earlier processing of predictable words. *PLoS Comput Biol* 21, e1013006

60. Bouton, S. et al. (2020) Audio-visual combination of syllables involves time-sensitive dynamics following from fusion failure. *Scientific Reports* 10, 18009

61. Olasagasti, I. et al. (2015) Prediction across sensory modalities: A neurocomputational model of the McGurk effect. *Cortex* 68, 61–75

62. Fontolan, L. et al. (2014) The contribution of frequency-specific activity to hierarchical information processing in the human auditory cortex. *Nature Communications* 5, 4694

63. Bouton, S. et al. (2018) Focal versus distributed temporal cortex activity for speech sound category assignment. *PNAS* 115, E1299–E1308

64. Gow, D.W. et al. (2008) Lexical influences on speech perception: a Granger causality analysis of MEG and EEG source estimates. *NeuroImage* 43, 614–23

65. van Wassenhove, V. (2013) Speech through ears and eyes: Interfacing the senses with the supramodal brain. *Frontiers in Psychology* 4, 1–17

66. Krampe, J. and Paparoditis, E. (2021) Sparsity concepts and estimation procedures for high-dimensional vector autoregressive models. *Journal Time Series Analysis* 42, 554–579

67. Mody, S.K. and Rangarajan, G. (2022) Sparse representations of high dimensional neural data. *Sci Rep* 12, 7295

68. Zhou, Z. et al. (2009) Analyzing brain networks with PCA and conditional Granger causality. *Human Brain Mapping* 30, 2197–2206

69. Chen, Y. et al. (2006) Frequency decomposition of conditional Granger causality and application to multivariate neural field potential data. *Journal of Neuroscience Methods* 150, 228–237

70. Gales, M. and Young, S. (2007) The Application of Hidden Markov Models in Speech Recognition. *FNT in Signal Processing* 1, 195–304

71. Moses, D.A. et al. (2016) Neural speech recognition: continuous phoneme decoding using spatiotemporal representations of human cortical activity. *Journal of Neural Engineering* 13, 056004

72. Lakretz, Y. et al. (2021) Mechanisms for handling nested dependencies in neural-network language models and humans. *Cognition* 213, 104699

73. Miller, P. (2016) Itinerancy between attractor states in neural systems. *Current Opinion in Neurobiology* 40, 14–22

74. Li, Y. et al. (2023) Dissecting neural computations in the human auditory pathway using deep
neural networks for speech. *Nat Neurosci* 26, 2213–2225

75. Akram, S. et al. (2016) Robust decoding of selective auditory attention from MEG in a competing-speaker environment via state-space modeling. *NeuroImage* 124, 906–917

76. Donhauser, P.W. and Baillet, S. (2020) Two Distinct Neural Timescales for Predictive Speech Processing. *Neuron* 105, 385-393.e9

77. Hamilton, L.S. et al. (2018) A Spatial Map of Onset and Sustained Responses to Speech in the Human Superior Temporal Gyrus. *Current Biology* 28, 1860-1871.e4

78. Muller, L. et al. (2018) Cortical travelling waves: mechanisms and computational principles. *Nat Rev Neurosci* 19, 255–268

79. Friston, K.J. (2019) Waves of prediction. *PLoS Biol* 17, e3000426

80. Pang (庞兆阳), Z. et al. (2020) Turning the Stimulus On and Off Changes the Direction of α Traveling Waves. *Neuro* 7, ENEURO.0218-20.2020

81. Stephen, E.P. et al. (2023) Latent neural dynamics encode temporal context in speech. *Hearing Research* 437, 108838

82. Medrano, J. et al. (2024) Linking fast and slow: The case for generative models. *Network Neuroscience* 8, 24–43

83. Humphries, M.D. (2021) Strong and weak principles of neural dimension reduction. *Neurons, Behavior, Data analysis, and Theory* 5

84. Tavoni, G. et al. (2022) Human inference reflects a normative balance of complexity and accuracy. *Nat Hum Behav* 6, 1153–1168

85. Aghagolzadeh, M. and Truccolo, W. (2014) Latent state-space models for neural decoding. in *2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society*, Chicago, IL, pp. 3033–3036

86. Klindt, D. et al. (2021) TOWARDS NONLINEAR DISENTANGLEMENT IN NATURAL DATA WITH TEMPORAL SPARSE CODING

87. Sani, O.G. et al. (2021) Modeling behaviorally relevant neural dynamics enabled by preferential subspace identification. *Nat Neurosci* 24, 140–149

88. Schneider, S. et al. (2023) Learnable latent embeddings for joint behavioural and neural analysis. *Nature* 617, 360–368

89. Zhou, D. and Wei, X.-X. (2020) Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE. *34th Conference on Neural Information Processing Systems (NeurIPS)*

90. Chiang, T. (2000) Catching crumbs from the table. *Nature* 405, 517