Title: BioOSS: A Bio-Inspired Oscillatory State System with Spatio-Temporal Dynamics
Venue: NeurIPS 2025
Accepted: yes
Keywords: oscillatory state system

spatio-temporal dynamics

bio-inspired model

time series tasks
Abstract: Today’s deep learning architectures are primarily based on perceptron models, which do not capture the oscillatory dynamics characteristic of biological neurons. Although oscillatory systems have recently gained attention for their closer resemblance to neural behavior, they still fall short of modeling the intricate spatio-temporal interactions observed in natural neural circuits. In this paper, we propose a bio-inspired oscillatory state system (BioOSS) designed to emulate the wave-like propagation dynamics critical to neural processing, particularly in the prefrontal cortex (PFC), where complex activity patterns emerge. BioOSS comprises two interacting populations of neurons: p neurons, which represent simplified membrane-potential-like units inspired by pyramidal cells in cortical columns, and o neurons, which govern propagation velocities and modulate the lateral spread of activity. Through local interactions, these neurons produce wave-like propagation patterns. The model incorporates trainable parameters for damping and propagation speed, enabling flexible adaptation to task-specific spatio-temporal structures. We evaluate BioOSS on both synthetic and real-world tasks, demonstrating superior performance and enhanced interpretability compared to alternative architectures.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors present a locally coupled oscillatory state space model with two populations of coupled neurons, governed by coupled PDEs. The authors evaluate the proposed model in Fourier space to allow for efficient recurrent computation through the well-known parallel associative scan operation. They evaluate the model's empirical performance on traditional sequence modeling tasks and demonstrate slightly improved performance over recent state of the art.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The study of spatiotemporal computation in a computationally efficient manner is very welcomed and the proposed methodology appears sound. It is encouraging that the authors appear to have gotten this proposed model to work in the existing SSM context given that it is quite a complex model. \n- The inclusion of different types of neurons inspired by Watakabe et al. 2023 is an interesting novelty and improves the works originality and significance. \n- The stability analysis of Section 3.4 is a nice addition and appreciated. \n- The experimental results are promising although the benchmarks used are all small scale and known to be noisy.\n\n**Weaknesses** \n- Related work section is extremely short and missing critical related work. Specifically the work of Keller & Welling (2023) (see citation below) explicitly builds a recurrent locally connected network of coupled oscillators for performing sequence modeling, also inspired by cortical spatiotemporal dynamics. Furthermore, this is far from the first work to build a computational model which leverages such dynamics, and the authors should acknowledge this in the paper and further describe how their current work differs. Without this comparison, the novelty and originality of their approach is very hard to discern. A non-exhaustive list of papers that the authors should both read and cite is listed below:\n      - Hughes, T. W., Williamson, I. A., Minkov, M., & Fan, S. (2019). Wave physics as an analog recurrent neural network. Science advances, 5(12).\n      - Rusch, T. K., & Mishra, S. (2021). Coupled oscillatory recurrent neural network (coRNN): An accurate and (gradient) stable architecture for learning long time dependencies. arXiv preprint arXiv:2010.00951. Retrieved from https://arxiv.org/abs/2010.00951.\n\t- Keller, T. A., & Welling, M. (2023). Neural wave machines: Learning spatiotemporally structured representations with locally coupled oscillatory recurrent neural networks. In Proceedings of the 40th international conference on machine learning. PMLR.\n\t-  Keller, T. A., Muller, L., Sejnowski, T., & Welling, M. (2024). Traveling waves encode the recent past and enhance sequence learning. In The twelfth international conference on learning representations. Retrieved from https:// openreview.net/forum?id=p4S5Z6Sah4\n\t- Balkenhol, J., Händel, B., Biswas, S., Grohmann, J., v. Kistowski, J., Prada, J., . . . Dandekar, T. (2024). Beyond- local neural information processing in neuronal networks. Computational and Structural Biotechnology Journal, 23, 4288-4305. Retrieved from https://www.sciencedirect.com/science/article/pii/S200103702400360X doi: https://doi.org/10.1016/j.csbj.2024.10.040\n\t- Jacobs, M., Budzinski, R. C., Muller, L., Ba, D., & Keller, T. A. (2025). Traveling waves integrate spatial information through time. arXiv preprint arXiv:2502.06034. Retrieved from [https://arxiv.org/abs/2502.06034](https://arxiv.org/abs/2502.06034).\n\t- Maksymov, I. S. (2023). Analogue and physical reservoir computing using water waves. Retrieved from https:// arxiv.org/abs/2306.09095\n\t- Liboni, L. H. B., Budzinski, R. C., Busch, A. N., Löwe, S., Keller, T. A., Welling, M., & Muller, L. E. (2023, November). Image segmentation with traveling waves in an ex- actly solvable recurrent neural network. arXiv. Retrieved 2024-11-19, from http://arxiv.org/abs/2311.16943 (arXiv:2311.16943)\n\t- Effenberger, F., Carvalho, P., Dubinin, I., & Singer, W. (2025). The functional role of oscillatory dynamics in neocortical circuits: A computational perspective. Proceedings of the National Academy of Sciences, 122(4), e2412830122. Retrieved from https://www.pnas.org/ doi/abs/10.1073/pnas.2412830122 doi: 10.1073/pnas .2412830122\n\t- Even more work in the neural field theory literature... \n- Because of a lack of related work and ablations, it is not clear how different the proposed model is different from prior work. For example, it is not clear how the proposed model differs from an accelerated version (through parallel scan) of the Neural Wave Machine work. Similarly, it is not clear how the model differs from a locally-connected version of the LinOSS model. (Clearly the proposed model has an additional type of neuron with different dynamics, but the inclusion of this additional complexity is unjustified without results showing that it improves performance over the simpler alternative.)\n- The definition and associated derivation of the model in section 3 is very hard to follow, despite the reviewer having experience with wave equations, and SSMs. Specifically, the space over which the neurons are defined should be stated before beginning with the derivation, and it should be made explicitly clear when switching from continuous to discrete spatial dimensions. \n    - Relatedly, equations 1 and 2 are a bit confusing because the image in figure 1 is defined over a discrete grid, while the equations appear to be implicitly defined over continuous space. These should be made explicit.\n- There are no ablations for the proposed architectural components (such as the choice of two types of neurons), and the justification for the modeling choice is very limited (with really only a single reference to Watakabe 2023 for motivation). \n- Despite the authors claiming that their model supports traveling waves, there is no analysis or visualization which demonstrates this. The Figures in the main text and appendix depict what looks like standing waves in the corresponding frequency bands. \n- In the paper the authors write: \"This paper primarily focuses on the exploration of oscillatory-based spatio-temporal model and sequential tasks. Therefore, we do not include Transformer models, as they are end-to-end frameworks\". \n\t- The use of the phrase 'end-to-end' here doesnt make sense. All of these models are end-to-end, and transformers are also used as sequence models, as the authors admit in the introduction, so the choice not to compare with Transformers appears more biased towards making their model look better rather than a true methodological difference.\n\nQUESTIONS:\n- Can the authors expand on the connection of their model to the results of Watakabe et al 2023? From my reading of the paper, it is not precisely clear how their specific choice of P and O neurons has much relation to these results which instead describe diffuse/patchy connections based on their associated targets. The presented BioOSS model does not seem to recapitulate these two types of projections. Is the idea that the diffuse connectivity is the 'modulation'? And the columns are the P neurons? Where are the patchy connections in this model?\n\n- Section 3.3 is too detailed for the main text and should largely be moved to the appendix in my opinion, with greater intuition for the proposed model and the resulting final equations in the main text. \n\n- Can the authors justify their choice of the particular dynamical system more explicitly? Either from the biological point of view, directly connecting it to natural observations (with explicit abstractions), or through ablation? For example, are the O neurons really necessary in this setup? What do they add that would be different from the Neural Wave Machine type of architecture, or LinOSS with local connectivity, for example? \n\n- Section 4.2 seems bizarrely out of place. What is the take-away from this analysis? It is clear that a dynamical system of oscillations will be dependant on the initalization, and a network of locally coupled oscillators will be dependant on connectivity eigenmode initalization. However this initalization does not appear relevant to any of the following experiments, and therefore seems to lack purpose in the text. I would suggest authors move this to the appendix in favor of including more related work, ablations, and justification for the model. \n\n- How were train-test splits for 4.3.1 chosen? This suite of tasks is known to be very specific to the choice of train-test split (from my own extensive experience with this data set).\n\n- Section 4.3.2 text is largely repetitive of the tables in the section. There is no need to repeat the exact numbers from the table in the text. This can be removed to make more space for substantive material as discussed above.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-22"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a new model inspired by current oscillatory sequence models as well as neuroscience findings. It can be described using coupled PDEs, which can be solved efficiently in the Fourier domain, where the state evolution is linear and associative scans are applicable (which have $\\log T$ time complexity). It reaches SOA performance on multiple tasks.\n\nSTRENGTHS AND WEAKNESSES:\n**Results and Significance**\n\n\n**Strenghts**: The results generally appear solid enough. On an appropriately chosen benchmark, the model outperforms existing architectures with comparable amounts of parameters and compute.\n\n**Weaknesses**: The authors write that the model is very hyper parameter sensitive (line 421). It is unclear how much time was spent hyper-parameter tuning relative to the baseline models. In what fidelity was the learning rate (between 0.000001 and 0.005) explored?\n\n\n**Clarity and originality**:\n\n\n**Strengths**: The introduction is well written and nicely motivates the architecture \n\n\n**Weaknesses**: The writing can be improved in two points, namely:\n\n*Comparison to previous work.* I think an important omission in the relation to previous work is from TA Keller & M Welling (on e.g., neural wave machines / traveling waves) which use a very similar setup of neuro-inspired oscillatory RNNs with spatial interactions (the latter being efficiently implemented with convolutions).\n\n*Section 3.2 to 3.5 can be improved significantly in my opinion.* First of all it contains some elaborate details that can be moved to the appendix, for instance section 3.2 is an almost page long derivation, just to end up with a well known euler based integration scheme. On the other hand some essential details (even just a brief description), such as how the PDEs are used in an ML architecture (does the model contain multiple layers of these PDS with MLPs in between?) are omitted. Where/when do we transform back from Fourier domain to time domain?\n\n**Minor**\n\n- There are some inconsistencies in whether $o$ and $p$ are subscripts or superscripts of $k$ (and sometimes they are missing at all).\n- I don´t understand line 114/115, how does the time-dependancy of the input play a role (or how is it addressed later?). \n- Line 118: isn’t this already diagonal by construction, why do we need to the intermediate block-diagonal step.\n- Line 134, you have not explained why you want this diagonal matrix, maybe add something along the lines of “... so that we can perform the scan operation on the diagonalised transition matrix, similar to in Orvieto et al. 2023” \n- Line 178. I don’t understand why this is not $f =  \\frac{\\theta}{ 2 \\pi dt}$.  A 2 dimensional linear system with c.c. eigenvalues with phase $\\theta$ results in a sine and cosine trajectory with $f =  \\frac{\\theta}{ 2 \\pi dt}$, no?\n\nQUESTIONS:\nSee above\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces BIoOSS, an extension of the recently proposed LinOSS model that incorporates locally coupled dynamics into the original framework.The paper provides an explanation how parallel scans can still be used for this framework and finally presents numerical results of BIoOSS mostly comparing it to the LinOSS model. However, the manuscript suffers from poor organization, lacks a clear motivation for introducing local coupling, and fails to demonstrate improved performance—especially when benchmarked against the originally published LinOSS results rather than the modified baselines reported in this paper (more in strengths and weaknesses).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* the paper extends a recently published model that has gained significant attention \n* the paper provides empirical results on various datasets\n\nWeaknesses:\n* The paper clearly extends and is fully based on the recently published LinOSS model of Rusch and Rus, 2024, e.g., the whole structure and representation follows the LinOSS paper almost verbatim. However, this is not explicitly stated and LinOSS is only cited twice before section 4 in the context of \"other oscillatory models\". I urge the authors to provide proper credit and clearly state that this works follows closely Rusch and Rus, 2024. \n\n* While BIoOSS does introduce coupling between nodes, the explicit formulation used in this manuscript lacks proper motivation. Why is this type of coupling used and not anything else? This is never explained nor motivated.\n\n* It's not clear why coupling is needed for the LinOSS model. Rusch and Rus, 2024 have shown that LinOSS is universal. What functions/operators can BIoOSS express that LinOSS cannot? I presume there is none. Moreover, LinOSS does have coupling mechanism (otherwise it would not be universal). However, this is done in between layers, i.e., the dense input weight matrix modulates and couples the outputs from the layer before, which by definition is a coupling mechanism. Why is that not sufficient? There needs to be more explanations and analysis.\n\n* The writing of the paper is poor, e.g., it's not clear how the underlying system is solved using parallel scans. In line 131-134: \"This associativity is essential because it enables the use of parallel scan operations, reducing the computational complexity [...]. Therefore, we need to make the eigendecomposition [...]\". How does the eigendecomposition help here? I presume the system is solved by diagonalizing it. However, this is never mentioned explicitly. \n\n* Line 155-156 says that P inverse only needs to be computed once. However, doesn't it need to be computed for every node i,j? How costly is that? Do you include it in the speed comparisons in the appendix?\n\n* Table 1 provides weaker results for LinOSS than stated in the official LinOSS paper. This is very problematic. If one considers the originally published results of LInOSS, it performs much better (and not worse as stated in this manuscript) than BIoOSS (67.8\\% for LinOSS vs 66.25\\% for BIoOSS).\n\n* The eigendecomposition provided is only an approximation. There is no further explanation or analysis provided how this affects the model behavior and performance.\n\n* The paper does not provide any deeper or theoretical insights of this model nor lacks a clear motivation of why this particular model is chosen.\n\nQUESTIONS:\nSee weaknesses\n\nLIMITATIONS:\nThe model might not scale well with large number of nodes. It does not provide any clear advantages over the original LinOSS model and performs similarly or worse in practice.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper builds on current literature on biologically inspired models, specifically on the recent LinOSS paper and proposed a new mathematical formulation for oscillatory neural networks, The paper claims this current formulation is better at spatio temporal correlations and shows results on several benchmarks\n\nSTRENGTHS AND WEAKNESSES:\nPros\nThe model and the discretization formuation are nice. \nThe explicit modeling of both spatial and temporal interactions allows for richer dynamics than standard sequence models, capturing structure closer to real neural circuits.\nBioOSS improves transparency and analysis of internal representations\nThe paper moves forward the field of oscillatory models.\nThe paper presents a reasonable suite of benchmarking tests.\n\nCons\nWhile biologically inspired, the model’s structural complexity (2D grids, dynamic interactions) may pose challenges for scaling to large data or high-resolution domains.\nComparisons to state-of-the-art deep models (transformers & SSMs) in high-impact benchmarks are needed to demonstrate practical utility. It would be great to see comparisons to Mamba\nIt would be great to go deeper into the mathematics of these models and identify beyond empirical results on benchmarks (because a benchmark is just a benchmark and BioOSS does not always have the best performance on the benchmarks) why and when this mathematical model will provide effective spatio-temporal correlations\nMore empirical work across diverse domains (e.g., vision, speech, control) would help ground claims of generalizability.\n\nQUESTIONS:\nHow do structured local interactions among p and o neurons influence the emergence of wave-like propagation patterns, and how critical are these interactions for encoding temporal dependencies?\n\nHow robust is BioOSS to noisy, sparse, or irregularly sampled time-series data compared to other oscillatory or recurrent models?\n\nCan you characterize mathematically the computation requirements for training and inference?\n\nCan you state clearly where does BioOSS clearly outperform standard models—and why?\n\nIs the interpretability gained from spatial frequency structure usable in downstream applications like clinical diagnostics or robotics?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}