Title: Kuramoto Orientation Diffusion Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: diffusion model

phase synchronization

neural oscillations
Abstract: Orientation-rich images, such as fingerprints and textures, often exhibit coherent angular directional patterns that are challenging to model using standard generative approaches based on isotropic Euclidean diffusion. Motivated by the role of phase synchronization in biological systems, we propose a score-based generative model built on periodic domains by leveraging stochastic Kuramoto dynamics in the diffusion process.  In neural and physical systems, Kuramoto models capture synchronization phenomena across coupled oscillators -- a behavior that we re-purpose here as an inductive bias for structured image generation. In our framework, the forward process performs \textit{synchronization} among phase variables through globally or locally coupled oscillator interactions and attraction to a global reference phase, gradually collapsing the data into a low-entropy von Mises distribution. The reverse process then performs \textit{desynchronization}, generating diverse patterns by reversing the dynamics with a learned score function. This approach enables structured destruction during forward diffusion and a hierarchical generation process that progressively refines global coherence into fine-scale details. We implement wrapped Gaussian transition kernels and periodicity-aware networks to account for the circular geometry. Our method achieves competitive results on general image benchmarks and significantly improves generation quality on orientation-dense datasets like fingerprints and textures. Ultimately, this work demonstrates the promise of biologically inspired synchronization dynamics as structured priors in generative modeling.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nStandard generative approaches based on isotropic Euclidean diffusion struggle with orientation rich inputs, which often exhibit coherent angular directional patterns. To tackle this problem, the work proposes to make use of stochastic Kuramoto dynamics in the diffusion process, borrowing from the idea that Kuramoto models capture synchronization phenomena across coupled oscillators. The reformulated forward process gradually collapse the data into a low-entropy von Mises distribution, and generate diverse patterns by reversing the dynamics with a learned score function.  The approach presented in this work enables structured destruction during forward diffusion and a hierarchical generation process that progressively refines global coherence into fine-scale details.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n1. The work introduces a new diffusion process, which preserves angular information, based on the mathematical Kuramoto model which describes synchronization. The transformed distribution (from the targeted distribution) behaves like a von Mises distribution at large timesteps. Nonetheless, these aspects make the work very interesting as the new diffusion process is based on a biological-plausible process. \n\n2. According to the work's experiment results, the new diffusion process demonstrates much better performance than the SDE-based Gaussian diffusion model in [Song et al. (2021)](https://arxiv.org/abs/2011.13456) on fingerprint and texture datasets.\n\n3. An advantage of the introduced diffusion process is the enabling of hierarchical generative process, which allows for a more coherent generative process resembling coarse-to-fine generation.\n\n## Weaknesses\n\n1. The work has limited results and the introduced approach underforms in terms of generating CIFAR10 images. Moreover, for NLL experiments on Earth and climate science datasets, the introduced approach only performs decently.\n\n2. The presence of the nonlinear drift in the stochastic Kuramoto model renders the marginal distribution intractable, while at each training step, it requires $O(T)$ operations to simulate the forward Markov chain.\n\n3. It is still unclear on the exact benefits of the new diffusion process over Gaussian and other processes. \n\n## Comment \n\nOverall, I think the work is interesting, but it has limited results and can be presented better to highlight the approach and also the problem it is trying to tackle. \n\nBy the way, there is an error in your supplementary regarding $\\nabla_{\\theta_t} \\log p(\\theta_t)$ on line 38 of section B.2, which should be $\\nabla_{\\theta_t} \\log p(\\theta_t) = (p(\\theta_t))^{-1} ~ \\nabla_{\\theta_t} p(\\theta_t)$.\n\nQUESTIONS:\n1. The generated CIFAR10 images, in my eyes, look good, but the FID scores do not reflect that. Are there any intuition why this diffusion process does not work well? Also, it would be nice for you to contrast and highlight the theoretical benefits of the Kuramoto model in contrast of the von Mises distribution as done in this [work](https://arxiv.org/pdf/2506.10576v1). FYI, I do not think you need to perform experimental comparisons to it --- just a brief explanation to build your reasoning for Kuramato model is fine.\n\n2. What if you scale your timesteps to higher than 1000? For example, you could try it with 4000 timesteps to see if the trend of FID decreasing is still maintained. \n\n3. You mentioned about the relation to blurring diffusion process in your text. Why don't you compare to them and perhaps extend this relationship as well?\n\nLIMITATIONS:\nYes, see Appendix A.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-17"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a method inspired by biology, Kuramoto Orientation Diffusion Models. The data is represented in phase domain, and the diffusion process follows explicit, non-linear, locally or globally coupled phase dynamics. This leads to an approximate von Mises distribution at equilibrium. The method is experimentally validated and compared with a baseline diffusion model on CIFAR10, fingerprint, Brodatz texture datasets, and in the appendix on earth and climate science datasets.\n\nSTRENGTHS AND WEAKNESSES:\nStrenghts:\n- The paper is well written and pleasant to read.\n- The paper explores in a creative way novel, fundamental ways to define generative diffusion models, inspired by biology.\n- The experiments are on multiple datasets.\n\nWeaknesses:\n- Concepts from biology can be a source of inspiration for many of us, however, I have trouble seeing really a profound or fundamental connection with the method and applications. Rather, the connection is rather vague and on an abstract level, not concrete. I will make this point more concretely in the weaknesses and questions listed below.\n- A crucial aspect in this line of work, is how exactly the data is represented. It is clear from the start that the data should be in the form of phase angles $\\theta$. Only on line 167-168 it is briefly explained that for image data (which means for all experiments in the main paper), *pixel intensities* are mapped to angles ([-1, 1] -> [-.9 pi, .9 pi]. Thus *not* angles in the spatial dimensions, for instance by representing the pixels in polar coordinates. This is very confusing to me and in my opinion troubles much of the motivation and context of this method.\n- Since the forward process is non-linear, and thus the conditional non-Gaussian, the method requires calculating the Markov chain (solving the forward SDE) during training, and also the score model is modelling the local transition $p(\\theta_t | \\theta_{t-1})$. These are all considerate changes compared to the baseline diffusion model. There are no ablations on these aspects, that could identify exactly why the model trains 'faster'. I would like to see more evidence on this.\n- The limitations and broader impact sections are in the appendix. Please put them in the main paper.\n\nQUESTIONS:\nQ1: As explained in Weaknesses, I would like to see ablations on the different aspects of your model. At the very least, for instance, train a model where $K(t)=0$, i.e., without coupling. This would then show us what the effect is of representing the data in phase coordinates, and of the specific diffusion process, sampling and training of your model.\n\nQ2: I assume $\\psi_{ref}$ is sampled uniformly from $[-\\pi, \\pi]$? And is thus sampled together with the starting von-Mises noise when denoising a sample?\n\nQ3: Am I correct in understanding that for the image datasets (CIFAR10, fingerprints and textures), you represent the images by mapping the pixel intensities to angles? If so, how exactly are rotation invariant datasets (fingerprints and textures) more relevant for your method?\n\nQ4: In deep learning, (hyper)spherical representation have already rich literature. See e.g.\n- https://arxiv.org/abs/1804.00891\n- https://arxiv.org/abs/2006.04437\n\nMore recently, there are many works defining diffusion models on (at least) SO(2), e.g.:\n- https://openreview.net/forum?id=BY88eBbkpe5\n- https://arxiv.org/abs/2312.11707\n\nYou might also be interested in this work where SDEs are defined on spherical manifolds, with a linear SDE: https://neurips.cc/virtual/2023/poster/71379\n\nHow do these relate to your work?\n\nQ5: Do you think representing the angles using 2-d coordinates could also be a (simpler) way to achieve your goals? Maybe then the forward conditional process could still be Gaussian? Related to this, can you comment on the choice of wrapped Gaussian distributions compared to von Mises for the diffusion process?\n\nQ6: How exactly is the Kuramoto concept relevant to your model, and to the experiments presented in the paper?\n\nLIMITATIONS:\nYes, but in appendix. This should be in the main paper.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors introduce Kuramoto orientation diffusion, which uses stochastic Kuramoto dynamics to evolve the pixels of an image, modeled as coupled oscillators, to a state of phase synchronization sampled from a von Mises distribution, analogous to the evolution of isotropic Euclidean diffusion to a Gaussian distribution. Because the Kuramoto drift function is nonlinear, the authors take care to learn the score function using wrapped Gaussian transition functions from one timestep to the next, rather than relying on a closed form expression. They also ensure that periodicity is enforced by mapping between angular and phase variables and ensuring the phase variables are \"wrapped\" to an appropriate value within the domain $[-\\pi, \\pi]$. The authors show that the phase-synchronization inductive bias built into Kuramoto orientation diffusion allows it to generate sharper images (improved FID scores) from datasets that are dominated by periodic patterns, such as textures and fingerprints, more efficiently (i.e. with fewer diffusion steps). The authors also show that this diffusion can generate natural scene images like those from CIFAR10, though this diffusion becomes less efficient than standard SGM after a large number of diffusion steps.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Novel diffusion method based on a well-known stochastic phase-synchronization process\n1. Clear performance gains on the SOCOFing fingerprint and Brodatz texture datasets over standard score-based generative models (SGMs), and respectable performance on natural images (CIFAR10).\n1. Well-written paper, easy to follow.\n\nWeaknesses:\n1. Only low resolution images were considered, with SOCOFing being the highest resolution at 1x96x96.\n1. No discussion on the choice of $\\psi_{ref}$ and its effect on results. I would imagine that a well-chosen $\\psi_{ref}$ could improve FID convergence.\n1. Some weakly supported claims about this diffusion (see Questions below).\n\nQUESTIONS:\n1. Is this method limited to low-resolution images? Why weren't higher-resolution datasets chosen? If the limited number of samples in the Brodatz was an issue, why wasn't a different dataset chosen, e.g. cellular imaging datasets like RxRx3 or JUMPCP, which contain many highly-textured objects per image?\n1. How important were the choices of noise schedules to the overall results? The schedules are presented but there is no discussion of how the particular choices were made, or what effect other choices had on the FID scores, generated image quality, etc.\n1. Please explain in more detail the claim made throughout the paper, and in particular in the Structured Destruction paragraph of Section 3.3, that states that objects within the image are preserved for more diffusion steps due to synchronized coupling. How do we know that this is true? How does Figure 5 support this claim?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel approach for building diffusion models, based on so-called Kuramoto dynamics. In this model, each pixel of an image is treated as an oscillator, characterized by a phase, which progressively evolves in the forward noise process towards an base distribution attractor. The model provides a mechanism to couple pixels together in both the noise and denoising process, which the authors say lead to a non-isotropic noise dynamics that is more prone to preserving edges and global features of ‘orientation-rich’ data. The paper provides the derivation of the necessary score matching loss function and provide some illustrations of their model on fingerprint and texture datasets compared to a standard diffusion model approach.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n- The model described here (and by that I mean the specific SDE) is new as far as I know for generative modeling, and the authors go through the derivations necessary for the score matching objective to use, as well as for some analysis of the asymptotic behavior of their SDE.\n- The paper is clear and adequately written\n\n**Weaknesses**:\n- **Unconvincing justification of why this model should be better for ‘orientation-rich’ data**. This is the main selling point suggested by the authors for their method, but it is not clear to me why this approach should be good for that. I understand the concept of having local pixels tending to be attracted to the same phase during the forward process, but that does not necessarily mean preserving edges or orientation because this happens directly in pixel space. If the diffusion was happening in a wavelet or curvelet basis, then I could agree that features such as edges would remain coherent during the noise process.\nI would recommend adding illustrations of that claim, maybe simply by running the forward Kuramoto process without noise, just to illustrate how the features of an image evolve under these dynamics. \n\n- **Limited generative modeling evaluation**: The paper only presents a comparison to SGM on 3 small image datasets.  On SOCOFing and Brodatz the proposed method seems to provide an advantage in terms of FID, but it is not completely clear if a different noise schedule or details of SGM training/sampling could impact these metrics. In general, comparing to a single generative model from 4 years ago is limited. I would recommend adding more thorough experimentations against different diffusion strategies and noise schedules to build a more complete picture. And I would be particularly interested in any illustration of the fact that the model is indeed more performant at orientations (maybe with zoomed-in view of the fingerprints for different models, at different steps of the reverse process).\n\nQUESTIONS:\n- Can the authors illustrate clearly the effect of their modelisation on preserving edges and orientation-rich features? This would help bring the main point of the paper home. I can imagine either showcasing different stages of the denoising process for fingerprint data under different models. Hopefully it should be pretty clear that the fingerprints are pretty salient early on. It might also be possible to compute summary statistics of the images that illustrate the emergence of these features early in the reverse diffusion process. \n\n- Can the authors add additional baselines to their generative modeling evaluation? In particular, it’s possible that the SGM noise schedule differ from the Kuramoto one (e.g. VP-SDE vs VE-SDE or even different noise schedules within the same class of SDE) and that could impact the generation quality without necessarily meaning that the Kuramoto process is better. More generally, I would recommend making the external baselines robust such that the reader can be convinced that the gains are specifically due to the phase synchronisation mechanism, and not because of any other implementation detail.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 1, "ORIGINALITY": 3, "DATE": "2025-07-07"}