Fractal Generative Models
Authors: Tianhong Li
Qinyi Sun
Lijie Fan
Kaiming He
Venue: Accepted by TMLR ()

Abstract: Modularization is a cornerstone of computer science, abstracting complex functions into atomic building blocks. In this paper, we introduce a new level of modularization by abstracting generative models themselves into atomic modules. Our method constructs generative models by recursively invoking atomic generative modules, resulting in architectures with fractal-like, self-similar properties. We call this new class of models fractal generative models. As a running example, we instantiate our fractal framework using autoregressive models as the atomic modules and examine it on the challenging task of pixel-by-pixel image generation. Our experiments show strong performance in both likelihood estimation and generation quality. We hope this work could serve as a starting point for future research into fractal generative models, establishing a new paradigm in generative modeling.

Published in Transactions on Machine Learning Research (12/2025)
Fractal Generative Models
Tianhong Li MIT
tionhong@mit edu
Qinyi Sun
wendysun@@äit edu
MIT
Lijie Fan Google DeepMind
@gooole..googee.coom com
Kaiming He
kaiming edu
MIT
Reviewed on OpenReview: https: openreview. net/ forun? 0k9kn61.01
5i
9--1
Sh++1
9-++1
9+++1
96+1
2011 2011
(a) generator
(b) fractal from the generator
Figure Fractal Generative Model. Four levels shown in this figure, better viewed zoomed in. In this instantiation, autoregrossive model for the fractal generator. By recursively calling autoregrossive models in autoregressive models, we build fractal-like framework with self-similarity across different levels.
Abstract
Modularization a cornerstone of computer science, abstracting complex functions into atomic building blocks. In this paper, we introduce new level of modularization by ab- stracting generative models themselves into atomic modules. Our method constructs gener- ative models by recursively invoking atomic generative modules, resulting in architoctures with fractal-like, self-similar properties. We call this new class of models fractal generative models. As running example, we instantiate our fractal framework using autoregressive models as the atomic modules and examine it on the challenging task of pixel-by-pixel image generation. Our experiments show strong performance in both likelihood estimation and generation quality. We hope this work could serve as a starting point for future research into fractal generative models, establishing new paradigm generative modeling.
1 Introduction
At the core of computer science lies the concept of modularization. For example, deep neural networks are built from atomie "layers" that serve as modular units (Szegedy et al., 2015). Similarly, modern generative models such as diffusion models (Song et al., 2020) and autoregressive models (Radford et al., 2018) are built from atomic "generative stops", each implemented by deep neural network. By abstracting complex functions into these atomic building blocks, modularization allows create more intricate systems by composing these modules.

---

Published in Transactions on Machine Learning Research (12/2025)
Building this concept, we propose abstracting generative model itself module to develop more ad- vanced generative models. Specifically, we introduce a generative model constructed by recursively invoking generative models of the same kind within itself. This recursive strategy results in a generative frame- work that cxhibits complex architectures with sclf-similarity across different levels of modules, as shown in Figure 1.
Our concept draws inspiration from the mathematical notion of fractals (Mandelbrot, 1983). While clas- sical fractals exhibit exact self-similarity at each recursion level, our framework adopts a looser, conceptual interpretation: each recursion involves generative modules of the same type but not necessarily sharing identical parameters. This relaxed definition provides greater flexibility, allowing each fractal level to adapt to the specific data distributions at different scales, a key advantage in modeling complex, real-world data. Reflecting this conceptual analogy, we term our framework "fractal generative models".
This conceptual interpretation also aligns well with fractal-like patterns widely observed in nature. Biological neural networks, for example, frequently exhibit fractal-like, scale-invariant connectivity (Bassett et al., 2006; Sporns, 2006; Bullmore & Sporns, 2009; Ansell & Kovács, 2024), suggesting that the brain's development largely adopts the concept of modularization, recursively building larger neural networks from smaller ones. Likewise, fractal patterns are prevalent in natural data, ranging from macroscopic structures such as clouds, tree branches, and flakfla,ss, to microscopic ones including crystals (Cannon et al., 2000), chromatin (Mirny, 2011), and proteins (Enright & Leitner, 2005). More generally, natural images can also be viewed through a fractal lens an image is composed of sub-images that are themselves images (although they may follow different distributions). Thus, it is natural to consider generative models whose modules recursively invoke similar generative functionalities.
In this work, we instantiate our fracial generative framework using autoregressive modules as the generalor* which defines the recursive generation rule (Figure 1). In our instantiation, each autoregressive model is composed of modules that are themselves autoregressive models. Concretely, each parent autoregressive block, modeling a high-dimennional data distribution, recursively spawns multiple child autoregressive blocks, each modeling lower-dimensional distributions, and each child block further spawns more autoregressive blocks. The resulting hierarchical structure thus exhibits fractal-like, self-similar patterns across multiple levels.
We examine this fractal instantiation on a challenging testbed: pixel-by-pixel image generation. Existing methods that directly model the pixel sequence do not achieve satisfactory results in both likelihood esti- mation and gencration quality (Hawthorne et al., 2022; Yu et al., 2023), as images do not embody a clear sequential order. Despite its difffcullly, pixel-by-pixel generation represents a broader class of important generative problems: modeling non-sequential data with intrinsic structures, which particularly important for many data types beyond images, such as molecular structures, proteins, and biological neural networks.
Our proposed fractal framework demonstrates strong performance on this challenging yet important task. It can generate raw images pixel-by-pixl (Figure 3) while achieving accurate likelihood estimation and high generation quality. We hope our promising results will encourage further explorations of fractal generative models in their design and applications, ultimately establishing a new paradigm for generative modeling.
2 Related Work
Fractals. A fractal is geometric structure characterized by self-similarity across different scales, often constructed by a recursive generation rule called generator (Mandelbrot, 1983). Fractals are widely observed in nature, with classic examples ranging from macroscopic structures such as clouds, tree branches, and snowflakes, to microscopic ones including crystals (Cannon al., 2000), chromatin (Mirny, 2011), and proteins (Enright & Leitner, 2005).
Beyond these more casily recognizable fractals, many natural data also exhibit near-fractal characteristics. Although they do not possess strict self-similarrty, they still embody similar multi-scale representations
We use the term "generator" following Mandelbrot (1983). In this paper, "generator" specifically refers to rule for recursively forming fractals.
2

---

Published in Transactions on Machine Learning Research (12/2025)
patterns, such images (Freeman et al., 1991; Lowe, 1999) and biological neural networks (Bassett et al., 2006; Sporns, 2006; Bullmore & Sporns, 2009). Conceptually, our fractal generative model naturally accommodates all such non-sequential data with intrinsic structures and self-similarity across different scales; in this paper, we demonstrate its capabilities with an image- based instantiation.
Hierarchical Representations. Extracting hierarchical pyramid representations from visual data has long been an important topic in computer vision. Many early land-engineered features, such as Steerable Filters, Laplacian Pyramid, and SIFT, employ scale-space analysis to construct feature pyramids (Burt & Adelson, 1987; Freeman et al., 1991; Lowe, 1999; 2004; Dalal & Triggs, 2005). In the context of neural networks, hierarchical designs remain important for capturing multi-scale information. For instance, SPPNet (He et al., 2015) and FPN (Lin et al., 2017) construct multi-scale feature hierarchies with pyramidal feature maps. Our fractal framework is also related to Swin Transformer (Liu et al., 2021), which builds hierarchical feature maps by attending to local windows at different scales. These hierarchical representations have proven effective in various image understanding tasks, including image classification, object detection, and semantic segmentation.
Hierarchical Generative Models. Hierarchical designs are also widely used in generative modeling. Many recent methods employ a two-stage paradigm, where a pre-trained tokenizer maps images into a compact latent space, followed by a generative model on those latent codes (van den Oord et al., 2017; Razavi et al., 2019; Esser et al., 2021; Ramesh et al., 2021). Another example, MegaByte (Yu et al., 2023), implements a two-scale model with a global and a local module for more efficient autoregressive modeling of long pixel sequences, though its performance remains limited. Some recent work also applies such hierarchical concepts in reasoning models and RNNs (Wang et al., 2025; Nzoyem et al., 2025).
Another line of research focuses scale-space image generation. Cascaded generative models (Yu et al., 2020; Ramesh et al., 2022; Ho et al., 2022; Pernias et al., 2023) train multiple generative models, such as diffusion models or normalizing flow models, to progressively generate images from low resolution to high resolution. More recently, scale-space autoregressive methods (Tian et al., 2024; Tang et al., 2024; Han et al., 2024) generate tokens one scale at a time using an autoregressive transformer. However, generating images without a tokenizer is often prohibitively expensive for these autoregressive approaches because the large number of tokens or pixels per scale leads to quadratic computational cost for the attention within cach scale.
Our fractal generative model builds upon hierarchical generative principles but uniquely integrates recursive, self-similar structures inspired by fractal geometry. Unlike conventional hicrarchical models that typically use multiple stages at fixed resolutions, our fractal framework recursively invokes generative modules of the same type across multiple image scales, analogous to the recursive self-similarity of natural fractal structures. This recursive design enables our approach to effectively capture intrinsically structured, non-sequential data, introducing novel form of hierarchical generative modeling.
Modularized Neural Architecture Design. Modularization is fundamental concept in computer science and deep learning, which atomizes previously complex functions into simple modular units. One of the earliest modular neural architectures was GoogleNet (Szegedy et al., 2015), which introduced the "Inception module" as a new level of organization. Later research expanded on this principle, designing widely used units such as the residual block (He et al., 2016) and the Transformer block (Vaswani, 2017). Recently, in the field of generative modeling, MAR (Li et al., 2024) modularize diffusion models as alomic building blocks to model the distribution of each continuous token, enabling the autoregressive modeling of continuous data. By providing higher levels of abstraction, modularization enables us to build more intricate and advanced neural architectures using existing methods as building blocks.
A pioneering approach that applies a modular unit recursively and integrates fractal concepts in neural architecture design is FractalNet (Larsson et al., 2016), which constructs very deep neural networks by recursively calling a simple expansion rule. While FractalNet shares our core idea of recursively invoking a modular unit to form fractal structures, it differs from our method in two key aspects. First, FractalNet uses small block of convolutional layers as its modular unit, while we use entire generative model, representing different levels of modularization. Second, FractalNet was mainly designed for classification tasks and thus outputs only low-dimensional logits. In contrast, our approach leverages the exponential scaling behavior of
3

---

Published in Transactions on Machine Learning Research (12/2025)
fractal patterns to generate large set of outputs (e.g., millions of image pixels), demonstrating the potential of fractal-inspired designs for more complex tasks beyond classification.
3 Fractal Generative Models
In this section, we introduce the fractal generative framework and illustrate how it can efficiently model high-dimennsonal data distributions by recursively composing atomic generative modules. We first outline the general framework and then present an instantiation using autoregressive models as the atomic modules.
3.1 Fractal Generative Framework
A fractal generative model recursively builds more powerful generative models from simpler atomic generative modules. Formally, cach fractal level consists of a generator 9i that transforms an output from the previous level into set of new oulputs {.++}]: {.++1} = (x;)). By recursively invoking similar generalive modules at each level (as illustrated Figure 1), this framework achieves exponential output growth with only a lincar increase in recursive depth. This property enables efficient modcling of complex, high-dimensional data.
Unlike classical fractals with strict self-similarity and parameter sharing, our framework employs flexible, conceptual fractal structure. Each fractal generator level uses independently parameterized modules, allow- ing the model to adapt to scale-specific data characteristics while keeping structural self-similarity across recursive levels.
3.2 Instantiation with Autoregressive Modules
We demonstrate the fractal generative framework by instantiating it with autoregressive models. Our goal is to model the joint distribution of large set of random variables Ti, ,N, but directly modeling it with a single autoregressive model is computationally prohibitive. To address this, we adopt divide-and-conge strategy. The key modularization is to abstract an autoregressive model as a modular unit that models a conditional probability distribution p(zlc). With this modularization, we can construct a more powerful autoregressive model by building it on top of multiple next-level autoregressive models.
Specifically, assume each autoregressive module handles sequences of length k, and the total number of variables is N = k², where (N) represents the number of recursive levels in our fractal framework. The joint distribution factorizes recursively as:
p(11'" (km) Il pl " (21," " F 1))--)))-1);
where each conditional distribution p(--.I... ..... with -n-1 variables is modeled by an autoregressive module at the next fractal level. By recursively calling such a divide-and-come annde-- process, our fractal framework can efficiently handle the joint distribution of 1n2 variables using levels of autoregressive models, each operating on a manageable sequence length k.
This recursive process represents standard joint distribution, our fractal autoregressive architecture not only significantly reduces computational costs compared to a single large autoregressive model but also captures the intrinsic hierarchical structure within the data.
strategy. By recursively decomposing the
Conceptually, as long as the data exhibits structure that can be organized in a divide-and-conquee manner, it can be naturally modeled within our fractal framework. To provide more concrete example, in the next section, implement this approach to tackle the challenging task of pixel-by-pixel image generation.
4

---

Published in Transactions on Machine Learning Research (12/2025)
Output from prev. generator
91
Transformer Blocks
93
24
Outputs for next generators
Figure Instantiation our fractal method on pixel-by-pixel image generation. In each fractal level, an auloregressive model receives the output from the previous generator, concatenates it with the corresponding image patches, and employs multiple transformer blocks to produce set of out- puts for the next generators.
Figure 3: Our fractal framework can generate high-quality images pixcl-by-pixel. We show the generation process of a 256x226 image by recursively calling autoregressive models in autoregressive models. We also provide additional example videos in the sup- plementary material to illustrate the generation pro- cess.
Table 1: Model configurations, parameters, and computational costs at different autoregressive levels for our large-size model. The computational costs are measured by GFLOPs per forward pass during training with batch size 1. Notably, thanks to the fractal design, the total computational cost of modeling 256x256 256 image only twice that of modcling a 64x64 image.
image size
seq.
#layers
hidden
#params (M)
#GFLOPs
16x 16
32
1024
403
215
64x64x3
92
4x4
512
208
3
3
128
0.6
15
91
16x
32
1024
403
215
256x256x3
02
4x
8
512
25
208
4x4
4
256
3
419
94
64
4 An Image Generation Instantiation
In this section, present a concrete instantiation of our fractal generative model applied to the challenging task of pixel-by-pixel by-pixel image generation. Although demonstrated here with image data, the underlying divide- and-conquer architecture can potentially extend to other structured, higl-dimensional domains. Next, wo first discuss the challenges and importance pixel-by-pixe image generation.
4.1 Pixel-by-pixel Image Generation
Pixel-by-pixcl image generation remains significant challenge in generative modeling because of the high dimensionality and complexity of raw image data. This task requires models that can efficiently handle a large mumber of pixels while effectively learning the rich structural patterns and interdependency between them. As a result, pixcl-by-pixe image generation has become challenging benchmark where most existing methods are still limited likelihood estimation and fail to generate satisfactory images (Child et al., 2019; Hawthorne al., 2022; Yu et al., 2023).

---

Published in Transactions on Machine Learning Research (12/2025)
Recently, several scale-space autoregressive methods have explored next-scale prediction strategies for image generation (Tian et al., 2024; Tang et. al., 2024; Han et al., 2024). However, these methods rely a single autoregressive model performing full attention across the entire pixel sequence at each scale, resulting in prohibitively high computational complexity for high-rcsolution images. For example, modeling a x56x256 image at the finest scale using these methods requires each attention matrix in the transformer to have approximately 4 10 elements, making direct modeling of pixels impractical.
Though challenging, pixel-by-pixel generation represents broader class of important nigh-dimensional gen- erative problems. These problems aim to generate data seinent-by-element but differ from long-sequence modeling in that they typically involve non-sequential data. For example, many structures-such as molec- ular configurations, proteins, and biological neural networks do not exhibit a sequent ial architecture yet embody very high-dimensional and structural data distributions. By selecting pixel-by- bixeeeell image genera- tion as an instantiation for our fractal framework, we aim not only to tackle a pivotal challenge computer vision but also to demonstrate the potential of our fractal approach in addressing the broader problem of modeling high-dimensional non-sequential data with intrinsic structures.
4.2 Architecture
As shown in Figure 2, each autoregressive model takes the output from the previous-leve generator as its input and produces multiple outputs for the next-level generator. It also takes an image (which can be patch of the original image), splits it into patches, and embeds them to form the input sequence for a transformer model. These patches are also fed to the corresponding next-level generators. The transformer then takes the output of the previous generator as separate token, placed before the image tokens. Based on this combined sequence, the transformer produces multiple outputs for the next-level generator.
Following common practices from vision transformers and image generative models (Dosovitskiy et al., 2020; Peebles & Xie, 2023), we set the sequence length of the first generator 9i to 256, dividing the original images into multiple 16 x 16 patches. The second-level generator then models each patch and further subdivides them into smaller patches, continuing this process recursively. To manage computational costs, we progressively reduce the width and the number of transformer blocks for smaller patches, as modeling smaller patches is generally easier than larger ones. At the final level, we use a very lightweight transformer to model the RGB channels of each pixel autoregressively, and apply a 256-way cross-entropy loss on the prediction. The exact configurations and computational costs for each transformer across different recursive levels and resolutions are detailed in Table 1. Notably, with our fractal design, the computational cost of modeling 256x256 image is only twice that of modeling a 64x64 image.
Following Li et al. (2024), our method supports different autoregressive designs. In this work, we mainly consider two variants: a raster-order, GPT-like causal transformer (AR) and random-orde BERT- like bidirectional transformer (MAR) (Figure 6). Both designs follow the autoregressive principle of next-token prediction, each with its own advantages and disadvantages, which we discuss in detail in Appendix B. We name the fractal framework using the AR variant as FractalAR and the MAR variant as FractalMAR.
4.3 Relation to Long-Sequence Modeling
Most previous work on pixel-by-ixxe generation formulates the problem as long-sequence modeling and leverages methods from language modeling to address it (Child al., 2019; Roy et al., 2021; Ren et al., 2021; Hawthorne et al., 2022; Yu et al., 2023). However, the intrinsic structures of many data types, including but not limited to images, are beyond one-dimensional sequences. Different from these methods, we treat such data as sets (instcad of sequences) composed of multiple elements and employ a divide-and-conquqe strategy to recursively model smaller subsets with fewer elements. This approach is motivated by the observation that much of this data exhibits a near-fractal structure: images are composed of sub-images, molecules are composed of sub-molecules, and biological neural networks are composed of sub-networks. Accordingly, generative models designed handle such data should be composed of submodules that are themselves generative models.
6

---

Published in Transactions on Machine Learning Research (12/2025)
Table 2: More fractal levels achieve better likelihood estimation performance with lower com- putational costs, measured on unconditional ImageNet 64x64 test set. The GFLOPs are measured per forward pass during training with batch size 1. The training time measured per training iteration on 1 H100 GPU with batch size 8. NLL is reported in bits/dim. The network configurations of 91, 92 and Js in Table 1.
Seg Len
#GFLOPs
Training time (s/iter)
NLL
AR, full-length
64x64x3
29845
N/A
N/A
MAR, full-longth
64x64x3
29845
N/A
N/A
FractalAR (2-level)
64x64
5516
4.63
3.34
FractalMAR (2-lovel)
64x64
5516
4.63
3,36
FractalAR (3-level)
16x16
4x4
3
438
0.28
3.14
PractalMAR (3-level)
16x16
4x4
438
3.15
4.4 Implementation
We now describe our fractal generative model implementation for both training and generation processes at a high level. Further details and hyper-parameters can be found in Appendix A. We also provide our source code the supplementary materials.
Training. We train our fractal model end-to-end directly on raw image pixels following a breadth-free manner through the fractal architecture. As shown in Figure 2, during training, each autoregressive module receives outputs from the previous level, processes them through transformer blocks, and generates embed- dings for the subsequent. fractal level. At the final fractal level, lightweight. autoregressive transformer predicts discrete RGB pixel intensities (from 0 to 255). We compute a cross-entropy loss at this pixel-level prediction and backpropagate through all fractal levels, optimizing the entire framework in an end-to-end manner.
Generation. Our fractal model generates images pixel-by-pixel following depth-first manner through the fractal hierarchy, as illustrated in Figure 3. Here et al., 2024)
the random-order generation scheme from MAR (Li example. The first-level autoregressive model 91 captures the interdependence between 16x16 image patches in each 256 image, and at cach step, it produces an embedding for the next level model Ø2 to generate a patch. Subsequent generators recursively produce outputs to guide the generation of smaller patches. Ultimately, the final-level autoregressive model generates individual pixel RGB values. This recursive generation strategy effectively captures dependencies across multiple image scales, resulting in high-quality generated images.
5 Experiments
We conduct extensive experiments on the ImageNet dataset (Deng al., 2009) with resolutions 64x64 and 256x256. Our evaluation includes both unconditional and class-conditional image generation, cover- ing various aspects of the model such likelihood estimation, fidelity, diversity, and generation quality. Accordingly, we report the negative log- lokelioeldoood (NLL), Frechet Inception Distance (FID) (Heusel et al., 2017), Inception Score (IS) (Salimans et al., 2016), Precision and Recall (Dhariwal & Nichol, 2021a), and visualization results for comprchensive asscssment our fractal framework.
5.1 Likelihood Estimation
We begin by evaluating our method on unconditional ImageNet 64x64 generation to evaluate its likelihood estimation capability. To demonstrate the effectiveness of our fractal framework, we compare the likelihood estimation performance of our framework with different numbers of fractal levels, as shown in Table 2. Modcling the entire 11x64x3==12,88-p0 computational costs and makes training infeasible. Moreover, two-level fractal framework that first models the entire pixel sequence and then the RGB channels requires over ten times the computation of our three-
sequence with a single autoregressive model incurs prohibitive

---

Published in Transactions on Machine Learning Research (12/2025)
Table 3: Comparison with other Table 4: Sample quality evaluation on ImageNet x56x256 pixel- likelihood-basee models on uncondi- level generation. FractalMAR is the only pixcl-by-pixel frame- tional ImageNet 64x64 test set. NLL is work in this table. We also provide the performance of other reported in bits/dim.
pixcl-level generation methods for reference.
type
NLL.A.
type
#paraas
FID
1S1
Pro.1
Roc.
IDDPM (Nichol & Dhaziwal, 2021)
diffusion
3.53
BigGAN-dorp (Brock et al., 2018)
0.87
0.28
FM (Lipman et al,, 2022)
diffusion
3.31
GigaGAN (Kang et al. 2023)
GAN
569M
3.45
225.5
0.84
0.61
(Bartosh al, 2024)
diffusion
StyleGAN-XT. (Saner et al., 2022)
GAN
166M
2.30
265.1
0.78 0.53
Wavelet Flow
TarFlow (Zhal 2024)
al., 2020)
norm.
flow
ADM Dhariwal & Nichol (2021h)
diffusion
654M
4.59
186.7
0.82 0.52
PixeCCNN (van dea Oord
2016)
All
3.57
Simple diffusion Hoogeboom al. (2023)
diffusion
2B
3.54
205.3
Combiner AIt (Ren et al., 2021)
AR
3.42
VDM++ Kingma & Gao (2023)
diffusion
2B
2.12
267.7
Perovver ARt (Hawthoree es al, 2022)
All
3.40
Hoogeboom (2024)
diffusion
MegaByte (Yu ol,, 2023)
Alt
JetFormer Tachannen et al. (2024)
AR+flow
2.8B
6.64
0.69
0.56
PractalMAR-B
fractal
186M
1,80
274.3
0.78
0.29
PractalMAR.
Fractal
PractalMAAIL
fractal
438M
7.30
334.9
0.79
0.44
FractalMAR-H
ftactal
818M
6.15
318.9
0.81
0.46
Figure 4: Pixcl-by-pixel generation results from FractalMAR-H ImageNet 256x256. Our fractal method can generale high-quality high-resolution images in pixel-by-pixel manner, with an average throughput of 1.29 seconds per image. More qualitative results are in Figure 7.
level fractal model. Employing more fractal levels is not only more computationally efficient but also improves likelihood estimation performance, likely because it better captures the intrinsic hierarchical structure of images.
We further compare our method with other likelihood-based models in Table 5. Our fractal generative model, instantiated with both causal and masked autoregressive fractal generators, achieves strong likelihood performance. In particular, achieves negative log-likelihood of 3.14 bits per dim, which outperforms the previous best autoregressive model (3.40 bits per dim) by a significant margin and remains competitive with advanced diffusion-based and sammaliiing-foo-bbeee methods. These findings demonstrate the effectiveness

---

Published in Transactions on Machine Learning Research (12/2025)
Ground truth
Masked input
Reconstructed
Ground truth
Masked input
Reconstructed
Figure 5: Conditional pixel-by-pixe prediction results, including image inpainting (first row left half), out- painting (second row left half), uncropping (outpainting on a large mask, first row right half), and class- conditional editing (inpainting with another class label, second row right half).
of our fractal framework on the challenging task of pixel-by-pixe image generation, highlighting its promising potential in modeling high-dimensional non-sequential data distributions.
5.2 Generation Quality
We also evaluate FractalMAR on the challenging task of class-conditional image generation at a resolution of 256, using four fractal levels. We report standard metrics including FID, Inception Score, Precision, and Recall following standard practices to evaluate its generation quality Table 4. Specifically, FractalMAR- Н achieves an FID of 6.15 and an Inception Score of 348.9, with an average throughput of 1.29 seconds per image (evaluated achieves strong Inception Score and Precision, indicating its ability to generate images with high fidelity and fine-grained details, as also demonstrated in Figure 4. However, its FID and Recall are relatively weaker, suggesting that the generated samples are less diverse compared to other methods. We hypothesize that this is due to the significant challenge modeling nearly 200,000 pixels in a pixel-by-iieel way. Nonetheless, these results highlight our method's effectiveness in not only accurate likelihood estimation but also the generation of high-quality images.
batch size of 1,024 on single Nvidia H100 PCIe GPU). Notably, our method
We further observe promising scaling trend: increasing the model size from 186M to 848M parameters substantially improves the FID from 11.80 to 6.15 and the Recall from 0.29 to 0.46. We expect that additional scaling of parameters could narrow the gap in FID and Recall even further. Unlike models that rely on tokenizers, our hothod is free from the reconstruction errors introduced by tokenization, suggesting the potential for uncapped performance gains with larger model capacities.
5.3 Conditional Pixel-by bixeel Prediction
We further examine the conditional pixel-by-pixel prediction performance our method using conventional tasks image editing. Figure 5 provides several examples, including inpainting, outpainting, uncropping, and class-conditional oditing. As shown in the figure, our method can accurately predict the masked pixels based on the unmasked regions. Additionally, it can effectively capture high-level semantics from class labels and reffect it in the predicted pixels. This is illustrated in the class-condiitonal editing example, where the model replaces cat's face with that of dog by conditioning on the dog's class label. These results demonstrate the effectiveness of our method in predicting unknown data given known conditions.
More broadly, by generating data element- element-by-elemeent bleelemen, our method provides a generation process that is more interpretable for humans compared approaches such as diffusion models or generative models operating in latent spaces. This interpretable generation process not only allows better understand how data is generated but also offers way to control and interact with the generation. Such abilities particularly
9

---

Published in Transactions on Machine Learning Research (12/2025)
important in applications like visual content creation, architectural design, and drug discovery. Our promising results highlight the potential of our approach for controllable and interactive generation, paving the way for future explorations in this direction.
6 Discussion and Conclusion
The effectiveness of our proposed fractal generative model, demonstrated through the challenging task of pixel-by-pixel generation, suggests new opportunities for designing generative models. highlights the potential of dividing complex data distributions into manageable sub-problems and addressing them by using existing generative models as modular units. We believe that fractal generative models are particularly suited for modeling data, with intrinsic structures beyond sequential orders. We hope that the simplicity and effectiveness of our method will inspire the research community to explore novel designs and applications of fractal generative models.
Broader Impacts. Our primary aim is to advance the fundamental research on generative models. Similar to all generative models, there are potential negative societal consequences if our approach is misused to produce disinformation or amplify biases. However, these considerations lie outside the scope of this paper and are therefore not discussed in detail.
10

---

Published in Transactions on Machine Learning Research (12/2025)
References
Helen S Ansell and István Kovács. Unveiling universal aspects of the cellular anatomy of the brain. Communications Plysics, 7(1):184, 2024.
Grigory Bartosh, Dmitry Vetrov, and Christian A Naessoth. Neural flow diff'usion models: Learnable forward process for improved diffusion modelling, arXiv preprinl orXiv:2404. 12940, 2024.
Danielle Bassett, Andreas Meyer-Lindenberg, Sophie Achard, Thomas Duke, and Edward Bullmore. Adaptive reconfiguration of fractal small-world human brain functional networks. Proceedings of the National Academy of Sciences, 103(51):19518-1D))
2006.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. arXiv preprint Xiv::11809 11096, 2018.
Ed Bullmore and Olaf Sporus. Complex brain networks: graph theoretical analysis structural and functional systems. Nature reviews neuroscience, 10(3):186 198, 2009.
Peter J Burt and Edward H. Adelson. The laplacian pyramid as compact image code. In Readings in computer vision, pp. 671 679. Elsevier, 1987.
JAMES W Cannon, WILLIAM J Floyd, and WALTER R Parry. Crystal growth, biological cell growth, and geometry. Pattern formation in biology: vision, and dynamics. Singapore: World Scientific, pp. 65-82, 2000.
Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. MaskGIT: Masked generative image Trans- former. In CVPR, 2022.
Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William Freeman, Michael Rubinstein, Yuanzhen Li, and Dilip Krishnan. Muse: Text-to-image generation via masked generative Transformers. In ICML, 2023.
Rewon Child, Scott Gray, Alec Radford, and Tlya Sutskever. Generating long sequences with sparse transformers. arXiv preprint arXi:: 10509, 2019.
Navneet Dalal and Bill Triggs. Histograms of oriented gradients for human detection. In 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), volume pp. 886-893. Icse, 2005.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: large-scale hierarchical image database. In CVPR, 2009.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. NewrIPS, 2021a.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural infor- mation processing systems, 34:8780-8794, 2021b.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint 00t:20000111990 2020.
Matthew B Enright and David M Leitner. Mass fractal dimension and the compactness of proteins. Physical Review Statistical, Nonlinear, and Soft Matter Physics, 71(1):011912, 2005.
Patrick Easer, Robin Rombach, and Bjorn Ommer. Taming Transformers for high-resolution image synthesis. Tn CVPR, 2021.
William T Freeman, Edwaar H Adelson, et al. The design and use steerable filters. IEFE Transactions on Pattern analysis and machine intelligence, 13(9):891-906 1991.
Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordlmis, Lukasz Wesolowski, Aapo Kyrola, Andrew Tulloch, Yangging Jia, and Kaiming He. Accurate, large minibatch SGD: Training ImageNet in hour. arXiv: 176, 02677, 2017.
Jian Han, Jinlai Liu, Yi Jiang, Bin Yan, Yuqi Zhang, Zehuan Yuan, Bingyue Peng, and Xiaobing Liu. Infinity: Scaling bitwise autoregressive modeling for high-resolution image synthesis. arXio preprint orXio:24 12. 2404411, 2024.
11

---

Published in Transactions on Machine Learning Research (12/2025)
Curtis Hawthorne, Andrew Jaegle, Cätälina Cangea, Sebastian Borgeaud, Charlie Nash, Mateusz Malinowski, Sander Dieleman, Oriol Vinyals, Malthew Bolvinick, Ian Simon, et al. General-purpose, long-contexi auloregressive modeling with perceiver ar. In International Conference on Machine Learning, pp. 8535-8558. PMLR, 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE transuctions pallern analysis and machinee intelligence, 37(9):1904 -9119999-911,1----999111991 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In CVPR, 2016.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by two time-scale update rule converge to a local nash equilibrium. In NIP, 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXio:2207. 12598, 2022.
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David Fleet, et al. Imagen video: High definition video generation with diffusion models. arXie preprint Arii222210. 0230000000,,,03, 2022.
Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. simple diffusion: images. arXiv preprint (rXi::2301.11093, 2023.
End-to-end diffusion for high resolution
Emiel Hoogeboom, Thomas Mensink, Jonathan Heek, Kay Lamerigts, Ruiqi Gao, and Tim Salimans. Simpler diffusion (sid2): 1.5 fid on imagenet512 with pixel-space diffusion. arXie preprint arXiv:2410.1900 19324,, 2024.
Minguk Kang, Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. Proceedings of the IEEE/CVE conference on computer vision and pallern recognilion, pp. 10124-10134, 10134, 2023.
Diederik Kingma and Ruiqi Gao. Understanding diffusion objectives the ELBO with simple data augmentation. In NewrIPS, 2023.
Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. Fractalnet: Ultra-deep neural networks without resid- uals. arXiv preprint 165500605007443 2016.
Tianhong Li, Huiwen Chang, Shlok Mishra, Han Zhang, Dina Katabi, and Dilip Krishnan. MAGE: Masked generative encoder unify representation learning and image synthesis. In CVPR, 2023.
Tianhong Li, Yonglong Tian, He Tá, Mingyang Deng, and Kaiming He. Autoregressive image generation without vector quantization. arXiv preprint arXim:246.11 1838, 2021.
Tsung-Yi Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pettern recognition, pp. 2117-2125, 2017.
Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.0274: 02747, 2022.
Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 10012-10022 2021.
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In ICLR, 2019.
David G Lowe. Object recognition from local scale-invariant features. In Proceedings of the seventh IEEE international conference on computer vision, volume pp. 1150 1157. Ieee, 1999.
David G Lowe. Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60:91 110, 2004.
Benoit Mandelbrot. The fractal geometry nature/revised and enlarged edition. New York, 1983.
Leonid A Mirny. The fractal globule as a model of chromatin architecture in the cell. Chromosome research, 19: 37 51, 2011.
12

---

Published in Transactions on Machine Learning Research (12/2025)
Debarshi Mustafi, Andreas H Engel, and Krzysztof Palczewski. Structure of cone pholoreceploos. Progress in retinal and eye research, 28(4):289 302, 2009.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusson probabilistic models. In ICML, 2021.
Roussel Desmond Nzoyem, Nawid Keshtmand, Enrique Crespo Fernandez, Idriss Tsayem, Raul Santos- Rodriguez, David AW Barton, and Tom Deakin. Weight-space linear recurrent neural networks. arXiv preprint arXi::2506.01155 2025.
William Peebles and Saining Xie. Scalable diffusion models with Transformers. ICCV, 2023.
Pablo Peruias, Dominic Rampas, Mats L Richter, Christopher J Pal, and Marc Aubreville. Würstchen: An efficient architecture for large-scale text-to-image diffusion models. orXiv preprint orXi::2306.000) 2023.
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by gener- ative pre-training. 2018.
Adilya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In ICML, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-condiitonal image generation with clip latents. orXiv preprint arXiv.2044.61122 2022.
Ali Razavi, Aaron Van den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with VQ-VAE-2. In NeurIPS, 2019.
Hongyu Ren, Hanjun Dai, Zihang Dai, Mengjiao Yang, Jure Leskovec, Dale Schuurmans, and Bo Dai. Combiner: Full attention transformer with sparse computation cost. Advances in Newral Information Processing Systems, 34: 22470-22482) 2021.
Aurko Roy, Mohammad Saffar, Ashish Vaswani, and David Grangier. Efficient content- based sparse attention with routing transformers. Transactions of the Association for Computational Lénguistics, 9:53 68, 2021.
Tim Salimans, lan Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training GANs. In NewrIPS, 2016.
Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. ACM SIGGRAPH 2022 conference proceedings, pp. 1-10, 2022.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. ar preprinl arXt::2010.02500 2020.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. Xáv preprint Nrx:::3303.0146. 2023.
Olaf Sporns. Small-world connectivity, motif composition, and complexity of fractal neuronal connections. Biosystems, 85())555-64, 2006.
Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1-9, 2015.
Haotian Tang, Yecheng Wu, Shang Yang, Enze Xie, Junsong Chen, Junyu Chen, Zhuoyang Zhang, Han Cai, Yao Lu, and Song Han. Hart: Efficient visual generation with hybrid autoregressive transformer. arXie preprint 2rXie:2410.1083 10812, 2024.
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. arXiv preprint arXix:2404.0888 0229000 2024.
Michael Tschannen, André Susano Pinto, and Alexander Kolesnikov. Jetformer: An autoregressive generative model raw images and text. arXiv preprint arxim:2 11. 19722, 2024.
Aaron van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, and Koray Kavukeuoglu. Conditional image generation with PixelCNN decoders. In NewrIPS, 2016.
13

---

Published in Transactions on Machine Learning Research (12/2025)
Aaron den Oord, Oriol Vinyals, and Koray Kavukcuoglu. Neural discrete representation learning. In NewrIPS, 2017.
A Vaswani. Attention is all you need. Advonces in Neural Information Processing Systems, 2017.
Guan Wang, Jin Li, Yuhao Sun, Xing Chen, Changling Liu, Yue Wu, Meng Lu, Sen Song, and Yasin Abbasi Yadkori. Hierarchical reasoning model. Xi preprint ar Xrx:22200..0 134, 2025.
Jason Yu, Konstantinos G Derpanis, and Marcus Brubaker. Wavelet flow: Fast training of high resolution normalizing flows. Advances in Newral Information Processing Systems, 53:6184-61999 6196, 2020.
Lili Yu, Dániel Simig, Colin Flaherly, Armen Aghajanyan, Luke Zettlemoyer, and Mike Lewis. Megabyle: Predicting million-byte sequences with multiscale transformers. Advances in Neural Information Processing Systems, 36: 78808-78823, 2023.
Shuangfei Zhai, Ruixiang Zhang, Preetum Nakkiran, David Berthelot, Jiatao Gu, Huangjie Zheng, Tianrong Chen, Miguel Angel Bautisla, Navdeep Jailly, and Josh Susskind. Normalizing flows are capable generative models. arXiv preprint Xiv:24 12. 06329, 2024.
14

---

Published in Transactions on Machine Learning Research (12/2025)
A Implementation Details
Here we provide more step-by-step implementation details of the training and generation process of our pixel-by-ixxe image generation instantiation. We also include the codes to reproduce our results in the supplementary materials. All codes and models will be made publicly available.
Training. Below is detailed step-by-ster description of the training procedure on 256 images:
At the highest fractal level, generator J1 receives the class embedding (or condition embedding) as input, along with embeddings of 16x16 image patches. The transformer within Ii processes these embeddings to produce output embeddings that serve as conditional inputs for the next fractal level 92- Due to the autoregressive nature the generator, the training can be done in a teacher-forcing way that produces embeddings and receives gradients from all patches in one training iteration.
The second-level generator 92 then takes these conditional embeddings from 91 and combincs them with embeddings of 4x4 patches in each 16x16 patch. The transformer of g2 processes these combined inputs and generates further refined embeddings to be used as conditional inputs for the subsequent fractal level 93.
Similarly, the third-level generator gs combines the conditional embeddings from 92 with the embeddings of all pixels in each 4x4 patch. Its transformer outputs embeddings that condition the final fractal level
94-
At the final fractal level 94. specializod pixcl-level autorogressive transformer prodicts discrete RGB values for individual pixels. These pixel-level RGB intensities are discretized into 256 levels, and cross-entropy loss is computed between these predictions and the ground-truth pixel values from the input images. Finally, the computed cross-entropy loss at the pixel level is backpropagated through all fractal levels and trains the entire framework in an end-to-end manner.
The backward pass is simply the automatic differentiation of the forward pass. In summary, each fractal generator receives conditioning signals from the previous level and produces new conditions for the next. The final level computes the cross-entropy loss, and gradients are propagated in reverse order.
When modeling high-resolution images with multiple levels of autoregressive models, we find it slightly helpful to include "guiding pixel" in the autoregrossive sequence. Specifically, the model first uses the output from the previous generator to predict the average pixel value of the current input image. This average pixel value then serves as an additional condition for the transformer. In this way, each generator begins with a global context before predicting fine-grained details. We apply this "guiding pixel" only in experiments on ImageNet 256x256.
Since our autoregressive model divides images into patches, can lead to inconsistent patch boundaries during generation. To address this issue, we provide the next-level generator not only the output at the current patch but also the outputs at the surrounding four patches. Although incorporating these additional inpuls slightly increases the sequence length, il significantly reduces the patch boundary artifacts.
By default, the models are trained using the AdamW optimizer (Loshchilov & Hutter, 2019) for 800 epochs (the FractalMAR-H model trained for 600 epochs). The weight docay and momenta for AdamW are 0.05 and (0.9, 0.95). We use a batch size 2048 for ImageNet 64x64 and 1024 for ImageNet #56x256, and a base learning rate (Ir) of 5e-5 (scaled by batch size divided by 256). The model is trained with 40 epochs linear Ir warmup (Goyal et al., 2017), followed by cosine schedule. The 64x64 model takes 3.5 days, and the x56x256 FractalMAR-L model takes 7.6 days on 32 H100 GPUs.
NLL computation. Since our method uses a divide-and-conquee approach by recursively applying autore- gressive decompositions, the computation of the NLL effectively becomes recursively computing the NLL for each autoregressive model. Specifically, the NLL an image is computed as
k
- -log((111 "K, =
" log (1) " [11,' ... (I((-))A--1)))
15

---

Published in Transactions on Machine Learning Research (12/2025)
This means the overall image NLL is simply the summation of individual NLL values each patch condi- tioned on previous patches. Furthermore, the NLL. of each individual patch itself is also recursively computed as the sum of NLL values of the smaller patches in it, and so on. The NLL on each pixel channel is computed w.r.t. the ground-truth pixel value using cross-entropy on 256 discrete values.
Generation. Here we describe the generation process step-by-step, using the generation of a. x56 image as an example. To provide additional clarity, we also include visualization videos illustrating this recursive generation process in the supplementary materials.
The first-level generator gi models the interdependence between the 16x16 patches in an 256 256 image. At each step of 91, it uses its transformer to produce an embedding for a x6x16 patch to generate, conditioned on the previously generated patches. Then, it asks the sccond-level generator 92 to generate this patch based on the embedding it produced.
The second-level generator 92 operates similarly but at finer scale. It models the interdependennce ween the 4x4 patches in a 16x 16 image. Specifically, at each step of 92, it uses its transformer to produce an embodding for the next 4x4 patch to generate, conditioned on proviously generated 4x4 patches and the output embedding from 91. Then, asks the third-level generator ga to generale this 4x4 patch based on this embedding.
The third-level generator a3 models and generates individual pixels within each 4x4 patch. For each step of gs, it employs its transformer to generate embeddings for the next pixel to generate, conditioned on the previously generated pixels in the patch and the output embedding from 92. Tt. then asks the final-level generator for the actual RGB value generation.
At the final fractal level, the generator 94 samples RGB values for each pixel autoregressively from proba- bility distributions produced by its transformer's output logits. These RGB predictions represent the final pixel values of the generated image.
Following common practices in the literature (Chang et al., 2022; 2023; Li et al., 2023), we implement classifier-free guidance (CFG) and temperature scaling for class-conditional generation. To facilitate CFG (Ho & Salimans, 2022), during training, the class condition is replaced with a dummy class token in 10% of the samples. At inference time, the model run with both the given class token and the dummy token, yielding two sets of logits le and Ly for each pixel channel. The predicted logits are then adjusted according to the following equation: 1 1= b, w (l. - lw), where w is the guidance scale. We employ linear CFG schedule during inference for the first-level autoregressive model, as described in Chang et al. (2023). We perform a sweep over different combinations of guidance scale and temperature to identify the optimal settings for each conditional generation model.
We also observed that CFG can suffer from numerical instability when the predicted probability for pixel value very small. To mitigate this issue, we apply top-p sampling with threshold of 0.0001 on the conditional logits before applying CFG.
Fractal Factorization. We provide a concrete walkthrough of the recursive factorization used in our fractal autoregressive framework. We aim to model a joint distribution over N random variables, p( .2,... NN). Directly factorizing this distribution with a single autoregressive model becomes impractical for large N. We therefore adopt a divide-and-conoee ann-uor strategy: we fix a block size & and choose the number of recursive levels n such that N km n log, N. At cach level, an autoregressive module models a conditional probability of the form p(a]c), where x denotes short sequence of length k and is the context produced by the previous level.
Now, we provide two-level example when N = x22 We partition the variables into k consecutive groups 1,...,k. By the chain rule,
G, = ( "...,,,,.....,,,, ia),== 1,
p(a maa = II G................
(1)
This conditional probability distribution between groups modeled by the level-1 autoregressive model. Then, each term the right is distribution over k variables, which we further factorize with lower-level
16

---

Published in Transactions on Machine Learning Research (12/2025)
(a) AR generator
(b) MAR generator
Figure 6: Two variants for antoregressive modeling. The AR variant models the sequence in raster- scan order using causal transformer. The MAR variant models the sequence in random order using bidirectional transformer: at each generation step, the model randomly selects masked positions to fill in, resulting in random sampling order. Both are valid generators to build our fractal framework.
AR module:
G G.-1)
II p 1y445 G....., Gi-1, ((-1))+1:.... fii 116-111+)11).
(2)
This conditional probability distribution within each group is modeled by the level-2 autoregressive model. Now, let. N k.. The top level groups the sequence into k contiguous blocks of size hn-1;
b p(11." xza) II p( (1 -e 21, 56 11)
Each conditional the right is again joint distribution over kn-1 variables, which is factorized by the neal fracial level using the same rule, now with exponent n-1. After n applications of this rule, we reach modules that operate on sequences of length k, completing the recursive factorization.
B
Additional Results
Table Comparison on conditional ImageNet 64x64 with classical generative models.
type
FID4
iDDMM (Nichol Dhariwal, 2021)
diffusion
2.92
StyleGAN-XL (Sauer al., 2022)
GAN
1.51
Consistency Model (Song et al., 2023)
consistency
4.70
MAR (Li et al., 2024)
AR-ddifussin
2.93
FractalAR
fractal
6.30
FractalMAR
fractal
2.76
Class-conditional ImageNet 64x64. We evaluate class-conditional image generation on ImageNet 64x64, reporting FID according to standard practice. Consistent with PixcICNN (van den Oord et al., 2016), we find that class conditioning has negligíble impact on NLL but substantially improves visual quality and FID. The results demonstrate that our fractal generative model can achieve competitive performance as classical generative models.
We also compare the performance of the AR and MAR variants, whose structures are illustrated in Figure 6, The AR variant leverages key-value caching to accelerate generation, whereas the MAR variant employs bidirectional atlention, which aligus more naturally with image modeling and enables the parallel prediction of multiple patches, thereby improving computational efficiency. As shown in the table, both of our models achieve favorable performance, with FractalMAR outperforming FractalAR overall, as also demonstratod Li et al. (2024). Therefore, we choose to use the MAR variant for conditional image synthesis on ImageNet at resolution of 256x256.
17

---

Published in Transactions on Machine Learning Research (12/2025)
Table 6: Different orders of channel modeling within cach pixel can slightly influence generation performance.
All models are trained for 400 epochs on ImageNet 64x64.
order
NLL4 FID
Y >bb Crr
N/A
3.55
B-R-G
3.17
3.32
G-RR+B
3.17
3.14
R-G-B
3.17
3.15
Modeling Pixcls. We also examine how different pixel modeling orders affect performance. We experi- mented with three autoregressive orders: RGB, GRB, and BGR, as well converting the RGB channels to the YCbCr color space. The results are summarized in Table 6. We found that while all orders achieved similar negative log-likelihood values, the FID varied slightly among the autoregressive orders (note that the NLL in the YCbCr space is not comparable with that in the RGB space). This variation likely because, akin human vision, the Inception model used to compute FID places greater emphasis on the red and green channels than on the blue channels (Mustafi al., 2009). Nonetheless, the choice of autoregressive order does not result in significant performance differences, demonstrating the robustness of our method.
Additional qualitative results. We further provide additional conditional generation results on ImageNet 256 in Figure 7, additional conditional pixel-by-pixe prediction results in Figure 8, and also include videos that demonstrate our gencration process in the supplementary material.
18

---

Published in Transactions on Machine Learning Research (12/2025)
Figure 7: Additional pixel-by-pine generation results from FractalMAR-H on ImageNet 256x256.
Ground truth
Masked input
Reconstrucced
Ground truth
Masked input.
Reconstructed
Figure 8: Additional conditional pixel-by-pixel prediction results, including image inpainting, outpainting, and uncropping.
19


Reviews:
[{"review_id": "qWwYfiOUNs", "reviewer": "Reviewer_JX8i", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors aim to establish a new paradigm with fractal generative models. The core idea is to recursively instantiate a generative model, thereby modeling a sequence problem across several hierarchical levels. This is achieved by recursively decomposing the joint distribution, which is purported to not only reduce computational costs compared to a single large autoregressive model but also to capture the intrinsic hierarchical structure within the data.\n\nThe biggest strength of this submission are in its novelty and strong results. The method is instantiated using autoregressive modules and evaluated on the challenging task of pixel-by-pixel image generation on ImageNet. On the 64x64 dataset, the model achieves strong likelihood estimation, reporting a negative log-likelihood of 3.14 bits/dim, which surpasses previous autoregressive models. On the 256x256 dataset, the model demonstrates high-quality, class-conditional generation, achieving an FID of 6.15 and an Inception Score of 348.9. The paper also provides qualitative results for conditional prediction tasks such as image inpainting, outpainting, and class-conditional editing.\n\nThe major weaknesses are linked to the evaluation protocol, implementation details, and presentation, as I mention in the Requested Changes below.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes, the claims are largely supported by the provided evidence. The experiments section, for instance, details the image completion and generation tasks in sufficient detail, providing clear evidence for the model's capabilities in this specific domain. The quantitative results presented in Tables 2, 3, and 4 are strong, demonstrating state-of-the-art performance in likelihood estimation for pixel-by-pixel models and highly competitive generation quality. The ablation in Table 2, which shows improved performance and lower computational cost with more fractal levels, provides direct support for the core \"fractal\" hypothesis. Furthermore, the qualitative results in Figures 4, 5, 7, and 8 are convincing, showing high-fidelity generation and plausible conditional edits, which visually substantiates the numerical claims.", "audience_yn": "Yes", "audience_explanation": "The paper addresses the significant challenge of generative modeling for high-dimensional, structured data, a problem of clear and persistent interest to the TMLR community. The work's strengths, which will attract audience interest, are numerous: conceptual novelty, computational efficiency, strong results, and promising potential. On the latter point, while tested on images, the fractal framework is conceptually general. It holds clear potential for other data modalities that exhibit intrinsic hierarchical or self-similar structures (e.g., molecular data, proteins, time-series), which will interest a broad segment of the community.", "requested_changes": "I think this is extremely strong work, but the following changes are suggested to strengthen it significantly. A few of them are critical for clarifying the method's novelty, correctness, and generalizability.\n\n1. _(Critical) Unaddressed Autodiff Implementation Challenges:_ A critical, unaddressed implementation challenge relates to automatic differentiation. The paper must include a detailed section discussing this. As formulated, a recursive, end-to-end trained model with $L$ levels would seem to require $L$-th order derivatives (e.g., Hessians for two levels, 3rd-order derivatives for three) to backpropagate gradients from the innermost loss to the outermost generator's parameters. This is computationally prohibitive and a potential flaw in the proposed training scheme. The authors must clarify how this is managed. If my analysis is incorrect, they must explain _why_ and provide a precise, technical description of the gradient flow.\n \n2. _(Clarification) Missing Citations and Positioning:_ The related work section is missing critical citations. The authors should include and discuss the following, as they appear highly relevant:\n \n    - \"Hierarchical Reasoning Model\", Wang et al., arXiv 2025. This paper's use of two levels of modeling (high-level and low-level) seems directly analogous to the fractal concept and should be addressed.\n        \n    - \"Weight-Space Linear Recurrent Neural Networks\", Nzoyem et al., arXiv 2025. The authors should differentiate their contribution from these works.\n        \n3. _(Clarification) Limited Testbeds and Task-Specific Justification:_ The reliance on pixel-by-pixel image generation as the sole testbed is a significant limitation. While an interesting example, its challenges are primarily dominated by the quadratic scaling at high resolutions. The authors should clarify that this task is not intrinsically challenging at low resolutions (e.g., 64x64). To substantiate the claims of generality, the method should be evaluated on additional testbeds, preferably non-image data, to demonstrate that the fractal approach is more than just an efficient architecture for images.\n    \n4. _(Critical) Lack of Ablation Studies:_ While I appreciate Table 2, I also think that the paper lacks crucial ablation studies. To confirm the method's potential, the authors must include an ablation on data _without_ a clear intrinsic hierarchical structure. This would validate whether the model's benefits are due to its fractal nature (as claimed) or if it is simply a well-performing hierarchical architecture in general.\n    \n5. _(Clarification) Parameter Scaling:_ The paper states, \"Each fractal generator level uses independently parameterized modules,\" which implies the total parameter count grows with depth. The only mitigation mentioned is reducing the width of inner Transformer blocks. This seems insufficient. The authors must provide a more thorough analysis of parameter scaling with depth and justify why this is not a significant drawback. How does it work with generative models different from Transformers?\n    \n6. _(Critical) Core Factorization Equation:_ The core factorization equation in Section 3.2 (\"The joint distribution factorizes recursively as...\") requires significant clarification. The notation is dense and not immediately obvious. The authors must provide a simpler, concrete example (e.g., for $n=2$ levels) to walk the reader through how the joint distribution is recursively decomposed.\n    \n7. _(Clarification) Loss Function:_ The authors should elaborate on the final loss computation.\n    \n    1. How is the 256-way cross-entropy loss applied across three independent RGB channels (is it a $3 \\times 256$ problem, or just $256$)?\n        \n    2. They must justify the choice of a cross-entropy loss over a simpler, and perhaps more perceptually relevant, MSE loss in RGB space, especially given that the data is continuous-valued before discretization.\n        \n8. _(Clarification) Figure 6:_ Figure 6(b) (MAR) is confusing. It is meant to illustrate a \"random order\" but still depicts patches in what appears to be a standard raster-scan order. The authors must clarify what \"random order\" means in this context or remake the figure to accurately reflect the MAR sampling process.\n    \n9. _(Clarification) Figure 1:_ The caption for Figure 1 states \"Four levels shown\" but visually, only three distinct generator levels appear to be depicted. This discrepancy should be corrected for clarity.\n    \n10. _(Clarification) Table 2:_ The naming in Table 2 is ambiguous. The third and fourth rows (\"AR, 2-level\", \"MAR, 2-level\") are not clearly differentiated from the \"FractalAR (3-level)\" and \"FractalMAR (3-level)\" models. Are the 2-level models not also \"FractalAR/MAR\"? The authors must clarify the precise architectures being compared in this table.", "broader_impact_concerns": "The authors did not include a Broader Impact Statement, even though powerful image generators like the proposed can significantly cause societal harm in terms of surveillance, deepfakes, etc.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "cfZoiX8rm7", "reviewer": "Reviewer_5J1v", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposed a new paradigm of modularization method, to abstract generative models into atomic models, involving a new class of models called fractal generative models. Fractal generative models recursively invoke atomic generative models, resulting in fractal self-similar structures. Numerical experiment of the proposed fractal framework on autoregressive model used on pixel-by-pixel image generation shows strong performance in both likelihood estimation and generation quality.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Numerical evidence is solid. Given the extent of great contributions in all other dimensions, the lacking of theoretical proofs is overall acceptable.", "audience_yn": "Yes", "audience_explanation": "This paper has great contribution and theoretical novelty, potentially preceding new lines of future researches.", "requested_changes": "No mandatory change is required. With that being said, the reviewer would prefer much more theories with rigorous proofs, in addition to the short theory provided in section 3.2.", "broader_impact_concerns": "N/A", "additional_comments": "This paper has great contribution and theoretical novelty, potentially preceding new lines of future researches. The numerical results are also good. Many implementation details are also provided. Therefore, I recommend accept.", "extra_scores": {}, "extra_text": {}}, {"review_id": "JuNATfvgmr", "reviewer": "Reviewer_JpUc", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Inspired by the concept of fractals, this paper proposes a new type of generative models, called fractal geenrative models. The fractal generative model generates high-dimensional data (e.g., images) by recursively calling layers—instantiated as autoregressive models. The intuition is that layers of autoregressive model leads to a more expressive model. This paper validates the effectiveness of fractal geenrative models on the task of pixel-by-pixel image generation.", "claims_and_evidence_yn": "Yes", "claims_explanation": "They provide the generated images in Figure 4, and compare with other types of generative models in Table 2,3.", "audience_yn": "Yes", "audience_explanation": "This paper would be of interest to researchers from the field of generative models.", "requested_changes": "Elaborate on the differences with MAR in the related work section.", "broader_impact_concerns": "None.", "additional_comments": "1. The key idea of the proposed method is formulated as the formula shown in Section 3.2. The equality in this formula holds **when** $x_1 \\ldots x_{(i-1)k^{n-1}}$ are the parents of $x_{(i-1)k^{n-1}} \\ldots x_{ik^{n-1}}$ in the underlying causal structure, which does not generally hold in the task of pixel-by-pixel image generation. So, I assume the RHS is simply an approximation of the joint probability on LHS. From this perspective, a single autoregressive model directly modeling the joint probability could lead to better performance compared with using fractal generative models. My question is: is there any result comparing their performance? and is there any result comparing their computational complexity?\n2. How to determine the number of recursive levels in experiments? Is there any ablation studies on this?\n3. The authors propose a fractal generative framework that could work in various scenarios. I just wonder if it is possible to apply this framework to 1) diffusion-model-based generators, 2) language generation tasks?\n4. Is there intuition why generating non-sequential data in a sequential manner may lead to good performance?", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)