GPSToken: Gaussian Parameterized Spatially-adaptive Tokenization for Image Representation and Generation
Authors: Zhengqiang ZHANG
Rongyuan Wu
Lingchen Sun
Lei Zhang
Venue: NeurIPS 2025 poster ()
Keywords: autoencoder
image generation
Primary area: deep_learning

Abstract: Effective and efficient tokenization plays an important role in image representation and generation. Conventional methods, constrained by uniform 2D/1D grid tokenization, are inflexible to represent regions with varying shapes and textures and at different locations, limiting their efficacy of feature representation. In this work, we propose **GPSToken**, a novel **G**aussian **P**arameterized **S**patially-adaptive **Token**ization framework, to achieve non-uniform image tokenization by leveraging parametric 2D Gaussians to dynamically model the shape, position, and textures of different image regions. We first employ an entropy-driven algorithm to partition the image into texture-homogeneous regions of variable sizes. Then, we parameterize each region as a 2D Gaussian (mean for position, covariance for shape) coupled with texture features. A specialized transformer is trained to optimize the Gaussian parameters, enabling continuous adaptation of position/shape and content-aware feature extraction. During decoding, Gaussian parameterized tokens are reconstructed into 2D feature maps through a differentiable splatting-based renderer, bridging our adaptive tokenization with standard decoders for end-to-end training. GPSToken disentangles spatial layout (Gaussian parameters) from texture features to enable efficient two-stage generation: structural layout synthesis using lightweight networks, followed by structure-conditioned texture generation. Experiments demonstrate the state-of-the-art performance of GPSToken, which achieves rFID and FID scores of 0.65 and 1.50 on image reconstruction and generation tasks using 128 tokens, respectively. Codes and models of GPSToken can be found at https://github.com/xtudbxk/GPSToken.

GPSToken: Gaussian Parameterized Tokenization for Image Representation and Generation
Zhengqiang Zhang Rongyuan Wu Lingchen Sun 2.2, Lei Zhang 1.2,1
'The Hong Kong Polytechnic University yuan. wu, chen. sun@connect. craienettpolyyu polyu. cal cslzzhng@comp. polyu. edu.
OPPO Research Institute
zhengqiang. zhang, rong-yuan.
Corrresponding author
Abstract
Effective and efficient tokenization plays an important role in image representa- tion and generation. Conventional methods, constrained by uniform 2D/ID grid tokenization, are inflexible to represent regions with varying shapes and textures and at different locations, limiting their efficacy of feature representation. In this work, we propose GPSToken, novel Gaussian Parameterized Spatially-adaptiv Tokenization framework, to achieve con-uniform image tokenization by leveraging parametric 2D Gaussians to dynamically model the shape, position, and textures of different image regions. We first employ an entropy-driven algorithm to partition the image into exxturehhomggenn eterize each region as 2D Gaussian (mean for position, covariance for shape) coupled with texture features. A specialized transformer is trained to optimize the Gaussian parameters, enabling continuous adaptation of position/shape and content-aware feature extraction. During decoding, Gaussian parameterized tokens are reconstructed into 2D feature maps through differentiable splatting renderer, bridging adaptive tokenization with standard decoders for end-to-end training. GPSToken disentangles spatial layout (Gaussian parameters) from texture features to enable efficient two-stage generation: structural layout synthesis using lightweight networks, followed by structure- structure-conditione conditioned texture generation. Exper- iments demonstrate the state-of-the-an performance of GPSToken, which achieves rFID and FID scores of 0.65 and 1.50 on image reconstruction and generation tasks using 128 tokens, respectively. Codes and models of GPSToken can be found at https:/ggthubb.coomnuuussssssS
regions of variable sizes. Then, we param-
1 Introduction
Recent advances in latent generative models such as VQGAN ΤI LDM [29]. MaskGIT 3
DiT (26). SiT 23 VAR 32], and SD3 [10] have revolutionized the research and application of image generation. Most of these methods adopt a two-stage framework. First, an auto-encoder is employed to convert original images into compact latent representations with reduced dimensionality (e.g., 256x256 + 32x32 in LDM), serving
effective "image tokenizer". Then, generative are trained the latent space, alleviating computational burdens while enabling high-quality generation. The primary goal of an image tokenizer is to learn effective representations through reconstruction tasks, encoding images into a latent space with minimal loss. Early methods such as VAE [20] transform images into continuous latent spaces. LDM 29 performs diffusion in the latent space, reducing computational cost while improving visual
models 29 26
23
32
10
40
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Ganssion Paneaco
"eocral Rettres
20Dgrid tokens
1D-grid toleons
(c) Our GPS-tokens
Ingur
GPS-tokens
Reconstraction
Input
GPS-tokens
(d) Visualizttion GPSTokom
Figure 1: Comparisons between (a) 2D- 2D-grid tokens, (b) 1D- ID-grid tokens and (c) our GPS-tokens. (d) Two visualization examples of the representation and reconstruction results GPSToken.
quality. In contrast to continuous representation, VQVAE 33 introduces discrete latent codes via vector quantization. Based on VQVAE, VQGAN I and MaskGiT train autoregressive models and achieve improved image generation performance. Beyond 2D grid tokenization, TiTok [37 transforms images into compact ID latent sequences, significantly reducing tokens. FlexTok [ and One-D-Piece [4] dynamically adjust counts, enhancing efficiency by concentrating key information early in the sequence. MAEToo [5] shows masked autoencoders yield discriminative latent spaces suitable for diffusion models.
Despite the significant progress achieved by existing image tokenization methods, the grid-based tokenization strategy used by them is inefficient and inflexible in representing the different regions with different contents natural images. As illustrated in Figs. (a) and (b), 2D-grid tokens represent local patches of fixed size and at fixed positions, matter whether the patch has complex structures or details, while ID-grid tokens encode globally contextualized information from the entire image, lacking spatially-adaptio representation ability.
In this paper, we propose GPSToken, a novel Gaussian Parameterized Spatially-adaptive kenization framework, to achieve non-uniform and flexible image tokenization. GPSToken pa- rameterizes each token with 2D Gaussian function, encoding both the positions and shapes of different regions in image. Specifically, shown in Fig. (c), each GPS-token consists of two components: the first component stores the standard deviation and position of the Gaussian function, representing region shape and location, while the second component represents the textural features of the region. Inspired by Gaussian Splatting [8], our GPSToken can rendered into 2D feature maps, facilitating seamless integration with conventional 2D decoders and enabling end-to and-to-end training.
To achieve apatially-adaptivy tokenization, we iteratively partition an image into regions varying sizes and shapes. The partitioned regions, though having different shapes and positions, will have similar amount of information in terms of entropy. The shape and position of each region are used to initialize the corresponding Gaussian parameters. Then, we utilize transformer, for which each query corresponds to a token, to refine these parameters and extract textural features. The shape-texture decomposition of GPSToken provides distinct advantages for image generation. With GPSToken, we can first generate Gaussian parameters that encode spatial layout (region shape/position), then synthesize texture features conditioned on the Gaussian geometric priors. The Gaussian priors act as structural constraints, simplifying texture generation while ensuring spatial consistency. This shape-texture decomposition approach aligns with how humans conceptualize images (structure-fin firsst, leteils-later), accelerating the model training and improving the generation quality.
Our method achieves significant improvements over existing methods in both image reconstruction and generation tasks. For image reconstruction, GPSToken achieves "rec. FID". PSNR and SSIM scores of 0.65, 24.06 and 0.657 on the ImageNet 256 reconstruction task using 128 tokens. For image generation, our model achieves a state-of-the-art FID of 1.50 on the ImageNet 256 generation task, surpassing recent methods such Titok [37], FlexTok [1. One-D-Piece [24] and MAETok 5 Our contributions are summarized as follows:

---

We propose GPSToken, an effective Gaussian parameterized spatially-adattive tokenization method for image representation and generation. GPSToken leverages 2D Gaussian functions to dynami- cally model varying region shapes and positions, significantly reducing representation redundancy in simple regions while achieving finer representation in texture-rich regions.
With GPSToken, we present shape-texture decomposition method for image generation, reducing generation complexity, accelerating model training, and improving generation quality.
Extensive experiments validate the effectiveness of GPSToken. Our work paves the way toward effective and efficient spatially-adaptive image representations, benefiting variety of vision tasks.
2 Related Work
Latent Generative Models. Latent models have gained significant attention in visual generation.
VAE [20] constructs continuous latent spaces with Gaussian priors, while VQVAE 33 couples codebooks with autoregressive modeling for discrete latent representation. VQGAN adversarial training and transformer-
I incorporates
autoregressive components, further improving generative performance. MaskGiT (3) refines discrete latent generation through scheduled parallel sampling, significantly accelerating inference. LDM [29] enables high-resolution synthesis by embedding diffusion in compressed latent spaces. DiT [26] demonstrates transformer scalability in latent diffusion, and SiT [23 extends DiT with flexible interpolation, offering versatile distribution mapping.
Image Tokenization. Image tokenization aims to create compact representations of high-dimensional images. Early methods often use VAE [20] for continuous tokenization and VQVAE 33 for discrete tokenization. VQVAE-2 (28) introduces multi-scale structure, while RQVAE [2]] builds extra codebooks to quantize residuals. DCAE 7 ensures quality at high compression ratios. MaskBit (34 proposes embedding- -rree autoencoder using bit tokens. Recently, 1D grid-based tokenization has gained attention for more compact representations. TiTok (37] is among the first to convert 2D images into ID latent tokens using masked transformers for encoding and decoding. SoftVQ (6] uses soft categorical posteriors to combine multiple codewords into one continuous token. Flex M and One-D--icee [24 project 2D images into variable-length, ordered ID sequences, allowing good reconstructions. MaeTok Tok (5) leverages mask modeling to learn semantically rich and reconstructive latent spaces, highlighting the importance of space structure for generation.
Despite the significant progress, grid-based methods remain inefficient and inflexible in capturing regions with varying content. To address this, we propose GPSToken, which parameterizes each token using 2D Gaussian function to encode region positions and shapes, allowing spatially adaptive alignment with local texture complexity. Note that while Gaussian [9] also uses 2D Gaussians, it simply replaces the original tokens in VQVAE with Gaussian distributions without spatial adaptivity. Besides, our GPSToken can decouple the visual generation process into layout synthesis and texture feature generation, while Gaussian Token does not possess a corresponding generator.
3 Methodology
In this section, we first describe the parameterization of GPSToken using 2D Gaussian functions, then present the detailed training procedure for obtaining GPSToken. The resulting tokens can be transformed into pixel-domain images through a decoder. Finally, leveraging the inherent shape- texture decomposition property of GPSToken, we propose a two-stage image generation pipeline to accelerate the training of generative models while improving their performance.
3.1 Gaussian Parameterized Tokenization
Processing images in pixel space is computationally expensive and increases model complexity.
To reduce cost, existing methods 33 28
37 5 employ image tokenizers that project an image x E H x3 into low-dimensional tokens E Ixxxe with & x W. However, current 2D/1D tokenizers are limited by rigid grid structures, restricting flexible representation of regions with varying sizes and contents. We propose GokToken, novel method that parameterizes tokens using 2D Gaussian functions, enabling adaptive and efficient modeling of complex visual regions.
3

---

Ihe overail feamework of GPSToken
Extracing features
Intializin Ganssaaz
Decoder
SA Token Refinenent
Token
Initialization
{..
9
Guassian parameters
legui
Ouiput
Tethol fistuurs /
RolAlign
Pop
Uniil enough
Append
MLP
Atta Block
regions
g'nir
Block
Block
Block
ML
The one moot complex
Divide
(b) Sparially-adatiive Token Initialization
(c) Spatially-aaptiive Token Refinement
Figure 2: (a) The overall framework of our GPSToken. (b) Spatially-adaptive Token Initialization. (c) adapiive Token Refinement.
2D Gaussian Parameterized Tokens. standard 2D Gaussian function p(x, y) is given by:
p(a, y) p(x,y)
(x px) 2p(x #>>( pv) (u nly 0,0y
exp
(I)
2(1 - )
o
where Z is the normalization constant, ox, Gg > O are the standard deviations along the x- and y-axes, and p [- 1] denotes the correlation coefficient. The means Mıx, M4y E R determine the center.
To reduce computation and focus on local regions, we modify the standard 2D Gaussian by restricting its spatial support to a bounded region centered at (Mz, Pyy and omitting the normalization constant. This design removes unnecessary computation while preserving fine details the region interest. Specifically, the modified Gaussian function is defined as:
p(x, v), if | - Pzl S so, and ly Myl S 80
g(r, v)
otherwise,
where s is a hyperparameter controlling the spatial support of the Gaussian function.
Using g. represent image via 1 Gaussian parameterized tokens R shown Fig. 1 (c). Each token contains two components z; = {g. f,)- The first component gi (o ,0
(3) p()) ,1e (2) (6)) (gray cuboids in Fig. 1 (c)) encodes spatial position and devia- tion of the Gaussian function. The second component f, (c-5) (orange cuboids) holds texture features, capturing detailed visual information from the corresponding region. This enables joint encoding of geometric and visual characteristics across image regions.
Splatting-Baze Based Rendering. Inspired by GS 18
we render GPS-tokens into 2D feature maps using splatting-based rendering. This is possible because each 2D Gaussian is continuous and can be Gaussian-parameteriiee the k-th channel of the rendered 2D feature map (x, y) can be obtained as follows:
sampled into 2D features. For example, given / Gaussian- parameterized tokens {zo, (20.21," Zi,
,
R(x, y.
Σ (,
Σ gi( v) [kk.
Advantages over Bounding Boxes and Segmentation Maps. Alternative approaches to representing image regions often rely on bounding boxes segmentation maps. Bounding boxes define regions using axis-aligned rectangles, while segmentation maps assign discrete labels to individual pixels. Compared with them, our each 2D Gaussian models anisotropic shapes with only five parameters (Pz: P'y P. az, ay); enabling compact and geometry addptivve representation that is both expressive and lightweight reducing
tokenization offers several key advantages. First,

---

the burden on downstream tasks. Second, the Gaussian function provides smooth, continuous weight distribution over pixels, naturally capturing uncertainty and modeling soft or ambiguous boundaries in natural images. Third, GPSToken is fully differentiable, enabling end-to-end training and seamless integration into existing gradient-based learning frameworks. contrast, bounding boxes are restricted to rigid, axis-aligned shapes and exhibit hard, non-differentiable boundaries. Segmentation maps, while precise, are high-dimensional, discrete, and inherently incompatible with differentiable optimization.
3.2 Spatially-ddaptive GPSToken Learning
Image tokenizers typically use an encoder-decoder framework, where the encoder maps the image x to latent representation z = Enc(x), and the decoder reconstructs it as X = Dec(z). Our GPSToken also follows this framework. As shown in Fig. 2 (a), we first apply an iterative algorithm to partition the image into regions of varying sizes based on texture complexity. Each region's position and size initialize the Gaussian parameters of the corresponding GPS-tokens, providing a coarse spatially- encoder refines GPS-tokens for fine-grained
adaptive representation. Next, a transformer- adaptation, adjusting the position, shape, and orientation according to regional textures. Finally, the GPSTokens are converted back 2D feature maps and passed through a decoder to reconstruct X.
Spatially-adapheee Token Initialization. As shown in 2 (b). we use an iterative algorithm to initialize Gaussian parameters aligned with local regions. Specifically, we maintain dynamic list of region candidates and iteratively split the most complex regions into simpler sub-regions until the target number is reached. We measure region complexity using gradient entropy. We compute the gradient magnitude map E via the Sobel operator [31], then calculate the information entropy H from the histogram of E. The overall metric is defined as:
512 m = has x H how x (- (-E i=1 4 log log((n))
where i and w are the spatial size of regions, qi is the probability of gradients in the i-th histogram bin, and À balances size and complexity. By integrating region size into the metric, promote division of larger regions. A higher m value indicates a larger and more complex region.
Once regions are determined, we associate the i-th GPSToken z; with the i-th region and initialize (6)
(8) its Gaussian parameters as gi (of") By p10) Mr ((0 { mi hy 0, yi where hi, w;
(e) (v) and σy to of w, and h, ensures full coverage during rendering. Please see Algorithm 1 in the Appendix for more details.
the height and width of regions, and (xz, yr) is its center. Setting oz
Spatially-adaptive Token Refinement. After obtaining the initialized Gaussian parameters, we em- play transformer-basee based encoder to refine these parameters to achieve fine-grained spatial adaptation, while simultaneously extracting the corresponding texture features f for each region.
As shown in Fig. c)), the encoder first projects the initial Gaussian parameters g init into query embeddings, which are then processed by attention blocks. To focus each embedding on its corre- sponding region, we include region-specific features as conditions. Specifically, we extract image features via residual blocks and use RolAlign [12] to obtain fixed-size features for each region. These are added the query embeddings before each attention block. This ensures that each query interacts with its local image features, improving alignment with regional textures.
Additionally, self-attention blocks enable query embeddings interact with each other, considering the global image layout during training. The encoder outputs residuals Ag for refining Gaussian parameters and textual features f for each token. The final GiPS-tokens are:
init
imst + Ag. f).
(5)
The refined Gaussian parameters g define the spatial layout and overall structure of the image, while f encode the textual patterns of Gaussians. They work synergistically to represent the whole image.
To illustrate the spatial adaptation of GPSTokens, we visualize g" ginit and g as Gaussian maps in Fig. 2 As shown in Fig. a), the initial map iitt aligns with the region partitions. Complex regions have denser Gaussians, while simpler ones use fewer, larger Gaussians. After encoder refinement, the parameters better match local textures. While g inst contains only axis-aligned Gaussians, the refined g includes rotated ones that align better with local structures, such as the dog's 's car edges.
5

---

Noises Rendering
Guzosian Toxtsal fratucs /
Dicoker
5. L. calltration
B. Higmae pditilication
Diflisiion
Simple Net
Layout condition
Callera
fg
Layout
Symthesis
(b) Conditional Texture Generation
(c) Visual Results
Figure 3: The overview of two-stage generation pipeline based on GPSToken.
During decoding, we first render the GPSTokens into 2D feature map using Eq. 3 then decode them into the reconstructed image. Following VQGAN [.]) we use combination of reconstruction loss Tree perceptual loss /pere, and adversarial loss Teddy during training.
3.3 GPSToken-driven Two-stage Image Generation
The shape-texture decomposition property of GPSToken naturally offers two-stage image generation pipeline, which first synthesizes the image layout using the shape information and then generates the image details using the texture features. This two-stage scheme simplifies the image generation process and improves the generation quality.
Layout Synthesis. As illustrated in Fig. (a), in the first stage, we focus on generating the overall structure of the image, which can represented by the Gaussian parameters.Nov that we use the initial Gaussian parameters gmtit instead of the final parameters because the initial Gaussian parameters are more decorrelated with the local textures and are easier to predict, while they are good enough to represent the image rough layout. Specifically, we first generate the g gnit using a simple generative model and then calibrate
for layout synthesis. This is
correct potential inaccuracies (see Fig. (c)). The calibration procedure consists of two steps: calibrate the means {Mz My } of each Gaussian to its nearest valid values and recompute {o oT p). obtaining the calibrated Gaussian init The detailed calibration can be found Algorithm in the Appendix.
parameters Seal
Texture Generation. After synthesizing the overall structure of the image, we enrich the generated layout with detailed textures (see Fig. 3 using diffusion models such as SiT (23] Specifically, first convert innt into embedding vectors and incorporate them as additional inputs in each timestep. This ensures that the newly generated texture features accurately reflect the constraints of the original layout while preserving structural consistency and rich details in the final image. In practice, the model predicts Gaussian parameter residual Ag and texture features f. The Gaussian parameters are updated by gl + Ag, while f captures the specific texture characteristics of each token. The results can be rendered and reconstructed into natural images using our GPSToken decoder.
The two-stage generation pipeline significantly reduces the complexity of image generation by decoupling geometric modeling from texture synthesis. The gninst acts structural constraint, simplifying the texture generation task while ensuring spatial consistency. It aligns with the human perception process (from structure to detail), Additionally, since layout synthesis is much easier than texture generation, we employ a simple network or cached database to produce results in the first stage, introducing minimal additional computation compared existing methods.
4 Experiments
4.1 Experimental Settings
Training Data and Settings. We train all models on the ImageNet dataset (30, which contains 1.28M training images and 50K validation images. During preprocessing, images are resized to 256 x 256 and center-cropped without additional augmentation beyond horizontal flipping. We implement three variants of GPSToken: GPS4oken-S64 (64 tokens), GPSToken-MID (standard setting, used

---

Table 1: Comparisons of 256 256 reconstruction task Imagenet val set. The top methods trained only with ImageNer are highlighted in red, blue and green. Note that "SDXL-VAE" is trained with rich amount of additional data other than Imagenet.
Method
Tokens
Params (MD
sample-level
distribution-levl
PSNR f
SSIM LPIPS
FID
sFID
sFID
2D Tokenization
SDXL-VAE 2
32x32
83.6
25.55
0.727
0.066
0.73
2.42
2.35
3.89
GaussianToker
18
32x32
130.6
22.40
0.597
0.112
1.70
4.62
3.63
4.71
VQVAF-f16 [11]
16x16
89.6
19.41
0.476
0.191
8.01
9.64
10.74
7.38
MaskGIT-VAE 3
16x
54.5
18.11
0.427
0.202
3.79
5.81
5.19
4.56
VAVAE 35
16x16
69.8
25.76
0.742
0.050
0.27
1.72
1.74
3.91
DCAE
8x8
323.4
23.62
0.644
0.092
0.98
4.82
2.59
5.02
ID Tokenization
SoftVQ
64
173.6
21.93
0.568
0.115
0.92
4.52
2.51
4.21
TiTok-B64 [37
64
204.8
17.01
0.390
0.263
1.75
4.51
2.50
4.21
TiTok-S128 137
128
83.7
17.66
0.413
0.220
1.73
7.25
3.25
5.52
MAETok 5
128
173.9
23.25
0.626
0.096
0.65
3.87
2.01
4.39
FlexTok [1]
256
949.7
17.69
0.475
0.257
4.02
8.00
4.88
6.12
One-D-Piece 24
256
83.9
17.74
0.420
0.210
1.54
6.96
2.93
5.36
MaskBit
256
54.5
21.07
0.539
0.142
1.29
4.72
3.08
4.09
GPSToken
S64
64
127.5
22.18
0.578
0.111
1.31
5,42
3.02
4.85
M128
128
127.8
24.06
0.657
0.080
0.65
3.28
2.18
3.96
1,256
256
128.7
28.81
0.809
0.043
0.22
1.31
1.65
3.77
default), and UPSToken-LL256 (256 tokens). For details about the training/inferrnce and network architectures, please refer to Appendix.
Evaluation Metrics. We conduct all evaluations the validation set of ImageNet. For reconstruction, we evaluate performance using both sample-level and distribution- indices. At the sample level, we use PSNR, SSIM, and LPIPS [39], which measure the similarity between reconstructed and original images, as metries. At the distribution level, we adopt FID 13 and sFID 25 to assess the overall distribution of reconstructed images. Specifically, measure the distribution consistency between reconstructed and input images, while using standard "FID" and "sFID" to evaluate the alignment with natural image distributions. For image generation, we employ FID to assess generation quality.
report "rec. FID" and "ree. sFID" to
4.2 Image Representation
Comparison Results. We evaluate the representation performance of GPSToken using the image reconstruction task. We compare GPSToken with existing 1D and 2D tokenization methods at 256 256 resolution, including SDXL-VAF III Gaussian Token 19. VQVAE [I]. MaskGiT- VAE VAVAE (35), DCAE 77, TiToK 37 SoftVQ (6), FlexTok 0, One-D-Piece (24], and
MAETok 5 As shown in Table I. PPSToken-L2266 achieves significantly better performance than the competing methods across both sample-level and distribution-Ieve metrics, even better than SDXL-VAE, which utilizes more tokens (1024 256) and is trained with rich amount of additional data. Compared to SDXL-VAE, GPSToken- L226 improves PSNR by 3.26, SSIM by 0.082, and reduces LPIPS by 0.023. It also achieves "rec. FID" of 0.22, "rec. sFID" of 1.31, an FID of 1.65, and an sFID of 3.77, outperforming all competitors. Note that VAVAE [35] leverages vision foundation models align latent features, yet it still lags behind GPSToken-L2250
the other hand, GPSToken-Min outperforms the competing methods using the same number of tokens on most metrics, obtaining "rec. sFID" of 3. 3.28 and .PPSS of 0.080. It also outperforms many methods that use more tokens. With only 64 tokens, GPSToken-SO also demonstrates promising performance, achieving "rec. FID" score of 1.31, highlighting the scalability of our approach.

---

SOXL
VAVAE
GPSTaken
AT IO DINK
AT TO TAMMI
HALE DANT
HALP DAAAI
EXPITING CEATTRR
BREWING WING CONICO
BRGN NO COMPANY
10L WING COMPANT
builywiine
bur & wine
Coussion prenbe
OCAE
MoeTok
PPSToken 2
M Bana maska
H HAT BINF
FARA
HAFE TOTA
Yru bY 10
DULL & CAAZAN
DILLI VENT ine
bur borlyavin
Mosk
Sofvi
PLIL H
F
AA IT FRONT
A EI GTANG
STNDEE Du
SAUOnIC & Oronabi
LAWALL IS
DELI MONOII
hilukyzy ME
hion Wy WIGG
TITOK-RES R64
TiTok 5128
FlexTok
One Piace
CIL CUILD
1mmp
SIRO
Figure 4: Visual comparisons on 256 256 reconstruction task.
(c) refined g Figure 5: Illustration of Spatial Adaptivity (SA). Left to right: the input visualization of init visualization of refined g. the reconstruction & with SA, the reconstruction & SA without SA, error map of &, and error map of &wh SA (darker blue indicates larger errors).
(a) inp
(b) ght
(d) GPSToke
(c) GPSToken wio SA
(1) erros of(
(g) errons (f()
We provide visual comparisons among GPSToken and its competitors in Figs. It can be observed that our GPSToken achieves significantly more accurate and clearer textures in complex regions, without compromising the performance in simpler areas.
Effectiveness of Spatial Adaptivity. GPSToken possesses spatial adaptivity (SA), enabling a region adaptive image representation. As shown in Fig. 5 with our SA initialization, the g gnntt is placed according to the regional complexity. More Gaussians are assigned to complex regions such as the human body, while sparse Gaussians are used in simpler regions. Based on this initialization, the refined further adjusts its positions, shapes, and orientations to better align with local textures.
8

---

1L
9u
9u
(a) imp
(b) g
(c) GPSToken
(d) User udjusted g
(c) GPSTokn wr adjusted g
Figure 6:
Adjustment of g By manually setting g init our GPSToken can
focus more on semantically important regions (e.g. text and faces) and achieve finer reconstruction.
psar: 19.26 ssim:0.600
psnr: 22.64 ssim:0.737
psnr: 26.03 ssim:0.834
psnr: 27.10 sxim:0.859
tokens
tokens 128 tokens (default)
tokens
Figure Adjustable Token Count at Inference. Token count can adjusted at inference for better quality-efficiency trade-off, even beyond default training setting (128 tokens).
can be clearly observed that with SA enabled, the maps exhibit significantly lower errors in complex regions (Figs.B3f-g) while the errors in simple regions (e.g., background) remain largely unchanged. This demonstrates that our GPSToken improves the representation in complex areas without compromising the quality in simpler ones.
Controllllll Adjustment of g init Additionally, GPSToken supports manual adjustment of gemit, allowing users to prioritize semantically important regions (e.g., faces text). An example is shown in Fig. where placing denser Gaussians in target areas results in clearer reconstructions.
Adjustable Token Count at Inference. GPSToken supports adjustable token count at inference (see Fig. 7). Unlike II 24], which only supports decreasing the number of tokens inference, GPSToken can also increase the number of tokens for improved quality.
Please refer to Appendix for more results of GPSToken on generalization performance, efficiency, ablation studies, and visualizations.
4.3 Image Generation
Comparison Results. We compare our approach against state-of-the-arn tokenizers for image generation, including MaskGIT 3, TYTok (37. FlexTok M1 SoftVQ (6]. ADM (8), One-D-Piece (24), DiT (26). SiT (23). REPA 38). D²iT 16), and MAETok 5 on 256 x 256 class-conditional
generation tasks. Quantitative results with classifier- frre guidance 14 are reported Tab. Our two-stage generator with 128 tokens outperforms all competing methods, achieving an FID of 1.50, highlighting the effectiveness GPSToken in providing a superior latent space for generative models, even with fewer tokens. With equal number of tokens (128), MAETok (5] under-performs our GPSToken-based generator, suggesting that our Gaussian-pares parasmte--prmmmeeriiieed tokenization offers distinct advantages. In contrast, the one-stage generator slightly under- perforrss the baseline. This discrepancy arises from the optimization challenges inherent the composition of Gaussian parameters g and textual features f within single token Our two-stage design (first generate g then synthesize f)), effectively addresses this issue, leading to significant enhancement over the baseline.
Faster Training. As illustrated in Fig. 8 our two-stage generator demonstrates significantly acceler- ated convergence compared to both the baseline and stesttage generator. Specifically, it achieves an FID-10K score of 25.48 within 100K iterations. In contrast, the SiT-XL/2 and one-stage generator reach scores of 25.41 and 26.20 after 300K and 500K iterations, respectively, indicating that our method is approximately 3x and 5x faster than them. This notable speed-up highlights the effec- tiveness of shape-texture decomposition simplifying the optimization process. More results on training efficiency can be found in Appendix.

---

Table 2: Comparisons 256 x 256 class-conditional image generation. The top methods
highlighted in red and blue. "." indicates the baseline.
Tokenizer
Generator
Method
Params (M)
Tokens
Params (M)
FID
Auto-regressive Models
MaskGIT [3
54.5
16x 16
227
6.18
FlexTok [1]
256
1,330
2.02
TiTok-S128
83.7
128
287
1.97
TiTok-Bo6 -664 [37
204.8
64
177
2.77
SoftVQ 6
173.6
64
675
1.78
Diffusion-based Models
ADM 8
23.24
3.94
24
83.9
256
2.35
DiT-XL/2 26
83.6
32x32
675
2.27
SIT-XL/2
83.6
32x32
675
2.06
REPA (38
83.6
32x32
675
1.79
D [16
>256
687
1.73
MAETok
173.9
128
675
1.67
Ours (one-stage)
127.8
128
675
2.13
Ours (two-stage)
127.8
128
33+675
1.50
SIXXL2 (hsoeline)
two-stege ceneratse ose stage generator
[ID)1K(())
5x
10
0
Training (x10'))
predicted g init
calibrated petr
g
synthesized image
Figure FID-10K training curves.
Figure Illustration of two-stage generation pipeline.
Qualitative Analysis of Generation Process. Fig. provides visual breakdown of the generation pipeline. Initially, g" captures a coarse structure but may include incomplete or misaligned regions in Gaussian maps. Our calibration algorithm addresses these issues and refines iini into semantically coherent and spatially accurate layout gindt Leveraging this calibrated layout, the second stage generates Gaussian parameters g that encode local texture orientation and scale. Consequently, the final image not only retains the global structure established by g" init but also achieves rich details, exemplified by the synthesized dog/rabbit images. More results can be found in Appendíx.
Conclusion
this paper, we introduced GPSToken, a spatially-adaptive image tokenization approach for effective image representation and generation. Unlike conventional grid-based 2D/1D tokenizers, GPSToken leveraged parametric 2D Gaussian distributions to model image content a non-unifee non-uniform and content- aware manner. Our method achieved strong performance using only 128 tokens per image, yielding rFID and FID scores of 0.65 and 1.50 on image reconstruction and generation tasks, respectively. By decoupling spatial layout from texture features, GPSToken enabled two-stage generation pipeline that supports flexible control over both structural and appearance attributes.
Limitations. While our approach has shown promising results, its heuristic initialization of Gaussian parameters may not always ensure optimal configurations. Future work could explore learning- based initialization to address this limitation. Additionally, designing specialized architecture for predicting Gaussian parameters in generative tasks could improve layout synthesis and potentially climinate the need for processing calibration, thereby enhancing overall performance.
10

---

References
[1] Roman Bachmann, Jesse Allardice, David Mizrahi, Enrico Fini, Oguzhan Fatih Kar, Elmira Amirloo, Alaaeldin El-Nouby, Amir Zamir, and Afshin Dehghan. Flextok: Resampling images into Id token sequences of flexible length. arXiv preprint Sadv:2552..39663 2025.
[2] Mattia Balestra, Marina Paolanti, and Roberto Pierdicca. Whu-rs19 abzsl: An attribute-based dataset for remote sensing image understanding. Remote Sensing, 17(14), 2025.
[3] Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11315-11333 11325, 2022.
[4] Du Chen, Liyi Chen, Zhengqiang Zhang, and Lei Zhang. Generalized and efficient 2d gaussian splatting for arbitrary-scall super-resolution. arXiv preprint arXiv:2501.6633 2025.
[5] Hao Chen, Yujin Han, Fangyi Chen, Xiang Li, Yidong Wang, Jindong Wang. Ze Wang. Zicheng Liu, Difan Zou, and Bhiksha Raj. Masked autoencoders are effective tokenizers for diffusion models. arXiv preprint ar arXiv:2502.0m 2544:2500203444 2025.
[6] Hao Chen, Ze Wang, Xiang Li, Ximeng Sun, Fangyi Chen, Jiang Liu, Jindong Wang, Bhiksha Raj. Zicheng Liu, and Emad Barsoum. voftvq-vae: Efficient 1-dimensional continuous tokenizer. arXiv preprint arXiv:2412. 10958, 2024.
[7] Junyu Chen, Han Cai, Junsong Chen, Enze Xie, Shang Yang, Haotian Tang, Muyang Li, Yao Lu, and Song Han. Deep compression autoencoder for efficient high-resolution diffusion models. arXiv preprint arXiv:2410. 10733, 2024.
[8] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-8794, 2021.
[9] Jiajun Dong, Chengkun Wang, Wenzhao Zheng, Lei Chen, Jiwen Lu, and Yansong Tang. Gaussiantoken: An effective image tokenizer with 2d gaussian splatting. arXiv preprint WrXi:22511115699 2025.
[10] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transform- ers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024.
[11] Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 12873-12883, 2021.
[12] Kaiming He, Georgia Gkioxari, Piotr Dollár, and Ross Girshick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961-2969, 2017.
[13] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge local nash equilibrium. Advances in neural information processing systems, 30, 2017.
[14] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv:2207.12598 2022.
arXiv preprint
[15] A. Hoover and M. Goldbaum. STructured Analysis of the Retina (STARE) Project //ww.cces clemson. edu/"ahoover" sstare July 2003.
http
[16] Weinan Jia. Mengqi Huang. Nan Chen, Lei Zhang, and Zhendong Mao. D2it: Dynamic diffusion transformer for accurate image generation. arXiv preprint arXiv:2504.0945. 2025.
[17] Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pages 4401-4410, 2019.

---

[18] Bernhard Kerbl, Georgios Kopanas. Thomas Leimkühler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 12 2023.
[19] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[20] Diederik Kingma, Max Welling, al. Auto-encoding variational bayes, 2013.
[21] Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1 11523-11532 2022.
[22] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740-755. Springer, 2014.
[23] Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijndent and Saining Xie. Sit: Exploring flow and diffusion-basee generative models with scalable interpolant transformers. In European Conference Computer Vision, pages 23-40. Springer, 2024.
[24] Keita Miwa, Kento Sasaki, Hidehisa Arai, Tsubasa Takahashi, and Yu Yamaguchi. One-d-piece: Image tokenizer meets controllle compression. arXiv preprint arXiv:2501.1006. 2025.
[25] Charlie Nash, Jacob Menick, Sander Dieleman, and Peter W Battaglia. Generating images with sparse representations. arXiv preprint 200.::20003038441 2021.
[26] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195-4205, 2023.
[27] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint (:Xxx:2307.01952 2023.
[28] Ali Razavi, Aaron Van den Oord, and Oriol Vinyals. Generating diverse with va-vae-2. Advances neural information processing systems, 32, 2019.
images
[29] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[30] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International journal of computer vision, 115:211-2533 2015.
journal
[31] Irwin Sobel and Gary Feldman. A 3x3 isotropic gradient operator for image processing. Pattern Classification and Scene Analysis, pages 271-272, 01 1973.
[32] Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. Advances in neural information processing systems, 37:84839-84865, 2024.
[33] Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
[34] Mark Weber, Lijun Yu, Qihang Yu, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang- Chieh Chen. Maskbit: Embedding- free image generation via bit tokens. arXiv preprint arXiv:2409. 16211, 2024.
[35] Jingfeng Yao, Bin Yang, and Xinggang Wang. Reconstruction vs, generation: Taming optimiza- tion dilemma in latent diffusion models. arXiv preprint arx.v:550..01423, 2025.
[36] Jingfeng Yao, Bin Yang, and Xinggang Wang. Reconstmuction vs. generation: Taming optimiza- tion dilemma in latent diffusion models. arXiv preprint arXiv:2501.01423; 2025.
12

---

[37] Qihang Yu, Mark Weber, Xueqing Deng, Xiaohui Shen, Daniel Cremers, and Liang Chich Chen. An image worth 32 tokens for reconstruction and generation. Advances in Neural Information Processing Systems, 37:128900-12866, 2024.
[38] Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint 24X1v:2410006940, 2024.
[39] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586-595, 2018.
[40] Zhengqiang ZHANG, Ruihuang Li, and Lei Zhang. Frecas: Efficient higher-resolution image generation via frequency-awan cascaded sampling, In The Thirteenth International Conference on Learning Representations.
13

---

Appendix
In this appendix, we provide the following materials:
A Spatially-adaptive token initialization and Gaussian calibration algorithms (referring to Sec. 3.2 and Sec. 3.3 of the main paper);
Experimental settings for reconstruction and generation (referring to Sec. 4.1 of the main paper);
Generalization on higher resolutions and other datasets (referring to Sec. 4.2 of the main paper);
D
FLOPs, memory and latency in reconstruction (referring to Sec. 4.2 of the main paper);
Ablation studies architecture designs and parameters (referring to Sec. 4.2 of the main paper);
Training and inference efficiency of GPSToken generators (referring Sec. 4.3 of the main
paper);
t
More visual comparisons and results reconstruction and generative tasks (referring to Sec. 4.2 and Sec. 4.3 of the main paper);
H Broader impacts of GPSToken and its generator.
A
Spatially-adaptive Token Initialization and Gaussian Calibration
The ppatially-adapiie token initialization algorithm is described in Sec. 3.1 of the main paper. It outlines a procedure for segmenting the entire image based on regional complexity and for initializing the Gaussian parameters of the GPS-tokens accordingly. The algorithm is summarized in Algorithm
Algorithm 1: Spatially- Token Initialization Algorithm
Input: image I; target token count I; metric hyper-paramette X; minimal size of region 8msn- Output: regions list L.; initialized Gaussian parameters {gimst .... : Initialize region candidate list L = (7):
iint
-6-1)
2 while L| < do
Calculate complexities {mo.. mmjri-) for regions in L using Eq. 4 in the main paper;
Let i. = {T: E I. at least one side I; is greater than Smin]; Let Tmax arg m m4;
Obtain size (w, h) of Imaxi if w * h then
Equally divide Jmux into two sub-regions Jı and Tz along the longer side; Update I. by replacing Taux with (7,, 12];
10
else
// stepl: width-wise division compute complexities Divide Imax into I1, In along width:
12
13
Calculate complexities m1, my for I,, In using Eq. in the main paper; // step2: hidth-wise division -> compute complexities
14
15
Divide Imax into I3. along height:
16
Calculate complexities m3, for Tz. Ta using Eq. 4 in the main paper; // step3: compare the complexities if min(m1, m2) & min(m3, ma) then
17
18
19
Update by replacing 1max with {11, I2];
20
else
21
Update by replacing Imux with {13, 14];
22
end
23
end
24
end
Obtain size (wi, h4) and center (Ii, y:) for each region 4 E (6) Initialize gnit (o ,y p()), pr py (8)
{ mi
hi
0, v.
14

---

The Gaussian calibration algorithm is described in Sec. 3.2 of the main paper. It is used to refine the predicted Gaussians during the layout synthesis phase. The algorithm is summarized Algorithm
Algorithm 2: Gaussian Calibration Algorithm
Input: predicted Gaussian parameters {go: -1-1 J: minimal size of region Smini image size W x H.
Output: calibrated Gaussian parameters {85al B1-13-
// step 1: calibrate the means (142) (yy
2 Let Ar = Ay = Smin'
Define grid G as all points in [0, W WW-1] H-1] spaced by (Ax, Ag):
(2) (0) g: to the nearest central point in G, obtaining (AE" pg)); Quantize each mean (Az My
5 // step 2: re-initialize sigmas via region partitioning
Initialize region list L = {00, W, 0, H));
7 while L| < do
Let m, be the count of quantized means in region I L;
Let L = {44 E L at least one side of 1; is greater than smin]:
10
Let 1mnx = arg mäx m4;
Let xe, x2, 3/,,,,, 22 +- max and (w, h) 1r2-x1,y2-u/)
11
if ④ h then
12
13
Replace max in L with its two equal sub-regions split vertically along the x-axis; elif w < h then Replace Imxx in L with its two equal sub-regions split horizontally along the y-axis;
15
16
else
17
if no Gaussian falls exactly on the split line x = (α1 + x2)//2 then
18
Replace mmux in L. with its two equal sub-regions split vertically along the x-axis;
19
else
20
Replace Imux in L with its two equal sub-regions split horizontally along the y-axis; end
21
22
end
23
end
Obtain size (wi, h4) and center (Iz; yi) for each region 7; E T;
25
Compute goal foz ,
cal (6) eat-(i) (i) pfal-(1), (f),
Me My
ma)-(1) nat oal-(1)) (1)
w. h 6o 6
0, 0,
}.
B Experimental Settings
Training and Inference Settings. For the image reconstruction task, train the encoder-decoder framework for IM steps with a batch size 96. The model first trained using only the reconstruction loss Lise for the initial 600K steps. Subsequently, the perceptual loss Lpere and adversarial loss Ladv 11 are incorporated for the remaining 400K steps to enhance texture details. We use the Adam optimizer [19 with fixed learning rate of 5 x 10-5. Additionally, we apply an exponential moving average (EMA) with a decay rate of 0.9999 to stabilize the training process. We set 5 for Eq. (in the main paper), À= 2.5 for Eq. 4 (in the main paper) and Smin 4 for Algorithm
For the image generation task, we adopt the velocity matching loss from SiT 23 and train the layout and conditional texture generators sequentially. Specifically, the layout synthesis model is trained for 500K iterations, and the conditional layout-to-texture generation model for 4M iterations. To mitigate overfitting to the conditions, we add 0.5 Gaussian noise to the condition during training of the conditional texture generator. Both models are trained with a batch size of 256 and a learning rate conducted
of 1 10 using the Adam optimizer. All experiments inference, we use 5tep ODE sampler 23 to predict giinit followed by 250-step SDE sampler 23) used in SiT (23) for texture synthesis. We set the classifier-fre guidance strength [14] to 1.5, following common practice. We set s 8min = 4 for Algorithm
eight A100 GPUs. During
Network Architecture. In the image reconstruction lask, the encoder architecture comprises two residual blocks for extracting image features, followed by 30 transformation blocks designed to process initial Gaussian parameters and extract textual features for each region. During the rendering
15

---

Table 3: Comparisons of 512 x 512 and 1024 1024 reconstruction task Imagenet val set.
sample-level
distribution-level
Method
Tokens
PSNR
SSIM
LPISS
FID
sFID
512 x 512
SDXL-VAE 27
64x64
28.42
0.817
0.059
0.271
1.36
VQVAE-f16 [
32x32
21.83
0.604
0.172
2.29
7.95
GPSToken- M128
512
26.74
0.764
0.073
0.367
1.93
GPSToken-L256
1024
32.00
0.887
0.039
0.175
0.699
1024
1024
SDXL-VAE 227
128x 128
33.27
0.909
0.057
0.113
0.561
VQVAE-F16 [1]
64x64
25.41
0.744
0.169
1,40
4.98
GPSToken-M
2048
31.22
0.873
0.072
0.236
1.24
GPSToken-L256
4096
37.71
0.955
0.031
0.055
0.276
stage, GPS-tokens are mapped into 64 64 2D feature maps. The decoder adopts the same architecture as the last three stages of the SDXL-VAE VAE (77 decoder but with double channels. JPPSToke-MM12 utilizes 128 tokens, each with 16 channels, whereas GPSToken-L256 employs 256 tokens, each with 32 channels, match the capacity of VAVAE [36]. GPSToken-S64 uses only 64 tokens, each with 16 channels, but incorporates 60 transformation blocks within the encoder.
For the image generation task, adopt the official SiT-S and SiT-XL architectures 23 backbone models. Specifically, SiT-S utilized for layout synthesis, while SiT-XL is employed for conditional layout-to-tex texture generation. To support layout conditions, use MLPs transform these conditions before adding them into each attention block in SiT-XL.
C Generalization of GPSToken on Higher Resolutions and Other Datasets
Higher Resolution. We evaluate pre-trained GPSToken-M128 and GPSToken-L222 - originally trained 256 256 images with 128 and 256 tokens, respectively on 512 x 512 and 1024 x 1024 images from the ImageNet validation set. We compare with SDXL-VAE 27 and VQVAE-f16 [I]. two public VAEs supporting reconstruction beyond training resolution. Following their practice, scale the token count quadratically with resolution. For example, we use 512 and 2048 tokens for 512 x 512 and 1024 1024 inputs, respectively, when using GPSToken-M128.
As shown in Table GPSToken shows strong generalization performance on resolution. Specifically, L2S50 achieves 32.00/0.885 (PSNR/SSIM) at 512 x 512 resolution and 37.7 17.71/0.955 at 1024 x 1024 resolution, outperforming SDXL VAE in both settings. On the other hand, all methods show higher reconstruction performance higher resolutions. GPSToken-M125 achieves better "rec. FID" of 0.236 at 1024 x 1024 than that (0.367) at 512 x 512. This is because higher resolutions provide more pixels for the same content, increasing local redundancy and structural consistency. The denser pixel sampling makes fine details easier to recover, simplifying the reconstruction task despite the larger input size.
More Datasets. We further evaluate on additional datasets: COC02017 22 (natural images), FFHQ [17] (faces), STARE [15] (medical images), and WHU_RS 2 (remote sensing). We
compare GPSToken with VAVAE (36) (256 tokens, 2D) and MAETok 5 (128 tokens, ID), repre- senting the state of the art in 2D and ID tokenization, respectively. As shown in Table GPSToken consistently outperforms both methods across all metrics and datasets under the same token counts.
Specifically, GPSToken-L222 L.256 achieves higher PSNR than VAVAE (256 tokens): 27.41 vs. 25.01 on COCO, 30.02 vs. 28.06 on FFHQ 37.60 vs. 36.32 on STARE, and 26.33 vs. 23.57 on WHU RS19. GPSToken-Mi M128 also yields better LPIPS than MAETok (128 tokens): 0.083 vs. 0.101 on COCO, 0.050 vs. 0.064 on FFHQ, 0.036 vs. 0.051 on STARE, and 127 vs. 0.5 on WHU_RS19 These results show that GPSToken performs well not only on natural images but also on other domains including medical and remote sensing - demonstrating strong generalization, robustness, and versatility for wide range of vision tasks.
16

---

Table 4: Comparison of 256 256 image reconstruction on COCO, FFHQ. STARE, and WHU RS19. For STARE and WHU RS19, we only report PSNR, SSIM, and LPIPS, which are more appropriate for evaluating reconstruction quality on non-photorealistic images than metrics such as "rec. FID"
Token Count
Method
PSNR
SSIM
LPIPS +
rec. FID 4
COCO
MAETok
22.67
0.623
0.101
8.91
128
GPSToken-M128
23.47
0.657
0.083
4.72
VAVAE
25.01
0.736
0.052
6.01
256
GPSToken-L255
27.41
0.794
0.035
2.23
FFHQ
MAETok
25.53
0.707
0.064
4.66
128
GPSToken-M128
26.35
0.745
0,050
3.72
VAVAE
28.06
0.808
0.027
1.95
256
GPSToken-L256
30.02
0.846
0.019
1.51
STARE
MAETok
32.98
0.818
0.051
128
GPSToken-M128
34.75
0.868
0.036
VAVAE
36.32
0.896
0.019
256
GPSToken-L256
37.60
0.915
0.014
WHU WHU_RS19
MAETok
21.73
0.506
0.195
128
GPSToken-M128
23.20
0.560
0.127
VAVAE
23.57
0.619
0.142
256
GPS5oken-L256
26.33
0.731
0.064
Table 5: Comparison of computational cost, memory usage, latency, and throughput for generating 256 x 256 images on an A100 GPU. Batch size for inference and 16 for training. Latency is averaged over 20 runs. Throughput is measured in samples per second. Training memory for FlexTok unavailable due to the lack of released training code.
FLOPs (G)
Memory (MB)
Latency (ms)
Throughput (sample/s)
Method
Encoder
Decoder
Train
Inference
VQVAE-f16
556
1014
78222
2627
72
111.44
TiTok-B64
220
973
45892
2359
96
83.44
BPSTooke-MM12
383
2689
50793
2567
180
44.56
Gaussian'Token
1706
2285
60781
5352
181
44.32
FlexTok
283
7665
7275
2714
2.88
D FLOPs, Memory and Latency for Reconstruction Task
provide profiling details on FLOPs, memory usage, latency, and throughput for the 256 256 image reconstruction task. As shown in Table 5] GPSToken incurs moderate computational cost. Although the FLOPS of GPSToken decoder are higher than those VQVAE-f16 33 and TYTok- B64 [37], its latency is competitive with GaussianToken ( and significantly better than FlexTok []] which employs a heavy autoregressive decoder. In terms of memory, GPSToken uses less GPU memory than Flex and GaussianToker during inference, and less training memory than VQVAE- f16, demonstrating favorable memory efficiency both phases.
E
Ablation Studies on Spatial Adaptivity Designs
Alation Studies on Components. described See. 3.2 of the main paper, our GPSToken em- ploys paatiall-addaptic token initialization ("Init.") followed by apatially-ddaptiv token refinement
17

---

Table 6: Ablation studies of our spatial adaptivity designs the 256 x 256 reconstruction task. indicates that the component is used. "Init." " and "Refine"" denote the spatially-adaptive token initialization and apatially-adaptiin token refinement, respectively.
Components
sample-level
distribution-leve
Method
Init. Refine.
PSNR
SSIM f LPIPS
FID
sFID
sFID
baseline
23.52 0.638
0.110
1.02
4.07
2.59
4.34
baseline+
24.00
0.654
0.100
0.81
3.59
2.37
4.31
GPSToken
24.06
0.657
0.080
0.65
3.28
2.18
3.96
input
g baseline+
baseline+
g of GPSToken
GPSToken
Figure 10: Illustration of "baseline+* and GPSToken. Left to right: the input image, visualization of
Gaussians of "baseline*" " the reconstruction of "baseline+", visualization Gaussians GPSToken, the reconstruction of GSPToken.
("Refine.") to progressively obtain coarse- and fine-grained spatial adaptations. We conduct ablation studies on GPSToken-M128 to validate the contribution of each component.
Table presents the quantitative results. The baseline refers the method that uses Gaussian- parameterized tokens without incorporating any spatially-adaptive components. The term "baseline+" denotes the method that additionally includes the "Refine." component. In contrast, GPSToken integrates both the "Init." and "Refine." components. As shown the table, "baseline+" yields improvements over the baseline across both sample-level and distribution-levt metries, with decrease of 0.01 in LPIPS and decrease of 0.21 in "rec. FID". These enhancements indicate the general improvement achieved by adjusting Gaussians to match local textures. Compared to metrics, achieving reductions of 0.19
"baseline+", GoSToken significantly improves distribution-les in FID and 0.35 in sFID, while showing slight improvements in sample-level metrics (an increase of 0.06 in PSNR and 0,003 in SSIM). This demonstrates the effectiveness of "Init." component, which reallocates more Gaussians from simple regions to complex ones, thereby capturing finer semantic details in texture-rich regions without compromising reconstruction performance.
As shown in Fig. TO without the "Init." component, the Gaussian maps from "baseline+" still roughly align in a 2D grid, even after refinement. This limits their ability to fit complex textures, only capturing edges in simple regions. In contrast, with the "Init." component, GPSToken aggregates more Gaussians in texture-rich areas, making it better suited to fit complex structures. This highlights the importance of "Init." component in achieving spatially adaptive representation of fine-grained visual contents.
Ablation Studies on Params. We further conduct experiments the selection of parameters X, and Smin- The results are shown in Table
Entropy Threshold X: As stated in Eq. 4 of the main paper, À balances region size and complexity. A larger X = 5 encourages Gaussians to concentrate on complex regions, leading to only minor performance degradation.
contrast, setting allocates Gaussians solely based on region size, resulting in a uniform spatial distribution. This causes significant drop in performance:
18

---

Table 7:
selection on 8, and Smin-
Hypeee-arameter
PSNR
SSIM
LPIPS
roc.
FID
23.52
0.638
0.110
1.02
2.59
X
24.06
0.652
0.083
0.68
2.23
17.55
0.439
0.344
160
165
24.07
0.657
0.080
0.66
2.18
7
24.06
0.658
0.080
0.66
2.16
24.05
0.656
0.080
0.66
2.18
Smin
16
24.03
0.657
0.081
0.66
2.19
ours (A 2.5, 5, Smin
4)
24.06
0.657
0.080
0.65
2.18
Table 8: Training and Inference Efficiency Comparison between SiT-X1/2 Baseline and GPSToken generator on ImageNet 256 256 with A100 GPU.
Method
Metric
500K
1000K
T-Mem
T-Thpt
I-Mem
I-Thpt
FID
19.07
14.50
Baseline
63684
0.63
9126
0.067
Time (h)
219
439
FID
9.57
7.61
Ours
41498
1.09
9636
0.129
Time (h)
128
256
Notes:
T-Mem: Training Memory (MB), T-Thpt: Training Throughput (iters/s),
I-Mem: Inference Memory (MB), I-Thpt: Inference Throughput (samples/s)
LPIPS increases from 0.08 to 0.11, and Fec rises from 0.65 to 1.02. We set À to 2.5 based on experimental experience.
Support Factor s: As stated in Eq. 2 of the main paper, controls the effective rendering support of each Gaussian. Performance degrades significantly when s but stabilizes for 2 This aligns with the 30 rule, i.e., 99.7% of the mass of a 2D Gaussian lies within three standard deviations from the mean. ensure full coverage, we set 5 in all experiments.
Minimal Region Size 8min: We set Smin in the calibration algorithm to match its value in the initialization algorithm, where 8man determines the minimum width or height of each region. We observe that increasing Smin from 4 to 16 results in negligible performance degradation. This is expected because, with 128 tokens representing 256x 256 image, the average spatial extent per token is approximately x2 pixels. Consequently, most of the segmented regions naturally have a width or height greater than or equal to 16, making the choice of 8min within this range largely inconsequential for the final tokenization.
F
Training/Inferrnce Efficiency of GPSToken Generators
We provide comprehensive computational benchmarks comparing the GPSToken generator with the SiT-XI/2 baseline in both training and inference, as shown in Table g At 1M iterations, GPSToken achieves a significantly lower FID score (7.61 vs. 14.50), with 42% less training time (256h vs. 439h), 73% higher training throughput (1.09 vs. 0.63 iters/s), and 35% lower VRAM consumption (41,498 MB vs. 63,684 MB). During inference, although VRAM usage increases slightly (9,636 MB vs. 9,126 MB), method nearly doubles the throughput 10.129 vs. 0.067 samples/s), reducing latency by approximately half.
These efficiency gains stem from two key design choices: (i) GPSToken reduces the number of effec- tive tokens, lowering computational and memory overhead: (ii) the two-stage generation framework simplifies the learning objective and stabilizes optimization, enabling faster convergence higher- quality solutions. Overall, GPSToken not only improves generation quality but also substantially reduces training cost and inference latency.
19

---

G More Visual Results
G.1 Results for Reconstruction Task
Visual Comparisons. We provide visual comparisons among GPSToken and its competitors in Figs.
1 and 12 can be observed that our GPSToken achieves significantly more accurate and clearer textures in complex regions, without compromising the performance in simpler areas.
SDXL-VAL -VAE
VAVAE
@PSToken-L255
GT
Gaussion Token
DCAE
MoeTok
GPSToken-MM M122
MoskGIT-VAE
MoskBiT
SoftvO
GPSToken-
TiTok-B64
liTok-S128
FlexTok
One-D-Pae Piece
Figure 11: Visual comparisons on 256 256 reconstruction task.
More Visual Results. Further visual results of our spatially adaptive designs are presented in Fig. 13 Fig. 14 illustrates the adjustment of the initial Gaussian parameters g-iiit to better focus on the regions of interest. Fig. shows the flexibility to adjust token counts during inference, demonstrating the adaptability of our approach under varying length.
G.2 Results for Generation Task
Fig. 16 shows a few images generated by our two-stage generator. One can see that our generator capable of synthesizing natural images depicting a wide variety of scenes. For instance, it successfully generates fine details objects such beetles, eagles, trucks, bags, mailboxes, golf balls, dinosaur fossils, and Furthermore, the generated images exhibit high visual quality characterized by
20

---

SDXL
VAVAE
GPSToken L256
Coussion oken
DCAE
MoeTok
GPSToken 28
MaskCil VAE
Mosk
SoftVC
GPSToken S64
TiTok 864
TiTok- $128
FlexTok
One-D One-D-Piech Piece
8
Figure 12: Visual comparisons on 256 256 reconstruction task.
sharp details and realistic textures demonstrating the generator's strong ability to synthesize diverse and photo-realistic images.
21

---

YGGSS
ببب
init
refined g
GPSToken
GT
Figure 13: More visual results of spatial adaptivity.
22

---

All a-quiver
ups Bueer
All as-unisee
(4) iup
tnCr
(c) C-PSTakmn
User adjusted git
(C) GPS anjusned gitst
Figure 14: More visual results on User- Coltroonteo Adjustment of init.
32 tokens
64 tokens
128 tokens (default)
256 tokens
GT
Figure 15: More visual results on Adjustment Token Count Inference.
23

---

LOUNER GETUC
OL
VELUI PPC
Figure 16: Visual result of 256 x 256 generation.
24

---

H
Broader Impacts
This work introduces GPSToken, spatially-adaptiie tokenization framework designed enable efficient and content-aware image representation. By offering flexible feature modeling, GPSToken enhances representational capacity, benefiting both computer vision researchers and downstream applications in domains such as medical imaging and creative design. Furthermore, its two-stage layout-texture synthesis approach reduces computational barriers for generative tasks, making accessible to individual users and small companies.
Despite its potential, the deployment of GPSToken also presents several risks. The ability to gen- erate high-quality synthetic media may be misused, potentially harming vulnerable populations through the spread of misinformation or deepfake technologies. Additionally, if trained biased datasets, the model may encode disparities in texture and shape representation, which could com- promise fairness particularly sensitive applications such as facial recognition. In safety-critical domains like autonomous driving or medical diagnosis, failures in accurate tokenization could lead to misinterpretation of complex visual scenes, with potentially dangerous consequences.
25

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: For example. in the "Experiments" section, we evaluate our method on both reconstruction and generation tasks to comprehensively validate all claims stated in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations in "Conclusion" section.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-texx system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
26

---

Answer:
[NA]
Justification:
We do not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide detailed experimental settings in "Experiments" section.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access hosted model (e.g., the case of a large language model), releasing of model checkpoint, or other means that are appropriate to the research performed.
Depending on the contribution, reproducibility
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c)
If the contribution is new model (e.g., large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with open-source dataset instructions for how construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
27

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
We will release the codes and models.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is acceptable answer. Papers cannot be rejected simply for not including code, unless this central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
//nipscc/ppie
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized ver- sions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification:
We provide detailed experimental settings "Experiments" section.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core of the paper to a level of detail that necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We follow same experimental settings and content of previous works, which do not include such experiments. We will take this into consideration in future works.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, statistical significance tests, least for the experiments that support the main
claims of the paper.
The factors of variability that the emor bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
28

---

The method for calculating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g,, Normally distributed errors). of errors is not verified.
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
It is OK to report 1- I-sigma error bars, but one should state it. The authors should preferably report 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide detailed settings in "Experiments" section, including the type of
GPUs.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't t make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We follow the guidelines.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no societal impact of the work performed.
Guidelines:
The answer NA means that there is societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
29

---

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairnsss considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications. let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mecha- nisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question: Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [No]
Justification: We follow previous works, which do not include such safeguards. The safeguards valuable. We will take this into consideration in future works.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We cited all the origin papers in the paper.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include a URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset.
For scraped data from particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
30

---

For existing datasets that are packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We will release the detailed document of codes and models.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
31

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy thttps://neurips cc/Conferences/ http:///aarrrppscccooooeereeeeee 2025/LL2 for what should should not be described.
32


Reviews:
[{"review_id": "LiPDRl9SlE", "reviewer": "Reviewer_EBwp", "summary": "The paper proposes GPSToken, a way to break an image into variable-sized “Gaussian” tokens instead of a fixed grid. Each token stores its position/shape (as a 2-D Gaussian) and a texture feature. A differentiable renderer turns these tokens back into an image-like map so standard networks can use them. For image generation, the authors first predict a coarse layout of Gaussians and then fill in texture with a diffusion model, achieving better FID scores on ImageNet-256 than previous tokenization methods while converging faster.", "questions": "- The tokenizer alone has >120 M parameters, and splatting requires per-pixel Gaussian evaluation. Please report (i) encoder/decoder FLOPs, (ii) GPU memory during training and inference, and (iii) wall-clock throughput compared with VQGAN and TiTok at equal image size. Demonstrating competitive efficiency would strengthen the practical impact.\n\n- Can GPSToken scale to 512×512 or 1024×1024 without exploding token counts? A small experiment or theoretical discussion on how σ scales with resolution—and whether the same model can tokenize non-natural images (e.g., medical, satellite)—would clarify robustness.", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Paper Strengths**\n- Model is large (≈128 M tokenizer + 675 M generator) and still needs 64–256 tokens, so actual memory/flops vs standard VQ/VAE remain unclear.\n- Many hyper-parameters (entropy threshold λ, support factor s, calibration algorithm) are given without sensitivity study.\n- Strong quantitative gains over many recent 1-D/2-D tokenizers on ImageNet reconstruction & generation.\n- Paper is well organized; code release promised.\n\n**Weaknesses**\n- Lacks wall-clock latency / memory benchmarks and statistical significance (no error bars).\n- Speed-ups are only theoretical; pooling overhead & two extra mat-muls may erode gains for small images—no profiling provide", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank this reviewer for the constructive comments and suggestion. We hope our following point-to-point responses can address this reviewer's concerns.\n\n---\n\n**Q1: Params, FLOPs, memory, latency and throughput.**\n\nIn terms of model size, our GPSToken tokenizer contains approximately 128M parameters, which is comparable to GaussianToken (130M) and larger than VQVAE-f16 (89M). Meanwhile, TiTok-B64 has 205M parameters and FlexTok has 950M. Overall, GPSToken is highly competitive in model scale.\n\nAs shown in the following table, GPSToken has moderate computational cost. While the FLOPs of its decoder are higher than those of VQVAE-f16 and TiTok-B64, its latency is competitive with GaussianToken and significantly better than FlexTok, which employs a heavy autoregressive decoder. In terms of memory, GPSToken uses less GPU memory than FlexTok and GaussianToken during inference, and less memory than VQVAE-f16 during training, exhibiting favorable memory efficiency in both phases.\n\n| Method | Params (M) | FLOPs (G) Encoder | FLOPs (G) Decoder | Memory (MB) Train | Memory (MB) Inference | Latency (ms) | Throughput (sample/s) |\n| - | - | - | - | - | - | - | - |\n| VQVAE-f16 | 89.6 | 556 | 1014 | 78222 | 2627 | 72$\\pm$0.05 | 111.44 |\n| TiTok-B64 | 204.8 | 220 | 973 | 45892 | 2359 | 96$\\pm$0.08 | 83.44 |\n| GPSToken-M128 | 127.8 | 383 | 2689 | 50793 | 2567 | 180$\\pm$0.15 | 44.56 |\n| GaussianToken | 130.6 | 1706 | 2285 | 60781 | 5352 | 181$\\pm$0.17 | 44.32 |\n| FlexTok | 949.7 | 283 | 7665 | - | 7275 | 2714$\\pm$3.11 | 2.88 |\n\n*Table: Comparison of parameter size, computational cost, memory usage, latency, and throughput for generating $256\\times256$ images on an A100 GPU. Batch size is 8 for inference and 16 for training. Latency is averaged over 20 runs with standard deviation reported. Throughput is measured in samples per second. Training memory for FlexTok is unavailable due to lack of released training code.*\n\nWhile GPSToken ranks mid-tier in standalone efficiency, it is worth mentioning that tokenizers typically play a minor role in the overall latency of generation pipelines. The diffusion generator dominates the inference time; hence, efficiency is not our primary design goal. For instance, generating eight $256\\times256$ images takes the diffusion model about 60 seconds, while decoding the latent with GPSToken requires only 0.15 seconds, which is negligible compared to the total latency.\n\nWe hope the above analysis clarifies the efficiency profile of GPSToken and demonstrates its practical viability in real-world pipelines.\n\n---\n\n**Q2: Hyper-parameter experiments.**\t\t\n\nThe experimental results on $\\lambda$, $s$, and $s_{\\min}$ (hyper-parameter in calibration algorithm) are as follows.\n\n| Hyper-parameter | PSNR | SSIM | LPIPS | rec. FID | FID |\n| - | - | - | - | - | - |\n| $\\lambda$ = 0 | 23.52 | 0.638 | 0.110 | 1.02 | 2.59 |\n| $\\lambda$ = 5 | 24.06 | 0.652 | 0.083 | 0.68 | 2.23 |\n| $s$ = 1 | 17.55 | 0.4388 | 0.3435 | 160 | 165 |\n| $s$ = 3 | 24.07 | 0.657 | 0.080 | 0.66 | 2.18 |\n| $s$ = 7 | 24.06 | 0.658 | 0.080 | 0.66 | 2.16 |\n| $s_{min}$ = 8 | 24.05 | 0.656 | 0.080 | 0.66 | 2.18 |\n| $s_{min}$ = 16 | 24.03 | 0.657 | 0.081 | 0.66 | 2.19 |\n| ours($\\lambda=2.5, s=5, s_{min}=4$) | 24.06 | 0.657 | 0.080 | 0.65 | 2.18 |\n\n**Entropy threshold $\\lambda$**: As stated in Eq. 4 of the main paper, $\\lambda$ balances region size and complexity. \nA larger $\\lambda=5$  encourages Gaussians to concentrate on complex regions, leading to minor performance degradation. \nIn contrast, setting $\\lambda = 0$ allocates Gaussians solely based on region size, resulting in a uniform spatial distribution. \nThis causes a significant drop in performance: LPIPS increases from 0.08 to 0.11, and rec.FID rises from 0.65 to 1.02.\nWe set $\\lambda=5$ to 2.5 based on experimental experience.\n\n**Support factor $s$**: As stated in Eq. 2 of the main paper, $s$ controls the effective rendering support of each Gaussian. \nPerformance degrades significantly when $s = 1$, but stabilizes for $s \\geq 3$. This aligns with the $3\\sigma$ rule --- 99.7\\% of the mass of a 2D Gaussian lies within three standard deviations from the mean. To ensure full coverage, we set $s = 5$ in all experiments.\n\n**Minimal region size $s_{min}$**: We set $s_{min}$ in the calibration algorithm to match its value in the initialization algorithm, where $s_{min}$ determines the minimum width or height of each region. We observe that increasing $s_{min}$ from 4 to 16 results in negligible performance degradation. This is expected because, with 128 tokens representing a 256$\\times$256 image, the average spatial extent per token is approximately 22$\\times$22 pixels. Consequently, most of the segmented regions naturally have a width or height greater than or equal to 16, making the choice of $s_{min}$ within this range largely inconsequential for the final tokenization.\n\n---\n\n**Q3: Speed-ups for two-stage generator.**\n\nExisting methods that use pooling layers for speed-up often sacrifice performance (e.g., DiT shows FID degradation from 19.47 to 43.01 with larger pooling strides). In contrast, our GPSToken delivers practical speed-ups in training.\n\nAs shown in the following table, compared to the baseline (SiT-XL/2), our method achieves better FID (9.57 vs. 19.07 at 500K iterations) while reducing training time by 42% (128h vs. 219h) and boosting throughput by 73% (1.09 vs. 0.63 iters/s) with a reduced memory usage. These results confirm tangible speed-ups of our two-stage generator.\n\n| Method | Metric | 500K | 1000K | T-Mem | T-Thpt |\n|-|- |- |- |- |- |\n| Baseline | FID | 19.07 | 14.50 | 63684 | 0.63 |\n| | Time (h) | 219 | 439 | | |\n| Ours | FID | 9.57 | 7.61 | 41498 | 1.09 |\n| | Time (h) | 128 | 256 | | |\n\n*Notes: T-Mem: Training Memory (MB), T-Thpt: Training Throughput (iters/s)*\n\n---\n\n**Q4: Higher resolution.**\n\nThanks for the insightful question on the scalability of GPSToken to higher resolutions.\n\nSimilar to prior work, GPSToken requires token count to scale linearly with pixel count for consistent reconstruction quality. As shown in the table below, 128 tokens for 256×256 images achieve performance (e.g., LPIPS: 0.080, FID: 2.18) comparable to 512 tokens for 512×512 or 2048 for 1024×1024. However, using only 128 tokens for 512×512 images sharply degrades performance (LPIPS: 0.274, FID: 21.0), indicating severe loss of detail.\n\n| Resolution       | Token Count | PSNR   | SSIM   | LPIPS | rFID  | FID   |\n|-|-|-|-|-|-|-|\n| $256 \\times 256$ | 128         | 24.06  | 0.657  | 0.080 | 0.65  | 2.18  |\n| $512 \\times 512$ | 128         | 21.92  | 0.586  | 0.274 | 18.13 | 21.01 |\n| $512 \\times 512$ | 512         | 26.44  | 0.754  | 0.084 | 0.61  | 2.04  |\n| $1024 \\times 1024$ | 512       | 23.47 | 0.729 | 0.257  | 7.52  | 9.54  |\n| $1024 \\times 1024$ | 2048      | 27.16    | 0.848 | 0.093    | 0.88  | 2.32  |\n\nThis is visually evident in Fig. 6 of the main paper (32 tokens at 256×256), where coarse structures remain but fine textures blur due to insufficient token density—similar to using 128 tokens for 512×512, supporting our observation.\n\nNotably, PSNR and SSIM improve at higher resolutions with linearly scaled tokens. This occurs because content within each token’s receptive field becomes smoother and less textured, easing reconstruction and inflating pixel-wise and structural similarity scores.\n\nRegarding the parameter $\\sigma$, which controls the spatial extent of each GPS-token, we hypothesize that it should scale with the average area covered per token. Specifically, the average area per token is proportional to $H \\times W / N$, where $H \\times W$ is the image resolution and $N$ is the number of tokens. To maintain consistent local modeling capacity, $\\sigma$ should scale with the square root of this area:\n$$\n\\sigma \\propto \\sqrt{\\frac{H \\times W}{N}}.\n$$\nThus, when resolution increases but $N$ remains fixed, $\\sigma$ will increase to cover larger regions, resulting in over-smoothing and loss of detail. When both resolution and $N$ scale proportionally, $\\sigma$ can remain unchanged, preserving local fidelity.\n\n---\n\n**Q5: Non-natural images.**\n\nTo evaluate the robustness and generalization of our GPSToken to non-natural images, we conducted experiments on public datasets: **STARE** (retinal fundus images in medical imaging) and **WHU_RS19** (satellite remote sensing images). We compare against VAVAE (256 tokens, a leading 2D tokenizer) and MAETok (128 tokens, a top-performing 1D tokenizer).\n\nIn the table below, we report PSNR, SSIM, and LPIPS scores, which are more suitable for evaluating reconstruction quality on non-photorealistic images compared to metrics like FID.  \n\n| Dataset        | Token Count | Method           | PSNR ↑ | SSIM ↑ | LPIPS ↓ |\n|-|-|-|-|-|-|\n| STARE          | 128         | MAETok           | 32.98  | 0.818  | 0.051  |\n|                |             | GPSToken-M128    | 34.75  | 0.868  | 0.036  |\n|                | 256         | VAVAE            | 36.32  | 0.896  | 0.019  |\n|                |             | GPSToken-L256    | 37.60  | 0.915  | 0.014  |\n| WHU\\_RS19      | 128         | MAETok           | 21.73  | 0.506  | 0.195  |\n|                |             | GPSToken-M128    | 23.20  | 0.560  | 0.127  |\n|                | 256         | VAVAE            | 23.57  | 0.619  | 0.142  |\n|                |             | GPSToken-L256    | 26.33  | 0.731  | 0.064  |\n\nGPSToken consistently outperforms competitors at comparable token counts, e.g., achieving higher PSNR (26.33 vs. 23.57) than VAVAE on WHU_RS19. Due to space limits, visual comparisons are deferred to the revision. GPSToken-L256 also better reconstructs fine details (e.g., capillaries, trees, cars) while preserving large-scale structures.\n\nThese findings suggest that GPSToken is not only effective on natural images, but also generalizes well to diverse image domains, making it a robust and versatile tokenizer for various vision tasks."}, {"review_id": "pCjkk8gE2v", "reviewer": "Reviewer_pWj1", "summary": "This paper addresses the limitations of conventional grid-based image tokenizers, which lack spatial adaptivity and content-awareness. The authors propose GPSToken, a Gaussian-parameterized tokenization framework that dynamically models image regions using 2D Gaussians to encode their shape, position, and texture. By disentangling layout from appearance, GPSToken enables a two-stage image generation pipeline—first synthesizing structural layouts, then generating textures. Experiments on ImageNet show that GPSToken achieves state-of-the-art performance on both image reconstruction and generation tasks, with significantly fewer tokens and faster convergence.", "questions": "1.\tWhat is the basis for setting ${\\sigma }_{x}^{\\left( i\\right) }$ and ${\\sigma }_{y}^{\\left( i\\right) }$ to $\\frac{1}{6}$ for ${w}_{i}$ and ${h}_{i}$ to ensure full coverage during rendering ? Is this choice empirically optimal? How sensitive are the results to this initialization?\n\n2.\tHow does the method perform on real-world datasets beyond ImageNet? Any plans to test on more diverse datasets (e.g., COCO, FFHQ)?？", "limitations": "The authors have briefly acknowledged the heuristic limitation in Gaussian initialization and the need for more specialized generation architecture. However, it would be helpful to explicitly discuss:\n\n1.\tPotential failure cases (e.g., cluttered scenes, occlusions).\n2.\tHow token count influences spatial bias or missed details.\n3.\tScalability to higher resolutions.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The research problem is meaningful, and the authors present their ideas clearly, with careful and thorough explanation of the modeling process. The writing is solid and communicates the technical components effectively.\n\nHowever, a key weakness is that while the core idea—using 2D Gaussians to dynamically model variable region shapes and positions—is appealing in reducing redundancy in simple regions and enabling finer granularity in complex ones. The motivation for introducing 2D Gaussian functions is not clearly established, as the paper lacks a deeper analysis connecting their specific properties to the concrete demands of the tokenization or generation tasks. I would encourage the authors to further articulate the theoretical motivation and necessity for using this specific formulation.\n\nIn addition, the experimental section seems to lack necessary ablation studies to justify design choices, and the setup feels somewhat underexplored. Stronger empirical evidence would improve the credibility and completeness of the work.", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank this reviewer for the constructive comments and suggestion. We hope our following point-to-point responses can address this reviewer's concerns.\n\n---\n\n**Q1: Motivation for 2D Gaussians in tokenization.**\n\nOur choice of 2D Gaussian functions for image region modeling is grounded in their mathematical properties, which align well with the requirements of adaptive image tokenization.\n\nStandard grid-based tokenization methods suffer from spatial redundancy—simple regions are over-partitioned, while complex ones lack sufficient resolution.  An ideal tokenization scheme should have the following properties:  \n\n(1) flexibly represent semantic regions of arbitrary location and scale;  \n(2) model fuzzy boundaries in a soft, probabilistic manner;  \n(3) be fully differentiable to enable end-to-end optimization with gradient-based learning; and  \n(4) maintain low parametric complexity to ease downstream generation tasks.\n\nAfter evaluating alternatives such as bounding boxes and segmentation masks, we find that 2D Gaussians offer a principled and effective solution:\n\n- **Spatial adaptivity:** Each 2D Gaussian is parameterized by a mean $\\mu = (\\mu_x, \\mu_y)$ and a covariance matrix (determined by $\\sigma_x$, $\\sigma_y$, and $\\rho$), enabling anisotropic shapes that adapt to region geometry. Multiple Gaussians can be combined to model complex structures.\n\n- **Soft boundary modeling:** The Gaussian density function provides a smooth, continuous weight distribution, naturally capturing the uncertainty and gradual transitions at object boundaries in natural images.\n\n- **End-to-end differentiability:** All parameters are differentiable, and feature aggregation from CNN or ViT feature maps can be performed via soft weighting (inspired by differentiable rendering in 3DGS), enabling seamless integration into gradient-based training.\n\n- **Parameter efficiency:** A 2D Gaussian requires only five parameters to describe a spatial region—$\\mu_x$, $\\mu_y$, $\\sigma_x$, $\\sigma_y$, $\\rho$—offering a compact yet expressive representation that avoids overburdening the generative decoder.\n\nIn contrast, bounding boxes are limited to axis-aligned rectangles and exhibit hard, non-differentiable boundaries. Segmentation masks offer precise shapes but they are high-dimensional, discrete, and incompatible with differentiable optimization. Therefore, our use of 2D Gaussians is not merely inspired by 3D Gaussian Splatting (3DGS), but is also a well-motivated choice that satisfies the core demands of adaptive, efficient, and learnable tokenization.  \n\nWe will add more discussions in the revision, particularly in the introduction and method sections.\n\n---\n\n**Q2: Missing ablation studies.**\n\nActually, we have included a comprehensive analysis in Section C of the Appendix (starting at L42). This section presents both quantitative results and visualizations of the learned Gaussian parameters, which could help elucidate the role of each component in GPSToken. Specifically, the **\"Refine.\"** module adjusts Gaussian distributions to better align with local textures and consistently improves performance across metrics. The **\"Init.\"** module reallocates Gaussians from homogeneous to texture-rich regions, enabling finer semantic modeling without degrading reconstruction quality.\n\nTo further justify our design and address the concern of this reviewer, we have conducted additional sensitivity studies on key hyperparameters, including entropy threshold $\\lambda$, support factor $s$, minimum region size $s_{\\min}$.\nPlease refer to our response to Q2 of Reviewer EBwp, where we show that the model performance remains stable within reasonable ranges and our default settings achieve favorable performance.  \nWe will better highlight these results in the revision.\n\n---\n\n**Q3: Why set $\\sigma_x^{(i)}, \\sigma_y^{(i)} = \\frac{1}{6}w_i, \\frac{1}{6}h_i$?**\n\nThis is based on the **3σ rule** — a well-established empirical principle for normal distributions — which states that nearly **99.7%** of data points of a normal distribution lie within the range of $\\mu \\pm 3\\sigma$. Based on this rule, each region is designed to have a width of approximately $6\\sigma$ (spanning from $\\mu - 3\\sigma$ to $\\mu + 3\\sigma$), ensuring the inclusion of the vast majority of relevant data points.\nNotably, this initialization has minimal impact on the final outcome, as we subsequently refine the $\\sigma$ values through transformer blocks in the encoder.\n\n---\n\n**Q4: Performance on COCO and FFHQ.**\n\nAs suggested, we further evaluate GPSToken on two additional real-world datasets: **COCO2017** and **FFHQ**, randomly sampling 5,000 images from each. As demonstrated in the following table, GPSToken consistently outperforms its competitors — MAETok (128 tokens) and VAVAE (256 tokens), representing the state of the art in 1D and 2D tokenization, respectively — across all metrics and datasets under the same token counts.\n\n| Dataset    | Token Count | Method         | PSNR ↑ | SSIM ↑ | LPIPS ↓ | rec. FID ↓ |\n|------------|-------------|----------------|--------|--------|---------|------------|\n| COCO2017   | 128         | MAETok         | 22.67  | 0.623  | 0.101   | 8.91       |\n|            |             | GPSToken-M128  | 23.47  | 0.657  | 0.083   | 4.72       |\n|            | 256         | VAVAE          | 25.01  | 0.736  | 0.052   | 6.01       |\n|            |             | GPSToken-L256  | 27.41  | 0.794  | 0.035   | 2.23       |\n| FFHQ       | 128         | MAETok         | 25.53  | 0.707  | 0.064   | 4.66       |\n|            |             | GPSToken-M128  | 26.35  | 0.745  | 0.050   | 3.72       |\n|            | 256         | VAVAE          | 28.06  | 0.808  | 0.027   | 1.95       |\n|            |             | GPSToken-L256  | 30.02  | 0.846  | 0.019   | 1.51       |\n\nSpecifically, GPSToken-L256 achieves the best results on all three benchmarks: 30.02 PSNR / 0.846 SSIM on FFHQ, 28.81 PSNR / 0.809 SSIM on ImageNet (see Tab. 1 in the main paper), 27.41 PSNR / 0.794 SSIM on COCO2017.\nThe observed performance trend -- FFHQ $>$ ImageNet $>$ COCO2017 -- aligns well with the inherent data complexity. The structured nature of human faces in FFHQ facilitates reconstruction, whereas ImageNet, with its 1,000 diverse object categories, presents a greater challenge. COCO2017, featuring complex scenes with multiple objects at varying scales and rich contextual interactions, poses the most difficult reconstruction task.\n\nThe consistent superiority across datasets with different characteristics underscores the generalization capability of GPSToken.\n\n---\n\n**Q5: More discussion on limitation.**\n\nThanks for the constructive comments. We will include more discussions in the **Limitations** section based on this reviewer's suggestion.\n\n**Potential Failure Cases.**\nGPSToken adaptively allocates fewer tokens to simple regions and more tokens to complex regions for efficient representation without compromising reconstruction. One limitation of this strategy is that it may offer limited benefit for uniformly simple images and allocate dense tokens everywhere for images with high-frequency details or noise, reducing its efficiency. Our method is robust to occlusions because token allocation depends on the observed complexity of the visible region, but not predefined object structures. Changes in local content caused by occlusion naturally influence the token distribution based on the resulting complexity. We will show examples of these cases in the revision. \n\n**Token counts.**  GPSToken supports flexible token allocation during inference, allowing adaptation to varying computational budgets or fidelity requirements. As demonstrated in Fig. 6 of the main paper, reducing the total number of tokens preserves global structure while gradually sacrificing fine-grained local details. In contrast, increasing the token count will enhance the local detail reconstruction across the image.\n\nCrucially, this adaptability does not introduce significant spatial bias in token distribution. Our region partitioning strategy balances both spatial extent and local complexity, with a minimum region size constraint during initialization. This design ensures that even under low-token regimes, no region will be entirely omitted, thereby maintaining a spatially balanced representation. \n\n**Higher resolution.** GPSToken demonstrates favorable scalability to higher-resolution images. Empirically, we observe that reconstruction quality (measured by PSNR and SSIM) remains consistent when scaling both image resolution and token count proportionally. For instance, as shown in our response to Q4 of reviewer EBwp, reconstructing a $512 \\times 512$ image using 512 tokens (or a $1024\\times 1024$ image using 2048 tokens) achieves comparable fidelity to reconstructing a $256 \\times 256$ image with 128 tokens. This result aligns with prior tokenization methods, suggesting that our approach preserves representational efficiency across resolutions."}, {"review_id": "0cjJ36cqzG", "reviewer": "Reviewer_xDKG", "summary": "GPSToken proposes a Gaussian parameterized spatially-adaptive tokenization framework to address the inflexibility of conventional 2D/1D grid tokenization in representing regions with varying shapes, textures, and locations. The framework achieves non-uniform image tokenization by: 1) partitioning images into texture-homogeneous regions using an entropy-driven algorithm; 2) parameterizing each region as a 2D Gaussian (mean for position, covariance for shape) with texture features; 3) optimizing Gaussian parameters via a specialized Transformer for continuous adaptation and content-aware feature extraction. During decoding, Gaussian-parameterized tokens are reconstructed into 2D feature maps using a differentiable splatting-based renderer, enabling end-to-end training with standard decoders. By decoupling spatial layout (Gaussian parameters) from texture features, GPSToken enables a two-stage generation pipeline: structural layout synthesis via lightweight networks, followed by structure-conditioned texture generation.", "questions": "1. In the \"GPSToken-driven Two-stage Image Generation\" framework, which specific loss function is employed to optimize the model, and how do these losses contribute to the overall generation quality and semantic consistency of the generated images?\n2. What are the detailed training recipe adopted in \"GPSToken-driven Two-stage Image Generation\"? For instance, are there any specific layers or components, such as the decoder, whose weights are frozen during certain training phases? And how do these strategies impact the model's convergence speed and performance in generating high-fidelity images?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. Unlike prior grid-based or 1D tokenization, GPSToken introduces a continuous, parametric representation of image regions using 2D Gaussians, which dynamically adapts to local structures. This is a departure from discrete or fixed-grid methods.\n2. The two-stage generation pipeline (structure-first, texture-later) aligns with human perception and reduces computational complexity, making it feasible for real-world applications. The ability to adjust token count at inference enhances flexibility for quality-efficiency trade-offs.\n3. The paper outlines the entropy-driven partitioning algorithm, Gaussian parameterization, and transformer refinement process in sufficient detail, including mathematical formulations (e.g., Eq. 2 for modified Gaussian, Eq. 3 for splatting). Experimental settings (dataset, tr-aining configurations, metrics) are well-documented.\n4. The paper provides extensive quantitative results on ImageNet, comparing against a wide range of state-of-the-art tokenizers (e.g., SDXL-VAE, TiTok, MAETok) across reconstruction and generation tasks. The performance gains (e.g., FID 1.64 with 128 tokens) are significant and supported by multiple metrics (PSNR, SSIM, LPIPS).\n\nWeakness:\n1.\tExperiments are primarily conducted on ImageNet 256*256. More experiments on ImageNet 512*512 or t2i could be conducted.\n2.\tThe paper highlights faster training convergence in line 63 but lacks concrete metrics (e.g., GPU hours, memory footprint) to quantify computational efficiency against baselines (e.g., SiT-XL/2). How does GPSToken optimize practical deployment costs? Please supplement with detailed computational benchmarks (e.g., training time per epoch, inference time, VRAM usage) and comparative analysis.", "was_revised": true, "final_justification": "Thanks for the response from the authors. The rebuttal has addressed my concerns. I hope the authors include the additional experiments in their manuscript to better support their claims.", "rebuttal": "We sincerely thank this reviewer for the constructive comments and suggestion. We hope our following point-to-point responses can address this reviewer's concerns.\n\n---\n\n**Q1: Higher resolution and T2I tasks**.\n\nRegarding higher-resolution image reconstruction tasks, we refer the reviewer to our response to Q4 of Reviewer EBwp, where additional experiments on $512\\times 512$ and $1024\\times 1024$ images are presented. The successful tokenization and faithful reconstruction at these scales indicate that our method can reliably encode fine-grained visual details -- a critical prerequisite for high-resolution generation.\n\nRegarding high-resolution or text-to-image (T2I) generation tasks, we acknowledge that additional experiments would further validate the applicability of our tokenizer in generative frameworks. However, such experiments generally require substantial computational resources, typically involving several weeks to months of training on large GPU clusters. Due to these practical constraints, we are currently unable to complete them within the rebuttal timeline. We plan to conduct these studies as part of our future work.\n\nWe believe that our experiments are well-aligned with recent advances in the context of image tokenizers. Many recent works (e.g., MaskBit (ICLR 2025), VAVAE (CVPR 2025)) primarily focus on reconstruction and generation tasks on ImageNet $256\\times 256$ images. Our experimental design follows this established convention, prioritizing the analysis of the tokenization process itself.\n\n---\n\n**Q2: Computational benchmarks**\n\nWe thank the reviewer for the insightful comment. In response, we provide comprehensive computational benchmarks by comparing GPSToken with the SiT-XL/2 baseline in both training and inference.\n\nAs shown in the table below, at 1M iterations, GPSToken achieves a significantly lower FID score (7.61 vs. 14.50), with 42% less training time (256h vs. 439h), 73% higher training throughput (1.09 vs. 0.63 iters/s), and 35% lower VRAM consumption (41,498 MB vs. 63,684 MB). During inference, although VRAM usage increases slightly (9,636 MB vs. 9,126 MB), our method nearly doubles throughput (0.129 vs. 0.067 samples/s), reducing latency by approximately half.\n\n| Method  | Metric     | 500K    | 1000K   | T-Mem (MB) | T-Thpt (iters/s) | I-Mem (MB) | I-Thpt (samples/s) |\n|---------|------------|---------|--------|------------|------------------|------------|--------------------|\n| Baseline | FID        | 19.07   | 14.50  | 63,684     | 0.63             | 9,126      | 0.067              |\n|         | Time (h)   | 219     | 439    |            |                  |            |                    |\n| Ours     | FID        | 9.57    | 7.61   | 41,498     | 1.09             | 9,636      | 0.129              |\n|         | Time (h)   | 128     | 256    |            |                  |            |                    |\n\n*Notes: T-Mem: Training Memory, T-Thpt: Training Throughput, I-Mem: Inference Memory, I-Thpt: Inference Throughput*\n\nThese efficiency gains are attributed to two key design choices:\n\n(i) GPSToken reduces the number of effective tokens, lowering computational and memory overhead;  \n(ii) the two-stage generation framework simplifies the learning objective and stabilizes optimization, facilitating faster convergence to higher-quality solutions.\n\nOverall, GPSToken not only improves generation quality but also significantly reduces training cost and inference latency.\n\n---\n\n**Q3: Loss functions for generators**.\n\nThank you for the question. We apologize for the lack of details in loss function for generators.\n\nBoth the layout and texture generators use the standard **velocity matching loss** from SiT: the model predicts the diffusion velocity $\\mathbf{v}_\\theta(\\mathbf{x}_t, t)$, and we apply an $L_2$ loss to match the target $\\mathbf{v}^*$:\n\n$$\n\\mathcal{L} = \\mathbb{E} \\left[ \\|\\mathbf{v}_\\theta(\\mathbf{x}_t, t) - \\mathbf{v}^*\\|^2 \\right].\n$$\n\nThis loss promotes high generation quality by modeling data dynamics accurately. In the layout stage, it encourages semantically coherent Gaussian arrangements. In the texture stage, it ensures fidelity and alignment with layout conditions, enhancing overall semantic consistency.\n\n--- \n\n**Q4: Training recipe for generators.**\n\nBoth the layout and texture generators are trained **independently and end-to-end**, with **no component frozen** — all parameters are updated during training.\n\nSpecifically, the **layout generator** uses a SiT-B/4 architecture and is trained to model $g_{init}$ (which encodes only the position and shape of initialized Gaussians) using the standard velocity matching loss. Since $g_{init}$ represents a simple geometric prior, its distribution is relatively easy to learn. The **conditional texture generator** employs a SiT-XL/2 architecture and learns to generate the refined layout $g$ and image features $f$, using $\\{g, f\\}$ as targets and the same velocity matching loss. Although modeling their joint distribution is challenging, conditioning on the generated $g_{init}$ — from which $g$ is refined — provides strong structural guidance, significantly reducing optimization difficulty.\n\nAt inference, a **calibration step** is employed to refine $g_{init}$ to correct minor errors (L190 in the main paper), mitigating potential misalignment from independent training.\n\nAs shown in Tab. 2 and Figs. 7/8 in the main paper, this strategy enables faster convergence and high-fidelity generation, validating the effectiveness of our decoupled yet coherent two-stage design."}, {"review_id": "cfQ4dPITuN", "reviewer": "Reviewer_pm2Q", "summary": "This paper introduces GPSToken, a Gaussian Parameterized Spatially-adaptive Tokenization framework for image representation and generation. \nUnlike conventional grid-based tokenizers, GPSToken uses parametric 2D Gaussians to dynamically model regions of varying shapes, positions, and textures, enabling non-uniform and content-aware tokenization. \nThe proposed method partitions images into texture-homogeneous regions, initializes Gaussian parameters based on entropy-driven complexity, and refines them via a transformer.\n A splatting-based renderer converts tokens back into feature maps for decoding. GPSToken decouples spatial layout (Gaussian parameters) from texture features, facilitating a two-stage generation pipeline that first synthesizes structural layouts and then generates textures.\n Experiments on ImageNet demonstrate state-of-the-art performance.", "questions": "1. In tab.2, I notice that the proposed method and MAETok have similar performance. However, the proposed method has a larger generator, and the MAETok has a larger tokenizer. Which model part do you think will influence the performance most? Are there any experimental results that can reflect this?", "limitations": "Yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "# Strengths\n\n1. The main idea of this paper is quite interesting, and the proposed methods are reasonable.\n\n2. The writing of this paper is easy to follow.\n\n\n# Weaknesses\n\n1. The experiments seem unfair in some cases. For example, in Tab.1, the parameter size of the VAVAE is only 69.8M, which is significantly smaller than the proposed method. \n\n2. I think it will be better to completely discuss the training overhead of the proposed method and compare it with existing SOTA methods instead of only discussing SiT-XL/2.\n\nI am not an expert in this field, and I may not understand all the strengths and weaknesses of this paper.", "was_revised": true, "final_justification": "Overall, I think leveraging parametric 2D Gaussians to model different image regions is a very interesting idea, and the authors also provide a reasonable pipeline to realize the proposed idea. \nWhile I initially had some concerns about the experiments, the authors addressed them satisfactorily in their rebuttal.\nThus, I maintain my positive score.", "rebuttal": "We sincerely thank this reviewer for the constructive comments and suggestion. We hope our following point-to-point responses can address this reviewer's concerns.\n\n---\n\n**Q1: Parameter size**.\n\nWe thank the reviewer for the insightful comment. Although VAVAE has fewer parameters (69.8M) than our method (128M), our model size is moderate among the 14 methods compared in Tab. 1 of the main paper. Moreover, direct comparison based solely on parameter count is not reasonable due to the significant architectural and training differences: SDXL-VAE uses residual blocks, TiTok employs transformers, VAVAE leverages a large pretrained vision model, FlexTok incorporates a generative decoder, and our GPSToken introduces Gaussians as a novel image representation. Notably, FlexTok has about eight times the number of parameters as our GPSToken, yet it performs significantly worse — demonstrating that parameter count alone does not determine performance.\n\nMore importantly, the number of tokens has a greater impact on the downstream generator than the tokenizer's parameter count, as it directly affects the optimization difficulty and the computational cost—note that the generator typically contains far more parameters (e.g., 708M in our case). Consequently, most prior works focus on improving reconstruction quality under the same token count, placing less emphasis on tokenizer parameter efficiency. In Tab. 1 of the main paper, under this standard and fair setting (same token count), our GPSToken achieves the best reconstruction performance across all metrics.\n\nWe will make these points clearer in the revision.\n\n---\n\n**Q2: Training overhead**.\n\nWe sincerely thank the reviewer for the insightful comment. We agree that a comprehensive discussion on the training overhead in comparison with existing SOTA methods is valuable.\n\nHowever, due to the substantial computational cost—our SiT-XL/2 baseline takes approximately 9 days for 400k iterations on our GPU cluster—conducting extensive retraining across multiple SOTA frameworks within the rebuttal period is unfortunately infeasible.\n\nInstead, we can provide an indirect comparison based on the reported results in literature. As stated in their papers, REPA achieves comparable performance to SiT-XL/2 (400k iters) in just 100k iterations, and MAETok reaches similar performance in about 50k iterations. In contrast, our GPSToken matches the 400k-iteration performance of SiT-XL/2 in approximately 150k iterations. Thus, in terms of convergence speed measured by iteration count, REPA and MAETok converge faster than our two-stage framework.\n\nHowever, per-iteration efficiency and wall-clock training time are equally important. REPA relies on a large pretrained vision model (DINOv2-L) on the base of SiT-XL/2, leading to high computational and memory overhead per step. In contrast, our method significantly improves per-step efficiency than SiT-XL/2 (see our response to Q2 of reviewer xDKG). Therefore, despite requiring more iterations, our approach achieves competitive or even superior wall-clock training speed in practice.\n\nIt should be noted that our method is orthogonal to existing acceleration techniques, such as auxiliary losses (e.g., in MAETok, REPA). We believe these techniques can be integrated into our framework for even greater efficiency. We also would like to clarify that the primary contribution of GPSToken is a novel Gaussian-based visual tokenization method and a two-stage generation framework, enabling effective image representation and structured generation. Reduced training overhead is a beneficial byproduct, not our main focus.\n\nWe will revise the manuscript to include a more comprehensive discussion on these aspects.\n\n---\n\n**Q3: Tokenizer vs. generator**.\n\nThank you for the insightful observation. The difference in model scale—our GPSToken generator is slightly larger while MAETok employs a relatively larger tokenizer—stems primarily from the disparate experimental setups and architectural choices across methods, rather than intentional design for performance tuning. Notably, our approach employs a conventional residual block-based decoder, whereas MAETok utilizes a transformer-based decoder. These fundamental architectural distinctions naturally lead to variations in parameter distribution and model size.\n\nRegarding which component—tokenizer or generator—has a greater influence on overall performance, this remains an open question in the field. Prior work exhibits diverse emphases: methods such as VQVAE and TiTok focus heavily on improving tokenizer fidelity under the premise that better reconstruction leads to stronger generative performance. In contrast, approaches such as REPA employ auxiliary losses to guide generator training, effectively treating the tokenizer as a secondary component. More recently, emerging frameworks such as FlexTok and REPA-E advocate for joint optimization or unified modeling of tokenization and generation, blurring the boundary between the two modules. This evolving landscape makes it challenging to isolate the relative importance of each part under the current paradigms.\n\nIn our view, the tokenizer plays a more critical role in the overall system. It shapes the structure of the latent space, thereby determining the intrinsic complexity of the generative task and establishing the theoretical performance ceiling for the generator. A poorly structured latent space can impose optimization barriers that even a powerful generator may struggle to overcome.\n\nBy far, there lack comprehensive empirical studies to systematically ablate or compare the contributions of tokenizers and generators under controlled settings. This gap is largely due to the significant computational resources required and the difficulty in establishing fair comparisons across diverse architectures, training objectives, and theoretical foundations. We believe that developing standardized benchmarks for modular evaluation—such as plug-and-play tests between tokenizers and generators—would be a valuable direction for future work."}]

Author rebuttal:

Meta-review: This paper tackles spatially-adaptive image tokenization with a Gaussian parameterized approach. In contrast to fixed spatial correspondence, tokens are initialized with an iterative splitting algorithm to represent regions of variable size. Empirical evidence shows its advantage in reconstruction quality, generation quality, and training convergence speed. The final version should reflect the suggestions from reviewers and the promises in the rebuttal.
Decision: Accept (poster)