Latent Representation Matters: Human-like Sketches in One-shot Drawing Tasks

Authors: victor  boutin
rishav  mukherji
aditya  agrawal
sabine  muzellec
thomas  fel
thomas  serre
rufin  vanrullen
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 10:00:55

Abstract: Humans can effortlessly draw new categories from a single exemplar, a feat that has long posed a challenge for generative models. However, this gap has started to close with recent advances in diffusion models. This one-shot drawing task requires powerful inductive biases that have not been systematically investigated. Here, we study how different inductive biases shape the latent space of Latent Diffusion Models (LDMs). Along with standard LDM regularizers (KL and vector quantization), we explore supervised regularizations (including classification and prototype-based representation) and contrastive inductive biases (using  SimCLR and redundancy reduction objectives). We demonstrate that LDMs with redundancy reduction and prototype-based regularizations produce near-human-like drawings (regarding both samples' recognizability and originality) -- better mimicking human perception (as evaluated psychophysically). Overall, our results suggest that the gap between humans and machines in one-shot drawings is almost closed.

Main paper: 
 introduction: 
 For cognitive scientists, human drawings offer a window into the brain, providing tangible insights into its visual and motor internal processes [1] . For instance, drawings have been used in clinical settings to screen for perceptual impairments following brain trauma or Alzheimer's disease [2, 3] , to assess perceptual disorders in autistic individuals [4] [5] [6] or to investigate perceptual changes during child development [7, 8] (see [1] for a recent review). Drawing tasks have also proven instrumental for exploring how the brain generalizes to novel visual categories [9] [10] [11] . Cognitive psychologists routinely use the one-shot drawing task to understand how human observers can reliably form new object categories from just one exemplar [12, 13] . From a computational viewpoint, this task is ill-defined because of the infinite number of possible sets of samples that could be associated with that exemplar. Yet, humans can effortlessly produce drawings that are not only easily recognizable but also original (i.e., sufficiently distinct from the reference exemplar) [12] . This remarkable capability suggests that the brain leverages powerful representational inductive biases -yet to be discoveredto form novel categories.Computer scientists have started to make progress in identifying some of the inductive biases for machine learning algorithms to learn from limited data. For one-shot classification tasks, a particularly effective representational inductive bias is to design an embedding space where samples of the same category, whether seen during training or not, cluster closely. This approach spans a wide range of models ranging from representations learned via contrastive objective functions [14] [15] [16] , prototypebased representations [17, 18] or metric matching losses [19, 20] . Conversely, for one-shot generation tasks, researchers have preferred architectural over representational inductive biases. For instance, novel architectures based on Generative Adversarial Networks (GANs) or Variational Auto-Encoders (VAEs) have incorporated forms of spatial attention [21] or contextual integration [22] [23] [24] . Recent advances in diffusion models [25, 26] make them particularly promising for one-shot generation tasks. Indeed, clever conditioning on a context vector [24] or directly using guidance from the exemplar [27] has led to powerful one-shot diffusion models [28] . Such a guidance mechanism has also proven successful in Latent Diffusion Models (LDMs) [29] , which use a Regularized AutoEncoder (RAE) to compress input data and a diffusion model to learn the RAE's latent distribution. These diffusion models have started to close the gap with humans in the one-shot drawing task [30] (see section 2 for related work on one-shot learning). While better conditioning mechanisms have driven improvements in one-shot generative models, the potential of shaping their input space with representational inductive biases inspired by one-shot classification remains largely unexplored. This raises the question: "Do representational inductive biases from one-shot classification help narrow the gap with humans in one-shot drawing tasks ?"In this article, we use Latent Diffusion Models (LDMs [29] ) to address this question. LDMs combine the flexibility of the Regularized AutoEncoder (RAE), in which one can seamlessly include various representational inductive biases in the latent space via regularization, with the high expressivity of the diffusion model. Herein, we study the impact of 6 different regularizers corresponding to distinct representational inductive biases. They are categorized into 3 groups. The first group, which serves as a baseline, includes the KL and the vector quantization regularization approaches typically used in LDMs [29] . The second group involves supervised regularizers: a classification loss that promotes discriminative features mapping with categorical training labels and a prototype-based objective function that clusters samples with their respective prototypes in an embedding space. The third group features contrastive learning regularization schemes with the SimCLR and Barlow losses. The SimCLR objective function keeps a sample and its augmented view close in the embedding space but far apart from other samples' views. In contrast, the Barlow loss ensures that features of similar samples are decorrelated from those of dissimilar ones.We compare those regularized LDMs against humans on the one-shot drawing task. Such a task offers a leveled playfield in which humans and machines can create sketches that are directly comparable using established evaluation frameworks [31, 30, 12] (see section 2 for related work). More specifically, our comparison focuses on two metrics to evaluate the quality of sketches produced by humans and machines -based on how distinct from the exemplar and how recognizable they are [31] -and on the alignment between humans' and machines' perceptual strategies. For the latter, we describe a novel method to generate importance maps highlighting category-diagnostic features in LDMs. These maps are then directly compared against importance maps derived from human observers obtained through psychophysics experiments. Our results show that LDMs using prototypebased and redundancy-reduction (with the Barlow twin objective) regularization techniques are further closing the gap with humans. These results are supported by both the sample's similarity and the feature importance maps alignment. Overall, our contributions can be summarized as follows:• We introduce novel representational inductive biases in Latent Diffusion Models. In particular, we draw inspiration from losses that have proven effective in one-shot classification tasks (with the prototype-based, Barlow and SimCLR objective functions) to regularize the latent space of LDMs. • We derive a novel explainability method to generate LDMs' feature importance maps that highlight category diagnostic features. • We systematically compare the sketches and feature importance maps derived from humans and machines, and we show that LDMs with prototype-based and Barlow regularization significantly narrow the gap with humans on the one-shot drawing task.Our work underscores the critical role of well-designed representational inductive biases in achieving human-like performance in one-shot drawing tasks. It also sets the stage for developing generative models that are better aligned with humans. 
related work: 
 Representation learning for one-shot classification tasks: Learning representations that bring unseen samples (from the query set) close to the exemplars (in the support set) has proven effective in one-shot classification. The historical approach, called metric learning, aims at creating a feature space in which the distances between the query and support sets are preserved [20, 19, 32, 33] . However, the limited number of samples in the support set restricts these networks' ability to recognize novel classes. This limitation becomes more pronounced in the one-shot setting as the support set contains only one sample (the exemplar). To address this, the field has shifted towards prototype-based representations. Rather than trying to preserve the distances between query and support samples, such networks learn an embedding space in which the query samples cluster near the support samples [17, 34, 35] . Contrastive learning, a self-supervised learning approach, offers another effective solution to mitigate sample scarcity by augmenting the training set. This method learns an embedding space where positive pairs (a sample and its augmented version) are close together, and distant from negative pairs (augmented views from different instances) [14, 15, [36] [37] [38] [39] . Among alternative methods, the SimCLR algorithm [14] uses a cosine similarity between samples whereas the Barlow-twins network [15] leverages the correlation matrix between features to dissociate positive and negative pairs. In this article, we use the prototype-based [17] , the SimCLR [14] and the Barlow twins [15] objectives to regularize RAEs latent space. For additional mathematical details, see section 4.1 for the prototype-based loss and section A.2.3 for SimCLR and Barlow.Generative models for one-shot image generation tasks: Some of the main techniques involve including information from the support set into the generative process, a method known as conditioning. For instance, the Neural Statistician uses a context vector containing summary statistics from the support sets, which is then concatenated with a VAE latent space [22, 24, 40] . Similarly, GANs leverage a compressed representation of the support set as a conditioning mechanism [23] . Such a mechanism has also been used successfully to either condition [41] [42] [43] 29] or guide the denoising process of diffusion models [27, 28] and latent diffusion models [29] . Here, we leverage LDMs with classifier-free guided diffusion models [27] . Such a diffusion process has been shown to well approximate human drawings in one-shot drawing tasks [30] .Human-machine comparison in one-shot drawing tasks: Cognitive scientists have developed various methods to compare the generalization abilities of machines and brains on drawing tasks. Lake et al. [44] introduced the Omniglot challenge in which both humans and machines are tasked with drawing symbols from categories represented by a single exemplar (see [45] for a review on the challenge). The authors evaluated the drawings' recognizability in a visual Turing test where humans (or classifiers) had to distinguish between human-drawn and machine-generated symbols [11] . Additional metrics, including classification uncertainty and semantic similarity, were also used to compare drawings produced by humans and machines under different time constraints [46, 8] . While these evaluation frameworks provide useful insights into a sample's recognizability, they do not measure how the diversity of model-generated samples compares to those created by humans. The "originality vs. recognizability" framework [31] mitigates this issue by adding the originality metric. An originality score quantifies the similarity between the original exemplar and its corresponding variations (see section 5.1 for details on this evaluation framework). This evaluation framework has been used to benchmark the generalization performance of mainstream generative models -Diffusion models [47] , GANs [48] and VAEs [49] -against humans in the one-shot drawing setting [30] .Although Diffusion models come closest to human performance, a noticeable gap remained in this study. In this article, we use the "originality vs. recognizablility" framework from Boutin et al. [31] to evaluate representational inductive biases in Latent Diffusion Models. In particular, we demonstrate that effective biases in one-shot classification tasks also prove efficient in the one-shot drawing task.Prototypical networks focus on learning an embedding space where data points cluster around a single prototype representation for each class. A prototype is originally defined as the mean vector of the embedded support points belonging to its class [17] . In the one-shot setting, the support set is reduced to one single sample. Therefore here the prototype and the exemplar are the same.To achieve the desired embedding space for the autoencoder we regularize the reconstruction loss with a protoype-based loss. The loss uses the pairwise ℓ 2 distance between samples and prototype to derive a probability distribution:EQUATIONIn Eq. 13, h P R θ (z y ) represents the projection of the prototype in the embedding space while h P R θ (z) represents the projections of the sample. See Algo. 2 for more details on the exact implementation of the prototype-based regularized RAE. L P R = -log(softmax(d)) 6 x = p θ (x|z) # decode 7 L = ∥x -x∥ 2 2 + L P R 8 π ← ∂L π A.2.As in VAEs [49] , the Evidence Lower Bound of the diffusion model could be recovered using Jensen's inequality [47] :E z0∼r(z0) log p ψ (z 0 |z y ) = E z0∼r(z0) log p ψ (z 0:T |z y )dz 1:T = E z0∼r(z0) log r(z 1:T |z 0 ) p ψ (z 0:T |z y ) r(z 1:T |z 0 ) dz 1:T = E z0∼r(z0) log E z 1:T ∼r(z 1:T |z0) p ψ (z 0:T |z y ) r(z 1:T |z 0 ) ≤ E z 0:T ∼r(z 0:T ) log p ψ (z 0:T |z y ) r(z 1:T |z 0 ) = -L V LBThe Variational Lower Bound could be written as a sum of KL terms [26] :L V LB = E r log r(z 1:T |z 0 ) p ψ (z 0:T |z y ) = E r log T t=1 r(z t |z t-1 ) p(z T |z y ) T t=1 p ψ (z t-1 |z t , z y )using Eq. ( 17) and ( 20)EQUATIONEQUATIONIn the previous equations, E r is a shortcut notation for E z 0:T ∼r(z 0:T ) . Note that in the optimization process, L T could be ignored because it doesn't depend on the model parameter ψ, this is a pure non-informative Gaussian distribution (see Eq. 22). L 0 is modeled by Ho et al. [47] using a separate neural network. L t is a KL between 2 Gaussians distributions, so it could be calculated with a closed form:EQUATIONWith μt (z t , z 0 ) and βt I the mean and the variance of r(z t-1 |z t , z 0 ), respectively. Using Eq. 18 we can express z 0 in a convenient way:EQUATIONTherefore on can simplify μt (z t , z 0 ) in Eq. 23:EQUATIONSimilarly, we can re-parameterize p ψ (z t-1 |z t , z y ) because z t is available as input at training time:EQUATIONOne can apply the closed form formula of the KL between 2 gaussians distributions to compute L t in Eq. 22:L t = E r 1 2 ∥σ 2 t ∥ 2 2 ∥μ t (z t , z 0 ) -µ ψ (z t , t)∥ 2 2 = E r 1 2 ∥σ 2 t ∥ 2 2 1 √ α t z t - 1 -α t √ 1 -ᾱt ϵ - 1 √ α t z t - 1 -α t √ 1 -ᾱt ϵ ψ (z t , t) 2 2with Eqs. 25 and 26EQUATIONWith further simplification of Eq. 27 [47] :EQUATION(28)EQUATIONQuestion: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: We have discussed the limitations in the supplementary information (see section A.8.6). There are 3 main limitations. First, the originality vs. recognizability framework might not be aligned with human perceptual judgment. Second, the long training time of 2-stages latent diffusion models prevents the wide adoption of the representational inductive biases we propose in this paper. Third, we have tested a limited number of regularizers, so other regularization techniques might be even better aligned with humans.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.Justification: We report error bars and pair-wise statistical tests on Fig. 5 (see A.8.4). Note that we did not compute error bars for Fig. 3 as our analysis relies on a fit made on tens of models.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper. • The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: We believe we conform with the NeurIPS code of ethics in every aspect. Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
datasets: 
 As done in previous work [31, 30, 11] , we use the Omniglot [11] and the QuickDraw-FS [30] datasets to compare humans and machines on the one-shot drawing task. These datasets, made of handwritten symbols or drawings, offer a fair environment for comparing the generation abilities of humans and machines [11, 46, 31, 30] . It is important to note that natural images generation is a task beyond human capability, making it unsuitable for a fair comparison between humans and machines.Omniglot contains 1, 623 categories of handwritten characters from 50 different alphabets, with 20 samples per class [11] . This article uses a downsampled version of the dataset (size: 48 × 48 pixels). We train the models on a training set composed of all available symbols minus 3 symbols per alphabet left aside for the test set (similar to [21] ). All the results on the Omniglot dataset are in the Appendix (see A.6).QuickDraw-FS is made from drawings of the Quick, Draw ! challenge [50] . In this challenge, human participants are asked to produce drawings in less than 20 seconds when presented with an object name. The categories are, therefore, made with semantically consistent samples that do not necessarily represent the same visual concept (e.g., the "phone" object category might contain corded phones, smartphones, phones with rotary dials, etc). The Quickraw-FS dataset mitigates this issue with categories representing the same visual concepts (see A.1 for more details). This dataset is ideally suited for purely visual one-shot generation tasks [30] . It contains 665 categories with 500 samples each. The training set is made of 550 randomly selected categories, and 115 are left aside for the testing set. We downsampled the drawings to 48 × 48 pixels to keep computational resources manageable.For each category in both datasets, we extract a 'prototypical' sample, selected in the center of the category cluster to condition the one-shot generative models (see A.1 for more details on the exemplar selection).4 One-shot Latent Diffusion Models 1 ), and a second stage consisting of a diffusion model that learns the latent distribution (orange boxes in Fig. 1 ).In the one-shot setting, the diffusion model is conditioned by z y , the latent representation of y. We call c the category label of the training set (a one-hot vector).Maths and Algorithms: Contrastive learning algorithms learn representations that are invariant under different distortions (i.e. data augmentations). Here we use two data-augmentation operators, τ A (•) and τ B (•), that transform the variations x into x A = τ A (x) and x B = τ B (x), respectively. We denote z A and z B the latent space projection of x A and x B , respectively (i.e. q ϕ (z A |x A ) and q ϕ (z B |x B )). Here, we use two different types of contrastive regularizations that are L SimCLR (see Eq. 14) and L Bar (see Eq. 15)EQUATIONEQUATION)with sim(x, y) i = ⟨x, y⟩ i ∥x∥ 2 ∥y∥ 2 (16) In these equations, b indexes the sample in a batch, i indexes the vector component of the embeddings, h I θ (z) and h B θ (z) are linear probe stacked on the RAE latent space. In the Barlow regularizer, we use λ = 5 × 10 -3 . For both networks, the linear probe projects in a space of size 128. This is important to observe that the scalar product in Eq. 14 is computed along the vector component dimension whereas this is computed along the batch dimension in Eq. 15. Said differently, in Eq. 14 sim computes a square matrix of size (batch size, batch size) (this is a pair-wise similarity matrix between samples) while it is of dimension (feature space dimension, feature space dimension) in Eq. 15 (this is a correlation matrix between vector's coordinate). We refer the reader to Algo. 3 and Algo. 4 for the pseudo-code of the SimCLR and the Barlow regularizers, respectively. 
regularized auto-encoders (raes): 
 To describe the RAE, we use a probabilistic formulation in which q ϕ (z|x) is the recognition model (or the encoder), and p θ (x|z) is the decoder. We train the RAEs by minimizing L RAE (Eq. 1). In this equation, the first term is a reconstruction loss (computed with a ℓ 2 distance), and the second term (L reg ) covers a wide range of regularization losses. L reg includes the representational inductive biases we study in this article.Those inductive biases fall into 3 groups: the standard LDM regularizers, the supervised regularizers, and the contrastive regularizers.EQUATIONStandard regularizers (KL and VQ): The KL divergence in Eq. 2 forces each coordinate of the latent vector to be distributed following a pre-determined distribution (e.g Gaussian distribution, as in the VAE [49] ). The vector quantized loss in Eq. 3 transforms the continuous latent code z into a discrete code z q using the nearest entry in a codebook Z = {e i } K i=1 with the quantization operator: z q = n Z (z) (s.t. n Z : z → arg min ei ∥z -e i ∥ 2 as in the VQ-VAE [51] ). This quantization operation being non-differentiable, backpropagation is achieved using a stop-gradient operation sg[•] to provide a gradient estimator. We provide an extensive mathematical description of the VQ-VAE in App. A.2.1.EQUATIONEQUATIONSupervised regularizers (Classif. and Proto.): The classification regularizer forces discriminative features by minimizing the cross-entropy between the true labels (c) and the softmax of the logits.Here the logits are learned by a linear layer (h CL θ ) stacked on the latent space (Eq. 4). While the classification loss is supervised by the true categorical labels, the prototype-based loss is supervised by the exemplars themselves (as in the Prototypical Net [17] ). The prototype-based loss learns a metric space in which classification can be performed by computing distances between the variations and their corresponding exemplars (i.e., the prototypes)(see Eq. 5). Here, the metric space is linked to the latent space of the RAE through a linear layer (h P R θ ). Intuitively, the prototype-based loss finds an embedding space where the variations will be close (in terms of ℓ 2 distance) from their prototypes. See A.2.2 for more details.EQUATIONEQUATIONContrastive regularizers (SimCLR and Barlow): Contrastive learning algorithms learn representations that are invariant under different distortions (i.e., data augmentations). Here we define two data-augmentation operators, τ A (•) and τ B (•), that transform the variations x into x A = τ A (x) and x B = τ B (x), respectively. We denote z A = q ϕ (•|x A ) and z B = q ϕ (•|x B ) the projection of x A and x B into the RAE latent space, respectively. The SimCLR regularizer is based on the InfoNCE loss: it maximizes the similarity between the representation of a sample and its augmented view while minimizing the similarity with negative pairs (augmented views of different instances) [14] . The Barlow regularizer (as in the Barlow twins [15] ) forces the cross-correlation matrix between z A and z B to be as close to the identity matrix as possible. This causes the embedding vectors of distorted versions of samples to be similar while minimizing the redundancy between the components of these vectors. Said differently, the SimCLR loss shapes the space based on the samples' similarity, while the Barlow operates on the correlation between the features of the samples. For conciseness, we have included the mathematical derivations and details on the data augmentation we used in App. A.2.3.We leverage standard convolutional architectures (from [52] ) to parametrize both the encoder and the decoder. The resulting autoencoder has a 1D bottleneck (d = 128 for QuickDraw-FS and d = 64 for Omniglot). We refer the reader to App. A.3.1 for complete architectural and training details of the RAE. In the rest of the article, we evaluate the impact of these regularizations by exploring the effect of β (see Eq. 1) on LDMs.The LDM second stage is a diffusion model that learns the data distribution in the latent space of the RAE. Diffusion models progressively denoise a pure noise z T ∼ N (0, I) into a clean latent representation z 0 := z through a sequence of partially denoised variables {z i } T i=1 . The goal is then to learn a transition probability p ψ (z t-1 |z t ) that approximates a noise injection operator ν t (.) so that z t = ν t (z 0 ) = √ ᾱt z 0 + √ 1 -ᾱt ϵ ( ᾱt is an hyperparameter of the diffusion schedule, and ϵ a Gaussian noise). The Denoising Diffusion Probabilistic Model (DDPM) [47] reduces the learning of p ψ (z t-1 |z t ) to the optimization of a simple autoencoder ϵ ψ trained to predict the noise from a degraded latent representation z t (see A.4 for mathematical justification):arg min ψ E z0∼q ϕ (.|x) zy∼q ϕ (.|y) ϵ ψ ν t (z 0 ), z y , t -ϵ 2 2 s.t. ϵ ∼ N (0, I) and t ∼ U(1, T ) (6)In Eq. 6, z y denotes the latent representation of the exemplar y. Eq. 6 could be interpreted as a denoising score matching objective [53] , so the optimal model ϵ ψ * matches the following score function:EQUATIONThe autoencoder-like model ϵ ψ (., z y , t) is a 1D Unet conditioned on the time variable t and z y (see A.4.3 for details on the architecture and the training of the Unet). Herein, we use a classifier-free guided version of the DDPM [27] with the following score function:EQUATIONThis formulation introduces a guidance scale γ (we use γ=1) to tune how much the conditioning signal influences the final score. Such a formulation has shown effective in one-shot settings [28, 30] . Note that each term on the RHS of Eq. 8 is computed with the same network ϵ ψ using Eq. 7. ϵ ψ is simply conditioned on a non-informative signal to compute log p ψ ⋆ (z t ). We remind the reader that the training of the diffusion model begins only after the RAE training is complete, and occurs exactly identically, regardless of the type of regularization used. The quality of images generated by the diffusion model thus directly serves to compare the different regularizations. The code to train all described models is available at http://anonymous.4open.science/r/LatentMatters-526B. 
originality vs. recognizabilty: 
 To compare humans and machines in the one-shot drawing task, we first use the originality vs. recognizability framework [31, 30] . This framework leverages 2 critic networks to evaluate the samples produced during the testing phase. The recognizability is quantified using the classification accuracy of a one-shot classifier [17] , while the originality is measured using the average distance between the variations and their corresponding exemplars. This distance is computed in the feature space of a self-supervised model [14] . Importantly, both human-drawn and machine-generated samples are evaluated using the same 2 critic networks. This ensures that any potential biases in the critic networks are minimized, leading to a more balanced comparative analysis. Note that the originality is normalized across all tested models to range between 0 and 1. Here, we use the same originality vs. recognizability framework setting as that used in Boutin et al. [30] . Importantly, the originality vs. recognizability plots should be interpreted based on how close the models are to the human data point (grey star in Fig. 3 ), rather than focusing solely on their individual originality or recognizability scores. In simple terms, a model that effectively mimics human drawings should fall near the human data point. Note also that there is an inherent trade-off between originality and recognizability: while recognizability assesses how likely the data point falls within the classifier decision boundary, originality measures how 'diffuse' the sample distribution is. Therefore a very original agent (producing highly diverse samples) will tend to have a low recognizability as the samples are likely to fall outside of the classifier decision boundary.In Fig. 3 , we first evaluate how increasing the regularization weights (i.e. the β in Eq. 1) for eachNo reg.Classif. regularizer (taken separately) affects the similarity of LDM samples to human drawings. To do so, we report the originality and the recognizability values for LDM samples trained with different β values (see data points in Fig. 3 ). We use a parametric fit (least curve fitting methods [54] ) to illustrate how increasing β affects these scores (see A. 5 for more details on the parametric fit computations). We observe a similar concave shape for all curves. As β starts increasing, the recognizability improves while the originality decreases (except for VQ regularizer). Beyond a certain β value, the recognizability declines, and the originality increases. In particular, the maximum recognizability values for KL and VQ (obtained with β KL = 10 -5 and β V Q = 5) match those of a diffusion model trained in the pixel space and barely exceed those of a non-regularized LDM (see Fig. 3a ). Increasing the weight of the prototype-based regularizer substantially reduces the distance to human compared to the classification regularizer (the minimal distance to human is 0.04 for β P R = 5•10 2 vs. 0.15 for β CL = 5, see Fig. 3b ). Among the contrastive regularizers, Barlow regularization significantly reduces the distance to human compared to the SimCLR one (the minimal distance to human is 0.08 with β BAR = 30 vs. 0.12 with β SimCLR = 10 -2 , see Fig. 3c ). A visual inspection of the samples tends to corroborate these results (see Fig. 2 and A.7 for more samples). We observe similar trends for all tested regularizers on the Omniglot dataset (see A.6).Barlow SimCLR KL VQOverall, our findings indicate that not all regularizers are created equal. For supervised regularizers (see Fig. 3b ), the prototype-based regularizer generates more recognizable samples compared to the classification regularizer. This is expected since the classifier focuses on separating categories in the training set, which may not be ideal for unseen categories in the one-shot setting [19, 17] .In contrast, the prototype-based regularizer clusters samples near their prototypes, leading to less For the LDMs with "standard" regularizers, the β is applied on the KL (L KL in Eq. 2) or on the VQ regularizers (L V Q in Eq. 3). b): For the supervised regularizers, the β is applied on the CL (L CL in Eq. 4) or on the prototype-based regularizers (L P R in Eq. 5). c): For the contrastive regularizers, the β is applied on the SimCLR (L SimCLR in Eq. 14) or on the Barlow regularizers (L Bar in Eq. 15). See A.5 for more information on the range of β we have explored for each regularizer. Larger data points indicate models whose performance is closer to that of humans for each type of regularization. For comparison, we include an LDM leveraging a non-regularized RAE (hexagon marker) and a diffusion model trained directly on the pixel space (cross marker). The human performance corresponds to the recognizability and originality computed on human drawings (shown with a grey star).overfitting and better transferability, which is valuable for few-shot tasks [55] . Our experiments confirm that the prototype-based regularizer generalizes better for one-shot drawing. In Fig. 3c , the Barlow regularization outperforms the SimCLR regularizer in recognizability, likely due to Barlow's effective feature disentangling [15] . These features transfer well to new datasets, making Barlow more suitable for the one-shot drawing task. Overall, our results demonstrate that effective representational inductive biases in few-shot learning also enhance performance in one-shot drawing.We now study the effect of the regularizers when they are used in combination. In particular, we have systematically explored the following combinations of regularizers Barlow + Prototype (Fig. 4a ), SimCLR. + Prototype (Fig. 4b ), KL + Prototype (Fig. 4c ), VQ + Prototype (Fig. 4d ). We observe that the Barlow + Prototype and the KL + Prototype combinations produced the most human-like samples. Those regularizer's combinations are particularly as in both cases the combined recognizability is significantly higher compared to using each regularizer alone. This suggests that clustering samples around their prototypes (using the Prototype regularizer) within a disentangled space (achieved via the KL or Barlow regularizer) enhances generalization. In contrast, the VQ + Prototype and the SimCLR + Prototype combinations show little to no improvements.While the originality vs. recognizability framework allows us to compare human and machine performances in the one-shot drawing task, it does not reveal the strategies each uses to generalize to new categories. To address this, we aim to compare the visual strategies more directly via feature importance maps. These maps emphasize the most salient features to recognize a drawing. Previous research has demonstrated that by summing the absolute values of the diffusion scores (∇ zt log p ψ (z t |z y )) throughout all diffusion steps, one can create heatmaps that highlight salient features in a diffusion model's generation process [30] . Here, we adapt this heuristic to make it compatible with LDMs. This involves projecting each intermediate noisy state (z t ) back to pixel space using the RAE's decoder (p θ (•|z t )). To do so, we use the chain rule, and we multiply each diffusion score by the Jacobian of the RAE decoder w.r.t x t (denoted J log p θ (•|zt) (x t )). For each variation x and its corresponding exemplar y, we can therefore compute a heatmap using Eq. 9 (see A.8.1 for mathematical details). Then, we average 10 of these heatmaps, obtained with the same exemplar but for different variations belonging to the same category. This process allows us to mitigate intra-class variations while focusing on category-specific features. We call this average the feature importance map (see A.8.2 to visualize feature importance maps).EQUATIONWe derived human feature importance maps using psychophysical data from Boutin et al. [30] (data shared by the original authors). The authors collected human saliency maps through an online psychophysics experiment based on a similar protocol to the ClickMe experiment [56] . In this experiment, participants were presented with drawings and were asked to draw on regions important for categorization (see App. S in [30] for more details on the experimental protocol). We averaged the heatmaps across participants and drawings within the same category to obtain the feature importance maps we compared with those of machines (see A.8.3 for visualizing feature importance maps).In Fig 5,  we compare humans and LDMs feature importance maps. For each regularizer, we select the LDMs that produce the most human-like sketches (highlighted with larger data points in Fig. 3 ). Note that we exclude the VQ-regularized LDM from this analysis because it produces irrelevant feature importance maps, possibly due to the non-differentiability of the quantization process (see Fig. A.15) . In Fig. 5a , we showcase examples of the obtained feature importance maps for all other LDMs' regularizations (see also A.8.2) and for humans (see also A.8.3). We qualitatively observe that the LDMs regularized with the Barlow and the prototype-based objectives tend to focus on sparse features. This particular aspect seems to be shared with the human feature importance maps. We compute the Spearman rank correlation to quantify the similarity between human and machine feature importance maps (see Fig. 5b ). To make sure that the correlation comparison between the different LDMs is significant, we have computed pairwise statistical tests (Wilcoxon signed-rank test, see A.8.4). Our results show that all considered regularizations correlate significantly more with human feature importance maps than non-regularized LDMs. In addition, the prototype-based regularizer produces the feature importance maps with the highest correlation with humans and is significantly above all other tested regularizations (p < 10 -3 ). In the human-alignment ranking, the Barlow-regularized LDM follows the prototype-based LDM, also showing a significantly higher Spearman correlation coefficient than KL, classification, SimCLR regularizers (p < 10 -3 ). All other pair-wise statistical tests (between KL, classification, SimCLR) are not significant enough to draw a meaningful ranking. 
conclusion: 
 In this article, we used Latent Diffusion Models (LDMs) to study the effect of representational inductive biases for one-shot drawing tasks. We explore 6 different regularizers: KL, vector quantization, classification, prototype-based, SimCLR and Barlow regularizers. We analyzed the human/LDMs alignment from two (independent) perspectives: their performance relative to humans on the one-shot drawing task (with the recognizability vs. originality framework in section 5.1) and the similarity of the underlying visual strategies (with the feature importance maps in 5.2). Overall, we observe a clear alignment between the 2 analyses on the following points:• All regularized LDMs have an optimal regularization weight (β) where they are more aligned with humans than their non-regularized counterparts.• The prototype-based regularizer is showing the best matches with human performance and attentional strategy. • In the one-shot drawing tasks, the samples' human-likeness could be further improved by combining the prototype-based regularizer with either the KL or the Barlow regularizers.In conclusion, we observe that all representational inductive biases "are not created equal". However, some of them (prototype-based and Barlow regularizers) do narrow the gap with humans in the one-shot drawing task. 
limitations: 
 In this article, we tested six representational inductive biases, a small number considering the extensive range available in the representation-learning literature. This field encompasses hundreds of inductive biases that have proven successful in one-shot classification tasks. Therefore, other representational inductive biases might align better with human performance, both in terms of sample similarity and visual strategy. Our goal wasn't to test all possible biases but to demonstrate that some of them can significantly narrow the gap with humans in one-shot drawing tasks.Another limitation of this article lies in the recognizability vs. originality framework we are using to evaluate the drawings. This framework leverages 2 critic networks to evaluate the sample's originality and recognizability. There's no guarantee these networks align with human perceptual judgments. Thus, the recognizability and originality scores might not reflect human perception accurately. However, since both human and model outputs are evaluated using the same pre-trained critic networks, the comparison remains fair. 
discussion: 
 It is noteworthy that the KL and VQ regularizers, commonly used to train LDMs on natural images (as in StableDiffusion [29] ) are not the best-performing regularizers in the one-shot drawing task. Our study indicates that the prototype-based and the Barlow regularizers, not tested yet on LDMs trained on natural images, hold a significant potential for enhancing their one-shot ability. Interestingly, the 2 inductive biases that align most closely with humans are directly related to prominent neuroscience theories. The prototype-based objectives provide an instantiation of the prototype theory of recognition and memory [57] [58] [59] [60] [61] , suggesting that humans use prototype similarity to recognize novel objects. Similarly, the Barlow regularization is inspired by Barlow's redundancy reduction theory [62, 63] , which posits that the brain encodes statistically independent features to eliminate redundancy (and minimize energy consumption). The effectiveness of these regularizations provides hints that the brain may use similar inductive biases to generalize to new categories. In terms of brain inspiration, although we use LDMs to model humans' one-shot generation abilities, we do not claim that these neural networks constitute a realistic model of brain processes. It is indeed unlikely that humans generate samples by iteratively denoising random noise. More biologically plausible generative models might further help to obtain better models of human behavior (e.g., see [64] [65] [66] [67] [68] ).With this paper, we highlight how specific representational inductive biases, included in the input space of generative models, can help bridge the gap with human capabilities. We believe these biases will allow advanced models to generalize and create as effectively as humans do, leading to exciting advancements in technology and creativity. 
 
 Appendix: A.1 QuickDraw-FS datasetThe QuickDraw-FS dataset is built from the samples of the Quick, Draw ! challenge [50] . In this online experiment (https://quickdraw.withgoogle.com), participants have to draw an object when presented with the category name. The resulting dataset is made of 345 object categories, with approximately 150, 000 drawings per category. The experimental protocol of the Quick, Draw ! challenge forces the participants to produce drawings that are semantically related to the category name, but those drawings do not necessarily represent the same visual concepts. For example, the "alarm clock" category includes digital and analogic types of alarm clocks, which represent 2 different visual concepts (see Fig. Let us define a codebook Z = {e i } K i=1 made of K elements (also called codewords). Each codeword has a dimension s : e i ∈ R s . The Vector-Quantized Variational AutoEncoder (VQ-VAE) [51] can be decomposed into 3 stages: i) an encoder q ϕ (z|x) mapping the input data x to a continuous latent vector z ∈ R d , ii) a discretizing operator denoted n Z (z) which transforms z into a discretized latent vector z q , and iii) a decoder p θ (x|z q ) mapping z q to a reconstructed image x. The discrete latent code z q is calculated using a nearest-neighbor look-up in the codebook Z (see Eq. 10). Said differently, each element of the continuous latent vector z i is replaced by the nearest e j in the codebook (here the i index corresponds to the i-th coordinate of z):EQUATIONz q could then be transformed into a discretized vector by mapping each codeword with its corresponding address in the codebook (e j → j). Note that this quantization process is equivalent to defining a posterior distribution following a K-way categorical distribution [51] .To learn the resulting networks, one naive way would be to minimize the following loss function :EQUATIONEq. 11 is a reconstruction loss in which the information first flows through the quantized encoder, (i.e. n Z (q ϕ (.|x))), to then produce a reconstructed image (i.e. log(p θ (x|z)).However, Eq. 11 cannot be directly optimized as it has no real gradient (the arg min function is not derivable). To minimize this loss function, the gradient is then approximated using a straight-through estimator [36] . The straight-through estimator involves copying the gradients from the decoder input to the encoder output. We refer the reader to line 5 in Algo. 1 for practical implementation of the straight-through gradient estimator. Intuitively, since z is supposed to be very close to z q , the gradient contains meaningful information for how the encoder has to change to minimize the reconstruction loss. During inference, the nearest embedding z q is computed using Eq. 10 and then fed to the decoder. Due to the straight-through operation, the codebook Z does not receive any gradient information from the reconstruction term. Therefore, the codebook is learned with the simplest dictionary learning algorithm that involves minimizing the ℓ 2 distance between the quantized vector z q and the continuous one z (i.e. ∥z -z q ∥ 2 2 ). This quantity cannot be directly minimized because there is no gradient flowing from z q to z. To mitigate this issue, it is replaced with the estimator term ∥sg[z q ] -z∥ 2 2 + ∥z q -sg[z]∥ 2 2 . The full VQ-VAE loss is described in Eq. 12: :EQUATIONThe following pseudo-code illustrates how the VQ-VAE is usually implemented (see Algo. 1). We follow a similar implementation:Algorithm 1: VQVAE pseudo-code Input: dataset D, model parameters π = (θ, ϕ, Z) 1 for x in D do 2 z = q ϕ (z|x) # encode 3 z q = n Z (z) # quantize 4 L reg = ∥sg[z q ] -z∥ 2 2 + ∥z q -sg[z]∥ 2 2 # Quantization loss, sg[.] = stop gradient 5 z q = z + sg[z q -z] # straight-through gradient estimator 6 x = p θ (x|z q ) # decode 7 L = ∥x -x∥ 2 2 + L reg 8 π ← ∂L ∂π A.2.Input:dataset D = {x}, model parameters π = (θ, ϕ) # x: variations 1 for (x, y) in D do 2 x A = τ A (x) # augment x in x A 3 x B = τ B (x) # augment x in x B 4 z A = q ϕ (z A |x A ) # encode x A 5 z B = q ϕ (z B |x B ) # encode x B 6 L reg = L SimCLR (z A , z B ) # see Eq. 14 7 x = p θ (x|z A ) # decode 8 L = ∥x -x∥ 2 2 + L reg 9 π ← ∂L π Algorithm 4: Barlow regularizer pseudo-code Input: dataset D = {x}, model parameters π = (θ, ϕ) # x: variations 1 for (x, y) in D do 2 x A = τ A (x) # augment x in x A 3 x B = τ B (x) # augment x in x B 4 z A = q ϕ (z A |x A ) # encode x A 5 z B = q ϕ (z B |x B ) # encode x B 6 L reg = L Bar (z A , z B ) # see Eq. 15 7 x = p θ (x|z A ) # decode 8 L = ∥x -x∥ 2 2 + L reg 9 π ← ∂L πAugmentations: The augmentations we use are the same for both regularizers (i.e. τ A (•) and τ B (•)), they are randomly picked among the following transformations:• Random resized crop: with a scale parameter ranging from (0.1, 0.9) and a ratio parameter ranging from (0.8, 1.2). The scale parameter tunes the upper and lower bound of the cropped area, and the ratio parameter defines the lower and upper bound for the aspect of the ratio of the crop. • Random affine transformation: with a rotation parameter varying from (-15 • to 15 • ), a translation (from -5 pixels to 5 pixels), a zoom (with a ratio from 0.75 to 1.25) and a shearing (from -10 • to 10 • ) • Random perspective transformation: apply a scale distortion with a certain probability to simulate 3D transformations. The scale distortion we have chosen is 0.5, and it is applied to the image with a probability of 50%In this section, we describe the mathematics behind the latent diffusion models. The following mathematical derivations are mostly derived from Sohl-Dickstein et al. [26] , Song and Ermon [25] , Ho et al. [47] , Rombach et al. [29] and are adapted to match the one-shot generation task and the notations of this paper. Those mathematical derivations are not necessary to understand this article but we include them to make it self-contained.Herein, we consider a pretrained Regularized AutoEncoder, with an encoder q ϕ (z|x) and decoder p θ (x|z) that map the input x ∈ R D to a latent representation z ∈ R d (d ≪ D) and inversely, respectively. In the following, we will call indifferently z or z 0 the latent variable corresponding to the input x. We will also call z the latent variable associated with the exemplar y. The goal of a diffusion model in a one-shot latent diffusion algorithm is to learn the conditional probability of z 0 given the latent representation of the exemplar z y , we call this probability distribution p ψ (z 0 |z y ).Diffusion models learn the transformation of a pure noise, called z T ∈ R d , into a fully denoised latent representation z 0 ∈ R d . This transformation is progressive, through a sequence of partially denoised latent representations 1) . In this sequence z t+1 is therefore sligthly more noisy than z t . The idea behind the diffusion model is to learn the transition probability p ψ (z t-1 |z t , z y ). To do so, diffusion models introduce a tractable noising process r(z t |z t-1 ) that gradually injects noise in the latent representation. An illustration of such a directed graphical model is shown in the forward (i.e. noise injection) and the reverse processes (i.e. noise removal), respectively. z y and z 0 are the latent representations of the exemplar image y and the image x, respectively (exemplified with skull drawings). z i corresponds to the sequence of partially corrupted latent representations. z y and z 0 are obtained using the RAE encoder q ϕ (z|x) and can be mapped to the input space using the RAE decoder p θ (x|z). The 'dummy' distributions located on top of the z i variables, illustrate the noise injection process, starting from an 'informative' multimodal distribution to a fully 'uninformative' Gaussian distribution.{z i } T -1 i=1 ∈ R d×(T -Here we describe, in mathematical terms, the noise injection process : (17) In Eq. 17, β t tunes the step size of the diffusion process. Using the successive product of Gaussian, this process could be reduced to a tractable noising operator ν t (.) that injects the right amount of noise at time t to obtain z t from z 0 :r(z 1:T |z 0 ) = T t=1 r(z t |z t-1 ) with r(z t |z t-1 ) = N (z t ; 1 -β t z t-1 , β t I) s.t. {β t ∈ (0, 1)} T i=1EQUATIONOne could then express the probablity of z t given z 0 in a closed form:EQUATIONThe denoising probabilistic process, recovering the latent representation z 0 from noise, could be parametrized as follows:EQUATIONwithp ψ (z t-1 |z t , z y ) = N (z t ; µ ψ (z t , t, z y ), σ 2 t I) p ψ (z T |z y ) = s(z T ) = N (0, I) A.4.The DDPM model we leverage is a 1D-UNet to perform the diffusion process over the latent embeddings. The architecture of the UNet is described in Herein we systematically vary the β parameter in Eq. 1 for each type of regularization and we evaluate its effect using the originality vs. recognizability framework. To visualize this effect while maintaining the order of the hyper-parameters, we use the parametric fit method described in [54] . This technic involves 2 simultaneous parametric fit: i) a polynomial fit (degree 2) between the hyperparameters and the originality values (shown in Herein we evaluate a LDM leveraging a RAE trained with the following loss (with L V Q ) in Eq. 3: EQUATIONHerein we evaluate a LDM leveraging a RAE trained with the following loss (with L P R ) in Eq. 5: Here we present a curve similar to Fig. 3 but for LDMs trained on the Omniglot dataset. We were unable to train a VQ-VAE with reasonable performance on this dataset, so we have excluded the VQ-regularized LDM from Fig. A.10. We believe this issue is due to improper hyperparameter tuning as the same regularizer works reasonably well on the QuickDraw-FQ dataset. We are actively working to resolve this problem.EQUATIONExcept for the VQ regularizer, we observe that all other regularizers follow a similar trend to those trained on the QuickDraw-FS dataset. In particular, the prototype-based and the Barlow regularizers outperform all others. We remind that p θ (x|z) is the decoder of the RAE, and that p ψ (z t-1 |z t , z y ) is the transition probability learned by the diffustion model. To make the mathematical derivations more concise, we define the following function :EQUATIONTo project each intermediate noisy state z t into the pixel, we feed them into the decoder. The resulting projection isx t = p θ,ψ (z t ) = p θ • p ψ (z t )For each time step of the diffusion process, the importance feature map quantifies how the absolute value of p θ,ψ (z t ) changes when one varies z t . ϕ(x, y) describes the accumulation, over all time steps, of these "local feature map":EQUATIONEQUATIONwith J p θ (x t ) the Jacobian of the function p θ w.r.t x t computed in p ψ (z t ). If we trade the functional notations for probabilistic ones we have:EQUATIONThe LDMs' feature importance maps have been computed on 25 different categories, for each of the six different regularization methods discussed in the paper. The feature maps were calculated by taking the average of n = 10 misalignment maps ϕ(x, y) as defined in Eq. 9. All shown feature importance maps correspond to the LDMs, for each regularizer, showing the shortest distance to humans. They correspond to larger data points in Fig. 3 . For comparison, feature importance maps have also been computed for humans for the same 25 categories. For humans, the feature importance maps are heatmaps representing the likelihood of a pixel being selected by a participant as part of the ClickMe-QuickDraw experiment (further details on the experiment provided in App. S of Boutin et al. [30] ). The same image used to calculate the misalignment maps for the LDMs is presented to the participants during the CliCkMe-QuickDraw experiment. Human consistency: To evaluate how humans agree with each other on the feature importance maps, we computed the human consistency. To do so we use a bootstrapping technique. For each category, we divided the participants into 2 populations (randomly selected), obtaining approximately 25 annotations (heatmaps) coming from different participants for each category. We then average those annotations within the same population (and the same category) to form population-wise feature importance maps. We finally compute the human consistency with the Spearman correlation between those population-wise feature importance maps. We obtain a spearman of 0.8845 (p < 5.10 -2 ).In this article, we tested six representational inductive biases, a small number considering the extensive range available in the representation-learning literature. This field encompasses hundreds of inductive biases that have proven successful in one-shot classification tasks. Therefore, other representational inductive biases might align better with human performance, both in terms of sample similarity and visual strategy. Our goal wasn't to test all possible biases but to demonstrate that some of them can significantly narrow the gap with humans in one-shot drawing tasks.Another limitation of this article lies in the recognizability vs. originality framework we are using to evaluate the drawings. This framework leverages 2 critic networks to evaluate the sample's originality and recognizability. There's no guarantee these networks align with human perceptual judgments. Thus, the recognizability and originality scores might not reflect human perception accurately. However, since both human and model outputs are evaluated using the same pre-trained critic networks, the comparison remains fair.Our approach leveraged two-stage generative models: the first stage compresses information and shapes the latent distribution with representational inductive biases (the RAE), and the second stage learns this latent distribution (the diffusion model). This type of architecture takes longer to train because it requires two separate training procedures. However, this limitation could be overcome by using an end-to-end training procedure for Latent Diffusion Models, which could streamline the process [71] .All the experiments of this paper have been performed using Quadro-RTX600 GPUs with 16 GB memory. The training time for the RAE is approximately 24 hours and 72 hours for the Diffusion model (96 hours overall). Note that as we have explored a large range of hyperparameters for all types of regularization, our paper is relatively extensive in terms of computations (600 models have been trained overall, but just a small part of them have been used in this article).This work does not present any foreseeable negative societal consequences. We think the societal impact of this work is positive. It might help the neuroscience community to evaluate the different mechanisms that allow human-level generalization and then better understand the brain.1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: Our main claim is that representational inductive biases in LDMs help to close the gap with humans on the one-shot drawing task. This claim is experimentally verified in Fig. 3 and Fig. 5 . Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 


Decision: While use of a combined evaluation of originality and recognizability is not novel in the field, the application of this evaluation framework to human sketches is a non-trivial contribution.  The testing of multiple regularizers, and the addition of combined regularizers, is likely to be impactful for the field of ML models of human-like sketching and character writing.  While the paper could be strengthened by adding evaluation of stroke features or a human evaluation of the human-likeness of the sketches, or a deeper analysis of the performance of the regularizers, I think the paper provides enough of a contribution to be useful to the field as it currently stands.

Reviews: review: paper_summary: paper_summary: This paper investigates how different representational inductive biases in Latent Diffusion Models affect their performance on one-shot drawing tasks, aiming to close the gap with human abilities. The authors explore six regularization techniques: KL divergence, vector quantization, classification, prototype-based, SimCLR, and Barlow twins. They evaluate these models using the originality vs. recognizability framework and a novel method for generating feature importance maps. The results show that prototype-based and Barlow regularizations significantly narrow the gap between LDMs and human performance in one-shot drawing tasks. These regularizers outperform standard LDM regularizers (KL and vector quantization) as well as classification and SimCLR regularizers. The authors also demonstrate that the feature importance maps of LDMs with prototype-based and Barlow regularizations align more closely with human attentional strategies. Additionally, they find that combining these two regularizers yields even better results. The study highlights the potential of incorporating specific representational inductive biases in generative models to achieve more human-like generalization capabilities, with implications for both AI advancement and understanding human cognitive processes.
strength_weakness: strength_weakness: - The paper doesn't provide a detailed analysis of how different components of the regularizers contribute to the overall performance. This makes it challenging to understand which specific aspects of each regularizer are most crucial.
- While the authors use human-derived feature importance maps, they don't include a human evaluation of the generated samples. Such an evaluation could provide additional insights into the perceived quality and human-likeness of the generated drawings.
- Although the authors briefly mention combining prototype-based and Barlow regularizers, this aspect is not thoroughly explored. A more systematic investigation of different regularizer combinations could potentially yield even better results.
questions: questions: - How well do your findings generalize to more complex datasets beyond simple sketches? Have you considered testing your approach on datasets with more detailed or realistic images?
limitations: limitations: N/A
paper_summary: paper_summary: i)This paper uncovers the impact of representational inductive biases on Latent Diffusion Models through one-shot tasks, particularly in the realm of human-like sketching. It compares three distinct groups of regularizers: a standard baseline, supervised methods, and a third group consisting of self-supervised techniques.

ii)This paper aims to uncover the strategies employed by each regularization method to generalize to novel categories.

iii)This paper conducts a comprehensive comparative analysis of the effectiveness of different regularization methods in one-shot drawing tasks. The study explores various dimensions, examining the performance and differences of various regularization strategies in such tasks.
strength_weakness: strength_weakness: i)This paper conducted an extensive analysis of the impact of different inductive biases in one-shot drawing tasks. However, the analysis remains superficial, merely briefly revealing the experimental results of various methods without delving into the fundamental reasons behind the differences in outcomes among the methods. Furthermore, the paper does not propose specific solutions to the research questions addressed.

ii)The experimental method adopted in this study is not limited to specific modalities and is not restricted by data scale. This method may be applicable to the generation tasks of more modalities(such as photos or text2img). However, the experiment specifically chose handwriting and sketches as the research subjects for one-shot generation tasks.

iii) The core argument proposed by this paper is the generation of samples that resemble human-drawn sketches or handwriting. However, the paper does not provide sufficient elaboration on how to quantify the “human-like” characteristics of the samples, especially with an in-depth analysis from the perspective of stroke features.
questions: questions: Major:

i)In one-shot drawing tasks, originality constitutes a key evaluation metric. However, when measuring the “human-like” characteristics of samples, the impact of originality is relatively minor, and its role in the evaluation process appears to be more limited.

ii)
Does this paper quantitatively analyze the stroke correlation between the generated sketches and those drawn by humans? Although the generalization curve has been considered in the evaluation of originality and recognizability, the paper does not explicitly reveal the interrelationship between the strokes.

iii)Has this study explored the reasons for or proposed hypotheses about the performance differences exhibited by various inductive bias methods in one-shot drawing tasks?

Minor:

i)Given that classification models may be influenced by their own biases or uneven distributions in the training data, the recognizability of the model does not necessarily equate to the “human-like” level of the samples. Does the paper take this potential issue into consideration?

ii)In the exploration of effective methods to improve the performance of one-shot drawing tasks, did this paper consider approaches other than simply adding the prototype-based and Barlow regularizers with weights?

iii)
In Figure 2, does the “pixel baseline” refer to the use of rasterized sketches during training or testing? On this point, this paper could analyze the differential impact of inductive biases when dealing with vector versus raster sketches.
limitations: limitations: No. How the evaluation criteria effectively align with human judgment or aesthetic standards is a question of considerable research value, especially in generative tasks. Particularly when dealing with sketches or handwriting, the consideration of strokes is an indispensable key dimension that cannot be overlooked.
paper_summary: paper_summary: The authors propose to explore how different regularizers impact the performance of LDM on one-shot sketch generation, with a specific focus on evaluating the similarity between the generated sketches and real ones. It reveals that prototype- and Barlow-based regularizers are among the best, and claims the gap between humans and machines in the one-shot drawing task is almost closed.
strength_weakness: strength_weakness: - This paper is more like an incremental work based on [30], the key idea of using the diversity vs recognizability framework, and importance maps to measure the generation quality of diffusion models is the same. It differs in extending the diffusion model into latent feature space and applying different regularizers, which seems minor.
- It is a bit over-claimed that the gap between humans and machines is almost closed on the one-shot drawing task by using LDM plus proper regularizers. The qualitative results shown are not as good as the actual sketches, suffering from clear blur and distortion. The experiments are conducted on relatively simple sketch cases, which makes it hard to justify its scalability.
- DDPM used in [30] for this task is not compared (with or without the same regularizers if possible), it would be helpful to understand how effectiveness of pushing the denoising into latent space.
questions: questions: - It is unclear how to construct the sketch codebook used for VQ-VAE.
limitations: limitations: Please refer to the weaknesses.
paper_summary: paper_summary: This paper investigates how different regularization techniques, applied to the latent space of Latent Diffusion Models (LDMs), impact their performance on one-shot drawing tasks. The Authors explore many regularization methods: KL divergence, vector quantization, classification, prototype-based, SimCLR, and Barlow Twins. 

They evaluate these methods against human performance using quantitative metrics (originality vs. recognizability) and qualitative analysis of feature importance maps. 

The results show that LDMs with prototype-based and Barlow Twins regularizations produce sketches that are most similar to human drawings in terms of both recognizability and originality.
strength_weakness: strength_weakness: Overall, I think that the paper makes a nice contribution to our understanding of how different regularization techniques affect the latent representations in generative models and their ability to produce human-like sketches.

The weakness I see regards the possibility to generalize from the "simple" datasets analyzed to more complex creative processes.
This study primarily focuses on the QuickDraw-FS dataset, with limited exploration of the Omniglot dataset. It not very clear to me how sound can be the extrapolation from these very simple (although relevant) contexts to more complex ones. I know that this does not provide a concrete and actionable insight, but I would appreciate a comment on this.
questions: questions: How generalizable are these findings to more complex drawings?
limitations: limitations: Limitations are adequately discussed, but they are not part of the main text (they are discussed in the Appendix, pag. 34).

Comments: title: Official Comment
comment: The author's comprehensive response in the rebuttal section effectively addresses most of my initial concerns, particularly by delving into the underlying mechanism of the regularizers' excellent performance. Based on this, I am inclined to give this paper a positive evaluation and change my votes to Borderline Accept.
Dear reviewers,

As the rebuttal period is closing, I am obligated to remind the reviewers to respond to the author rebuttals if possible. As reviewers vmMB, ZReU, and 1Pi8 all lean towards rejection, I especially request these reviewers to respond to the rebuttal before the deadline tomorrow (Aug 13 11:59pm AoE) and to indicate whether the rebuttal addressed any of your concerns.

Gratefully,

AC
Should there be any remaining issues, we are more than willing to engage in further discussion.
Should there be any remaining issues, we are more than willing to engage in further discussion.
Should there be any remaining issues, we are more than willing to engage in further discussion.
Should there be any remaining issues, we are more than willing to engage in further discussion.