Synthesizing Images on Perceptual Boundaries of ANNs for Uncovering and Modulating Individual Human Percepts

Authors: Chen Wei, Chi Zhang, Jiachen Zou, Haotian Deng, Dietmar Heinke, Quanying Liu
Primary area: applications to neuroscience & cognitive science
Keywords: Perceptual variability, Diffusion model, Object Recognition, Behavior Modulation, Behavioral Alignment

Abstract:
Human decision-making in cognitive tasks and daily life exhibits considerable variability, shaped by factors such as task difficulty, individual preferences, and personal experiences. Understanding this variability across individuals is essential for uncovering the perceptual and decision-making mechanisms that humans rely on when faced with uncertainty and ambiguity.  In this study, we present a counterfactual-based approach to investigate the subject-level decision-making behaviors and reveal the underlying perceptual mechanisms by synthesizing visual stimuli. First, we developed an efficient generative model that samples along an artificial neural network (ANN)’s perceptual boundary, generating image samples designed to induce high variability in human perception. Using these generated samples, combined with behavioral data from 246 human participants across 116,715 trials, we constructed the varMNIST dataset. Then, we presented a subject-specific fine-tuning approach to align the perceptual variability of ANNs with that of humans. It allows us to successfully predict human decision-making behaviors on varMNIST. Finally, we verified the ability to selectively manipulate individual behaviors by generating tailored controversial stimuli, which highlighted significant inter-subject perceptual variability. Together, our work illuminated key distinctions between human and machine perceptual variability and established an effective strategy for manipulating individual decision-making behaviors. This study paves the way for artificial intelligence models with personalized perceptual capabilities.

Decision: reject
Reviewer scores: 6, 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1729572330530

thread_type:
official_review

<thread_object>
creator: reviewer_vi4h

<content>
summary:
This paper presents a counterfactual generation-based method to study subject-level perceptual variability in humans. To automatically and scalably synthesize stimulus samples for human studies, the authors propose a generative model that samples images from the perceptual boundaries of artificial neural networks (ANNs) and integrates a post-hoc data curation procedure involving human input. The generative model guides the diffusion model using two approaches: (i) uncertainty guidance, which maximizes the entropy of the ANN’s predictions, and (ii) controversial guidance, which maximizes the prediction discrepancy between two different ANN models. Using this method, the paper introduces a synthetic dataset, varMNIST, to systematically investigate subject-level perceptual variability in three areas: (i) human behavioral analysis in image recognition tasks that evoke perceptual variability, (ii) alignment between ANN and human perception, and (iii) subject-level perceptual variability in response to controversial stimuli.

strengths:
1. The paper provides interesting results from human studies, particularly in aligning perceptual boundaries between ANNs and human perception.
2. It introduces a novel end-to-end system to generate datasets for cognitive studies on human perception and its relationship with vision ANNs.
3. The paper conducts diverse empirical evaluations, complemented by human studies, to analyze the proposed datasets and uncover insights into human perception.

weaknesses:
1. The technical contribution of uncertainty and controversial guidance is marginal, as these concepts already exist in prior works, and the paper does not offer significant system engineering innovations.
2. The post-hoc analysis of human studies lacks rigor. The findings on human perception are not supported by statistical significance, and the empirical results are not discussed in depth. For example, the paper notes inter-subject and subject-group differences in controversial stimuli but does not explore the factors contributing to these perceptual differences.

questions:
- The formulation in lines 185 and 192 seems inconsistent. The left-hand side introduces $f_{\phi}(\cdot)$, but the right-hand side does not include $f_{\phi}$​ or the parameter $\phi$. It would be helpful to rewrite these equations more formally.
- The cluster analysis results in eight distinct clusters, suggesting high perceptual variability among participants. However, what characteristics of the participants contribute to this variability? Is it due to demographic attributes, experimental environment, or purely biological differences? Additionally, why does the analysis yield eight clusters?
- When interpreting results from human experiments, statistical significance should be considered. For example, in Figure 5, the difference between the "finetuned" and "subject-finetuned" results is minimal, yet the paper describes this as a "significant difference in perceptual variability across participants." To ensure validity in human studies, a stricter approach with p-values would be appropriate.
Could you clarify when uncertainty guidance and controversial guidance are used? Are they applied simultaneously or separately to generate images? In line 255, the paper states that images are generated using both guidance methods. Does this mean they are integrated to create a single image, or are they used separately to generate different subsets of images? If the latter, has there been any quantitative ablation study comparing their effectiveness in generating ambiguous samples for human perception?
- As noted in the paper, many prior works on adversarial examples and counterfactual explanations also leverage diffusion models to create natural samples. This setup is not novel.
- The technical novelty of this work is limited. The use of classifier guidance in diffusion models for conditional generation is a well-known technique, and post-processing samples with human evaluation is a standard dataset curation method. While the paper highlights uncertainty guidance, controversial guidance, and the end-to-end system as contributions, the controversial guidance was previously introduced by Golan (2020), and uncertainty guidance resembles untargeted adversarial attacks, which also occur at the classifier’s decision boundary, yielding high-entropy predictions. From a systems perspective, the paper does not offer system-level engineering innovations to enhance the generation process. Therefore, the main contribution lies in the problem addressed and its application to cognitive studies of human perception.
- There is a typo in line 310: "perveptual" should be corrected to "perceptual."
</content>
</thread_object>

</thread>
<thread>
creation date:
1729759228876

thread_type:
official_review

<thread_object>
creator: reviewer_lgkn

<content>
summary:
The paper is concerned with understanding variability in perceptual judgments between different people. It introduces a method for generating controversial stimuli, i.e. stimuli that elicit different classifications from different neural networks, in order to generate a dataset of human behavioral responses with high variability (varMNIST). Different classifiers are then finetuned on individual subjects' responses in varMNIST. These subject-specific classifiers are then used to generate a new set of controversial stimuli tailored to differences between individual subjects.

strengths:
The idea of generating a dataset with the goal of having high variability between human behavioral responses is compelling and the paper nicely combines controversial stimuli and diffusion-based image generation to achieve this. I am very fond of the general approach of analyzing inter-individual variability instead of simply using neural networks to predict average human behavior and I think this is a promising direction. Overall, I found the paper well written and easy to follow. It starts with a clear introduction and description of what it sets out to do and Figure 2 really helps to understand the general idea of the dataset generation.

weaknesses:
Unfortunately, I found it hard to understand some of the specifics of the methods and analyses, leading to several points of confusion and remaining questions, which are listed below and under "Questions". This also affects the reproducibility of the results, which I think would be really difficult to do just from the description of the methods in the paper, especially without any code.

Furthermore, I am not convinced that the results presented in the paper actually deliver on some of its central claims, that the generated images "more closely resemble natural images" or that the authors successfully "modulated individual behavior, unveiling significant inter-individual differences in perceptual variability". Details on these points also follow below. I admit that some of this might be due to me misunderstanding details of the analyses performed in the paper, which the authors might be able to clarify.

1. At first, I was confused about the dataset construction, in particular about what is taken as "ground truth" for varMNIST. After reading Section 3 "Human experiment and dataset construction", I got the impression that varMNIST contains 4,741 images, each with a label. After looking at Fig. 5b and reading Appendix A.3 and A.4, I think I understood that varMNIST contains a data point per subject per image, resulting in  102,021 labels in total. This should be clarified in the main text.
2. The abstract states that varMNIST contains 242,900 images, while Appendix A.4 suggests that it contains 102,021 after additional filtering. It is not clear which number applies to the actual dataset.
3. The discussion claims that "by utilizing carefully designed controversial stimuli, we selectively modulated individual behavior". I assume that this is about the controversial stimuli that were designed based on the subject-finetuned models in Section 6. I agree that this kind of analysis is a promising idea to investigate inter-individual variability in behavior. However, if I understand correctly, the newly generated stimuli are not actually tested in behavior. The way it is currently written, the paragraph "Controversial stimuli between subjects exhibit typical patterns" is misleading. It reads as if it is about disagreements between subjects in the experiment, but it is actually based on disagreements between models (which is clarified in the caption for Fig. 7). This is problematic because the subject-finetuned models seem to have a very low accuracy for quite a few subjects (see Fig. 5b). This should be clarified in the main text and the limitations of the analysis should be discussed. Whether the controversial designed for the subject-finetuned networks actually agree with the perceptual judgments of those subjects is an empirical questions that could be tested with a follow-up experiment. 
4. The discussion of related work in the Methods section "Diffusion model as a regularizer to introduce prior information" seems a bit imprecise. In particular, it states that "a common issue was that the generated images lacked sufficient naturalness and failed to significantly influence human perception". But taking a quick look at one of the cited references (Gaziv et al., 2024), it seems that the generated images, while not necessarily being natural, did significantly influence human perception. The same is true for some of the other cited studies.
5. The claim that the proposed method results in images that "more closely resemble natural images" compared to previous methods for controversial stimuli is doubtful. Previous methods (e.g. Gaviz et al., 2024), which the authors of the present paper criticize as unnatural, are applied to photographic images like ImageNet. The present study is concerned with MNIST-style handwritten digits. It is not clear how the results transfer to photorealistic domains and if the diffusion-based approach would also lead to more "natural" images in these cases.
6. A couple of points about data visualization, which sometimes make it hard to follow the arguments made about the data.
    a. If I am not mistaken, Fig. 5 presents the same data in three different ways (average accuracy bar plot, single-subject accuracy point plot and kernel density). This could be simplified with something like a rainclould plot (https://wellcomeopenresearch.org/articles/4-63), which contains a point plot, kernel density estimate, and the average in a single plot.
    b. The color scheme is confusing. In Fig. 5a, different shades of red and blue indicate different models, while in Fig. 5b, some of the same colors are used to indicate training schemes. In Fig. 5c, the same traning schemes are then represented by different colors. In Fig. 6, again the same colors are used to represent out-cluster, in-cluster and subject-level predictions, and in Fig. 7 they represent different datasets. My recommendation is to use different colormaps to represent different concepts / distinctions.

Minor points:
- Fig. 7b is not referenced in the text
- The paragraph "Controversial stimuli between subjects constructed by finetuned models. " states that controversial stimuli were generated "using the algorithm in Section 3.1" (l. 401-402): There is no Section 3.1, so which algorithm was actually used?

questions:
- About the analysis of the performance of the finetuned models (Fig. 5)
    - Was the varMNIST dataset split into a training and test set? Or were the models evaluated on the same images and labels on which they were finetuned? In other words, how can we be sure that the models are not overfitting on the individual subject data?
    - Does the "subject-finetuned (varMNIST-subject)" evaluation mean that each model that was finetuned on an individual subject's data was then tested on the data of that subject?
- Figure 6b: for some of the subjects, the subject-finetuned model performs worse at predicting in-subject compared to in-cluster. What is the explanation for this?
- How was the parameter $\alpha$, which trades off the entropy or controversy loss with the MSE loss, set?
- "we found that experimental images with response times between 300 and 5000 ms and entropy values between 0.5 and 2.5 best met our experimental requirements" (l. 716-717). What exactly were the experimental requirements?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730665117639

thread_type:
official_review

<thread_object>
creator: reviewer_squf

<content>
summary:
This paper presents a novel approach to studying human perceptual variability using artificial neural networks (ANNs). The authors develop a generative model that samples from the perceptual boundaries of ANNs to create synthetic MNIST images, inducing high variability in human perception. This method facilitates the construction of the varMNIST dataset, which is used for behavioral experiments involving 346 participants and over 240,000 trials. The research highlights that aligning the perceptual variability between ANNs and humans can predict human decision-making.

strengths:
This paper presents an innovative approach to studying human perceptual variability by generating synthetic MNIST images from ANN perceptual boundaries, which are useful for extensive human behavioral experiments.

weaknesses:
1. Scope of the dataset: The proposed method is only evaluated on MNIST, which limits the generalizability of the findings to real-world applications. While MNIST is effective for the initial exploration of perceptual variability, it does not capture the complexity inherent in more diverse visual datasets. This restricts the study's ability to address more nuanced or complex perceptual phenomena that may arise in real-world visual recognition tasks. For instance, datasets that include natural scenes, complex objects, or varied lighting and perspectives could provide more robust insights into human perceptual variability and ANN alignment.
2. Lack of quantitative comparison to baseline methods: Although the paper discusses the limitations of baseline methods in the related work section, it does not provide a quantitative comparison to those baseline approaches. This omission makes it difficult to objectively assess how much better sampling from the perceptual boundaries of ANNs performs in terms of aligning with human perceptual variability. A quantitative comparison would strengthen the paper's claims by demonstrating measurable improvements over baseline methods.
3. Trade-off between alignment and performance: While the paper shows that fine-tuning ANNs with human behavioral data improves perceptual alignment, there is a trade-off in performance on standard classification tasks, as pointed out by the authors. This suggests that optimizing models for perceptual alignment with human behavior may come at the cost of reducing their effectiveness for standard tasks.

questions:
See Weaknesses
</content>
</thread_object>

</thread>
<thread>
creation date:
1730714714147

thread_type:
official_review

<thread_object>
creator: reviewer_vtkk

<content>
summary:
The aim of this paper is to generate ambiguous digit-like images to study perceptual processing in humans. To this end, the authors propose a diffusion-based method for generating counterfactual digit-like images that produce conflicting predictions from humans; the diffusion process is guided by two kinds of signals, termed the uncertainty signal and the controversial signal. The uncertainty guidance encourages the diffusion model to produce images for which an AI visual classifier is confused between two labels for the images, while the controversial guidance encourages generation of images where two different classifiers produce diverging labels for each image. Further, the diffusion is guided by a mean-squared error loss to encourage generated images to be like real handwritten digits. This method is shown to be effective, as the generated images are stylistically similar to real images, but are confusing for humans to classify. Behavioral experiments conducted by the authors corroborate these claims. The authors term these generated images, together with the human classification data, the varMNIST dataset. Analysis of human prediction patterns for images from the varMNIST dataset shows slight positive correlation between an AI classifier’s prediction entropy and the entropy of human predictions on varMNIST images. By training AI classifiers using the human predictions, the authors showed that the varMNIST dataset can be used to identify controversial stimuli which reveal patterns of individual differences between subjects in predicting such ambiguous images.

strengths:
The paper is well-motivated by existing work in cognitive science looking to study human perception with ambiguous stimuli. It also follows a recent body of work using AI models to study human perception. In this regard, the paper is tackling an interesting and important problem. There are several strengths of this paper, the most important of which are described below:

* The synthetic visual stimuli generated by the diffusion model using uncertainty and controversial guidance are of high quality — examples in the paper are stylistically similar to real digit-like images while evoking confusion among human subjects. Further, there seems to be significant variation in the degree of confusion (measured by entropy of human predictions) generated by the images. This kind of variability is highly desirable for such datasets which are intended for cognitive science research into human perception. 
* Analysis in the paper shows that there is some degree of correlation between the entropy of visual AI classifiers and human classifiers on the images from the varMNIST dataset. This shows that the (uncertainty or controversial)-guided diffusion process generates synthetic images which capture some aspects of human perceptual boundaries.
* The authors’ use of the varMNIST dataset and the predictions from subjects to generate controversial stimuli that uncover individual differences amongst subjects is fascinating. 
* The authors do a good job situating the paper in the context of existing work. The Introduction and Related Work sections do a good job putting this work in the context of existing research, while the Discussion section does a good job describing the contributions, implications and limitations of the work.

weaknesses:
* The paper only considers the generation of ambiguous digit-like images, whereas for the purpose of human perception, natural images are much more relevant. In fairness, this weakness is acknowledged by the authors; nevertheless it remains a weakness.
* Important implementation details, such as the network and architecture details for the image generation, are missing in the paper. For the diffusion process itself, the appendix states that the authors used a pre-trained diffusion model. However, the architecture and the pretraining dataset are not described. Further, for the guidance signals, the architectures of the classifiers $f_1$ and $f_2$ have not been discussed. This significantly hampers both the reproducibility of the paper and the soundness of the work. 
* Behavioral experiments require carefully designed checks to ensure that the dataset is balanced and not biased towards specific stimuli. Further, statistical analysis must be done after data collection to ensure that no bias inadvertently makes it into the dataset. The paper is lacking on both those counts. Some of these concerns are listed below:  
    * The distribution over the 10 digits to be reference images in the final 4741 images is not presented. How many of the 4741 were generated using ’0’ or ‘1’ as the reference image? How many of these images were generated by uncertainty guidance and how many by controversial guidance?
    * The authors claim that the generated images have a random distribution over the range of entropies. This claim however is not supported by statistical tests. The histogram can be misleading since they are notoriously sensitive to the choice of parameters like the bin size.
    * In the random trials, how are the images to be displayed to subjects selected? Were there choices made to ensure that a subject sees the same number of synthetic images generated by using ‘0’ as the reference digit as the number of synthetic images generated by using ‘1’? Any kind of accidental subject-level bias can strongly affect patterns of individual differences studied later in the paper.

* The description for the finetuning process was quite unclear. 
    * Were the labels of the reference images used as the correct labels to calculate the accuracy on varMNIST? I’m not sure about such a choice because the motivation for the dataset is to generate ambiguous images and having fixed labels for such images seems counterintuitive.
    * What are the targets used to finetune the models when using population-level data? Is it the average over predictions for a particular image over all subjects?
 
* The writing in the paper can be improved to improve clarity and readability of the paper.
    * One of the main contributions of the paper is the varMNIST dataset and the diffusion process used to generate the images. It is surprising that a lot of highly relevant details regarding the image generation are included only in the appendix. I would strongly recommend the authors to update the manuscript to include pertinent details, at least briefly, in the main paper.
    * The same point holds about the behavioral experiments. Details such as the number of subjects, number of trials and rationale for choosing the criteria for selecting the 4741 images are better placed in the main paper because they are central to the paper’s contributions.

* A couple of minor typos:
    * perveptual -> perceptual in section 5 title (line 310)
    * There is no section 3.1 (line 402)

questions:
I have several questions which I would like the authors to respond to. I have stated them in the 'Weaknesses' section of the review, but I will list them here for the authors' convenience:
* How many of the 4741 were generated using ’0’ or ‘1’ as the reference image? How many of these images were generated by uncertainty guidance and how many by controversial guidance?
* How statistically significant are the claims of randomness of the distribution of human prediction accuracy?
* How are images to be displayed to subjects selected? What design choices were made to ensure that subjects were equally likely to view images generated by different reference images?
* What is the correct label for an image from the varMNIST dataset? What was the rationale for the choice you made?
* In finetuning the models, what are the targets used in the case of population-level alignment?

In addition, I have a couple of additional questions for which I would appreciate the authors' clarifications:
* Why is the MSE loss important? The authors claim that bias in the prior distribution of MNIST causes generated images to be biased towards specific digits. Would this not be solved by sampling from MNIST so that all digits are equally likely to be selected as reference images?
* Why is model entropy (Fig 4a) centered around 0?
* Will the varMNIST dataset along with the results of the behavioral experiments be made publicly available?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732499885995

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Global Response (pt. 1)

comment:
# Global Rebuttal

We sincerely thank the reviewers for their thorough review and valuable comments on our paper. We have carefully reviewed and fully considered the suggestions and recommendations provided by the four reviewers. Based on this feedback, we have conducted additional experiments and analyses and made comprehensive revisions to the manuscript while preserving the original conclusions. Since we have updated the structure of the manuscript to better address the reviewers’ concerns, we have summarized the revisions into the following four key points instead of highlighting specific changes in the text.

### 1. Behavioural experiment of manipulating human perceptual variability. (Sec. 5 and Fig. 5)

Based on the concerns raised by reviewer Lgkn, we conducted a lab-based human behavioral experiment, building upon the original Section 6 (now revised as Section 5). The aim of the experiment was to validate whether the high-variability stimuli generated by our method can efficiently align with individual responses and whether the generative model can produce stimuli that elicit significant controversial behavior between two participants. This experiment would significantly enhances the validity of our paper.

Through directional, participant-specific precise manipulation (e.g., **Participant A selects digit *4* while Participant B selects digit *5* on the same stimulus**), we demonstrated that our model successfully captured the key factors driving differences between participants. Specifically, we conducted two rounds of experiments with 18 participants:
Round 1: Participants completed a digit recognition task with stimuli from the varMNIST dataset to determine initial behavioral data.
Round 2: Using the data collected in Round 1, we fine-tuned the models and generated a new set of experimental stimuli with the generative model.

Our results demonstrated that our method significantly **improved the targeted ratios across almost all participant pairs**. This indicates that, **when presented with the same stimulus, both participants were more inclined to choose the numbers aligned with their respective guidance targets**, further validating our model’s ability to decode and manipulate perceptual variability.
(**see Fig. 5**)


### 2. Deeper analysis of behavioral data and human-model alignment (Sec. 3.3.2 and Sec. 4, Fig. 3 & 4)

Based on the suggestions from reviewer vTKk, SqUF, Lgkn and vi4H, we performed a more in-depth and comprehensive analysis of the behavioral data and human-model alignment.

- **Quantitative analysis on varMNIST**: We conducted a quantitative analysis that includes 1) new evaluation metrics; 2) ANN variability can arouse human variability; 3) Guidance strategy, classifier and target influence the guidance outcome (**see Sec. 3.3.2, Fig. 3 and Fig. A.10, A.11, A.12**).
  
- **Alignment analysis**: We conducted a comprehensive analysis of the alignment between human behaviors and models in group and individual level. Specifically, we discussed: 1) Fine-tuning improves both group-level and individual-level prediction performance; 2) Different classifiers exhibit inconsistent performance; 3) Human variability can be predicted by models; 4) Performance of behavior prediction across images with varying entropy levels. (see **Sec. 4, Fig. 4 and Fig. A.13, A.14, A.15**)

### 3. New written content

Based on the suggestions from reviewer vTKk, Lgkn and vi4H, we made the following revisions:

1) **Article structure:** We reorganized the article into three sections: (**Section 3**) "Collecting human perceptual variability," (**Section 4**) "Predicting human perceptual variability," and (**Section 5**) "Manipulating human perceptual variability." Additionally, we provided a comprehensive overview of the content in the Introduction. We also updated **Fig. 1** and the para 3 of Introduction to better reflect the revised structure and clarified that our core contribution lies in proposing a systematic paradigm for studying human perceptual variability.
2) **Revised figures:** We redesigned figures, **including** methodological illustrations for each section (**Fig. 1 and Fig. 2**), and added several result and evaluation figures (**Fig 3, Fig 4b&d and Fig5 b&c**).  
3) **Methods and experimental details:** The main methods and experimental content are detailed in the main text (**Sec. 3**), with additional details provided in the Appendix (**Appendix A 2&3**).
</content>
</thread_object>

</thread>


Metareview:
metareview:
This submission introduces a methodology for generating stimuli that elicit varied perceptual responses across both neural network classifiers and human subjects, while also demonstrating the capacity to fine-tune neural networks to account for individual human perceptual differences. The original manuscript focused exclusively on the MNIST dataset for experimental validation. While the investigation of perceptual variability and its alignment between artificial and biological vision systems represents a valuable research direction, reviewers initially expressed concerns regarding the reliance on non-naturalistic MNIST data. The authors subsequently provided additional experiments using ImageNet, but these results were submitted after the response period had concluded. This late submission of significant new experimental results would necessitate another round of review for proper evaluation.