Visual Question Answering Driven Eye Tracking Paradigm for Identifying Children with Autism Spectrum Disorder

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 06:19:34

Abstract: As a non-contact method, eye-tracking data can be used to diagnose people with Autism Spectrum Disorder (ASD) by comparing the differences of eye movements between ASD and healthy people. However, existing works mainly employ a simple free-viewing paradigm or visual search paradigm with restricted or unnatural stimuli to collect the gaze patterns of adults or children with an average age of 6-to-8 years, hindering the early diagnosis and intervention of preschool children with ASD. In this paper, we propose a novel method for identifying children with ASD in three unique features: First, we design a novel eye-tracking paradigm that records Visual Question Answering (VQA) driven gaze patterns in complex natural scenes as a powerful guide for differentiating children with ASD. Second, we contribute a carefully designed dataset, named VQA4ASD, for collecting VQA-driven eye-tracking data from 2-to-6-year-old ASD and healthy children. To the best of our knowledge, this is the first dataset focusing on the early diagnosis of preschool children, which could facilitate the community to understand and explore the visual behaviors of ASD children; Third, we further develop a VQA-guided cooperative ASD screening network (VQA-CASN), in which both task-agnostic and task-specific visual scanpaths are explored simultaneously for ASD screening. Extensive experiments demonstrate that the proposed VQA-CASN achieves competitive performance with the proposed VQA-driven eye-tracking paradigm. The code and dataset will be publicly available.

Main paper: 
 introduction: 
 Autism Spectrum Disorder (ASD) is a disorder of very early brain development. Current clinical diagnosis methods such as Social Figure 1 : The fixation maps of healthy people in two paradigms. In (a) free-viewing task, due to the absence of specific guidance, the diversity of the eye movement patterns is evident, where different subjects prefer to observe different parts of the cow. Instead, in (b) VQA task, guided by the explicit question, healthy subjects invariably watch the dog's eyes and the frisbee, which could reduce the intravariance and benefit the ASD screening.Responsiveness Scale-2 (SRS-2) [5] and Repetitive Behavior Scale-Revised (RBS-R) [4] all depend greatly upon symptomatic observations and subjective judgments from clinicians. Therefore, developing automatic and objective ASD screening methods is an urgent need for the early diagnosis of ASD people.It is evidenced that the eye movement patterns of people with ASD are significantly different from those of healthy people, thus revealing the potential of applying eye-tracking techniques to subjectively diagnose people with ASD [12, 14, 15, [28] [29] [30] [31] 38] . Inspired by this, many works [6, 11, 26, 34, 39] try to analyze the eye-tracking data using machine-learning-based methods for diagnosis. Recently, benefiting from the strong learning ability of deep-learning technologies, some methods adopt deep neural networks [1, 7, 22, 40] and achieve promising results in diagnosing ASD.However, early eye-tracking-based methods [2, 35, 41] usually utilize a simple visual search paradigm for identifying ASD, where only restricted or unnatural stimuli (e.g., faces or letters) in isolation or even stimuli with low-level features are used for analysis. Recent studies [7, 22] attempt to develop new paradigms under complex scenes with natural background to analyze the difference between ASD and healthy people, but they mainly rely on the simple free-viewing paradigm, such as watching images or videos without any specific purpose. Due to the lack of explicit guidance in the free-viewing style, the eye movements are susceptible to individual preference, leading to large intra-variance (i.e., the eye movements of different healthy people are highly dissimilar when viewing the same image). As shown in Fig. 1a , the gaze patterns of different healthy people are significantly diverse when watching the same image, making the ASD classification quite challenging. Besides, since collecting eye-tracking data from preschool children is much more difficult, the eye-tracking-based datasets are mainly taken from adults [3, 20, 22] or children with an average age of 6-to-8 years [12, 16] , which highly limits the early diagnosis and intervention of young children.To tackle the above issues, in this paper, we propose a novel eye-tracking paradigm in which the Visual Question Answering (VQA) task is designed to collect distinctive eye movement patterns, which provides a better understanding of how visual attention is deployed in the brain of children with ASD in real-life behaviors. In our eye-tracking experiments, the images combined with the corresponding questions are shuffled and presented in consecutive trials. The subjects are required to search the related regions of the images to answer the given questions. Introducing the VQAdriven eye-tracking paradigm provides powerful guidance to get the subject concentrated, which can alleviate the effect of large intra-variance in the free-viewing paradigm. As shown in Fig. 1b , with the guidance of the VQA task, the gaze patterns of different healthy people are highly similar, which can benefit the following identification. Moreover, the VQA task requires the subjects to interact and reason for answering the questions, and thus can partially reflect the interaction ability of the subjects, which also helps the accurate prediction of ASD classification.To evaluate the effectiveness of the VQA paradigm, we launch recruitment for preschool children diagnosed with and without ASD to voluntarily take part in our experiments. So far, we have successfully collected effective eye movements from 42 ASD and 26 typically developed (TD) children (control group). There are no statistical differences between the two groups in terms of visual acuity, age, etc. To our knowledge, our collected dataset, named VQA4ASD, is the first eye movement dataset focusing on 2-6-yearold children, allowing the community a better comprehension of the mechanism of how visual attention influences ASD's behaviors.Furthermore, built upon the collected dataset, we propose a novel model, VQA-guided cooperative ASD screening network (VQA-CASN), which consists of two branches to identify ASD and healthy people jointly. In the proposed VQA-CASN, one branch captures the eye movement features in a task-agnostic manner, where the differences in gaze patterns across the whole image are utilized for classification. The other branch only focuses on the differences in the regions of interest, i.e., question-related regions, aiming at reducing the variance of TD group to better differentiate ASD and healthy people in a task-specific manner. These two branches jointly learn the subject's attention under the VQA task, which provides a more efficient solution in real clinical scenarios. We conduct a series of experiments on our collected dataset and achieve state-of-the-art performance.Overall, this paper has four main contributions:• We design a novel eye-tracking paradigm where Visual Question Answering (VQA) driven gaze patterns are employed for ASD screening. The proposed paradigm can not only mitigate the large intra-variance in the previous free-viewing paradigm but also assess the interaction ability of ASD children through visual patterns.  
related work 2.1 eye tracking paradigm: 
 Current ASD diagnosis is usually performed by subjective and manual screening methods. However, these subjective measurements are both time-consuming and clinically demanding. Since it has been verified that the visual attention of people with ASD presents distinctive characteristics from healthy people, recent studies [25, 26, 36, 39] aim at developing objective and automatic screening methods to distinguish the ASD individuals, where the subjects are asked to freely view a set of images or videos and an eye tracker is used to collect the subjects' eye movements.In order to collect distinctive gaze patterns for identifying ASD precisely, various paradigms have been explored. Gong et al. [16] use dynamic images with repetitive movements. Hochhauser et al. [20] utilize changed blindness paradigm and social films. Chita-Tegmark et al. [8] present visual and auditory stimuli in pairs. Au-Yeung et al. [3] display the paired visual stimuli simultaneously. Some works [27, 33] use the visual search paradigm with simple stimuli to ask the subjects to locate the predefined objects. Recently, [7, 22] develop image free-viewing (IV) paradigm, where the subjects watch natural images or videos. Such free-viewing paradigm cannot provide useful guidance, leading to a large variance among healthy people. In contrast, our VQA-driven task can alleviate the negative effect of intra-variance under explicit guidance.The visual search paradigm is most related to our VQA paradigm, but our work is superior to the previous visual search paradigm in three aspects: 1) Previous visual paradigms mainly use restricted or unnatural stimuli (e.g., faces or letters) in isolation or even stimuli with only low-level features. Our VQA paradigm explores visual attention with more natural stimuli (e.g., complex scenes taken with a natural background), providing a better understanding of how attention is deployed in children with ASD when viewing the real world. 2) Our VQA paradigm designs more kinds of questions (e.g., how many, is it, where, what) than visual search (count), which can reveal the impairments of both communication and social interaction in ASD children. More importantly, the VQA task has been fully studied in the multimedia field, so abundant datasets and models can bring new insights about visual saliency for ASD screening. 3) Most previous visual search paradigms published in medical-related articles did not release the dataset. So far, only [12] has released an image-viewing dataset collected from school children and adults. In this work, we collect the first VQA-driven dataset focusing on identifying preschool children with ASD, which is not explored in most previous works due to the difficulty of data collection. We believe that our dataset can largely enrich the studies of automatic computer-aided diagnosis and multimedia applications.To analyze the collected eye-tracking data, some works introduce machine learning-based methods [26, 37, 39] to distinguish ASD. Thanks to the powerful representation abilities of deep learning techniques, deep learning-based methods have achieved significant performance in ASD screening. Jiang and Zhao [22] propose the first deep neural network to identify ASD. Chen and Zhao [7] present to use multi-modality (i.e., photo-taking and image-viewing) in ASD screening and achieve state-of-the-art performance.Compared with previous works which usually adopt a singlebranch architecture, our proposed dual-branch network has an additional task-specific branch that focuses on analyzing the gaze patterns in question-related regions. Benefiting from the explicit guidance of the VQA task, adding the task-specific branch can reduce the large variance of the gaze patterns within the TD group, leading to a better diagnosis performance. 
dataset: 
 In this section, we construct a VQA-driven eye-tracking dataset, named VQA4ASD, for the early diagnosis of ASD children with two objectives: (1) to reduce the intra-variance in the control group by introducing explicit guidance; (2) to reveal the social and communication skills, especially the ability to receive questions and make corresponding reactions, which indicates the social and attentional deficits of subjects. Compared with the existing datasets [7, 22] , our VQA4ASD provides the first VQA-driven eye movement data collected from preschool children, ranging from 2 to 6 years, which can benefit the early diagnosis of preschool children with ASD. Some VQA stimuli and the corresponding questions of ASD and control group are provided in Fig. 2 . More examples are given in the supplementary material.To construct our VQA4ASD dataset for diagnosing ASD children, we choose the images and the corresponding questions from VQA v2.0 dataset [17] , where 25 images are selected from real scenarios and 25 images are selected from abstract scenarios. To guarantee that the images contain a large variety of scenes, we carefully select the images covering common scenarios of daily life for children. Specifically, the abstract scene images include 1 indoor image with single person, 2 outdoor images with single person, 2 indoor images with multiple persons, 6 outdoor images with multiple persons, 7 indoor images with people and animals, and 7 outdoor images with people and animals. The real scene images include 1 image with natural scene, 1 image with single person, 2 images with person and animals, 3 images with multiple persons, 5 images with single person and objects, 5 images with animals, and 8 images with buildings or objects. Additionally, we also arrange an image freeviewing task accompanied by VQA task for a fair comparison with free-viewing paradigm (see details in Section 5.1). Besides, since children with ASD may show a center bias [12, 39] and some questions of VQA may be difficult for preschool children, all of the questions we choose are relatively simple and most of the answers to the questions cannot be directly found in the center of the screen. The distribution of question types is shown in Fig. 3 . We can see that the types of questions are abundant, including "How much", "Is this", "What", and "Where", to ensure that gaze patterns can be recorded in various scenarios.We collect eye-tracking data from 68 preschool children aged children are determined by an expert-level clinician and evaluated by Autism Behavior Checklist (ABC) [24] and Childhood Autism Rating Scale (CARS) [32] , which can exclude unqualified children who may also suffer from other non-ASD developmental disabilities. Before the experiments, we obtained informed consent from children's parents. Besides, due to social interaction disorder, it is difficult for children with ASD to cooperate on finishing the experiment tasks and even calibration, and several TD children also fail to complete all experiments due to losing patience and interest. Therefore, only 24 children with ASD and 24 TD, whose ages are 4.0 ± 1.2 and 4.6 ± 0.8, complete all experiments.We use SensoMotoric Instruments (SMI) IView X RED eye tracker to record the subjects' eye movements by showing the stimuli on a displayer with 1280 × 768 resolution.During training, we set the sampling frequency of the eye tracker to 120 Hz. In order to avoid the subjects from losing their concentration due to the long-time continuous experiment, we split VQA tasks into two parts, each consists of 25 images. Before each trial begins, a five-point calibration is attached to ensure the precision of the eye movement data. During each trial, a gray image is first displayed for three seconds and accompanied by the corresponding question, and then the image is presented for five seconds. We ask the question again during the image presentation to avoid the situation where children do not hear the question clearly. When we observe that children are distracted from the displayer, we will remind them to view the screen with a non-leading sentence (e.g., "Let's look at the screen"). The duration of each subject finishing the VQA experiments is about 15 minutes.For the obtained raw eye movement data, we need to further process the data for the training of the diagnosis model. In the raw data, the sequences of continuous fixation events separated by saccades or blinks are considered fixations. So we first generate the location of each fixation by averaging the continuous fixation events' horizontal and vertical positions. The duration of the averaged fixation is calculated as the time from the first fixation event of the fixation sequence to the first non-fixation event after that. Since the subject's first fixation is often the last fixation position on the previous gray image, it is meaningless and cannot reflect the subject's interest. In order to eliminate such possible distractions, we discard the first fixation for all subjects. Finally, we generate eye movements that are rounded to integers and recorded for further training and testing.It should be noted that all eye movement data in our dataset are recorded, including the visual stimuli, the corresponding questions, the raw data, as well as processed data, which will be made publicly available upon publication of the work.The comparison between the previous dataset Saliency4ASD [12] and our VQA4ASD are summarized in Table 1 . We can see that our collected VQA4ASD involves more individuals and younger children. Besides, VQA4ASD includes both VQA and image freeviewing (IV) paradigms, which facilities the fair comparison between different paradigms.Table 1 : The advantages of VQA4ASD over Saliency4ASD [12] .Saliency4ASD [12] VQA4ASD (ours) #Subjects 28 (  
method 4.1 overview: 
 The overview of our proposed VQA-guided cooperative ASD screening network (VQA-CASN) is shown in Fig. 4 . The intuition behind our design is to cooperate with the task-agnostic (the gaze patterns across the whole image) and task-specific (the gaze patterns within the question-related region), which can largely reduce the intra-variance of TD group and further leverage the identification performance under the guidance of VQA task. Specifically, VQA-CASN contains three modules, i.e., image feature extraction, fixation sequence feature extraction, and classification. The first component of our model is extracting feature representations of images, which is shared between the following two branches. Then, in the fixation sequence feature extraction module, we design a dual-branch architecture for extracting both task-agnostic and task-specific features of the fixation sequences. The task-agnostic branch focuses on extracting features from the whole fixation sequence. The task-specific branch attempts to obtain local features from VQA task-related regions. Finally, after fusing the two above features and the location and duration of fixations, we feed the concatenated features to a fully connected (FC) layer with a sigmoid function to compute the probability of suffering from ASD in the final classification module by weighting the contributions of different gaze patterns. Benefiting from the cooperation of the above modules, VQA-CASN can simultaneously learn the difference of gaze patterns on both task-agnostic and question-related regions, where two branches are complementary to each other and yield favorable accuracy.In order to learn the effective image representation of the subject's scanpath, we use the popular feature extractor ResNet-50 [19] to learn the image features. For one image 𝑥, we extract a 2048-𝑑 feature from the top convolutional layer of ResNet-50 as the image feature of each fixation point which is shared in the following two-branch module.After obtaining the image feature of the image 𝑥, given the corresponding gaze pattern of one subject watching the image 𝑥, we further generate the feature of the fixation sequence by a dualbranch architecture, where two branches adopt the same network structure, i.e., a variant [18] of Long Short Term Memory (LSTM) We first use a ResNet-50 [19] to extract the image feature which is concatenated with the location and duration of the fixation point. Then, we design a dual-branch architecture to extract the feature for the fixation sequence. Specifically, the task-agnostic branch captures the feature across the whole image using LSTM [18] network, and the task-specific branch acquires the LSTM feature within the question-related region, which is generated according to the fixation map of the TD group. Finally, the two features are concatenated and fed to a fully connected (FC) layer with sigmoid function for identification.network [21] , to encode the temporal information of the gaze pattern. Note that the two LSTMs do not share the parameters to capture abundant features for different purposes.Task-agnostic branch. In order to analyze the difference between ASD and TD across the whole image, we first extract the feature of the fixation sequence in a task-agnostic manner. Specifically, we first extract the 2048-𝑑 visual feature at each fixation and concatenate it with the location and duration as a 2051-𝑑 feature of each fixation. Then we stack the feature of each location in temporal order to generate the feature of the fixation sequence. To reduce the interference of the redundant fixations, we manually control the maximum number of fixations as 𝐿 and discard the rest fixations. Consequently, we get the feature representation of the fixation sequence denoted as 𝐹 ∈ R 𝐿×2051 .To obtain more powerful features, we further feed 𝐹 into the LSTM [18] in the task-agnostic branch to capture the temporal information by updating the hidden state repeatedly. We treat the final hidden state as the task-agnostic feature 𝑆 𝑡𝑎 of one gaze pattern as follows:EQUATION)where 𝜙 is the parameters of LSTM in task-agnostic branch.Task-specific region generation. Thanks to the VQA task, except for the task-agnostic feature, we can pay more attention to the task-specific regions to further differentiate the gaze patterns between ASD and healthy people. Before generating the task-specific features, we should know where the regions are related to the question and answer. We achieve this goal by regarding the fixation maps of the control group as possible question-related regions. This strategy is reasonable since our VQA task is performed in an interactive Q&A manner, which is much more attractive to healthy children, so they are very likely to cooperate in the collection process, which guarantees that the TD group's fixation maps are highly relevant with the corresponding questions and answers. We have empirically validated this by comparing the results with manually labeling the task-relevant regions. The results are relatively close (see Sec. 5.4 for details). Considering that manual labeling requires extra annotations, we therefore design the task-specific branch by averaging all TD children's fixation maps, making our model easily extendable to other new VQA stimuli because it does not require any additional annotations.Specifically, we first gather the fixation points of the control group by setting the location of fixation points to 1 and other pixels to 0. Then we follow [12] to obtain the averaged fixation map 𝑀 of the control group by applying the Gaussian smoothing and min-max normalization. Subsequently, we can get the task-specific region 𝑅 by using binarization on the averaged fixation map.Task-specific branch. The task-specific branch focuses on the gaze patterns within the question-related regions, so we first obtain the task-specific feature of the fixation sequence, denoted as 𝐹 𝑡𝑠 ∈ R 𝐿×2051 , with the guidance of the task-specific region 𝑅 by replacing the feature that falls outside the question-related region with 0.Similar to the task-agnostic branch, the task-specific feature of the fixation sequence 𝐹 𝑡𝑠 is then forwarded into LSTM [18] in the task-specific branch to capture the question-related feature of the gaze pattern 𝑆 𝑡𝑠 :EQUATIONwhere 𝜑 is the parameters of LSTM in task-specific branch.Once the 𝑆 𝑡𝑎 and 𝑆 𝑡𝑠 have been obtained, they are concatenated and forwarded to a multilayer perceptron (MLP), i.e., an FC layer with sigmoid function, to acquire the predicted score ŷ which can estimate whether the gaze pattern belongs to a subject with ASD or not.EQUATIONwhere 𝜃 is the parameters of the MLP layer.As different visual stimuli show distinct abilities for diagnosing ASD, we design an adaptive weighting loss to better differentiate ASD and TD. Specifically, for each visual stimulus, we calculate the cosine similarity of the corresponding fixation maps of ASD and TD groups to define a weighted cross entropy loss:EQUATIONEQUATIONwhere 𝑀 is the fixation map of the subject for single visual stimulus.M𝐴𝑆𝐷 , M𝑇 𝐷 represent the averaged fixation map of ASD group and TD group for single visual stimulus, respectively. 𝑦 ∈ {0, 1} and ŷ ∈ {0, 1} denote the ground-truth label and the predicted score (1 is ASD and 0 is TD).In the inference, given one subject, we generate the final subject's predicted score by averaging the predictions obtained from all stimuli using (4) as a weighting function, and a pre-defined threshold of 0.5 is used to identify ASD. 
experiment settings: 
 Dataset. We conduct all experiments on our VQA4ASD dataset which is divided into two parts, the abstract and real scenes, named VA and VR, respectively. Besides, in order to give a comparison with previous approaches [7] and facilitate future works, we also arrange an additional image free-viewing (IV) paradigm to collect the eye movements while constructing VQA4ASD dataset. We set the ratio of the number of IV images to VQA images to 1:1 and conduct experiments on the same subjects who complete all tasks. The dataset consists of a total of 4800 samples, with 2400 samples for IV and VQA, respectively. The details of three kinds of paradigms are as follows:• IV (Image free-viewing, abbreviated as IV) is collected by asking the subjects to freely view images without guidance. It contains the most distinctive top 50 images out of a total of 300 images according to a Fisher-score-based image selection strategy [7, 22] from Saliency4ASD dataset [12] . • VA (Abstract scene in VQA4ASD, abbreviated as VA) includes 25 clipart-style images selected from the abstract scenes in VQA v2.0 dataset [17] . • VR (Real scene in VQA4ASD, abbreviated as VR) includes 25 images selected from the real scenes in VQA v2.0 dataset [17] .The details of VA and VR have been given in Sec. 3. Training details. All images are resized to 800 × 600 and normalized (i.e., make the mean and standard deviation of the images 0 and 1). The parameters of ResNet-50 [19] are initialized using pre-trained parameters on ImageNet [10] . The size of the hidden state of LSTM [18] is 512. The length of the fixation sequence 𝐿 is 14. During the training stage, we set the batch size to 10. All models are trained for 10 epochs using the Adam [23] optimizer with the learning rate of 10 -3 and the weight decay of 10 -5 . The gradient clip strategy is applied by keeping the gradient no larger than 10.Evaluation metrics. Following previous work [7] , we use the leave-one-subject-out method for cross-validation and evaluate the models with four metrics, including accuracy (Acc), sensitivity (Sen), specificity (Spec), and area under the ROC curve (AUC). In addition to evaluating the subject-level performance of each subject, we also evaluate the the scanpath-level performance of each gaze pattern to validate which model can select more distinctive gaze patterns for diagnosis.Baseline methods. We are the first to propose VQA-driven eyetracking data for ASD diagnosis, and Jiang et al. [22] and Chen et al. [7] use the free image-viewing task, which are the most related works and thus are chosen as our compared methods. For fairness, we use the same hyper-parameters in [22] and [7] except the batch size of 10. Besides, we also implement a non-deep-learning simple baseline to verify the effectiveness of VQA paradigm, where the Fisher score method [13] is used to extract the gaze pattern's feature [7, 22] followed by a linear Support Vector Machine (SVM) classifier [9] for classification.Qualitative analysis. As shown in Fig. 1 , the guidance of the question effectively gather the TD subjects' attention on the dog's eyes. In contrast, TD subjects will view the different parts of the cow in the free-viewing task. Furthermore, as shown in Fig. 5 , the visual patterns of TD subjects and ASD subjects show significant difference in VQA task. Since TD subjects can communicate and interact with us normally, their gaze patterns are highly related to the questions. In contrast, due to the social interaction disorder, ASD subjects do not respond to the questions and will view the images according to their personal consciousness.Quantitative analysis. To show the superiority of the VQA paradigm, we make comprehensive comparisons between different paradigms. We perform experiments on VQA (VA+VR) and IV paradigms using four models, i.e., Fisher score method, Jiang and Zhao [22] , Chen and Zhao [7] , and our proposed VQA-CASN. The comparisons are summarized in Table 2 .For the simple Fisher score method, we can see that the VQA paradigm achieves better performance than IV in all metrics. Considering that the 50 images of IV paradigm are chosen from 300 images based on the analysis of collected gaze patterns [7] , our paradigm simply chooses 50 images from VQA dataset [17] without any guidance of gaze patterns, but still achieves better performance, which can suggest the effectiveness of the designed paradigm.Besides, we compare two paradigms using the model proposed by Jiang and Zhao [22] . Compared with IV paradigm, our VQA paradigm gains better performance in terms of accuracy, sensitivity, and AUC in subject-level (e.g., an AUC improvement of 20.3%). Noted that the model [22] fuses the scanpath features before prediction, so we do not evaluate the performance on scanpath level. We also compare two paradigms using the model proposed by Chen and Zhao [7] . Compared with IV paradigm, our VQA paradigm achieves fixation maps of ASD and TD group, and the bottom two rows are the images with higher weights with significant diverse fixation maps of ASD and TD group. Effect of the location and duration of fixation points. To verify the effect of the location and duration of fixation points for generating distinctive representations, we remove the location and duration and the result is given in the fourth row of Table 4 . We can see that adding location and duration is beneficial for improving performance in terms of Sen and AUC.Effect of using TD group's fixation map as the task-specific region. To verify the effect of TD group's fixation map as the taskspecific region, we use the manually labeled task-specific region and the result is given in the fifth row of Table 4 . We can see that using manually labeled task-specific region can lead to a performance drop in terms of Acc, Sen and AUC (4.1%, 8.3% and 0.3%).Length of fixation sequence. We investigate the effect of the length of fixation sequence 𝐿. As shown in Fig. 7 , by setting 𝐿 to 14, we achieve the best performance in terms of AUC. 
conclusion: 
 In this paper, we propose a novel VQA-driven paradigm for ealy diagnosis ASD children. We construct the first VQA4ASD dataset, which contains the data from 2-to-6-year-old preschool children. Moveover, we design a dual-branch VQA-CASN model to identify ASD and healthy individuals. The results show the effectiveness of our proposed VQA paradigm and the VQA-CASN model. 
 
 Appendix:  


Decision: In the first round of reviews, this paper got 2 borderline acceptances and 1 borderline rejection. The authors submitted a rebuttal, which satisfied two of the reviewers. Taking into account the reviews, the rebuttal, the final comments by the reviewers after the rebuttal and the interesting and innovative topic addressed in the paper, I lean towards recommending this paper for acceptance as poster.

Reviews: review: paper_summary: paper_summary: This paper conducts a study that uses VQA-driven eye-tracking techniques for ASD diagnosis.
strength_weakness: strength_weakness: 1. To the best of our knowledge, this seems the first work studies VQA based eye movement for ASD.
2. Due to the disease of ASD, it seems significant to conduct VQA base eye tracking for ASD.
limitations: limitations: 1. The age ranges from 2-6, it is hard to let them understand the question, even for high functioning ASD and env for normal 2 years old children. This is the most important thing in the experiment, and should be seriously answered.
2. The dataset size seems small, only 25+25 images were included.
paper_summary: paper_summary: The authors create a VQA-driven eye-tracking dataset for early diagnosis of ASD in children with two goals: (1) to reduce intra-variance in the control group by providing explicit guidance; (2) to reveal social and communication skills, particularly the ability to receive questions and respond, indicating social and attentional deficits. In addition, they develop a VQA-guided cooperative ASD screening network (VQA-CASN), in which both task-agnostic and task-specific visual scan paths are explored simultaneously for ASD screening. The results show the effectiveness of their proposed VQA paradigm and the VQA-CASN model.
strength_weakness: strength_weakness: This work is interesting and well-written.
limitations: limitations: Comments 1: This dataset is generated by 2-6-year-old children. Does it appear that the age factor affects the understanding of the question. It is suggested that the authors add the performance of model at different age stages.

Comments 2: In the Results, evaluation metrics (e.g., sensitive (Sen) and accuracy (Acc)) are not written in a harmonized way, i.e., sometimes it's a full name, sometimes it's an abbreviation.

Comments 3: For the VQA-CASN, it is suggested that the authors consider more novel feature fusion methods to replace the concatenation.
paper_summary: paper_summary: ## Paper Topic And Main Contributions:

The paper proposes a new Visual Question Answering for early diagnosis of preschool children to explore the visual behaviours of ASD children. Additionally, it proposes the VQA-guided cooperative ASD screening network (VQA-CASN) to explore the eye tracking paradigm. The paper aims to identify preschool children with ASD by focusing on proposed eye-tracking paradigm, new dataset and model (by capturing eye movement features in a task-agnostic manner and differences in the regions of interest).

Eye tracking paradigm empowered by VQA4ASD dataset and VQA-guided cooperative ASD screening network (VQA-CASN) can be mentioned as the main contributions of the paper.
strength_weakness: strength_weakness: ## Strengths:

1. In medical application level, the usage of gaze feature for early diagnosis of preschool children is new.

2. In idea level, VQA-CASN sounds really promising. Additionally, I believe that this idea can be explored more in different applications.
limitations: limitations: ## Weakness:

1. Generally, the main purpose of using the human gaze feature is to capture the relevant objects or actions to support relevant hypotheses. Therefore, this approach is not novel enough for the scientific level.

2. According to the references cited by the authors, this dataset cannot be considered novel either. However, a new dataset has been explored for younger subjects (as compared in the paper). Additionally, a Gaze features-enhanced dataset has already been explored for eye-tracking paradigms in different applications before. Please, re-explore the literature.

3. The VQA dataset is one of the priorities of this paper, and there is no detailed explanation about the questions.

## Questions For The Authors:

This is an important task, and the paper focuses on preschool ASD children. Questions in the VQA dataset are also crucial. Eye tracking is a paradigm followed, and the proposed hypothesis is tested with the model and results. However, questions are key points. I believe that you have already been careful about them, but I would like to see some clarification in the paper. I felt unconvinced about the questions and preparations.

1. What are the professions (pedagogues, medical doctors, teachers, etc.) of people who designed the questions? 

2. Were the questions prepared manually or AI-generated?

3. Were the questions designed inspired by pedagogical guidance?

4. What are the specific reasons for preparing the proposed questions (in the paper and appendix)?

5. How do these questions meet your expectations?

6. What are the protocols or fundamental rules for preparing the questions?

7. How can you prove these questions are easy tasks for 2-6-year-old children?

8. Please provide some statistical information about the subjects and experiment environments.

9. How big were the figures shown to the subjects? (I think that affects children's saccades and fixations.)

10. Additionally, how many 2 or 3-year-old children joined the experiments? I ask this question because some questions in the supplementary seem really hard for them. Because some questions consist of “How many…”. Therefore, a questions rises: How can 2 or 3-year-old children give the right answer? Even if you do not expect them to give the right answer, and you expect them to check the right objects, those children can be confused while checking the correct object and numbers. Consequently, what are the prediction results of your model in such cases?

11. Please provide some statistical data about VQA4ASD in terms of the number of questions, word cloud distributions, etc.

12. If the new questions are collected with new figures and scenes, why were only questions and answers prepared on Saliency4ASD? In short, what is the key point for collecting a new dataset with new figures and scenes?

13. Isn’t it enough to curate only new QA pairs relevant to the Saliency4ASD dataset? If the only difference between the new dataset and Saliency4ASD is questions, then the preparation of that new dataset may not convince the community about why we need a new dataset.

## Typos Grammar Style And Presentation Improvements:

1. This paper is well organized and written.