RTify: Aligning Deep Neural Networks with Human Behavioral Decisions

Authors: yu-ang  cheng
ivan  felipe rodriguez
sixuan  chen
kohitij  kar
takeo  watanabe
thomas  serre
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 01:54:25

Abstract: Current neural network models of primate vision focus on replicating overall levels of behavioral accuracy, often neglecting perceptual decisions' rich, dynamic nature. Here, we introduce a novel computational framework to model the dynamics of human behavioral choices by learning to align the temporal dynamics of a recurrent neural network (RNN) to human reaction times (RTs). We describe an approximation that allows us to constrain the number of time steps an RNN takes to solve a task with human RTs. The approach is extensively evaluated against various psychophysics experiments. We also show that the approximation can be used to optimize an ``ideal-observer'' RNN model to achieve an optimal tradeoff between speed and accuracy without human data. The resulting model is found to account well for human RT data. Finally, we use the approximation to train a deep learning implementation of the popular Wong-Wang decision-making model. The model is integrated with a convolutional neural network (CNN) model of visual processing and evaluated using both artificial and natural image stimuli. Overall, we present a novel framework that helps align current vision models with human behavior, bringing us closer to an integrated model of human vision.

Main paper: 
 introduction: 
 Categorizing visual stimuli is crucial for survival, and it requires an organism to make informed decisions in dynamic and noisy environments. This critical aspect of visual perception has driven the development of computational models to understand and replicate these processes. Traditionally, the field has followed two distinct paths.On the one hand, image-computable vision models are used to predict behavioral decisions during (rapid) visual categorization tasks ranging from models of early- [1] [2] [3] , mid- [4] [5] [6] and high-level vision [7] [8] [9] (see [10] for a review). More recently, these earlier models were superseded by deep convolutional neural networks (CNNs), which have become the de-facto choice for modeling behavioral decision [11] [12] [13] [14] . Models are typically evaluated by estimating confidence scores computed for individual images, which are then correlated with similar scores derived for human observers(such as the proportion of correct human responses for each image). Such metrics ignore human reaction times (RTs); hence, current vision models only partially account for human decisions.On the other hand, decision-making models have been used to explain how visual information gets integrated over time -predicting behavioral choices and RTs jointly. Notably, mathematical models, exemplified by evidence accumulation models such as the drift-diffusion [15] [16] [17] The input is a visual stimulus represented by random moving dots, but the model can also accommodate color images and video sequences. We take a pretrained task-optimized RNN and use a trainable function f w to transform the activity of the network into a real-valued evidence measure, e t , that will be integrated over time by an evidence accumulator, Φ t . When the evidence accumulator reaches the threshold θ, processing stops, and a decision is taken. The time step at which the accumulated evidence passes this threshold τ θ is taken as the model RT for this stimulus.accumulators [18, 19] , have been quite successful in modeling an array of behavioral data (see [20] for a review). In addition, mechanistic models, including the Wong-Wang (WW) model, have provided insights into the underlying neural mechanisms [21, 22] . However, these efforts have primarily relied on traditional psychophysics tasks using simple, artificial stimuli, such as Gabor patterns [23] and random moving dots [24] . Beyond these easily parameterizable stimuli, these models have not been extended to deal with more complex, natural stimuli.While both vision and decision-making models have contributed distinctively to our understanding of visual processes, a complete understanding of human vision will require their integration to explain the whole dynamics of visual decision-making. Recent neuroscience studies have leveraged recurrent neural network (RNN) models as the starting point for this integration [25] (see [26] for a review). More generally, recurrent processing has been shown to be necessary to account for both behavioral and neural recordings in object recognition tasks [27, 28] .Two promising approaches have been described that leverage RNNs to bridge the gap between decision-making and vision models [29, 30] . In [29] , an RNN was trained to solve a visual classification task using the backpropagation-through-time (BPTT) learning algorithm. The entropy of the RNN outputs is computed at each timestep and used as a proxy for the network confidence. A decision is taken when the entropy reaches the threshold, halting further recurrent computation. In this approach, the recurrence steps are not differentiable, which prevents the use of gradient methods and inherently limits the complexity of the corresponding decision function. As we will show, the resulting model struggles to predict the entire distribution of RTs. Besides, the method is only applicable when human RTs are available because it requires an extensive search for the correct threshold value to fit human RT data.An alternative approach was described in [30] . In [30] , a convolutional RNN was trained on a visual classification task using contractor recurrent back-propagation (C-RBP). Besides, instead of searching for an optimal threshold to fit human RT data, a surrogate time-evolving 'uncertainty' metric was estimated with evidential deep learning [31] . In this framework, model outputs are treated as parameters of a Dirichlet distribution, with the width of the distribution reflecting uncertainty. Remarkably, the resulting approach fits a range of experimental data well, without any supervision from human data. It is true that uncertainty and RTs are tightly coupled and are both affected by task difficulty. However, uncertainty and RTs are conceptually different. Experimental results show that uncertainty and RTs can be positively or negatively correlated [32] , and even double dissociated [33] under different experimental conditions. Therefore, a good model of uncertainty is not guaranteed to be a good model of RTs. As we will show in our results (see similar findings in the original paper Fig. 6 ), uncertainty is only correlated with human RTs in extreme cases (very certain and very uncertain), and the model struggles to account for more "typical" stimuli, predicting fixed RTs for them.Here, based on extensive cognitive neuroscience research [34] [35] [36] [37] , we introduce a novel trainable module called RTify to allow an RNN to dynamically and nonlinearly accumulate evidence. First, this module can be trained to learn to make human-like decisions using direct human RT supervision.Our results suggest that incorporating a dynamic evidence accumulation process, compared to the entropy heuristics used in [29] , can help better capture human RTs. Second, we show how the same general approach can also be used to train an RNN to learn to solve a task with an optimal number of time steps via self-penalty. Our results show that human-like RTs naturally emerge from such ideal-observer models without explicit supervision from human RT data. Hence, our framework is general enough to allow the fitting of human RT data as done in [29] and/or the training of a neural architecture that can optimally trade speed and accuracy via self-penalty as done in [30] .Contributions: Overall, our work makes the following contributions: (i) We present RTify, a novel computational approach to optimize the recurrence steps of RNNs to account for human RTs. This enables dynamic nonlinear evidence accumulation learned through back-propagation (a) to fit human data or (b) optimally balance speed and accuracy. (ii) We comprehensively demonstrate the effectiveness of our framework for modeling human RTs across a diverse range of psychophysics tasks and stimuli. Our method consistently outperforms alternatives with and without explicit training on human data. (iii) As an illustrative example of the framework's potential, we extend the WW decision-making model [22] to create a biologically plausible, multi-class compatible, and fully differentiable RNN module. We show that the enhanced neural circuit can be used as a drop-in module for CNNs to fit human RTs.2 RTify: Overview of the method First, we explain how our RTify module is applied to a pre-trained RNN. Then, we will explain how to tune a deep-learning RNN-based implementation of the WW model to RTify feedforward networks.We start with a task-optimized RNN with hidden state h t , which remains frozen. We then train a learnable mapping function f w : R k → R that summarizes the state of the neural population at each time step t by mapping the RNN hidden state h t to some "evidence": e t = f w (h t ). At every time step, the evidence is integrated via an "evidence accumulator" Φ t = t i=1 e i , and when the accumulated evidence passes a learnable threshold θ, the model is read out, and a decision is made. The time step at which the accumulated evidence first passes this threshold is given by τ θ (Φ) = min{t : Φ t > θ}, and is treated as model RTs. In summary, τ θ (Φ) is directly influenced by the threshold θ and by w through Φ t .To align the model RTs with human RTs, or to penalize the model for excessive time steps, we need to optimize a loss function over τ θ (Φ). In the most general case, we first consider F (τ θ (Φ)) as our loss function to illustrate how we approximate its gradient. Since our goal is to minimize F , we will need to calculate the gradient ∂F (τ θ (Φ)) ∂θ and ∂F (τ θ (Φ)) ∂w . Following the chain rule, we getEQUATIONThis means a crucial step involves estimating the gradient of τ θ (Φ) over the trainable parameters w and θ of the RTify.The primary challenge that arises when extracting the gradient of τ θ (Φ) over the trainable parameters w and θ is the non-differentiability of τ θ (Φ), which prevents the direct use of the backpropagation algorithm. This is because τ θ (Φ) lies in the integer space and requires non-differentiable operations such as the minimum function and the inequality. We will relax the original formulation to circumvent this issue, enabling us to approximate the gradient.Assume that Φ(t) is a continuous function on the closed interval [1, N ], representing the accumulation of evidence over time. Define τ θ (Φ) = min{t ∈ [1, N ] : Φ(t) > θ}, which is the earliest time when Φ(t) exceeds the threshold θ.We can use a first-order Taylor expansion to find the following approximation:EQUATIONFig. S1 provides a visual intuition for the approximation, and the full derivation can be found in the SI A.1. This leads to the following gradients for our trainable parameters:EQUATIONSince F and Φ are both differentiable by nature, the gradients in Eqs. 2 and 3 are all computable after this approximation.Under this framework, we consider two different scenarios: Training with human data directly ("supervised"; see Section 2.1) or training with "self-penalty", which involves no explicit human data but uses a penalty term that spontaneously leads to decision times similar to human RTs (e.g., achieving an optimal speed-accuracy trade-off; see Section 2.2). 
predicting human decisions with direct "supervision": 
 Human behavioral decisions in the random dot motion (RDM) tasks used in decision-making studies [16, 24, 38] are typically summarized as histograms similar to those shown in Fig. 2 . Here, histograms are computed for RTs for correct and incorrect trials corresponding to individual experimental conditions (such as coherence levels shown here; see section 3 for details). Moreover, RTs for incorrect trials are turned into negative RTs. Combined with correct RTs (which stay positive), one single histogram is used for capturing both accuracy (the proportion of positive values) and RTs. To measure the goodness of fit between human RTs and model RTs, we use a mean squared error loss (MSE) between histograms of model RTs and human RTs.In the object recognition task [39] , only RTs averaged across all participants were available. We can match human data on a stimulus-by-stimulus basis using the negative correlation loss between model and human RTs.Our framework allows us to develop an "ideal-observer" RNN model explicitly trained to balance the computational time required for solving a particular classification task and its own accuracy for the task, i.e., a speed-accuracy trade-off.To achieve this, we add a regularizer to the cross-entropy loss to encourage the RNN to jointly maximize task accuracy while minimizing the computational time needed to solve the task. With l as the output logits of the network, ŷ as the output probabilities of the network, and y as the ground truth, we can write our penalty term for a single sample as:EQUATION)where λ is a hyperparameter for controlling the strength of the penalty, l y refers to the logit value of the correct label, τ θ is the model decision time. This penalty means that the model will be penalized for using too much time, especially for higher confidence (higher l y ).To integrate temporal dynamics into feedforward neural networks (e.g., CNNs), we describe an RNN module that approximates the WW neural circuit model [22] . The original WW model isA Incorrect CorrectExtracting RTs with supervision (fitting human RTs)Extracting RTs via self penalty (leveraging RNN dynamics)B Incorrect Correct Incorrect Correct Incorrect Correct Incorrect Correct Incorrect Correct Incorrect Correct Incorrect CorrectFigure 2 : RTified model evaluation on a RDM task [24] . Human data are shown as a gray shaded area, and model fits are shown for (A) the "supervised" setting where human behavioral responses are used to train the models and (B) the "self-penalized" setting where no human data is used. Our approach (green) outperforms the two alternative approaches (brown), i.e., entropy-thresholding [29] for the "supervised" and uncertainty proxy [30] for the "self-penalized" settings (see Fig. 4 for MSE comparisons and Fig. S3 for all coherences). a biophysically-realistic neural circuit model of two-alternative forced choices via the temporal accumulation of sensory evidence in two distinct neural populations. It takes a constant scalar input (representing a stimulus parameter such as the degree of coherence for randomly moving dot stimuli).It outputs an RT when the activity of either population reaches a decision threshold.However, the original WW model has limitations: its parameters must be manually tuned to fit observed data, and it is restricted to binary classification tasks with simple artificial parametric stimuli (e.g., Gabor patterns). To overcome these limitations, we extend the WW model. First, we replace the scalar input with a feedforward neural network (e.g., a CNN), enabling the model to process complex stimuli such as natural images. Second, we generalize the model from two populations to M populations, effectively increasing the number of neural populations to handle multi-class classification problems. Third, we RTify the model to make all parameters trainable via backpropagation. This allows the model to automatically learn the optimal parameters to fit human RTs (see Fig. Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: The limitations of the proposed method are discussed, the scope and assumptions are reflected.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
experiments: 
 In this section, we validate our RTify framework on two psychophysics datasets: the RDM dataset [24, 38] and a natural image categorization dataset [39] . As a side note, all models were trained on single Nvidia RTX GPUs (Titan/3090/A6000) with 24/24/48GB of memory each. All training can be completed in approximately 48 hours. We develop a multi-class compatible and fully differentiable RNN module based on the WW model [21, 22] . This module is implemented as an attractor-based RNN, and is stacked on top of a feedforward neural network. The feedforward neural network first takes an image as the input. Outputs from classification units of the network are then sent to RTified WW (A). Information is accumulated by multiple populations of neurons in RTified WW while they compete with each other (B). A decision is made and the process stops when one of the populations reaches a threshold. The number of time steps needed for the RTified WW to reach the threshold is used to predict human RT (C).The RDM task is a classic experimental paradigm used to test temporal integration that has been extensively used in psychophysics [40] , human imaging [41] , and electrophysiology studies [42] . The stimuli in this task consist of dots moving on a screen toward a predefined direction vs. randomly. For each time step, each dot only has a specific probability (coherence) (0.8%, 1.6%, 3.2%, 6.4%, 12.8%, 25.6%, or 51.2%) to move towards the pre-defined direction, making the task non-trivial. The participants must integrate motion information across time and report it when they are sufficiently confident. The original experimental data are from [24, 38] , where 21 young adult participants performed around 40,000 trials in total over 4 consecutive days.First, we trained an RNN consisting of 5 convolutional blocks (Convolution, BatchNorm, ReLU, Max pooling) and a 4096-unit LSTM with BPTT. In the original experiment, the stimuli were shown on a 75 Hz CRT monitor for up to 2 seconds, and therefore, we also trained our RNN for the RDM stimuli for 150 frames. The RNN was trained for 100 epochs using the Adam optimizer with a learning rate of 1e-4 at full coherence (c = 99.9%) for the first 10 epochs as a warm-up and 1e-5 at all coherence levels for the remaining 90 epochs. Next, we trained our two different RTify modules. For fitting human RTs, it was trained for 10,000 epochs, and for self-penalty, it was trained for 20,000 epochs. In both cases, the Adam optimizer were used, and the weights of the task-optimized RNN were frozen while training the RTify modules.We trained the first RTify module to predict human RTs by fitting human RT distributions. Here, positive RTs refer to RT with correct choices, and negative RTs refer to RTs with incorrect choices. Therefore, one distribution incorporates both speed and accuracy information from behavioral choices. Results are shown in Fig. 2 . Our RTify model can predict the full RT distribution across all coherence levels. In comparison, the entropy-thresholding approach by [29] fails to capture the full distribution (see Fig. 2 for coherence = 51.2% to 6.4% and Fig. S3 for all coherences). Importantly, our method A B Figure 4 : (A) MSE comparisons for the RDM task [24] for all coherence levels. The RTified model trained in the "supervised" setting (i.e., with human behavioral responses; green solid line) performs better (lower MSE) than entropy-thresholding [29] (brown solid line) under all coherence levels. Similarly, the RTified model trained in the "self-penalized" setting (i.e., without human data; green dash line) performs better than uncertainty proxy [30] (brown dash line). With the help of our RTified WW module (orange solid line), a convolution neural network (C3D) can also fit the data better than entropy-thresholding [29] . (B) Classification accuracy comparisons between pretrained and RTified models for the RDM task [24] . The RTified model trained with human RTs data in the "supervised" setting (green solid line) and in the "self-penalized" setting (green dash line) achieve human-like classification accuracy under all coherence levels compared with the pretrained model without RTify (green dotted line). With the help of our RTified WW module (orange solid line), a CNN (C3D) matches human accuracy better than the pretrained model without RTify (orange dotted line).surpasses entropy-thresholding approach [29] (two-sided Wilcoxon signed-rank test, p < .05; for MSE comparisons, see Fig. 4 ).It is well known that there is a trade-off between RTs and accuracy in cognitive tasks. To investigate this relationship further, we extended our analysis to examine whether the model's accuracy would approximate human accuracy when it was fitted solely on human RTs without using human accuracy data. Given that conventional RT distributions encompass both RTs and accuracy information, we restricted our fitting procedure to the positive part of the distribution. That is, RTs corresponding to correct responses to prevent inadvertently incorporating human accuracy into the model. Remarkably, by fitting the model on human RTs only, the model naturally reached a classification accuracy comparable to human performance (see Fig. 4 ).We also trained a self-penalized RTify module without any human data. This ideal-observer RNN model was trained to minimize the time steps needed for solving the RDM task (see section 2.2 for details). Human-like RTs emerge naturally in this neural network (See Fig. 2 ). Our model predicts RT data much better than previous approaches, which use a measure of uncertainty computed over the RNN as a surrogate metric. As can be seen, the resulting model tends to overfit the modes of the distribution.(see Fig. 2 for coherence = 51.2% to 6.4% and Fig. S3 for all coherences). Our method surpasses uncertainty proxy approach [30] (two-sided Wilcoxon signed-rank test, p < .05, for MSE comparison, see Fig. 4 ). We also checked the model classification accuracy before and after self-penalized RTify. Interestingly, we also found the self-penalized RTified RNN demonstrated a human-like classification accuracy (see Fig. 4 ).Unlike previous visual decision-making models, we want to show that our method can also be applied to natural images and multi-class datasets. Specifically, we consider an object recognition task, a classic paradigm used extensively in computer vision [43, 44] and cognitive neuroscience studies [45, 46] . The original data is from [39] , where 88 participants perform the task through Mturk. The stimuli in this task belong to 10 categories, and for each category, there are 20 natural imagesExtracting RTs via self penalty (leveraging RNN dynamics)Extracting RTs with supervision (fitting human RTs) A BFigure 5 : RTified model evaluation on an object categorization task [39] . Model vs. human RT predictions for our RTified model (green) vs. alternative approaches (brown) (A) in the "supervised" setting where human behavioral responses are used to train the model and (B) the "self-penalized" setting where no human data is used. Solid lines are linear regression fits between model and human RTs. Crossed-shaded areas and the dashed lines are controls to show the fits after removing the highest model RTs. Our approach outperforms the two alternative approaches, i.e., entropy-thresholding [29] for the "supervised" setting and uncertainty proxy [30] for the "self-penalized" setting.taken from the COCO dataset [47] and 112 synthetically generated images with different backgrounds and object positions.We first train our RNN with BPTT to perform a 10-way classification task. In the original study, participants performed a binary classification task. However, since the individual binary pairs were not saved, we trained the model in a 10-class classification task. We used Cornet-S [48] pretrained on the Imagenet dataset [43] because it was used in the original study and it achieves a relatively high brainscore in terms of explaining neural activities [49] . We trained the network for 100 epochs, using the Adam optimizer with a learning rate of 1e-4 and a learning scheduler (StepLR) with a step size of 2,000. We used 20 timesteps (instead of 2 in the original model) for the IT layer in the Cornet to achieve high temporal resolution. Results show that our RNN achieves 75.2% on a held-out test set. Similarly, we trained our two different RTify modules. For fitting human RTs, it was trained for 100,000 epochs, while for self-penalty, it was trained for 10,000 epochs with a learning scheduler (StepLR) with a step size of 2,000 and a gamma of 0.3. In both cases, Adam optimizers were used, and the weights of the task-optimized RNN were frozen while training the RTify modules.We then extracted RTs from our RNN to fit human RTs. Notably, model RTs significantly correlate with the human RT observed in the psychophysics experiment (r = .51, p < .001). Besides, our method also surpasses entropy-thresholding [29] (bootstrapping shows that our method is superior to theirs with a probability of 99.9%). We also extract RTs from an ideal-observer RNN model trained with a time self-penalty (see section 2.2 for details). We show here that model RTs are significantly correlated with human RTs (see Fig. 5 , r = .40, p < .001). This failed to be captured by uncertainty proxy [30] . We argue this in two ways. First, bootstrapping shows that our method is superior to theirs with a probability of 87.1%. Second, and most importantly, although the uncertainty seems to correlate with human RTs in the dataset (r = .36, p < .001), it only applies to high-uncertainty cases.When trials with high model uncertainty are excluded, uncertainty shows no significant correlation with human RTs (r = .05, p = 0.32). However, using our method, the correlation remains strong and significant (r = .41, p < .001).Given the prevalence of feedforward networks (e.g. CNNs) and their incredible performance in visual tasks, a natural question is how to align such networks in the temporal domain of decision-making. We thus developed a biologically plausible, multi-class, differentiable RNN module based on the WW recurrent circuit model [21, 22] . This module can be stacked on top of any neural network, even if not recurrent, and can be used to align model RTs with human RTs.For the RDM task, we take a 3D CNN with 6 convolutional blocks (convolution, BatchNorm, ReLU, Max pooling) and an MLP. We train the network for 100 epochs using the Adam optimizer with a learning rate of 1e-4 at full coherence (c = 99.9%) for the first 10 epochs as a warm-up and 1e-5 at all coherence levels for the remaining 90 epochs. Since this model is not an RNN, it has no temporal dynamics. Therefore, we drop in our WW module and further train the WW module for 5,000 epochs using the Adam optimizer with a learning rate of 1e-4, a StepLR scheduler with a step size of 1,000 and gamma of 0.3, and a grad clip at 1e-5. Interestingly, when we RTify the C3D model using the WW module, it is able to capture the distribution of human RTs across all coherence levels, see Fig. 6A for coherence = 51.2% to 6.4% and Fig. S4 for all coherences, for MSE comparison see Fig. 4 ). Furthermore, we also observed a human-like classification accuracy for the model when it is solely trained to fit human RTs but not human accuracy (see Fig. 4 ).Similarly, we take a VGG-19 pre-trained on Imagenet for the object recognition task and fine-tune it on the dataset provided by Kar et al. [39] in a 10-class classification way for the abovementioned reasons. We train the model for 100 epochs using a batch size of 32. The optimizer was AdamW, with a learning rate of 1e-5. We use a OneCycleLR scheduler adjusted after 10 epochs of warm-up. Results show that our RNN achieves 81.6% on a held-out test set. We further train the WW module for 100,000 epochs using the Adam optimizer with a learning rate of 1e-4 and a grad clip at 0.0001. By using the multi-class version of the WW model, we show that the RTify VGG also exhibited a significant correlation with human data (r = .49, p < .001, see Fig. 6B ).Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes] Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer:[Yes]Justification: The code and datasets used in the experiments are publicly available in our GitHub, with detailed instructions provided.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
conclusion: 
 We have described a computational framework to train RNNs to learn to dynamically accumulate evidence nonlinearly so that decisions can be made based on a variable number of time steps to approximate human behavioral choices, including both decisions and RTs. We showed that such optimization can be used to fit an RNN directly to human behavioral responses. We also showed that such a framework can be extended to an ideal-observer model whereby the RNN is trained without human data but with self-penalty that encourages the network to make a decision as quickly as possible.Under this setting, human-like behavioral responses naturally emerge from the RNN -consistent with the hypothesis that humans achieve a speed-accuracy trade-off. Finally, we provided an RNN implementation of a popular neural circuit decision-making model, the WW model, as a trainable deep learning module that can be combined with any vision architecture to fit human behavioral responses. Our computational framework provides a way forward to integrating image-computable models with decision-making models, advancing toward a more comprehensive understanding of the brain mechanisms underlying dynamic vision.Limitations Certain limitations will need to be addressed in future work. Most of the human data used in our study remains relatively small-scale and is limited primarily to synthetic images because more naturalistic benchmarks only include behavioral choices [44, 50] and lack RT data. To properly evaluate our approach and that of others, larger-scale psychophysics datasets using more realistic visual stimuli will be needed. There is already evidence that large-scale psychophysics data can be used to effectively align AI models with humans [51, 52] . We hope this work will encourage researchers to collect novel internet-scale benchmarks that include both behavioral choices and RTs.Broader Impacts As AI vision models become more prevalent in our daily lives, ensuring their trustworthy behavior is increasingly important [53, 54] . Our framework contributes to this effort by exploring how to align certain aspects of models' behavior with human responses in specific contexts. While our approach is limited to predicting RT distributions, it constitutes a first step toward more human-aligned AI models.A Appendix / Supplemental material EQUATIONProof. By definition,τ * θ (Φ t ) = min{t ∈ R [1,N ] : Φ t > θ}.Let us consider a small change δΦ. By the Taylor expansion, we can write:τ * θ (Φ t + δΦ) ≈ τ * θ (Φ t ) + ∂τ * θ ∂Φ t δΦ.On the other hand, t is considered a continuous value. By our definition, we can induce a small change in the value of Φ t by introducing a small change in time δΦ, which we can write in the following way:Φ t+δt ≈ Φ t + ∂Φ t ∂t δtWithout loss of generality, let us take δt = ∂Φt ∂t -1 δΦ and thenΦ t+δt ≈ Φ t + ∂Φ t ∂t ∂Φ t ∂t -1 δΦ = Φ t + δΦ Now, τ * θ (Φ t + δΦ) = min{t ∈ R [1,N ] : Φ t + δΦ > θ} ≈ min{t ∈ R [1,N ] : Φ t+δt > θ} = min{t ∈ R [1,N ] : Φ t+( ∂Φ t ∂t ) -1 δΦ > θ}Note that, by definition, if the evidence is evolving by Φ t+∆ . It just means the whole function is moving along the t-axis, and therefore the minimum time to pass the threshold would be given by τ * θ -∆, thereforemin{t ∈ R [1,N ] : Φ t+( ∂Φ ∂t ) -1 δΦ } = min{t ∈ R [1,N ] : Φ t } - ∂Φ t ∂t -1 δΦ = τ * θ (Φ) - ∂Φ ∂t -1δΦ Finally, we can join the two sides and obtain:τ * θ (Φ t + δΦ) ≈ τ * θ (Φ) + ∂τ * θ ∂Φ δΦ = τ * θ (Φ) - ∂Φ ∂t -1 δΦFrom this, we can deduce:EQUATIONSo the first part of Eq 2 in the main text is proved.Similarly, we can derive Combining excitatory and inhibitory currents 7:∂τ * θ ∂θ ≈ 1 Φ t -Φ t-1 . 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes] Justification: Comprehensive details on the training and test settings, including data splits, hyperparameters, and optimizer types, are provided.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: The experimental results are presented with appropriate information about the statistical significance.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: The paper specifies the types of compute resources used.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: We discussed the broader impacts of the proposed method, including potential applications.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: The paper does not involve the release of high-risk data or models.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes]Justification: All external assets used in the paper are properly credited.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA]Justification: The paper does not release new assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [Yes]Justification: The paper only involves published human data where all relevant experimental information can be found in the original papers.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [Yes]Justification: The paper only involves published human data where all IRB information can be found in the original papers.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper. • We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: H 1 ← f(J 11 • s 1 -J 12 • s 2 + I 0 + I 1 + I OU noise1 ) 8: H 2 ← f(J 22 • s 2 -J 21 • s 1 + I 0 + I 2 + I OU noise2 ) 9:Calculate rate of change 10:ds 1 dt = -s 1 τ s + H 1 • (1 -s 1 )11:ds 2 dt = -s 2 τ s + H 2 • (1 -s 2 ) 12:Update s 1 , s 2 , decision times 13: end while 14: return decision timesFigure S2 : Illustration of the RTified WW circuit. To RTify feedforward neural networks, we used an RNN based on the WW circuit. we consider a binary classification on random moving dots for illustration (A). When receiving a visual input, the two populations sensitive to left/right directions compete with each other (B) and accumulate evidence until one of them reaches a threshold (C). The number of time steps needed for the RNN to reach the threshold is defined as the "model RT" and is used to predict human RT. We provide pseudo-code for a more detailed description of the circuit (D). Here, Figure S3 : RTified model evaluation on a RDM task [24] across all coherences. Human data are shown as a gray shaded area, and model fits are shown for (A) the "supervised" setting where human behavioral responses are used to train the models and (B) the "self-penalized" setting where no human data is used. Our approach (green) outperforms the two alternative approaches (brown), i.e., entropy-thresholding [29] for the "supervised" and uncertainty proxy [30] for the "self-penalized" settings.f (x) = γ • max(ax -b, 0)/(1 -exp(-d(ax -b))) isIncorrect Correct Incorrect Correct Incorrect Correct Incorrect Correct Incorrect Correct Incorrect Correct Incorrect CorrectExtracting RTs with supervision (fitting human RTs)Figure S4 : RTified WW model on a RDM task [24] across all coherences. We combined our RTified WW module with a 3D CNN to fit human RTs collected in an RDM task [24] . Human data are shown as a gray shaded area and model results are shown as orange solid lines.1. Claims Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: The abstract and introduction clearly state the main contributions of the paper. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.Justification: The research conforms to the NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 


Decision: This submission addresses the challenge of modeling human reaction times (RTs) using recurrent evidence accumulation. The main contribution of this work is a differentiable approximation of simulated RTs. This approximation enables the incorporation of RTs, operationalized as the number of recurrent computation steps until an adaptive stopping criterion is met, into the training loss. This approach is employed to train networks either to fit human RTs or to reach decisions as rapidly as possible. In the latter application—training networks to reach decisions quickly—the authors demonstrate improvements over state-of-the-art methods. To the best of my knowledge, the former application—directly fitting human RTs—is novel.

Most of the reviewers believe that the contribution of this work warrants publication at NeurIPS, and I share their assessment. However, the reviewers commonly pointed out several weaknesses: overreaching conclusions, insufficient discussion of limitations, and a lack of clarity in some technical sections, particularly regarding the Wong-Wang RNN models. The authors have committed to addressing these issues in the revised manuscript. It is crucial that the authors provide well-documented and functional public code. Another point to consider in the revision is the omission of a relevant reference: Rafiei et al. (2024), published online shortly before the submission deadline:

Rafiei F, Shekhar M, Rahnev D. The neural network RTNet exhibits the signatures of human perceptual decision-making. Nature Human Behaviour. 2024 Jul 12:1-9.

Reviews: review: paper_summary: paper_summary: The authors present RTify, a novel approach that leverages Recurrent Neural Networks (RNNs) to model decision response times. RTify offers a dual benefit: it can align human and RNN response times, and it can self-supervise RNNs to optimize the speed-accuracy tradeoff. Through evaluations on both synthetic and natural image recognition tasks, the authors demonstrate that RTify outperforms existing methods (Goetschalckx et al., 2024) in fitting human response times, achieving superior results in both supervised and unsupervised settings.
strength_weakness: strength_weakness: Weaknesses:
- Although the proposed RTify method is a significant extension of prior art in modeling human response times, it bears notable similarities with existing approaches, such as Adaptive Computation Time [1] and its adaptation to ConvRNNs, AdRNNs in [2]. A more explicit discussion of these similarities and differences would strengthen the paper. Specifically, the analogy between [1, 2]'s halting probability ($p_t$, $P_t$) and RTify's evidence accumulation ($e(t)$, $\phi(t)$), as well as the resemblance between the self-penalized loss function in Eq. 4 and the objective functions in [1] and [2], warrants clarification. The key distinction between RTify and ACT appears to be the learnability of $\tau$ in RTify, whereas ACT treats this threshold as a hyperparameter (to clarify, I find closing this gap to be a strong contribution); a clear discussion of this difference (and any others that I have missed) would be beneficial for the readers.
- In lines 67-69, the authors argue that prior work fails to provide a mechanistic account of the underlying decision processes governing temporal dynamics. However, the current work I believe also suffers from this limitation. This paper may further benefit from a more detailed mechanistic explanation of the temporal dynamics of decision-making in RTify. While the paper demonstrates impressive empirical results, a deeper understanding of the underlying mechanisms would further enhance its impact.
- Minor: please check for spelling errors (e.g. add subscript t to $\phi$ in Figure 1, "Neural Netwok" -> "Neural Network" in same figure etc.). Changing the variable name for halting step $\Tau_\tau$ would improve readability as the threshold for evidence accumulation ($\tau$) also uses the same alphabet. 

References:
1. Graves, A. (2016). Adaptive computation time for recurrent neural networks. arXiv preprint arXiv:1603.08983.
2. Veerabadran, V., Ravishankar, S., Tang, Y., Raina, R., & de Sa, V. (2024). Adaptive recurrent vision performs zero-shot computation scaling to unseen difficulty levels. Advances in Neural Information Processing Systems, 36.
questions: questions: NA. Please refer to my review above.
limitations: limitations: Yes, the authors have sufficiently discussed limitations of their proposed work.
paper_summary: paper_summary: The paper introduces a new framework for training vision systems using human reaction times. The approach allows for dynamic integration of visual reasoning with decision making by incorporating human behavior over time. The approach is shown to be beneficial over a range of psychophysics tasks.
strength_weakness: strength_weakness: * The paper doesn’t explain why human RTs are useful. I read the introduction a few times and couldn’t explain the intuition about what incorporating RTs is solving. At some level, I believe there is an intuitive explanation: reaction times give a distribution of responses over time and therefore, this measure directly incorporates decision-making (a process of time) and visual reasoning at each time step. 
* A more robust explanation of human RT would be extremely helpful to explain more of the deep intuition. This might be entirely obvious but the paper makes RT seem like a novel contribution without explaining why it might be important. 
* It was a bit hard to connect your results to the reported performance of the models. Could results like ones reported in line 277 be put in a table? 
* (Nit) I found figure 7a a bit hard to understand. What do the correct and incorrect arrows mean?
questions: questions: * I was wondering how RT impacted performance for a while. Was there a good sense of this? Maybe the presentation could be cleaned up to present this more systematically. 
* I wasn’t able to get a sense of runtime for the WW circuit. This may also need discussion.
limitations: limitations: * I feel like some limitations were missing. For example, even though you don’t theoretically need human data in the self-supervised set-up, from my perspective, there is some reliance on human data for validation of this approach. Does this bottleneck the amount of data you can use?
paper_summary: paper_summary: Here the authors present a method for fitting neural network outputs to reaction times. The main technical step is setting up the calculation of a reaction time as an accumulation of a decision signal over time steps and creating a differentiable computation of the time when this accumulation crosses a threshold. Here the authors use a simple linear interpolation between the time points to allow reaction times between the time points and a differentiable dependency on the input signal for accumulation. The authors use this possibility to optimise reaction times to match average reaction times of human observers and to minimise reaction time of networks optimised for performance. By using an existing network architecture (“Wong-Wang”) the authors are able to create realistic reaction time distributions as well.
strength_weakness: strength_weakness: While the method presented here is fairly general it is still fairly specific in terms of the data it applies to: Timed categorisation tasks without cues, planning, rewards etc. This is a step forward from just accuracy, but clearly far from all decision making behaviour of humans.

While the deep neural network based methods are sensible comparisons, the adapted Wong-Wang architecture received little test in this manuscript. For insights into the modelling, I believe a more thorough discussion of the concrete model used and which aspects of it are relevant for getting the RT distributions right could improve my trust into this method.

The data used here for the natural image categorisation task is not a great choice. While it has a substantial amount of images it is an MTurk dataset with corresponding limitations in accuracy of stimulus presentations and measurements. There are plenty of better controlled psychophysics datasets that could be used for evaluations of RT models.
questions: questions: Can this approach be extended towards creating predictions for other manipulations of reaction time?
limitations: limitations: While I don’t think the authors are hiding any negative social impacts or severe limitations, I do think they are vastly overestimating the broader impact of a method for predicting RT distributions based on neural networks. I like this work, but it does not allow alignment with the full spectrum of human behavioural data, it will not make DNNs more trustworthy for now and there is exactly one supervised and one less supervised situation, not many.
paper_summary: paper_summary: The  paper proposes a novel framework to align the temporal dynamics of RNNs and human reaction times for visual recognition tasks. The framework starts from a task-optimized RNNs and trains a function $f_w$ to transform the activity (hidden state) into a real-valued evidence measure $e(t)$ (learnable) that will be integrated over time by an evidence accumulator (seems to be a summation). To train the stopping threshold and the function $f_w$, authors propose 2 approaches: supervised and self-supervised.
strength_weakness: strength_weakness: 1. It sounds like the paper writing is a bit overcomplicated. The actual approach is applying RNN (in the WW form that is not clear) on top of the model like CNN, and then learning to reduce the number of RNN iterations via the supervision or via regularization. 
2. Benchmarks selected for visual experiments are not strong. Why not to consider CIFAR10/100, ImageNet, COCO detection etc? 
3. Details of WW models are not mentioned in the main paper. Probably, listing PyTorch pseudo code for forward pass will be helpful here. 
4. The problem of adaptive compute is not novel for RNNs. Please consider comparing with other methods in the filed like ACT ("Adaptive Computation Time for Recurrent Neural Networks" by Alex Graves).
questions: questions: 1. The formulation of $\Phi$ function is not clear, and how the loss is actually propagated in (3) and (4). 
2. Figure 3, the Wong-Wang model is mentioned, the structure should be illustrated for the reader to understand it without going to the reference. This is because RTifing is one of the main contributions of the paper. 
3. If understand correctly, the dynamic computation time comes only from WW model, and the CNN stays exactly the same, and runs only once. Probably CNN run time dominates the full model, please comment on this.
limitations: limitations: Not listed in the paper as a separate section.

Comments: title: Update on review
comment: Dear reviewer, 

In summary your feedback can be listed in three main points. Let us  respond to each one of them : 

1. *WW is still not clearly explained for practitioners, and providing the source code would help with this* 

**We provided the source code in the original submission. See wong_wang.py** in the supplementary.zip,  for details on the implementation of WW. And **we emphasize** that our method is suitable for any reasonable decision-making circuit besides WW, in fact any RNN. 

2. *"I also don't see a significant breakthrough in the method itself, as it seems to be an RNN on top of features"*

We respectfully disagree. Our main approach is not simply to place an RNN on top of features, but to have the **RNN adaptively decide to stop based on the features of the RNN itself**.Our technical breakthrough lies in the differentiability of time steps, which is neither solved in previous adaptive computing methods for RNNs (e.g. ACT), nor solved in human decision models (e.g. DDM).  As we pointed out in our rebuttal. Unlike previous phenomenological models (e.g. DDM), our method allows us to develop image-computable, nonlinear adaptive models of decision making.

3. *"I suggest that the authors focus on explaining the approach in more detail and ensure that the explanation benefits practitioners who are not experts in the field of reaction time."*

We thank the reviewer for the feedback on explaining the method to a broader audience, which we will take into account for the camera-ready version. In general, however, we believe that this paper provides an important tool for neuroscience researchers to link the phenomenon of reaction times to neural mechanisms of decision making.
I would like to thank the authors for the rebuttal. Perhaps due to my limited knowledge in the field of human reaction time, I am unable to fully grasp the potential of this paper. My current feedback is that WW is still not clearly explained for practitioners, and providing the source code would help with this. I also don't see a significant breakthrough in the method itself, as it seems to be an RNN on top of features. Therefore, I maintain my score of Borderline Reject; however, I would not oppose the acceptance of this paper given the more positive reviews from others. I suggest that the authors focus on explaining the approach in more detail and ensure that the explanation benefits practitioners who are not experts in the field of reaction time.
Thank you for the careful response. I have read it over and believe all my concerns are addressed. My familiarity with psychophysics tasks is a bit low so I will keep my score as is to reflect my lack of familiarity with the domain.
I read the author rebuttals and came to the conclusion that my initial assessment was fair. This provides a nice solution to the relatively narrow field of using human RTs for fitting or evaluation.

Actually reducing the overstatements will be appreciated!