To Learn or Not to Learn, That is the Question — A Feature-Task Dual Learning Model of Perceptual Learning

Authors: xiao  liu
muyang  lyu
cong  yu
si  wu
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 11:43:34

Abstract: Perceptual learning refers to the practices through which participants learn to improve their performance in perceiving sensory stimuli. Two seemingly conflicting phenomena of specificity and transfer have been widely observed in perceptual learning. 

Here, we propose a dual-learning model to reconcile these two phenomena. The model consists of two learning processes. One is task-based learning, which is fast and enables the brain to adapt to a task rapidly by using existing feature representations. The other is feature-based learning, which is slow and enables the brain to improve feature representations to match the statistical change of the environment. Associated with different training paradigms, the interactions between these two learning processes induce the rich phenomena of perceptual learning. Specifically, in the training paradigm where the same stimulus condition is presented excessively, feature-based learning is triggered, which incurs specificity, while in the paradigm where the stimulus condition varies during the training, task-based learning dominates to induce the transfer effect. As the number of training sessions under the same stimulus condition increases, a transition from transfer to specificity occurs. 

We demonstrate that the dual-learning model can account for both the specificity and transfer phenomena observed in classical psychophysical experiments. We hope that this study gives us insight into understanding how the brain balances the accomplishment of a new task and the consumption of learning effort.

Main paper: 
 introduction: 
 Perceptual learning refers to the practices through which human participants learn to improve their performances of perceiving sensory stimuli [1] . Perceptual learning has been widely used as a research paradigm to ascertain the learning strategies in the brain. Over the years, two seemingly contradictory phenomena have been widely found in perceptual learning. One is specificity, referring to that after learning, the improved performance of participants is restricted to the trained location or the trained feature of the stimulus [2, 3] . The other is transfer, referring to that after learning, the improved performance is transferable to untrained locations or untrained features of the stimulus [4, 5, 6] . In early experiments in which participants were typically trained excessively under the same stimulus condition (e.g., either at the same retinal location or presenting the same stimulus feature), specificity was predominantly found, and stimulus features used in experiments include contrast [7, 8] , orientation [9, 10, 11, 12] , spatial frequency [13, 11, 12, 14] , motion direction [15, 16, 17] , and retinal location [18, 19, 20, 14] . Thus, once in a while, specificity was regarded as a hallmark distinguishing perceptual learning from other learning types [3] . Later on, enriched training paradigms were used, and it was found that perceptual learning can also exhibit the dominating transfer effect. For instances, by manipulating the task difficulty [21, 22, 23, 24] , the training intensity [25, 26, 27] , or the training protocol [28, 29, 30, 31, 32, 33, 34, 35, 36] , the originally observed specificity-dominating effect become transfer-dominating. Related works. A large volume of computational models has been proposed in the literature to unveil the mechanism of perceptual learning. Since specificity was predominantly reported in early experiments, early computational models mainly focused on exploring the neural mechanism underlying specificity. For instance, Poggio et al. proposed the hyper basis function model (HyperBF) [37] , which considers that the enhanced perception performance comes from the changes of basis functions or connection weights of neurons; Teich and Qian proposed a model which considers that the enhanced performance comes from the changes of tuning curves of low-level neurons [38] ; Petrov et al. proposed the reweighting model, which considers that the enhanced performance comes from the read-out weights from feature representations to decision neurons [39, 40] . All these models implicitly assume that perceptual learning only incurs local changes in the network, and hence, the learning effect is localized and not transferable to other parts of the neural system needed for processing untrained conditions. Later on, motivated by the observation of transfer, new computational models focusing on the transfer mechanism were proposed. For examples, Dosher et al. refined their reweighting model by including an extra layer with network-wide connections to implement translation-invariant feature representation, which achieves a certain level of transfer effect [41] ; Solgi et al. introduced an extra off-line learning procedure which generalizes the local learning result to untrained locations/features, making the learning performance transferable [42] ; Li et al. employed a convolutional neural network (CNN) to simulate perceptual learning [43] , utilizing that the CNN implements translation-invariant feature extraction, and hence their model can explain some transfer phenomena. While these models may solve the transfer problem, they fail to account for specificity, which was also vividly observed in experiments. Overall, perceptual learning still lacks a unified computational model accounting for both specificity and transfer that have been observed in different experiments.In this study, we propose a novel computational model to reconcile the seemingly conflicting phenomena of specificity and transfer in perceptual learning. Our model is based on the fact that in the experiments observing specificity, participants were often excessively trained under the same stimulus condition (either at the same retinal location or presenting the same stimulus feature); while in the experiments observing transfer, the stimulus condition was varied during the training (either the location or the stimulus feature). This indicates that excessive training of the same stimulus condition is critical to induce specificity. To account for both specificity and transfer arising from different training paradigms, we propose a dual-learning framework for perceptual learning in the brain. The framework consists of two intertwined learning processes: one is task-based, and the other is feature-based. Specifically, task-based learning aims to quickly adapt to a new task by utilizing existing neural representations of the external world. It enables the brain to master new skills quickly and is transferable to untrained conditions. On the other hand, feature-based learning aims to refine neuronal representations to reflect the statistical changes of the external world. It enables the brain to represent the external world faithfully and it only takes effect when the external environment is substantially changed. We argue that in the experiments exhibiting transfer, task-based learning dominates; while in the experiments exhibiting specificity, feature-based learning dominates, and the latter is triggered by the neural system being excessively exposed to the same stimulus condition, triggering feature-based learning to catch up the statistical change of the external environment. Since featured-based learning leads to localized changes in feature representations adhered to the trained condition, its effect is not transferable to untrained conditions.To substantialize the dual-learning framework, we built a hierarchical neural network model, which consists of three sequential stages of information processing responsible for, respectively, feature extraction, feature-based learning, and task-based learning. Specifically, we use a classical basis function network [37] to implement feature extraction representing preliminary image processing, a feedforward network with unsupervised Hebbian learning to implement feature-based learning, and a CNN with global max pooling to represent task-based learning. We set the rate of feature-based learning to be much smaller than that of task-based learning, such that task-based learning will dominate initially in training, and feature-based learning will take effect after the same stimulus condition is presented excessively. We demonstrate that the dual-learning model can well explain the specificity and transfer phenomena observed in classical psychophysical experiments.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: The main claims in the abstract and introduction are well-aligned with the contributions and scope of the paper.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
specificity vs. transfer in perceptual learning: 
 Various tasks, ranging from the discrimination of basic visual features such as Vernier or orientation to the recognition of visual images such as motion or face, have been applied to study perceptual learning. These experiments unveiled three generic characteristics of perceptual learning, including specificity, transfer, and a transition from transfer to specificity when the number of training sessions under the same stimulus condition increases. They are reviewed below.We use a Vernier discrimination task to introduce the specificity effect [28] . In this task, participants learned to discern which of two vertical orientations was more leftward (or equivalently rightward) (Fig. 1A(i) ). The stimuli were always presented at the same retinal location. Through intensive training, participants progressively improved their sensitivity to the offset between two orientations until reaching a threshold (the blue dashed line in Fig. 1A (ii)). After training, it was found that participants could not generalize their enhanced perceptual performances to untrained retinal locations, displaying the effect of specificity (red squares in Fig. 1A (ii)). We use an orientation discrimination task to introduce the transfer effect [34] . In this task, participants learned to distinguish which of two successively presented orientations was more clockwise (or equivalently more anticlockwise) (Fig. 1B(i) ). By training with varied combinations of retinal location and orientation, participants progressively improved their sensitivity to the offset between two orientations until reaching a threshold (the green dashed line in Fig. 1B (ii)). After training, it was found that participants could generalize their enhanced perceptual performances to untrained conditions, displaying the effect of transfer (red circles in Fig. 1B (ii)).The above two experiments highlight the importance of the number of training sessions under the same stimulus condition in inducing the specificity or the transfer effect. It is expected that with the increased number of training sessions under the same condition, the learning effect should go from transfer to specificity gradually. Indeed, this was confirmed experimentally in an orientation discrimination task [26] . In this task, participants underwent different numbers of training sessions under the same condition, followed by a switch to untrained conditions to measure the extent of transfer (Fig. 1C(i) ). It was found that with the increased number of training sessions under the same condition, the perceptual performance of participants was improved (the left panel of Fig. 1C (ii)), whereas the transfer effect was decreased (the right panel of Fig. 1C (ii))Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes]Justification: The paper includes a discussion section that addresses the limitations of the work.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
the dual-learning model: 
 To reconcile the phenomena of specificity and transfer in perceptual learning, we propose a duallearning model. As depicted in Fig. 2 , the model consists of three sequential information processing stages, which are feature extraction, feature-based learning, and task-based learning. We use the Vernier discrimination task as an example to introduce the model. 3.1 The model structure Feature Extraction. This models the functions of the retina, the Lateral Geniculate Nucleus (LGN), and the input layer of V1 in the visual pathway, extracting preliminary features from input images to form retinotopic feature representations (see Fig. 2A ). There is no plasticity at this stage, and we employ the HyperBF network [37] to model feature extraction. Denote I t (x) the input image presented at trial t and F * t (x, θ) the extracted feature representation, which is expressed as:EQUATIONwhere x and θ denote the position and orientation feature in the image, respectively. The variable x ′ represents the position of neighboring pixels, used in the convolution operation. The Gabor function Feature-Based Learning. This models the plasticity in the early visual cortex, which refines feature representations to capture the statistical change of the external environment (see Fig. 2B ). For simplicity, we employ a feedforward network with unsupervised Hebbian learning to implement this process. Denote F t (x, θ) the refined feature representations at trial t, which is given byG(x-x ′ , θ) = 1/2exp -(x -x ′ ) 2 /(2σEQUATIONwhere W t (x, x ′ ) represents the feedforward connection of the network (Fig. 2B ). W t (x, x ′ ) is updated following the competitive Hebbian learning rule, which is expressed asEQUATIONwith η as the learning rate. The detail of ∆W t (x, x ′ ) is presented in Sec. A.2 in the Appendix. This learning rule enhances neuronal representations at locations where the stimulus is frequently presented, thereby inducing location-specific changes in feature representations.Task-Based Learning. This models the information read-out process in the higher visual cortex (see Fig. 2C ). We use a small convolutional network with three convolutional layers, followed by global max pooling, to implement this process. Denote z * t the decision at trial t, which takes a value of (0, 1) representing the upper orientation is more leftward or more rightward, respectively. During decision-making, if the value is greater than 0.5, it is classified as 1; if it is less than 0.5, it is classified as 0. Its value is calculated by,EQUATIONwhere Φ(•) represents the multi-layer convolution operation that integrates task-related information from the feature representation F t (x, θ). We used a Vernier discrimination task to analyze the properties of the dual-learning model. The model outputs 0 if the upper orientation is more leftward compared to the lower one and outputs 1 otherwise. We trained the model at a fixed location and then tested its performances at three untrained locations having increasing distances to the trained one (Fig. 3A ). For the details, see Sec. B in the Appendix. We conducted the below ablation studies.First, we froze feature-based learning and only enabled task-based learning. As shown in Fig. 3B , the discrimination accuracies at both trained and untrained locations increase gradually with train epochs, indicating that task-based learning facilitates transfer over locations. This property comes from the fact that task-based learning employs a maxing pooling operation, which is translation-invariant.Second, we froze task-based learning and only enabled feature-based learning. The weights of the readout layer were initialized after completing task-based learning, as described above. As shown in Fig. 3C , We see that the discrimination accuracy at the trained location keeps a relatively high value, whereas the accuracies at untrained locations drop quickly to a nearly chance level (50%). This demonstrates that feature-based learning induces specificity, reinforcing the model's performance at the trained location while diminishing its transfer to untrained locations. To display the location-specific change in feature representation, we calculated neural representation changes at different locations during the training. As shown in Fig. 3D , we see that as the training goes on, the neural representation at the trained location remains stable (high similarity), while the neural Finally, we analyzed the model performance with both feature-based and task-based learning enabled.In particular, we set the rate of feature-based learning to be relatively much smaller than that of task-based learning. Although a direct numerical comparison between the two learning rates is not meaningful due to differences in their roles and magnitudes, the chosen configuration allows us to observe a clear distinction in the dynamics of the two learning processes. As shown in Fig. 3E , we observe that: 1). The combination of feature-based and task-based learning accelerates the training process at the train location, requiring fewer epochs to reach 90% accuracy compared to using taskbased learning alone (gray line in Fig. 3 , mean epochs: task-based learning = 84.7, combined featureand task-based learning = 81.3). This difference is statistically significant (t = 3.90, p < 0.001), based on 50 repetitions for each condition. 2). The training promotes transfer to untrained locations initially, but as time goes on, the transfer effect decreases, while the specificity effect increases. This shift reflects that due to different learning rates, task-based learning dominates initially, which induces transfer; while feature-based learning gradually takes effect, which induces specificity.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes] Justification: The authors have stated that they will provide a GitHub link with the data and code after the anonymous review process.Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
reproducing classical findings in perceptual learning: 
 In this section, we used the dual-learning model to reproduce the classical findings in perceptual learning. We employed a Vernier discrimination task as an example, as this paradigm has been widely used in psychophysical experiments. In these experiments, the discrimination threshold, defined as the intensity of stimuli that participants can accurately discriminate with about an 80% success rate, was used to measure the learning effect. In our simulations, we adjusted the offset between two vertical orientations in the task to create a series of difficulty levels and chose the offset value corresponding to 80% model correctness as the threshold (for details, see Sec. C in the Appendix).In the first experiment, we adopted the training paradigm similar to that in [28] . Specifically, we trained the model with fixed orientation and location (ori1_loc1) and evaluated model performances across various combinations of orientation and location before and after the training (Fig. 4A ).The results are presented in Fig. 4B , which show that the model exhibits a significantly improved performance in the trained condition (ori1_loc1, blue diamonds), whereas this improvement is not transferable to untrained locations (ori1_loc2, light red squares), untrained orientations (ori2_loc1, medium red triangles), or combinations of untrained location and orientation (ori2_loc2, dark red circles). Fig. 4C further summarizes the learning improvements under different conditions, showing good agreement with the experimental data [28] . The results demonstrate that our model successfully replicates the classical specificity phenomenon observed in perceptual learning. In the second experiment, we adopted the training paradigm similar to that in [34] . Specifically, the model was trained with stimuli presented at varying locations, either in a random or sequential order, and the transfer effect was evaluated at an untrained location (Fig. 5A ). The results are presented in Fig. 5B-C , showing that the model's learning effect is successfully transferred to the untrained location. Fig. 5D further summarizes the learning improvements when trained across multiple locations either randomly or sequentially. These results agree well with the experimental data [34] , demonstrating that our model successfully replicates the classical transfer phenomenon in perceptual learning.In the third experiment, we adopted a training paradigm similar to that in [26] . Specifically, the number of training sessions was varied, and the transfer effect was evaluated using a new condition that combined an untrained location and orientation (Fig. 6A ). The results are shown in Fig. 6B . In the trained condition (left panel), performance thresholds gradually decreased with the increasing number of training sessions. However, in the untrained condition (right panel), the trend reversed: the more sessions completed in the trained condition, the higher thresholds were observed in the untrained condition. Fig. 6C further summarizes the progression of learning improvements during the training and transfer phases, agreeing well with the experimental data [26] . Thus, our model successfully replicates the classical phenomenon of transition from transfer to specificity with the increasing number of training sessions.In the fourth experiment, we adopted the training paradigm called double training similar to that in [28] . In this experiment, following the first step training of the classical specificity task as in Fig. 1 In summary, the adaptability of perceptual learning is governed by the interaction between specific feature-based learning at lower levels and transferable task-based learning at higher levels. Conditionspecific training leads to the dominance of feature-based learning, resulting in significant specificity.In contrast, training with varied conditions allows task-based learning to dominate, enhancing transfer. This difference enables a transition from transfer to specificity as stimulus repetitions increase.Although not exclusively from the Vernier discrimination task, the observed changes in specificity and transfer are influenced by training paradigms rather than the tasks themselves. The simplicity of the Vernier discrimination task, requiring only a single stimulus presentation per trial, underscores its utility in illustrating these principles without the need for complex processing, making it ideal for our model demonstrations.  
conclusions and discussions: 
 All learning agents, including the brain, face a fundamental challenge: balancing task performance with the cost of learning. Intuitively, if the agent does not need to account for the feature distribution in the external world, it can quickly adapt by reusing existing feature representations to complete tasks efficiently and at low cost. However, when the agent identifies meaningful statistical patterns in the environment, it can refine its feature representations to enhance precision and effectiveness. Although beneficial, this adaptation is both resource-intensive and slow, as the agent must first distinguish between genuine environmental changes and random fluctuations. In essence, to learn or not to learn, is a generic question faced by all learning agents.In this work, starting from the goal of reconciling the conflicting phenomena of specificity and transfer in perceptual learning, we present a solution of the brain, i.e., the dual-learning framework. This framework consists of two learning processes: a task-based one and a feature-based one. Specifically, task-based learning is fast, which enables the agent to learn to accomplish a task rapidly by using existing feature representations; while feature-based learning is slow, which enables the agent to improve feature representations to reflect the statistical change of the external environment.Our dual-learning model successfully replicates and elucidates classical experimental findings related to specificity and transfer in perceptual learning. It reveals that the interaction between the slowchanging, specific feature learning at the early visual pathway and the flexible, transferable task learning at the higher visual pathway governs the adaptability of perceptual learning. Typically, learning tends to adjust the readout of neural representations (i.e., task-based learning) rather than altering the representations themselves (i.e., feature-based learning), unless there is a significant change in the statistical properties of external information. Thus, the default state of learning favors transfer. However, in traditional experimental paradigms, the frequent repetition of stimuli leads to the dominance of feature-based learning, thereby exhibiting significant specificity; by limiting the repetition of stimuli, task-based learning can dominate, thereby enhancing transfer. As such, perceptual learning can display a transition from transfer to specificity as the number of stimulus repetitions increases.Our dual-learning model can be regarded as a computational modeling implementation of the twostage model theory [44] . While the two-stage model theory aims to address the contradictions between task-related and task-unrelated perceptual learning, it posits the existence of feature-based and task-based plasticity within perceptual learning. Thus, our model is a practical realization of this theory. Furthermore, the dual-learning framework aligns with the reverse hierarchy theory [45] , which suggests that learning processes invert the sequence of visual information processing from top-down. As tasks become more difficult and specific, lower levels of the neural hierarchy are increasingly engaged, aligning learning outcomes with enhanced task specificity. This alignment illustrates how the dual-learning model not only accommodates but also substantiates theoretical perspectives on perceptual learning's hierarchical nature, providing a robust framework for exploring how different learning mechanisms interact within the brain's architecture.Limitation and future work. In this study, as a first step to elucidate the notion of dual-learning, we have built a very simple network model without including many biological details of the visual pathway. In future work, we will extend the current model to include more biological details and apply the model to explain more cognitive functions of the brain. In essence, the dual-learning framework reflects that the brain employs a learning strategy to balance the accomplishment of a new task and the consumption of learning effort, that is, if the statistics of the environment are unchanged, low-cost task-based learning is applied; otherwise, high-cost feature-based learning is triggered. The same balance requirement is faced by other learning agents. We, therefore, expect that the dual-learning framework has the potential to be applied in AI applications, and we will explore this issue in future work.EQUATIONwhere x and θ define the preferred position and orientation of the basis function.We configured the network with 40 × 18 basis functions to uniformly cover all possible positions and angles. Specifically, the 40 basis functions span the positional space, evenly distributing across different position scales, while the 18 basis functions cover the angular space, with each basis function representing a 10-degree increment (i.e., 18 = 180 • /10 • ). It is important to note that while x traditionally represents a two-dimensional position within the image, we simplified the model by setting the size of the basis function to the height of the image, thus reducing the twodimensional position variable to one dimension. This simplification does not affect the outcome of our experiments. As defined in Eqs. 2-3 in the main text, the refined feature representations F t (x, θ) at each trial t are computed based on the feedforward connection weights W t (x, x ′ ). Below are the mathematical details. 
experimental setting/details: 
 Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: The paper provides comprehensive details regarding the training and testing procedures.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in the Appendix, or as supplemental material. Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. Justification: The paper does not provide detailed information on the computer resources needed to reproduce the experiments. However, this omission is justified because the model used is very simple and does not impose significant computational demands. Therefore, specifying resources like compute workers, memory, or execution time was deemed unnecessary for the context of this study.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes]Justification: The paper discusses potential societal impacts by highlighting the benefits of learning, which can contribute to advancements in fields like medical imaging and autonomous vehicles. It also addresses possible negative impacts, such as the specificity of learning. This balanced discussion demonstrates the authors' awareness of the broader. Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA] Justification: The paper poses no such risks. Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected? Answer: [Yes] Justification: The authors properly credit the creators of the code and data used in the paper by citing the original source. Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [Yes]Justification: The documentation of new assets introduced in the paper will be assessed once the supplementary material, including code and data, is made available. This will ensure that all necessary details and instructions are provided to facilitate the reproduction and understanding of the introduced assets.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: The paper does not involve crowdsourcing nor research with human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained? 
 
 Appendix: The refined feature representations are updated using a convolution process that integrates feedforward connection weights with previous feature representations:F t (x, θ) = norm x ′ W t (x, x ′ ) * F * t (x ′ , θ) . (A3)The feedforward connection weight W t (x, x ′ ) from position x ′ in the previous layer to position x is defined as:EQUATION)The code is available on Github at https://github.com/XiaoLiu-git/ToLearnOrNotToLearn.Here, A t+1 (x ′ ) represents the magnitude of the connection, while a controls the spread of the connection. For simplification, we consider a → 0, the connection weight simplifies to:lim a→0 W t+1 (x, x ′ ) = A t+1 (x ′ ) if x = x ′ , 0 otherwise. (A5)This indicates that, in the limit, the connections become fully localized, creating direct, one-to-one mappings between corresponding positions.The normalization operation norm(•) and the convolution operation * are consistent with previous descriptions.The strength of feedforward connections, A t (x), is updated following competitive Hebbian learning rules [46] , aimed at reinforcing location-specific changes in the neuronal representations.First, competitive learning identifies strong responses from prior feature representations for weight updates:EQUATIONEQUATIONwhere F 0 and F * 0 are thresholds set to 0.6, effectively selecting the top 20% of neurons based on response strength.Then, Hebbian learning computes and updates the connection strength A t (x):EQUATIONEQUATIONwhere ∆A t (x)represents the updated value for weights, η f is the learning rate, and mean θ (•) calculates the average over different orientations, considering the columnar structure of orientation preference in the primary visual cortex. As defined in Eq. 4 of the main text, the decision z * t at each trial t is computed using a convolutional network followed by global max pooling. The decision z * t takes a value of 0 or 1, representing whether the upper orientation is more leftward (0) or more rightward (1), respectively. During decision-making, the output is classified as 1 if the value exceeds 0.5; otherwise, it is classified as 0.The CNN used in our model is a streamlined convolutional neural network comprising three convolutional layers.• Layer 1: Inputs a single channel and uses a 3x3 convolutional kernel to output 6 channels.• Layer 2: Processes the 6 channels with another 3x3 kernel, producing 10 channels.• Layer 3: Compresses the 10 channels into a single output channel using a 3x3 convolution.The network employs ReLU activation after the first two convolutional layers to introduce nonlinearity, followed by flattening and a 1D max pooling on the final output to ensure a well-structured output.1. Task-Based Learning Only: Feature-based learning is disabled, isolating the effect of task-based learning to on model performance across different locations.2. Feature-Based Learning Only: Task-based learning is disabled, and only feature-based learning operates. Initialize the weights as the weight completing task-based learning.During training, the similarity between the feature representations F * t (x, θ) and the refined representations F t (x, θ) under different stimulus locations (1 trained location and 3 untrained locations) and orientations (0 • , 45 • , 90 • ) is measured. The similarity is calculated as:EQUATION)where n denotes the total number of representations. In this study, Vernier discrimination stimuli were generated as described previously. The stimuli were available in two sizes: 400 × 200 pixels (used for the first, third, and fourth experiments; Fig. 4A , Fig. 6A and Fig. 7A ) and 800 × 200 pixels (used for the second experiment; Fig. 5A ).Each Vernier stimulus consists of two Gabor patches with orthogonal orientations: ori1 (θ = 0 • ; horizontal orientation) and ori2 (θ = 90 • ; vertical orientation), as described in Eq. A11.In the first, third, and fourth experiments, the Vernier stimulus was centered at two locations: loc1 (100 , 100) and loc2 (300, 100). In the second experiment, two additional locations were added: loc3 (500, 100) and loc4 (700, 100).The offset O between the two Gabor patches varied across 10 difficulty levels, defined as:EQUATIONwhere D ∈ [1, 10] is the difficulty level, N d = [n d ] is the rounded difficulty noise, and:n d ∼ N µ, σ 2 d , σ d = 0.5. (A18)The probability density function ψ (n d ) for the truncated normal distribution was as follows:ψ (n d ) =      0, n d ≤ a ϕ(µ,σ 2 d ;n d ) Φ(µ,σ 2 d ;b)-Φ(µ,σ 2 d ;a) , a < n d < b 0, b ≤ n d (A19)where ϕ and Φ were the probability density and cumulative distribution functions, respectively. The values a = -1 and b = 1 are the truncation points.The background noise N is generated as:N = a noise • n, n ∼ N µ n , σ 2 n (A20)where a noise was the noise intensity, n was the raw noise, and N followed a normal distribution with mean µ n = 1 and variance σ 2 n = 2. The stimulus images generated in each trial were mutually independent.All the results in this study were obtained by independently training 100 models with the same set of parameters under different experimental conditions (e.g., variations in random initialization and training stimulus noise).To compare with human perception experiments, we measured the difficulty thresholds for each model at every test point. A threshold represents the intensity at which a difference between two stimuli is just detectable, with stimuli below it considered subliminal. Here, we used the constant stimuli method to measure the threshold at which model accuracy reached 80%, comparable to the 79% threshold in human subject experiments [28] [26] [30][34] . The constant stimuli method is a standard psychophysical approach for measuring perceptual thresholds [47] , involving the presentation of stimuli at several constant levels and fitting a psychometric function to the responses.At each test point, we evaluated model accuracy across 10 difficulty levels. Each level included 20 positive and 20 negative samples, totaling 40 trials. The model's accuracy at each level was used to fit a psychometric function based on the cumulative normal distribution:EQUATION)where A i denotes the accuracy at difficulty level D i , µ f it and σ f it are the mean and standard deviation of the fitted distribution, and the ϵ i is the error between the observed and fitted accuracy. And C denoted the chance level accuracy, which was 0.5 for this binary forced-choice task.The goal of the fitting process is to minimize the total error:EQUATIONwhere n point = 10. Once fitted, the threshold difficulty D threshold corresponding to an 80% accuracy level calculated as:EQUATION)The fitting process was conducted using the scipy.optimize.curve_fit function from the Python scipy library, with upper bounds for µ f it and σ f it set to 30 and 13, respectively, based on initial pre-training results. The cumulative distribution function and its inverse were computed using scipy.stats.norm.cdf and scipy.stats.norm.ppf.In addition to directly presenting the threshold values, we also calculated the Percent Improvement (PI) for each experiment to quantify the effect of learning. The PI for model m, denoted as I m was defined as:I m = -(threshold af term -threshold bef orem ) threshold bef orem (A24)where threshold bef orem and threshold af term were the threshold measured before and after learning, respectively. Pre-and Post-Testing We performed pre-and post-testing on all training and transfer conditions, before the formal training (after pre-training) and after the formal training, respectively. We measured the 80% threshold of all the training and transfer conditions and calculated the improvement using the method mentioned in Sec. C.2.• Training Conditions: ori1_loc2, ori1_loc3, ori1_loc4, ori2_loc1, ori2_loc2, ori2_loc3, ori2_loc4• Testing Condition: ori1_loc1• Stimulus Noise Intensity: a = 0.6• Learning Rate: η f = 0.3 (feature-based learning), η t = 0.001 (task-based learning)Pre-training Same as Sec. C.3.1.Formal Training For the random training condition, the trials were shuffled, while for the rotation condition, trials were presented sequentially in dictionary order. Other procedures were the same as Sec. C.3.1.Pre-and Post-Testing Testing followed the same protocol as described in Sec. C.3.1.• Training Condition: ori1_loc1• Transfer Condition: ori2_loc2• Stimulus Noise Intensity: a = 1.2• Learning Rate: Pre-and Post-Testing Testing followed the same protocol as in Sec. C.3.1.η f = 3 (feature-In Fig. 6B , we argue that at the final transfer session, the thresholds are from low to high as the training sessions increase. The results of the t-tests conducted on the data are presented below in Table A4 , which contains the p-values of T-tests for points at the final transfer session in Fig. 6B . As observed, the statistical differences between any two conditions of the experiment are significant. This finding aligns with the results depicted in Fig. 1C (ii), indicating the consistency across different parts of the study. The model was trained for 10 sessions under both the initial and double training conditions. Other details were similar to Sec. C.3.1.Pre-and Post-Testing Pre-and post-testing followed the same protocol as in Sec. C.3.1.Outlier Removal Some results shown in Fig. 7B and Fig. 7C appear inconsistent, which may cause confusion (e.g. the threshold of ori1_loc2 at session 10 in the training condition is lower than that at session 0 in Fig. 7B , indicating an improvement in perception. However, in Fig. 7C , the improvement of ori1_loc2 after the first step training is around 0, suggesting no learning). This inconsistency arises due to that Fig. 7B presents the average of 100 simulation runs, while the improvement in Fig. 7C was calculated by averaging the rate of improvement from each simulation, where a few outliers affected the results. After excluding 4 outliers (with z-scores of improvements greater than 4), the results are consistent (see Fig. A1 ). We conducted two additional ablation studies to highlight the importance of the relative speeds of feature-based and task-based learning. The results, based on 100 simulations, confirm that both slow feature-based learning and fast task-based learning are essential for replicating perceptual learning phenomena in our model.As shown in Fig. A2 , we increased the learning rate of feature-based learning by tenfold and replicated the four experiments from Sec. 2 in the main text. The detailed results are as follows: First Experiment Accelerated feature-based learning led to significant modifications in feature representations, resulting in degraded performance at untrained locations (especially for ori1_loc2).Second Experiment Due to continuous changes in experimental conditions and accelerated learning, the model failed to stabilize representations. As a result, performance in the task was compromised, and human-like transfer was not achieved.Third Experiment With increased training sessions, differences between conditions diminished, and learning curves became entangled (especially between T8 and T12). The statistical results, shown in Table A5 , indicate that the p-values increased compared to Table A4 , reflecting reduced significance between conditions. As depicted in Fig. A3 , we decreased the learning rate of task-based learning by tenfold. This adjustment significantly degraded learning effectiveness, as the model struggled to master tasks efficiently.First Experiment Although the model could not fully master the task, a comparison with Fig. A2 shows no significant difference in transfer effects between training and transfer conditions. Second Experiment Despite the model's poor mastery of tasks, the improvements gained during learning transferred well to new conditions.Third Experiment The slower learning disrupted performance consistency across conditions, leading to entangled learning curves. Statistical analysis, shown in Table A6 , reveals increased p-values compared to Table A4 , indicating smaller differences between conditions. NeurIPS Paper ChecklistJustification: The research adheres to the NeurIPS Code of Ethics by ensuring transparency, reproducibility, and fairness in reporting experimental results.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 


Decision: In this study, the authors proposed a dual-learning framework in which the interplay between the rapid, supervised, task-based learning and the slow, unsupervised, feature-based learning to explain a wide range of phenomena in perceptual learning. Although the reviewers had several concerns about the motivation/model design/Hebbian learning mechanisms, the rebuttal successfully convinced the reviewers. I am inclined to accept the paper.

Reviews: review: paper_summary: paper_summary: The paper puts forth a theoretical framework for perceptual learning, in which two separate learning processes contribute to learning a perceptual task (a fast, flexible task-based learning that relies on existing feature representations; and a slow, task-specific feature learning). Repeated learning sessions with the same stimulus conditions triggers feature-based learning, which will be specific to the training stimulus conditions. The framework is instantiated as a neural network model, and is used to account to both specific and transfer phenomena in psychophysical experiments.
strength_weakness: strength_weakness: One important limitation of this work is that the networks appear to be trained from scratch on the “probe task” (e.g., vernier acuity). In contrast, with human perception, the person’s visual system is tuned over their lifetime (presumably via some self-supervised objective), and then they are put into an experimental setting where 1-2 hours per day for N days the perform this new task. So the impact in terms of perceptual experience is set within this very broad context of “learning to see”, and a visual system that’s capable of much more than just the task at hand (e.g., vernier acuity). I suppose this begs the question of whether this all works as well if the network is initially pretrained (say on some self-supervised learning objective), and then the same types of experiments are run. I don’t see in principle why these same ideas wouldn’t apply.

Minor: When the computational framework is first introduced, the different architectural choices are unmotivated (a classical basis function network for image processing — What is this and why this? A Hebbian network for feature learning – Why a Hebbian network? Why is feature learning a separate step from the basis function learning? And then a CNN with max pooling for task-based learning. Why a CNN for this purpose, and a Hebbian network for the feature learning?). I realize these choices are all motivated, they just come out of the blue all at once, and it’s hard for the reader to grok the motivation for this setup.

Minor: The methods introduce the Feature Extraction as akin to the retina or LGN, but what’s described appears to be an orientation-tuned Gabor Wavelet Basis Set (and orientation-tuning is believe to arise later in V1 and beyond).

Minor: Can you provide more information on the architecture of the task-based CNN?

Minor: Can the authors provide further justification/clarification for the different learning algorithms implemented at each level of the network? e.g., Why shouldn’t Hebbian learning play a role “all the way to the top” of the task network? Is this biologically motivated, or a convenient way to implement assumptions of the model (e.g., location-specific learning)?

Minor: Other related work includes “How variability shapes learning and generalization” (Raviv et al, Trends in Cognitive Sciences)
questions: questions: Can the authors clarify what makes the task-based learning “fast”? Is this accomplished via learning rate hyperparameter tuning?
limitations: limitations: OK
paper_summary: paper_summary: 1. The paper proposes a dual-learning model to reconcile two seemingly contradictory phenomena in perceptual learning: specificity and transfer.
2. The model consists of two learning processes:
   - Task-based learning: Fast, enables quick adaptation to new tasks using existing neural representations.
   - Feature-based learning: Slow, refines neural representations to reflect changes in the environment.
3. The model is implemented as a hierarchical neural network with three stages:
   - Feature extraction
   - Feature-based learning
   - Task-based learning
4. The interactions between these two learning processes explain the observed specificity and transfer effects in perceptual learning experiments:
   - Specificity occurs when feature-based learning dominates (due to excessive training on the same stimulus).
   - Transfer occurs when task-based learning dominates (due to varied training conditions).
5. The model successfully reproduces key experimental findings in perceptual learning, including:
   - Specificity in condition-specific training
   - Transfer in varied training conditions
   - Transition from transfer to specificity with increased training sessions
   - Transfer effects in double training paradigms
strength_weakness: strength_weakness: I am not a computational neuroscientist and only follow this field very sparsely. I found the paper interesting but can not judge the novelty of the paper and its methodology and results. Nevertheless, I would argue that the experimental section is missing ablations or theoretical results to explain the (many) moving bits and pieces , and therefore authors design choices, of the proposed model.
questions: questions: Please ablate the model. 
1) Is the first hand designed feature extractor necessary? What happens you mess this part up?
2) What happens if you train  the middle part with backprop?
3) What happens if you train always the first and the last part with backprop simultaneously. 

Generally I think the proposed model seems quite ad how and potentially makes intuitive sense. Please back up design choices with experimental results. Show me when model behaviors start misalignment with the expected findings of perceptual learning.
limitations: limitations: Seems ok to me.
paper_summary: paper_summary: In this article, the authors propose a novel model that accounts for two different phenomena observed in human learning: i) specificity, a feature-based mechanism restricted to the very specific statistics of the environment condition, and ii) transfer, a task-based mechanism that allows to transfer knowledge to untrained locations or features. The model proposed by the author is made of a stack of 3 components: A feature extraction stage (mimicking LGN), a Feature-based learning stage (mimicking early processing), and a task-based learning (mimicking upper-processing). Training this model on a vernier discrimination task, the authors demonstrated that it accounts for numerous psychophysics phenomena: i) specificity in condition-specific training, ii) transfer in various training conditions, and ii) transition from transfer to specificity when trained on an increased number of training sessions.
strength_weakness: strength_weakness: I have noted various (rather minor) points that needed to be better clarified/explained (see question section). My only concern is about the generalization of the proposed model to more complex tasks (including for example natural image discrimination). I fully agree that simple tasks like the Vernier discrimination are a very good starting point to propose a novel model accounting for human learning, but scaling such models to more complex data would have drastically improved the impact of this article. Overall I don’t think this scaling-up argument is enough to reject this article, but this is something the authors need to keep in mind if they want their article to reach a more ‘global’ audience. Note that I am willing to increase my rating if the points (in the question section) are properly addressed/clarified.
questions: questions: 1 - In my understanding, feature-based is supposed to account for selectivity, which is thought to happen when over-training is taking place (i.e. when stimuli are relatively similar, and there that the environment is relatively unchanged). But it seems to contradict this sentence (l75-77): ‘Feature-based learning … only takes place when the external environment is substantially changed ». On my understanding if the environment is « substantially changed » then we are not in the over-training condition... Am I missing something? Could you explain?

2 - In Figure 2A, this is not clear what loc(x) corresponds to ? Is it encoded as the x,y coordinate of the stimulus? Please give more detail.

3 - In Appendix 1 (Eq.A4), you suggest a=0, which involves two divisions by zero (one in the exponential multiplier, and another inside the exponential). Is that a typo or is that a=0? If this is the case how do you handle the division by zero? 

4 - What is the grey line in Figure 3?

5 - In Fig 3.C. How do you explain that the accuracy for the training sample is decreasing with only feature-based learning? I would expect to see this accuracy improving if the feature-based learning model well the specificity phenomena as we observe an increase in performance for human data (Fig 1Aii ).

6 - I am not sure the claim line 173 is true: « feature based learning… reinforces the model’s performance at the trained location ». This is the opposite trend that is observed in Fig 3C… Could you discuss this point? 

7 - In Fig 3D: There is no mention of the method used to compute the similarity in Fig 3D. Could you elaborate? 

8 - Line 182: «  The combination of feature-based and task-based learning accelerates the training process at the train location ». This is not visually obvious when we compare Fig 3B and 3E… Could you quantify the difference in convergence speed? Ideally, make sure this difference is statistically significant.

9 - Could you explain a bit more about the difficulty threshold in section 4? And what is the difference between the discrimination threshold and the difficulty threshold? Is that the same? If I understood well the difficulty threshold is the offset between the two Gabor filters (the smaller the offset the harder the task). But in line 191, you introduce the discrimination threshold, which seems to be more related the the intensity of the stimuli than the offset. Could you clarify this point? 

10 - Figure 6: Could run a statistical test to make sure the T4 is indeed below T8 (in Fig6b, in the transfer setting? 

11 - Not sure what you mean in line 218: In the untrained condition, the threshold first decreases and then increases or saturates… I am not sure to see that in Fig 6B… could please be more accurate here 

### Typo : 
* Fig2C : learining —> learning
* Line 144: Sec. A.1 —> Sec A.2
limitations: limitations: The author properly discusses the limitations of their paper.
paper_summary: paper_summary: This paper aims to replicate a variety of results from the perceptual learning literature using a model with two different forms of learning. It shows results that capture how specificity occurs under certain training conditions and transfer occurs under others. The two forms of learning are different in terms of supervision and location of weight changes.
strength_weakness: strength_weakness: The modeling choices made in this work made it hard to interpret the results and also made the connection to known biology more difficult.  Also there are some issues in the interpretation of the results the authors provide at times.

Specifically:

In the feature-based learning method, the authors tie the weights from different orientations at the same location (this is justified by reference to cortical columns). I don't understand the mechanism that would cause all neurons representing a single location to have the same weight updates. Rather, I would expect there to be orientation-specific learning as orientation is represented differently by different cells. 

Different task-based networks are trained for different orientations. The authors do not make this very clear in the main text, but it is of course very important for understanding the results. What is this meant to correspond to biologically? (of course the rest of the ventral stream doesn't change based on which orientation is being used.) The authors also refer to this form of learning as fast, but it includes the learning of many more parameters than the feature-based learning and takes many epochs. 

The task-based network also performs convolutions over a space where dimensions represent orientation and space. This means that a downstream cell only gets input from 3 nearby spatial locations and 3 nearby orientations (in the first layer). The spatial specificity is warranted as cells do have spatially restricted receptive fields. But what is the justification for the restricted orientation connections? Furthermore, the fact that the weights are convolutional in this space means that there may actually be orientation transfer in these networks (for the same reason there is location transfer when the task network is trained alone). However this was not tested because different task networks were used for different orientations. The location transfer exhibited by the task-based network also depends on weights being tied (i.e. using a convolution). What is the biological explanation for this feature of the model?

The Hebbian learning (combined with normalization) seems to really mess up the representations at non-trained locations. This is more extreme than simply not transferring to them. How is this interpreted? 

To help with interpretation, it would be good if the ablation studies could be run on all the experimental results.
questions: questions: Is Eqn 2 actually a convolution? It seems like each spatial location has its own separate weight that the activity is multiplied by so I don't see what about it is convolutional. (Also it isn't necessary to include Eqn A4 if the only scenario used is $\alpha=0$). 
What values are $A_{t=0}$ initialized as?

The main text makes reference to the task-based network as being a convolutional layer, but it seems like it is actually 3. Also, are there nonlinearities between this layers?

For the third experiment the authors say "At the untrained condition, the threshold first decreases and then increases or saturates, indicating a transition from transfer to specificity". I'm not sure where the increase or saturation is seen. The data is a little noisy but still looks like a continued decrease on average.

I am confused about how the locations of the gabor filters are spread out across the image. Are they overlapping or no? if I just take the width and divide by 40 (as suggested by line 406), the filters would be 20 pixels wide but the standard deviation is 30 so that can't be right. Is there a stride that's missing? 

In fig 7, isn't the relative lack of transfer to ori1_loc2 inconsistent with the experimental results? Also 7b makes it look like there is improvement for ori1_loc2 but 7c doesn't. Also it seems the labels are wrong on the right hand side of 7c. 

It is possible the authors broke anonymity on line 105 (no need to confirm or deny, just a reminder to be careful)

There are typos in line 145 and 148.
limitations: limitations: The authors should probably speak more to the limitations due to the non-biological components of the model discussed here.

Comments: title: response to author
Thank you!
comment: Thank you for your positive remarks. We will incorporate your suggestions in the paper. Your suggestions have greatly improved our paper, please let us know if you have other questions or concerns.
Thanks for the response. Under the assumption that the authors will make the stated changes to better explain the model mechanisms, I have increased my score.
Thank you for the instructive comments.

- Regarding question 1: We agree that the detailed neural mechanisms for perceptual learning in the brain remain unclear. The goal of this study is not to assert that the nervous system actually employs the models/learning algorithms we used in this work, but rather to propose a plausible explanatory framework for perceptual learning. As suggested by the reviewer, in the revised manuscript, we will discuss in much more detail about the biological plausibility of the models/learning algorithms to clarify the underlying assumptions.

- Regarding questions 2 & 3: Thanks for the suggestion. We will further clarify the settings of the models in the paper to avoid potential misunderstandings. Our previous statement that "a single task-based learning module handling different directions [...] would not qualitatively change our results" is based on the current structure and settings of our model.

- Regarding question 4: Thanks for the comments. We should have made our points more clearly. In the paper [24], the authors found that with more training in the first task, the initial threshold in the new task became higher, and so did the final achieved threshold (thus the learning performance was decreased compared to the less trained case). This indicates that extensive training in the first task can alternate neural representation, increasing the difficulty of learning the new task (as shown in our Fig1C; and Fig2 in [24], comparing the training of two and twelve blocks, T2 black and T12 green curves).
I thank the authors for their response.
With respect to the answer to Q1: The behavioral studies that show location transfer is possible do not directly support this specific mechanism of location transfer. That would require evidence on the neurophysiological level. I believe the second point is trying to say that changes in top-down modulatory influence might be responsible for aiding location transfer (but the mechanism is vague and not implemented here). In any case, the authors should acknowledge this modeling gap more directly in the main text so as to not mislead the reader about the mechanisms of the model. Furthermore, it will help explain some trends in results such as the slight transfer to orientation 2 at location 1 seen in Figure 4B.      

WRT Q 2&3: The authors say "Since our study focuses on location specificity", but the study is presenting results about orientation transfer as well.  So it does seem important to reflect on how the choice of readout model/task-based learning influences that process. To even test orientation transfer requires several epochs of training of a different readout model; the details of this model/training process could influence the results. It is also said that "a single task-based learning module to handle different orientations [..] will not change our results qualitatively." My point is that it could, depending on how it is built. That is why I'm advocating for more transparency in the main text about how the model is built, how transfer is tested, and how these design choices might impact or explain the results. I think a reader would benefit from the author's careful reflection on these issues. 

Q4: I looked at reference 24 and I don't see much evidence of extended training causing a decrease in performance at the transfer location. There were maybe 1 or 2 individual subjects where the average discrimination threshold on the first transfer session was slightly higher than the threshold on the first training session, but I'm not clear if these were even statistically significant differences. The more common trend however was a slight decrease in threshold on the first transfer sessions (i.e. partial transfer, even for extended training).
Thank you for the improved score. We are grateful for the valuable suggestions you provided, which have enhanced the clarity of our work. We will incorporate these insights into our revised manuscript.
Thank you for the score bump. We are pleased that the ablation study you suggested has enhanced the clarity of our work, and we will include these changes in our revised manuscript.
I am convinced by the author's rebuttal. I increase my rating to 7
Thank you for the additional clarification and data. I will raise my score to 5, but highlight my lack of knowledge wrt the biological plausibility and relevance of the algorithm for neuroscienctists - so I can not judge the contribution of the paper fairly.