To Learn or Not to Learn, That is the Question — A Feature-Task Dual Learning Model of Perceptual Learning
Authors: Xiao Liu
Muyang Lyu
Cong Yu
Si Wu
Venue: NeurIPS 2024 poster (2024)
Keywords: perceptual learning
specificity
transfer
plasticity
Primary area: neuroscience_and_cognitive_science

Abstract: Perceptual learning refers to the practices through which participants learn to improve their performance in perceiving sensory stimuli. Two seemingly conflicting phenomena of specificity and transfer have been widely observed in perceptual learning. 

Here, we propose a dual-learning model to reconcile these two phenomena. The model consists of two learning processes. One is task-based learning, which is fast and enables the brain to adapt to a task rapidly by using existing feature representations. The other is feature-based learning, which is slow and enables the brain to improve feature representations to match the statistical change of the environment. Associated with different training paradigms, the interactions between these two learning processes induce the rich phenomena of perceptual learning. Specifically, in the training paradigm where the same stimulus condition is presented excessively, feature-based learning is triggered, which incurs specificity, while in the paradigm where the stimulus condition varies during the training, task-based learning dominates to induce the transfer effect. As the number of training sessions under the same stimulus condition increases, a transition from transfer to specificity occurs. 

We demonstrate that the dual-learning model can account for both the specificity and transfer phenomena observed in classical psychophysical experiments. We hope that this study gives us insight into understanding how the brain balances the accomplishment of a new task and the consumption of learning effort.

To Learn or Not to Learn, That is the Question - -A Tasture-Task Dual Learning Model of Perceptual Learning
Xiao Liu! Muyang Lyu Cong Yu²", and Si Wu
PPeking-Tsinghin Center for Life Sciences, Academy for Advanced Interdisciplinary Studies,
Beijing Key Laboratory of Behavior and Mental Health,
IDG/McGovemn Institute for Brain Research, Center of Quantitative Biology, Peking University
School of Psychological and Cognitive Sciences, Peking University (xiaoliu23, yucong, (onu)pkku.eeee edu. on 1yumuyangaatu. pku. edu.
Abstract
Perceptual learning refers the practices through which participants learn to im- prove their performance in perceiving sensory stimuli. Two seemingly conflicting phenomena specificity and transfer have been widely observed in perceptual learning. Here, we propose dual-learning model to reconcile these two phenom- ena. The model consists of two learning processes. One is task-based learning, which is fast and enables the brain to adapt task rapidly by using existing feature representations. The other is feature-based learning, which is slow and enables the brain to improve feature representations to match the statistical change of the environment. Associated with different training paradigms, the interactions between these two learning processes induce the rich phenomena of perceptual learning. Specifically, in the training paradigm where the same stimulus condition is presented excessively, feature-based learning is triggered, which incurs speci- ficity, while in the paradigm where the stimulus condition varies during the training, task-based learning dominates induce the transfer effect. As the number of train- ing sessions under the same stimulus condition increases, a transition from transfer specificity occurs. We demonstrate that the dual-learnigg model can account for both the specificity and transfer phenomena observed in classical psychophysical experiments. We hope that this study gives us insight into understanding how the brain balances the accomplishment of new task and the consumption of learning effort.
Introduction
Perceptual learning refers to the practices through which human participants learn to improve their performances of perceiving sensory stimuli 11. Perceptual learning has been widely used as research paradigm to ascertain the learning strategies in the brain. Over the years, two seemingly contradictory phenomena have been widely found in perceptual learning. One is specificity, referring to that after learning, the improved performance of participants is restricted to the trained location or the trained feature of the stimulus 2 3. The other is transfer, referring to that after learning, the improved performance is transferable to untrained locations untrained features of the stimulus [4]
'These authors contributed equally to this work and should be considered co-corresponding authors.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

5
In early experiments in which participants were typically trained excessively under the same stimulus condition (e.g.. either at the same retinal location or presenting the same stimulus feature), specificity was predominantly found, and stimulus features used in experiments include contrast [7 X motion direction [15 16 17], and retinal 19 20, 14 Thus, once in while, specificity was regarded as a hallmark distinguishing
orientation [0 11 12, spatial frequency [3 12
location [88
perceptual learning from other learning types [3]. Later on, enriched training paradigms were used, and it was found that perceptual learning can also exhibit the dominating transfer effect. For instances, by manipulating the task difficulty [2
23 24 the training intensity [25] 26, 27,, or the training the originally observed specificity specifiity--omminting effect
protocol 28 29 30 31
32
33
34
35
36
become cransfr-dommnattee
Related works. A large volume of computational models has been proposed in the literature to unveil the mechanism of perceptual learning. Since specificity was predominantly reported in early experiments, early computational models mainly focused on exploring the neural mechanism under- lying specificity. For instance, Poggio et al. proposed the hyper basis function model (HyperBF) [37. which considers that the enhanced perception performance comes from the changes of basis func- tions or connection weights of neurons; Teich and Qian proposed model which considers that the enhanced performance comes from the changes of tuning curves low-level neurons [38]; Petrov et al. proposed the reweighting model, which considers that the enhanced performance comes from the read-out weights from feature representations to decision neurons [39] [0]. All these models implicitly assume that perceptual learning only incurs local changes in the network, and hence, the learning effect is localized and not transfcrable to other parts of the neural system needed for pro- cessing untrained conditions. Later on, motivated by the observation of transfer, new computational models focusing on the transfer mechanism were proposed. For examples, Dosher et al. refined their reweighting model by including extra layer with network-wide connections to implement -nnvariinnnnaaiiantt translation- invariant feature representation, which achieves a certain level of transfer effect [41]; Solgi et al. introduced an extra off-line learning procedure which generalizes the local learning result to untrained locations/feattres, making the learning performance transferable [42]: Li et al. employed convolutional neural network (CNN) to simulate perceptual learning [R]. utilizing that the CNN implements translation- -unslltion-invariantt feature extraction, and hence their model can explain some transfer phenomena. While these models may solve the transfer problem, they fail account for specificity, which was also vividly observed in experiments, Overall, perceptual learning still lacks a unified computational model accounting for both specificity and transfer that have been observed in different experiments.
In this study, we propose novel computational model to reconcile the seemingly conflicting phenomena of specificity and transfer in perceptual learning. Our model is based on the fact that in the experiments observing specificity, participants were often excessively trained under the same stimulus condition (either at the same retinal location or presenting the same stimulus feature); while in the experiments observing transfer, the stimulus condition was varied during the training (either the location or the stimulus feature). This indicates that excessive training of the same stimulus condition is critical to induce specificity. To account for both specificity and transfer arising from different training paradigms, we propose dual-learning framework for perceptual learning in the brain. The framework consists of two intertwined learning processes: one is task-based and the other is feature- based. Specifically, task-based learning aims to quickly adapt to new task by utilizing existing neural representations of the external world. It enables the brain to master new skills quickly and is transferable to untrained conditions. On the other hand, bease learning aims to refine neuronal representations to reflect the statistical changes of the external world. enables the brain to represent the external world faithfully and it only takes effect when the external environment is substantially changed. We argue that in the experiments exhibiting transfer, task-based learning dominates; while in the experiments exhibiting specificity, feature- learning dominates, and the latter is triggered by the neural system being excessively exposed to the same stimulus condition, triggering feature-based learning to catch up the statistical change of the external environment. Since deatured-based learning leads to localized changes in feature representations adhered to the trained condition, its effect is not transferable to untrained conditions.
To substantialize the dual-learning framework, we built a hierarchical neural network model, which consists of three sequential stages of information processing responsible for, respectively, feature extraction, feature-based learning, and task-based learning. Specifically, we use a classical basis function network (37] to implement feature extraction representing preliminary image processing,
2

---

feedforward network with unsupervised Hebbian learning to implement canture--ased learning, and a CNN with global max pooling to represent task-based learning. We set the rate of feature-based learning to be much smaller than that of task-based learning, such that task-based learning will dominate initially in training, and feature-based learning will take effect after the same stimulus condition is presented excessively. We demonstrate that the dual-learning model can well explain the specificity and transfer phenomena observed in classical psychophysical experiments.
2 Specificity vs. transfer in perceptual learning
Various tasks, ranging from the discrimination of basic visual features such as Vernier or orientation to the recognition of visual images such motion or face, have been applied to study perceptual learning. These experiments unveiled three generic characteristics of perceptual learning, including specificity, transfer, and a transition from transfer to specificity when the number of training sessions under the same stimulus condition increases. They are reviewed below.
We use a Vernier discrimination task to introduce the specificity effect 288] In this task, participants learned to discern which of two vertical orientations was more leftward (or equivalently rightward) (Fig. (i)). The stimuli were always presented at the same retinal location. Through intensive training. participants progressively improved their sensitivity to the offset between two orientations until reaching threshold (the blue dashed line in Fig. [iii).. After training, it was found that participants could not generalize their enhanced perceptual performances untrained retinal locations, displaying the effect of specificity (red squares in Fig. (ii).
(i)
B (i)
(i)
Training
Transfer
NW SE
NE oz SW
(ii)
(ii
(ii)
Veriier (ancmin)
8
Oricatation threshold (dog)
Mean contrast thresholds
Train 2 sessions Train sessions
Train 8 sessioms Train 12 sessions
7
20
the
0.
oril locl oril loc2
nandom Gabbor
9.
N 10
single Gabor
Training
Transfer
Scssion
Session
Scssion
Figure 1: Specificity vs. transfer in perceptual learning. A. An example of specificity, adapted from
(28]. (i) A Vernier discrimination task: discerning the offset between two vertical orientations. (ii) The threshold of detectable offset at the training retinal location decreased with training sessions (blue diamonds), in comparison to pre-/post-training thresholds at an untrained retinal location (red squares). B. An example of transfer, adapted from [34.. (i) An orientation discrimination task: Gabor stimuli varied across 47 location/orientation conditions were used for training. The red circle and arrow highlight the location/orientation not included in training but tested pre-/post-training. (ii) The orientation discrimination threshold with random Gabors over 47 stimulus conditions decreased with training sessions (green cireles) in comparison pre-/post-training thresholds
untrained
condition (red circles). C. Transition from transfer to specificity, adapted from (26). (i) An orientation discrimination task: retinal locations and orientations of visual stimuli used in training and transfer assessment. (ii) Left panel: the contrast sensitivity threshold decreased with training sessions (black, yellow. purple, and green lines denoting 4, 8, and 12 numbers of training sessions, respectively). Right panel: the transfer effect decreased with the number of training sessions.
We use orientation discrimination task to introduce the transfer effect (3) In this task, participants learned to distinguish which of two successively presented orientations was more clockwise (or equivalently more anticlockwise) (Fig. (3)). By training with varied combinations of retinal location and orientation, participants progressively improved their sensitivity to the offset between two orientations until reaching threshold (the green dashed line in Fig, [iii). After training, it
3

---

was found that participants could generalize their enhanced perceptual performances to untrained conditions, displaying the effect of transfer (red circles in Fig. [T(i)).
The above two experiments highlight the importance of the number of training sessions under the same stimulus condition in inducing the specificity the transfer effect. It is expected that with the increased number of training sessions under the same condition, the learning effect should go from transfer to specificity gradually. Indeed, this was confirmed experimentally in an orientation discrimination task [26]. In this task, participants underwent different numbers of training sessions under the same condition, followed by switch to untrained conditions measure the extent of transfer (Fig. (C)).. It was found that with the increased number of training sessions under the same condition, the perceptual performance of participants was improved (the left panel of Fig. [Cii)). whereas the transfer effect was decreased (the right panel of Fig. [[i)
3 The dual-learning model
To reconcile the phenomena of specificity and transfer in perceptual learning. we propose dual- learning model. As depicted in Fig. 2 the model consists of three sequential information processing stages, which are feature extraction, feature-basse learning, and task-based learning. We use the Vernier discrimination task as an example to introduce the model.
A Feature extraction
B Feature-based learning
C Task-based learning
ori Refined rep. F(X,0)
Input image I(x)
Feature rep. (x,00
Decision
Figure 2: Overview of the dual-learning model. The Vernier discrimination task is used as an example. A. Feature extraction. It involves using basis functions transform image [(x) into feature representations F. (x,0), 0), where x and denote the position and orientation features. B. Feature-based learning. It refines feature representations to F. (x, 0) reflect the statistical changes of external inputs. The feedforward connections are updated following the Hebbian learning rule, and they are strengthened locations where stimuli are presented excessively, inducing location-specific changes in feature representations. C. Task-based learning. Using convolutional layers and global max pooling, integrates the task-relevant information from feature representations (x, 0) make the decision 2
3.1 The model structure
Feature Extraction. This models the functions of the retina, the Lateral Geniculate Nucleus (LGN), and the input layer of V1 in the visual pathway, extracting preliminary features from input images to form retinotopic feature representations (see Fig, A There is plasticity at this stage, and we employ the HyperBF network [37] to model feature extraction. Denote (x) the input image presented at trial and F7 (xx 8) the extracted feature representation, which is expressed as:
F (x, 0) norm [G(x x',0) * 4xx)),
(I)
where x and 0 denote the position and orientation feature in the image, respectively. The variable represents the position of neighboring pixels, used in the convolution operation. The Gabor function G(x-x',0) 1/2exp [-(x xx'22(200 2 cos 22xx Acos(O) + vh. with og the standard

---

deviation of the Gaussian envelope, À the spatial frequency, and v! the phase offset (for details, see Sec. A.I in the Appendix). The symbol represents the convolution operation and the symbol norm(-) denotes normalization operation, i.e., norm = (F - / /Fstt with F and mean
std the mean and standard deviation of the representation variable F*.
Feature-Based Learning. This models the plasticity in the early visual cortex, which refines feature representations to capture the statistical change of the external environment (see Fig. (B.. For simplicity, we employ a feedforward network with unsupervised Hebbian learning to implement this process. Denote Fx(x, 0) the refined feature representations at trial 4, which given by
F((x, norm Wx(xxx)*FF3 Fx(x
where Wx(x,') x') represents the feedforward connection of the network (Fig. updated following the competitive Hebbian learning rule, which is expressed as
2B). (x(x, is
(x)) = (x x') + wAW, (x,x')
with 1) as the learning rate. The detail of (W, x') is presented in Sec. A.2 in the Appendix. This learning rule enhances neuronal representations at locations where the stimulus is frequently presented, thereby inducing cecation-specifit changes in feature representations.
Task-Based Learning. This models the information read-out process in the higher visual cortex (see Fig. (C). We use small convolutional network with three convolutional layers, followed by global max pooling, to implement this process. Denote 27 the decision at trial which takes a value of (0, 1) representing the upper orientation is more leftward or more rightward, respectively. During decision- mocision-making.. if the value is greater than 0.5, it is classified as 1; if it is less than 0.5, it is classified as O. Its value is calculated by,
2/
where ((-) represents the multi-layer convolution operation that integrates task-relater information from the feature representation (x, 8). The max(.) function indicates the max pooling operation, which realizes translation-invariant invariant computation and supports transferable learning. The sigmoid function, expressed sigmoid(>) = 1/(1 exp(#)) converts the output of the max pooling into probability, representing the final decision. The true decision at each trial is denoted as 24, taking only binary values 0 or 1. Task-based learning updates the parameters in (b-) by minimizing the cross entropy loss, Le = [z,ln(z;) + (1 - 2, 21)nn(1 - z7]], and backpropagation is used (for details, see Sec. A.3 in the Appendix).
3.2
The interplay between feature- and task-based learning
We used a Vernier discrimination task to analyze the properties of the dual-lcarning model. The model outputs 0 if the upper orientation is more leftward compared to the lower one and outputs 1 otherwise. We trained the model at a fixed location and then tested its performances at three untrained locations having increasing distances the trained one (Fig. 3)) For the details, see Sec. B in the Appendix. We conducted the below ablation studies.
First, we froze feature-based learning and only enabled task-based learning. As shown in Fig. 33 the discrimination accuracies both trained and untrained locations increase gradually with train epochs, indicating that task-based learning facilitates transfer over locations. This property comes from the fact that task-based learning employs a maxing pooling operation, which is translation-invart
Second, we froze task-based learning and only enabled feature-based learning. The weights of the readout layer were initialized after completing task-based learning, as described above. As shown in Fig. BC, We see that the discrimination accuracy at the trained location keeps a relatively high value, whereas the accuracies (50%). This demonstrates that feature- baaed learning induces specificity, reinforcing the model's performance at the trained location while diminishing its transfer untrained locations. To display the location-specife change in feature representation, we calculated neural representation changes different locations during the training. As shown in Fig. 3D, we see that as the training goes on, the neural representation at the trained location remains stable (high similarity), while the neural
untrained locations drop quickly a nearly chance level
5

---

Input image
B
C.
70
70
60
60
50
loc
loc
loc
0
40
80
120
160
200
40
50
120
160
200
Train
Test
Epochs
Epochs
0.9
Accuracy (%)
Similarity
0.8
0.7
on 45
oct 90"
120
180
Epochs
Epochs
Figure 3: Properties of the dual-learning model. A. A Vernier discrimination task. Upper: the training stimulus. Lower: testing stimuli at three untrained locations. Discrimination accuracy vs. training epochs with only task-based learning on. It displays transfer effects to all untrained locations. C. Discrimination accuracy vs. training epochs with only feature- -ased learning on, following the completion of training in panel B. The model performances untrained locations drop dramatically, display the effect of specificity. D. Similarity feature representations before and after feature-based learning. Solid lines represent vertically oriented (0") stimuli used in training, dashed lines represent diagonally oriented (45%) stimuli, and dotted lines represent horizontally oriented (90%) stimuli. E. Discrimination accuracy
training epochs with both feature-based and task-based learning In panels and E, the gray lines indicate the number epochs required to reach 90% accuracy, The results presented are averaged over 50 repetitions. For more details, see Sec. B in the Appendix.
representations at untrained locations change dramatically, This indicates that feature-based learning induces location-specific changes feature representations.
Finally, we analyzed the model performance with both feature- based and task-based learning enabled. In particular, set the rate of feature-based learning to be relatively much smaller than that of task-based learning. Although a direct numerical comparison between the two learning rates is not meaningful due to differences in their roles and magnitudes, the chosen configuration allows observe a clear distinction the dynamics of the two learning processes. As shown in Fig. 3 we observe that: 1). The combination of feature- based and task-based learning accelerates the training process at the train location, requiring fewer epochs to reach 90% accuracy compared to using task- based learning alone (gray line in Fig. 3 mean epochs: task-based learning 84.7, combined feature- and task-based learning = 81.3). This difference is statistically significant (t = 3.90, < 0.001), based on 50 repetitions for each condition. 2). The training promotes transfer untrained locations initially, but as time goes on, the transfer effect decreases, while the specificity effect increases. This shift reflects that due different learning rates, task-based learning dominates initially, which induces transfer, while feature-base learning gradually takes effect, which induces specificity.
Reproducing classical findings in perceptual learning
In this section, we used the dual-learning model to reproduce the classical findings in perceptual learning. We employed Vernier discrimination task as an example, as this paradigm has been widely used in psychophysical experiments. In these experiments, the discrimination threshold, defined as the intensity of stimuli that participants can accurately discriminate with about an 80% success rate, was used to measure the learning effect. In our simulations, we adjusted the offset between two vertical orientations in the task to create a series of difficulty levels and chose the offset value corresponding to 80% model correctness as the threshold (for details, see Sec. Cin the Appendix).

---

In the first experiment, we adopted the training paradigm similar to that in 128. Specifically, we trained the model with fixed orientation and location (oril loc1) and evaluated model performances across various combinations of orientation and location before and after the training (Fig. A The results are presented in Fig. 43 which show that the model exhibits significantly improved performance in the trained condition poril_lo11, blue diamonds), whereas this improvement is not transferable to untrained locations (oril (oril_loc2, light red squares), untrained orientations (ori2 (ori2_locl, locl, medium red triangles), combinations of untrained location and orientation (ori2_loc2, dark red circles). Fig. 4C further summarizes the learning improvements under different conditions, showing good agreement with the experimental data (28). The results demonstrate that model successfully replicates the classical specificity phenomenon observed in perceptual learning.
A
B
ozi
25
Training
80
Difficulty threshold
20
Improvement (%)
15
40
O oril Joel
Transfer
ocil loc2
10
or12 locl
20
loc2
Joe2
0
Learming
Transfer
Session
Figure 4: Specificity in perceptual learning via condition-speciin training. A Vernier discrimina- tion task similar to that in [28). The visual stimulus used for training (top, loc1 |_oril) and an example of stimuli with the different location and orientation for transfer evaluation (bottom, loc2 (oc2_ori2), B. Learning curves and thresholds for pre- and post-testing in different conditions. The threshold at the trained condition (oril (oril_locl, I, 1, blue diamonds) decreases significantly after training, while thresholds at dark
untrained conditions (oril_ka l_o_2, light red squares; ori2_loc] medium red triangles; red circles) do not exhibit significant decline. C. Statistical results of training and transfer improve- ments. They show substantial gain at the trained condition improvement at untrained conditions (oril loc2, light red; ori2 loc1, medium red; loc2, dark red).
blue bar) and negligible or no
In the second experiment, we adopted the training paradigm similar to that in 34]. Specifically, the model was trained with stimuli presented at varying locations, either a random or sequential order, and the transfer effect was evaluated at untrained location (Fig. [5A]. The results are presented Fig. BCC, showing that the model's learning effect is successfulTy transferred to the untrained location. Fig. 5D further summarizes the learning improvements when trained across multiple locations either randomly or sequentially. These results agree well with the experimental data [34), demonstrating that our model successfully replicates the classical transfer phenomenon in perceptual learning. In the third experiment, we adopted a training paradigm similar to that in 26 Specifically, the number of training sessions was varied, and the transfer effect was evaluated using new condition that combined an untrained location and orientation (Fig. (()). The results are shown in Fig. GB. In the trained condition (left panel), performance thresholds gradually decreased with the increasing number of training sessions. However, in the untrained condition (right panel), the trend reversed: the more sessions completed in the trained condition, the higher thresholds were observed in the untrained condition. Fig. 66C further summarizes the progression of learning improvements during the training and transfer phases, agrecing well with the experimental data [26]. Thus, our model successfully replicates the classical phenomenon of transition from transfer to specificity with the increasing number of training sessions.
In the fourth experiment, we adopted the training paradigm called double training similar to that in 28 In this experiment, following the first step training of the classical specificity task as in Fig. [ we introduced second step training with stimulus at new location and orientation. After double training, we evaluated the model's ' transfer performance under conditions untrained in either step. The results, shown in Fig. 73, reveal that after double training, the original specificity effect in the
7

---

A
B
Random
C
Rotating
D
Random
Rotating
Training
64
randoms Gaboe
64
80
Difficulty threshold
single Gaboe
Difficalty threshold
single Gaber
Improvement (%)
12
60
40
Transfer
20
Session
6
10
6 Session
10
Leaminng Transfer Leaningg Transfer
Figure 5: Transfer in perceptual learning via varied training conditions. A. A Vernier discrimination task similar to that in [34] During training, visual stimuli were presented at three distinct locations with two orientations: horizontal and vertical (top). For transfer evaluation, a single stimulus was presented at new, untrained location (bottom), highlighted with a red circle. B. Random training condition. The learning curve and thresholds for training across multiple conditions randomly. The threshold at the training conditions (dark blue circles) decreases significantly, and the transfer condition (red circles) shows a similar decline. C. Rotating training condition. The learning curve and thresholds for training with stimuli rotating. The threshold at the training conditions (light blue circles) decreases significantly, and the transfer condition (red circles) shows a similar decline. D. Summary of learning and transfer improvements in different conditions.
Training
30
Train sessions (12)
Tran sessions (T))
Difffeulty threshald
25
Trai sessioss (144)
Tran 12 sessions (112)
Imgruveementt (%)
40
30
Transfer
20
15
10
2
10
10
12
78
T12
Session
Transfer
Figure 6: Transition from transfer specificity with the increased number of training sessions. A. A Vernier discrimination task similar to that in [26]. Visual stimulus used for training (top, loel _or_) and stimuli for transfer evaluation (bottom, loc2_ori2). B. The learning curves under different training (left) and transfer (right) conditions. The thresholds are depicted with different colored curves representing different numbers of training sessions: gray sessions), light red (4 sessions), medium red (8 sessions), and dark red (12 sessions). C. Summary of transfer improvements with varied training sessions. Each bar represents the improvement in transfer performance following 2, 8, or 12 training sessions, respectively.
first step training now becomes transferable. The underlying reason is intuitively understandable. The second step training modified the feature representations adhered to the first step training, and hence reduced specificity (or equivalently increased transfer). Fig. 7 further summarizes the learning improvements across different training steps and conditions, demonstrating that our model successfully replicates the double training phenomenon in perceptual learning.
In summary, the adaptability of perceptual learning governed by the interaction between specific feature-based learning at lower levels and transferable task-based learning at higher levels. Condition- specific training leads the dominance In contrast, training with varied conditions allows task-based learning to dominate, enhancing transfer. This difference enables transition from transfer to specificity as stimulus repetitions increase. Although not exclusively from the Vernier discrimination task, the observed changes in specificity and transfer are influenced by training paradigms rather than tasks themselves. The simplicity of the Vernier discrimination task, requiring only single stimulus presentation per trial, underscores its utility in illustrating these principles without the need for complex processing, making ideal for our model demonstrations.
Teature-based learning, resulting in significant specificity,

---

A
13
oril
Training
_ii_leel
eri2_ [ne__ocl
locl
25
oril Joc2
oei2_locd
loch.
Difficulty threshold
Improvement (%)
20
40
Joc1
Double training
10
20
oril
loc2
on12
Learning
Transfer
Double training
Transfer
Session
Figure 7: Transfer in perceptual learning via double training. A. A Vernier discrimination task similar to that in [28]. The visual stimulus used for the first step training (top, (ooll) and the visual stimulus used for the second step training (bottom, loc2_ ori2). B. Learning curves and thresholds for pre- and post-testing in different conditions. The left panel corresponds to the results of the first step training in Fig. 4B. The right panel displays the results of double training. Notably, the threshold at the second trained condition (ori2_loc2, deep red circles) decreases significantly after double training, and thresholds untrained locations (oril_ 1ori__ light red squares) untrained orientations (ori2_loc1, medium red triangles) also exhibit notable declines. C. Statistical results of training and transfer improvements. The left panel corresponds the results of the first step training as in Fig. CC. The right panel displays the results of double training. A substantial gain at the second trained condition (ori2_loc2, deep red bar) and untrained conditions (oril_lock) light red; ori2_loc1, medium red)
Conclusions and discussions
All learning agents, including the brain, face a fundamental challenge: balancing task performance with the cost of learning. Intuitively, if the agent does not need to account for the feature distribution in the external world, it can quickly adapt by reusing existing feature representations to complete tasks efficiently and at low cost. However, when the agent identifies meaningful statistical patterns in the environment, it can refine its feature representations to enhance precision and effectiveness. Although beneficial, this adaptation is both resource- -nouurceinteesive and slow, as the agent must first distinguish between genuine environmental changes and random fluctuations. In essence, to learn or not to learn, is generic question faced by all learning agents.
In this work, starting from the goal of reconciling the conflicting phenomena of specificity and transfer in perceptual learning, we present a solution of the brain, i.e., the dual-learning framework. This framework consists of two learning processes: task-based one and feature-based one. Specifically, task-based learning is fast, which enables the agent to learn to accomplish task rapidly by using existing feature representations; while feature- learning is slow, which enables the agent to improve feature representations to reflect the statistical change of the external environment.
Our dual-learning model successfully replicates and elucidates classical experimental findings related to specificity and transfer in perceptual learning. It reveals that the interaction between the slow- changing, specific feature learning the early visual pathway and the flexible, transferable task learning at the higher visual pathway governs the adaptability of perceptual learning. Typically, learning tends to adjust the readout of neural representations (i.e., task-based learning) rather than altering the representations themselves (i.e., feature-based learning). unless there is a significant change in the statistical properties of external information. Thus, the default state of learning favors transfer. However, in traditional experimental paradigms, the frequent repetition of stimuli leads to the dominance of feature- based learning, thereby exhibiting significant specificity; by limiting the repetition of stimuli, task-based learning can dominate, thereby enhancing transfer. As such, perceptual learning can display a transition from transfer to specificity as the number of stimulus repetitions increases.
Our dual-learning model can be regarded as a computational modeling implementation of the two- stage model theory [44] While the two-stage model theory aims to address the contradictions between rask-related and task-unrelated perceptual learning, it posits the existence of feature-based and fask-based plasticity within perceptual learning. Thus, our model is a practical realization of

---

this theory. Furthermore, the dual-learning framework aligns with the reverse hierarchy theory [5] which suggests that learning processes invert the sequence of visual information processing from top-down. As tasks become more difficult and specific, lower levels of the neural hierarchy are increasingly engaged, aligning learning outcomes with enhanced task specificity. This alignment illustrates how the dual-learning model not only accommodates but also substantiates theoretical perspectives on perceptual learning's hicrarchical nature, providing a robust framework for exploring how different learning mechanisms interact within the brain's s architecture.
Limitation and future work. In this study, as a first step to elucidate the notion of dual-learning, have built very simple network model without including many biological details of the visual pathway. In future work, we will extend the current model to include more biological details and apply the model to explain more cognitive functions of the brain. In essence, the dual-learning framework reflects that the brain employs a learning strategy to balance the accomplishment of task and the consumption of learning effort, that is, if the statistics of the environment unchanged, low-cost baske learning is applied: otherwise, high-cost feature-based learning is triggered. The same balance requirement is faced by other learning agents. We, therefore, expect that the dual-learning framework has the potential to be applied in Al applications, and we will explore this issue in future work.
Acknowledgments and Disclosure of Funding
This work was supported by the National Natural Science Foundation of China (no. T2421004, S. S.W.). the Science and Technology Innovation 2030-Brain Science and Brain-inspired Intelligence Project (no. 20212D0200204, S. S.W.), and the STI2030-Major Projects grant (no. 20227/0200600, C.Y.).
10

---

References
[1] E Gibson. Perceptual Learning. Annual Review of Psychology, 14(1):29-56, 1963.
[2] Charles D. Gilbert, Mariano Sigman, and Roy E. Crist. The Neural Basis of Perceptual Learning. Neuron, 3165):681667,, 2001.
[3] Manfred Fahle and Tomaso Poggio, editors. Perceptual Learning, MIT Press, Cambridge, Mass, 2002.
[4] Zhong-Lin Lu and Barbara Anne Dosher. Current directions in visual perceptual learning. Nature Reviews Psychology, 1(11):654-668 September 2022.
[5] Barbara Dosher and Zhong-Lin Lu. Visual Perceptual Learning and Models. Annual Review of Vision Science, 3(1):343-363, September 2017.
[6] Dov Sagi. Perceptual learning in Vision Research. Vision Research, 51(13):1552-1566, July 2011.
[7] Yael Adini, Dov Sagi, and Misha Tsodyks. Context-enabled learning in the human visual system. Nature, 415(66733)7700-79, February 2002.
[8] Cong Yu, Stanley A. Klein, and Dennis M. Levi. Perceptual learning in contrast discrimination and the (minimal) role of context. Journal of Vision, 4(3):4, March 2004.
[9] Manfred Fahle and Shimon Edelman. Long-term learning in vernier acuity: Effects of stimulus orientation, range and of feedback. Vision Rexearch, 33(3)397-412, February 1993.
[10] Adriana Fiorentini and Nicoletta Berardi. Perceptual learning specifie for orientation and spatial frequency. Nature, 287 44, 1980.
[11] Adriana Fiorentini and Nicoletta Berardi. Learning in grating waveform discrimination: Speci- ficity for orientation and spatial frequency. Vision research,
1981. [12] Poggio, M Fahle, and Edelman. Fast perceptual learning in visual hyperacuity. Science, (56(5059)10018-1000 May 1992.
[13] Manfred Fahle. Human Pattern Recognition: Parallel Processing and Perceptual Learning. Perception, 23(4):411 12-71, April 1994.
[14] Paul T. Sowden, David Rose, and Ian R.L. Davies. Perceptual learning of luminance contrast detection: Specific for spatial frequency and retinal location but not orientation. Vision Research, 12(10):1249-122 May 2002.
[15] Karlene Ball and Robert Sekuler. Specific and Enduring Improvement in Visual Motion Discrimination. Science, 218(4573):697-698 November 1982.
[16] Tiffany Saffell and Nestor Matthews. Task-specific perceptual learning on speed and direction discrimination. Vision research, 43(12): 1365-13344 2003.
[17] Lucia M. Vaina, V. Sundareswaran, and John Harris. Learning to ignore: Psychophysics and computational modeling of fast learning of direction in noisy motion stimuli. Cognitive Brain Research, 203):155-163, July 1995.
[18) Manfred Fahle. Perceptual learning: A case for early selection. Journal of vision, 4(10)444, 2004.
[19] Manfred Fahle and Michael Morgan. No transfer of perceptual learning between similar stimuli the same retinal position. Current Biology, 6(3):292-297, March 1996.
[20] Karni and D Sagi. Where practice makes perfect in texture discrimination: Evidence for primary visual cortex plasticity. Proceedings of the National Academy of Sciences, 88(1 ::496- 4970, June 1991.
[21] Merav Ahissar and Shaul Hochstein. Task difficulty and the specificity of perceptual learning. Nature, 18766631) May 1997.

---

[22] S.-C. Hung and A. R. Seitz. Prolonged Training at Threshold Promotes Robust Retinotopic Specificity in Perceptual Learning. Journal of Neuroscience, 34(25):8423-8433 8431, June 2014.
[23] Zili Liu and Daphna Weinshall. Mechanisms of generalization in perceptual learning. Vision Research, 40(1):77-109, January 2000.
[24] P. Jeter, B. A. Dosher, A. Petrov, and Z. L. Lu. Task precision at transfer determines specificity of perceptual learning. Journal of Vision, 9(3)1-1, March 2009.
[25] Kristoffer C. Aberg, Elisa M. Tartaglia, and Michael H. Herzog. Perceptual learning with Chevrons requires a minimal number of trials, transfers to untrained directions, but does not require sleep. Vision Research, 49(16):2087-2094 August 2009.
[26] Pamela E. Jeter, Barbara Anne Dosher, Shiau-Hur Liu, and Zhong-Lin Lu. Specificity of perceptual learning increases with increased training. Vision Research, 50(19):1928-1940, September 2010.
[27] Avi Karni and Dov Sagi. The time course of learning visual skill. Nature, 365(6443):250-25555 September 1993.
[28) Lu-Qi Xiao, Yun Zhang, Rui Wang, Stanley A. Klein, Dennis M. Levi, and Cong Yu. Complete Transfer of Perceptual Learning across Retinal Locations Enabled by Double Training. Current Biology, 18(24):1922-1926, December 2008
[29] Ting Zhang, Lu-Qi Xiao, Stanley A. Klein, Dennis M. Levi, and Cong Yu. Decoupling location specificity from perceptual learning of orientation discrimination. Vision Research, (0(4):368-374
February 2010.
[30] Rui Wang, Yun Zhang. Stanley A. Klein, Dennis M. Levi, and Cong Yu. Task relevancy and demand modulate double-training enabled transfer of perceptual learning. Vision Research, 61:33-38, May 2012.
[31] Jun- Yun Zhang, Lin-Juan Cong, Stanley A. Klein, Dennis M. Levi, and Cong Yu. Perceptual Learning Improves Adult Amblyopic Vision Through Rule-Based Cognitive Compensation. Investigative Opthalmology & Visual Science, 55(4):2020, April 2014.
[32] Rui Wang, Jie Wang. Jun-Yun Zhang, Xin-Yu Xie, Yu-Xiang Yang, Shu-Han Luo, Cong Yu, and Wu Li. Perceptual Learning at Conceptual Level. The Journal of Neuroscience, 36(7):2238-222
February 2016.
[33] Ying-Zi Zi Xiong, Jun-Yun Zhang, and Cong Yu. Bottom-up and top-down influences untrained conditions determine perceptual learning specificity and transfer. eLife, 5:e14614, July 2016.
[34] Xin- Yu Xie and Cong Yu. A new format of perceptual learning based evidence abstraction from multiple stimuli. Journal of Vision, 20(2):5, February 2020.
[35] Hila Harris, Michael Gliksberg, and Dov Sagi. Generalized Perceptual Learning in the Absence of Sensory Adaptation. Current Biology, 22(19):1133-111, October 2012.
[36] Giorgio L. Manenti, Aslan S. Dizaji, and Caspar M. Schwiedrzik. Variability in training unlocks generalization in visual perceptual learning through invariant representations. Current Biology. 33(5):817-826.e; March 2023.
[37] Tomaso Poggio, Shimon Edelman, and Manfred Fahle. Learning of visual modules from examples: A framework for understanding adaptive visual performance. CVGIP: Image Understanding, 56(1 (:22330,, July 1992.
[38] Andrew Teich and Ning Qian. Learning and Adaptation in Recurrent Model of V1 Orientation Selectivity. Journal of Neurophysioloy,, 2944)22086-2100, April 2003.
[39] Alexander A. Petrov, Barbara Anne Dosher, and Zhong-Lin Lu. The Dynamics of Perceptual Learning: An Incremental Reweighting Model. Psychological Review, 112(4):15-774,, 2005.
12

---

[40] Alexander A. Petrov, Barbara Anne Dosher, and Zhong-Lin Lu. Perceptual learning without feedback in non-stationary contexts: Data and model. Vision Research, 46(19):117731997 October 2006.
[41] Barbara Anne Dosher, Pamela Jeter, Jiajuan Liu, and Zhong-Lin Lu. An integrated reweighting theory of perceptual learning. Proceedings the National Academy of Sciences, 110633):1367-- 13683, August 2013.
[42] M. Solgi, T. Liu, and J. Weng. A computational developmental model for specificity and transfer in perceptual learning. Journal of Vision, 13(1):7-7, January 2013.
[43] Li K. Wenliang and Aaron R. Seitz. Deep Neural Networks for Modeling Visual Perceptual Learning. The Journal of Neuroscience, 38(27):6028-60M) 6044, July 2018.
[44] Kazuhisa Shibata, Dov Sagi, and Takeo Watanabe. Two-stage model perceptual learning: Toward a unified theory: Two-stage model in perceptual learning. Annals of the New York Academy of Sciences, 1316(1):18-22)) 2828 May 2014.
[45] Merav Ahissar and Shaul Hochstein. The reverse hierarchy theory of visual perceptual learning. Trends in Cognitive Sciences, 8 8(10):457-464, October 2004.
[46] Kenneth D. Miller. Synaptic Economies: Competition and Cooperation in Synaptic Plasticity. Neuron, 17(3)377-374, 1996.
[47] Donald Laming and Janet Laming. F. Hegelmaier: On memory for the length of a line. Psychological Research, 54(4):233-239 1992.
13

---

Appendix
A
Model Details
The hicrarchical neural network model used in this study consists of three sequential stages:
A.1 Feature Extraction
As defined in Eq. of the main text, the feature representation Fx(x, 0) is calculated as follows: norm [G(x x' [x')],
(x, 0)
(A1)
where G(x - 0') represents the basis function, preferred for its ability to capture specific position and orientation features from the input image. The symbol represents the convolution operation, and the norm(.) denotes a normalization operation, i.e., norm(F* = (F* - F / F F meann and F the mean and standard deviation of the representation variable /²*.
std= with
The selected basis function a Gabor function characterized by a sinusoidal wave modulated by Gaussian envelope. It is mathematically expressed as:
xx)² (x
x' cos
G(x- x',)) exp
exp
COS
2T
(A2)
2n²
where x and 0 define the preferred position and orientation of the basis function.
We configured the network with 40 18 basis functions uniformly cover all possible positions and angles. Specifically, the 40 basis functions span the positional space, evenly distributing across different position scales, while the 18 basis functions cover the angular space, with each basis function representing 10-degree increment (i.e., 18 180 150%/10)). It is important to note that while traditionally represents a two-dimensional position within the image, we simplified the model by setting the size of the basis function to the height of the image, thus reducing the two- dimensional position variable to one dimension. This simplification does not affect the outcome of our experiments.
Table Al below summarizes the parameters used for the Gabor function in our experiments.
Table Al: Gabor Parameters for Basis Function
Amplitude
Go
0.5 0.01 cycles/pixel
Spatial frequency
Phase
T/22 radians
Standard deviation
30 pixels
A.2 Feature-Based Learning
As defined in Eqs. 2 in the main text, the refined feature representations Ft(x, 0) at each trial are computed based on the feedforward connection weights Wx(xx''). Below are the mathematical details.
A.2.1 Feedforward Convolution Process
The refined feature representations are updated using a convolution process that integrates feedforward connection weights with previous feature representations:
F2 0)
norm
Σ
F
(A3)
The feedforward connection weight Wx x') from position x' the previous layer to position x is
defined
Ax41(x))
xx
(x
Wx+1xx,))
cxp
(A4)
2ma
2a
The code is available Github at http:///wg.g.a.......ooooo/a/aaa
14

---

Here, A++1(x') represents the magnitude of the connection, while controls the spread of the connection. For simplification, we consider a 0, the connection weight simplifies to:
{ A (x') 0
if x',
lim W a-0
(A5)
otherwise.
This indicates that, in the limit, the connections become fully localized, creating direct, one-to-one mappings between corresponding positions.
The normalization operation norm(-) and the convolution operation consistent with previous descriptions.
A.2.2 Competitive Hebbian Learning Updating
The strength of feedforward connections, Ax(x), is updated following competitive Hebbian learning rules [46], aimed at reinforeing location-specific changes in the neuronal representations.
First, competitive learning identifies strong responses from prior feature representations for weight updates:
Fx, sign [Pt- (x. 0) Fo], (x) = sign [F (x, - F
(A6)
(A7)
where Fo and Fö thresholds 0.6, effectively selecting the top 20% of neurons based response strength.
Then, Hebbian learning computes and updates the connection strength A.(x):
AA+(x) meang [ meang [x, 0)
(A8)
(x) A,(x) + ())
(A9)
where Ax,(x) (xeeresssst the updated value for weights, 7/8 is the learning rate, and meang(.) cal- culates the average over different orientations, considering the columnar structure of orientation preference in the primary visual cortex.
Table A2
below summarizes the parameters used for the ceature-based Learning in our experiments.
Table A2: Parameters for Feature- Learning
Parameter
Symbol
Value
Threshold of (x 8)
Fo 0.6
Threshold of F2 (x, ()
Fo
0.6
Learning rate of fcature-based learning
me 0.003
A.3 Task-Based Learning
As defined in Eq. of of the main text, the decision 2 at each trial is computed using a convolutional network followed by global max pooling. The decision 2t takes value of 0 or 1, representing whether the upper orientation is more leftward (0) or more rightward (1), respectively. During decision-making, the output is classified as I if the value exceeds 0.5; otherwise, it is classified 0.
The CNN used in our model is streamlined convolutional neural network comprising three convolu- tional layers.
Layer 1: Inputs a single channel and uses a 3x3 convolutional kernel to output 6 channels.
Layer Processes the channels with another 3x3 kernel, producing 10 channels.
Layer 3: Compresses the 10 channels into single output channel using 3x3 convolution.
The network employs ReLU activation after the first two convolutional layers to introduce non- linearity, followed by flattening and ID max pooling on the final output to ensure well-structured output.
15

---

The learning updates the network parameters through cross entropy loss minimization: T (z,ln(=f) + (1 zx)ln(1 =;)]
(A10)
where backpropagation is used for efficient parameter adjustment with an Adam optimizer. The learning rate Th. is set 0.001 to ensure gradual and steady learning.
B
Experimental Protocol for Ablation Studies
B.1
Stimulus
The Vernier discrimination images used in this study were adapted from the Matlab code in 41 Each image has a size of 800 x 200 pixels (Fig. [A). The stimulus within each image consists of 200 x 200 pixels Vernier pattern overlaid on background of visual noise.
The Vernier stimulus consists of two Gabor patches arranged either horizontally or vertically. single Gabor patch is defined by:
x² + + y2
G3 = Go cos|2mt cos 0 + ysin 0) + x/} exp
(A11)
20²
where the parameters are described in Table A3
Table A3: Gabor Parameters for Stimulus
Parameter
Symbol
Value
Amplitude
Go
0.6
Spatial frequency
0.02 cycles/pixel
Orientation
09 or 90% (vertical or horizontal stimulus)
Phase
/T/2 radians
Standard deviation
25 pixels
The Gabor patches are offset by value O that is randomly generated for each trial:
= (2 + Na).
(A12)
Na [na],
(A13)
nd N (0.0.5").
(A14)
where [-] denotes rounding to the nearest integer, and ny follows a normal distribution with a mean of O and a standard deviation of 0.5.
The background noise is generated by:
Noise anoise . (1,2²),
(A15)
where Onoise is the noise amplitude, and the normal distribution has mean of and standard deviation of
B.2 Simulation Details
The model is trained at a fixed location and tested at three untrained locations with increasing distances from the trained location, as illustrated in Fig. 3 of the main text.
Procedure:
Train the model for 200 epochs, using batches of 16 trials each.
Measure the discrimination accuracy at both trained and untrained locations to assess the model's performance.
Average all results over 50 repetitions to ensure the robust and reliable findings.
To analyze the contributions of feature-based and task-based learning, three specific ablation studies conducted:
16

---

1. Task-Based Learning Only: Feature-based learning is disabled, isolating the effect of task-based learning to on model performance across different locations.
2. ceature-Based Learning Only: Task-based learning is disabled, and only Seature-based learning operates. Initialize the weights as the weight completing task-based learning. During training, the similarity between the feature representations F," (x, 8) and the refined representations Fx(x, 0) under different stimulus locations (1 trained location and 3 untrained locations) and orientations (0", 45', 90º) is measured. The similarity is calculated as:
n -
Similarity
(A
(
where denotes the total number of representations.
3. Combined Feature-Based and Task-Based Learning:
Both learning modes are activated, but feature- based learning uses lower rate(0.00001) compared task-based learning (0.001). The model starts with weights initialized at the beginning of learning, which helps evaluate how the combined learning dynamics influence the speed and effectiveness of training across different spatial contexts.
C Reproduction of Experimental Phenomena
C.1
Stimulus
In this study, Vernier discrimination stimuli were generated described previously. The stimuli were available in two sizes: 400 x 200 pixels (used for the first, third, and fourth experiments; Fig. A Fig. 6A and Fig- A and 800 200 pixels (used for the second experiment; Fig. 5A).
Each Vernier stimulus consists of two Gabor patches with orthogonal orientations: oril 0%: horizontal orientation) and ori2 (0 90%; vertical orientation), described in Eq. AIT
In the first, third, and fourth experiments, the Vernier stimulus was centered two locations: loc1 (100, 100) and loc2 (300, 100). In the second experiment, two additional locations were added: loc3 (500, 100) and loc4 (700, 100).
The offset between the two Gabor patches varied across 10 difficulty levels, defined as:
O 2 (2D Na),
(A17)
where D E [1, 10) is the difficulty level, No [na] is the rounded difficulty noise, and:
nd N (p, оa 0.5.
(A18)
The probability density function v (na) for the truncated normal distribution was as follows:
Rdd S a < nd < b b < na where e and & were the probability density and cumulative distribution functions, respectively. The values a -1 and are the truncation points.
{
0,
o(H.afona) ((u.03:b) ( ((H,o
/ (na)
(A19)
0,
The background noise N generated as:
N anoise nI, N (pms x
(A20)
where anoise was the noise intensity, n was the raw noise, and N followed normal distribution with mean pn and variance of 2. The stimulus images generated in each trial were mutually independent.
17

---

C.2 Measuring Difficulty Threshold
All the results in this study were obtained by independently training 100 models with the same set of parameters under different experimental conditions (e.g., variations in random initialization and training stimulus noise).
To compare with human perception experiments, we measured the difficulty thresholds for each model every test point. A threshold represents the intensity which difference between two stimuli is just detectable, with stimuli below considered subliminal. Here, we used the constant stimuli method measure the threshold at which model accuracy reached 80%, comparable
the 79% threshold in human subject experiments 28 (26] [30] (34]. The constant stimuli method is a standard psychophysical approach for measuring perceptual thresholds 47 involving the presentation of stimuli at several constant levels and fitting psychometric function to the responses.
At each test point, evaluated model accuracy across 10 difficulty levels. Each level included 20 positive and 20 negative samples, totaling 40 trials. The model's 's accuracy at each level was used to fit psychometric function based the cumulative normal distribution:
of 2 2m (-
D. exp ( (t-16t8 1faa22
A, f
dt + (1 (1-C)+C )
(A21)
20 fit
where A, denotes the accuracy difficulty level Di, Mfitt and fit the mean and standard deviation of the fitted distribution, and the is the error between the observed and fitted accuracy.
And C' denoted the chance level accuracy, which 0.5 for this binary forced-choice task.
The goal of the fitting process is to minimize the total error:
min pffee fSt
(A22)
where Apoint = 10. Once fitted, the threshold difficulty Dthreshold corresponding to an 80% accuracy level calculated as:
D Dubreshold = f- (Athreshata) Athreshold 0.8.
(A23)
The fitting process was conducted using the scipy. oot.iiiz..mmize..... curve_fii function from the Python scipy library, with upper bounds for for and fūr set to 30 and 13, respectively, based on initial pre-training results. The cumulative distribution function and its inverse were computed using scipy.stats stats norm. cdf and staipysstts. norm.pof. ppf.
In addition to directly presenting the threshold values, we also calculated the Percent Improvement (PI) for each experiment to quantify the effect of learning. The PI for model m, denoted In was defined as:
Im (threeshold ftorm thresholdse
(A24)
Uhresholdee fore..
where thresholde forem and thrreholdd fterm were the threshold measured before and after learning, respectively.
C.3
Experimental Simulation
C.3.1 First Experiment
Training Condition: oril_low
Testing Conditions: oril oril_loc2) sol__loe,,00 cri2_oocl, loc2
Stimulus Noise Intensity: 0.6
Learning Rate: 7 = 0.3 (feature-based learning), Th. 0.001 (task-based learning)
Pre-training Since the model also needed to measure the threshold before formal training (i.e., pre-testing), at this time, due to the random initialization of the model parameters, the stimulation accuracy rate all difficulties was the chance level (i.e., 50%). and the psychometric function was a horizontal line, which could not measure the threshold. Therefore, the model needed to be pre-trained
18

---

before formal training. During pre-training, the model was trained with tasks set at difficulty level 10 (the easiest) all training and transfer conditions. Each condition consisted of 16 trials, forming one training batch, Stimuli of all conditions were sequentially trained batch by batch for 20 epochs. Stimuli of the two orientations trained shared feature-bast based learning model component and two distinct aask-based model components.
Formal Training The formal training was conducted under the training condition at difficulty level 10. Every 16 trials formed a training batch, and 20 training batches constituted one training session. During the training process, both the ceature-based learning model component and the task based model component corresponding to that orientation were trained. After completing training across 10 training sessions, the traditional training phase was completed.
Pre- and Post- Post-Testing We performed pre- and post-testing on all training and transfer conditions, before the formal training (after pre-training) and after the formal training, respectively. We measured the 80% threshold of all the training and transfer conditions and calculated the improvement using the method mentioned in Sec.
C.3.2 Second Experiment
Training Conditions: loc2, oril [oc3,oc3, oril oril_loc4, ori2_locl, ori2 _o__, ori2 loc3, ori2_loc4
Testing Condition: oril_loc!
Stimulus Noise Intensity: = 0.6
Learning Rate: = 0.3 (feature-based learning), 1h 0.001 (task-based learning)
Pre-training
Same as Sec. C.3.
Formal Training For the random training condition, the trials were shuffled, while for the rotation
condition, trials were presented sequentially in dictionary order. Other procedures were the same as Sec. C33.1
Pre- and Post-Testing
Testing followed the same protocol described in Sec. C.3.1
C.3.3
Third Experiment
Training Condition: oril_loc1
Transfer Condition: ori2_loc2
Stimulus Noise Intensity: 1.2
Learning Rate: = 3 (feature- based learning), 7), = 0.001 (task-based based learning)
Pre-training Same as Sec. C.3.1
Formal Training For the first training phase, the model was trained the training condition for 4, 8, or 12 sessions according to different conditions. For the transfer training phase, models of all the conditions were trained the transfer condition for 10 sessions. Other details were similar to Sec. CC3.1
Pre- and Post-Testing
Testing followed the same protocol as in Sec. C.3.1
Statistical Comparison In Fig. 63, we argue that at the final transfer session, the thresholds are from low to high as the training sessions increase. The results of the t-tests conducted on the data are presented below in Table which contains the p-values T-tests for points at the final transfer session in Fig. 63. As observed, the statistical differences between any two conditions of the experiment are significant. This finding aligns with the results depicted Fig. [C(i), indicating the consistency across different parts of the study.
19

---

Table A4: Statistical Comparison of the Third Experiment
T2
T4
T8
T12
T2
1
0.0054
2c-11
2.1e-21
T4
0.0054
I
0.00026
1.4e-09
T8
2e-11
0.00026
1 0.02
0.02
T12
2.1e-21
1.4c-09
C.3.4
Fourth Experiment
Training Condition: oril_ pri__ocl
Double Training Condition: ori2_loc2
Testing Conditions: oril ori1_loc2, ori2_loc1
Stimulus Noise Intensity: 1.2
Learning Rate: = 3 (feature- based learning), = 0.001 (task-based basse learning)
Pre-training Same as Section C.3.1
Formal Training The model trained for 10 sessions under both the initial and double training
conditions. Other details were similar to Sec. C.3.1
Pre- and Post- mest-Testing Pre- and post-testing followed the same protocol as in Sec. C.3.
Outlier Removal Some results shown in Fig. 7B and Fig. 7C appear inconsistent, which may confusion (e.g. the threshold of oril _oclllo22aa at session 10 in the training condition is lower than that at session 0 in Fig. [B. indicating
improvement in perception. However, in Fig. 7CC the improvement of oril oril_loo aril_loc22n0 after the first step training is around O, suggesting no learning). This inconsistency arises due to that Fig. 7 presents the average of 100 simulation runs, while the improvement in Fig. 7C was calculated by averaging the rate of improvement from each simulation, where a few outhiers affected the results. After excluding 4 outliers (with z-scores of improvements greater than 4), the results are consistent (see Fig. Al
oril
Training
oril locl
loc1
loc
oci
oril loc2
ori2 loc2
loc2
Difficuity threshold
Improvement (%6
or12 Joe
Double training
15h
10:
0
10
10
Double training
leamnng
Transfer
Transfer
Session
Figure A1: Transfer in perceptual learning via double training after removing four sets of outliers. The detailed figure deseription can be found in Fig. 7
C.4 Additional Ablation Study
We conducted two additional ablation studies to highlight the importance the relative speeds of feature-based and task-based learning. The results, based on 100 simulations, confirm that both slow feature-based learning and fast task-based learning are essential for replicating perceptual learning phenomena in model.
20

---

C.4.1
Accelerating Feature-based Learning
As shown in Fig. A2 increased the learning rate of feature-based learning by tenfold and replicated four experiments from Sec. in the main text. The detailed results are as follows:
Exp.1
Esp33
A
A
Sooee
Taeeay
stage
stope
Wandeer
Exp.2
Exp.4
D2 by threddn
Figure A2: Accelerating feature-base learning results. Repeat the experiments from Sec. 2 with tenfold increase in the learning rate for -based learning. Experiments 1-4 repeated from those shown in Sec. 2 The results include statistics from 100 simulation experiments, with all other parameters consistent with the model in the main text.
First Experiment Accelerated ceature-based learning led significant modifications in feature representations, resulting in degraded performance at untrained locations (especially for oril oril_loc2).
Second Experiment Due to continuous changes in experimental conditions and accelerated learn- ing, the model failed to stabilize representations. As a result, performance in the task was compro- mised, and human-like like transfer was not achieved.
With increased training sessions, differences between conditions diminished, and learning curves became entangled (especially between T8 and T12). The statistical results, shown in Table A5 indicate that the p-values increased compared to Table A4 reflecting reduced significance between conditions.
Third Experiment
Table A5: Statistical Comparison of the Third Experiment with Accelerating Feature-
Feature-based based Learning
T2
T4
T8
T12
T2
1
0.019
4.9e-05
1.1e-0
T4
0.019
0.083
0.025
T8
4.9e-05
0.083
0.61
T12
5.1e-06
0.025
0.61
Fourth Experiment Double training no longer achieves the transfer of learning, and performances in both testing (transfer) conditions are even worse after double training.
C.4.2 Slowing Down Task- Learning
As depicted in Fig. A3
we decreased the learning rate of task-based learning by tenfold. This adjustment significantly degraded learning efffcttiveness, as the model struggled master tasks efficiently.
First Experiment Although the model could fully master the task, comparison with Fig. A2 shows significant difference in transfer effects between training and transfer conditions.
21

---

Exp.1
Exp
Sas undeentt
Tan 12 snssoneti
Tariing Transfer
Exp.2
Exp.4
ketsing
DRuty treastd
off treshee
Repowerenn (N)
OFF Co Breant
Voiiing d Trander
Figure A3: Slowing down task-based learning results. Repeat the experiments from Sec. 2 with learning rate for task-based learning decreased by factor of ten. Experiments 1-4 are repeated from those shown in Sec. 2 The results include statistics from 100 simulation experiments, with all other parameters consistent with the model in the main text.
Second Experiment Despite the model's poor mastery of tasks, the improvements gained during learning transferred well to new conditions.
Third Experiment
The slower learning disrupted performance consistency across conditions, leading to entangled learning curves. Statistical analysis, shown in Table A6 reveals increased p-values compared to Table A4 indicating smaller differences between conditions.
Table A6: Statistical Comparison of the Third Experiment with Slowing Down Task-based Learningg
T2
T4
T8
T12
T2
0.68
0.97
0.76
T4
0.68
0.71
0.48
T8
0.97
0.71
0.73
T12
0.76
0.48
0.73
1
Fourth Experiment Due to poor learning outcomes, double training failed to achieve transfer. However, comparison with Fig. shows minimal disruption in transfer conditions.
22

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims in the abstract and introduction are well-aligned with the contributions and scope of the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
The paper includes a discussion section that addresses the limitations of the work.
Justification:
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiitiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
23

---

Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in the Appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes].
Justification: The paper provides detailed descriptions of the experimental procedures, the code, and the data.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
24

---

Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: The authors have stated that they will provide a GitHub link with the data and code after the anonymous review process.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code permitted.
6. Experimental Setting/eetails
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper provides comprehensive details regarding the training and testing procedures.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in the Appendix, or as supplemen- tal material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper reports error bars and provides appropriate information about the statistical significance of the experiments. These error bars are clearly defined in the figures, representing s.m.m.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
25

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [No]
Justification:
The paper does not provide detailed information the computer resources needed to reproduce the experiments. However, this omission is justified because the model used is very simple and does not impose significant computational demands. Therefore, spec- ifying resources like compute workers, memory, or execution time was deemed unnecessary for the context of this study.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: The research adheres to the NeurlPS Code Ethies by ensuring transparency, reproducibility, and fairness in reporting experimental results.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
26

---

Justification: The paper discusses potential societal impacts by highlighting the benefits of learning, which can contribute advancements in fields like medical imaging and autonomous vehicles. It also addresses possible negative impacts, such as the specificity of learning. This balanced discussion demonstrates the authors' awareness of the broader.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The authors properly credit the creators of the code and data used in the paper by citing the original source.
Guidelines:
The answer NA means that the paper does not existing assets.
27

---

The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the datasets.paperrsst paperswi thcode. cow datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors are encouraged to reach out to the asset's ss creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
The documentation of new assets introduced in the paper will be assessed once the supplementary material, including code and data, made available. This will ensure that all necessary details and instructions are provided to facilitate the reproduction and understanding of the introduced assets.
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
28

---

Answer:
[NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
29


Reviews:
[{"review_id": "Zom5Xu0BKQ", "reviewer": "Reviewer_UGym", "summary": "The paper puts forth a theoretical framework for perceptual learning, in which two separate learning processes contribute to learning a perceptual task (a fast, flexible task-based learning that relies on existing feature representations; and a slow, task-specific feature learning). Repeated learning sessions with the same stimulus conditions triggers feature-based learning, which will be specific to the training stimulus conditions. The framework is instantiated as a neural network model, and is used to account to both specific and transfer phenomena in psychophysical experiments.", "questions": "Can the authors clarify what makes the task-based learning “fast”? Is this accomplished via learning rate hyperparameter tuning?", "limitations": "OK", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "The paper proposes a novel computational model to explain when specificity and transfer is observed in perceptual learning. The crux of the proposed dual-learning theory is that experiments exhibiting transfer are dominated by task-based learning (which operates over existing feature representations and is adaptable), and that experiments exhibiting specificity are dominated by feature-based learning triggered by excessive exposure to the same stimulus condition triggering feature-learning to adapt to these new environment statistics.\n\nThe computational framework is instantiated as a neural network model with feature extraction (via a set of basis functions), feature-based learning which modifies the feature representations via hebbian learning, and a task network (CNN that performs the task). This network recapitulates an impressive array of human psychophysical phenomena, showing specificity when repetition of the same stimulus conditions is high, and transfer when there is more variability in the stimulus conditions. The framework impressively captures nuances of learning dynamics across multiple task variations.\n\nAs such, the work provides a strong framework for understanding diverse and seemingly contradictory findings in human psychophysics. It also presents an opportunity for impact on machine learning theory, if the slow-feature-learning and fast-task-learning framework has broader benefits for machine learning systems (e.g., could this inform approaches to life-long continual learning research?).", "weaknesses": "One important limitation of this work is that the networks appear to be trained from scratch on the “probe task” (e.g., vernier acuity). In contrast, with human perception, the person’s visual system is tuned over their lifetime (presumably via some self-supervised objective), and then they are put into an experimental setting where 1-2 hours per day for N days the perform this new task. So the impact in terms of perceptual experience is set within this very broad context of “learning to see”, and a visual system that’s capable of much more than just the task at hand (e.g., vernier acuity). I suppose this begs the question of whether this all works as well if the network is initially pretrained (say on some self-supervised learning objective), and then the same types of experiments are run. I don’t see in principle why these same ideas wouldn’t apply.\n\nMinor: When the computational framework is first introduced, the different architectural choices are unmotivated (a classical basis function network for image processing — What is this and why this? A Hebbian network for feature learning – Why a Hebbian network? Why is feature learning a separate step from the basis function learning? And then a CNN with max pooling for task-based learning. Why a CNN for this purpose, and a Hebbian network for the feature learning?). I realize these choices are all motivated, they just come out of the blue all at once, and it’s hard for the reader to grok the motivation for this setup.\n\nMinor: The methods introduce the Feature Extraction as akin to the retina or LGN, but what’s described appears to be an orientation-tuned Gabor Wavelet Basis Set (and orientation-tuning is believe to arise later in V1 and beyond).\n\nMinor: Can you provide more information on the architecture of the task-based CNN?\n\nMinor: Can the authors provide further justification/clarification for the different learning algorithms implemented at each level of the network? e.g., Why shouldn’t Hebbian learning play a role “all the way to the top” of the task network? Is this biologically motivated, or a convenient way to implement assumptions of the model (e.g., location-specific learning)?\n\nMinor: Other related work includes “How variability shapes learning and generalization” (Raviv et al, Trends in Cognitive Sciences)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your encouraging and valuable comments. We streamline reviewers' main concerns and address them one-by-one.\n\n**Q1:** In the current study, we have used a feature extraction module that remains unchanged during the learning process to reflect the pre-processing of visual inputs in the brain. This setup mimics the resulting capability of the human visual system that has undergone extensive exposure to diverse environments throughout life. In the typical psychophysical experiments which last at most hours, we regard this part of pre-processing as unchanged. To accomplish the Vernier discrimination task in this work, we have employed a simple basis function network for pre-processing. Certainly, if more complicated tasks are included, we can employ more sophisticated pre-trained models.\n\n**Q2:**  As for the choice of models and learning methods, please refer to the overall rebuttal. \n\n**Q3:**  Indeed, as pointed out by the reviewer, orientation tuning emerges in the visual cortex. Our description of feature extraction in the paper was not precise. The feature extraction module in our model should include the input layer of V1 (responsible for extracting visual features). We will revise the statement in the manuscript accordingly.\n\n**Q4:**  The CNN used in our model is a streamlined convolutional neural network comprising three convolutional layers. \n- The initial layer, layer 1, inputs a single channel and uses a 3x3 convolutional kernel to output 6 channels. \n- This is followed by layer 2, which processes these 6 channels through another 3x3 kernel to produce 10 channels. \n- The concluding layer, layer 3, compresses these 10 channels into a single output channel using a 3x3 convolution. \n\nThe network employs ReLU activation after the first two convolutional layers to add non-linearity and includes a sequence of flattening and a 1D max pooling on the final output to structure the output appropriately. \n\n**Q5:**  As for the model choice, please refer to our overall rebuttal. Indeed, the most fundamental difference lies in the objective functions of the two learning processes. Utilizing Hebbian learning (Hebb) or backpropagation (BP) mainly simplifies the model implementation.\n\n**Q6:**  From the perspective of the dual-learning framework, variability enhances generalizability primarily because it prevents feature-based learning from confining learning effects to specific feature combinations, thus allowing task-based learning to dominate and exhibit transferability.\n\nOn the other hand, specificity also plays an important role in learning. For instance, according to Reicher's study in 1969, native English speakers, due to extensive reading in English, can recognize words significantly faster than individual letters or strings that do not conform to phonetic rules. This indicates that prolonged training on specific types of inputs can significantly enhance the efficiency and accuracy of processing these inputs.\n\nOverall, our model aims to simulate these learning dynamics observed in real-world scenarios. It emphasizes the interaction between feature-based and task-based learning, highlighting their joint effects on perceptual recognition and task execution capabilities. Both learning are valuable for the brain to adapt to and interact with complex environments.\n\n**Q7:** Yes, we set a much higher learning rate for task-based learning, resulting in the network parameters related to task-based learning being updated much more quickly, which allows the model to rapidly adapt to the task replying on the existing feature representations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oItx3kg5mv", "reviewer": "Reviewer_n6Xq", "summary": "1. The paper proposes a dual-learning model to reconcile two seemingly contradictory phenomena in perceptual learning: specificity and transfer.\n2. The model consists of two learning processes:\n   - Task-based learning: Fast, enables quick adaptation to new tasks using existing neural representations.\n   - Feature-based learning: Slow, refines neural representations to reflect changes in the environment.\n3. The model is implemented as a hierarchical neural network with three stages:\n   - Feature extraction\n   - Feature-based learning\n   - Task-based learning\n4. The interactions between these two learning processes explain the observed specificity and transfer effects in perceptual learning experiments:\n   - Specificity occurs when feature-based learning dominates (due to excessive training on the same stimulus).\n   - Transfer occurs when task-based learning dominates (due to varied training conditions).\n5. The model successfully reproduces key experimental findings in perceptual learning, including:\n   - Specificity in condition-specific training\n   - Transfer in varied training conditions\n   - Transition from transfer to specificity with increased training sessions\n   - Transfer effects in double training paradigms", "questions": "Please ablate the model. \n1) Is the first hand designed feature extractor necessary? What happens you mess this part up?\n2) What happens if you train  the middle part with backprop?\n3) What happens if you train always the first and the last part with backprop simultaneously. \n\nGenerally I think the proposed model seems quite ad how and potentially makes intuitive sense. Please back up design choices with experimental results. Show me when model behaviors start misalignment with the expected findings of perceptual learning.", "limitations": "Seems ok to me.", "rating": 5, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper proposes a simple, novel dual-learning model that effectively reconciles the conflicting phenomena of specificity and transfer in perceptual learning. With the help of this model, the authors successfully reproduces classical findings from perceptual learning experiments. The paper is easy to follow and experiments seem sound.", "weaknesses": "I am not a computational neuroscientist and only follow this field very sparsely. I found the paper interesting but can not judge the novelty of the paper and its methodology and results. Nevertheless, I would argue that the experimental section is missing ablations or theoretical results to explain the (many) moving bits and pieces , and therefore authors design choices, of the proposed model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the valuable comments of the reviewer. \n We streamline reviewers' main concerns and address them one-by-one.\n\n**Q1:**  On the removal of the feature-extraction module.\nThe feature extraction module is akin to a vision representation extractor that has been trained through extensive experiences over time. If we remove or disrupt this module, other parts of the model will not be able to operate effectively, rendering the entire model ineffective.\n\n**Q2:**  On the choice of Hebbian learning or backpropagation. \nIn the dual-learning framework, feature-based learning \naims to capture the distribution variation of external stimuli, which is task-independent and requires a significant number of observations. Therefore, we used slow and unsupervised Hebbian learning\nto achieve this goal. \nIn contrast, task-based learning is directly linked to task execution using the existing representations, and it can be quickly achieved by modifying the read-out weights. We therefore chose the fast and supervised BP to implement this part. Overall, the choice of the learning method is based on the biological plausibility.\n\n**Q3:**  The newly added ablation studies.\nWe add two ablation studies to highlight the importance of the relative speeds of two learning processes.\n\n**1. Accelerating feature-based learning.** As shown in Fig.1 of the attached PDF, we increase the learning rate of feature-based learning by tenfold and replicate the four experiments in the paper. The results are: \n\n- **In  Exp1**, the results do not differ significantly from the original model, but the accelerated feature learning significantly modifies feature representations, leading to significantly degraded performances at untrained locations (ori1\\_loc2). \n- **In Exp2**, due to continuously changing experimental conditions and the accelerated feature learning rate, the representations are constantly altered, preventing the model from performing well in the learning task and failing to achieve human-like transfer. \n- **In Exp3**, the transferability of learning in new conditions diminishes with increasing training numbers, but with significant changes in feature-based learning, the differences in learning curves under new conditions are no longer distinct and become entangled, showing a decreasing trend in learning gain (except for T12), with increased error bars.  However, when these differences are statistically analyzed using a T-test compared to the results in the main text (as detailed in the reply to the Reviewer Wypk), the p-values have increased, suggesting reduced statistical significance between conditions.\n  |     | T2    | T4    | T8    | T12   |\n  |-----|-------|-------|-------|-------|\n  | T2  | 1   | 0.049| 0.00088| 0.12 |\n  | T4  | 0.049  | 1      | 0.29     |0.5|\n  | T8  |0.00088  | 0.29   | 1 |0.053|\n  | T12 | 0.12 | 0.5 | 0.053       |   1|\n\nTable 1: Statistical comparison of Exp3 with accelerating feature-based learning\n\n- **In Exp4**, double training no longer achieves the transfer of learning, and performances in both transfer conditions are even worse after double training.\n\n**2. Slowing down task-based learning:** As depicted in Fig.2 of the attached PDF, we decreased the learning rate of task-based learning by tenfold. As expected, this adjustment resulted in a degradation of the learning effect, rendering the model less capable of mastering the tasks efficiently.\n\n- **In Exp1**, although the model cannot fully master the current task, a comparison with Exp1 in Fig.1 reveals that there is no significant difference in the effects of training on other conditions.\n  \n- **In Exp2**, especially under random conditions, it is observed that despite the model's inability to master the current task effectively, the improvements brought about by learning still transfer to new conditions.\n  \n- **In Exp3**, due to the model's inability to effectively master the current task, the learning curves for all four different conditions are almost entangled. Significance testing shows that p-values have increased, indicating smaller differences between different training conditions:\n  \n  |     | T2    | T4    | T8    | T12   |\n  |-----|-------|-------|-------|-------|\n  | T2  | 1     | 0.68  | 0.089 | 0.94  |\n  | T4  | 0.68  | 1     | 0.02  | 0.76  |\n  | T8  | 0.089 | 0.02  | 1     | 0.095 |\n  | T12 | 0.94  | 0.76  | 0.095 | 1     |\nTable 2: Statistical comparison of Exp3 with slowing down task-based learning\n\n- **In Exp4**, due to poor learning outcomes, double training did not facilitate the transfer of learning. However, when compared with Exp4 in Fig.1, it can be seen that there is minimal disruption to transfer conditions.\n\nDue to time constraints, the above ablation study results are only statistical outcomes from 20 simulations. We plan to conduct 100 experimental simulations consistent with the main text and will perform a more detailed analysis. However, we do not expect significant deviations from the current ablation study results. \n\nThese two sets of ablation studies demonstrate that both slow feature-based learning and fast task-based learning are necessary for our model to reproduce perceptual learning phenomena.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "28ogtxpFFz", "reviewer": "Reviewer_Wypk", "summary": "In this article, the authors propose a novel model that accounts for two different phenomena observed in human learning: i) specificity, a feature-based mechanism restricted to the very specific statistics of the environment condition, and ii) transfer, a task-based mechanism that allows to transfer knowledge to untrained locations or features. The model proposed by the author is made of a stack of 3 components: A feature extraction stage (mimicking LGN), a Feature-based learning stage (mimicking early processing), and a task-based learning (mimicking upper-processing). Training this model on a vernier discrimination task, the authors demonstrated that it accounts for numerous psychophysics phenomena: i) specificity in condition-specific training, ii) transfer in various training conditions, and ii) transition from transfer to specificity when trained on an increased number of training sessions.", "questions": "1 - In my understanding, feature-based is supposed to account for selectivity, which is thought to happen when over-training is taking place (i.e. when stimuli are relatively similar, and there that the environment is relatively unchanged). But it seems to contradict this sentence (l75-77): ‘Feature-based learning … only takes place when the external environment is substantially changed ». On my understanding if the environment is « substantially changed » then we are not in the over-training condition... Am I missing something? Could you explain?\n\n2 - In Figure 2A, this is not clear what loc(x) corresponds to ? Is it encoded as the x,y coordinate of the stimulus? Please give more detail.\n\n3 - In Appendix 1 (Eq.A4), you suggest a=0, which involves two divisions by zero (one in the exponential multiplier, and another inside the exponential). Is that a typo or is that a=0? If this is the case how do you handle the division by zero? \n\n4 - What is the grey line in Figure 3?\n\n5 - In Fig 3.C. How do you explain that the accuracy for the training sample is decreasing with only feature-based learning? I would expect to see this accuracy improving if the feature-based learning model well the specificity phenomena as we observe an increase in performance for human data (Fig 1Aii ).\n\n6 - I am not sure the claim line 173 is true: « feature based learning… reinforces the model’s performance at the trained location ». This is the opposite trend that is observed in Fig 3C… Could you discuss this point? \n\n7 - In Fig 3D: There is no mention of the method used to compute the similarity in Fig 3D. Could you elaborate? \n\n8 - Line 182: «  The combination of feature-based and task-based learning accelerates the training process at the train location ». This is not visually obvious when we compare Fig 3B and 3E… Could you quantify the difference in convergence speed? Ideally, make sure this difference is statistically significant.\n\n9 - Could you explain a bit more about the difficulty threshold in section 4? And what is the difference between the discrimination threshold and the difficulty threshold? Is that the same? If I understood well the difficulty threshold is the offset between the two Gabor filters (the smaller the offset the harder the task). But in line 191, you introduce the discrimination threshold, which seems to be more related the the intensity of the stimuli than the offset. Could you clarify this point? \n\n10 - Figure 6: Could run a statistical test to make sure the T4 is indeed below T8 (in Fig6b, in the transfer setting? \n\n11 - Not sure what you mean in line 218: In the untrained condition, the threshold first decreases and then increases or saturates… I am not sure to see that in Fig 6B… could please be more accurate here \n\n### Typo : \n* Fig2C : learining —> learning\n* Line 144: Sec. A.1 —> Sec A.2", "limitations": "The author properly discusses the limitations of their paper.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "This article is well written, is well motivated, and leverages simple, yet very informative, experiments that could be easily compared to human data. Overall it was pleasant to read. Great work!", "weaknesses": "I have noted various (rather minor) points that needed to be better clarified/explained (see question section). My only concern is about the generalization of the proposed model to more complex tasks (including for example natural image discrimination). I fully agree that simple tasks like the Vernier discrimination are a very good starting point to propose a novel model accounting for human learning, but scaling such models to more complex data would have drastically improved the impact of this article. Overall I don’t think this scaling-up argument is enough to reject this article, but this is something the authors need to keep in mind if they want their article to reach a more ‘global’ audience. Note that I am willing to increase my rating if the points (in the question section) are properly addressed/clarified.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the careful and valuable comments. \nWe streamline reviewers' main concerns and address them one-by-one.\n\n**Q1:** On the goal of feature-based learning. In this study, we argue that the goal of feature-based learning is to capture the statistical characteristics of external features. Therefore, it only takes effect when there is a significant change in the distribution of external features. In our experiments, when the stimuli are presented repeatedly at the same location for many times, \ntriggering the sense of statistical change of external stimuli by the brain,\nthe weight changes associated with feature-based learning become pronounced, eventually dominating the model's performance and demonstrating specificity. This mechanism highlights how feature-based learning adapts to repetitive patterns, enhancing the model’s ability to specialize in recognizing consistent (salient) features in its environment.\n\n**Q2:** Apologies for the confusion. Indeed, $\\mathbf{x}$ should be a vector indicating the position. Specifically, for the task at hand, $loc(\\mathbf{x}) $represents the coordinates as $\\mathbf{x}=(x,y)$, where $x$ and $y$ denote the horizontal and vertical coordinates, respectively.\n\n**Q3:** Apologies for the confusion. Here, we actually consider the case of limit. Specifically:\n$$\n\\lim_{a \\to 0} W_{t+1}(\\mathbf{x}, \\mathbf{x}^\\prime) = \\frac{A_{t+1}(\\mathbf{x}^\\prime)}{\\sqrt{2 \\pi} a} \\exp \\left[-\\frac{(\\mathbf{x} - \\mathbf{x}^\\prime)^2}{2 a^2}\\right].\n$$\nAs the width $ a $ approaches zero, the Gaussian kernel becomes very sharp, indicating that weight updates are concentrated at $ \\mathbf{x}=\\mathbf{x}^\\prime $. In this limit scenario, significant weight updates occur only when the input and output positions are exactly the same, corresponding to the one-to-one connection. \n\n**Q4 and Q8:** In Fig.3B and Fig.3E, the gray line indicates the number of training epochs required for the model to achieve a 90\\% accuracy rate. This metric serves as the reference point for assessing the convergence speed of the model under different learning conditions.\n\nUnder the task-based learning only condition (as shown in Figure 3B), the model requires approximately 75 training epochs to reach 90\\% accuracy.\nHowever, under the condition of combining task-based and feature-based learning (as shown in Figure 3E), the model converges faster, needing only about 68 training epochs to achieve the same level of accuracy.\n\nA statistical significance was found between these two conditions, with a p-value of approximately 0.00148 (< 0.005) in one hundred simulations.\n\n**Q5-7:** Questions 5 to 7 are closely related. Feature-based learning is independent of task requirements, so it is possible that it may reduce the model's performance. For human subjects, feature-based learning and task-based learning occur simultaneously, thus they cannot be compared separately. This explains why we do not observe enhanced feature processing at the trained locations in Fig.3C. Rather, we should focus on Fig.3D, where we assess similarity by calculating the correlation between the representations before and after feature-based learning. The correlation is calculated as follows:\n$$\ncorr = \\frac{n \\left(\\sum F^*_t F_t \\right) - \\left(\\sum F^*_t \\right)\\left(\\sum F_t \\right)}{\\sqrt{\\left[n \\sum (F^*_t)^2 - \\left(\\sum F^*_t \\right)^2\\right] \\left[n \\sum (F_t)^2 - \\left(\\sum F_t \\right)^2\\right]}}.\n$$\nHere, $ F^*_t $ and $ F_t $ represent the representations before and after the feature-based learning module, respectively, and $ n $ is the total number of representations. \n\n**Q9:** In Section 4, all our model simulations utilize difficulty thresholds. Specifically, we adjusted the offset between two vertical Gabor filters in the task to create a series of difficulty levels, and we selected the difficulty level that corresponds to 80\\% accuracy of the model as the threshold (mentioned in line 193 and Appendix C2). The \"discrimination threshold\" introduced in line 191, however, refers to the threshold used in psychophysical experiments to assess learning performance. \nAlthough these two types of thresholds differ, their roles are similar: they are both used to measure the limit at which the model achieves a certain performance under specific conditions.\n\n**Q10:** The results of the t-tests conducted on the data from Fig.6 are presented below, highlighting the comparison between T4 and T8. As observed, the statistical differences between these two conditions are not significant. This finding aligns with the results depicted in Fig.1C(ii), indicating the consistency across different parts of the study.\n\n|     | T2                 | T4                 | T8                 |T12                 |\n|-----|-------------------|-------------------|--------------------|--------------------|\n| **T2** | 1                 | 0.025  | 4.1e-05            | 3e-07 |\n| **T4** | 0.025             | 1      | 0.075              |0.0018|\n| **T8** | 4.1e-05           | 0.075  | 1                  |0.11|\n| **T12** | 3e-07             | 0.0018 | 0.11               |1|\n\nTable 1: Statistical comparison of Fig.6\n\n**Q11:** We revised the description as follows: \"The results are presented in Fig.6B, which show that the thresholds decrease gradually in the trained condition (left panel in Fig.6B). In conjunction with Fig.3E, it can be observed that in the untrained condition, the threshold should first decrease and then either increase or saturate. This indicates that with increased training, there is a transition from transfer to specificity at untrained locations.\" This describes how training intensity influences learning dynamics differently in trained versus untrained locations.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bNNKLZMrqo", "reviewer": "Reviewer_VB2J", "summary": "This paper aims to replicate a variety of results from the perceptual learning literature using a model with two different forms of learning. It shows results that capture how specificity occurs under certain training conditions and transfer occurs under others. The two forms of learning are different in terms of supervision and location of weight changes.", "questions": "Is Eqn 2 actually a convolution? It seems like each spatial location has its own separate weight that the activity is multiplied by so I don't see what about it is convolutional. (Also it isn't necessary to include Eqn A4 if the only scenario used is $\\alpha=0$). \nWhat values are $A_{t=0}$ initialized as?\n\nThe main text makes reference to the task-based network as being a convolutional layer, but it seems like it is actually 3. Also, are there nonlinearities between this layers?\n\nFor the third experiment the authors say \"At the untrained condition, the threshold first decreases and then increases or saturates, indicating a transition from transfer to specificity\". I'm not sure where the increase or saturation is seen. The data is a little noisy but still looks like a continued decrease on average.\n\nI am confused about how the locations of the gabor filters are spread out across the image. Are they overlapping or no? if I just take the width and divide by 40 (as suggested by line 406), the filters would be 20 pixels wide but the standard deviation is 30 so that can't be right. Is there a stride that's missing? \n\nIn fig 7, isn't the relative lack of transfer to ori1_loc2 inconsistent with the experimental results? Also 7b makes it look like there is improvement for ori1_loc2 but 7c doesn't. Also it seems the labels are wrong on the right hand side of 7c. \n\nIt is possible the authors broke anonymity on line 105 (no need to confirm or deny, just a reminder to be careful)\n\nThere are typos in line 145 and 148.", "limitations": "The authors should probably speak more to the limitations due to the non-biological components of the model discussed here.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper tackles an interesting topic, one that is ripe for modeling influence. \n\nTechnically they are able to replicate many findings.", "weaknesses": "The modeling choices made in this work made it hard to interpret the results and also made the connection to known biology more difficult.  Also there are some issues in the interpretation of the results the authors provide at times.\n\nSpecifically:\n\nIn the feature-based learning method, the authors tie the weights from different orientations at the same location (this is justified by reference to cortical columns). I don't understand the mechanism that would cause all neurons representing a single location to have the same weight updates. Rather, I would expect there to be orientation-specific learning as orientation is represented differently by different cells. \n\nDifferent task-based networks are trained for different orientations. The authors do not make this very clear in the main text, but it is of course very important for understanding the results. What is this meant to correspond to biologically? (of course the rest of the ventral stream doesn't change based on which orientation is being used.) The authors also refer to this form of learning as fast, but it includes the learning of many more parameters than the feature-based learning and takes many epochs. \n\nThe task-based network also performs convolutions over a space where dimensions represent orientation and space. This means that a downstream cell only gets input from 3 nearby spatial locations and 3 nearby orientations (in the first layer). The spatial specificity is warranted as cells do have spatially restricted receptive fields. But what is the justification for the restricted orientation connections? Furthermore, the fact that the weights are convolutional in this space means that there may actually be orientation transfer in these networks (for the same reason there is location transfer when the task network is trained alone). However this was not tested because different task networks were used for different orientations. The location transfer exhibited by the task-based network also depends on weights being tied (i.e. using a convolution). What is the biological explanation for this feature of the model?\n\nThe Hebbian learning (combined with normalization) seems to really mess up the representations at non-trained locations. This is more extreme than simply not transferring to them. How is this interpreted? \n\nTo help with interpretation, it would be good if the ablation studies could be run on all the experimental results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for the very detailed comments. \nWe streamline reviewers' main concerns and address them one-by-one.\n\n**Q1:** On the biological plausibility of location-specific plasticity. Based on a number of experimental evidence, we believe that location-specific plasticity is possible in certain conditions.\n\n- First, the Double Training experiments (Xiao et al. 2008, Zhang et al. 2010, Wang et al. 2012, Zhang et al. 2014, Wang et al. 2016, and Xiong et al. 2016 ) and those involving attention to induce transfer (Donovan et al. 2015, 2018, 2020) have indicated that the location-specific plasticity is possible. For instance, Wang et al. (2012) showed that \"Double Training\" involving stimuli unrelated to orientation, such as distinguishing between a Gabor stimulus and the letter E, could achieve transfer to untrained locations. Xiong et al. (2016) used two experimental paradigms: one in which subjects were presented stimuli without awareness (bottom-up driven), and the other subjects were informed of the presence of stimuli but no such stimuli actually presented (top-down driven), and in both cases, the transfer effect to untrained locations was observed.\n\n- Second, neuroimaging studies have suggested that the results of perceptual learning can transform the processing of location information mediated by top-down signals into those realized by bottom-up implementations. For instance, fMRI findings from Sigman et al. (2005) showed that during the initial stages of learning a visual search task, there was significant activity in the frontal-parietal network, suggesting a pronounced top-level regulatory role; however, as the learning progressed, the frontal-parietal network activity decreased, while the activity in the lower visual cortex increased, \n\nTaken together, we argue that neurons with different orientation preferences can achieve location-specific plasticity through top-down signals (such as attention). The columnar structure can be seen as a way to support position-specific learning.\n\n**Q2:** On the use of different read-out networks. In our simplified model, we only extract features related to angle and position, necessitating the use of different task-based networks to read out different orientation information. We agree that in real biological systems having much more complicated structures, a single network module can process different orientation information. Nevertheless, since the focus of this study is on elucidating the dual-learning framework, in particular, the effect of different speeds of the two learning processes, we could extend the current model to have a single task-based learning module to handle different orientations, which will not change our results qualitatively.  \n\n**Q3:** On the use of CNN. The CNN in our model is solely for simulating the task-based learning process with the capability of location generalization. Indeed, there may also be some degree of orientation transfer. Since our study focuses on location specificity, we employed different task-based networks for different orientation discrimination to avoid this interference. It ensures that the learning effect is specific to the location without confusion with orientation changes.\n\n**Q4:** Indeed, the combination of Hebbian learning with normalization can induce distorted feature representations at untrained locations and subsequently cause degraded learning effects at untrained locations. However, this phenomenon is not biologically implausible.\nIn real human subject experiments as cited in Fig 1C, it shows that repeated training at specific locations can indeed cause a decrease in subjects' performances at untrained locations.\n\n**Q5:** Equation 2 presents a general learning rule. In practice, for simplicity, we simplified the operation. As shown in Equations A3 and A4, we considered \n$$\n\\lim_{a\\to0} W_{t+1}(\\mathbf{x}, \\mathbf{x}^\\prime) = \\frac{A_{t+1}(\\mathbf{x}^\\prime)}{\\sqrt{2 \\pi} a} \\exp \\left[-\\frac{(\\mathbf{x} - \\mathbf{x}^\\prime)^2}{2 a^2}\\right],\n$$\nwhich shows that feature-based learning employs a Gaussian kernel to perform weighted convolution on representations. For simplicity, we considered the scenario where parameter $a$ approaches zero. In this case, significant weight updates occur only when the input and output positions are exactly the same, effectively creating the one-to-one connection between neurons at identical locations.\n\n**Q6:** See reply to Q4 of the Reviewer UGym.\n\n**Q7:** See reply to Q11 of the Reviewer Wypk.\n\n**Q8:** Apologies for the typo. It should be \"divide by 10\".  \n\n**Q9:** Apologies for the confusion, indeed the labeling of \"loc1\\_ori2\" and \"loc2\\_ori1\" in Fig.7b were reversed. After correcting the labeling, it can be seen that our results are consistent with the experimental results in Xiao et al. 2008 (Fig.3). \n\nAfter re-evaluating the improvement results shown in Fig.7b and Fig.7c, we find that the inconsistency arises due to that Fig.7b presented the average of 100 simulation runs, while the improvement in Fig.7c was calculated by averaging the rate of improvement from each simulation, where a few outliers affected the results. After excluding 4 outliers, the results are consistent, see Fig.3 in the attached PDF. \n\n**Q10:** Thanks for the reminder.\n\n**Q11:** Typos will be corrected.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We acknowledge the very careful and valuable comments of all reviewers. We realize that there are common concerns about the aim of this study and the models we used to demonstrate the framework. In the below, we briefly summarize the motivation and main results of this work to clarify these concerns.  

Overall, in this work, we explored the neural mechanisms underlying perceptual learning and proposed a dual-learning framework in which the interplay between **the rapid, supervised, task-based learning** and **the slow, unsupervised, feature-based learning** generates the rich phenomena of perceptual learning. Our model reconciles the seemingly conflicting phenomena of specificity and transfer observed in diverse experiments. 
To our knowledge, our work is the first one that uses a computational model to elucidate the dual-learning framework for perceptual learning.

Since our focus is on elucidating the dual-learning framework, we have chosen as simple as possible, and meanwhile, as biologically plausible as possible, models to implement each part of learning. Nevertheless, if other models can capture the characteristics of one part of dual-learning, they can be used in our framework.  
- Specifically, in our model, the feature extraction module is dedicated to transforming images into visual features, representing a stable representation system in the brain that rarely changes. To accomplish the visual discrimination task in this work, we employed the classical basis function network. Certainly, if more complicated tasks are considered, more sophisticated pre-trained models can be used. 
- The feature-based learning module aims at learning the variation in the statistical distribution of visual inputs. It takes effect only when significant changes have occurred in the distribution of visual inputs, and hence it is relatively slow and goes in an unsupervised manner. We employed the classical Hebbian learning with a slow learning rate to implement feature-based learning. 
- The task-based learning module is driven by the task at hand, aiming to extract task-relevant information from existing representations to accomplish the task rapidly. Hence, we used a CNN model with supervised learning (such as backpropagation) having a relatively large learning rate to implement this learning process.

To highlight the importance of the interplay between
the rapid task-based learning and the slow feature-based learning, we have carried out an additional ablation study (see the attached pdf Fig.1 and Fig.2),
which varies the learning rates of two learning processes, and demonstrates that the relative speed ratio between two learning processes is essential for perceptual learning.

Meta-review: In this study, the authors proposed a dual-learning framework in which the interplay between the rapid, supervised, task-based learning and the slow, unsupervised, feature-based learning to explain a wide range of phenomena in perceptual learning. Although the reviewers had several concerns about the motivation/model design/Hebbian learning mechanisms, the rebuttal successfully convinced the reviewers. I am inclined to accept the paper.
Decision: Accept (poster)