Different Reward Processing in the Primate Prefrontal Cortex and Striatum Using a Reward Inference Task

category learning task. The researchers found in the late learning stage (category-performing stage), striatal neurons couldn’t represent category information whereas prefrontal neurons did14. Besides, it’s proved to be possible that hippocampus could concatenate 2 elements, which were not actually linked to each other (e.g., A-C), based on prior knowledge that both of the 2 elements had been linked with another element before (e.g., A-B, B-C)96,115. In Shohamy’s fMRI experiments, participants first learned that, e.g. stimulus X1 is paired with X2, stimulus Y1 is paired with Y2, then participants learned X1 is paired with a reward and Y1 is paired with nothing. At last participants were asked to choose between X2 and Y2, despite the fact that X2 and Y2 had never been paired with reward or non-reward directly, participants showed their preference to X2. The association between X2 and reward emerged in the learning phases rather than testing phases, and hippocampus supported the dynamic integration of overlapping stimuli (X1 and reward were both linked with X2). However, different with the prefrontal neurons, neurons in the hippocampus could predict C after seeing A without any direct experience that had linked A to C, however, no categorical information was found to be encoded in the hippocampus. Albeit our task was a transitive inference task and the monkeys could predict the reward by a deductive approach that always recall the successive corresponding stimulus after seeing each cue (e.g., recall B1 after seeing N1, then recur C1 after recalling B1, and definitely come up with the information that C1 is LR/SR), the emergence of categorical information of the stimuli represented by stimulus-reward neurons provided the monkeys an alternative inductive approach to complete the task, i.e., generalized an abstract rule that all the stimuli A1, B1, C1 and N1 belonged to C1 group/red group by repeatedly observing the relationships between those stimuli. Then after seeing the 1st cue (e.g., N1) the monkeys could recall that N1 was an element of C1 group, further by recalling the prior knowledge that C1 group was LR/SR, the monkeys would predict the reward outcome of the current trial. It’s reasonable to consider such categorical information made it possible to perform transitive inference in the LPFC rather than in the striatum. Stimuli in the same group were bound together according to their functional equivalence119, and were encoded by the neurons in the LPFC which could predict/infer reward outcomes as well. Pigeons were found to be able to use categorical information to guide their behavior. Vaughan trained pigeons to discriminate 2 photographic slides sets (Set A and Set B) consisted of photos of trees105. Pigeons at first were trained to response to Set A rather than Set B, later, the response contingency reversed and then revered again repeatedly. If the pigeons got familiar with the task, the first 4~5 trials were enough for the pigeons to response correctly afterwards when the stimulus-response contingency was shifted. They could pass the feature (response or non-response) of some stimuli to the rest of the stimuli in the same category without experiencing all the stimuli in Set A and B by trial and error. Categorical information emerges from experiences and can in turn guide behavior when subjects are confronted with new circumstances in the form of inferences. Inferences don’t need to employ categorical information, but categorical information may facilitate inferences. Category is an efficient and flexible way to store diverse information. Features of one category can be extended when new members containing new features are added to that category. Soon old members carry forward those new features. Exchanging features between new and old members is a process of inferences, and the results of inferences don’t need always to be true. Category can emerge from functional equivalence but don’t restrict equivalence to members which makes categorical representation flexible. In fact, individual neuron in the PFC can represent multiple categories (e.g. animals and cars) 14. In deep learning neural networks of a language model, the learned word vectors for Tuesday and Wednesday were found to be similar to the word vectors for Sweden and Norway59. That means both a node in an artificial neural network and a real neuron in a biological neural network can represent different categories. Extension to new features or new categories helps animals to generate new information in their minds. So far, we know basal ganglia can encode value information of stimuli which could be used by subjects to infer reward outcomes of a stimulus. Specifically, different parts of the striatum can encode different types of value, such as flexible and stable values37. Disjunctive inference processes found in the striatum is intriguing compared with other recent studies. Value-based reward prediction is a common paradigm in investigating functions of basal ganglia. The basal ganglia can’t master a complicated model of the environment, e.g. categorical information of the stimuli, however it can grasp some basic rules of our experimental design (if one group of stimuli is linked to the larger reward, then the other must be linked to the smaller reward). The striatum does not need dopaminergic prediction error to learn the reward outcomes, rather it can provide predictive information to dopaminergic neurons. In an experiment similar to our disjunctive inference paradigm, if a reward outcome was inferable, despite the truth that a stimulus was linked to a large/small reward for the first time, dopamine neurons responded less astonished98. Hence, merely focusing on value representation of the striatum is not enough, in the future we have to dig deep into the mechanism that how the striatum learns the structures of complex external environment without prediction error signals. Although, anatomically there are dense connections from the LPFC to the striatum, it doesn’t seem the LPFC have sent all its information to the striatum, otherwise, the caudate or the putamen could have predicted the reward in the very first trial with new stimulus. The results suggest that the reward information is not processed first in one area and then send to the others. Since the putamen needed more than one trial to learn the new stimulus-reward contingency compared to the caudate, these three areas might have processed reward information independently. Further investigation is needed to clarify what kind of information is transferred among these areas. Prefrontal BS and NS reward neurons LPFC neurons were classified into the BS and NS groups by k-means method based on their shapes of averaged spike waveforms. The observed differences in physiological responses between the two classes of neurons indicate that they resembled putative pyramidal cells and interneurons. First, the distributions of waveform durations were bimodal (see Fig12. F-I22). The NS neurons had shorter durations, and the BS neurons had longer durations. Intracellular and whole-cell recording studies with histological methods have demonstrated that pyramidal cells have broader waveform durations than do interneurons61,68,10,28. Therefore, most of the BS neurons in current study should be pyramidal cells and most of NS neurons should be interneurons. Second, the NS neurons showed significantly higher firing rates (see Fig13. A-C22), which is a typical characteristic of interneurons10,28,13. Intracellular studies have found that interneurons have higher firing rates than pyramidal cells when stimulated by current injection or visual stimuli62. The mean baseline activity of BS neurons (5.4 Hz) was also in line with the typical mean rate of 4-8Hz for pyramidal cells97,114. Third, intracellular and anatomical studies have estimated that roughly 70-80% of cortical neurons are pyramidal cells and that the remaining neurons are interneurons10,9,49,50. Consistent with these findings, we found that 77.7% of our sample neurons were BS neurons, and 22.3% of them were NS neurons. Fourth, the NS neurons showed shorter visually evoked latencies than did the BS neurons. Intracellular experiments have found that interneurons have significantly larger and faster EPSPs than pyramidal cells79. This may cause a lower threshold for spike generation and shorter latencies for EPSP-spike coupling in interneurons. Fifth, the BS neurons had significantly higher local variation values than did the NS neurons, indicating that the BS neurons showed a relatively burst-like pattern95. In addition, the BS neurons showed longer peak mode time in ISI distributions than did the NS neurons. These spike train statistical differences between the two classes of neurons were in agreement with statistical observations in pyramidal cells and interneurons79,57,69. Overall, the distinct properties in spike waveforms, response patterns and spike trains between the two classes of neurons suggested that our sample of LPFC neurons were classified into putative pyramidal cells and putative interneurons correctly based on their spike waveforms. Both the BS and NS Reward neurons discriminated the preferred from non-preferred reward conditions (see Fig 14A and B22). Their population activities were stronger in the preferred than non-preferred reward conditions. To determine whether a neuron encodes reward information or not, we usually compare the neuronal activity on trials containing different rewards. If one neuron shows activity differences on different rewarded trials, we say this neuron encodes reward information; otherwise, this neuron does not encode reward information. This type of analysis has been often used in the literature of reward experiments75,54,76, but it does not tell us what type of reward information the reward neuron encodes. To further investigate this issue, the activity of a reward neuron is compared in different time epochs