A Flexible Online Classifier using Supervised Generative Reconstruction During Recognition

Matching the brain’s flexibility and ability to quickly incorporate new information remains difficult using recognition algorithms. In addition, recognition scenarios in which humans require more time to recognize do not inherently require more time in recognition algorithms. This work proposes a recognition algorithm that uses dynamics in an opposite way from traditional recognition algorithms. Recognition algorithms also know as classifier algorithms function in two phases: a learning phase and a testing phase. Most classifier algorithms utilize feedforward connections during testing. They use dynamic feedforward-feedback signals to learn the feedforward connections during the learning phase. A supervised generative classifier is proposed where the key innovation is that the dynamic feedforward-feedback component is essential during the testing as opposed to the learning phase. This changes the form which learned information is stored. The new form represents the fixed points or solutions of the network. This form allows for more intuitive symbolic-like weighs, more-flexible online learning, and the dynamics involved in the recognition phase emulates cognitive phenomena. Brain-like architecture, capabilities, and limits associated with this model suggest the brain may perform recognition and store information using a similar approach. Introduction Recognition is an essential foundation upon which cognition and intelligence is based. Without recognition the brain cannot interact with the world e.g.: form internal understanding, memory, logic, display creativity, or reason. Subsequently the form of recognition information, as determined through the underlying neuron architectures and connection weights, can greatly affect memory and cognition. For example, whether sound is processed in the frequency or time domain will affect the ease of storing, comparing, and recognizing certain aspects of sound. Analogously certain forms of connections and weights may be more optimal for flexible learning. Many recognition algorithms have been developed over the past half century. For the most part, they share a commonality that 1) they perform classification based on a simple use of feedforward weights during testing and 2) the feedforward weights are determined during learning though dynamic gradient descent methods (using feedforward-feedback methods). Recognition methods can be labeled based on their testing configuration. These classifiers are designated as feedforward. Feedforward methods are known to have difficulties with online learning: updating or changing information independently. Such changes can lead to Catastrophic interference and catastrophic forgetting e.g. (French 1999; McClelland, McNaughton, & O’Reilly 1995). Without retraining old information, information can be quickly lost. Moreover the strategy of batch learning, learning all possible patterns at once, is not scalable for large data sets (Bottou & Le Cun 2004). Although some strategies have been proposed to ameliorate this problem, it is avoided here by modifying the form of information. The key innovation presented here is a supervised recognition algorithm that uses a feedforwardfeedback configuration to implement dynamics that converge through gradient descent during testing. This allows learning to be much simpler using symbolic-like expectations and the weights to represent fixed points. Moreover, changes required for online learning are localized when information is stored using fixed points, thus online problems are avoided. Table 1: Comparisons between feedforward and feedforward-feedback supervised classifiers Structure During Recognition Dynamics During Weights Relations Modification of Fixed-Points Inherent Cognitive Phenomena Feedforward Learning W Sub-Symbolic Distributed No FeedforwardFeedback Testing M Symbolic Localized Yes