An excellent summary. I believe that a lot of the work we're doing on NN classifiers is based on a rather shaky notion of bootstrapping a collection of platonic ideals, one at a time.

Lately I've been thinking about this from the opposite perspective, as I have a lot of cats and a large and rather neurotic German Shepherd (all rescue animals that basically came up to us). I mention the dog being neurotic because he went through a phase of reacting to anything in his size class, such as rocking horses and small goods trolleys, and is still inclined to react to things like his own shadow or reflection in a window if it catches him unawares. He's very interested in the cats but given his size and volume they don't appreciate his attentions and I worry more about him catching a stray claw in the eye than I do about him biting one of them, so currently they have to admire each other from a distance. I spend a lot of time with the dog every day (partly because my wife isn't able to handle him physically) and we also look after and train a neighbor's Shepherd pups.

Not being a dog person I've had to spend a lot of time learning how to think like a dog, or model the dogs' model of the world, and make gradual modifications to it, often by subverting existing behaviors. ISTM that the trouble with back-propagation algorithms in NN classifiers is that we are establishing and then polishing a recognition reflex, and at best this is going to yield only insect-like behavior. Now such hardwired behavior in insects can be pretty elaborate and impressive, more so in social insects; but it's an inherently reactive model. Consider instead a neural network with two outputs, GET and IDENTIFY. The IDENTIFY part is more or less the same as what we have now; when stimulated it is trained to classify input as DOG or CAT, and if it does so correctly we 'reward' it by propagating those output weightings back into the network.

But I'd like to see (or be made aware of) one that also tries to GET input, either input it has already classified or new and therefore 'exciting' input. My dogs sometimes want predictable rewards, such as snacks, access to the outside, or various kinds of play (so do my cats but they have different and rather more zero-sum approaches to getting what they're after). But they also need a degree of novelty and mental challenge which requires them to expand their world model. I've been experimenting for a while now with hide-and-seek games using a ball for the oldest dog and (casually) monitoring how he divides his time between iterating over known hiding locations and novel ones, sequences of known locations and so on. Of course he can 'cheat' by using his sense of smell to rack where I've been, but I've managed to train him out of that by using various cheats of my own to make that unreliable.

My takeaway from this is that successful world-modeling is not its own reward but seems to be emergent from a framework of interdependent reflexive behavior, and require a temporal as well as a spatial dimension. Indeed, one of the trickiest aspects of this game was getting the dog to wait for me. Right now I have to show him the ball, then have him stay while I wander away(completely out of his sight) for some indeterminate length of time, following which I return and tell him to start searching, which I may or may not monitor directly. So he has to model both that I'll be coming back and that I've left something in the environment for him to retrieve. He worked this part out quite quickly and it took some extra time to get him to wait for my say-so before doing the retrieval as soon as he sees me reappear.

I feel that for our machine learning progress to advance we need both first-order classifiers of the kind described in this article and second-order classifiers that are orthogonal to the first and classify the first-order classification events into meta-patterns.

I'd be grateful if anyone could point out work already being done in this area. I don't keep up with the academic literature in any sort of organized way so this is probably a case of reinventing the wheel.