Great post, and thank you for the link to Hinton's coursera page - I didn't know about that. I also hope to learn a thing or two from your github code. But it was so depressing to read this:

> Now, when I say Artificial Intelligence I’m really only referring to Neural Networks. There are many other kinds of A.I. out there (e.g. Expert Systems, Classifiers and the like) but none of those store information like our brain does (between connections across billions of neurons).

This is a middle-brow dismissal of almost the entire field of A.I. because it does not meet an unnecessarily narrow restriction. (Which, by the way, neural nets don't either. Real neurons are analog-in, digital-out, stochastic processes with behavior influenced by neural chemistry and with physical interconnectivity and timing among other things not accurately modeled at all by any neural net. It's closer modeling to the mechanisms of the brain, but far from equivalent and as a CogSci student you should know that.)

A.I. is the science of building artifacts exhibiting intelligent behavior, intelligence being loosely defined as what human minds do. But in theory and in practice, what human minds do is not the same thing as how they do it.

The human mind does appear to be a pattern matching engine, with components that might indeed be well described as a hidden Markov model or restricted Boltzmann machine. It may be that our brains are nothing more than an amalgamation of some 300 million or so interconnected hidden Markov models. That's Ray Kurzweil's view in How to Create a Mind, at any rate.

However it is a logical fallacy to infer that neural nets are the only or even the best mechanism for implementing all aspects of human-level intelligence. It's merely the first thing evolution was able to come up with through trial and error.

Take the classical opposite of neural nets, for example: symbolic logic. If given a suitable base of facts to work from and appropriate goals, a theorem prover on your cell phone could derive all mathematics known up to the early 20th century (and perhaps beyond), without the possibility of making a single mistake. And do it on a fraction of the energy you spend splitting a bill and calculating tip. A theorem prover alone does not solve the problem of initial learning of ontologies or reasoning about uncertainty in a partially observable and even sometimes inconsistent world. But analyzing memories and perception for new knowledge is a large part of what human minds do (consciously, at least), and if you have a better tool, why not use it?

Now I myself am enamored by Hilton-like RBM nets. This sort of unassisted deep-learning is probably a cornerstone in creating a general process for extracting any information from any environment, a central task of artificial general intelligence. However compared with specialized alternatives, neural nets are hideously inefficient for many things. Doesn't it make sense then to use an amalgam of specialized techniques when applicable, and fall back on neural nets for unstructured learning and other non-specialized tasks? Indeed this integrative approach is taken by OpenCog, although they plan to use DeSTIN deep-learning instead of Hilton-esque RBM's, in part because the output of DeSTIN is supposedly more easily stored in their knowledge base and parsed by their symbolic logic engine.