October  UTC  As I said above Deep learning is the name of a class of algorithms Feedforward neural nets restricted Boltzmann machines deep neural networks and deep belief networks are all mathematical models How the models are trained that is how model parameters are chosen to optimize some loss function is another topic entirely People had looked at deep neural network architectures before deep learning techniques were developed but it was the deep learning techniquesunsupervised pretraining layer by layerthat made deep networks more practical at the time With todays GPU based algorithms the need for pretraining has lessened The hype surrounding deep learning has muddied the concept to point that people like Michael Jordan claim it is just a synonym for neural network But the backlash from current hype doesnt change the historical importance of deep learning algorithms in generating interest and good results in deep neural network models or representational learning Schmidhubers review has a good discussion of these issues