Started implementing unsupervised layer-wise pre-training.
Implemented a kind of unsupervised pre-training algorithm, to poor results. Various improvement and bug fixes, found a "deep" architecture that I can train efficiently with momentum. Seems 3x3 convolutions are tricky to train somehow. Notably, switched to ReLUs + random gaussian initialization for FC layers.
Implemented decaying learning rate schedule with validation set.