rasmusbergpalm/DeepLearnToolbox — pull request #4 by carlcotner (2012-03-29 15:12:24+00:00)
Title: Experimental

Hi Rasmus,

I probably didn't quite do this optimally, but I wanted to get it out anyway. Feedback desired. I know that the nnff2.m and nnbp2.m files need to be folded back in, and I will do that once you've looked at everything.

nninit.m is the proposed replacement for nnsetup.m. It has two advantages: (1) the struct nn doesn't have to be pre-defined; (2) nnsetup() is not idempotent: nnsetup(nn, x, y) != nnsetup(nnsetup(nn, x, y), x, y). (This can come up like this at the command line: nn = nnsetup(nn, x, y); [buch of work]; nn = nnsetup(nn, x, y); % want to reinitialize; % nn is now different!)

nnexamples.m still needs to be cleaned up one way or another.

I have the feeling that the optimization abstraction can still be made one level simpler, but I haven't figured that out yet. Sometimes it takes other examples (or someone else's eyes) to see the whole thing. Passing in nn to nnJ.m seems to be the easiest way to let nnJ know about the shape of the neural net, even though it is also feels slightly redundant. nnunpackWb.m and nnrepackWb.m follow the technique from Ng's class.

A small pretense of added option passing included in some places.

There are a few other small consistency issues that I will take care of (and don't quite remember at the moment).

Oh, also perhaps fmincg.m should be removed; it doesn't seem to help training.

Carl