You made me realize that self.rng is set up for MLPs, but not layers.

As for reproducability, it isn't a problem, since like now the top MLP is determining the theano_rng (like before, and is seeded) and pass it to layers.

What do you suggest for the theano_rng parameter.
Do you think it's ok that it must be not None for bottom layers, but can be None for MLPs (and then the top MLP use the same method that before to create the theano_rng).