When the coefficients are 0, the resulting cost for l1 and l2 weight dec...

They could return a float64 instead of a float32, when floatX=float32.
This is dangerous. If someone provided somothing with a different dtype, it could downcast/change from int to float32. I think you should change the loop at line 695, and remote 0. from the layer_costs. That way, it can't cause upcast. Then, you will need to return theano.tensor.constant(0, dtype=floatX) if layer_costs is empty.