BVLC/caffe — pull request #717 by qipeng (2014-07-17 18:47:47+00:00)
Title: Lrelu layer

Implemented the Leaky ReLU unit described in this paper

Maas, Andrew L., Awni Y. Hannun, and Andrew Y. Ng. "Rectifier nonlinearities improve neural network acoustic models." ICML Workshop on Deep Learning for Audio, Speech, and Language Processing. 2013.

which shares similar sparse activation properties with the ReLU, but was shown easier to optimize.