Highway State Gating for Recurrent Highway Networks: improving information flow through time
Revisiting Activation Regularization for Language RNNs