Dual Rectified Linear Units (DReLUs): A replacement for tanh activation functions in Quasi-Recurrent Neural Networks
SUBWORD LANGUAGE MODELING WITH NEURAL NETWORKS