Before revision:  We also show that gating gives rise to a novel, discontinuous transition to chaos, where the proliferation of critical points is decoupled with the appearance of chaotic dynamics ; the nature of this chaotic state is characterized in detail. Using the DMFT and RMT, we produce phase diagramsfor gated RNN . Finally, we address the gradients by leveraging the adjoint sensitivity framework to develop a DMFT for the gradients.
After revision:  We also show that gating gives rise to a novel, discontinuous transition to chaos, where the proliferation of critical points is decoupled with the appearance of chaotic dynamics (dynamical complexity), contrary to a seminal result for additive RNNs. Critical surfaces and regions of marginal stability in the parameter space are indicated in phase diagrams, thus providing a map for principled parameter choices for ML practitioners . Finally, we address the gradients by leveraging the adjoint sensitivity framework to develop a DMFT for the gradients.