In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long shortterm memory networks (LSTMs) at the cellular level. We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods. The ranked contribution of individual cells to the network's output is computed by analyzing a set of interpretable metrics of their decoupled step and sinusoidal responses. As a result, our method is able to uniquely identify neurons with insightful dynamics, quantify relationships between dynamical properties and test accuracy through ablation analysis, and interpret the impact of network capacity on a network's dynamical distribution. Finally, we demonstrate the generalizability and scalability of our method by evaluating a series of different benchmark sequential datasets.
Hasani et al. REF ranks the contribution of individual cells to the final output to help understand LSTM hidden state dynamics.
Response Characterization for Auditing Cell Dynamics in Long Short-term Memory Networks
2019 International Joint Conference on Neural Networks (IJCNN)
2019 International Joint Conference on Neural Networks (IJCNN)
Computer Science Mathematics