Question: How do they evaluate interpretability in this paper?
Answer: we suggest to estimate the interpretability of $k$ th component as $ \operatorname{interp}_k W = \sum _{i,j=1}^N W_{i,k} W_{j,k} \left(W_i \cdot W_j \right). $