  Pretrained transformers achieve the state of the art across tasks in natural
language processing, motivating researchers to investigate their inner
mechanisms. One common direction is to understand what features are important
for prediction. In this paper, we apply information bottlenecks to analyze the
attribution of each feature for prediction on a black-box model. We use BERT as
the example and evaluate our approach both quantitatively and qualitatively. We
show the effectiveness of our method in terms of attribution and the ability to
provide insight into how information flows through layers. We demonstrate that
our technique outperforms two competitive methods in degradation tests on four
datasets. Code is available at https://github.com/bazingagin/IBA.
