Before revision:  Through the lens of attention, we analyze the inner workings of the Transformer and explore how the model discerns structural and functional properties of proteins .
After revision:  In this work, we demonstrate a set of methods for analyzing protein Transformer models through the lens of attention .