Deep neural networks such as GoogLeNet and ResNet have achieved impressive performance in tasks like image  classification. To understand how such performance is achieved, we can probe a trained deep neural network by studying neuron activations, i.e., combinations of neuron firings, at any layer of the network in response to a particular input. With a large number of inputs, we aim to obtain a global view of what neurons detect by studying their activations. We ask the following questions: What is the shape of the activation space ? What is URLanizational principle behind neuron activations ? How are the activations related within a layer and across layers? Applying tools from topological data analysis, we present TopoAct, a visual exploration system to study topological summaries of activation vectors for a single layer as well as the evolution of such summaries across multiple layers . We present exploration scenarios using TopoAct that provide valuable insights towards learned representations of an image classifier . We expect TopoAct to give a topological perspective that enriches the current toolbox of neural network analysis, and to provide a basis for network architecture diagnosis and data anomaly detection.
Deep neural networks such as GoogLeNet , ResNet, and BERT have achieved impressive performance in tasks like image and text classification. To understand how such performance is achieved, we probe a trained deep neural network by studying neuron activations, i.e., combinations of neuron firings, at various layers of the network in response to a particular input. With a large number of inputs, we aim to obtain a global view of what neurons detect by studying their activations. In particular, we develop visualizations that show the shape of the activation space , URLanizational principle behind neuron activations , and the relationships of these activations within a layer . Applying tools from topological data analysis, we present TopoAct, a visual exploration system to study topological summaries of activation vectors . We present exploration scenarios using TopoAct that provide valuable insights into learned representations of neural networks . We expect TopoAct to give a topological perspective that enriches the current toolbox of neural network analysis, and to provide a basis for network architecture diagnosis and data anomaly detection.
R
and ResNet
, ResNet, and BERT
A
and text
D
can
R
any layer
various layers
R
We ask the following questions: What is the
In particular, we develop visualizations that show the
R
? What is
,
R
? How are the activations related
, and the relationships of these activations
R
and across layers?
.
D
for a single layer as well as the evolution of such summaries across multiple layers
R
towards
into
R
an image classifier
neural networks