Deep neural networks such as GoogLeNet and ResNet have achieved superhuman performance in tasks like image classification. To understand how such superior performance is achieved, we can probe a trained deep neural network by studying neuron activations, that is , combinations of neuron firings, at any layer of the network in response to a particular input. With a large set of input images , we aim to obtain a global view of what neurons detect by studying their activations. We ask the following questions: What is the shape of the space of activations? That is , what is URLanizational principle behind neuron activations , and how are the activations related within a layer and across layers? Applying tools from topological data analysis, we present TopoAct, a visual exploration system used to study topological summaries of activation vectors for a single layer as well as the evolution of such summaries across multiple layers. We present visual exploration scenarios using TopoAct that provide valuable insights towards learned representations of an image classifier  .
Deep neural networks such as GoogLeNet and ResNet have achieved impressive performance in tasks like image classification. To understand how such performance is achieved, we can probe a trained deep neural network by studying neuron activations, i. e. , combinations of neuron firings, at any layer of the network in response to a particular input. With a large number of inputs , we aim to obtain a global view of what neurons detect by studying their activations. We ask the following questions: What is the shape of the activation space? What is URLanizational principle behind neuron activations ? How are the activations related within a layer and across layers? Applying tools from topological data analysis, we present TopoAct, a visual exploration system to study topological summaries of activation vectors for a single layer as well as the evolution of such summaries across multiple layers. We present exploration scenarios using TopoAct that provide valuable insights towards learned representations of an image classifier . We expect TopoAct to give a topological perspective that enriches the current toolbox of neural network analysis, and to provide a basis for network architecture diagnosis and data anomaly detection .
R
superhuman
impressive
D
superior
R
that is
i. e.
R
set of input images
number of inputs
R
space of activations? That is , what is
activation space? What is
R
, and how
? How
D
used
D
visual
A
. We expect TopoAct to give a topological perspective that enriches the current toolbox of neural network analysis, and to provide a basis for network architecture diagnosis and data anomaly detection