# Representing Deep Neural Networks Latent Space Geometries with Graphs
## 3 Methodology
### Deep Learning  
We start by introducing basic deep learning (DL) concepts, referring the reader to Reference [3] for a more in-depth overview. A DL architecture is an assembly of layers that can be mathematically described as a function \(f\), often referred to as the \"network function\" in the literature, that associates an input tensor \(\mathbf{x}\) with an output tensor \(\mathbf{\hat{y}}=f(\mathbf{x})\). This function is characterized by a large number of trainable parameters \(\theta\). In the literature, many different approaches have been proposed to assemble layers to obtain such network functions [17]. While layers are the basic unit, it is also common to describe architectures in terms of a series of _blocks_, where a block is typically a small set of connected layers. This block representation allows us to encapsulate non-sequential behaviors, such as the residual connections of residual networks (Resnets) [17], so that, even though layers areconnected in a more complex way, the blocks remain sequential, and the network function can be represented as a series of cascading operations:  
\[f=f^{t_{\max}}\circ f^{t_{\max}-1}\circ\cdots\circ f^{t}\circ\cdots\circ f^{1}, \tag{1}\]  
where each function \(f^{t}\) can represent a layer, or a block comprising several layers, depending on the underlying DL architecture. Thus, each block is associated with a subfunction \(f^{t}\). For example, in the context of Resnets [17], the architecture is composed of blocks as depicted in Figure 1.  
A very important concept for the remainder of this work is that of intermediate representations, which are the basis for the LGGs (defined in Section 3.3) and corresponding applications (Section 4). **Definition 1** (Intermediate representation). : _We call intermediate representation of an input \(\mathbf{x}\) the output it generates at an intermediate layer or block. Starting from Equation (1), and denoting \(F^{\ell}=f^{\ell}\circ\cdots\circ f^{1}\), we define the intermediate representation at depth \(\ell\) for \(\mathbf{x}\) as \(\mathbf{x}^{\ell}\triangleq F^{\ell}(\mathbf{x})\). Or, said otherwise, \(\mathbf{x}^{\ell}\) is the representation of \(\mathbf{x}\) in the latent space at depth \(\ell\)._  
Initially, the parameters \(\theta\) of \(f\) are typically drawn at random. They are then optimized during the training phase so that \(f\) achieves a desirable performance for the problem under  
Figure 1: Simplified depiction of a residual network (Resnet) with eight residual blocks. consideration. The dimension of the output of \(f\) depends on the task. In the context of classification, it is common to design the network function such that the output has a dimension equal to the number of classes in the classification problem. In this case, for a given input, each coordinate of this final layer output is used as an estimate of the likelihood that the input belongs to the corresponding class. A network function correctly maps an input to its class if the output of the network function, \(\hat{\boldsymbol{y}}\), is close to the target vector of the correct class \(\boldsymbol{y}\). **Definition 2** (Target vector). : _Each sample of the training set is associated with a target vector of dimension \(C\), where \(C\) is the total number of classes. Thus, the target vector of a sample of class \(c\) is the binary vector containing 1 at coordinate \(c\) and 0 at all other coordinates._  
In this work, we also introduce the notion of label indicator vector, which it is important to differentiate from that of target vector. The label indicator vector is defined on a batch of data points, instead of individually for each sample, as follows:  
**Definition 3** (Label indicator vector). : _Consider a batch of \(B\) data points.