## 3 The Proposed Approach
### Latent Graph Attention (LGA)  
LGA is a computationally inexpensive graph-based attention network designed to improve the contextual information for any given image. It spatially constructs chained attention through the use of multiple layers of the graph, thereby taking into account the information of the entire route when computing the relation between two distant pixels in the latent space. LGA directly operates on the latent feature map and produces additional attention maps, which can directly be concatenated with the original feature map. Due to its simplistic nature, LGA can be incorporated with minimal effort into existing architectures. Figure 2 shows how LGA can be plugged into some popular yet challenging image-to-image translation tasks. **Architecture of LGA. ** Figure 3 presents a schematic overview of the process of our LGA module. As input, it takes a feature map (\(F^{\text{in}}\in\mathbb{R}^{H\times W\times C}\), where \(H\), \(W\) and \(C\) denote the height, width and number of channels, respectively). It constructs a graph using this feature map (as shown in Figure 4). Every spatial feature on the feature map, with dimensions of \(1\times 1\times C\), is considered a node of the graph, leading to \(N=HW\) nodes in the entire graph, as shown in Figure 4.  
Figure 2: We present the novel concept of latent graph attention (LGA), which can be easily integrated in wide variety of applications and architectures. Three challenging and open problems are considered as example applications of LGA in this article. _Learning the weights of the graph edges._ Edges should be constructed in the graph depending on the directions in which a strong spatial relation is to be learned. We construct edges only between immediate spatial neighbors based on a 8-shaped connectivity pattern (see Figure 4). This leads to nine connections per node--one self-connection and eight with neighboring nodes. The weights of the edges are learnable, and each of these weights is generated using a one-layered CNN comprising a \(1\times 1\) convolution layer. Since we have nine edges, there are a total of nine such mini-networks that learn the edges. _Construction of adjacency matrix._ The learned edges are used to construct an adjacency matrix (\(A\)). It is an \(N\times N\) matrix, where \(A_{ij}\in A\) denotes the weight of the edge connecting  
Figure 4: The top row shows the conversion of feature map \(X_{i}\) to graph nodes. Each cell of \(X_{i}\) in an \(H\times W\) plane (at all depths) is considered as one node in the graph. The graph information is extracted by 9 kernels—1 for each direction. Edge maps (in top row) correspond to the input, gt, and predicted triplet (bottom left). The middle-bottom figure shows how the node intensities represent the edge weights. The bottom-right figure shows how connectivity strength information propagates through recursive LGA layers. Figure 3: Operations performed by the LGA layer are shown in the expanded view. Starting with the encoder’s feature maps (\(F^{\text{in}}\)), edge maps are created using 2D convolution. The dotted arrow between the _edge map_ and _adjacency matrix_ implies that this transfer happens only once, even if the LGA layer is repeated multiple times. Next, the normalized adjacency matrix is used to calculate the output (\(X_{i+1}\)) for the \(i\)th LGA layer. The LGA contrastive loss (\(\mathcal{L}_{\text{LGA}}\)) is computed between the output of the LGA module (\(F^{\text{out}}\)) and the ground truth. \(X_{i+1}\) becomes the input to the \(i+1\)th LGA layer. the \(i\)th node with the \(j\)th node in the graph. For the sake of convergence, \(A\) is normalized to form \(A^{*}\). This normalization ensures that the feature vectors of all the nodes remain within a unit polyspsphere and are of comparable strengths with respect to each other. We define \(A^{*}=AD^{-1}\), where \(D_{ij}=\sum_{j}A_{ij}+\epsilon\). Here, \(\epsilon\) is added to avoid zero division.