## 3 The Proposed Approach
### Latent Graph Attention (LGA)
The matrix (\(A\)) is normalized based on the weights of the outgoing edges, since it provides better results and stability during the training process. _Message passing._ With this construction, LGA facilitates message passing between immediate neighbors in the latent space. To propagate information farther away, multiple such graphs are stacked to build a graph network that constitutes the LGA module. The information propagation at each LGA layer (Figure 3) can be stated as  
\[X_{i+1}=f(X_{i}A^{*}). \tag{1}\]  
where \(X_{i}\) and \(X_{i+1}\) denote the input and output of the \(i\)th layer, respectively (\(X_{0}\) indicates \(F^{in}\), and \(f\) is a learnable function used to change the dimensions of the feature vector of each node). **Learning in LGA. ** The core of LGA relies on constructing a representative directed graph. Learning in this graph is guided by the direct loss (\(\mathcal{L}_{\text{LGA}}\)) on the output feature map (\(F^{\text{out}}\)). The main goal of LGA is to capture neighboring context via message passing such that nodes corresponding to the same information have a similar distribution. Each node in the graph belongs to a particular spatial location in the image. We assume that each node corresponds to a particular patch in the GT (\(X^{\text{GT}}\)). This patch is calculated either via receptive field estimation of each node or by dividing the \(X^{\text{GT}}\) in \(N\) equal patches, where every node corresponds to a unique patch. Then, we calculate the similarity between these patches. For example, for the segmentation task, we calculate whether the patches are of the same classes. For image restoration tasks such as image dehazing, we calculate the structural similarity index measure (SSIM) between the patches; if the SSIM is greater than a predefined threshold, the patches are considered similar and not otherwise. The nodes corresponding to similar patches are likely to have similar distributions. \(\mathcal{L}_{\text{LGA}}\) penalizes \(F^{\text{out}}\) if nodes belonging to similar patches have different distributions or nodes corresponding to non-similar patches have similar distributions. This loss is defined such that LGA learns to predict richer representations to avoid learning an identity mapping between \(F^{\text{in}}\) and \(F^{\text{out}}\). **LGA contrastive loss. ** This novel loss term helps in the learning of our LGA module. Mathematically, LGA contrastive loss can be stated as  
\[\mathcal{L}_{\text{LGA}}=E_{P,P_{i}}\Bigg{(}\mathcal{C}_{ij}\log\Bigg{(} \frac{V_{ij}^{2}}{U_{ij}}+1\Bigg{)}+\mathcal{C}_{ij}\log\Bigg{(}\frac{U_{ij}} {V_{ij}^{2}}+1\Bigg{)}\Bigg{)}, \tag{2}\]  
\(V_{ij}=D(F^{\text{out}}_{i},F^{\text{out}}_{j})\) and \(U_{ij}=D(F^{\text{in}}_{i},F^{\text{in}}_{j})\), where \(F^{\text{in}}\) and \(F^{\text{out}}\) denote the input and output feature maps for the LGA module, respectively, and \(D(\cdot)\) is a divergence function, such as KL-divergence or mean square error. The variables \(\mathcal{C}\) and \(\bar{\mathcal{C}}\) are Boolean, and their values depend on the similarity between neighboring nodes in the graph. For example, for nodes \(i\) and \(j\), we look at the GT labels of patches containing them. Furthermore, we aggregate the labels to assign a single label to the node. For example, one aggregate measure could be to assign the majority class label to the node. Let \(Agg(\cdot)\) denote the aggregate function and \(i\) and \(j\) denote two nodes from the graph; then, \(\mathcal{C}=1\) if \(Agg(i)=Agg(j)\) and 0 otherwise. Next, we present the interpretation and consequence of using this loss function. We note that \(U_{ij}\) is determined by the input to the LGA module and does not update as LGA learns. LGA essentially learns by tweaking the value of \(V_{ij}\), which is determined by the output of LGA. Note that a larger diversity value \(D_{ij}\) indicates larger differences in the distributions of nodes \(i\) and \(j\). With these points, we now assess how the loss function behaves and influences the learning of LGA in different situations.