SlowGuess/NeurIPS_2023_OCR huggingface dataset:
learning a weighted combination of filter-specific representations. The combined representations for downstream tasks, considering  $K$  filters from filter bank  $\mathbf{F}$ , are as follows:
$$
Z = \sum_ {i = 1} ^ {K} \alpha_ {i} \phi^ {\prime} \left(E _ {\theta} \left(\mathbf {X}, \mathbf {F} _ {i}\right)\right) \tag {5}
$$
The parameters  $\alpha_{i}$ 's are learnable. Additionally, the function  $\phi^{\prime}(.)$  represents either the RFF projection or an identity transformation, depending on whether  $E_{\theta}(\mathbf{X},\mathbf{F}_{i})$  is low-dimensional or not. A classifier model (e.g. logistic regression) consumes these embeddings, where we train both the  $\alpha_{i}$ 's and the weights of the classifier. Fig 2 illustrates this process. Notably, semi-supervised methods like [28, 7, 17] differ as they learn both encoder and coefficients from labeled data, while our method pre-trains the encoder and learns task-specific combinations of filter-specific representations.
**Training Details:** We define a single-layer graph convolutional network (GCN) with shared weights  $(\Theta)$  across all filters in the filter bank  $(\mathbf{F})$  as our encoder. Therefore, the encoder can be expressed as follows:  $E_{\theta}(\mathbf{X},\mathbf{F}_i) = \psi (\mathbf{F}_i\mathbf{X}\Theta)$ . It is important to note that  $\mathbf{F}_i$  represents a normalized filter with self-loops, which ensures that its eigenvalues are within the range of [0, 2]. The non-linearity function  $\psi$  refers to the parametric rectified linear unit [16]. As we work with a single graph, we obtain the positive samples by sampling nodes from the graph. Using these sampled nodes, we construct a new adjacency list that only includes the edges between these sampled nodes in filter  $\mathbf{F}_i$ . On the other hand, the corruption function  $\mathcal{C}$  operates on the same sampled nodes. However, it randomly shuffles the node features instead of perturbing the adjacency list. In essence, this involves permuting the node features while maintaining a consistent graph structure. This action introduces a form of corruption, altering each node's feature to differ from its original representation in the data. Similar to [44], we employ a straightforward readout function that involves averaging the representations across all nodes for a specific filter  $\mathbf{F}_i$ :  $\mathcal{R}(\mathbf{H}_i) = \sigma \left(\frac{1}{N}\sum_{j = 0}^{N}h_j^{\mathbf{F}_i}\right)$  where  $\sigma$  denotes the sigmoid non-linearity. We utilize a bilinear scoring function  $D_{\omega}(.,.)$ , whose parameters are also shared across all filters, where  $D_{\omega}(h_j^{\mathbf{F}_i},h_g^{\mathbf{F}_i}) = \sigma (h_j^{\mathbf{F}_iT}\mathbf{W}h_g^{\mathbf{F}_i})$ . We learn the encoder and discriminator parameters by optimising Eq. 4. While we could use various filter banks, we specifically employ the filter bank corresponding to GPRGNN  $(\mathbf{F}_{\mathrm{GPRGNN}})$  for all our experiments. However, we conduct an ablation study (see 6.5) comparing  $\mathbf{F}_{\mathrm{GPRGNN}}$  with  $\mathbf{F}_{\mathrm{BERNET}}$ . Additional training details are available in 8.3.
We conducted a series of comprehensive experiments to evaluate the effectiveness and competitiveness of our proposed model compared to SOTA models and methods. These experiments address the following research questions: [RQ1] How does FiGURE, perform compared to SOTA unsupervised models? [RQ2] Can we perform satisfactorily even with lower dimensional representations using projections such as RFF? [RQ3] Does shared encoder decrease performance? [RQ4] What is the computational efficiency gained by using lower dimensional representations compared to methods that rely on higher dimensional representations? [RQ5] Can alternative filter banks be employed to
recover good quality representations? [RQ6] How does FiGURe combine information from different filters (in  $\mathbf{F}_{\mathrm{GPRGNN}}$ ) in a task-dependent manner?
Datasets and Setup: We evaluated our model on a diverse set of real-world datasets, which include both heterophilic and homophilic networks, to assess its effectiveness. Similar to previous works, we utilized the node classification task as a proxy to evaluate the quality of the learned representations. Please refer to 8.2 for detailed information about the benchmark datasets.
Baselines: In our comparison against baselines, we considered common unsupervised approaches, such as DEEPWALK and NODE2VEC, and state-of-the-art mutual information-based methods, namely DGI, MVGRL, GRACE, and SUGRL. We also include the performance numbers of the widely used GCN for reference. Unless stated otherwise, we use a 512-dimensional representation size for all reported results, following prior research. For additional details please refer to 8.3 and 8.4.
6.1 RQ1:FiGURe versus SOTA Methods
Table 1: Contains node classification accuracy percentages on homophilic and heterophilic datasets.  $\mathrm{FiGURe}_{32}^{\mathrm{RFF}}$  and  $\mathrm{FiGURe}_{128}^{\mathrm{RFF}}$  refer to FiGURe trained with 32 and 128 dimensional representations, respectively, and then projected using RFF. Other models, including  $\mathrm{FiGURe}_{512}$  and all baselines are trained at 512 dimensions and do not utilize RFF. Higher numbers indicate better performance. It is worth noting that  $\mathrm{FiGURe}_{512}$  outperforms or remains competitive with the baselines in all cases. The rightmost column Av.  $\Delta_{gain}$  represents the average accuracy  $\%$  gain of  $\mathrm{FiGURe}_{512}$  over the model in that row, averaged across the different datasets. If a model is labeled as 'OOM' for a dataset, it's excluded from Av.  $\Delta_{gain}$  calculation. Magenta, Green and Blue represent the  $1^{\mathrm{st}}$ ,  $2^{\mathrm{nd}}$  and  $3^{\mathrm{rd}}$  best performing models, for a particular dataset.
HETEROPHILIC DATSETS	HOMOPHILIC DATSETS
SQUIRREL	CHAMELEON	ROMAN-EMPIRE	MINESWEEPER	ARXIV-YEAR	CORA	CITESEER	PUBMED	OGBN-ARXIV	Av. Δgain
DEEPWALK	38.66 (1.44)	53.42 (1.73)	13.08 (0.59)	79.96 (0.08)	41.05 (0.10)	83.64 (1.85)	63.66 (3.36)	80.85 (0.44)	64.02	13.48
NODE2VEC	42.60 (1.15)	54.23 (2.30)	12.12 (0.30)	80.00 (0.00)	39.69 (0.09)	78.19 (1.14)	57.45 (6.44)	73.24 (0.59)	60.20	15.78
DGI	39.61 (1.81)	59.28 (1.23)	47.54 (0.76)	82.51 (0.47)	40.59 (0.09)	84.57 (1.22)	73.96 (1.61)	86.57 (0.52)	65.58	6.61
MVGRL	39.90 (1.39)	54.61 (2.29)	68.50 (0.38)	85.60 (0.35)	OOM	86.22 (1.30)	75.02 (1.72)	87.12 (0.35)	OOM	4.39
GRACE	53.15 (1.10)	68.25 (1.77)	47.83 (0.53)	80.22 (0.45)	OOM	84.79 (1.51)	67.60 (2.01)	87.04 (0.43)	OOM	5.54
SUGRL	43.13 (1.36)	58.60 (2.04)	39.40 (0.49)	82.40 (0.58)	36.96 (0.19)	81.21 (2.07)	67.50 (1.62)	86.90 (0.54)	65.80	8.64
FiGURExRF	48.89 (1.55)	65.66 (2.52)	64.61 (0.92)	85.28 (0.71)	41.30 (0.21)	82.56 (0.87)	71.25 (2.20)	83.91 (0.69)	66.58	3.65
FiGURExRF	48.78 (2.48)	66.03 (2.19)	67.01 (0.56)	85.16 (0.58)	41.94 (0.15)	86.14 (1.13)	73.34 (1.91)	83.56 (0.34)	69.11	2.53
FiGURex512	52.23 (1.19)	68.55 (1.87)	70.99 (0.52)	85.58 (0.49)	42.26 (0.20)	87.00 (1.24)	74.77 (2.00)	88.60 (0.44)	69.69	0.00
Table 2: Comparing node classification with the widely-used GCN,  $\mathrm{{FiGUR}}{\mathrm{e}}_{512}$  achieves strong results even without task-specific labels, highlighting its ability to learn quality representations.
SQUIRREL	CHAMELEON	ROMAN-EMPIRE	MINESWEEPER	ARXIV-YEAR	CORA	CITESEER	PUBMED	OGBN-ARXIV
GCN	47.78 (2.13)	61.43 (2.70)	73.69 (0.74)	89.75 (0.52)	46.02 (0.26)	87.36 (0.91)	76.47 (1.34)	88.41 (0.46)	69.37 (0.00)
FiGUREn12	52.23 (1.19)	68.55 (1.87)	70.99 (0.52)	85.58 (0.49)	42.26 (0.20)	87.00 (1.24)	74.77 (2.00)	88.60 (0.44)	69.69 (0.00)
Table 3: Mean epoch time (in seconds) on the two large datasets for different embedding sizes. For lower dimensional embeddings, there is a significant speedup.
512 dims	128 dims	32 dims
ARXIV-YEAR	1.24s	0.75s	0.72s
OGBN-ARXIV	0.92s	0.74s	0.72s
We analyzed the results in Table 1 and made important observations. Across homophilic and heterophilic datasets,  $\mathrm{FiGURe}_{512}$  consistently outperforms several SOTA unsupervised models, except in a few cases where it achieves comparable performance. Even on the large-scale datasets ARXIVYEAR and OGBN-ARXIV  $\mathrm{FiGURe}_{512}$  performs well, demonstrating the scalability of our method. Two baseline methods MVGRL and GRACE run into memory issues on the larger datasets and are accordingly reported OOM in the table. We want to emphasize the rightmost column of the table, which shows the average percentage gain in performance across all datasets. This metric compares the improvement that  $\mathrm{FiGURe}_{512}$  provides over each baseline model for each dataset and averages these improvements. This metric highlights that  $\mathrm{FiGURe}_{512}$  performs consistently well across diverse datasets. No other baseline model achieves the same consistent performance across all datasets as  $\mathrm{FiGURe}_{512}$ . Even the recent state-of-the-art contrastive models GRACE and SUGRL experience
average performance drops of approximately  $5\%$  and  $10\%$ , respectively. This result indicates that  $\mathrm{FiGUR e}_{512}$  learns representations that exhibit high generalization and task-agnostic capabilities. Another important observation is the effectiveness of RFF projections in improving lower dimensional representations. We compared FiGUR e at different dimensions, including  $\mathrm{FiGUR e}_{32}^{\mathrm{RFF}}$  and  $\mathrm{FiGUR e}_{128}^{\mathrm{RFF}}$ , corresponding to learning 32 and 128-dimensional embeddings, respectively, in addition to the baseline representation size of 512 dimensions. Remarkably, even at lower dimensions, FiGUR e with RFF projections demonstrates competitive performance across datasets, surpassing the 512-dimensional baselines in several cases. This result highlights the effectiveness of RFF projections in enhancing the quality of lower dimensional representations. Using lower-dimensional embeddings reduces the computation time and makes FiGUR e faster than the baselines. The computational efficiency of reducing dimension size becomes more significant with larger datasets, as evidenced in Table 3. On ARXIV-YEAR, a large graph with 169, 343 nodes, 128-dimensional embeddings yield a 1.6x speedup, and 32-dimensional embeddings yield a 1.7x speedup. Similar results are observed in OGBN-ARXIV. For further insights into the effectiveness of RFF projections, see Section 6.2, and for computational efficiency gains, refer to Section 6.4. In Table 2, we include GCN as a benchmark for comparison. Remarkably,  $\mathrm{FiGUR e}_{512}$  remains competitive across most datasets, sometimes even surpassing GCN. This highlights that  $\mathrm{FiGUR e}_{512}$  can capture task-specific information required by downstream tasks, typically handled by GCN, through unsupervised means. When considering a downstream task, using  $\mathrm{FiGUR e}_{512}$  embeddings allows for the use of computationally efficient models like Logistic Regression, as opposed to training resource-intensive end-to-end graph neural networks. There are, however, works such as [6], [13], and [5], that explore methods to speed up end-to-end graph neural network training. In summary, FiGUR e offers significant computational efficiency advantages over end-to-end supervised graph neural networks. Performance can potentially improve further by incorporating non-linear models like MLP. For detailed comparisons with other supervised methods, please refer to 8.4. It's noteworthy that both OGBN-ARXIV and ARXIV-YEAR use the ARXIV citation network but differ in label prediction tasks (subject area and publication year, respectively). FiGUR e demonstrates improvements in both cases, showcasing its task-agnostic and multi-task capabilities due to its flexible node representation. This flexibility enables diverse tasks to extract the most relevant information (see Section 5.2), resulting in strong overall performance. Note that in 8.5, we have also performed an ablation study where the depth of the encoder is increased.
6.2 RQ2: RFF Projections on Lower Dimensional Representations
Table 4: Node classification accuracy percentages with and without using Random Fourier Feature projections (on 32 dimensions). A higher number means better performance. The performance is improved by using RFF in almost all cases, indicating the usefulness of this transformation
RFF	CORA	CITESEER	SQUIRREL	CHAMELEON
DGI	×	81.65 (1.90)	65.62 (2.39)	31.60 (2.19)	45.48 (3.02)
✓	81.49 (1.96)	66.50 (2.44)	38.19 (1.52)	56.01 (2.66)
MVGRL	×	81.03 (1.29)	72.38 (1.68)	37.20 (1.22)	49.65 (2.08)
✓	80.48 (1.71)	72.54 (1.89)	39.53 (1.04)	56.73 (2.52)
SUGRL	×	65.35 (2.41)	42.84 (2.57)	31.62 (1.47)	43.20 (1.79)
✓	70.06 (1.24)	47.03 (3.02)	38.50 (2.19)	51.01 (2.26)
GRACE	×	76.84 (1.09)	58.40 (3.05)	38.20 (1.38)	53.25 (1.58)
✓	79.15 (1.44)	63.66 (2.96)	51.56 (1.39)	67.39 (2.23)
FiGURRe32	×	82.88 (1.42)	70.32 (1.98)	39.38 (1.35)	53.27 (2.40)
✓	82.56 (0.87)	71.25 (2.20)	48.89 (1.55)	65.66 (2.52)
In this section, we analyse the performance of unsupervised baselines using 32-dimensional embeddings with and without RFF projections (see Table 4). Despite extensive hyperparameter tuning, we could not replicate the results reported by SUGRL, so we present the best results we obtained. Two noteworthy observations emerge from these tables. Firstly, it is evident that lower dimensional embeddings can yield meaningful and linearly separable representations when combined with simple RFF projections. Utilising RFF projections enhances performance in almost all cases, highlighting the value captured by MI-based methods even with lower-dimensional embeddings. Secondly,  $\mathrm{FiGURRe}_{32}^{\mathrm{RFF}}$  consistently achieves superior or comparable performance to the baselines, even in lower dimensions. Notably, this includes SUGRL, purported to excel in such settings. However, there is a 2-
$3\%$  performance gap between GRACE and our method for the SQUIRREL and CHAMELEON datasets. While GRACE handles heterophily well at lower dimensions, its performance deteriorates with homophilic graphs, unlike  $\mathrm{FiGURRe}_{32}^{\mathrm{RFF}}$  which captures lower frequency information effectively. Additionally, our method exhibits computational efficiency advantages for specific datasets in lower dimensions. Please refer to 8.6: for discussions with regards to the RFF algorithm, 8.7: for analysing the RFF behaviour and community structure, 8.8: for experiments using other random projection methods, 8.9 and 8.10: for ablation studies with regards to projecting to higher dimensional spaces via RFF, and 8.11: for issues related to including RFF in training. Overall, these findings highlight the potential of RFF projections in extracting useful information from lower dimensional embeddings and reaffirm the competitiveness of FiGURe over the baselines.
6.3 RQ3: Sharing Weights Across Filter Specific Encoders
Table 5: A comparison of the performance on the downstream node classification task using independently trained encoders and weight sharing across encoders is shown. The reported metric is accuracy. In both cases, the embeddings are combined using the method described in 5.2
CORA	CITESEER	SQUIRREL	CHAMELEON
INDEPENDENT	86.92 (1.10) %	75.03 (1.75) %	50.52 (1.51) %	66.86 (1.85) %
SHARED	87.00 (1.24) %	74.77 (2.00) %	52.23 (1.19) %	68.55 (1.87) %
Our method proposes to reduce the computational load by sharing the encoder weights across all filters. It stands to reason whether sharing these weights causes any degradation in performance. We present the results with shared and independent encoders across the filters in Table 5 to verify this. We hypothesize that, sharing encoder weights embeds diverse filter representations in a common space, improving suitability for combined representation learning. This enhances features for downstream tasks, in some cases boosting performance. Experimental results confirm that shared weights do not significantly reduce performance; sometimes, they even enhance it, highlighting shared encoders' effectiveness in reducing computational load without sacrificing performance.
6.4 RQ4: Computational Efficiency
Table 6: Mean epoch time (in milliseconds) averaged across 20 trials with different hyperparameters. A lower number means the method is faster. Even though our method is slower at 512 dimensions, using 128 and 32 dimensional embeddings significantly reduces the mean epoch time. Using RFF as described in 6.2 we are able to prevent the performance drops experienced by DGI and MVGRL.
DGI	MVGRL	FiGUR\( e_{512} \)	FiGUR\( e_{128}^{RFF} \)	FiGUR\( e_{32}^{RFF} \)
CORA	38.53 (0.77)	75.29 (0.56)	114.38 (0.51)	20.10 (0.46)	11.54 (0.34)
CITESEER	52.98 (1.15)	102.41 (0.99)	156.24 (0.56)	30.30 (0.60)	17.16 (0.51)
SQUIRREL	87.06 (2.07)	168.24 (2.08)	257.65 (0.76)	47.72 (1.40)	23.52 (1.14)
CHAMELEON	33.08 (0.49)	64.71 (1.05)	98.36 (0.64)	18.56 (0.39)	11.63 (0.48)
To assess the computational efficiency of the different methods, we analyzed the computation time and summarized the results in Table 6. The key metric used in this analysis is the mean epoch time: the average time taken to complete one epoch of training. We compared our method with other MI based methods such as DGI and MVGRL. Due to the increase in the number of augmentation views, there is an expected increase in computation time from DGI to MVGRL to FiGUR. However, as demonstrated in 6.2, using RFF projections allows us to achieve competitive performance even at lower dimensions. Therefore, we also included comparisons with our method at 128 and 32 dimensions in the table. It is evident from the results that our method, both at 128 and 32 dimensions, exhibits faster computation times compared to both DGI and MVGRL, which rely on higher-dimensional representations to achieve good performance. This result indicates that FiGUR is computationally efficient due to its ability to work with lower-dimensional representations. During training, our method,  $\text{FiGUR} e_{32}^{\text{RFF}}$ , is  $\sim 3x$  faster than DGI and  $\sim 6x$  times faster than MVGRL. Despite the faster computation,  $\text{FiGUR} e_{32}^{\text{RFF}}$  also exhibits an average performance improvement of around  $2\%$  across the datasets over all methods considered in our experiments. Please refer to 8.12 and 8.13 for further discussions.
6.5 RQ5: Experiments on Other Filter Banks
Table 7: Accuracy percentages for various filter banks in conjunction with FiGURE. Specifically,  $\mathbf{F}_{\mathrm{BERNNET}}^{3}$  and  $\mathbf{F}_{\mathrm{BERNNET}}^{11}$  refer to the  $\mathbf{F}_{\mathrm{BERNNET}}$  filter bank with  $K$  set to 3 and 11, respectively. Similarly,  $\mathbf{F}_{\mathrm{CHEBNET}}^{3}$  and  $\mathbf{F}_{\mathrm{CHEBNET}}^{11}$  represent the  $\mathbf{F}_{\mathrm{CHEBNET}}$  filter bank with  $K$  set to 3 and 11, respectively.
CORA	CITESEER	SQUIRREL	CHAMELEON
FBERNNET3	85.13 (1.26)	73.38 (1.81)	37.07 (1.29)	53.95 (2.78)
FBERNNET11	86.62 (1.59)	73.97 (1.43)	43.48 (3.80)	62.13 (3.66)
FCHEBNET3	83.84 (1.36)	71.92 (2.29)	40.23 (1.58)	60.61 (2.03)
FCHEBNET11	76.14 (6.80)	59.89 (8.94)	52.46 (1.10)	67.37 (1.60)
FGPRGNN	87.00 (1.24)	74.77 (2.00)	52.23 (1.19)	68.55 (1.87)
To showcase the versatility of our proposed framework, we conducted an experiment using Bernstein and Chebyshev filters, as detailed in Table 7. The results indicate that using  $\mathbf{F}_{\mathrm{GPRGN}}$  leads to better performance than BERNNET and CHEBNET filters. We believe this is happening due to the latent characteristics of the dataset. [17, 28] have shown that datasets like CHAMELEON and SQUIRREL need frequency response functions that give more prominence to the tail-end spectrum. GPRGNN filters are more amenable to these needs, as demonstrated in [28]. However, different datasets may require other frequency response shapes, where BERNNET and CHEBNET filters may excel, and give better performance. For instance,  $\mathbf{F}_{\mathrm{BERNNET}}$  may better approximate comb filters, as their basis gives uniform prominence to the entire spectrum. Our framework is designed to accommodate any filter bank, catering to diverse dataset needs. Further discussions are in 8.14.
6.6 RQ6: Combining information from different filters (in FGPRGNN)
To analyze how FiGURe combines representations from different filters, we present alpha coefficients for the highest-performing split in Table 8, utilizing  $\mathbf{F}_{\mathrm{GPRGNN}}$  on: CORA, CITESEER, SQUIRREL, and CHAMELEON (coefficients may vary for different splits within the same dataset) [28]. GPRGNN filters adapt well to heterophilic datasets like CHAMELEON and SQUIRREL, emphasizing spectral tails with significant weightage on the  $\mathbf{A}^2$  filter. In contrast, homophilic datasets require low-pass filters, achieved by assigning higher weightage to the A and  $\mathbf{A}^3$  filters. Achieving these filter shapes with other methods like BERNNET and CHEBNET is possible but more challenging for the model to learn. We believe that this is why GPRGNN filters consistently outperform other filter banks on the datasets we've examined. However, it's important to note that some datasets may benefit from different response shapes, where BERNNET and CHEBNET filters might be more suitable. This table validates our hypothesis about the efficacy of GPRGNN filter coefficients, with  $\mathbf{A}^3$  dominating in homophilic datasets (CORA, CITESEER) and  $\mathbf{A}^2$  in heterophilic datasets (CHAMELEON, SQUIRREL).
Table 8: We present the alpha coefficients obtained from the best-performing split utilizing GPRGNN filters for CORA, CITESEER, SQUIRREL, and CHAMELEON datasets
I	A	A2	A3
CORA	18.2	0	0	35.95
CITESEER	0	0	0	0.48
SQUIRREL	0	0	15.3	0
CHAMELEON	0	0	8.93	0.1
Our work demonstrates the benefits of enhancing contrastive learning methods with filter views and learning filter-specific representations to cater to diverse tasks from homophily to heterophily. We have effectively alleviated computational and storage burdens by sharing the encoder across these filters and focusing on low-dimensional embeddings that utilize high-dimensional projections, a technique inspired by random feature maps developed for kernel approximations. Future directions involve expanding the analysis from [2] to graph contrastive learning and investigating linear separability in lower dimensions, which could strengthen the connection to the random feature maps approach.
[38] O. Platonov, D. Kuznedelev, M. Diskin, A. Babenko, and L. Prokhorenkova. A critical look at the evaluation of GNNs under heterophily: Are we really making progress? In International Conference on Learning Representations (ICLR), 2023.
[39] Z. Qin, W. Sun, H. Deng, D. Li, Y. Wei, B. Lv, J. Yan, L. Kong, and Y. Zhong. cosformer: Rethinking softmax in attention. In International Conference on Learning Representations, 2022.
[40] A. Rahimi and B. Recht. Random features for large-scale kernel machines. In Neural Information Processing Systems (NeurIPS), 2007.
[41] A. Rahimi and B. Recht. Weighted sums of random kitchen sinks: Replacing minimization with randomization in learning. In Neural Information Processing Systems (NeurIPS), 2008.
[42] D. I. Shuman, S. K. Narang, P. Frossard, A. Ortega, and P. Vandergheynst. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE Signal Processing Magazine, 2013.
[43] N. Tremblay, P. Gonçalves, and P. Borgnat. Design of graph filters and filterbanks. Cooperative and Graph Signal Processing, 2017.
[44] P. Velicković, W. Fedus, W. L. Hamilton, P. Lio, Y. Bengio, and R. D. Hjelm. Deep graph infomax. In International Conference on Learning Representations (ICLR), 2019.
[45] Y. Zhu, Y. Xu, F. Yu, Q. Liu, S. Wu, and L. Wang. Deep Graph Contrastive Representation Learning. In ICML Workshop on Graph Representation Learning and Beyond, 2020.
8 Supplementary Material
1 Introduction 1
2 Related Work 2
3 Problem Setting 2
4 Preliminaries 3
4.1 Filter Banks 3
4.2 Random Feature Maps for Kernel Approximations 3
5 Proposed Approach 3
5.1 Unsupervised Representation Learning 3
5.2 Supervised Representation Learning 5
6 Experimental Results 6