SlowGuess/NeurIPS_2023_OCR huggingface dataset:
this section, we focus on temporal link prediction, i.e., we try to forecast future interactions based on the historical transactions. Specifically, we investigate the snapshot-based representation for temporal link prediction via graph neural networks [36, 25]. Since each edge  $e$  has a timestamp  $\tau_{e}$ , we utilize a sequence of graph to depict the temporal transaction network  $G = \{\mathcal{G}_t\}_{t=1}^T$ , where each snapshot is a static graph  $\mathcal{G}_t = (\mathcal{N}_t, \mathcal{E}_t, \mathcal{T}_t)$  with  $\mathcal{E}_t = \{e \in \mathcal{E} | \tau_e \leq t\}$ . Through modeling the sequential information of different graph snapshots, we could leverage the information available up to time  $t$  to forecast possible edges at time  $t + 1$ . Although various approaches have been proposed for temporal link prediction, it is unclear whether they can achieve satisfied performance in this highly active and large-scale temporal transaction network.
Graph Neural Networks. GNNs have gained great success in numerous learning tasks on graphs including node classification [36, 25], graph classification [84, 83] and link prediction [82, 75]. The objective of GNN is to acquire effective node representations through iteratively aggregating messages from its local neighborhood. Specifically, the  $l$ -th layer of a GNN model can be formulated as follows:
$$
\mathbf {h} _ {v} ^ {l} = \operatorname {A G G} ^ {l} \left(\left\{\operatorname {M S G} ^ {l} \left(\mathbf {h} _ {u} ^ {l - 1}, \mathbf {h} _ {v} ^ {l - 1}\right) | u \in \mathcal {N} (v) \right\}, \mathbf {h} _ {v} ^ {l - 1}\right)
$$
where  $\mathbf{h}_v^l$  is the node representation for node  $v$  at layer  $l$  and  $\mathcal{N}(v)$  represents node  $v$ 's neighborhood.  $\mathrm{MSG}(\cdot)$  indicates the message-passing function, which propagates information from its neighbors.  $\mathrm{AGG}(\cdot)$  is the aggregation function, which updates its representation with node's neighborhood representations. For a  $L$  layers GNN, it aggregates information from the  $L$ -hop neighborhood. Different GNN architectures can have different message-passing and aggregation functions. The original GNN model is designed for static graphs, thus it cannot capture the underlying temporal information within the graph. To incorporate the evolving property, existing works utilize RNNs [28, 14] to aggregate information from different graph snapshots. Hence, the dynamic GNN models have much more parameters compared with the vanilla GNNs, which is more difficult to scale to large graphs due to the back propagation through time constraints.
Models. We compare a number of recent state-of-the-art dynamic GNN models. (1) Dyngraph2vec [21] captures the temporal transitions with a deep architecture with dense and recurrent layers. (2) TGCN [87] integrates GCN with GRU to learn complex spatial and temporal dependencies. (3) EvolveGCN [56] use a RNN to adapt the graph convolutional network parameters. (4) GCRN [62] generalize TGCN with either GRU or LSTM. Moreover, it uses ChebNet [15] to encode the graph structure information, and separate GNNs are applied to compute different gates of RNNs. (5) DynGEM [22] utilizes deep autoencoder to perform graph embedding, then local and global constraints are employed to keep node representations being stable over time. (6) Roland [81] is an efficient learning framework designed for temporal GNNs, which updates its parameters through a combination of incremental training and meta-learning strategies [19, 20].
Settings. For dataset, we remove all the transactions associated with the Null address, which results in 3.13 million nodes and 23.13 million edges in the directed graph. We set node feature as 1 for all the nodes and utilize area under the curve (AUC) as well as mean reciprocal rank (MRR) as evaluation metrics. For every node  $u$  connected by a positive edge  $(u, v)$  at time  $t$ , we randomly select 100 negative edges originating from node  $u$ . Subsequently, we determine the rank of the score for edge  $(u, v)$  among all the sampled negative edges. AUC characterizes the probability of ranking positive node more highly than negative nodes. MRR denotes the average of reciprocal ranks computed across all nodes. Following the settings in Roland [81], we utilize two different train-test splits: fixed-split and live-update. Among them, the fixed-split setting assesses the models using all the edges from the last 20% graph snapshots. Although it is widely used in the existing works, the fixed-split might produce misleading results based on edges merely from the last few graph snapshots, since the graph structure could constantly evolve in the real world scenario. To eliminate this bias, we also utilize live-update split to test the models, which evaluates their performance over all the available graph snapshots. 10% of edges are used to determine the early-stop condition in this setting.
Results. We present the link prediction results in Table 3. Specifically, we use three different time granularities (i.e., days, weeks and months) to construct the graph snapshot sequences, which results in 1,657 graph snapshots, 253 graph snapshots and 60 graph snapshots, respectively. All the models' performance is evaluated under two different settings, i.e., fixed-split and live-update. We notice that as the time granularity becomes coarser, the models' performance drops. This is because the NFT transaction network is highly active, which is about 68 million transaction volume per day on average in 2021. Thus, it will be more difficult to predict all the possible links in a long time period. Meanwhile, we can observe very high AUC scores in a daily time granularity, which indicates the
Table 3: Temporal link prediction performance in fixed split and live-update settings. We repeat experiments with three different seeds to report the mean as well as standard deviation of AUC and MRR. We also present the results under different time snapshot granularities, e.g., days, weeks and months. OOM means out-of-memory.
Models	Fixed Split
Snapshot Days	Snapshot Weeks	Snapshot Months
AUC	MRR	AUC	MRR	AUC	MRR
Dyngraph2vec	OOM	OOM	OOM	OOM	OOM	OOM
TGCN	53.64±1.60	14.24±2.60	61.55±6.24	36.16±6.60	74.87±4.99	45.97±4.46
EvolveGCN	OOM	OOM	OOM	OOM	OOM	OOM
GCRN-GRU	95.86±0.03	71.48±0.49	93.14±0.18	68.44±0.05	86.74±0.80	58.23±1.23
GCRN-LSTM	94.12±0.92	68.51±2.36	92.90±0.43	67.66±0.31	86.44±0.92	58.71±0.84
DynGEM	OOM	OOM	OOM	OOM	OOM	OOM
Roland-MA	95.93±0.15	66.34±0.23	93.53±0.13	65.06±0.43	86.23±0.85	54.93±1.42
Roland-MLP	65.46±6.10	43.76±5.94	73.34±7.87	42.04±16.8	85.88±2.22	57.58±4.12
Roland-GRU	73.33±11.5	49.45±8.91	91.48±1.67	66.28±1.86	86.59±0.88	59.37±1.54
Live Update
Dyngraph2vec	OOM	OOM	OOM	OOM	OOM	OOM
TGCN	58.22±5.76	17.77±10.7	59.94±8.44	22.67±19.3	75.01±2.66	43.16±0.75
EvolveGCN	OOM	OOM	OOM	OOM	OOM	OOM
GCRN-GRU	80.95±1.92	39.13±0.39	85.34±0.26	46.08±1.43	81.40±0.34	43.68±0.39
GCRN-LSTM	79.12±2.14	37.83±1.16	84.73±0.34	42.89±3.34	81.24±0.80	41.47±3.00
DynGEM	OOM	OOM	OOM	OOM	OOM	OOM
Roland-MA	90.47±0.66	49.79±0.95	88.74±0.37	50.77±1.13	83.93±0.95	47.11±1.16
Roland-MLP	56.32±8.06	22.16±15.4	70.88±10.5	40.91±10.8	79.38±4.47	46.51±2.61
Roland-GRU	60.04±5.08	27.29±6.26	75.93±19.2	48.66±13.7	84.36±0.46	50.75±0.83
models have a better capability of distinguishing the positive and the negative edges in this short time scenarios. Also, the performance in live-update setting is a little lower than the fixed-split setting. This is caused by the pattern shifts in different graph snapshots, which implies the patterns indeed evolve over the time. Therefore, we can conclude that it is necessary to model the NFT transaction network in a finer time granularity, and at the meantime this will result in longer graph snapshot sequence, which puts forward more challenges to model the temporal information.
5.2 Temporal Node Classification
Node classification plays a crucial role in understanding the attributes, behaviors, and relationships of nodes in temporal graphs, offering valuable insights in diverse applications. By predicting the label of nodes at a particular time  $t$ , we gain a temporal perspective that deepens our understanding of how nodes evolve over time and their dynamic characteristics. Specifically, we focus on categorizing nodes according to their transaction behaviors. They can be generally classified into five distinct classes: daily traders, weekly traders, monthly traders, yearly traders and the remaining traders. We first filter out nodes that only have one transaction. Then, each node's maximum transaction interval is calculated. If the maximum interval is within one day, we call it daily trader. Likewise, if the maximum interval is within one week and larger than one day, we call it weekly trader, and so forth. This process results in a large-scale directed graph with about 1.80 million nodes and 21.83 million edges. Following the setting of EvoloveGCN [56], we use the first  $80\%$  graph snapshots as training set, the following  $10\%$  graph snapshots as validation set and the last  $10\%$  graph snapshots as test set. Similar to temporal link prediction task, we use the same set of GNN models and add a multi-class classification layer. Furthermore, nodes' degrees are encoded as features. The number of transactions between two nodes and their latest interaction timestamp are transformed as edge features. Two commonly used metrics (i.e., accuracy and recall) are employed to assess the model's performance.
Results. The node classification results are illustrated in Table 4 and key observations are as follows. Three granularities, i.e., days, weeks and months, are utilized to generate the temporal graph snapshot sequences. In accordance with the temporal link prediction task, we can draw similar conclusions. Roland could consistently outperform other baselines, and its variant Roland-MA (moving-average) stands out due to its parameter-free design and remarkable capability to capture evolving patterns. On the other hand, TGCN fails to achieve satisfactory performance because of the gradient vanishing problem in longer sequences. GCRN-GRU and Roland-GRU address this issue by utilizing separate
Table 4: Node classification performance in fixed split setting. We repeat experiments with three different seeds to report the mean as well as standard deviation of Accuracy and Recall.
Models	Snapshot Days	Snapshot Weeks	Snapshot Months
Accuracy	Recall	Accuracy	Recall	Accuracy	Recall
Dyngraph2vec	OOM	OOM	OOM	OOM	OOM	OOM
TGCN	18.48±2.66	31.15±3.16	43.97±4.57	32.99±2.34	47.45±3.49	32.53±2.95
EvolveGCN	OOM	OOM	OOM	OOM	OOM	OOM
GCRN-GRU	41.06±3.30	34.75±2.93	46.78±0.72	34.79±0.42	47.42±2.16	28.97±3.22
GCRN-LSTM	46.14±3.29	35.19±3.61	48.04±2.37	31.58±1.75	49.32±2.01	35.39±1.49
DynGEM	OOM	OOM	OOM	OOM	OOM	OOM
Roland-MA	51.02±2.01	28.77±3.23	50.39±0.45	26.33±3.95	47.96±2.69	22.33±3.07
Roland-MLP	48.46±3.18	30.62±3.94	47.59±3.39	31.67±3.62	45.74±4.75	35.04±3.47
Roland-GRU	49.88±2.15	33.38±3.91	46.63±3.07	33.85±1.48	50.04±0.37	32.17±2.72
Figure 3: Five most common wash trading patterns.
GNNs or leveraging previous and current snapshots. Furthermore, the scalability of several baselines is limited and they experience out-of-memory (OOM) issues. Among the three time granularities, the month granularity is generally the most difficult scenarios. This is because with a month granularity, the time period between snapshots is longer compared with the day and week granularities. As a result, it becomes more challenging to accurately predict the temporal patterns and changes in node behaviors. The longer time gap between snapshots increases the complexity of modeling the evolving dynamics of the network and introduces more uncertainty, leading to decreased performance in terms of classification accuracy and recall. Therefore, we can conclude that given the highly active NFT transaction network, it is essential for the model to use a finer time granularity. Nonetheless, this choice results in longer graph snapshot sequences, presenting additional difficulties in accurately capturing the temporal information.
5.3 Continuous Subgraph Matching
Continuous Subgraph Matching (CSM) plays a vital role in numerous real-time graph applications. The goal of CSM is to identify and report the occurrences of a given query graph  $Q$  within a temporal graph stream  $G$ . It can be utilized in various scenarios. For example, when setting the query graph  $Q$  as wash trading patterns in e-commerce, CSM could identify the anomaly transaction patterns in the graph via exactly matching [58]. Similarly, representing query graph as rumor patterns in social network could help to detect and prevent the spread of rumors [74]. In this subsection, we set the query graphs as the frequent wash trading patterns in the NFT transactions. According to the definition in [73, 47], wash trading is an activity where the seller is on the both sides of the trade. The goal of wash trading is to influence the price or create the illusion that the item is very popular, which produces artificial activities in the marketplace. Wash trading has been prohibited by many countries. However, due to the anonymous nature of the blockchain, wash trading has become a severe issue in the NFT market<sup>6</sup>, which accounts for a large volume in the whole NFT transactions. For instance, the CryptoPunk's 9,998-th NFT was traded between two wallets for 124,457 ETH (about $534 million USD), in which the buyer paid to the seller, then the seller transferred the money back to the buyer. Thus, it is important to detect the wash trading transactions to uncover the anomaly behaviors and reduce the risks in the market. We resort to CSM to identify the wash trading transactions. Five most common wash trading patterns are shown in Figure 3. As can be seen, all of them contain at least one cycle. Here is a toy example that exemplifies Pattern 1: Address A (0x744...282) initiated the sale of Azuki token 1,215 to Address B (0xd39...263). Subsequently, Address B sold the token to Address
<sup>6</sup>https://cryptopotato.com/over-33-of-nft-volume-is-wash-trading-bitscrunch-ceo-interview/
Table 5: Comparison of different frameworks on query time.
Query Patterns	Model Query Time (ms)
Queries	Counts	SymBi	Graphflow	TurboFlux	RapidFlow
p1	19,338	1.22×104	1.11×104	1.11×104	5.93×102
p2	2,243,232	1.19×104	1.44×104	1.51×104	6.13×102
p3	3,012,738	1.11×104	1.13×104	1.08×104	5.83×102
p4	9,472,960	1.24×104	1.43×104	6.06×104	6.45×102
p5	3,154,355,868	1.34×105	4.24×105	7.72×104	5.84×102
C (Oxeaaa...c0f), and eventually, Address C sold it back to Address A. These transactions happened within half an hour, and interestingly, the token's price surged from 8.98 ETH to 11.99 ETH. Such activity can raise suspicions of wash trading. We will use them as query graphs in the experiments to simulate the wash trading detection procedure. More details are given in Appendix E.
Results. Table 5 shows the key results of continuous subgraph matching. As we can see, RapidFlow [68] is the most efficient algorithm, which demonstrates about 18-724x speedups compared with the remaining frameworks. This is because the query reduction technique can significantly expedite the query procedure through an optimized matching order. We also observe that no algorithm can dominate others in different query patterns except RaviadFlow. Among them, SymBi [50] and Graphflow [33] perform quite worst on pattern 5, which need 10x more time to produce the results. SJ-Tree [13] and IEDyn [30, 31] cannot output the results within the time limit. The reason is that SJ-Tree encounters memory exhaustion in the majority of cases, and IEDyn's index update bears too much overhead because of the maintenance for constant-delay enumeration. Furthermore, the number of matched subgraphs increase from pattern 1 to pattern 5, however the query time does not have too much difference in almost all the frameworks. We can conclude that these four frameworks are applicable to large-scale graphs with dense structures. They can effectively serve as a bridge to generate ground truth for continuous subgraph matching, providing valuable support for the training of deep graph learning models.
In this paper, we propose the concept of Live Graph Lab, which includes blockchain based temporal graphs that are openly accessible, fully recorded, and dynamically evolving over time. Specifically, we instantiate a live graph using the NFT transaction network and investigate its dynamic properties in a temporal graph analysis perspective. Our findings reveal both similar and distinct characteristics when compared to traditional networks like social networks, citation networks, and the Web. Through comprehensive experiments on the live graph, we uncover numerous insightful discoveries. The proposed live graph overcomes the limitations of existing datasets and enhances the diversity in graph research. We believe that the live graph lab will become an indispensable resource for the graph community and open up new opportunities. There are also various other potential use cases, such as identifying whether an account holds a specific type of tokens or predicting the range of tokens that the account possesses, etc.
7 Border Impact and Limitation
The Live Graph Lab can facilitate researchers from graph community by offering comprehensive blockchain based graphs via an easily accessible manner. Insights from this research can be directly applied to improve the design, security, and user experience of NFT platforms, leading to sustainable growth of the ecosystem. Meanwhile, as the live graph constantly records all the NFT transactions in the Ethereum blockchain, the possibility of encountering malicious activities could become a concern, such as bot transactions or wash trading, etc. It might also cause potential negative societal impacts. Since the dataset consists of complete transactions associated with the wallet addresses, it could enable the tracking of each wallet's behaviors, habits, and financial activities. This kind of tracking could be exploited for targeted advertising, manipulation, or surveillance. The dataset could also make it possible for malicious actors to analyze the transaction patterns and manipulate the NFT market, which could lead to unfair practices, price manipulation, and market instability.
This research is supported by the National Research Foundation, Singapore under its Industry Alignment Fund – Pre-positioning (IAF-PP) Funding Initiative. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of National Research Foundation, Singapore. The authors would like to thank reviewers for their helpful comments. The authors would also like to thank Zhengtao Jiang for the development of website.
[78] Bryan White, Aniket Mahanti, and Kalpdrum Passi. Characterizing the opensea nft marketplace. In Companion Proceedings of the Web Conference 2022, WWW '22, page 488-496, New York, NY, USA, 2022. Association for Computing Machinery.
[79] Feng Xie, Zhen Chen, Jiaxing Shang, Xiaoping Feng, and Jun Li. A link prediction approach for item recommendation with complex number. Knowledge-Based Systems, 81:148-158, 2015.
[80] Steve Y Yang and Jinhyoung Kim. Bitcoin market return and volatility forecasting using transaction network flow properties. In 2015 IEEE Symposium Series on Computational Intelligence, pages 1778-1785. IEEE, 2015.
[81] Jiaxuan You, Tianyu Du, and Jure Leskovec. Roland: graph learning framework for dynamic graphs. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2358-2366, 2022.
[82] Muhan Zhang and Yixin Chen. Link prediction based on graph neural networks. Advances in neural information processing systems, 31, 2018.
[83] Zhen Zhang, Jiajun Bu, Martin Ester, Jianfeng Zhang, Zhao Li, Chengwei Yao, Dai Huifen, Zhi Yu, and Can Wang. Hierarchical multi-view graph pooling with structure learning. IEEE Transactions on Knowledge and Data Engineering, 2021.
[84] Zhen Zhang, Jiajun Bu, Martin Ester, Jianfeng Zhang, Chengwei Yao, Zhi Yu, and Can Wang. Hierarchical graph pooling with structure learning. arXiv preprint arXiv:1911.05954, 2019.
[85] Zhen Zhang, Hongxia Yang, Jiajun Bu, Sheng Zhou, Pinggang Yu, Jianwei Zhang, Martin Ester, and Can Wang. Anr!: attributed network representation learning via deep neural networks. In Ijcai, volume 18, pages 3155-3161, 2018.
[86] Lin Zhao, Sourav Sen Gupta, Arijit Khan, and Robby Luo. Temporal analysis of the entire ethereum blockchain network. In Proceedings of the Web Conference 2021, pages 2258-2269, 2021.
[87] Ling Zhao, Yu jiao Song, Chao Zhang, Yu Liu, Pu Wang, Tao Lin, Min Deng, and Haifeng Li. T-gen: A temporal graph convolutional network for traffic prediction. IEEE Transactions on Intelligent Transportation Systems, 21(9):3848-3858, 2019.
[88] Zibin Zheng, Shaoan Xie, Hong-Ning Dai, Weili Chen, Xiangping Chen, Jian Weng, and Muhammad Imran. An overview on smart contracts: Challenges, advances and platforms. Future Generation Computer Systems, 105:475-491, 2020.
A Detailed Background
In this section, we explain the terminologies related to Ethereum blockchain, transactions and NFTs, etc.
A.1 Blockchain and Ethereum
Blockchain, a distributed ledger technology, has drawn continuous attention recent years. Blockchains are made up of securely linked blocks with cryptography techniques [53], where each block contains information of the previous block (e.g., cryptographic hash). Then, consensus algorithms or protocols are applied to validate transactions and keep them being consistent. By this way, the blockchain transactions are immutable, traceable and publicly available. Ethereum is a decentralized, programmable blockchain, which means users can construct various decentralized applications on the blockchain. Ether (ETH) is the native cryptocurrency of Ethereum, and every transaction incurred in the Ethereum needs a specific fee paid in ETH. According to the market capitalization, ETH is the second-largest cryptocurrency behind Bitcoin.
A.2 Account and Transaction
Ethereum account is the key to access and explore the Ethereum ecosystem. Every Ethereum account is associated with a unique address, akin to an email address for an inbox. The address can be used to receive or send funds to the corresponding account. Accounts in the Ethereum can be classified into two types: (1) Externally-owned account (EOA) and (2) Contract account. All the accounts are denoted as a 64 character hex string.
Transactions are messages sent from one account to another account. One of the simplest transaction is transferring ETH from one account to another, which will change the state of the Ethereum Virtual Machine (EVM) and need to be broadcast to the whole Ethereum network. Each transaction requires amount of fees to pay for the computation. The key information in a transaction includes the receive address, the sender address, value, data, gas limit, and the max fee per gas, etc. There are three categories of transactions within the Ethereum: 1) regular transactions, which indicates the transactions between two externally-owned accounts; 2) contract deployment transactions, which are special transactions without receive addresses; 3) execution of a contract, whose receive address is the smart contract address. When the transaction is submitted, its life cycle can be simplified into the following three steps: 1) an externally-owned account sends a transaction and generates a transaction hash; 2) the transaction is broadcast across the network; 3) a validator verifies the transaction and includes it into a block. Once the transaction is successfully executed, it can never be altered.
A.3 Smart Contract and Non-Fungible Token