SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Ethical use of the data includes using it for benchmarking and improving models for the detection of money laundering activity. We foresee significant positive societal impacts from such uses of our data. Money laundering has a huge cost for society in itself, but more importantly, money laundering enables a whole range of criminal activities to continue, ranging from phishing attacks to human trafficking. Detecting money laundering transactions can help authorities uncover such activities and identify the criminals behind them. Additionally, the data could be used for pretraining detection models before fine-tuning on real data.
Given that the dataset is synthetic, there is no risk of it containing personally identifiable information or offensive content. Moreover, researchers do not need to take special care with its use, but they should bear in mind that performance might not translate one-to-one to real data.
C Hyperparameter Tuning of Machine Learning Models
GBT Baselines. We use successive halving [26] for the hyperparameter tuning of the LightGBM and XGBoost models. Successive halving starts by randomly sampling  $x_0$  model parameter configurations. Each configuration of model parameters is evaluated using a fraction  $r_0 \leq 1$  of the initial training set. The algorithm then finds the best  $x_0 / \eta$  configurations, with  $\eta > 1$ . These best configurations are used in the next round of successive halving using a fraction  $\eta \times r_0$  of the train set. This process continues until the fraction of the training set used for the evaluation reaches 1. We use different successive halving configurations for different datasets, as shown in Table 9. Furthermore, the parameter ranges from which  $x_0$  initial parameter configurations are sampled are shown in Table 10.
GNN Baselines. We used random sampling to identify a good range of GNN hyperparameters. A second round of random sampling was conducted with this narrower range to pick our final set of hyperparameters. We varied the following hyperparameters: the number of GNN layers, hidden embedding size, learning rate, dropout, and minority class weight (for the weighted loss function).
Table 10: Model parameter ranges used at for the hyperparameter tuning of GBT models. The small parameter range, indicated with Range-small, is only used for hyperparameter tuning of LightGBM models related to HI-Large and LI-Large datasets. The large parameter range shown in column Range-large is used for other datasets.
LightGBM	XGBoost
Parameter	Range-small	Range-large	Parameter	Range
num_round	(32, 512)	(10, 1000)	num_round	(10, 1000)
num_leaves	(32, 256)	(1, 16384)	max_depth	(1, 15)
learning_rate	(0.001, 0.01)	10(-2.5,-1)	learning_rate	10(-2.5,-1)
lambda_12	(0.01, 0.5)	10(-2,2)	lambda	10(-2,2)
scale_pos_weight	(1, 10)	(1, 10)	scale_pos_weight	(1, 10)
lambda_11	10(0.01,0.5)	10(0.01,0.5)	colsample_bytree	(0.5, 1.0)
subsample	(0.5, 1.0)
The exact ranges we used are listed in 11. The number of random samples was set to between 10 to 50, depending on the training time of the model on a particular dataset. To get our final results, we use the hyperparameters with the best validation score to train four models initialized with different random seeds.
Table 11: Model parameter ranges used for hyperparameter tuning of GNN models. The hyperparameters were optimized on the small datasets and used for all GNN models.
Parameter Ranges	GIN (+EU)	PNA
hidden embedding size	(16,72)	(16,64)
learning rate	(0.005,0.05)	(0.0001, 0.002)
number of GNN layers	(2,4)	(2,4)
dropout	(0,0.5)	(0, 0.2)
minority class weight	(6,8)	(6,8)
D Graph Feature Preprocessor Configuration
The Graph Feature Preprocessor (GFP) processes transactions represented as temporal edges in a streaming fashion. The input of this preprocessor is a batch of temporal edges that the preprocessor inserts into its in-memory dynamic graph and extracts various graph features from this graph, such as scatter-gather patterns, simple cycles, and vertex statistics. The output of the preprocessor is the same batch of edges with these additional graph-based features. This library is available as part of the Snap ML library [49], and the experiments in this paper were performed using version 1.14 of Snap ML. More details on GFP are available in the documentation [48].
The graph-based features for our experiments are extracted using the batch size of 128. The GFP is configured to generate features based on scatter-gather patterns, temporal cycles, simple cycles of length up to 10, and vertex statistics. We set the GFP to use a time window of six hours for scatter-gather patterns and a time window of one day for the rest of the graph-based features. The vertex statistic features are computed using the "Amount" and "Timestamp" fields of the basic transaction features (see Figure 1a). This configuration is used for all datasets and experiments that use GFP in this paper.
For each dataset, GFP processes transactions in the increasing order of their timestamps. This ordering ensures that the graph-based features for each transaction are extracted using the past data. As a result, transactions from the training set will not contain graph-based features computed using the validation or test sets. Processing transactions in such a way prevents data leakage.
E Additional GNN Experiments
Our GNN code is included with the supplementary material and available publicly on GitHub  ${}^{4}$  under an Apache License. The GNNs are implemented using PyTorch Geometric version 2.3.1 [21] and PyTorch version 2.0.1.
All the baseline GNN experiments were run on an internal cluster on Nvidia Tesla V100 GPUs. Table 12 shows the runtimes when training the different GNN baselines on the AML Small and Medium datasets. The size of the GNN models was kept the same: 2 GNN layers and a hidden embedding size of 64. The total GPU time, including initial experiments and hyperparameter optimization, is estimated to have been around 1000 GPU hours.
Table 12: Total training times (TTT) and inference performance in Transaction per Second (TPS) for all GNN baselines on the AML Small and Medium datasets using an Nvidia Tesla V100 GPU.
Model	HI-Small	LI-Small	HI-Medium	LI-Medium
TTT (s)	TPS	TTT (s)	TPS	TTT (s)	TPS	TTT (s)	TPS
GIN	22703	17210	29655	15784	85652	7101	85994	7316
GIN+EU	26046	11844	36625	11640	85753	4981	85887	5097
PNA	27745	16557	39648	14994	85654	6725	85827	6819
In addition to the minority class F1 scores in Table 2, we include more fine-grained results detailing the precision and recall rates of the GNN-based models. Precision evaluates the accuracy of laundering predictions, recall measures the model's ability to identify all laundering instances, while the F1 score is the harmonic mean of precision and recall. Table 13 shows the laundering recall rates and Table 14 shows the corresponding precision scores. In Figure 9, we give example precision-recall curves, which visually capture the trade-off between precision and recall across different decision thresholds. The examples are taken from the best-performing seed from the best-performing model (PNA) on all small and medium AML datasets.
Table 13: Minority class recall rate (%) for the GNN-based models. HI indicates a higher illicit ratio. LI indicates a lower illicit ratio.
Model	HI-Small	LI-Small	HI-Medium	LI-Medium
GIN [65, 25]	38.16 ± 5.92	14.59 ± 2.37	39.86 ± 3.61	8.07 ± 9.32
GIN+EU [11, 15]	55.41 ± 5.96	23.26 ± 2.87	48.06 ± 6.45	5.51 ± 6.82
PNA [60]	53.15 ± 2.26	16.43 ± 2.62	47.42 ± 4.30	20.44 ± 0.66
Table 14: Minority class precision (%) for the GNN-based models. HI indicates a higher illicit ratio. LI indicates a lower illicit ratio.
Model	HI-Small	LI-Small	HI-Medium	LI-Medium
GIN [65, 25]	27.40 ± 7.98	5.14 ± 3.42	47.78 ± 5.20	5.23 ± 3.60
GIN+EU [11, 15]	42.29 ± 10.69	21.60 ± 10.83	52.56 ± 8.27	12.13 ± 19.35
PNA [60]	58.48 ± 10.67	17.37 ± 5.80	69.12 ± 4.26	36.50 ± 10.51
F Additional GBT Experiments
Figure 11 shows the precision-recall curves for XGBoost models trained on all AML datasets using graph-based features created with Graph Feature Preprocessor. These curves correspond to the results from the row "GFP+XGBoost" shown in Table 2. The steep slope of each curve after the red dot, which represent a data point obtained using the prediction threshold of 0.5, indicates that it is challenging to obtain higher recall without significantly degrading the precision. Note that it is possible to achieve higher precision of these XGBoost models by simply reducing the recall by a few percent.
<sup>4</sup>https://github.com/IBM/Multi-GNN
(a) HI-Small
(b) LI-Small
(c) HI-Medium
(d) LI-Medium
Figure 9: Precision-Recall Curves for the best-performing PNA model for all AML datasets. The red dot indicates the F1 score obtained using the prediction threshold of 0.5.
(a)
(b)
Figure 10: Effect of sharing the data and LightGBM models across banks for the (a) LI-Medium and (b) LI-Large datasets.
Figure 10 shows performance on a per-bank basis using the LI datasets. The experiment is explained in Section 4 and the plot is analogous to Figure 6, but here we are using the LI-Medium and LI-Large datasets. These dataset contain fewer illicit transactions compared to the HI-Medium and HI-Large datasets (see Table 4). Therefore, it is more challenging to build a machine learning model for a bank using only local data of these LI datasets compared to the banks of HI datasets. In this case, the average minority-class F1 score across 30 banks shown in Figure 10 is only  $4.9\%$  for LI-Medium and  $8.7\%$  for LI-Large. Nevertheless, the improvement in minority-class F1 score of the shared graph, shared model case compared to the private graph, private model case is still significant. Sharing the transaction graph and the global model across the banks increases the average minority-class F1 score to  $20.8\%$  for LI-Medium and to  $22.1\%$  for LI-Large.
(e) HI-Large
(f) LI-Large
Figure 11: Precision-Recall Curves for the XGBoost models for all AML datasets trained using graph-based features of GFP. The red dot indicates the F1 score obtained using the prediction threshold of 0.5.
G Datasheet
We include a datasheet based on the framework set forward by Gebru et al. [22]. Some parts of the proposed datasheet are omitted since our datasets are synthetic.
For what purpose was the dataset created? The dataset was created to test, develop, and improve machine-learning models for financial crime detection. In particular, the datasets focus on identifying money laundering transactions.
Who created the dataset and on behalf of which entity? Erik Altman on behalf of IBM.
Who funded the creation of the dataset? IBM
What do the instances that comprise the dataset represent? The datasets are synthetic financial transaction networks. Each node in the network represents an account/entity and each directed edge represents a transaction from one account to another. Edge features detail the amount, currency, and type of transaction amongst other properties. The datasets also contain transactions that are labeled as money laundering. All the information is simulated. No real account or transaction details were used to create the datasets.
How many instances are there in total? There are 6 datasets. Each dataset consists of one graph. The number of transactions (samples) ranges from 5M to 180M.
Does the dataset contain all possible instances or is it a sample (not necessarily random) of instances from a larger set? The datasets are synthetic. Any number of transactions could be generated.
What data does each instance consist of? When used for transaction classification, the edges can be considered instances. Each instance consists of a set of transaction features (incl., amount, currency, date, time, and type). In addition, each transaction is part of a whole network of transactions, and since the network topology plays an important role, the position of the instance in the whole network could be considered a "part" of the instance.
Is there a label or target associated with each instance? Yes.
Are there recommended data splits (e.g., training, development/validation, testing)? Yes.
Are there any errors, sources of noise, or redundancies in the dataset? Not to the knowledge of the authors.
Is the dataset self-contained, or does it link to or otherwise rely on external resources (e.g., websites, tweets, other datasets)? The dataset is self-contained.
G.3 Collection Process, Uses, Distribution and Maintenance
Please refer to Section 4 and Appendix A for details about the generation process. The current usage is detailed in the paper and potential uses are described in Appendix B. The Kaggle page acts as the single source of distribution [50]. The dataset will be maintained there.