SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Erik Altman<sup>1</sup> Jovan Blanuša<sup>2</sup> Luc von Niederhäusern<sup>2</sup>  
Béni Egressy<sup>3*</sup> Andreea Anghel<sup>2</sup> Kubilay Atasu<sup>2</sup>
<sup>1</sup>IBM Watson Research, Yorktown Heights, NY, USA
$^{2}$ IBM Research Europe, Zurich, Switzerland  $^{3}$ ETH Zurich, Switzerland
ealtman@us.ibm.com {jov, lvn}@zurich.ibm.com
begressy@ethz.ch {aan, kat}@zurich.ibm.com
With the widespread digitization of finance and the increasing popularity of cryptocurrencies, the sophistication of fraud schemes devised by cybercriminals is growing. Money laundering – the movement of illicit funds to conceal their origins – can cross bank and national boundaries, producing complex transaction patterns. The UN estimates 2-5% of global GDP or $0.8 - $2.0 trillion dollars are laundered globally each year. Unfortunately, real data to train machine learning models to detect laundering is generally not available, and previous synthetic data generators have had significant shortcomings. A realistic, standardized, publicly-available benchmark is needed for comparing models and for the advancement of the area.
To this end, this paper contributes a synthetic financial transaction dataset generator and a set of synthetically generated AML (Anti-Money Laundering) datasets. We have calibrated this agent-based generator to match real transactions as closely as possible and made the datasets public. We describe the generator in detail and demonstrate how the datasets generated can help compare different machine learning models in terms of their AML abilities. In a key way, using synthetic data in these comparisons can be even better than using real data: the ground truth labels are complete, whilst many laundering transactions in real data are never detected.
Money laundering refers to the movement of illicit funds to conceal their origin and make them appear to come from legitimate sources. Laundering schemes often cross bank and national boundaries, producing complex transaction patterns. Even though the amount of laundering is difficult to estimate precisely, the UN puts the global figure at up to $2 trillion per year [6] as noted in the abstract.
Other, narrower, figures help illustrate the amount of laundering. The CEO of Danske Bank resigned in 2019 after it was revealed that the Estonian branch alone may have laundered  $230 billion [1]. Finextra estimates that in 2017 there may have been$ 200 billion in laundering solely in U.S. online sales [4]. Bank fines can be in the billions of dollars for failing to follow stringent standards on laundering [2]. As such, it is valuable to have high-quality data to help build models to detect laundering. Alas, for legal and privacy reasons, real data is not generally available. Even if real data were available, its labeling is inherently poor, as most laundering often goes undetected [7, 5].
Synthetic financial data is an emerging area to help address a multitude of challenges with real data and the algorithms used for analysis. These challenges include privacy and differential privacy, competitive advantage, small sample sizes, what-if scenarios, poorly labeled real data, bias, consistent
*This work was performed when Béni Egressy was with IBM Research Europe.
(a)
Trans. ID	Timestamp	Source bank ID	Source Account	Target bank ID	Target Account	Amount	Currency	Payment type
0	3 MAY 2019 12:45	1	A	1	C	1400	USD	Cheque
1	15 MAY 2019 07:34	2	B	1	C	710	EUR	ACH
2	18 MAY 2019 16:55	3	E	1	C	950	USD	Credit card
3	1 JUN 2019 10:06	1	C	3	D	1200	CHF	Wire
4	27 JUN 2019 13:18	3	E	3	D	2300	EUR	Credit card
5	7 JUL 2019 11:14	3	D	1	A	1100	USD	Credit card
6	14 JUL 2019 09:37	2	B	3	E	650	USD	ACH
7	20 JUL 2019 14:02	3	E	3	D	2500	USD	Wire
Figure 1: Financial transactions in (a) tabular format and in (b) graph format.
baseline data for regulators, and more. Realistic synthetic data can help address many of these issues but has its own challenges. For example, does synthetic data match real data and desired scenarios? Can synthetic data be constructed without access to instances of real data being imitated? We address these questions in Sections 2 and 3.
Relational tables are the most common way of storing financial data. Figure 1a illustrates financial transactions stored in a tabular format, where each row is a transaction that transfers financial assets from a source account to a destination account, potentially held in different banks. The amount, the timestamp, the payment currency, and the payment type are also indicated for each transaction.
Graphs offer a natural representation for financial transaction data. A graph-based data representation exposes the connectivity of the underlying data objects and makes it possible to extract complex patterns of interactions between them. The graph-based representation of transactions shown in Figure 1a is illustrated in Figure 1b. In a financial transaction graph, the nodes typically represent accounts and the directed edges between the accounts represent financial transactions. Several different transactions between the same two accounts can take place at different points in time as illustrated in Figure 1b. Therefore, financial transaction graphs are essentially directed multigraphs [10]. This general graph structure underlies many financial crime analysis techniques [63, 43].
Financial crime refers to illegal acts committed to obtain financial gain. Financial crime is often linked with suspicious account activities. For instance, in financial transaction graphs, a cycle represents a series of transactions in which the money initially sent from one bank account returns back to the same account; the existence of such cycles is a strong indicator of money laundering [58]. Figure 1b provides a clear visual depiction of cycles in money laundering, which consists of the highlighted transactions from Figure 1a. Similarly, in cryptocurrency transaction networks, criminals use sophisticated mixing and shuffling schemes to cover the traces of their activities [34]. Such schemes can usually be expressed in terms of subgraph structures, such as scatter-gather and bipartite patterns [58, 55, 33, 46] Efficient discovery of such suspicious patterns of interactions across account holders can enable timely and cost-effective detection of criminal activities and their perpetrators.
Our contributions can be listed as follows:
- A synthetic financial transaction generator, which builds a multi-agent virtual world in which some of the agents are criminals with illicit income to launder. This synthetic approach provides perfect information about which transactions are laundering and the specific patterns, such as cycles, that are used in each instance. This perfect information is not available in real data. Section 3 provides additional details.
- A set of realistic, standardized AML datasets that can be used for developing and benchmarking new money laundering detection models. The datasets cover a range of sizes and difficulties and are all made publicly available.
- A set of initial experiments with Graph Neural Networks (GNNs) and with Gradient Boosted Trees (GBT) that demonstrate the use of the datasets and provide baselines for future work. Importantly, our GNN code is open source. In addition, our GBT results can be reproduced using publicly available tools. Our experimental evaluation is presented in Section 4.
- Observations in Section 7 about how our data release is ethically sound and supports the fight against money laundering and other forms of financial crime.
Despite the importance of good data for money laundering detection, legal, privacy, and competition concerns mean that real financial transaction data is not generally available [52, 31]. Even in the
Table 1: Comparison to key previous synthetic AML data.
MLDP	AMLsim	AMLworld (This Work)
Do placement?	✓	✓
Do layering?	✓	✓	✓
Do integration?	✓	✓
Model 8 key patterns?	✓	✓
Model other patterns?	✓	✓
Model multiple banks?	✓
Model multiple currencies?	✓
Complex entity graph?	✓
Model transfers, payments, credits, etc.?	Transfers only	Transfers only	✓
# of Transactions	2,340	1.32 M	180 M
Laundering rates	3/5	1/762	1/807 & 1/1,750
restricted cases when anonymized data is available, ground truth labels are not known and data labeling is inherently poor since laundering often goes undetected [7, 5, 6].
For example, one of the few studies to use "real" data is from Starnini et al. [56] which reports 6 months of data from a major Italian bank with 180M transactions. Although myriad laundering patterns are possible, this work looks for only two patterns (aka smurfs or motifs), and there is no guarantee that all instances of these two patterns are detected. In addition, this data was made available only to Starnini et al. and appears not to be available for use by other investigators.
On a smaller scale, Harris et al. [24] received 2,827 transaction history summaries for 4,469 customers of Scotia Bank in Canada, with the goal of assessing which customers pose a high risk for laundering.
To get around these data availability issues, there have been proposals across many domains to generate synthetic data [53, 45, 44, 32, 40, 61, 37]. However, to the best of our knowledge, existing AML efforts suffer from one or more of the following shortcomings: (i) the synthetic data generators need access to real data, which they can then mimic; (ii) the AML data lacks authoritative islaundering labels; or (iii) the data produced lacks important attributes of real data.
The small set of previous synthetic AML efforts are agent-based, like ours. However, agent-based simulations can vary significantly in their approach, e.g. modeling more or less detail, or using a larger or smaller number of agents. For activities like money laundering, with only a tiny fraction of illicit transactions (i.e. minority class transactions), modeling fidelity and scale matters. Table 1 outlines key aspects of money laundering data. Table 1 also notes the capabilities and differences between our approach AMLworld, detailed in Section 3, and two of the most prominent previous efforts to generate synthetic laundering data: Money Laundering Data Production (MLDP) [38, 39] and AMLsim [63, 57, 9]. As Table 1 suggests, the capabilities of AMLworld are significantly more advanced than those of its predecessors, making for more robust data sets. (Terms like placement, layering, and integration will be defined in Section 3, as will the "8 key patterns.") Higher numbers of transactions and lower laundering rates are generally more realistic, and therefore more useful. The 2,340 transactions and  $60\%$  laundering rate for MLDP suggest a tiny, unrealistic scenario. Section 3.3 provides additional detail about AMLworld and AMLSim and about the techniques used in AMLworld to enhance realism.
GNNs [65, 59, 14, 30, 23, 15, 35, 62] are powerful machine learning models specifically designed for relational data (graphs). In particular, they can also be used for financial crime detection in transaction networks. Cardoso et al. [15] and Weber et al. [64] use GNNs for money laundering detection, Kanezashi et al. [28] use GNNs for phishing detection on the Ethereum blockchain, and Rao et al. [47] uses a GNN to detect fraudulent transactions. Graph Substructure Network, proposed by Bouritsas et al. [14], takes advantage of pre-calculated subgraph pattern counts to improve the expressivity of GNNs. GNNs could also be used to count subgraph patterns, such as in Chen et al. [17], which can enable detecting patterns associated with financial crime.
3 Synthetic Datasets
Abundant labeled data is available for key domains such as images, speech, natural language processing, and recommendation engines. However, due to concerns regarding privacy, law, and competitive
(a) Fan-out
(b) Fan-in
(c) Gather-scatter
(d) Scatter-gather
Figure 2: Laundering Patterns Modelled
e) Simple cycle
(f) Random
(g) Bipartite
(h) Stack
advantage, little or no financial transaction data is broadly available. To this end, we: (1) introduce our detailed AMLworld generator, and (2) make multiple datasets produced by AMLworld public.
3.1 Motivation for Synthetic Data
Aside from the difficulty of accessing real financial data with laundering transactions, real data has shortcomings that synthetic data can help overcome:
- Individual banks see only their own transactions – not the transactions at the multitude of other institutions with which the bank's customers transact.
- Ground truth labels are usually incomplete, with many instances of laundering being missed. As a result, many AML models mislabel laundering transactions (false negatives).
- Identifying complex money laundering patterns in real data requires additional work – if it is even possible for individual banks (please see the first bullet point).
The conclusive ground truth labels in synthetic data enable the precise comparison of different models. Additionally, the multiple banks in synthetic data enable measurement of how much performance would improve if banks could share their data in a federated learning setup. The scenario of an individual bank can easily be simulated by filtering the synthetic data.
3.2 AML patterns
In their AMLSim generator, Suzumura and Kanezashi [57] introduced a set of 8 patterns $^2$  often used in money laundering. These 8 patterns are illustrated in Figure 2. Among its many capabilities outlined in Section 3.3, our AMLworld also generates these 8 patterns.
To be more precise, a fan-out pattern of vertex  $v$ , shown in Figure 2a, is defined by the outgoing edges of  $v$  connecting  $v$  to at least  $k \geq 2$  different vertices [58]. Analogously, for a fan-in pattern, shown in Figure 2b,  $v$  is connected to  $k \geq 2$  different vertices via incoming edges. A gather-scatter pattern combines a fan-in pattern with a fan-out pattern of the same vertex, as illustrated in Figure 2c [55]. A fan-out pattern of a vertex  $v$  and a fan-in pattern of a vertex  $u$  form a scatter-gather pattern if the fan-out and the fan-in patterns connect vertices  $v$  and  $u$ , respectively, to the same set of intermediate vertices [55]. Figure 2d shows an example of a scatter-gather pattern with four intermediate vertices.
A simple cycle pattern, shown in Figure 2e, is a sequence of edges that starts and ends with the same vertex and visits other vertices at most once [27, 12]. Figure 1 shows a cycle in the financial transaction graph, in which node A launders money through nodes C and D. A Random pattern, depicted in Figure 2f, is similar to the cycle pattern in Figure 2e – except with random funds are not returned to their original account. As such, random can be viewed as a random walk among accounts owned or controlled (e.g. through shell companies). A bipartite pattern in Figure 2g moves funds from a set of input accounts to a set of output accounts. Finally, a stack pattern in Figure 2h extends the bipartite pattern by adding an additional bipartite layer.
In these cases, as with all the laundering patterns, the laundering entity owns or controls all accounts (nodes) used. For example, an entity may "control" shell companies, that appear to "own" an account. Over the years, rules have become stricter about who is the Ultimate Beneficial Owner (UBO) of an account. However, accurate UBO designations can be difficult to enforce or detect. Note also that some entities own or control more nodes than others, and thus can have larger patterns or more variation in their node selection.
AMLSim
Randomly decide transactions at each time step in accordance with histogram config files
Figure 3: Overview of AMLSim.
Figure 4: Overview of AMLworld Simulation.
3.3 What is new in our AMLworld generator?
Our AMLworld generator attempts to improve on the pioneering efforts of AMLSim [57]. Table 1 in Section 2 gives a breakdown of the differences. For more detail, Figure 3 depicts AMLSim and Figure 4 depicts AMLworld. As the figures suggest, AMLworld models a much richer set of characteristics.
For example, AMLworld models the entire money laundering cycle [19]: (1) Placement: sources like smuggling of illicit funds; (2) Layering: mixing the illicit funds into the financial system; and (3) Integration: spending the illicit funds.
AMLSim focuses only on (2) Layering – hence the patterns described in Section 3.2. Placement in AMLworld can come from 9 sources of criminal activity: extortion, loan sharking, gambling, prostitution, kidnapping, robbery, embezzlement, drugs, and smuggling. Money received and the frequency of these criminal enterprises vary by activity and by the acting entity. Illicit funds are then layered into the financial system (under A in Figure 4). Integration happens under B at the bottom right of Figure 4 when criminal entities decide how to deploy their funds. AMLSim by contrast handles only layering. To support this detailed laundering model AMLworld tags illicit funds from their origin (e.g. smuggling proceeds) through all transactions used to launder those funds.
AMLworld also has more detailed modeling of population and transaction characteristics, and laundering activity is not limited to specific patterns. Section 3.4 outlines other key attributes of AMLworld including the AMLworld mechanisms to support "Select Counter Party" at the upper right of Figure 4.
2 Some AML papers refer to patterns as smurfs or motifs.
Figure 5: Illustration of the entity / circular flow graph representing our Virtual World Model.
3.4 The Virtual World Model
At its foundation AMLworld builds a multi-agent virtual world of banks, individuals, and companies – with individuals and companies buying items, and making bank transfers to get supplies, pay salaries, pay pensions, etc. The underlying model has good and bad actors, with bad actors doing things like smuggling as noted in Section 3.3.
The bad actors' attempts to launder their illicit funds result in money-laundering transactions. In addition to the laundering patterns discussed above,  $AML_{\text{world}}$  also makes some laundering transfers happen "naturally", e.g. a criminal boss launders money by paying employees, buying supplies, etc.
The data produced is a set of transactions as illustrated in Figure 1, with each transaction labeled as laundering (illicit) or not. AMLworld has been used to generate billions of transactions in a variety of currencies including dollars, euros, yuan, yen, bitcoin, and more.
The actions of all agents in our virtual world are governed by statistical distributions. Thus the model and data are not based on obfuscating or anonymizing real individuals, but are based on real statistics. Likewise, our model uses no seed of real transactions from which to expand. Everything is synthetic. More specifically, as depicted in Figure 5, the underlying virtual world model of AMLworld uses a complex entity graph where different entities are connected in many ways. For example, a company serves as an employer, but may also offer retail sales or supply products or services to other companies and individuals. An individual or company may also own other companies or parts (shares) of companies. Each entity (person or company) owns one or more bank accounts directly and via subsidiaries.
The entity graph in Figure 5 is essentially a variant of the Circular Flow graphs used to measure GDP (Gross Domestic Product) [3]. As such any financial actor (person or company) can be a (blue) major node, and each such major node can own one or more bank accounts. Edges in the graph reflect financial flows between these nodes. (Flows are often bidirectional, but for simplicity, Figure 5 shows only a subset of important flow directions.)
Transactions are generated based on these relationships and at appropriate frequencies. For example, salaries are typically paid weekly, biweekly, or monthly (as noted in Figure 4). Interest is almost always paid monthly. In addition, a small number of laundering transactions are conducted by entities with funds from illicit activities. Since the generator knows the source of all funds, it knows which transfers are laundering.
Returning to Figure 5, the circular flow graph representing our virtual world illustrates several other important aspects of our approach:
- The circular flow graph enables the generation of a financial transaction graph (see Figure 1) – with a list of all transactions between entities during the time period of simulation.
- There are multiple types of entities in the model, including individuals, corporations, partnerships, and sole proprietors. Banks are a special type of entity in the model as they are the locus of most financial transactions among the other types.
- Entities such as people and companies have bank accounts through which their transactions are conducted. Each person and each company may have multiple bank accounts across multiple banks. This spreading can further legitimate pursuits such as saving and spending, but can also facilitate illegal activity such as obfuscating fund sources during laundering.
- Ownership of companies is cross-linked, e.g. company X may own company Y in whole or part (e.g. through stock). Individuals may also own companies in whole or part. This ownership pattern serves two functions for AMLworld: (1) it represents the legitimate complexity of the world, and (2) it allows layers of shell companies to be created through
which bad actors can launder their ill-gotten funds. These shell companies may have accounts at arbitrary banks, e.g. their accounts may be at different banks than the primary bank accounts of their controlling entity.
- The graph structure in Figure 5 is arbitrary, e.g. the graph is not bipartite between merchants and consumers as can be the case in modeling credit card transactions [8].
The AMLworld simulator parameterizes population and transaction information to facilitate: (a) tuning; (b) modeling future changes in behavior; and (c) updating to better match real behavior. For (c), authoritative real data is not always available. Feedback loops in AMLworld also enforce a degree of tuning to make statistics consistent with observed behavior. For example, if results show too many transactions per person per month, then parameters may be adjusted. Some events are more important than others, e.g. moving money to avoid a negative bank account balance takes precedence in most cases over moving money to get a higher interest rate. Since parameters are not all independent, a certain amount of iteration is needed to arrive at a consistent set of parameters.
As suggested by point (b) of the previous paragraph, parameterization is also useful for checking what-if scenarios, e.g. what if there were more laundering, or what if there were changes to banking laws? Synthetic data enables these explorations, which are not possible with real data.
Finally, we note the laundering tag on financial transfers is transitive in \(AML_{\text{world}}\). If \(\mathbf{A}\) pays \(\mathbf{B}\) \\(100 in illicit funds, and \(\mathbf{B}\) then pays \\)50 of those funds to \(\mathbf{C}\), and \(\mathbf{C}\) pays \\(25 of the \\)50 to \(\mathbf{D}\), then not only is the initial \\(100 considered laundering, but the \\)50 and the \\)25 are also. This transitivity is supported by detailed tracking; \(AML_{\text{world}}\) propagates this tag through all transactions, including co-mingling of funds from legitimate and non-legitimate sources. \(AML_{\text{world}}\)’s complete labeling of laundering is nearly impossible with real-world data.
3.5 Public Availability of Synthetic Data
We have used AMLworld to create several synthetic AML datasets that have been made available on Kaggle [50]. Data is divided into two high-level groups: HI and LI, with higher and lower illicit ratios (laundrying) respectively. Both HI and LI are further divided into small, medium, and large datasets, with large having  $175M - 180M$  transactions. Due to space constraints, Table 4 in the Appendix provides details about the datasets.