Original: Federated Continual Learning to Detect Accounting Anomalies in Financial Auditing
Revised: Federated Continual Learning to Detect Accounting Anomalies in Financial Auditing

Original: Abstract
Revised: Abstract

Original: The International Standards on Auditing require auditors to collect reasonable assurance that financial statements are free of material misstatement.
Revised: The International Standards on Auditing require auditors to collect reasonable assurance that financial statements are free of material misstatement.

Original: At the same time, a central objective of Continuous Assurance is the ‘real-time’ assessment of digital accounting journal entries.
Revised: At the same time, a central objective of Continuous Assurance is the ‘real-time’ assessment of digital accounting journal entries.

Original: Recently, driven by the advances in artificial intelligence, Deep Learning techniques have emerged in financial auditing to examine vast quantities of accounting data.
Revised: Recently, driven by the advances in artificial intelligence, Deep Learning techniques have emerged in financial auditing to examine vast quantities of accounting data.

Original: However, learning highly adaptive audit models in decentralised and dynamic settings remains challenging.
Revised: However, learning highly adaptive audit models in decentralized and dynamic settings remains challenging.

Original: It requires the study of data distribution shifts over multiple clients and time periods.
Revised: It requires the study of data distribution shifts over multiple clients and time periods.

Original: In this work, we propose a Federated Continual Learning framework enabling auditors to learn audit models from decentral clients continuously.
Revised: In this work, we propose a Federated Continual Learning framework enabling auditors to learn audit models from decentral clients continuously.

Original: We evaluate the framework’s ability to detect accounting anomalies in common scenarios of organizational activity.
Revised: We evaluate the framework’s ability to detect accounting anomalies in common scenarios of organizational activity.

Original: Our empirical results, using real-world datasets and combined federatedcontinual learning strategies, demonstrate the learned model’s ability to detect anomalies in audit settings of data distribution shifts.
Revised: Our empirical results, using real-world datasets and combined federatedcontinual learning strategies, demonstrate the learned model’s ability to detect anomalies in audit settings of data distribution shifts.

Original:  Introduction
Revised:  Introduction

Original: The International Standards in Auditing (ISA) demand auditors to collect reasonable assurance that financial statements are free from material misstatements, whether caused by error or fraud [58, 27].
Revised: The International Standards in Auditing (ISA) demand auditors to collect reasonable assurance that financial statements are free from material misstatements, whether caused by error or fraud [58, 27].

Original: The term fraud refers to ‘the abuse of one’s occupation for personal enrichment through the deliberate misuse of an organisation’s resources or assets’ [80].
Revised: The term fraud refers to ‘the abuse of one’s occupation for personal enrichment through the deliberate misuse of an organisation’s resources or assets’ [80].

Original: According to the Association of Certified Fraud
Revised: According to the Association of Certified Fraud

Original: Examiners (ACFE) , organisations lose 5% of their annual revenues due to fraud [1].
Revised: Examiners (ACFE) , organisations lose 5% of their annual revenues due to fraud [1].

Original: The ACFE highlights that respondents experienced a median loss of USD 100K in the first 7-12 months after a fraud scheme begins.
Revised: The ACFE highlights that respondents experienced a median loss of USD 100K in the first 7-12 months after a fraud scheme began.

Original: Detecting fraud and abusequickly is critical since the longer a scheme remains undetected, the more severe its financial, reputational and organisational impact [54].
Revised: The timely detection of fraud and abuse is critical since the longer a scheme remains undetected, the more severe its financial, reputational and organisational impact [54].

Original: Over the last decade, the ongoing digital transformation  has fundamentally changed the nature, recording and volume of audit evidence [83].
Revised: The digital transformation of the last decade has fundamentally changed the nature, recording and volume of audit evidence [83].

Original: Nowadays, organizations record vast quantities of digital accounting records, referred to as Journal Entries (JEs) , in Enterprise Resource Planning (ERP) systems [21].
Revised: Nowadays, organizations record vast quantities of digital accounting records, referred to as Journal Entries (JEs) , in Enterprise Resource Planning (ERP) systems [21].

Original: For the audit profession, this unprecedented exposure to large volumes of accounting records offers new opportunities to obtain audit-relevant insights [2].
Revised: For the audit profession, this unprecedented exposure to large volumes of accounting records offers new opportunities to obtain audit-relevant insights [2].

Original: Lately, accounting firms also foster the development of Deep Learning (DL) capabilities [38] to learn advanced models to audit digital journal entry data.
Revised: Lately, accounting firms also foster the development of Deep Learning (DL) capabilities [38] to learn advanced models to audit digital journal entry data.

Original: To augment a human auditor’s capabilities DL models are applied in various audit tasks, such as accounting anomaly detection [68, 87, 50] illustrated in Fig.
Revised: To augment a human auditor’s capabilities, DL models are applied in various audit tasks, such as accounting anomaly detection [68, 87, 50] illustrated in Fig.

Original: 1, audit sampling [65, 62] or notes analysis [71, 55].
Revised: 1, audit sampling [65, 62] or notes analysis [71, 55].

Original: However, learning DL-enabled models in auditing is still in its infancy, exhibiting two main limitations.
Revised: However, learning DL-enabled models in auditing is still in its infancy, exhibiting two main limitations.

Original: First, most of today’s audit models are trained 
Revised: First, most of today’s audit models are trained from

Original: [Table]
Revised: [Table]

Original: e
Revised: e

Original: [Table]
Revised: [Table]

Original: A
Revised: A

Original: [Table]
Revised: [Table]

Original: r
Revised: r

Original: [Table]
Revised: [Table]

Original: t e k
Revised: t e k

Original: [Table]
Revised: [Table]

Original: r
Revised: r

Original: [Table]
Revised: [Table]

Original: t e
Revised: t e

Original: [Table] [Table]
Revised: [Table] [Table]

Original: -on
Revised: -on

Original: [Table]
Revised: [Table]

Original: Entries
Revised: Entries

Original: O
Revised: O

Original: P
Revised: P

Original: [Figure]
Revised: [Figure]

Revised: [Table]

Revised: [Figure]

Original: [Table] [Figure] 
Revised: [Table] [Figure] [Table]

Original: [Table] [Figure] [Table] [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: 🕵‍‍
Revised: 🕵‍‍

Original: [Table]
Revised: [Table]

Original: [Figure]
Revised: [Figure]

Original: (AEN) based accounting anomaly detection setting [64, 68].
Revised: (AEN) based accounting anomaly detection setting [64, 68].

Original: Journal entries corresponding to a high reconstruction error are selected for detailed audit procedures.
Revised: Journal entries corresponding to a high reconstruction error are selected for detailed audit procedures.

Original: from scratch on stationary client data of the in-scope audit period, e.g., a financial quarter or year [72].
Revised: scratch on stationary client data of the in-scope audit period, e.g., a financial quarter or year [72].

Original: Disregarding that organizations operate in environments in which activities rapidly and dynamically evolve [78, 77, 24].
Revised: Disregarding that organizations operate in environments in which activities rapidly and dynamically evolve [78, 77, 24].

Original: New business processes, models, or departments are constantly introduced, while current ones are redesigned or discontinued.
Revised: New business processes, models, or departments are constantly introduced, while current ones are redesigned or discontinued.

Original: Second, audit models are often trained centrally onthe data of a single client, e.g. , the organization ‘in-scope’ of the audit [24].
Revised: Second, audit models are often trained centrally on a single client’s data, e.g. , the organization ‘in-scope’ of the audit [24].

Original: Despite the fact, that audit firms often audit multiple organisations operating in the same industry [26].
Revised: Although audit firms often audit multiple organizations operating in the same industry [26].

Original: Such ‘peer audit clients’ [37] are affected by similar economic and societal factors, e.g. , supply-chains, market cycles [10].
Revised: Such ‘peer audit clients’ [37] are affected by similar economic and societal factors, e.g. , supply-chains, market cycles [10].

Original: In 1995 Thrun et al.
Revised: In 1995 Thrun et al.

Original: proposed a machine learning setting in which a model incrementally learns from a stream of experiences [75].
Revised: proposed a machine learning setting in which a model incrementally learns from a stream of experiences [75].

Original: The underlying idea of Continual Learning (CL) refers to a progressive model adaption; once new information becomes available [52].
Revised: The underlying idea of Continual Learning (CL) refers to a progressive model adaption; once new information becomes available [52].

Original: In 2017 McMahan proposed Federated Learning (FL) , a learning setting enabling distributed clients to collaboratively train models under the orchestration of a central server [46].
Revised: In 2017 McMahan proposed Federated Learning (FL) , a learning setting enabling distributed clients to collaboratively train models under the orchestration of a central server [46].

Original: A key idea of FL is creating learning synergies while preserving data privacy [17].
Revised: A key idea of FL is creating learning synergies while preserving data privacy [17].

Original: Ultimately, the Federated Continual Learning (FCL) of highly adaptive audit models can be viewed as a central objective of Continuous Assurance [78, 77].
Revised: Ultimately, the Federated Continual Learning (FCL) of highly adaptive audit models can be viewed as a central objective of Continuous Assurance [78, 77].

Original: However, learning such models in dynamic and decentralised audit settings remains challenging due to the stability-plasticity dilemma [22].
Revised: However, learning such models in dynamic and decentralised audit settings remains challenging due to the stability-plasticity dilemma [22].

Original: The FCL setting requires to cope with two types of data distribution shifts [20, 9, 30], as described in the following:
Revised: The FCL setting requires to cope with two types of data distribution shifts [20, 9, 30], as described in the following:

Original: • A temporal distribution shift between time step t 0 and t 1 denotes a change in distributions p t 0 ( X ) ̸
Revised: • A temporal distribution shift between time step t 0 and t 1 denotes a change in distributions p t 0 ( X ) ̸

Original: = p t 1 ( X ) .
Revised: = p t 1 ( X ) .

Original: In the CL setting, such shifts potentially result in the loss of a model’s ability to perform on p t 0 ( X ) after learning from p t 1 ( X ) referred to as catastrophic forgetting [36].
Revised: In the CL setting, such shifts potentially result in the loss of a model’s ability to perform on p t 0 ( X ) after learning from p t 1 ( X ) referred to as catastrophic forgetting [36].

Original: • A client distribution shift between client ω 0 and ω 1 denotes a divergence of the distributions p {+ω 
+}
Revised: • A client distribution shift between client ω 0 and ω 1 denotes a divergence of the distributions p {+ω 
+}

Original: ω
Revised: ( X )

Original: X ) ̸ = p ω  X ) .
Revised: ̸ = p ω 1 ( X ) .

Original: In the FL setting, such shifts cause the divergence of client models that eventually result in non-convergence of the server model referred to as model interference [32].
Revised: In the FL setting, such shifts cause the divergence of client models that eventually result in non-convergence of the server model referred to as model interference [32].

Original: In this work, we investigate the practical application of FCL in financial auditing to improve the assurance of financial statements.
Revised: In this work, we investigate the practical application of FCL in financial auditing to improve the assurance of financial statements.

Original: In summary, we present the following contributions:
Revised: In summary, we present the following contributions:

Original: • We propose a novel federated continual learning framework that enables auditors to incrementally learn industry-specific models from distributed data of multiple audit clients.
Revised: • We propose a novel federated continual learning framework that enables auditors to incrementally learn industry-specific models from distributed data of multiple audit clients.

Original: • We demonstrate that the framework enables audit models to retain previously learned knowledge that is still relevant and replace obsolete knowledge with novel information.
Revised: • We demonstrate that the framework enables audit models to retain previously learned knowledge that is still relevant and replace obsolete knowledge with novel information.

Original: • We conduct an extensive evaluation of scenarios of common organizational activity patterns using real-world datasets to illustrate the setting’s benefit in detecting accounting anomalies.
Revised: • We evaluate scenarios of common organizational activity patterns using real-world datasets to illustrate the setting’s benefit in detecting accounting anomalies.

Original: Ultimately, we view FCL as a promising,but under-explored learning setting in contemporary auditing.
Revised: Ultimately, we view FCL as a promising, under-explored learning setting in contemporary auditing.

Original: The remainder of this work is structured as follows:
Revised: The remainder of this work is structured as follows:

Original: In Section 2, we provide an overview of related work.
Revised: In Section 2, we provide an overview of related work.

Original: Section 3 follows with a description of the proposed framework to learn federated and privacy-preserving models from vast quantities of JE data.
Revised: Section 3 follows with a description of the proposed framework to learn federated and privacypreserving models from vast quantities of JE data.

Original: The experimental setup and results are outlined in Section 4 and Section 5.
Revised: The experimental setup and results are outlined in Section 4 and Section 5.

Original: In Section 6, the work concludes with a summary.
Revised: In Section 6, the work concludes with a summary.

Original:  Related Work
Revised:  Related Work

Original: The application of ML in financial audits triggered a sizable body of research by academia [2, 72] and practitioners [73, 16].
Revised: The application of ML in financial audits triggered a sizable body of research by academia [2, 72] and practitioners [73, 16].

Original: This section presents our literature study focusing on the (i) unsupervised, (ii) federated, and (iii) continual learning of accounting data representations.
Revised: This section presents our literature study focusing on the (i) unsupervised, (ii) federated, and (iii) continual learning of accounting data representations.

Original: Representation Learning denotes a machine learning technique that allows systems to discover relevant data features to solve a given task [6].
Revised: Representation Learning denotes a machine learning technique that allows systems to discover relevant data features to solve a given task [6].

Original: Nowadays, most ML methods used in financial audits depend on ‘human’ engineered data representations [13].
Revised: Nowadays, most ML methods used in financial audits depend on ‘human’ engineered data representations [13].

Original: Such techniques encompass, Naive Bayes
Revised: Such techniques encompass, Naive Bayes

Original: [Table]
Revised: [Table]

Original: Collaborative Client
Revised: Collaborative Client

Original: Client
Revised: Client

Original: [Table]
Revised: [Table]

Original: E
Revised: E

Original: [Table]
Revised: [Table]

Original: +75FKxWLxoGcJ9SI8FixkBGsj+XZlGGE9IZhnrbmfNea+XXVqzgJonbgFqUKBtm9/DUcxSSMqNOFYqYHrJNrLsNSMcDovD1NFE0ymeEwHhgocUeVli+BzdGGUEQpjaZ7QaKH+3shwpNQsCsxkHlOtern4nzdIdXjjZUwkqaaCLA+FKUc6RnkLaMQkJZrPDMFEMpMVkQmWmGjTVdmU4K5+eZ106zX3qla/b1Sbt0UdJTiDc7gEF66hCXfQhg4QSOEZXuHNerJerHfrYzm6YRU7p/AH1ucP6yuTQQ==</latexit> … … … … ✗ ✗ ✗ ✗ ✗ ✗ ✗ … … ✗ ✗ ✗ ✗ ✗ ✗ … … ✗ ✗ ✗ ✗ ✗✗ ✗ classification [5], network analysis [45], univariate and multivariate attribute analysis [3], cluster analysis [74], transaction log mining [33], or business process mining [29, 81].
Revised: +75FKxWLxoGcJ9SI8FixkBGsj+XZlGGE9IZhnrbmfNea+XXVqzgJonbgFqUKBtm9/DUcxSSMqNOFYqYHrJNrLsNSMcDovD1NFE0ymeEwHhgocUeVli+BzdGGUEQpjaZ7QaKH+3shwpNQsCsxkHlOtern4nzdIdXjjZUwkqaaCLA+FKUc6RnkLaMQkJZrPDMFEMpMVkQmWmGjTVdmU4K5+eZ106zX3qla/b1Sbt0UdJTiDc7gEF66hCXfQhg4QSOEZXuHNerJerHfrYzm6YRU7p/AH1ucP6yuTQQ==</latexit> … … … … ✗ ✗ ✗ ✗ ✗ ✗ ✗ … … ✗ ✗ ✗ ✗ ✗ ✗ … … ✗ ✗ ✗ ✗ ✗✗ ✗ classification [5], network analysis [45], univariate and multivariate attribute analysis [3], cluster analysis [74], transaction log mining [33], or business process mining [29, 81].

Original: With the advent of
Revised: With the advent of

Original: Deep Learning (DL), representation learning techniques have emerged in financial audits [72, 49, 55].
Revised: Deep Learning (DL), representation learning techniques have emerged in financial audits [72, 49, 55].

Original: Nowadays, the application of DL in auditing JEs encompasses, autoencoder neural networks [68], adversarial autoencoders [66], or variational autoencoders [87].
Revised: Nowadays, the application of DL in auditing JEs encompasses autoencoder neural networks [68], adversarial autoencoders [66], or variational autoencoders [87].

Original: Lately, self-supervised learning techniques have been proposed to learn representations for multiple audit tasks [62].
Revised: Lately, self-supervised learning techniques have been proposed to learn representations for multiple audit tasks [62].

Original: Continual Learning (CL) describes a setting where a model continuously trains on a sequence of experiences [75].
Revised: Continual Learning (CL) describes a setting where a model continuously trains on a sequence of experiences [75].

Original: The challenge of forgetting in model learning has been studied extensively [19, 15, 76, 40].
Revised: The challenge of forgetting in model learning has been studied extensively [19, 15, 76, 40].

Original: Rehearsal techniques replay samples from past experiences [59, 28, 12] to conduct knowledge distillation [56], while generative rehearsal techniques replay synthesised samples [69].
Revised: Rehearsal techniques replay samples from past experiences [59, 28, 12] to conduct knowledge distillation [56], while generative rehearsal techniques replay synthesized samples [69].

Original: Recent methods use gradients of previous tasks to mitigate gradient interference [44, 11].
Revised: Recent methods use gradients of previous tasks to mitigate gradient interference [44, 11].

Original: Regularisation techniques consolidate previously acquired knowledge and restrict parameter updates in model optimization.
Revised: Regularisation techniques consolidate previously acquired knowledge and restrict parameter updates in model optimization.

Original: LwF [42], uses knowledge distillation to regularize parameter updates.
Revised: LwF [42], uses knowledge distillation to regularize parameter updates.

Original: EWC [36] directly regularises the parameters based on their previous task importance.
Revised: EWC [36] directly regularises the parameters based on their previous task importance.

Original: In SI [85] regularization is applied in a separate parameter optimization step.
Revised: In SI [85], regularization is applied in a separate parameter optimization step.

Original: Dynamic architecture techniques prevent forgetting by parameter reuse or increase [60, 48] or.
Revised: Dynamic architecture techniques prevent forgetting by parameter reuse or increase [60, 48] or.

Original: Lately, CL has been deployed in a variety of application scenarios, such as, healthcare [39], machine translation [4], and financial auditing [24].
Revised: Lately, CL has been deployed in a variety of application scenarios, such as healthcare [39], machine translation [4], and financial auditing [24].

Original: Federated Learning (FL) enables entities to collaboratively train ML models under the orchestration of a central trusted entity under differential privacy [31, 17].
Revised: Federated Learning (FL) enables entities to collaboratively train ML models under the orchestration of a central trusted entity under differential privacy [31, 17].

Original: FedAvg [46] aggregates the client’s model parameters by computing a weighted average based on the number of training samples.
Revised: FedAvg [46] aggregates the client’s model parameters by computing a weighted average based on the number of training samples.

Original: PATE [51] aggregates knowledge transferred from a teacher model that is trained on separated data of student models.
Revised: PATE [51] aggregates knowledge transferred from a teacher model that is trained on separated data of student models.

Original: FedAdagrad , FedYogi , and FedAdam [57] use adaptive optimizer variations to improve FL convergence.
Revised: FedAdagrad , FedYogi , and FedAdam [57] use adaptive optimizer variations to improve FL convergence.

Original: FedProx [41] trains the local with a proximal term that restricts the updates to the central model.
Revised: FedProx [41] trains the local with a proximal term that restricts the updates to the central model.

Original: FedCurv [70] aims to minimise the model dispersity across clients by adopting a modified version of EWC [36].
Revised: FedCurv [70] aims to minimise the model dispersity across clients by adopting a modified version of EWC [36].

Original: Recent works [84, 79] also introduce Bayesian non-parametric aggregation policies.
Revised: Recent works [84, 79] also introduce Bayesian non-parametric aggregation policies.

Original: Nowadays, FL learning is applied in sensitive application scenarios, such as, healthcare [14], financial risk modelling [86], and financial auditing [63].
Revised: Nowadays, FL learning is applied in sensitive application scenarios, such as healthcare [14], financial risk modelling [86], and financial auditing [63].

Original: Federated Continual Learning (FCL) addresses the challenge of distributed distribution shifts over time and has only been preliminary studied [9].
Revised: Federated Continual Learning (FCL) addresses the challenge of distributed distribution shifts over time and has only been preliminary studied [9].

Original: SCAFFOLD [32] applies variance reduction in the local updates to correct for a client’s concept drifts.
Revised: SCAFFOLD [32] applies variance reduction in the local updates to correct for a client’s concept drifts.

Original: FedWeIT [82] decompose a model into global shared and sparse task-specific parameters to prevent inter-client interference.
Revised: FedWeIT [82] decompose a model into global shared and sparse task-specific parameters to prevent inter-client interference.

Original: CDA-FedAvg [9] extends FedAVG by distribution-based drift detection and a long-short term memory for rehearsal.
Revised: CDA-FedAvg [9] extends FedAVG by distribution-based drift detection and a long-short term memory for rehearsal.

Original: VER [53] conducts an additional server-side rehearsal-based training using a VAE’s [35] embedding statistics or actual embeddings.
Revised: VER [53] conducts an additional server-side rehearsal-based training using a VAE’s [35] embedding statistics or actual embeddings.

Original: FedDrift [30] uses local drift detection and hierarchical clustering to learn multiple global concept models.
Revised: FedDrift [30] uses local drift detection and hierarchical clustering to learn multiple global concept models.

Original: To the best of our knowledge, this work presents the first step towards the federated continual learning of highly adaptive audit models in financial auditing.
Revised: To the best of our knowledge, this work presents the first step towards the federated continual learning of highly adaptive audit models in financial auditing.

Original:  Methodology
Revised:  Methodology

Original: We consider an unsupervised audit learning setting where a tabular dataset D denotes a population of i = 1 , 2 ,...,I JEs.
Revised: We consider an unsupervised audit learning setting where a tabular dataset D denotes a population of i = 1 , 2 ,...,I JEs.

Original: Each entry x i = { x 1 i ,x 2 i ,...,x Ji ; x 1 i ,x 2 i ,...,x Ki } , consists of j = 1 , 2 ,...,J categorical accounting attributes and k = 1 , 2 ,...,K numerical attributes.
Revised: Each entry x i = { x 1 i ,x 2 i ,...,x Ji ; x 1 i ,x 2 i ,...,x Ki } , consists of j = 1 , 2 ,...,J categorical accounting attributes and k = 1 , 2 ,...,K numerical attributes.

Original: The individual attributes encompass the journal entry details, such as posting date, amount, or general ledger.
Revised: The individual attributes encompass the journal entry details, such as posting date, amount, or general ledger.

Original: Each JE is generated by a particular organizational activity c ℓ , e.g. , an organizational unit, entity or business process.
Revised: Each JE is generated by a particular organizational activity c ℓ , e.g. , an organizational unit, entity or business process.

Original: Furthermore, C = { c 1 ,c 2 ,...,c L } denotes the set of all ℓ = 1 , 2 ,...,L activities.
Revised: Furthermore, C = { c 1 ,c 2 ,...,c L } denotes the set of all ℓ = 1 , 2 ,...,L activities.

Original: In our audit setting we distinguish two types of ‘anomalous’ JEs [8] that auditors aim to detect [68, 87, 50]:
Revised: In our audit setting, we distinguish two types of ‘anomalous’ JEs [8] that auditors aim to detect [68, 87, 50]:

Original: • Global Anomalies correspond to entries that exhibit unusual or rare individual attribute values, e.g. , rarely used ledgers or unusual posting times.
Revised: • Global Anomalies correspond to entries that exhibit unusual or rare individual attribute values, e.g. , rarely used ledgers or unusual posting times.

Original: Such anomalies often correspond to unintentional mistakes, are comparably simple to detect, and possess a high error risk.
Revised: Such anomalies often correspond to unintentional mistakes, are comparably simple to detect, and possess a high error risk.

Original: • Local Anomalies correspond to entries that exhibit unusual attribute value correlations, e.g. , rare co-occurrences of ledgers and posting types.
Revised: • Local Anomalies correspond to entries that exhibit unusual attribute value correlations, e.g. , rare co-occurrences of ledgers and posting types.

Original: Such anomalies might correspond to intentional deviations, are comparably difficult to detect, and possess a high fraud risk.
Revised: Such anomalies might correspond to intentional deviations, are comparably difficult to detect, and possess a high fraud risk.

Original: Inspired by a human auditor’s learning process, we introduce an audit framework comprised of three interacting learning settings, namely (i) representation , (ii) continual , and (iii) federated learning.
Revised: Inspired by a human auditor’s learning process, we introduce an audit framework comprised of three interacting learning settings, namely (i) representation , (ii) continual , and (iii) federated learning.

Original: Figure 2 illustrates the interactions of the settingswhich are described in the following.
Revised: Figure 2 illustrates the interactions of the settings described in the following.

Original: First, in the representation learning setting Autoencoder Networks (AENs) [25] are trained to learn a comprehensive model f θ of a given accounting data distribution p ( D = { x 1 ,x 2 ,...,x I } ) .
Revised: First, in the representation learning setting Autoencoder Networks (AENs) [25] are trained to learn a comprehensive model f θ of a given accounting data distribution p ( D = { x 1 ,x 2 ,...,x I } ) .

Original: In general, the AEN architecture comprises two non-linear functions, usually neural networks referred to as encoder and decoder .
Revised: In general, the AEN architecture comprises two non-linear functions, usually neural networks referred to as encoder and decoder .

Original: The encoder q ψ ( · ) , with parameters ψ , learns a representation z i ∈ R d 1 of a given input x i ∈ R d 2 , where d 1 < d 2 .
Revised: The encoder q ψ ( · ) , with parameters ψ , learns a representation z i ∈ R d 1 of a given input x i ∈ R d 2 , where d 1 < d 2 .

Original: In an attempt to achieve x i ≈ ˜ x
Revised: In an attempt to achieve x i ≈ ˜ x

Original: i , the decoder p ϕ ( · ) , with parameters ϕ , learns a reconstruction ˜ x i ∈
Revised: i , the decoder p ϕ ( · ) , with parameters ϕ , learns a reconstruction ˜ x i ∈

Original: R k of the original input.
Revised: R k of the original input.

Original: Throughout the training process, the AEN model parameters θ : ψ ∪ ϕ are optimized ∀ x i ∈ D , as defined by [23]:
Revised: Throughout the training process, the AEN model parameters θ : ψ ∪ ϕ are optimized ∀ x i ∈ D , as defined by [23]:

Original: [Equation]
Revised: [Equation]

Original: (1) where θ ∗ denotes the optimal model parameters and x i a single JE.
Revised: (1) where θ ∗ denotes the optimal model parameters and x i a single JE.

Original: Upon successful training, the learning success is quantified by the model’s
Revised: Upon successful training, the learning success is quantified by the model’s

Original: reconstruction error
Revised: reconstruction error

Original: L Rec ( f θ , D , ˜ D ) given D and its reconstruction ˜
Revised: L Rec ( f θ , D , ˜ D ) given D and its reconstruction ˜

Original: D .
Revised: D .

Original: In our audit setting, similar to Hawkins et al.
Revised: In our audit setting, similar to Hawkins et al.

Original: [23], we interpret an individual JEs reconstruction error magnitude as its deviation from regular posting patterns.
Revised: [23], we interpret an individual JEs reconstruction error magnitude as its deviation from regular posting patterns.

Original: JEs that correspond to a high L Rec magnitude are selected for a detailed audit [68, 61] as shown in Fig.
Revised: JEs that correspond to a high L Rec magnitude are selected for a detailed audit [68, 61] as shown in Fig.

Original: Second, in the continual learning setting , an AEN model f θ learns from accounting data D that is observed as a stream of M disjoint experiences { E ( t ) } T t =1 [75].
Revised: Second, in the continual learning setting , an AEN model f θ learns from accounting data D that is observed as a stream of M disjoint experiences { E ( t ) } T t =1 [75].

Original: The data of the t -th experience D ( t ) encompasses data instances of ℓ different activities {C ( t ) ℓ } Lℓ =1 and an individual activity C ( t ) ℓ corresponds to ρ data observations { x( t ) ℓ,s } ρs
Revised: The data of the t -th experience D ( t ) encompasses data instances of ℓ different activities {C ( t ) ℓ } Lℓ =1 and an individual activity C ( t ) ℓ corresponds to ρ data observations { x

Original:  =1 .
Revised: [Equation] t ) ℓ,s } ρs =1 .

Original: With progressing experiences, the parameters θ ( t ) of a given AEN model are then continuously optimized, as defined by [19]:
Revised: With progressing experiences, the parameters θ ( t ) of a given AEN model are then continuously optimized, as defined by [19]:

Original: [Equation]
Revised: [Equation]

Original: where θ ∗ ( t − 1) denotes the optimal parameters of the previous experience E ( t − 1) .
Revised: where θ ∗ ( t − 1) denotes the optimal parameters of the previous experience E ( t − 1) .

Original: In our audit setting each experience dataset D ( t ) exhibits JEs of L activities {C ( t ) 1 , C ( t ) 2 ,..., C ( t ) L } , where in general D ( t 1 ) ̸
Revised: In our audit setting each experience dataset D ( t ) exhibits JEs of L activities {C ( t ) 1 , C ( t ) 2 ,..., C ( t ) L } , where in general D ( t 1 ) ̸

Original: = D ( t 2 ) given t 1 ̸
Revised: = D ( t 2 ) given t 1 ̸

Original: = t 2 .
Revised: = t 2 .

Original: Throughout the learning process, the distinct datasets D ( t ) may be non-iid and a model has only access to the data of an activity C ( t ) ℓ ∈ D ( t ) during the time of the t -th experience.
Revised: Throughout the learning process, the distinct datasets D ( t ) may be non-iid, and a model has only access to the data of an activity C ( t ) ℓ ∈ D ( t ) during the time of the t -th experience.

Original: Although, a single organizational activity C ℓ might generate JEs in multiple experiences.
Revised: Although, a single organizational activity C ℓ might generate JEs in multiple experiences.

Original: Third, in the federated learning setting , a central AEN model f ( t ) θ is collaboratively learned by M decentral clients { ξ ω } M ω =1 and coordinated by a server [31].
Revised: Third, in the federated learning setting , a central AEN model f ( t ) θ is collaboratively learned by M decentral clients { ξ ω } M ω =1 and coordinated by a server [31].

Original: In each experience E ( t ) , a single client ξ ω exhibits access only to its private data subset D ( t ) ω ∈ D ( t ) .
Revised: In each experience E ( t ) , a single client ξ ω exhibits access only to its private data subset D ( t ) ω ∈ D ( t ) .

Original: To initiate the learning a synchronous model optimisation protocol is established that proceeds in rounds r = 1 , 2 ,...,R .
Revised: To initiate the learning, a synchronous model optimisation protocol is established that proceeds in rounds r = 1 , 2 ,...,R .

Original: Each round, the server broadcasts its central AEN model f
Revised: Each round, the server broadcasts its central AEN model f

Original: ( t,r ) θ to a selection of available clients.
Revised: ( t,r ) θ to a selection of available clients.

Original: Subsequently, each client conducts a decentral training of the central model on its data subset D ( t ) ω .
Revised: Subsequently, each client conducts a decentral training of the central model on its data subset D ( t ) ω .

Original: Upon training completion, the clients send their updated models f ( t,r +1) θ,ω to the server.
Revised: Upon training completion, the clients send their updated models f ( t,r +1) θ,ω to the server.

Original: The server then aggregates the parameters of the decentral models to create an updated central AEN model f ( t,r +1) θ , as defined by [46]:
Revised: The server then aggregates the parameters of the decentral models to create an updated central AEN model f ( t,r +1) θ , as defined by [46]:

Original: [Equation]
Revised: [Equation]

Original: E E
Revised: E E

Original: E E
Revised: E E

Original: E 

Revised: E

Original: D ep t s .
Revised: D ep t s .

Original: Audit Client 

Revised: Audit Client

Original: E
Revised: E

Original: E E
Revised: E E

Original: E E 

Revised: E E

Original: Collaborative Client
Revised: Collaborative Client

Original: E E
Revised: E E

Original: E E
Revised: E E

Original: E 20 Experiences

Revised: E 20 Experiences

Original: D ep t s .
Revised: D ep t s .

Original: Collaborative Client
Revised: Collaborative Client

Original: E E
Revised: E E

Original: E E
Revised: E E

Original: E 20 Experiences

Revised: E 20 Experiences

Original: Collaborative Client1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 

Revised: Collaborative Client1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Original: R E C
Revised: R E C

Original: Audit Client 0 1 - Philadelphia:
Revised: Audit Client 0 1 - Philadelphia:

Original: FedAvg & Sequential Fine-Tuning     1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 

Revised: FedAvg & Sequential Fine-Tuning     1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Original: R E C
Revised: R E C

Original: Audit Client 0 1 - Philadelphia: FedProx & Elastic Weight Consolidation     1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Experiences
Revised: Audit Client 0 1 - Philadelphia: FedProx & Elastic Weight Consolidation     1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 Experiences

Original: R E C
Revised: R E C

Original: Audit Client 0 1 - Philadelphia: FedProx & Experience Replay     where |D ( t ) ω | denotes the number of data observations privately held by ξ ω .
Revised: Audit Client 0 1 - Philadelphia: FedProx & Experience Replay     where |D ( t ) ω | denotes the number of data observations privately held by ξ ω .

Original: In our audit setting, a central audit firm coordinates the learning as illustrated in Fig.
Revised: In our audit setting, a central audit firm coordinates the learning as illustrated in Fig.

Original: We distinguish two categories of decentral FCL clients.
Revised: We distinguish two categories of decentral FCL clients.

Original: First, an audit client ξ ′ 1 ‘in-scope’ or subject of the audit.
Revised: First, an audit client ξ ′ 1 ‘in-scope’ or subject of the audit.

Original: Second, a set of collaborating ‘peer’ or collaborative clients { ξ ω } M ω =2 contributing to the learning of a central audit model.
Revised: Second, a set of collaborating ‘peer’ or collaborative clients { ξ ω } M ω =2 contributing to the learning of a central audit model.

Original: In each experience, the central model is applied to detect anomalies in the audit client’s data.
Revised: In each experience, the central model is applied to detect anomalies in the audit client’s data.

Original:  Experimental Setup
Revised:  Experimental Setup

Original: In this section, we describe the (i) dataset, (ii) learning setting, and (iii) experimental details to detect accounting anomalies in an FCL setting.
Revised: In this section, we describe the (i) dataset, (ii) learning setting, and (iii) experimental details to detect accounting anomalies in an FCL setting.

Original: We provide additional experimental details in the appendix.
Revised: We provide additional experimental details in the appendix.

Original: City Payments Datasets: To evaluate the FCL capabilities and allow for reproducibility, we use three publicly available datasets of real-world payment data.
Revised: City Payments Datasets: To evaluate the FCL capabilities and allow for reproducibility, we use three publicly available datasets of real-world payment data.

Original: The datasets exhibit high similarity to
Revised: The datasets exhibit high similarity to

Original: ERP accounting data, e.g. , typical manual payments or payments runs (SAP T-Codes: F-35, F-110):
Revised: ERP accounting data, e.g. , typical manual payments or payments runs (SAP T-Codes: F-35, F-110):

Original: • The City of Philadelphia payments: 2 238,894 payments of 58 city departments in 2017.
Revised: • The City of Philadelphia payments: 2 238,894 payments of 58 city departments in 2017.

Original: • The City of Chicago payments: 3 108,478 payments of 54 city departments in 2021.
Revised: • The City of Chicago payments: 3 108,478 payments of 54 city departments in 2021.

Original: • The City of York payments: 4 102,026 payments of 49 city departments in 2020-22.
Revised: • The City of York payments: 4 102,026 payments of 49 city departments in 2020-22.

Original: Each dataset comprised of categorical and numerical attributes is described in the appendix.
Revised: Each dataset comprised of categorical and numerical attributes is described in the appendix.

Original: Federated Continual Learning: We establish, a systematic FCL setting encompassing (i) a central coordinating audit firm, (ii) a single decentral audit client ξ ′ 1 , and (iii) three decentral collaborating clients { ξ 2 ,ξ 3 ,ξ 4 } .
Revised: Federated Continual Learning: We establish a systematic FCL setting encompassing (i) a central coordinating audit firm, (ii) a single decentral audit client ξ ′ 1 , and (iii) three decentral collaborating clients { ξ 2 ,ξ 3 ,ξ 4 } .

Original: The M = 4 client models are learned in a federated setup, each over a different continual data stream {D ( t ) ω } T t =1 of T = 20 experiences [53].
Revised: The M = 4 client models are learned in a federated setup, each over a different continual data stream {D ( t ) ω } T t =1 of T = 20 experiences [53].

Original: Each experience, exhibits R = 5 learning rounds, while each round encompasses η = 1,000 training iterations.
Revised: Each experience, exhibits R = 5 learning rounds, while each round encompasses η = 1,000 training iterations.

Original: The data of an individual client experience D ( t ) ω , exhibits payments of L = 5 activities.
Revised: The data of an individual client experience D ( t ) ω , exhibits payments of L = 5 activities.

Original: A single experience activity C ω,ℓ ∈ D ( t ) ω corresponds to ρ = 1,000 randomly sampled payments generated by a particular city department.
Revised: A single experience activity C ω,ℓ ∈ D ( t ) ω corresponds to ρ = 1,000 randomly sampled payments generated by a particular city department.

Original: The experience streams {D ( t ) ω } T t =1 simulate data distribution shifts according to three common client activity scenarios.
Revised: The experience streams {D ( t ) ω } T t =1 simulate data distribution shifts according to three common client activity scenarios.

Original: Each scenario corresponds to a different sparse client activity, where payment data of individual city department activities C ℓ is randomly observed.
Revised: Each scenario corresponds to a different sparse client activity, where payment data of individual city department activities C ℓ is randomly observed.

Original: The distinct scenarios, illustrated in
Revised: The distinct scenarios, illustrated in

Original: Fig 2, are described in the following:
Revised: Fig 2, are described in the following:

Original: • [Scenario 1] simulates a situation where the audit client ξ ′ 1 exhibits sparse payment activities, e.g., due to discontinued or periodic business processes.
Revised: • [Scenario 1] simulates a situation where the audit client ξ ′ 1 exhibits sparse payment activities, e.g., due to discontinued or periodic business processes.

Original: The distribution shifts at ξ ′ 1 eventually yield catastrophic forgetting.
Revised: The distribution shifts at ξ ′ 1 eventually yield catastrophic forgetting.

Original: The activities of the collaborating clients { ξ ω } M ω =2 remain constant.
Revised: The activities of the collaborating clients { ξ ω } M ω =2 remain constant.

Original: • [Scenario 2] simulates a situation where collaborating clients { ξ ω } M ω
Revised: • [Scenario 2] simulates a situation where collaborating clients { ξ ω } M ω

Original: =
Revised: =

Original: 2 exhibits sparse payment activities, e.g., due to a carve-out or merger.
Revised: 2 exhibits sparse payment activities, e.g., due to a carve-out or merger.

Original: The distribution shifts at { ξ ω } M ω =2 eventually yield severe client model interference affecting ξ ′ 1 .
Revised: The distribution shifts at { ξ ω } M ω =2 eventually yield severe client model interference affecting ξ ′ 1 .

Original: The activities of the audit client ξ ′ 1 remain constant.
Revised: The activities of the audit client ξ ′ 1 remain constant.

Original: • [Scenario 3] simulates a situation where an audit client ξ ′ 1 and the collaborating clients { ξ ω } M ω
Revised: • [Scenario 3] simulates a situation where an audit client ξ ′ 1 and the collaborating clients { ξ ω } M ω

Original: =2 exhibit sparse payment activities.
Revised: =2 exhibit sparse payment activities.

Original: The scenario simulates a common setting in which client activities opt-in and opt-out with progressing experiences causing model interference and forgetting.
Revised: The scenario simulates a common setting in which client activities opt-in and opt-out with progressing experiences causing model interference and forgetting.

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: Accounting Anomaly Detection: In each experience, we inject 20 global and 20 local anomalies 5 into each payment activity C ξ ′ 1 ,ℓ of the audit client ξ ′ 1 ( ≈ 4% of each payment activity).
Revised: Accounting Anomaly Detection: In each experience, we inject 20 global and 20 local anomalies 5 into each payment activity C ξ ′ 1 ,ℓ of the audit client ξ ′ 1 ( ≈ 4% of each payment activity).

Original: To quantitatively assess the anomaly detection capability of the FCL audit setting, we determine the audit client’s average precision ( AP ) of the detected anomalies over the stream of learning experiences { E ( t ) } T t = [61].
Revised: To quantitatively assess the anomaly detection capability of the FCL audit setting, we determine the audit client’s average precision ( AP ) of the detected anomalies over the stream of learning experiences { E ( t ) } T t = [61].

Original:  Experimental Results
Revised:  Experimental Results

Original: In this section, we present the experimental results of scenarios 1-3.
Revised: In this section, we present the experimental results of scenarios 1-3.

Original: Our evaluation encompasses several federated (FL) and continual learning (CL) strategies.
Revised: Our evaluation encompasses several federated (FL) and continual learning (CL) strategies.

Original: For each strategy, we report the audit client’s ξ ′ 1 average global anomaly AP global and local anomaly AP local detection precision.
Revised: For each strategy, we report the audit client’s ξ ′ 1 average global anomaly AP global and local anomaly AP local detection precision.

Original: In Fig.
Revised: In Fig.

Original: 4, we report the [Scenario 1] audit client ξ ′ 1 anomaly detection results using FedAvg [47] model aggregation.
Revised: 4, we report the [Scenario 1] audit client ξ ′ 1 anomaly detection results using FedAvg [47] model aggregation.

Original: For all datasets, the CL strategies, Replay [56], LwF [42], and EWC [36], outperform the from Scratch [24] learning and Sequential fine-tuning baselines.
Revised: For all datasets, the CL strategies, Replay [56], LwF [42], and EWC [36], outperform the from Scratch [24] learning and Sequential fine-tuning baselines.

Original: The results show the ability of the CL strategies ability to mitigate catastrophic forgetting in a sparse audit client setting.
Revised: The results show the ability of the CL strategies ability to mitigate catastrophic forgetting in a sparse audit client setting.

Original: In Fig.
Revised: In Fig.

Original: [Table]
Revised: [Table]

Original: we report the [Scenario 2] audit client ξ ′ 1 anomaly detection results conducting sequential model fine-tuning.
Revised: we report the [Scenario 2] audit client ξ ′ 1 anomaly detection results conducting sequential model fine-tuning.

Original: For all datasets, the FL strategies, FedYogi [57], FedProx [41], and Scaffold [32], outperform the baseline of FedAvg [47].
Revised: For all datasets, the FL strategies, FedYogi [57], FedProx [41], and Scaffold [32], outperform the baseline of FedAvg [47].

Original: In addition, the FL strategies mitigate model interference closing the gap towards Single [63] client learning in a sparse collaborating client setting.
Revised: In addition, the FL strategies mitigate model interference closing the gap towards Single [63] client learning in a sparse collaborating client setting.

Original: [Table]
Revised: [Table]

Original: In Tab. 1, we report the [Scenario 3] audit client ξ ′ 1 anomaly detection results of combining FL and CL strategies.
Revised: In Tab. 1, we report the [Scenario 3] audit client ξ ′ 1 anomaly detection results of combining FL and CL strategies.

Original: The strategies yield a superior precision for both anomaly classeswhen compared to the FedAvg-Sequential baseline.
Revised: The strategies yield a superior precision for both anomaly classes compared to the FedAvgSequential baseline.

Original: The obtained results suggest that certain strategies are advantageous in detecting particular anomaly classes.
Revised: The obtained results suggest that certain strategies are advantageous in detecting particular anomaly classes.

Original: For smaller (larger) AEN models we observe, that FedProx ( Scaffold ) learning successfully regulates dissimilar model convergence.
Revised: For smaller (larger) AEN models, we observe that FedProx ( Scaffold ) learning successfully regulates dissimilar model convergence.

Original: The results provide initial empirical evidence, as illustrated in Fig.
Revised: The results provide initial empirical evidence, as illustrated in Fig.

Original: 3, that the strategies mitigate distributedcontinual distribution shifts in a ‘real-world’ audit setting.
Revised: 3, that the strategies mitigate distributed-continual distribution shifts in a ‘real-world’ audit setting.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we proposed financial audits as a novel application area of Federated Continual Learning (FCL).
Revised: In this work, we proposed financial audits as a novel application area of Federated Continual Learning (FCL).

Original: We demonstrated the benefits of such a learning setting by detecting accounting anomalies under different data distribution shifts usually observable in organizational activities.
Revised: We demonstrated the benefits of such a learning setting by detecting accounting anomalies under different data distribution shifts usually observable in organizational activities.

Original: We believe that
Revised: We believe that

Original: FCL will enable the audit profession to enhance its assurance services and thereby contributes to the integrity of financial markets.
Revised: FCL will enable the audit profession to enhance its assurance services and thereby contributes to the integrity of financial markets.

Original: In future work, we aim to investigate (i) novel learning strategies, (ii) potential attack scenarios, and (iii) the addition of differential privacy.
Revised: In future work, we aim to investigate (i) novel learning strategies, (ii) potential attack scenarios, and (iii) the addition of differential privacy.

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: The research conducted by Marco Schreyer was funded by the Mobi.
Revised: The research conducted by Marco Schreyer was funded by the Mobi.

Original: Doc mobility grant of the
Revised: Doc mobility grant of the

Original: University of St.Gallen (HSG) under project no.
Revised: University of St.Gallen (HSG) under project no.

Original: A Appendix
Revised: A Appendix

Original: In the appendix, we provide additional details of the datasets, data preprocessing, federated continual learning scenarios, and experimental setup applied to detect accounting anomalies in an FCL setting.
Revised: In the appendix, we provide additional details of the datasets, data preprocessing, federated continual learning scenarios, and experimental setup applied to detect accounting anomalies in an FCL setting.

Original: A.1 Datasets and Data Preprocessing:
Revised: A.1 Datasets and Data Preprocessing:

Original: In our experiements, we use three publicly available datasets of real-world financial city payment data.
Revised: In our experiments, we use three publicly available datasets of real-world financial city payment data.

Original: The datasets exhibit high similarity to ERP accounting data, e.g. , typical manual payments (SAP T-Code: F-53) or payment runs (SAP T-Code: F-110):
Revised: The datasets exhibit high similarity to ERP accounting data, e.g. , typical manual payments (SAP T-Code: F-53) or payment runs (SAP T-Code: F-110):

Original: • The City of Philadelphia payments 6 encompass a total of 238,894 payments generated by 58 city departments in 2017.
Revised: • The City of Philadelphia payments 6 encompass a total of 238,894 payments generated by 58 city departments in 2017.

Original: Each payment exhibits 10 categorical and 1 numerical attribute(s).
Revised: Each payment exhibits 10 categorical and 1 numerical attribute(s).

Original: • The City of Chicago payments 7 encompass a total of 108,478 payments generated by 54 distinct city departments in 2021.
Revised: • The City of Chicago payments 7 encompass a total of 108,478 payments generated by 54 distinct city departments in 2021.

Original: Each payment exhibits 6 categorical and 1 numerical attribute(s).
Revised: Each payment exhibits 6 categorical and 1 numerical attribute(s).

Original: • The City of York payments 8 encompass a total of 102,026 payments generated by 49 city departments in 2020-22.
Revised: • The City of York payments 8 encompass a total of 102,026 payments generated by 49 city departments in 2020-22.

Original: Each payment exhibits 11 categorical and 2 numerical attribute(s).
Revised: Each payment exhibits 11 categorical and 2 numerical attribute(s).

Original: For each dataset D , we pre-process the

Original:  original payment line- item attributes to (i) remove semantically redundant attributes and (ii) obtain an encoded representation of each payment.
Revised: For each dataset D , we pre-process the original payment line-item attributes to (i) remove semantically redundant attributes and (ii) obtain an encoded representation of each payment.

Original: The following preprocessing is applied to the distinct payment attributes of the datasets:
Revised: The following preprocessing is applied to the distinct payment attributes of the datasets:

Original: • For each categorical attribute in D , the attribute values x ki are converted into one-hot numerical tuples of bits ˆ x ki ∈ { 0 , 1 } υ , where υ denotes the number of unique attribute values in x k .
Revised: • For each categorical attribute in D , the attribute values x ki are converted into one-hot numerical tuples of bits ˆ x ki ∈ { 0 , 1 } υ , where υ denotes the number of unique attribute values in x k .

Original: • For each numerical attribute in D , the attribute values x li are scaled, according to scale ( x li ) =
Revised: • For each numerical attribute in D , the attribute values x li are scaled, according to scale ( x li ) =

Original: ( x li − min ( x l ) / ( max ( x l ) − min ( x l )) , based on the min and max attribute value in x l , where In the following, the pre-processed journal entries are denoted as ˆ x ∈ ˆ X and the corresponding individual attributes as ˆ x i =
Revised: ( x li − min ( x l ) / ( max ( x l ) − min ( x l )) , based on the min and max attribute value in x l , where In the following, the pre-processed journal entries are denoted as ˆ x ∈ ˆ X and the corresponding individual attributes as ˆ x i =

Original: { ˆ x 1 i , ˆ x 2 i ,..., ˆ x Ji ; ˆ x 1 i , ˆ x 2 i ,..., ˆ x Ki } , where the index j = 1 , 2 ,.
Revised: { ˆ x 1 i , ˆ x 2 i ,..., ˆ x Ji ; ˆ x 1 i , ˆ x 2 i ,..., ˆ x Ki } , where the index j = 1 , 2 ,...,

Original: ..,J denotes the categorical accounting attributes and the index k
Revised: J denotes the categorical accounting attributes.

Original:  = 1 , 2 ,...,K denotes the numerical attributes.
Revised: The index k = 1 , 2 ,...,K denotes the numerical attributes.

Original: A.2 Federated Continual Learning (FCL) Scenarios
Revised: A.2 Federated Continual Learning (FCL) Scenarios

Original: For each of the M federated clients { ξ ω } M ω =2 , we create a different city payment data stream encompassing
Revised: For each of the M federated clients { ξ ω } M ω =2 , we create a different city payment data stream encompassing

Original: T = 20 experiences.
Revised: T = 20 experiences.

Original: The data stream of single federated client ξ ω is defined by
Revised: The data stream of single federated client ξ ω is defined by

Original: D ω =
Revised: D ω =

Original: {D (1) ω , D (2) ω ,..., D (20) ω } .
Revised: {D (1) ω , D (2) ω ,..., D (20) ω } .

Original: For each single experience dataset D ( t ) ω , we randomly sample payments from
Revised: For each single experience dataset D ( t ) ω , we randomly sample payments from

Original: L = 5 distinct payment activities C ℓ .
Revised: L = 5 distinct payment activities C ℓ .

Original: Thereby, each payment activity corresponds to a different city department.
Revised: Thereby, each payment activity corresponds to a different city department.

Original: In our experiments, we sampled from the following city departments to create the data streams for each dataset respectively:
Revised: In our experiments, we sampled from the following city departments to create the data streams for each dataset respectively:

Original: • The selected City of Philadelphia departments are (i) ‘42 Commerce’, (ii) ‘52 Free Library’, (iii) ‘10 Managing Director’, (iv) ‘11 Police’, and (v) ‘14 Health’.
Revised: • The selected City of Philadelphia departments are (i) ‘42 Commerce’, (ii) ‘52 Free Library’, (iii) ‘10 Managing Director’, (iv) ‘11 Police’, and (v) ‘14 Health’.

Original: • The selected City of Chicago departments are (i) ‘Dept. of Family and Suppport Services’, (ii) ‘Dept. of Aviation’, (iii) ‘Chicago Department of Transportation’, (iv) ‘Department of Health’, and (v) ‘Department of Water Management’.
Revised: • The selected City of Chicago departments are (i) ‘Dept. of Family and Suppport Services’, (ii) ‘Dept. of Aviation’, (iii) ‘Chicago Department of Transportation’, (iv) ‘Department of Health’, and (v) ‘Department of Water Management’.

Original: • The selected City of York departments are (i) ‘Adult Social Care’, (ii) ‘Economy Regeneration and Housing’, (iii) ‘Housing and Community Safety’, (iv) ‘Transport Highways and Environ.’, and (v) ‘School Funding and Assets’.
Revised: • The selected City of York departments are (i) ‘Adult Social Care’, (ii) ‘Economy Regeneration and Housing’, (iii) ‘Housing and Community Safety’, (iv) ‘Transport Highways and Environ.’, and (v) ‘School Funding and Assets’.

Original: For each dataset, the selected departments exhibit a high payment posting activity.
Revised: For each dataset, the selected departments exhibit a high payment posting activity.

Original: Furthermore, the departments correspond to different municipal duties.
Revised: Furthermore, the departments correspond to different municipal duties.

Original: In total we created three FCL activity scenarios as described in Sec. 4 and illustrated in Fig.
Revised: In total, we created three FCL activity scenarios as described in Sec. 4 and illustrated in Fig.

Original: For each client payment activity C ( t ) ω,ℓ , the payments are sampled according to three scenario configurations, denoted as [Scenario 1-3] in the following.
Revised: For each client payment activity C ( t ) ω,ℓ , the payments are sampled according to three scenario configurations, denoted as [Scenario 1-3] in the following.

Original: Figure 6 illustrates the bar charts of each sampling configuration and FCL client.
Revised: Figure 6 illustrates the bar charts of each sampling configuration and FCL client.

Original: For each client configuration, a bar observable for client activity C ℓ and experience E ( t ) denotes that ρ = 1,000 city payments are sampled in the corresponding scenario.
Revised: For each client configuration, a bar observable for client activity C ℓ and experience E ( t ) denotes that ρ = 1, city payments are sampled in the corresponding scenario.

Original: E
Revised: E

Original: E E
Revised: E E

Original: E
Revised: E

Original: E 20 Experiences

Revised: E 20 Experiences

Original: D ep t s .
Revised: D ep t s .

Original: Audit Client 

Revised: Audit Client

Original: E
Revised: E

Original: E E
Revised: E E

Original: E
Revised: E

Original: E 20 Experiences

Revised: E 20 Experiences

Original: D ep t s .
Revised: D ep t s .

Original: Collaborative Client
Revised: Collaborative Client

Original: E E
Revised: E E

Original: E E
Revised: E E

Original: E 20 Experiences

Revised: E 20 Experiences

Original: D ep t s .
Revised: D ep t s .

Original: Collaborative Client
Revised: Collaborative Client

Original: E E
Revised: E E

Original: E E
Revised: E E

Original: E 20 Experiences

Revised: E 20 Experiences

Original: D ep t s .
Revised: D ep t s .

Original: Collaborative Client
Revised: Collaborative Client

Original: E E
Revised: E E

Original: E E
Revised: E E

Original: E 20 Experiences

Revised: E 20 Experiences

Original: D ep t s .
Revised: D ep t s .

Original: Audit Client 

Revised: Audit Client

Original: E
Revised: E

Revised: E

Original: E {+E E 20 Experiences
 D ep t s .+}
Revised: E {+E E 20 Experiences
 D ep t s .+}

Original:  E
Revised: Collaborative Client E E E

Original: E
Revised: E

Original: E 20 Experiences

Revised: E 20 Experiences

Original: D ep t s . Collaborative Client
Revised: D ep t s . Collaborative Client

Original: E E
Revised: E E

Original: E E
Revised: E E

Original: E 20 Experiences

Revised: E 20 Experiences

Original: D ep t s .
Revised: D ep t s .

Original: Collaborative Client EE
Revised: Collaborative Client E

Original: E[-E E 20 Experiences
 D ep t s .-]
Revised: E[-E E 20 Experiences
 D ep t s .-]

Original: Collaborative Client E E E
Revised: E

Original: E
Revised: E

Original: E 20 Experiences

Revised: E 20 Experiences

Original: D ep t s .
Revised: D ep t s .

Original: Audit Client 

Revised: Audit Client

Original: E
Revised: E

Original: E E
Revised: E E

Original: E
Revised: E

Original: E 20 Experiences

Revised: E 20 Experiences

Original: D ep t s .
Revised: D ep t s .

Original: Collaborative Client
Revised: Collaborative Client

Original: E E
Revised: E E

Original: E E
Revised: E E

Original: E 20 Experiences

Revised: E 20 Experiences

Original: D ep t s .
Revised: D ep t s .

Original: Collaborative Client
Revised: Collaborative Client

Original: E E
Revised: E E

Original: E E
Revised: E E

Original: E 20 Experiences

Revised: E 20 Experiences

Original: D ep t s .
Revised: D ep t s .

Original: [Figure]
Revised: [Figure]

Original: A.3 Representation Learning Setting Details
Revised: A.3 Representation Learning Setting Details

Original: We use a symmetrical encoder q ψ and decoder p ϕ architecture, in all our experiments.
Revised: We use a symmetrical encoder q ψ and decoder p ϕ architecture, in all our experiments.

Original: Furthermore, we applied two architectural setups, each designed to detect a different activity of injected anomalies in both datasets.
Revised: Furthermore, we applied two architectural setups, each designed to detect a different activity of injected anomalies in both datasets.

Original: A ‘shallow’ architecture, as shown in Tab. 2, is designed to detect global anomalies and a ‘deep’ architecture, as shown in Tab. 3, is designed to detect local anomalies.
Revised: A ‘shallow’ architecture, as shown in Tab. 2, is designed to detect global anomalies and a ‘deep’ architecture, as shown in Tab. 3, is designed to detect local anomalies.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: In both architectures, we apply Leaky-ReLU non-linear activations with a scaling factor α = 0 .except in the encoder’s bottleneck and decoder’s final layer, which comprise Tanh activations.
Revised: In both architectures, we apply Leaky-ReLU non-linear activations with a scaling factor α = 0 .except in the encoder’s bottleneck and decoder’s final layer, which comprise Tanh activations.

Original: We use a batch size of γ = 16 journal entries per training iteration η , apply Adam optimization [34] with β 1 = 0 .
Revised: We use a batch size of γ = 16 journal entries per training iteration η , apply Adam optimization [34] with β 1 = 0 .

Original: 9 , β 2 = 0 .
Revised: 9 , β 2 = 0 .

Original: 999 , and early stopping once the reconstruction loss L Rec converges.
Revised: 999 , and early stopping once the reconstruction loss L Rec converges.

Original: Given an encoded journal entry ˆ x i and its reconstruction ˜ x i = p ϕ ( q ψ ( x i )) , we compute a combined loss of a binary cross-entropy error L BCE loss and a mean squared error L MSE loss, as defined by [67]:
Revised: Given an encoded journal entry ˆ x i and its reconstruction ˜ x i = p ϕ ( q ψ ( x i )) , we compute a combined loss of a binary cross-entropy error L BCE loss and a mean squared error L MSE loss, as defined by [67]:

Original: L Rec ( f θ , { x i , ˜ x i } ) =
Revised: L Rec ( f θ , { x i , ˜ x i } ) =

Original: ϑ
Revised: ϑ

Original: [Equation]
Revised: [Equation]

Original: (cid:88)
Revised: (cid:88)

Original: [Equation]
Revised: [Equation]

Original: where J denotes the number of categorical attributes, K the number of numerical attributes, and ϑ balances the categorical and numerical attribute losses.
Revised: where J denotes the number of categorical attributes, K the number of numerical attributes, and ϑ balances the categorical and numerical attribute losses.

Original: For each categorical attribute ˆ x ji , we compute the L BCE , as defined by:
Revised: For each categorical attribute ˆ x ji , we compute the L BCE , as defined by:

Original: [Equation]
Revised: [Equation]

Original: where υ = 1 , 2 , ..., Υ corresponds to the number of one-hot encoded attribute dimensions of a given categorical attribute.
Revised: where υ = 1 , 2 , ..., Υ corresponds to the number of one-hot encoded attribute dimensions of a given categorical attribute.

Original: For each numerical attribute ˆ x ki , we compute the L MSE , as defined by:
Revised: For each numerical attribute ˆ x ki , we compute the L MSE , as defined by:

Original: [Equation]
Revised: [Equation]

Original: We balance both loss terms in L MSE to account for the high number of categorical attributes in each city payment dataset and set ϑ = 23 in all experiments.
Revised: We balance both loss terms in L MSE to account for the high number of categorical attributes in each city payment dataset and set ϑ = 23 in all experiments.

Original: A.4 Continual Learning Setting Details
Revised: A.4 Continual Learning Setting Details

Original: We set the number of CL experiences T = 20 in all our experiments.
Revised: We set the number of CL experiences T = 20 in all our experiments.

Original: In Experience Replay (ER) [59] learning, we set the replay buffer size N B = 1 , 000 samples, which seemed sufficient in all datasets.
Revised: In Experience Replay (ER) [59] learning, we set the replay buffer size N B = 1 , 000 samples, which seemed sufficient in all datasets.

Original: The buffer keeps a stratified sample of all departments observed until the current experience.
Revised: The buffer keeps a stratified sample of all departments observed until the current experience.

Original: For
Revised: For

Original: Elastic Weight Consolidation (EWC) [36] we found that α EWC = 500 preserves an optimal degree of plasticity.
Revised: Elastic Weight Consolidation (EWC) [36], we found that α EWC = 500 preserves an optimal degree of plasticity.

Original: For Learning without Forgetting (LwF) [42] we determined that a
Revised: For Learning without Forgetting (LwF) [42] we determined that a

Original: α LwF = 1 . 2 yields a good knowledge distillation.
Revised: α LwF = 1 . 2 yields a good knowledge distillation.

Original: We reset the optimizer parameters upon each experience to avoid past experience information transfer through the optimizer state.
Revised: We reset the optimizer parameters upon each experience to avoid past experience information transfer through the optimizer state.

Original: We adapted the different CL strategies from the algorithmic implementations of ER, LwF, and EWC of the Avalanche v0.2.1 9 library [43].
Revised: We adapted the different CL strategies from the algorithmic implementations of ER, LwF, and EWC of the Avalanche v0.2.1 9 library [43].

Original: A.5 Federated Learning Setting Details
Revised: A.5 Federated Learning Setting Details

Original: The models are trained for R = 5 communication rounds per experience, while each round encompasses η = 1 , 000 training iterations.
Revised: The models are trained for R = 5 communication rounds per experience, while each round encompasses η = 1 , 000 training iterations.

Original: In each experiment, we set the number of participating clients to M = 4 assuming a real-world peer audit client setting.
Revised: In each experiment, we set the number of participating clients to M = 4 assuming a real-world peer audit client setting.

Original: In FedProx [41] learning, we found that setting the proximal term to µ = 1.2 yields a good regularization of the local model updates.
Revised: In FedProx [41] learning, we found that setting the proximal term to µ = 1.2 yields a good regularization of the local model updates.

Original: For the SCAFFOLD [32] algorithm, we use the Option II implementation proposed in [32] by re-using previously computed gradients to update the control variate c + i .
Revised: For the SCAFFOLD [32] algorithm, we use the Option II implementation proposed in [32] by re-using previously computed gradients to update the control variate c + i .

Original: In all our experiments we built upon the FedAvg strategy and the FL framework implementation of the Flower v0.19.0 10 library [7].
Revised: In all our experiments we built upon the FedAvg strategy and the FL framework implementation of the Flower v0.19.0 10 library [7].

Original: A.6 Anomaly Detection Details
Revised: A.6 Anomaly Detection Details

Original: In each experience, we randomly inject 20 global and 20 local anomalies into each payment activity
Revised: In each experience, we randomly inject 20 global and 20 local anomalies into each payment activity

Original: C ξ ′ 1 ,ℓ of the in-scope audit client ξ ′ 1 ( ≈ 4% of each payment activity).
Revised: C ξ ′ 1 ,ℓ of the in-scope audit client ξ ′ 1 ( ≈ 4% of each payment activity).

Original: To create both classes of anomalies, we use the Faker v14.2.0 11 library [18] using five distinct random seed initialisation of the random anomaly sampling mechanism.
Revised: To create both classes of anomalies, we use the Faker v14.2.0 11 library [18] using five distinct random seed initialisation of the random anomaly sampling mechanism.

Original: To quantitatively assess the anomaly detection capability of the FCL audit setting, we determine the in-scope audit client’s Average Precision ( AP ) over the sorted payments reconstruction errors.
Revised: To quantitatively assess the anomaly detection capability of the FCL audit setting, we determine the in-scope audit client’s Average Precision ( AP ) over the sorted payments reconstruction errors.

Original: The AP summarises the precision-recall curve as defined by:
Revised: The AP summarises the precision-recall curve as defined by:

Original: [Equation]
Revised: [Equation]

Original: where P i ( L Rec ) = TP/ ( TP + FP ) denotes the detection precision, and R i
Revised: where P i ( L Rec ) = TP/ ( TP + FP ) denotes the detection precision, and R i

Original: ( L Rec )
Revised: ( L Rec )

Original: = TP/ ( TP + FN ) denotes the detection recall of the i- th reconstruction error threshold.
Revised: = TP/ ( TP + FN ) denotes the detection recall of the i- th reconstruction error threshold.