Automatic Control Group Generation

Techniques are disclosed for automatically generating and updating a control group. In disclosed techniques, a server computer system trains, using a plurality of transactions, a machine learning model. During training the machine learning model learns a feature distribution of both a current set of control group (CG) transactions and a current set of non-control group (non-CG) transactions included in the plurality of transactions. The system inputs the current set of CG transactions into the trained machine learning model. Based on the output of the trained machine learning model for the current set of CG transactions, the system modifies the current set of CG transactions to generate an updated set of CG transactions. Based on the updated set of CG transactions, the server performs one or more preventative measures for a transaction processing system. The disclosed techniques may advantageously improve the accuracy e.g., of a transaction processing system.

BACKGROUND

Technical Field

This disclosure relates generally to data security, and, more specifically, to techniques for automatically detecting anomalous behavior e.g., for improved security.

Description of the Related Art

As more and more transactions are conducted electronically via online transaction processing systems, for example, these processing systems become more robust in managing transaction data as well as detecting suspicious and unusual behavior. Many transaction requests, for example, may be generated with malicious in intent, which may result in wasted computer resources, network bandwidth, storage, CPU processing, monetary resources, etc., if those transactions are processed. Some transaction processing systems attempt to analyze various transaction data for previously processed and currently initiated transactions to identify and mitigate malicious behavior such as requests for fraudulent transactions.

DETAILED DESCRIPTION

Traditionally, control groups have been used during experimentation for comparison purposes to test the overall effectiveness of a new feature, characteristic, drug, etc. being introduced to an experimental group. As such, the accuracy of such experimentation depends on the representativeness of control group of examples relative to an overall population of examples. In the context of machine learning, control groups may be used to both train and test the overall accuracy of a machine learning model. Over time, however, a control group representing a given population (e.g., of users, transactions, patients, etc.) may no longer be representative of the overall population. For example, populations are generally temporal in nature and, as such, change with time. As one specific example, a population of transactions may increase in volume (e.g., during holiday months, the number of online electronic transactions increases significantly relative to non-holiday months), the types of transactions being conducted may change, etc.

In addition to becoming less representative over time, in some situations, control groups may introduce loss. In the context of online electronic transactions, as the overall population of transactions grows, the potential for loss associated with transactions that are included in the control group for this overall population increases. For example, because fraudulent transactions are often included in the control group (to represent that fraudulent transactions occur within the overall population) and because transactions included in the control group are automatically approved (authorized to proceed), these transactions cause a system processing such transactions to incur loss (e.g., financial loss). In this example, if one or more fraudulent transactions included in the control group are for a high dollar amount relative to other transactions, these transactions cause the transaction processing system to incur even greater loss than if such transactions were for a lower dollar amount.

The disclosed techniques use machine learning techniques to automatically generate and update control groups such that these control groups accurately represent the overall population they are intended to represent. In addition, while updating a control group, the system selects examples for the control group based on a particular feature. In the context of online electronic transactions, the system selects transactions for a control group based on a dollar amount feature in addition to selecting transactions that are highly representative of the overall transaction population to avoid loss associated with this feature. In particular, the disclosed techniques combine a neural network (e.g., a Dragonnet) with a VAE to learn the feature distribution of both a current set of control group (CG) of transactions and non-control group (non-CG) transactions (the rest of the transaction population that is not included in the current control group). During training, the neural network calculates propensity scores for transactions to predict whether these transactions are likely to be CG or non-CG transactions. As part of the propensity score calculation, the neural network also uses weights that are based on a dollar amount optimality (e.g., fraudulent transactions with low dollar amounts are weighted such that they are predicted to be control group transactions). Based on the prediction from the neural network, transactions are sent through either a CG portion of the neural network or a non-CG portion of the neural network. These two separate portions learn the feature distribution of the CG transactions and non-CG transactions, respectively. Based on the neural network knowing the feature distribution of non-CG population, the disclosed system uses the trained neural network to evaluate whether CG transactions are indeed representative of the overall transaction population and to alter the control group accordingly.

In some situations, traditional models used to automatically select transactions for a control group become biased over time. For example, if a control group selection model is more likely to select a fraudulent transaction for a control group than a non-fraudulent transaction, then this control group selection model has become biased when selecting transactions. The disclosed machine learning model used to select control group transactions alleviates model bias by learning the feature distribution of a current control group as well as the feature distribution of the overall transaction population (non-control group transactions) and evaluating transactions in the current control group using the portion of the machine learning model that has learned the feature distribution of the overall transaction population. As such, the disclosed machine learning model is able to accurately select representative examples for a control group (as well as remove unrepresentative examples).

In various situations, control groups may be used to provide a set of examples (e.g., 1.5-2.5% of the total population of examples) that is adequately representative of the overall population of examples. Further, control groups may be used to measure various benchmarks. In the context of electronic transactions, a control group may be used to measure: loss rates (e.g., how much money is PayPal losing on a daily, monthly, yearly, etc. basis), how well sub-populations are responding to fraud prevention measures compared to other sub-populations (e.g., transactions initiated in North America vs. transactions initiated in South America), whether greater numbers of fraud are occurring in a first geographic region as compared to a second geographic region, etc. Further, in the context of electronic transactions, the control group may be used to train a classifier model to classify transactions (as suspicious or not).

The disclosed techniques may advantageously improve the representativeness of control groups relative to the overall population of examples the control group is attempting to represent. In the context of online electronic transaction, this may, in turn, advantageously improve transaction security. For example, transaction classifiers trained using a control group of transactions generated using the disclosed techniques will more accurately detect suspicious or fraudulent transactions relative to transaction classifiers trained using control groups selected via traditional techniques (e.g., manually). In this example, the disclosed techniques decrease loss (e.g., financial) due to the higher catch rate of transaction classifiers trained using an automatically selected control group. Further in this example, the disclosed techniques decrease financial loss due to lower dollar amount transactions being included in the control group. In the context of a clinical trial, a control group of users selected using the disclosed techniques may advantageously be used to more accurately determine patients receive a drug (treatment group) and which patients will receive a placebo (control group).

As used herein, the term “control group” is intended to be construed according to its well-understood meaning in the context of machine learning, which includes a subset of a set of data that is representative of the set of data and that is used to train machine learning models. For example, a control group may include labeled transactions that have been authorized such that classifications for the transactions (e.g., fraudulent or non-fraudulent) are known (e.g., enough time has passed that fraudulent transactions included in this subset of transactions have been reported as fraudulent). In disclosed techniques, transactions included in a control group are selected from an overall population of transactions (e.g., transactions in the control group make up a portion of the overall transaction population). In disclosed techniques, a control group (including both fraudulent and non-fraudulent transactions) as well as a subset (including fraudulent transactions) of the non-control group transaction population are used to train a machine learning classifier to classify transactions. Once the classifier is trained, the disclosed techniques test the accuracy of this classifier using only transactions in the control group (both fraudulent and non-fraudulent). In some embodiments, transactions in the control group used for training are “out-of-time” transactions. For example, a first set of transactions included in the control group have timestamps in the year 2020, while a second set of transactions included in the control group have timestamps in the year 2021. In this example, control group transactions in the year 2020 are used to train the classifier, while control group transactions in the year 2021 are used to test the classifier.

Example Control Group Generation System

FIG.1block diagram illustrating an example system configured to automatically generate control groups. In the illustrated embodiment, system100includes one or more computing devices110, database150, server computer system120, which in turn includes control group selection module130, machine learning classifier140, and trained machine learning classifier145.

In the illustrated embodiment, server computer system120receives requests102to initiate transactions from one or more computing device110. For example, computing devices110are user computing devices (e.g., a cellular device, desktop computer, a tablet, a wearable device, etc.) and requests102are requests to initiate one or more online electronic transactions. In the illustrated embodiment, server computer system120inputs requested transactions into trained machine learning classifier145. Based on classifications output by trained machine learning classifier145for the requests transactions, server computer system120sends transaction decisions122to one or more computing devices110. Transaction decisions122indicate whether the requests102for transactions are authorized (transactions are allowed to proceed) or not authorized (transactions are rejected).

In order to generate trained machine learning classifier145, server computer system120in the illustrated embodiment retrieves transactions162from database150. Transactions162are completed transactions that have been authorized by server computer system120. Transactions162include both fraudulent and non-fraudulent transactions. Transactions162make up the general population of transactions (e.g., for PayPal™) that are completed transactions (e.g., authorized and finalized transactions and rejected and terminated transactions). Server computer system120selects a subset of transactions162to be a control group for the overall transaction population. Transactions162stored in database150include known labels (e.g., tags indicating whether these transactions are fraudulent or not). For example, database150may store transactions that were authorized and allowed to proceed, but were later determined to be fraudulent and labeled as such. As another example, database150may store transactions that were approved and were later confirmed to be not fraudulent and are, therefore, stored with a non-fraudulent label. Database150may also store various metadata (e.g., features) for transactions162that may be used by system120when training machine learning classifier140and when generating a control group. Server computer system120executes control group selection module130to train a machine learning model160using transactions162. Machine learning model160may be used to generated control groups. This model160may be a Dragonnet model combined with a variational auto encoder (VAE), for example. Model160may be any of various types of machine learning models or combinations of machine learning models, including neural networks, regression models, decision trees, etc. Model160may be combined with other types of auto encoders other than VAEs including regularized autoencoders, concrete autoencoders, etc. The machine learning model160is described in detail below with reference toFIGS.2,3, and5.

Control group selection module130, in the illustrated embodiment, generates an updated control group134of transactions from a current set of control group transactions output by machine learning model160. Server computer system120, in the illustrated embodiment, trains machine learning classifier140using the updated control group134. Once server computer system120is satisfied with the training of machine learning classifier140, system120executes trained machine learning classifier145to classify transactions.

During training of machine learning model160, control group selection module130generates an updated control group134by adding or removing, or both transactions from a current set of control group transactions selected by model160during training based on learning the feature distribution of the selected set of control group transactions and the non-control group transactions selected by model160.

In this disclosure, a “module” operable to perform designated functions are shown in the figures and described in detail (e.g., control group selection module130). As used herein, a “module” refers to software or hardware that is operable to perform a specified set of operations. A module may refer to a set of software instructions that are executable by a computer system to perform the set of operations. A module may also refer to hardware that is configured to perform the set of operations. A hardware module may constitute general-purpose hardware as well as a non-transitory computer-readable medium that stores program instructions, or specialized hardware such as a customized ASIC.

Although the disclosed techniques are generally described with reference to transactions, the disclosed machine learning techniques may be implemented to select any of various types of examples for control groups, including, for example, medicine in a clinical trial, fertilizers for plant growth trials, individuals to consume food in food sensitivity tests, etc. In some situations, the disclosed machine learning techniques may be implemented to perform reject inferencing for credit card applications. In such situations, building a control group may be expensive since including credit card applications that have been approved but turn out to be malicious or credit card applications from individuals having a low credit score may cause a credit provider to incur financial loss. Further, declined credit card applications are often under-represented (or, in some cases, not represented at all). As such, the disclosed techniques may be implemented to derive a control group that includes example credit card applications that are cost effective while also being the most representative of the declined (rejected, potentially fraudulent credit card applications) when assessing credit-worthiness.

Example Control Group Selection Module

Turning now toFIG.2, a block diagram is shown illustrating example training of a representation model. In the illustrated embodiment, a training example202is shown in which control group selection module130trains representation model260(a machine learning model) to identify control group transactions and non-control group transactions. Control group selection module130inputs transactions162into representation model260which outputs both reconstructed CG transactions222and reconstructed non-CG transactions242. In some embodiments, representation model260is a Dragonnet model combined with a VAE.

Representation model260, in the illustrated embodiment, includes propensity model layers210, CG branch220, classification branch230, and non-CG branch240. Representation model260receives transactions162as input during training. The propensity model layers210of representation model260include a neural network layer that calculates propensity scores for transactions162. Based on these propensity scores, propensity model layers210predict whether transactions162are either CG transactions212or non-CG transactions216. Representation model260sends the predicted CG transactions212to the CG branch220, the non-CG transactions216to the non-CG branch240, and both types of transactions212and216to classification branch230. (In this way, CG and non-CG branches220and240of the representation model260are conditioned on the propensity score.)

In some embodiments, during training, representation model260predicts, based on a predetermined weight associated with a particular transaction feature, whether transactions included in the plurality of transactions are CG transactions212or non-CG transactions216. During training, control group selection module130weights certain features of transactions162prior to inputting these transactions into propensity model layers210. For example, control group selection module130artificially weights transactions162based on the values of a dollar amount feature of these transactions. As one specific example, module130may assign higher weights to transactions that have a low dollar amount feature. In this specific example, the assigned weights cause the propensity model layers210to learn to put more emphasis on these transactions, such that representation model260is more likely to classify such transactions as CG transactions212. As another specific example, control group selection module130may assign higher weights to a dollar amount feature itself of a given transaction (rather than assigning a weight to the given transaction).

Note that, the weighting performed by module130is a way of artificially constraining representation model260when classifying transactions, to keep the model from transactions with undesirable features. For example, weighting may prevent the model from selecting high-dollar fraudulent transactions to be control group transactions. After weights are assigned to various transactions (e.g., based on the value of a dollar amount feature), transactions included in the control group do not have the same weights and, therefore, the multi-variable (feature) distribution of the control group is diverse (e.g., the representation model260will train harder on some example transactions than others during training). Control group selection module130may similarly weight any of various features of transactions, such that representation model260trains harder on such features or transactions that include certain values for those features (e.g., a location, an IP address, a type of transaction, etc.).

Propensity model layers210execute cost functions during training to predict whether transactions are CG or non-CG. In particular, the cost function executed by propensity model layers210may be experimented to be weighted by predetermined weights (e.g., to rectify under-representation of sparsely represented examples) as well as by the value of a dollar amount feature. The cost function might be optimized based on various different underlying objectives (e.g., weighting low-dollar value transactions greater than high-dollar value transactions). In some situations, the cost function is a hybridized set of loss functions that are applicable to a cohort of training examples (e.g., transactions) that can be optimized. In this way, the disclosed techniques not only discover which transaction examples are the most representative of the overall transaction population, but also the transaction examples that are the most cost-effective. This is particularly true given that transactions allocated to control groups are not declined, even if fraudulent. As one example, control group selection module130may assign predefined control group weights to transactions during the propensity score calculation performed by propensity model layers210to cause representation model260to train harder on under-represented types of transactions. Propensity model layers210are discussed in further detail below with reference toFIG.5.

Classification branch230of representation model260determines tags (i.e., classifications) for both CG transactions212and non-CG transactions216. For example, classification branch230determines whether transactions predicted as CG or non-CG by propensity model layers210are fraudulent or not. For example, classification branch230determines CG tags232for respective CG transactions indicating whether these transactions are fraudulent or not. Similarly, classification branch230determines non-CG tags234for respective non-CG transactions indicating whether these transactions are fraudulent or not. Classification branch230sends CG tags232corresponding to respective CG transactions212to CG branch220and sends non-CG tags234corresponding to respective non-CG transactions216to non-CG branch240.

CG branch220of representation model260receives CG transactions212from propensity model layers210and CG tags232from classification branch230and generates reconstructed CG transactions222. In this way, the CG branch220of representation model260learns the multi-variable distribution of control group transactions. For example, CG branch220includes a variational auto encoder that encodes features of CG transactions212using an encoder, learns the distribution of these features while they are compressed, and then reconstructs the CG transactions212using a decoder. In some embodiments, representation model260concatenates CG tags232to CG transactions212as they are input to CG branch220. For example, a CG tag232corresponding to a given CG transaction212will be assigned to that transaction prior to being input to CG branch220.

In some embodiments, during training, control group selection module130compares the CG tags232and non-CG tags234output by classification branch230with known labels for respective transactions. Based on tags output by classification branch230not matching (or being more than a threshold amount different from) the known labels for CG transactions212and the known labels for non-CG transactions, control group selection module130may reinforce the learning of the classification branch230to improve the classification accuracy of representation model260. That is, control group selection module130may decide to train representation model260further based on this model exhibiting poor classification performance.

Similar to the CG branch220, the non-CG branch240attempts to learn the feature distribution of non-CG transactions216by encoding and then decoding these transactions to produce reconstructions242of non-CG transactions. In addition, representation model260concatenates the non-CG tags234output by classification branch230to non-CG transactions216prior to these transactions being input to non-CG branch240.

In some embodiments, representation model260described above with reference toFIG.2includes two separate neural networks that are trained using similar techniques to those discussed below with reference to a single neural network model (e.g., a Dragonnet model) and executed in combination to achieve a similar outcome to a single, multi-branched model. For example, the model260shown inFIG.2might be implemented using two neural networks, where a first neural network executes the propensity model layers210and the CG branch220, while a second neural network executes the propensity model layers210and the non-CG branch240.

FIG.3is a block diagram illustrating example execution of a trained representation model. In the illustrated embodiment, a trained model execution example304is shown in which control group selection module130executes a trained representation model365(the trained version of the representation machine learning model260discussed above with reference toFIG.2).

In the illustrated embodiment, example304shows the situation in which control group selection module130inputs transactions362(which might be the same as transactions162) to trained representation model365. The propensity model layers210predict which of the transactions362are CG transactions312. Control group selection module130then causes trained representation model365to input CG transactions312into the non-CG branch240. Non-CG branch240outputs a reconstruction344of the CG transactions312. Non-CG branch240reconstructs the CG transactions312by feeding the CG transactions through an encoder and decoder pipeline that previously learned the distribution of non-CG transactions. If the non-CG branch240is able to accurately reconstruct the CG transactions312, then these transactions are representative of the overall transaction population. Said another way, if non-CG branch240, which knows the feature distribution of non-CG transactions, is able to recreate CG transactions, then these CG transactions have the same or similar feature distribution to non-CG transactions. The determination of whether the reconstructions344of CG transactions312, generated by non-CG branch240, are similar to the original CG transactions312is discussed in detail below with reference toFIG.4.

Example Divergence

Turning now toFIG.4, a block diagram is shown illustrating an example divergence module. In the illustrated embodiment, server computer system120includes machine learning classifier140and control group selection module130, which in turn includes trained representation model365, reconstruction module430, control group alteration module420, and divergence module410.

In the illustrated embodiment, control group selection module130executes trained representation model365by inputting transactions562(which might be the same as transactions162and/or362) into the model265. Control group selection module130then inputs the reconstruction244of CG transactions312output by trained representation model365and the CG transactions312predicted by propensity model layers210of model265(such as those shown inFIG.3) into reconstruction module430.

Reconstruction module430, in the illustrated embodiment, determines reconstruction error432for one or more of the reconstructions244of CG transactions312generated by the non-CG branch240of model265(as shown inFIG.3). For example, reconstruction module430determines a different between CG transactions312and their corresponding reconstructions244. The reconstruction error432output by reconstruction module430indicates the error of the non-CG branch240when reconstructing CG transactions312. In this example, any CG transactions that the non-CG branch of trained representation model365is not able to reconstruct within some threshold accuracy is not representative of the overall population of transactions.

Control group alteration module420, in the illustrated embodiment, removes transactions from the current set of CG transactions312based on these transactions having a threshold amount of reconstruction error432. Said another way, if the non-CG branch was not able to accurately reconstruct various CG transactions, then these transactions may be removed from the current control group. Transactions having the least amount of reconstruction error will be more representative of the overall transaction population than transactions with a greater amount of reconstruction error. In this way, control group alteration module420identifies and selects a subset of transactions from a current set of CG transactions312, adds additional non-CG transactions, removes unrepresentative transactions, etc. to generate an altered control group422of transactions562(i.e., control group alteration module420selects a set of transactions from the general transaction population that are the most representative of the overall transaction population).

Control group selection module130, in the illustrated embodiment, inputs the altered control group422into divergence module410. Divergence module410determines various divergence scores412for the current set of CG transactions312and the altered control group422generated by control group alteration module420. Divergence module410executes a divergence algorithm to determine a difference between a current set of CG transactions312and non-CG transactions (transactions not included in the control group). Divergence module410also executes a divergence algorithm to determine a difference between the altered control group422and non-CG transactions. For example, control group selection module130performs a verification process for the altered control group422prior to using this control group for training, testing, etc. In this example, divergence module410may execute a Kullback-Leibler (KL) divergence algorithm to measure the difference between two probability distributions (a current control group and the overall non-CG population as well as the altered control group422and the overall non-CG population).

Control group alteration module420may compare the divergence scores412output by divergence module410for the current set of CG transactions312and the altered control group422to ensure that the altered control group422did indeed improve the representativeness of the control group relative to the original (current set) of CG transactions312. In this way, module130ensures that the updates to the control group (e.g., adding or removing example transactions) have not significantly increased the divergence between the CG and non-CG populations (relative to the divergence between the original CG transaction and non-CG transactions), but rather have decreased (improved) the divergence.

In the illustrated embodiment, control group alteration module420outputs an updated control group134. The updated control group134may be the same as altered control group422or may be a further altered version of altered control group422. In some embodiments, based on comparing the two divergence scores412(e.g., divergence of the altered control group has increased relative to the divergence measured between the original control group and the non-CG population), control group alteration module420performs additional alterations to the updated control group134. For example, based on comparing the two divergence scores412, control group alteration module420may further determine to remove and/or add transactions to altered control group422to generate updated control group134. Server computer system120, in the illustrated embodiment, uses the transactions in the updated control group134to train a machine learning classifier140as discussed above with reference toFIG.1.

Turning now toFIG.5, a block diagram is shown illustrating an example Dragonnet VAE with multiple different branches. Dragonnet VAE model590, in the illustrated embodiment, includes non-fraud CG branch540, fraud CG branch550, non-fraud non-CG branch560, fraud non-CG branch570, and neural network layers510(one example of propensity model layers210), which in turn include a CG classification layer520, a fraud classification layer525, and an objective function layer530. In some embodiments, the Dragonnet VAE model590includes multiple separate VAE branches for reconstructing and learning the feature distribution of respective combinations of fraudulent, non-fraudulent, control group, and non-control group transactions.

Dragonnet VAE model590, in the illustrated embodiment, receives transactions562and inputs them to CG classification layer520and fraud classification layer525. CG classification layer520determines whether transactions562are CG transactions212or non-CG transactions216. Based on these classifications, neural network layers510calculate a classification loss function to determine the accuracy of the CG classification layer520in predicting whether transactions are control group transactions or not. Neural network layers510send CG transactions212and non-CG transactions216to objective function layer530. Fraud classification layer525determines whether transactions562are fraudulent or not. Based on the fraud tags522for respective transactions562, neural network layers510calculate a classification loss function to determine the accuracy of fraud classification layer525in predicting whether transactions are fraudulent or not. Fraud classification layer sends fraud tags522(indicating fraudulent or not fraudulent) to objective function layer530.

Objective function layer530, in the illustrated embodiment, combines fraud tags522with appropriate CG transactions212and non-CG transactions216and sends the appropriate transactions to the corresponding branches540-570. For example, objective function layer530sends non-fraud CG transactions532to non-fraud CG branch540, fraud CG transactions534to fraud CG branch550, non-fraud non-CG transactions536to non-fraud non-CG branch560, and fraud non-CG transactions538to fraud non-CG branch570. Objective function layer530minimizes an objective function that includes the combination of the two different losses calculated by CG classification layer520and fraud classification layer525. (Although not shown inFIG.5, layers520and525pass the results of calculating their respective loss functions to objection function layer530.) Non-fraud CG branch540, in the illustrated embodiment, outputs reconstructions544of non-fraud CG transactions. Fraud CG branch550, in the illustrated embodiment, outputs reconstructions554of fraud CG transactions. Non-fraud non-CG branch560, in the illustrated embodiment, outputs reconstructions564of non-fraud non-CG transactions. Fraud non-CG branch570, in the illustrated embodiment, outputs reconstructions574of fraudulent non-CG transactions.

Example Method

FIG.6is a flow diagram illustrating a method600for automatically updating a control group, according to some embodiments. The method shown inFIG.6may be used in conjunction with any of the computer circuitry, systems, devices, elements, or components disclosed herein, among other devices. In various embodiments, some of the method elements shown may be performed concurrently, in a different order than shown, or may be omitted. Additional method elements may also be performed as desired. In some embodiments, server computer system120performs the elements of method600.

At610, in the illustrated embodiment, a server computer system trains, using a plurality of transactions, a machine learning model, where during training the machine learning model learns a feature distribution of both a current set of control group (CG) transactions and a current set of non-control group (non-CG) transactions included in the plurality of transactions. In some embodiments, during training, the machine learning model predicts, based on a predetermined weight associated with a particular transaction feature, whether transactions included in the plurality of transactions are to be included in the current set of CG transactions or the current set of non-CG transactions. For example, the disclosed system may assign greater weight to transactions having a larger value for a dollar amount feature and may assign less weight to transactions having a smaller value for the dollar amount feature, such than the machine learning model trains harder on the transactions having larger values for the dollar amount feature. In addition, the assignment of weights may be based on a classification for transactions (e.g., whether the transaction is fraudulent or not). In some embodiments, this causes the disclosed machine learning model to select low-dollar amount transactions to be included in a control group of transactions.

In some embodiments, training the machine learning model further includes concatenating output of a third portion of the machine learning model indicating classifications for CG transactions to transactions input to a portion of the machine learning model for reconstructing CG transactions. In some embodiments, training the machine learning model further includes concatenating output of the third portion of the machine learning model indicating classifications for non-CG transactions to transactions input to a portion of the machine learning model for reconstructing non-CG transactions.

At620, the server computer system inputs, into the trained machine learning model, the current set of CG transactions. In some embodiments, the inputting includes inputting the current set of CG transactions into a non-CG portion of the machine learning model, where the current set of CG transactions that are predicted by the machine learning model during training are predicted by a CG portion of the machine learning model. The machine learning model may be a Dragonnet model with a CG branch and a non-CG branch. In some embodiments, both a CG portion and a non-CG portion of the Dragonnet model are executed using variational auto encoders (VAEs). In some embodiments, a third portion of the Dragonnet model classifies transactions. In some embodiments, predicting whether transactions included in the plurality of transactions are CG transactions or non-CG transactions is further based on one or more predefined weights for one or more transaction included in the plurality of transactions.

At630, a server computer system modifies, based on output of the trained machine learning model for the current set of CG transactions, the current set of CG transactions to generate an updated set of CG transactions. In some embodiments, modifying the current set of CG transactions includes determining reconstruction error of the non-CG portion of the machine learning model by comparing reconstructions of CG transactions output by the non-CG portion with corresponding CG transactions. In some embodiments, modifying the current set of CG transactions includes removing, based on the reconstruction error, one or more CG transactions from the current set of CG transactions to generate the updated set of CG transactions. In some embodiments, the machine learning model includes: a branch for reconstructing non-suspicious CG transactions, a branch for reconstructing suspicious CG transactions, a branch for reconstructing non-suspicious non-CG transactions, and a branch for reconstructing suspicious non-CG transactions.

In some embodiments, modifying the current set of CG transactions based on output of the non-CG portion of the machine learning model further includes performing a first comparison using a divergence algorithm, including comparing transactions in the current set of CG transactions with non-CG transactions included in the plurality of transactions. In some embodiments, modifying the current set of CG transactions further includes performing a second comparison using the divergence algorithm, wherein performing the divergence algorithm includes comparing transactions in the updated set of CG transactions with non-CG transactions included in the plurality of transactions. In some embodiments, modifying the current set of CG transactions further includes comparing results of the first comparison and the second comparison and, based on comparing the results, adding one or more non-CG transactions to the updated set of CG transactions. In some embodiments, based on comparing the results, the modifying includes removing one or more non-CG transactions from the updated set of CG transactions. In some embodiments, the divergence algorithm is a KL divergence algorithm, a contrastive divergence algorithm, a restricted Boltzmann machine, etc.

At640, the server computer system performs, based on the updated set of CG transactions, one or more preventative measures for a transaction processing system. In some embodiments, performing the one or more preventative measures includes training, using the updated set of CG transactions, a machine learning classifier, where the trained machine learning classifier is usable to generate an authorization decision for newly requested transactions. For example, if a user computing device requests to initiate an online electronic transaction, server computer system120(or some other system) may execute the trained machine learning classifier to determine a suspiciousness classification for the requested electronic transaction. Based on the suspiciousness classification indicating that the requested electronic transaction is fraudulent, the server computer system120may deny the requested transaction.

Example Computing Device

Turning now toFIG.7, a block diagram of one embodiment of computing device710(which may also be referred to as a computing system) is depicted. Computing device710may be used to implement various portions of this disclosure. Computing device710may be any suitable type of device, including, but not limited to, a personal computer system, desktop computer, laptop or notebook computer, mainframe computer system, web server, workstation, or network computer. The server computing system120shown inFIG.1and discussed above is one example of computing device710. As shown, computing device710includes processing unit750, storage712, and input/output (I/O) interface730coupled via an interconnect760(e.g., a system bus). I/O interface730may be coupled to one or more I/O devices740. Computing device710further includes network interface732, which may be coupled to network720for communications with, for example, other computing devices.

In various embodiments, processing unit750includes one or more processors. In some embodiments, processing unit750includes one or more coprocessor units. In some embodiments, multiple instances of processing unit750may be coupled to interconnect760. Processing unit750(or each processor within750) may contain a cache or other form of on-board memory. In some embodiments, processing unit750may be implemented as a general-purpose processing unit, and in other embodiments it may be implemented as a special purpose processing unit (e.g., an ASIC). In general, computing device710is not limited to any particular type of processing unit or processor subsystem.

Storage subsystem712is usable by processing unit750(e.g., to store instructions executable by and data used by processing unit750). Storage subsystem712may be implemented by any suitable type of physical memory media, including hard disk storage, floppy disk storage, removable disk storage, flash memory, random access memory (RAM—SRAM, EDO RAM, SDRAM, DDR SDRAM, RDRAM, etc.), ROM (PROM, EEPROM, etc.), and so on. Storage subsystem712may consist solely of volatile memory, in one embodiment. Database150, discussed above with reference toFIG.1is one example of storage subsystem712. Storage subsystem712may store program instructions executable by computing device710using processing unit750, including program instructions executable to cause computing device710to implement the various techniques disclosed herein.

I/O interface730may represent one or more interfaces and may be any of various types of interfaces configured to couple to and communicate with other devices, according to various embodiments. In one embodiment, I/O interface730is a bridge chip from a front-side to one or more back-side buses. I/O interface730may be coupled to one or more I/O devices740via one or more corresponding buses or other interfaces. Examples of I/O devices include storage devices (hard disk, optical drive, removable flash drive, storage array, SAN, or an associated controller), network interface devices, user interface devices or other devices (e.g., graphics, sound, etc.).

Various articles of manufacture that store instructions (and, optionally, data) executable by a computing system to implement techniques disclosed herein are also contemplated. The computing system may execute the instructions using one or more processing elements. The articles of manufacture include non-transitory computer-readable memory media. The contemplated non-transitory computer-readable memory media include portions of a memory subsystem of a computing device as well as storage media or memory media such as magnetic media (e.g., disk) or optical media (e.g., CD, DVD, and related technologies, etc.). The non-transitory computer-readable media may be either volatile or nonvolatile memory.