SYSTEM AND METHOD TO OPTIMIZE PROCESSING PIPELINE FOR KEY PERFORMANCE INDICATORS

A computer-implemented system, platform, computer program product, and/or method for optimizing a data analytics suspicious activity detection pipeline that includes identifying a data analytics suspicious activity detection (SAD) pipeline for optimization; inputting desired key performance indicators for the data analytics suspicious activity detection (SAD) pipeline; gathering key performance indicators from previous runs of the data analytics suspicious activity detection (SAD) pipeline; identifying candidate pipeline configurations for simulation runs; running simulations of the candidate pipeline configurations; analyzing the simulations of the candidate pipeline configurations; and identifying the optimal pipeline configuration for the desired key performance indicators.

FIELD

The present application relates generally to information handling, data processing, and/or data analytics, and more particularly to systems, platforms, computer program products, and/or methods for detecting suspicious activities, e.g., suspicious financial transactions and/or insurance claims.

BACKGROUND

There is a regulatory need for monitoring financial transactions and activity of account holders at financial institutions and monitoring insurance claims to detect any suspicious, fraudulent, and/or criminal activity such as, for example, money laundering, terrorist financing, credit card theft, and/or insurance claim fraud. Governmental anti-money laundering (AML) and other regulations may require a financial institution to monitor for activities and behavior indicative of criminal or fraudulent activity. Detecting activity indicative of fraudulent or criminal activity is increasingly difficult due to the large amount of data and information, such as, for example, numerous financial transactions, numerous insurance claims, and numerous parties/entities.

There have been developed electronic systems and data analytical processes to detect activity indicative of fraud, criminal behavior, and other suspicious activity. These advanced information and data processing systems discover, define, and detect data patterns within relationship networks, for example, a large-scale network of financial transactions, insurance claims, and the like involving numerous parties and transactions, that are indicative of suspicious activity and behavior.

In the financial crime scenario and/or insurance claims processing, the electronic systems and data analytical processes create alerts that are often reviewed by analysts and managers, each having different roles in the analysis and vetting process. In typical scenarios, new data is imported into the electronic data analytics system, for example a day's financial transactions, new insurance claims, and/or updated insurance claims processing, and electronic analytical processes are run for example on all the data, old and new data, and the electronic data analytics system generates alerts. The alerts can take many forms and generally will flag a person or incident that is suspicious and might require further review and analysis. The alert goes to a case manager who reviews the alert, and if the alert is credible, then it might be investigated by an internal team, and if the internal team finds the alert credible then typically a suspicious activity report or suspicious claim report is prepared.

Suspicious activity processing pipelines are long running processes that often involve multiple data sets, data pre-processing, data clean-up, data filtering, data transformation, data analytic models, rules, algorithms, and/or ensemble models to produce results. Often multiple simulations are run to produce the best possible run. Customers or clients often supply the data processing entity with guidelines on the Key Performance Indicators (KPIs), such as, for example, time for the pipeline line to complete, the amount of CPU and/or memory the pipeline can consume, and/or the F1 score (an indicator of how well the pipeline performs (e.g., the probability of the pipeline)) for the pipeline results. The KPIs can be expressed as values, or can be expressed as boundaries (such as upper and lower limits) to indicate flexibility. Often the data science team focuses on maximizing the model accuracy (the F1 score) and loses focus on the overall performance of the suspicious activity detection (SAD) pipeline. It would be advantageous to provide a system, platform, computer program product, and/or method to optimize the suspicious activity detecting (SAD) pipeline for the desired KPIs.

SUMMARY

The summary of the disclosure is given to aid understanding of, and not with an intent to limit, the disclosure. The present disclosure is directed to a person of ordinary skill in the art. It should be understood that various aspects and features of the disclosure may advantageously be used separately in some circumstances or instances, or in combination with other aspects, embodiments, and/or features of the disclosure in other circumstances or instances. Accordingly, variations and modifications may be made to the system, platform, processing pipeline, their architectural structure, the computer program product, and/or their method of operation to achieve different effects. In this regard it will be appreciated that the disclosure presents and describes one or more inventions, and in aspects includes numerous inventions as defined by the claims.

One or more embodiments of a system, platform, computer program product, and/or method is disclosed for optimizing a data analytics suspicious activity detection (SAD) pipeline that includes: identifying a data analytics suspicious activity detection (SAD) pipeline for optimization; inputting desired key performance indicators for the data analytics suspicious activity detection (SAD) pipeline; gathering key performance indicators from previous runs of the data analytics suspicious activity detection (SAD) pipeline; identifying candidate pipeline configurations for simulation runs; running simulations of the candidate pipeline configurations; and analyzing the simulations of the candidate pipeline configurations. In one or more approaches, the system, platform, computer program product, and/or method includes identifying the optimal pipeline configuration for the desired key performance indicators. The system, platform, computer program product, and/or method in an aspect optionally includes identifying hotspot tasks that were not susceptible to improvement by the method of optimizing the data analytics suspicious activity detection pipeline and/or identifying and removing tasks that have no or negligible effect on the data analytics suspicious activity detection pipeline.

According to one or more aspects, the system, platform, computer program product, and/or method optionally includes applying fine tune variations that reorganize the order of steps, tasks, and/or functions in the data analytics suspicious activity pipeline, where in an approach applying the fine tune options is based upon the dependencies of the tasks in the data analytics suspicious activity detection pipeline. Identifying candidate pipeline configurations for simulation runs includes in one or more embodiments, identifying at least one of the group consisting of removing tasks, reconfiguring tasks, updating tasks, re-ordering tasks, and combinations thereof. In an optional approach identifying the candidate pipeline configurations includes using the gathered key performance identifiers to identify candidate pipeline configurations for simulation runs, and in an optional aspect using the gathered key performance identifiers includes identifying tasks that have an outsized effect on one or more of the key performance identifiers. In another optional approach, identifying the candidate pipeline configurations includes creating a matrix of candidate pipeline configurations for simulation runs. In a further embodiment, the candidate pipeline configurations identified for simulation runs are limited to the top N candidate pipeline configurations.

In an embodiment, the computer programming product includes instructions that, when executed by at least one hardware processor, configure the at least one hardware processor to perform one or more of the steps, tasks, and/or functions described herein, and the system and/or platform includes a non-transitory memory storage device storing program instructions; and a hardware processor having circuitry and logic to execute said program instructions, wherein the hardware processor is in communication with said memory storage device and in response to executing said program instructions, is configured to perform the steps, task, and/or functions described herein.

The foregoing and other objects, features, and/or advantages of the invention will be apparent from the following more particular descriptions and exemplary embodiments of the invention as illustrated in the accompanying drawings wherein like reference numbers generally represent like parts of the illustrative embodiments of the invention.

DETAILED DESCRIPTION

The following description is made for illustrating the general principles of the invention and is not meant to limit the inventive concepts claimed herein. In the following detailed description, numerous details are set forth in order to provide an understanding of the system, platform, computer program product, method, and/or techniques for optimizing a pipeline for detecting suspicious activity, however, it will be understood by those skilled in the art that different and numerous embodiments of the system, platform, computer program product and/or their method of operation may be practiced without those specific details, and the claims and disclosure should not be limited to the arrangements, structures, systems, assemblies, subassemblies, platforms, modules, functional units, circuitry, embodiments, features, aspects, instructions, programming, processes, methods, techniques, and/or details specifically described and shown herein. Further, particular features, aspects, arrangements, structures, systems, assemblies, subassemblies, platforms, modules, functional units, circuitry, embodiments, instructions, programming, processes, methods, techniques, details, etc. described herein can be used in combination with other described features, aspects, arrangements, structures, systems, assemblies, subassemblies, platforms, modules, functional units, circuitry, embodiments, instructions, programming, processes, methods, techniques, details, etc. in each of the various possible combinations and permutations.

The following discussion omits or only briefly describes conventional features of information processing systems and data networks, including electronic data analytics programs or electronic risk assessment tools, platforms, computer program products and/or methods configured and adapted to detect suspicious activity and generate alerts in connection therewith, for example, financial and/or insurance claim transactions, which should be apparent to those skilled in the art. It is assumed that those skilled in the art are familiar with data processing and data analytics including large scale data processing/analytics (also referred to as information/data processing systems) and their operation, and the application of data analytics, including data analytics systems and processes to detect suspicious activity that may be indicative of fraud and/or other criminal behavior. It may be noted that a numbered element is numbered according to the figure in which the element is introduced, and is typically referred to by that number throughout succeeding figures.

FIG.1illustrates an example computer system functioning as an electronic risk assessment tool100implementing methods to detect suspicious activity in the domain of financial services, insurance claims processing, and related industries, e.g., insurance claim fraud and financial fraud detection. In one or more embodiments, such a system tool100may be employed by or for a financial institution or insurance company who may issue, or a regulatory authority who may receive, a suspicious activity report or suspicious claim report concerning a particular party, organization, and/or transaction(s). The suspicious activity report or claim may be issued by an insurance company or financial institution against an entity or party who has been found to participate in suspicious activity, e.g., the insurance company has found a suspicious claim undertaken by a given party. In insurance fraud, a “suspicious” entity may be a doctor, auto body shop, a claimant, or any party. Suspicious entities may be placed on a “watch” list.

In the context of detecting suspicious financial activity and/or suspicious insurance claim processing, risk assessment tool100, according to one embodiment, is a computer system, a computing device, a mobile device, or a server configured to run risk assessment software applications and models. In some aspects, computing device100may include, for example, personal computers, laptops, tablets, smart devices, smart phones, or any other similar computing device, an embodiment of which is described in more detail inFIG.9.

Computing system100includes one or more hardware processors152A,152B (also referred to as central processing units (CPUs)), a memory150(e.g., for storing an operating system, application program interfaces (APIs) and program instructions), a network interface156, a display device158, an input device159, and any other features common to a computing device. In one or more aspects, computing system100may, for example, be any computing device that is configured to communicate with one or more networks and/or websites125including a cloud-based or web-based server120over a public or private communications network99. For instance, network125may include an insurance company or financial institution that records/stores information, e.g., multiple insurance claims and/or financial transactions occurring between numerous parties (entities). Such insurance claims and/or electronic transactions may be stored in a database130B with associated entity, claims, and/or financial information stored in related database130A. Further, as shown as part of system100, there can be a local memory and/or an attached memory storage device160, or a remote memory storage device, e.g., a database, accessible via a remote network connection for input to the system100.

In the embodiment depicted inFIG.1, processors152A,152B may include, for example, a microcontroller, Field Programmable Gate Array (FPGA), or any other processor that is configured to perform various operations. Communication channels140, e.g., wired connections such as data bus lines, address bus lines, Input/Output (I/O) data lines, video bus, expansion busses, etc., are shown for routing signals between the various components of system100. Processors152A,152B are configured to execute instructions as described below. These instructions may be stored, for example, as programmed modules in an associated memory storage device150.

Memory150may include, for example, non-transitory computer readable media in the form of volatile memory, such as random-access memory (RAM) and/or cache memory or others. Memory150may include, for example, other removable/non-removable, volatile/non-volatile storage media. By way of non-limiting examples only, memory150may include a portable computer diskette, a hard disk, a random-access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Network interface156is configured to transmit and receive data or information to and from a network and/or web-site server120, e.g., via wired or wireless connections. For example, network interface156may utilize wireless technologies and communication protocols such as Bluetooth®, WIFI (e.g., 802.11a/b/g/n), cellular networks (e.g., CDMA, GSM, M2M, and 3G/4G/4G LTE, 5G), near-field communications systems, satellite communications, via a local area network (LAN), via a wide area network (WAN), or any other form of communication that allows computing device100to transmit information to or receive information from the server120.

Display158may include, for example, a computer monitor, television, smart television, a display screen integrated into a personal computing device such as, for example, laptops, smart phones, smart watches, virtual reality headsets, smart wearable devices, or any other mechanism for displaying information to a user. In one or more aspects, display158may include a liquid crystal display (LCD), an e-paper/e-ink display, an organic LED (OLED) display, or other similar display technologies. In an aspect, display158may be touch-sensitive and may also function as an input device. Input device159may include, for example, a keyboard, a mouse, a touch-sensitive display, a keypad, a microphone, or other similar input devices or any other input devices that may be used alone or together to provide a user with the capability to interact with the computing device100.

With respect to configuring the computer system as a risk assessment tool100for detecting suspicious activity, e.g., insurance fraud and/or financial fraud, and creating alerts, the local or remote memory160may be configured for storing information and associated meta-data. Such captured and stored data can include, but is not limited to: parties, claims, accounts, transactions, relationships, and associated metadata obtained from claims, transactions and/or data stored in the electronic databases130A,130B. Alternately or in addition, the entity data, entity relationships, claims, transactional data and meta-data162can be stored in a separate local memory storage device attached to the computer system100.

Memory150of computer system100in one or more embodiments stores processing modules that include programmed instructions adapted to perform risk assessment as it relates to detecting suspicious activity, e.g., financial fraud, money laundering, insurance fraud, and creating alerts or SARs.

In an embodiment, one of the programmed processing modules stored at the associated memory150include a data ingestion module165that provides instructions and logic for operating circuitry to access/read large amounts of data (e.g., parties, claims/accounts, transaction data) for use by other modules that process and analyze the data to form and output alerts. In one or more embodiments, the input data for data ingestion module165comprises parties, accounts, claims, transactions, etc. For example, where a financial institution, such as for example a bank, desires to determine if there is a money laundering scheme or other fraud, for example as a result of governmental oversight, or regulations, such as for example anti-money laundering (AML) laws, the input data can comprise: the transactions occurring with or being processed by the financial institution; the parties to any financial transaction with or through the financial institution; and account information (the customers) of the financial institution. In the case of an insurance organization and the like, the input data can comprise: the parties doing business with the insurance organization; the claims made with the insurance organization; policy information (the customers) of the insurance organization, the identity of any agencies or brokers that were involved in underwriting the policy, and any parties involved in treating the claim, e.g., auto body shop fixing the motor vehicle, physician treating patient, etc. The examples above are not limiting and there can be other situations where the system will have application, and additional or other input data can be provided.

In an embodiment, memory150includes a Suspicious Activity Detection (SAD) Pipeline Module170(also referred to as a SAD Pipeline Platform170) that provides instructions and logic for operating circuitry to process and analyze data, typically large amounts of data, to form a suspicious activity detection (SAD) pipeline that generates and output alerts. Suspicious Activity Detection (SAD) Pipeline Platform170(also referred to as SAD Platform170, SAD Pipeline Platform170, or SAD Pipeline Generator170) preferably contains one or more models to determine a suspicious activity risk probability based on the variables, and/or data. Based on the data and the models, alerts and/or SARs can be produced by the SAD Platform170, and an analyst can analyze the alert, and provide feedback as to a potential risk level of a party, insurance claim, and/or transaction.

SAD Platform170can contain and call up one or more models to process and analyze the data and provide associated alerts for review by an analyst. SAD Platform170in an embodiment can include a Risk-by-Association analyzer that provides instructions and logic for operating circuitry which can run a probabilistic risk model to generate a risk-by-association analysis scores. In an embodiment, another optional processing module stored at the associated computer memory150, and in an aspect as part of the SAD Pipeline Platform170, is a pattern determination module or model that employs logic and instructions for detecting any data patterns indicative of suspicious activity or behavior in the transaction network that is indicative of criminal and/or fraudulent activity. SAD Platform/Module170can further include and invoke supervised (or unsupervised) machine learning (ML) techniques through, for example, ML Model175, for detecting suspicious activity indicative of criminal activity, e.g., fraud, as known in the art, e.g., supervised learning using a regression model to predict a value of input data (classification) and unsupervised learning (clustering) techniques. Based on features and metadata relating to a party, transactions, claims, environmental data, industry data, location data, other data, and changes to the data that are captured, techniques employing Hidden Markov Models or Artificial Neural Networks may alternatively or additionally be employed to compute a risk associated with a particular party, a transaction, a claim, and/or changes in data states.

In an embodiment, SAD Pipeline Platform170includes an optional graph and build module/model172, e.g., in memory150, and provides instructions and logic for operating circuitry to form/build graphs, e.g., relationship networks, for use in producing alerts and accessing risk, and/or read data from graphs. Graph build module172is shown as included within the SAD Pipeline Platform170but can be a separate module/model from the SAD Pipeline Platform170. Other modules or models can include rules model. Further details and explanation of the SAD Pipeline Platform170is discussed below, and the SAD Pipeline Platform170is not limited to the modules or models discussed above and any number of models can be used in the SAD Pipeline Platform170. In an aspect, the ingestion module165would load the received input data, the SAD Pipeline Platform170determines from the input data, which may, for example, be party data, account data, transaction data, claims data, industry data, geographical data, and other data, including metadata, whether to issue an alert. In most instances, large amounts of data are input and processed by the data ingestion module165.

Memory150optionally includes a supervisory program having instructions for configuring the computing system100to call one or more, and in an embodiment all, of the program modules and invoke the suspicious activity detection (SAD) pipeline operations of the SAD Pipeline Platform170in risk assessment tool100. In an embodiment, the supervisory program calls methods and provides application program interfaces (APIs) for running SAD Pipeline Platform170for generating a suspicious activity detection (SAD) pipeline for producing SARs and/or alerts using the data ingested by the data ingestion module165.

At least one application program interface (API)195is invoked in an embodiment to receive input data from an “analyst”, e.g., a domain expert trained in finance/financial matters particularly in the context of detecting criminal and/or fraudulent activity. Via API195, the analyst receives SARs and/or alerts. The analyst reviews the alert and, in an embodiment, can provide feedback information to either escalate or close the investigation of a particular party, transaction, etc. depending on the risk. Further investigation can lead to the issuance of a suspicious activity report (SAR).

In an embodiment, computing system and/or risk assessment tool100can also include a system, platform, computer program product, and/or computer programming module180to optimize the suspicious activity detection (SAD) pipeline based upon the desired Key Performance Indicators (KPIs). In one or more embodiments, the Suspicious Activity Detection (SAD) Pipeline Optimizer Module180is configured to capture requirements for the suspicious activity detection (SAD) pipeline, and in an aspect to understand the dependency of different pre-processing, filters, data transform functions, models, and/or steps involved in the suspicious activity detection (SAD) pipeline. In one or more embodiments, the Optimizer180generates one or more configurations for candidate runs, performs one or more simulations for the candidate configurations, and captures the key performance indicators (KPIs) for the candidate runs and simulations. The Optimizer180, according to an embodiment, compares the reported KPIs for the various candidate runs and/or simulations against the defined/desired KPIs and in an aspect makes a recommendation on possible pipeline configurations to meet the desired KPIs.

The Suspicious Activity Detection (SAD) Pipeline Optimizer Module180in one or more embodiments provides instructions and logic for operating circuitry to optimize the suspicious activity detection (SAD) pipeline configuration. According to one or more approaches, optimizing the SAD pipeline can include, but is not limited to, re-ordering of steps, reconfiguring and/or adjusting tasks (filtering, transform functions, data processing, and/or models), identifying “hot spots” (e.g., filters, transform functions, models, tasks that could not be optimized), and/or identifying models, data processing tasks, and/or feature engineering steps that can be removed without significant compromise to the desired/defined KPIs.

In an embodiment, SAD pipeline Optimizer Platform180includes an optional graph and build module/model182, e.g., in memory150, and provides instructions and logic for operating circuitry to form/build graphs for use in optimizing the suspicious activity detection pipeline, and/or read data from graphs. Graph build module182is shown as included within the Optimizer Platform180but can be a separate module/model from the Optimizer Platform180. Other modules or models can include rules model. Further details and explanation of the Optimizer Platform180is discussed below, and the Optimizer Platform180is not limited to the modules discussed below and any number of modules can be used in the Optimizer Platform180. In an aspect, the ingestion module165would load the received input data to the SAD Pipeline Module170, the SAD Pipeline Module170would configure a suspicious activity detection (SAD) pipeline to generate alerts, Optimizer Platform180would receive the desired KPIs and the pipeline to be optimized (and optionally the fine tune options), and the Optimizer Platform180determines from the desired KPIs and inputted pipeline, whether and how to reconfigure, adjust, and/or remove preprocessing tasks to optimize the suspicious activity detection (SAD) pipeline (e.g., generate revised SAD pipeline) for the desired/defined KPIs.

FIG.2discloses further details about the SAD Pipeline Module170, sometimes also referred to as a SAD Pipeline Generator or SAD Pipeline Generator Framework170, that is designed and configured to generate a suspicious activity detection (SAD) pipeline that creates one or more alerts280. For purposes of clarityFIG.2discloses functional units210for the Module170while eliminating some of the details, specifics, and features disclosed in system100inFIG.1. The functional units210, and/or processing units associated with the functional units210, contain instructions and/or logic for operating circuitry to perform the respective functions and operations of the functional units. For example, the functional units210contain instructions and/or logic executable by circuitry, for example in a processor, computer, server, etc., to perform one or more of the process steps performed by the respective functional unit210.

Module170in the embodiment ofFIG.2discloses functional units210including Traceability Manager220, Pipeline Run Manager230, Pipeline Configuration Manager240, Pipeline Executor250, and Case Manager260. Pipeline Configuration Manager240configures and maintains a suspicious activity detection (SAD) Pipeline300(SeeFIG.3) by, for example, determining what models to call, what data205to use, and how to use the data. The pipeline configuration can be captured in a table or as a JSON object. Pipeline Configuration Manager240also maintains the revisions of the pipeline configurations such as, for example, what version, who modified/created the SAD pipeline, and when it was modified/created. Pipeline Configuration Manager240further maintains details of the pipeline such as input data, the pipeline flow, which filters are used and their versions, which transform functions are used and their versions, which models are used and their versions, which code/module are used and their versions.

The Pipeline Executor250executes the suspicious activity detection (SAD) pipeline, including reading the pipeline configuration for a given pipeline and its version and executes the steps in the pipeline. Pipeline Executor250updates the execution pipeline run configuration and runs an Execution Pipeline300(SeeFIG.3) task-by-task to generate and output alerts280. The Pipeline Run Manager230maintains the run details, e.g., for each pipeline. The Pipeline Run Manager230manages and tracks what programs/models are run, including when and what is run each time a pipeline run is executed, such as, for example, when the pipeline run started and completed, and the outcome or result of the run. The Pipeline Run Manager230knows the runtime information of any given pipeline and can be used to find which runs are impacted for a given change. The Traceability Manager220maintains the traceability of different artifacts in the system. The Traceability Manager220can help to identify the pipeline runs for a given alert by tracing the alerts and what models, programs, filters, data, transform functions, insights and/or events were used to generate the alerts280. The Case Manager260in an embodiment is typically used by a user or analyst to review the alerts generated by the system. If the alert is suspicious enough, in an aspect it can be moved to an investigation queue, or if the alert is a false positive, the analyst or supervisor can close the case.

The Module170uses the functional units210to generate and run the SAD pipeline to prepare alerts280based upon data205ingested and/or received into the system200. For example, the Module170receives additional data on a periodic basis, such as for example every evening. The Module170through input from an analyst or other user, or optionally as part of a program, selects programs or models to run in order to generate alerts280. The module170to generate the alert280can take many forms and can have more or less functional units210than described in connection withFIG.2. The Module170can be run to generate alerts on a periodic basis, such as, for example at predetermined times, and/or manually by a user, immediately or any desired time.

FIG.3illustrates a suspicious activity detection (SAD) (e.g., fraud detection) execution pipeline300generated and/or created by suspicious activity detection (SAD) pipeline module170. Pipeline300in an embodiment includes receiving and/or ingesting data205, illustrated as data subset205′ (d1), data subset205″ (d2), and data subset205′″ (d3) inFIG.3, where for example d1data subset205′ represents data, d2data subset205″ represents parties, and d3data subset205′″ represents incidents or claims. It can be appreciated that the amount of data can and likely is much larger and numerous than the data205illustrated inFIG.3. The detection pipeline300can include one or more tasks305, for example one or more filters320, one or more Transform Functions330, one or more Models340, and one or more Code Units350to process the data205and produce, generate, and/or output an Alert280. The detection pipeline300illustrates a flow that takes data as an input and performs a number of tasks305to process the data to generate an output280, which can optionally include one or more insights370.

Pipeline300includes one or more filters320, illustrated as Filter1(322), Filter2(324), and Filter3(326) inFIG.3, that are applied to the data205, e.g., to data d1(205′), data d2(205″), and data d3(205′″). Filters320process the data205or data subsets (e.g.,205′), for example, by filtering rows and columns, to obtain the desired data or data subsets in, for example, the desired format. Different filters320can be applied to different data205as shown inFIG.3, the same filter320can be applied to different data, or the same filter320can be applied to different data. The detection pipeline300can apply one or more transform functions330which further process and transform the data. One example of a transformation function330can be an “aggregate” function, which aggregates the physical damages or injury bills against a particular claim. Another example can be a simple function which computes whether the loss date is a weekday or a weekend, and whether during business hours or after business hours. In pipeline300, the Transform Functions330are Transform Function332(Transform Function1), Transform Function334(Transform Function2), Transform Function335(Transform Function3), Transform Function336(Transform Function4), and Transform Function338(Transform Function5). More or less Transform Functions330can be used in detection Pipeline300. Different incident models340can be applied in detection pipeline300. The one or more filters320and one or more Transform Functions330generate one or more feature sets345that are input and/or submitted to the one or more models340. The one or more features or feature sets345are data205that has been filtered, selected, and transformed into a form for use by the selected model340. In other words, the various models340desire and/or require certain data in a certain format in order to run and generate an output. In execution pipeline300, for example, the feature345′ (e.g., feature1) output from Transform Function334is fed into Model342(Model1).

The models340, also referred to as programs340, in one or more embodiments receive input as feature set345and will output or generate, depending upon the amount of processing performed by the model, insights370, alerts280, and/or values (data) that can, for example, be further processed to create insights370or alerts280. In an embodiment, the output from the Module170and/or detection pipeline300is an alert280, and the alert280in one or more aspects is output directly from the one or more models340. Additionally, or alternatively, the output from the one or more models340can be one or more insights370that are used to generate one or more alerts280. The models340can be, for example, a risk-by-association analyzer, a pattern determination model, a rules model, a machine learning (ml) model, etc. More or less models340are contemplated, and pipeline300is illustrated with three models340, e.g., Model342(Model1), Model344(Model2) and Model346(Model3).

In a further embodiment, the one or more models340can produce values (data) used to replace variables in a narrative template to create insights370, and/or values that can be used to generate an alert280. That is, based upon the model output, the detection pipeline300/Module170(e.g., a Transform Function330and/or a further model340) can replace the variables in a narrative and provide the resulting narrative as an insight370. For example, after applying the one or more models340the data output from the models(s)340may undergo further processing and manipulation by further transform function(s)330, and by one or more models340, to produce one or more insights370. In one or more embodiments, because the model output is technical output and users of the system typically respond to text language, e.g., sentences, that they can understand, the model output can be run through a further optional Transform Function and/or a further model to construct a narrative referred to as an insight370. In pipeline300, for example, the output from Models342and344are received by Transform Function338which further processes the data for insertion into a narrative where the output from Transform Function338is received by Model346that inserts the values from Transform Function338into the narrative and Model346outputs one or more insights370. As a further example, a model342returns two values [Cluster A, 10] where A is the cluster ID and 10 is the number of frauds in the cluster. Model342or another Model can place or substitute the values from model342into a narrative template to produce an insight370. So where the narrative template is defined as “Party <party_id> is in Cluster <cluster id> which contains <number_of_fraud_parties> parties”, the model342or another model will take the values and produce the insight, “Party A is in cluster A which contains 10 fraud parties” as an insight370output by model342or another model.

Optional Code350in pipeline300may be applied to further convert the data. For example, code350can be a look-up table to score the output of the case. Model346(Model3) in pipeline300can output a numeric score or value and Code1350can determine if the numerical score or value is above a threshold, and if so can generate or output an alert280. It should be recognized that the output of the pipeline300, or the model340may be in the form of “low” risk, “medium risk”, or “high” risk, but it should be recognized that the output can be in other forms or formats. In one or more embodiments, input data205is fed into the module170, and in an embodiment into detection pipeline300, where the input data205is converted into features345that are fed into one or more models340where in one or more aspects the one or more models340generate an output as one or more insights370, and one or more insights can be used to generate an alert280.

It should be recognized that the Detection Pipeline300can include one or more execution pipelines, or sub-branches302that perform one or more tasks305. For example, Filter322(Filter1), Transform Function332(Transform Function1), Transform Function334(Transform Function2), and Model342(Model1) illustrates detection pipeline302. Each sub-pipeline302can have different task units that undergo different tasks305, e.g., different filters, different transform functions, and/or different models, but can also share filters, transform functions, models, and data. For example, in the insurance industry, the SAD Pipeline Module can produce an auto insurance pipeline, a worker's compensation pipeline, a property pipeline, and/or a medical provider pipeline, to name just a few. Each pipeline will take or use different data and perform different tasks305to process the data differently to detect/identify potential suspicious activity. For example, the auto pipeline takes all auto insurance claims as input data and the worker's compensation pipeline is fed with worker's compensation claims.

An analyst typically reviews the alert280output by the SAD pipeline300to determine how to proceed, and, in particular, typically reviews the insights370and/or events that lead to the alert280being generated in the particular case in order to determine how to proceed. The alert280typically relies upon one or more insights370, and the alert or report is typically presented with the insights370which form the basis for the alert280. For example, insight1could be that the average number of transactions has changed by twenty percent (20%); insight2could be that the customer is part of a high-risk cluster which has a number of known frauds; insight3could be that the transactions are associated with a high-risk geographic zone; and insight4could be that there was extended inactivity in the account before the recent transactions. The SAD Pipeline Generator Module170for example can have a Case Manager260that contains the alerts280and/or insights370, and the user or analyst opens the case manager program260and reviews the alerts280, insights370. The analyst reviews the alert280and/or insights370, and determines how to proceed, including for example to close the alert280if the alert is believed to be a false positive, or if it is believed that there is likely some fraud and needs further investigation the case can be escalated to “investigation”. In instances the analyst can decide whether to bring the alert280to a supervisor or manager to determine what actions to take.

Often data science teams investigating suspicious activity and building SAD pipelines, focus on the overall performance of the suspicious activity detection (SAD) pipeline to maximize the F1 score, where the F1 score is an indicator of how well the model is performing (maximizing the probability of the pipeline). Instead of maximizing the F1 score, in one or more embodiments the pipeline is optimized to account for multiple, preferably all the, desired Key Performance Indicators (KPIs), e.g., the time to complete, amount of CPU usage, amount of memory usage, and/or F1 score. In one or more embodiments a SAD pipeline optimizer180takes into account and maximizes the SAD pipeline for multiple KPIs.

FIG.4diagrammatically illustrates an overview of a system400that receives desired key performance indicators (KPIs)410, a SAD pipeline300, and optional reconfiguration and readjustment (e.g., fine tune) options412for the SAD pipeline300, as inputs into Suspicious Activity Detection (SAD) pipeline optimizer module180to provide/produce an optimized SAD pipeline300′ for the desired KPIs410.FIG.5discloses further details about the suspicious Activity Detection (SAD) Pipeline Optimizer Module180, sometimes also referred to as an Optimizer System180, Optimizer Governance Framework180, and/or Optimizer Module180, that is designed and configured to optimize a SAD pipeline for desired KPIs, e.g., can reconfigure the SAD pipeline by adjusting, changing, removing, and/or re-ordering, tasks, filters, transforms, models, parameters, etc. Key Performance Indicators (KPIs) are generally abstracted as “resource”, “time”, and “quality”, where “resource” can be measured as CPU and/or memory (RAM or disk space), “time” can be measured as total time to complete the job (or task), and “quality can be measured as accuracy of output, e.g., F1 score. Key Performance Indicators (KPIs) can be defined and/or measured at the global pipeline level or at the task lever, e.g., for Model1, Model2, etc.

For purposes of clarity,FIG.5discloses functional units520for the Optimizer Module180while eliminating some of the details, specifics, and features. The functional units520, and/or processing units associated with the functional units520, contain instructions and/or logic for operating circuitry to perform the respective functions and operations of the respective functional units520. For example, the functional units520contain instructions and/or logic executable by circuitry, for example in a processor, computer, server, etc., to perform one or more of the process steps performed by the respective functional unit520.

Module180in the embodiment ofFIG.5includes Task Level KPI Capturer522, Pipeline Flow Optimizer524, Candidate Simulation Identifier526, Pipeline Simulations Generator528, and Report Generator530. Task Level KPI Capturer522performs simulation to understand the KPI values at the task level, e.g., the resource requirements, time requirements, and effect on the accuracy (F1 score) for each task305(e.g., data filter tasks, data transform tasks, and/or model tasks) in the detection pipeline300. The Task Level KPI Capturer522in an embodiment scores the tasks305in the pipeline300against the pipeline KPIs. That is the Task Level KPI Capturer522captures and/or records for each task305the various KPIs utilized by that particular task305. For example, the time to run, use of resources (CPU/memory), and/or effect on model accuracy (F1 score) is recorded for each task305. The KPIs for each task can be recorded in a database, chart, or other format, preferably in a manner that is easily obtainable and ready to use.

The Pipeline Flow Optimizer524optionally reorganizes the steps/tasks to optimize the pipeline run. For example, the Pipeline Flow Optimizer524could convert parallel tasks to sequential tasks or convert sequential (serial) tasks in the SAD pipeline to parallel tasks based upon the operations performed in the pipeline300. In an aspect, the Pipeline Flow Optimizer524looks at the dependencies of the pipeline and tries to optimize the pipeline flow by, for example, converting serial tasks to parallel tasks to lessen the time required to process data in the pipeline300. For example, if a first task can be processed independently of a second task, e.g., the input of the second task is not dependent upon the output of the first task, then the second task can be performed parallel with or before the first task, in addition to being performed after the first task. These optional reorganizations, reconfigurations, and/or readjustments performed by the Pipeline Flow Optimizer524can be referred to as fine-tune options or fine tune variations. The fine tune variations can be optionally applied by the Pipeline Flow Optimizer524when optimizing the pipeline.

Candidate Simulation Identifier526computes and determines the candidate pipeline configuration(s) for simulation. In one or more embodiments, the candidate pipeline configurations identified by the Candidate Simulation Identifier526utilizes the information collected by the Task Level KPI Capturer522to determine the candidate pipelines likely to have the most success by looking at the KPI values recorded for each task. For example, the Candidate Simulation Identifier526can determine which tasks are the most resource intensive (take the most CPU, most memory, and/or most time to run) and look to reconfigure and/or remove those tasks. The Candidate Simulation Identifier526can also determine which tasks have the least effect on the accuracy score (F1 score) and look to reconfigure and/or remove those tasks. In an embodiment, the Candidate Simulation Identifier526can identify the top pipeline candidates based upon information obtained by the Task Level KPI Capturer522, and, in an aspect, can identify the top fifty, hundred, or some other threshold or percentage of candidate pipelines. In another embodiment, the Candidate Simulation Identifier526can use a brute force method of determining and/or identifying a matrix of most every conceivable variation of the pipeline as a candidate pipeline.

Pipeline Simulation Generator528runs the simulation for each candidate pipeline identified in the Candidate Simulation Identifier526. During the simulations run for each pipeline identified in the Candidate Simulation Identifier526by the Pipeline Simulation Generator528, various parameters, including for example resource usage, time to run, F1 score, and KPIs at the task level as well as the pipeline level are captured and recorded.

Report Generator530analyzes the simulation results from the Pipeline Simulation Generator528and generates a report. The Report Generator530includes Optimal Configuration Identifier532, Hot Spot Identifier534, and Non-Performing Tasks (Models) Identifier536. Optimal Configuration Identifier532identifies the optimal SAD pipeline for meeting the desired KPIs among the various candidate SAD pipelines. Hot Spot Identifier534identifies tasks that could not be optimized and which in an aspect negatively affect and count against one or more the desired KPIs. For example, the Hot Spot Identifier534could identify a task that consumes 10% of the desired CPU usage, and 15% against the desired time to complete. Non-Performing Tasks Identifier536identifies tasks that do not contribute to the KPIs, e.g., the tasks can be removed from the SAD pipeline and the SAD pipeline would still meet the KPIs. The tasks identified by the Non-Performing Tasks Identifier536according to one approach could be considered redundant tasks.

FIGS.6-7illustrate schematic block diagrams of a pipeline300that has been generated by SAD Pipeline Generator Module170that has been fed into the SAD Pipeline Optimizer180with the desired KPIs, and the pipeline300is optimized by adjusting, modifying, reconfiguring, updating, reordering, and/or removing tasks305to provide revised SAD pipeline300′. As shown inFIG.6Transform Function334(Transform Function2) and Model344(Model2) are identified for removal as they have negligible effect on improving F1 score for the pipeline300and consume resources and expend KPI capital. The configuration of Model342is identified for a configuration update inFIG.6, and Transform Function335(Transform Function3) has been identified as a hot spot as variations did not result in optimizing the SAD pipeline300or the task performed at Transform Function335.FIG.7is a schematic block diagram ofFIG.6illustrating the optimized SAD pipeline300′ where Transform Function334and Model344have been removed, the configuration of Model342has been updated to Model342′, and Transform Function335has been identified as a hotspot335′. Hotspot335′ in an embodiment indicates that task335′ was not able to be optimized and is still expensive in the SAD pipeline300′ in terms of resources (KPIs) consumed. In an aspect, task335′ is tagged as a hotspot so the data scientists and/or user of the system can review the hotspot task335′.

FIG.8is an exemplary flowchart in accordance with one embodiment illustrating and describing a method800of optimizing a pipeline for detecting suspicious activity based upon the desired Key Performance Indicators (KPIs). While the method800is described for the sake of convenience and not with an intent of limiting the disclosure as comprising a series and/or a number of steps, it is to be understood that the process800does not need to be performed as a series of steps and/or the steps do not need to be performed in the order shown and described with respect toFIG.8, but the process may be integrated and/or one or more steps may be performed together, simultaneously, or the steps may be performed in the order disclosed or in an alternate order.

The method800in one or more embodiments includes a data analytics pipeline for receiving data and detecting suspicious activity or behavior, and in an aspect in response to detecting suspicious activity generating an alert. The data analytics suspicious activity detection (SAD) pipeline can be generated in an embodiment by a data analytics system and/or platform, e.g., SAD Pipeline Module170. The method800includes in an embodiment at810receiving the data analytics (SAD) pipeline to be optimized, including its name, and the desired KPIs for the SAD pipeline. The desired KPIs, for example, can be the time for the SAD pipeline to run, the amount of CPU and/or memory, and/or the F1 score. The KPIs can be expressed and input as values or ranges, including minimums and maximums. The desired KPIs are typically received from the client and input into a data analytics system and/or platform, e.g., SAD pipeline Optimizer180, to optimize the SAD pipeline. For example, a client may desire the SAD pipeline to have a run time of less than 8 hour and have an F1 score better than 0.76. Other KPIs values and/or ranges are contemplated. The SAD pipeline is typically generated by a module, e.g., SAD Pipeline Module170, in a system, and received by an optimizer module, e.g., SAD Pipeline Optimizer180. In one or more embodiments, a user inputs the name of the SAD pipeline to be optimized and the system and/or platform, e.g., SAD Pipeline Optimizer180, can retrieve the pipeline to be optimized.

Fine tune variations can optionally be inputted at810. These fine tune variations (or options) can in an aspect be at the task level. In one or more embodiments, the fine tune variations, include reorganizing the pipeline tasks, for example, changing the order of steps depending upon their dependencies. For example, changing parallel steps to sequential steps, or changing serial steps to parallel steps. At810, the fine tune variations can optionally be input into a system or platform.

At820, the KPI values from previous runs of the pipeline to be optimized are retrieved and/or gathered. In one or more embodiments, the KPIs are retrieved from the pipeline run manager, e.g., Pipeline Run Manager230. The pipeline KPIs from the previous runs is generally captured by the Task Level KPI Capturer522. At830, optionally the fine tune variations are applied to simulate and optimize the pipeline based upon the pipeline dependencies. In an aspect optimizing the pipeline flow is performed in a Pipeline Flow Optimizer524. At830, the ordering of the SAD pipeline tasks can be reconfigured, e.g., serial tasks can be converted to parallel tasks and vice-versa. Block830is an optional step and is not necessary to be applied in this manner at this stage of optimizing the SAD pipeline.

At840, identify the candidate SAD pipeline configurations for simulation runs. That is, at840, candidate SAD pipelines that have different configurations are identified for simulation runs to determine the effect the changes have on the KPIs for the different SAD pipeline configurations. In one or more embodiments, the KPIs captured at the task level is a starting point to identify the different SAD pipeline configurations that could have success in optimizing the SAD pipeline. For example, tasks that had an outsized effect, e.g., a large or small effect, on the KPI of the pipeline and/or task are identified, and candidate SAD pipeline configurations can be derived from changes based upon that information. Tasks that consumed large amounts of CPU, memory, and/or run time for example can be targeted for reconfiguration, while tasks that have a small effect on the F1 score can likewise be targeted for reconfiguration, e.g., removal from the pipeline.

In this manner, at840a number of candidate SAD pipelines can be identified for simulations runs for determining the optimized SAD pipeline for the desired KPIs. In one or more embodiments, a number “N” candidate SAD pipelines that are reconfigured from the original SAD pipeline can be identified for simulation runs based upon using the KPIs gathered from the original pipeline simulation run and using that information to attempt to identify those SAD pipeline configurations likely to have the greatest effect on optimizing the SAD pipeline. In one or more embodiments, a brute force model can be used to create a matrix of all possible SAD pipelines as a manner of identifying at840candidate SAD pipeline configurations for simulation runs.

At850the candidate SAD pipeline configuration runs identified at840are simulated. In one or more embodiments, the KPIs for each simulation run of each SAD pipeline reconfiguration are captured and/or recorded. The KPI information for each candidate SAD pipeline can be recorded on the task level and/or global pipeline level. At860the simulation results, e.g., the simulation runs of the candidate SAD pipeline reconfigurations, are analyzed. For example, analysis of the simulation results at860can identify the optimal SAD pipeline configuration at870, and/or identify hotspot tasks at880, and/or identifying non-preforming tasks at890. The non-performing tasks can be removed and/or targeted for removal from the SAD pipeline. In one or more embodiments, analyzing the simulation runs at860can result in generating a report, and the report can identify the optimal SAD pipeline configuration, identify the hotspots, and/or the non-performing tasks.

FIG.9illustrates an example computing system in accordance with the present invention. It is to be understood that the computer system depicted is only one example of a suitable processing system and is not intended to suggest any limitation as to the scope of use or functionality of embodiments of the present invention. For example, the system shown may be operational with numerous other special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and/or configurations that may be suitable for use with the system shown inFIG.9may include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems or devices, and the like.

In some embodiments, the computer system may be described in the general context of computer system executable instructions, embodied as program modules stored in memory16, being executed by the computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks and/or implement particular input data and/or data types in accordance with the present invention.

The components of the computer system may include, but are not limited to, one or more processors or processing units12, a memory16, and a bus14that operably couples various system components, including memory16to processor12. In some embodiments, the processor12may execute one or more modules10that are loaded from memory16, where the program module(s) embody software (program instructions) that cause the processor to perform one or more method embodiments of the present invention. In some embodiments, module10may be programmed into the integrated circuits of the processor12, loaded from memory16, storage device18, network24and/or combinations thereof.

The computer system may also communicate with one or more external devices26such as a keyboard, a pointing device, a display28, etc.; one or more devices that enable a user to interact with the computer system; and/or any devices (e.g., network card, modem, etc.) that enable the computer system to communicate with one or more other computing devices. Such communication can occur via Input/Output (I/O) interfaces20.

Moreover, a system according to various embodiments may include a processor, functional units of a processor, or computer implemented system, and logic integrated with and/or executable by the system, processor, or functional units, the logic being configured to perform one or more of the process steps cited herein. What is meant by integrated with is that in an embodiment the functional unit or processor has logic embedded therewith as hardware logic, such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc. By executable by the functional unit or processor, what is meant is that the logic in an embodiment is hardware logic; software logic such as firmware, part of an operating system, part of an application program; etc., or some combination of hardware or software logic that is accessible by the functional unit or processor and configured to cause the functional unit or processor to perform some functionality upon execution by the functional unit or processor. Software logic may be stored on local and/or remote memory of any memory type, as known in the art. Any processor known in the art may be used, such as a software processor module and/or a hardware processor such as an ASIC, a FPGA, a central processing unit (CPU), an integrated circuit (IC), a graphics processing unit (GPU), etc.

It will be clear that the various features of the foregoing systems and/or methodologies may be combined in any way, creating a plurality of combinations from the descriptions presented above. If will be further appreciated that embodiments of the present invention may be provided in the form of a service deployed on behalf of a customer to offer a service on demand.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Unless otherwise specifically defined herein, all terms are to be given their broadest possible interpretation including meanings implied from the specification as well as meanings understood by those skilled in the art and/or as defined in dictionaries, treatises, etc. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. The corresponding structures, materials, acts, and equivalents of all elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present invention has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the invention in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the invention. The embodiment and terminology was chosen and described in order to best explain the principles of the invention and the practical application, and to enable others of ordinary skill in the art to understand the invention for various embodiments with various modifications as are suited to the particular use contemplated.