Machine learning archive mechanism using immutable storage

An apparatus and method for providing an immutable audit trail for machine learning applications is described herein. The audit trail is preserved by recording the machine learning models and data in a data structure in immutable storage such as a WORM device, a cloud storage facility, or in a blockchain. The immutable audit trail is important for providing bank auditors with the reasons for lending or account opening reasons, for example. A graphical user interface is described to allow the archive of machine learning models to be viewed.

BACKGROUND

Prior Application

This application is a priority application.

Technical Field

The system, apparatuses and methods described herein generally relate to machine learning techniques, and, in particular, to a mechanism for creating immutable archives.

Description of the Related Art

The banking industry has been criticized in the past for making decisions on lending, account opening and check cashing procedures based on the neighborhoods in which customers resided. This was seen as a form of racism because certain neighborhoods were predominantly filled with one race or another. Since these criticisms, banks and other lending organizations have instituted procedures to assure that their banking activities avoid any type of prejudices. The requirements to fairly conduct banking activities are codified in the Equal Credit Opportunity Act and the Fair Housing Act in the United States.

Up until the last few years, decisions on whether to loan money or open accounts were determined by bank employees who were trained in compliance with Fair Lending expectations. Bank Examiners then audited the banks for compliance.

In recent years, banks have turned to machine learning techniques to combat fraud in lending and in bank account opening. These machine learning algorithms are taught, given data sets of customers along with fraud activates. The machine develops its own algorithms based on the patterns in the data. Since the machine learning focuses solely on the data in the training sets, there is no way to enforce public policy techniques to assure that the Fair Lending rules are in compliance.

For instance, if a certain neighborhood has a high incidence of fraud, the computer, as part of its algorithm to detect fraud clusters and route banking activities away from these areas of fraud, may determine that certain neighborhoods are high fraud risks, and the machine may restrict banking activities in those neighborhoods. There is a potential that this machine learning behavior will open up a bank to accusations of violations of the Fair Lending rules.

To avoid or limit liability, the reasons why an adverse action is taken should be saved so that the evidence of the reasoning behind the decision is retained. The saving of the reasoning must be in a form that prevents modification.

There are a number of techniques using in computing to avoid data loss and to assure the integrity of the data. Write-once-read-many devices such as write only compact disk devices (CR-R) and write only Digital Versatile Disc Recordable (DVD-R) devices provide a hardware solution that creates an immutable record of the data. Microsoft, in their Azure system, has created a Blob storage functionality that includes an immutable parameter that provides a software cloud immutable storage.

Blockchain technology also provides an immutable ability to store the reasoning concerning a decision so that a bank examiner or auditor can double check the decision. Blockchain technology is unmodifiable yet viewable, and can be added to without impacting the previously stored data. This allows new archive information to be added to the chain without impacting the integrity of the previously stored information.

There is a need in the banking industry to provide an immutable archive of the reasoning behind a decision to provide defensible parameters around the decision. The present inventions address these issues.

BRIEF SUMMARY OF THE INVENTION

An apparatus for archiving machine learning models is described herein. The apparatus is made up of a special purpose server with an immutable storage facility connected. The apparatus further includes an application that executes code on the special purpose server that sends data to a machine learning engine receives a result from the machine learning engine. A machine learning model, that is updated generally, is integrated with the machine learning engine. Every time the machine learning model is updated it is stored by the special purpose server in the immutable storage facility.

In some embodiments, the immutable storage facility is a blockchain, or a write-once-read-many storage product, or a software cloud immutable storage. Furthermore, customer data (which could relate to banking) could also be stored in the immutable storage facility by the special purpose server. This data could also include the machine learning result. The customer data could be stored each time the machine learning engine is called by the application. The machine learning model could be updated periodically or whenever the customer data is used to train the machine learning model.

A method for archiving machine learning models is also described here. The method is made up the steps of (1) receiving data from an application at a machine learning engine running on a special purpose server, (2) calling a machine learning model by the machine learning engine, (3) executing the machine learning model using the data to determine a result, (4) returning the result to the machine learning engine and to the application, (5) updating the machine learning model by the special purpose server, and (6) storing the machine learning model in an immutable storage facility when the machine learning model is updated.

The immutable storage facility could be a blockchain, a write-once-read-many storage product, or a software cloud immutable storage in some embodiments of the method. The steps of the method could also include (7) storing customer data in the immutable storage facility by the special purpose server. In some embodiments, the customer data relates to banking. The customer data could be stored each time the machine learning engine is called by the application and could also include the machine learning result. In some embodiments, the machine learning model is updated periodically or when the customer data is used to train the machine learning model.

DETAILED DESCRIPTION

There is a strong need in the machine learning industry to provide an audit trail for machine learning models, so that bank regulators, legal teams, and internal auditors can verify that the decisions by the machine learning software was done without bias.

FIG. 1shows a block diagram of a machine learning architecture with the storage of the machine learning model103archived in immutable storage106. The training dataset102is first run through a training module101to develop the machine learning model103. As the training module101processes the training dataset102, the training module101plots the training data on a multi-dimensional graph and then attempts to create a mathematical model around the dataset. A sample training data set can be seen in the customer information inFIG. 5, items504,505,506,507,508and the result510.

A sample model is seen inFIG. 2, although displayed in a very simple, two-dimensional form. Positive data205and negative data204are plotted on an x-axis202and a y-axis201. The data is run through a curve matching algorithm, and the result is a linear equation y=Ax−B, which is shown onFIG. 2as line203. For example, the x value could be the customer's assets504and the y value could be the customer's income506. This is a very simple model. Using real world data, the model could be a seven or ten dimensional quadratic equation with scores of constants. To use this model, the machine learning engine function104is called. The machine learning engine function104returns a Boolean to the application105that specifies whether the input parameters x and y predicts positive205or negative204data. In this case, the machine learning engine function is a one line function: return (y>((A*x)−B)).

For a banking model103, the input parameters may include such factors as total assets504, total debt505, total income506, total expenses507, zip code508, and other criteria, and the model may decide whether to allow or deny a loan. In some uses, the model recommends and a human reviews the model's recommendation (called supervised machine learning), either approving or disapproving the model's result. If the human overrules the machine, the results are fed back into the training set so that the machine/model learns from the mistake.

Every time the model103is updated, the model is stored in the immutable storage106so that there is an auditable trail of why decisions were made by the machine. The immutable storage106could be a form of write once, read many hardware device such as a compact disk devices (CR-R) and write only Digital Versatile Disc Recordable (DVD-R) device. Read only memory chips could also be used in some embodiments. A version of the Secure Digital flash memory card exists in which the internal microprocessor does not allow rewrites of any block of the memory. There are multiple vendors providing Magnetic Storage technologies including (but not limited to) NetApp, EMC Centera, KOM Networks, and others. Prevention of rewrite is done at the physical disk level and cannot be modified or overridden by the attached computer.

Microsoft, in their Azure system, has created a Blob storage functionality that includes an immutable parameter that provides a software cloud immutable storage. Iron Mountain offers a WORM (Write-once-read-many) storage product. Both of these cloud services offer the ability to time limit items in the immutable storage, so that document retention policies can be enforced. Thus, while the stored records are immutable, they also expire after a predetermined time.

Blockchain technology also provides an immutable ability to store the reasoning concerning a decision so that a bank examiner or auditor can double check the decision. Blockchain technology is unmodifiable yet viewable, and can be added to without impacting the previously stored data. This allows new archive information to be added to the chain without impacting the integrity of the previously stored information.

A simple blockchain is nothing more than a list of transactions that is chained or linked together using novel data structures, and some basic cryptographic principles. Anyone can make a blockchain, and create blocks. To create secure, immutable blockchains, the blockchain must be distributed, and the ability to create blocks must be more effort and cost than the value of what is being protected. The rules need to provide mathematical certainty from how transactions are “signed” through to how much “proof of work” needs to accompany a block.

There are many parts to the secure blockchain's rulebook, and each and every part is essential to the scheme's overall security; no single element (including the blockchain) is enough to secure the transactions. The rules ensure that the computing power required to tear out a recent block, alter a transaction, then re-create all subsequent blocks is more than any attacker and even most governments could amass.

Any of these technologies could be used to provide the immutable storage106. In many embodiments, the training module101is run periodically, so the model103is updated periodically. This means that a reasonably small number of model103versions need to be stored. However, in some embodiments, particularly in supervised machine learning models, the model is updated in real time (or close to real time) as a user corrects the machine's decisions. In these cases the model103would need to be stored with high frequency in the immutable storage106. Special hardware may need to be implemented to store the volume of models103and at a performance that can handle the rapidity in which the model103changes.

In some embodiments, the data (FIG. 5) from the application105is also stored in the immutable storage106to allow a complete audit trail of inputs and model changes. This is useful in quality assurance applications to assure that the model behaves the same way when provided a set of test data. This data storage also provides a bank examiner or audit with all of the inputs, models, and outputs for a banking decision, allowing for verification of the reasoning behind the decision.

A model audit graphical user interface107could also be provided to review the history of the model changes in the immutable storage106. This audit GUI107could allow search capabilities as well as an ability to list when and how often the model changed. Graphical capabilities to visually present the model are features of some embodiments (i.e. show the curve on an x-, y-, z-axis, similar toFIG. 2).

In addition the audit GUI107could be used to reset the model103back to a certain time frame to allow selective running of the model with a certain set of data. Another feature could set the model103to a certain time or to certain values, allowing the machine learning engine104to resume using the GUI107selected model103. This feature is useful if the machine has been incorrectly trained through improper supervised learning, allowing the impact to be reversed by rewinding the model.

The model audit GUI107is not permitted to delete or alter the data in the immutable storage106, although the GUI107may be allowed to add to the immutable storage106to annotate the model records. This allows comments on why a historical or a new model was forced by the GUI107into the machine learning model103. The GUI107could also have facilities for printing or exporting one or more models to another program.

In some embodiments, the data input504,505,506,507,508to the model103is also stored in the immutable storage106. In this embodiment, the GUI107could allow the running of a data set or a range of data sets to be run through the current model103and the results returned to the GUI107.

Looking toFIG. 3, we see one possible embodiment of the model stored in a record300in the immutable storage106. The first data field in the record300is a link to the next record301in the linked list of records300. In most embodiments, the next record in the linked list contains the previous version of the model103(newest to oldest). In other embodiments, the record sizes could be fixed and the next record301would be unnecessary. The next record301could run the linked list from the oldest to the newest in other embodiments without detracting from the invention, although buffering techniques would need to be employed to hold the newest record300until the next model is to be saved so that the next record link301is known before writing. In other embodiments, the location is allocated before use so that it is known when the previous record is stored. In some embodiments, this link301is a two element array with one link pointing to the next record and the other link pointing to the previous record. In still another embodiment, a separate table of links to records is stored separate from the data to point to the records.

The next data field in the record300is the model version302. This is an optional field that specifies the version of the model103and could simply be an incremented counter.

A timestamp303is the next field in the record300, and records both a time and a data value. The timestamp303could be a two element array of the timestamp when the model began use and the timestamp when the model ended use. In other embodiments, either the beginning or ending timestamp is stored, and the other time implied by the previous or next record.

A machine learning algorithm304is next stored in the record300in this example embodiment. The algorithm304could be a string describing the type of curve used or a constant specifying the type of curve or algorithm used. In the example used inFIG. 2this would be a linear algorithm. In other models could specify a range, a power, a logarithmic, a polynomial, an exponential, a moving average, or a linear forecast algorithms, for example. Combinations or more complicated algorithms could be specified as well.

The next field specifies the order305of a polynomial or other formula. This field is not used in all algorithms. This field is in theFIG. 2example is set to one, because a linear equation is a first order polynomial.

The next four or more fields contain constants306,307,308,309for the algorithm. In theFIG. 2example, constants A and B are used, so field Constant A306will have the value of A and field Constant B307will have the value of B. The other constant fields308,309are of no concern and could be set to zero or left as random values.

Because of the complexities of machine learning algorithms, special purpose computing may be needed to build and execute the machine learning model described herein.FIG. 4shows one such embodiment. The user views the user interface107described here on a personal computing device such as a personal computer, laptop, tablet, smart phone, monitor, or similar device401. The personal computing device401communicated through a network402such as the Internet, a local area network, or perhaps through a direct interface to the server403. The server403is a high performance, multi-core computing device with significant storage facilities404in order to store the training data102for the model103. Since this training data102is continuously updated in some embodiments, this data must be kept online and accessible so that it can be updated. In addition, the real-time editing of the model103as the user provides feedback to the model103requires significant processing power to rebuild the model as feedback is received.

The server403is a high performance computing machine electrically connected to the network402and to the storage facilities404. Furthermore, the server403is electrically connected to the immutable storage405. Immutable storage facilities405are not common to most servers403, but are used in the inventions described herein. In some embodiments, the immutable storage405is located in the cloud and indirectly connected to the server403through a network.

In order to preserve a complete archive of the machine learning transactions, some embodiments will also store all data that is run through the model103in the immutable storage106. This data could look similar to that illustrated inFIG. 5.

The first data field in the record500is a link to the next record501in the linked list of records500. In most embodiments, the next record in the linked list contains the previous version of the model103(newest to oldest). In other embodiments, the record sizes could be fixed and the next record501would be unnecessary. The next record501could run the linked list from the oldest to the newest in other embodiments without detracting from the invention, although buffering techniques would need to be employed to hold the newest record500until the next model is to be saved so that the next record link501is known before writing. In other embodiments, the location is allocated before use so that it is known when the previous record is stored. In some embodiments, this link501is a two element array with one link pointing to the next record and the other link pointing to the previous record. In still another embodiment, a separate table of links to records is stored separate from the data to point to the records.

The next data field in the record500is the model version502. This field specifies the version of the model103that processed the data500. In some embodiments, rather than a version number, a link to the immutable storage106record300for the model is stored in field502.

A timestamp303is the next field in the record300. The timestamp303represents the date and time when the record is processed by the model103.

After the timestamp field303, the input parameters for the model are stored: customer assets504, customer debt505, customer income506, customer expenses507, and customer zip code508. Any of these customer input fields could be deleted or additional fields added without detracting from the inventions herein. These fields, in most implementations, would be the same fields in the training dataset102, with the addition of the correct result510.

The next field that needs to be stored in the data record500is the machine learning result509. This preserves the outcome of the model based on the above customer data set504,505,506,507,508. In most embodiments, this value is a Boolean.

The final field is the human result510. This is the result that a trainer or supervisor of the model103gives to the given user dataset504,505,506,507,508. If this value does not agree with the machine learning result509, then the machine model may need to be re-trained using the data. In some embodiments, a comment may be attached to the record500to memorialize the trainer's thinking in overriding the model103. In some machine learning embodiments, the result510is not based on a human trainer but on another machine or real-world result. For instance, if the loan approved by the machine learning model defaults, then the result510may change to a negative value, and the data used to retrain the model.

The foregoing devices and operations, including their implementation, will be familiar to, and understood by, those having ordinary skill in the art.

The above description of the embodiments, alternative embodiments, and specific examples, are given by way of illustration and should not be viewed as limiting. Further, many changes and modifications within the scope of the present embodiments may be made without departing from the spirit thereof, and the present invention includes such changes and modifications.