Multiple sources of data in a bayesian system

Data for a transaction is modeled by receiving a source set of data. The source set of data comprises data representing a plurality of transactions stored in a source transaction database. An estimation model for modeling data for a transaction is received. A mapping between the source set of data and a model parameter database is received. The model parameter database comprises a plurality of model parameters for the estimation model. The parameters extracted from the model parameter database and the source set of data in a Bayesian framework are combined using a parameter estimation engine to obtain an updated set of model parameters. The updated set of model parameters is stored in the model parameter database.

BACKGROUND OF THE INVENTION

Statistical modeling systems input historical or known data and output a model or decision based on the input data. For example, a statistical distribution function may be output where the exact “shape” of the function depends upon one or more parameters (e.g., the first parameter of the distribution function has a first value determined by the statistical modeling system, the second parameter has a second value, etc.). In one example, a linear model is output where the linear model is described by the function y=β0+β1x+ε and the parameters in that example are β0and β1. In general, the more information input to a statistical modeling system the better the quality of the resulting model. However, there may be a number of issues which prevent additional information from being used. For example, a set of known or historical data may be owned or managed by an entity that is unwilling to share information because of competitive reasons and/or legal reasons (e.g., the information may include sensitive personal and/or financial information). It would be desirable to develop statistical modeling systems that overcome some of these issues so that more information can be used.

DETAILED DESCRIPTION

FIG. 1is a diagram showing an example of a traditional modeling system. In the example shown, source set of data104is a collection of known or historical data. Source set of data104is owned by or managed by an entity for which the statistical modeling (performed by modeling system100) is being performed. In some cases, modeling system100is owned and/or managed by some other entity, such as a company that specializes in modeling.

Data from source set of data104is sent or otherwise accessed by modeling system100via network102. In various embodiments, a network includes a variety of networking devices and/or technologies including (but not limited to) wired/wireless communication, Internet related protocols, etc.

In one example, source set of data104is associated with a bank and system100is associated with a financial services company hired by the bank to determine prices for the bank's products (e.g., rates on loans offered by the bank). In a closed loop system, the bank sets its prices (or some other adjustable value) based on the decision or estimate made by system100, the market response to the new price is observed (e.g., whether or not consumers are taking loans with the new rates), and this information is fed back into source data set104.

In another example, source set of data104contains data related to a medical research or trial. For example, a medical study may focus on which of two similar drugs is better for treating a particular disease and source set of data104includes the medical history of a study group (including prescribed drugs, dosages, how long a particular drug was prescribed for, etc.). In that application, modeling system100may access the stored medical information in source set of data104and generate models of the response or efficacy of the medicine.

In general, any data may be used and operated on using the techniques disclosed herein. Although certain specific applications or types of data are disclosed herein (e.g., financial data), the techniques disclosed herein are not necessarily limited to those types of data or applications.

In general, when generating or outputting a model or estimate, it is better to have more information. In the example shown, a decision is made based on information owned or managed by one particular entity, such as a single bank or a single group of medical study participants. Other information may exist (e.g., information owned by another bank or another group of medical study participants) but is not used in the system ofFIG. 1. The following system shows an embodiment of a Bayesian system that uses multiple sources of data.

FIG. 2Ais a diagram showing an embodiment of a multi-source Bayesian system at a first point in time. In the example shown, multi-source Bayesian modeling system200includes parameter generator203and model parameter database201. Parameter generator203is used to generate parameters based on known data. These parameters generated by generator203are then stored in database201. When a model is desired, the stored parameters in model parameter database201are used to generate the model.

In the example shown, data A (202a) is owned or managed by a first entity (e.g., a first bank or a first researcher), data B (202b) is owned or managed by a second entity, and source set of data104is owned or managed by a third entity. This is merely one example. In another embodiment, data A and B (202aand202b) are owned by the same entity and source set of data104is owned by some other entity. In some embodiments, a source set of data (e.g.,104) includes the same or similar types of information as other groups of data (e.g., data A and B202aand202b) and/or have the same types or form of data (e.g., records of individual events or transactions).

As used herein, a source set of data refers specifically to data that is owned or managed by an entity for which a model is being generated (e.g., a customer in this figure). For the example ofFIG. 2A, a model is generated for the entity associated with source set of data104. If a model were being generated for some other entity, then data104would be referred to simply as “data” and not a “source set of data”.

In general, a given source set of data i can be expressed as Si(n, {right arrow over (y)}, X) where n is the number of observations for that particular source set of data, {right arrow over (y)} is a vector of responses, and X is a n by k matrix of variables. The vector {right arrow over (y)} represents the outcomes of the quantity being modeled and its values are known or historic values of y. The matrix X includes other information associated with or pertaining to each particular response for a given observation. For example, if the vector {right arrow over (y)} is whether or not a customer accepts an offered loan, the matrix X may include city and state where the applicant lives, income, marital status, age, FICO score, loan amount, terms of loan, etc. If high blood pressure medication is being modeled or estimated, then the vector {right arrow over (y)} may be blood pressures of study participants and the matrix X includes gender, age, whether the person smokes, etc. In some embodiments, data A and B (202aand202b) can be expressed using the same general function.

In this example, parameter generator203accesses data A and B (202aand202b, respectively) via network102. Using this information, parameters are generated by generator203and are passed to model parameter database201where they are stored. In various embodiments, the particular parameters generated depend upon the particular model used by the system and assumed distribution of the response or error. For example, in some configurations a linear model (i.e., y=β0+β1x+ε is used and the parameters β0and β1are generated or otherwise obtained from data A and B when the error distribution is Gaussian. In some other embodiments, a different model is used which corresponds to a different set of parameters. For example, suppose an exponential model (i.e. y=exp(α0+α1x)+ε) is used where the y are assumed to follow a Poisson distribution. In such embodiments, the parameters α0and α1are obtained for data A and B (202aand202b) and are stored in model parameter database201.

In some embodiments, two different sets of parameters are generated and stored for two different sets of data. In other words, it is not necessary for the same, homogenous set of parameters to be used for all data that is processed. For example, data A (202a) may be modeled using a first model having a first set of parameters (e.g., β0and β1). Values for that first set of parameters are generated by generator203and are stored in database201. In some embodiments, data B (202b) uses a different model and values for a second set of parameters (e.g., α0and α1) are generated and stored. In some embodiments, the same set of parameters is used for data A and B (202aand202b). For example, a first set of parameter values are generated and stored for data A (e.g., β0Aand β1A) and a second set of parameter values are generated and stored for data B (e.g., β0Band β1B). In some embodiments, when processing a group of data, a user specifies to parameter generator203which model (and thus, which parameters) to generate for a given group of data. In some embodiments the model and corresponding parameters are automatically selected.

In some applications, the number of parameters generated by generator203and stored in database201is much smaller than the number of samples in an original or “raw” data set. In some cases, data A and/or B (202aand202b) includes hundreds of thousands or millions of records whereas database201includes hundreds or thousands of parameters. In some cases, the records describe an individual event or transaction, for example a car loan. The record may include information over the lifetime of the event, for example beginning from an application process (e.g., applicant's credit score, down payment, etc.) to how the loan was concluded (e.g., paid off by borrower, defaulted, etc.). If both data A and data B (202aand202b) are modeled using a linear model (i.e., y=β0+β1x+ε) then only two parameters are stored.

FIG. 2Bshows an embodiment of a multi-source Bayesian system at a second point in time.FIG. 2Bshows the system ofFIG. 2Aat a later point in time. In this example, parameters have been generated using data A and B and are stored in model parameter database201. Data A and B (202aand202b) in this particular example are no longer available via network102and the original data can no longer be accessed.

Data from a source set of data (104) is accessed by multi-source Bayesian modeling system200via network102. Using the source set of data (104) and parameters in model parameter database (201) a decision is made or a model is estimated by multi-source Bayesian modeling system200. In some embodiments, this is performed by first having parameter generator203generate parameter values using source set of data104(e.g., by mapping variables in source set o data104to corresponding parameters using a map) and then storing the parameter(s) in database201. Using the information stored in database201(e.g., including the parameter(s) generated from data A (202a), data B (202b), and source set of data (104)), an estimate or model is generated for the customer. Later, the parameters generated and stored in database201can be used for another customer if desired.

In various financial embodiments, a model or other decision made by multi-source Bayesian modeling system200is customized or generalized. A customized decision is one that is made for a particular person, entity, and/or transaction. For example, if a person wants a line of credit using their home as collateral, a personalized decision is one that is made only for that person and typically the terms of the offer (e.g., for a line of credit) are not known a-priori. In contrast, a generalized price is determined ahead of time for one or more groups. For example, people may be divided into one or more groups based on income and/or credit score. For each group, the people in a given group are offered the same price or terms.

Returning toFIG. 2B, suppose the source set of data (104) is associated with a bank and the bank hires a financial services company to determine prices for the bank's products. Multi-source Bayesian modeling system200determines optimal parameters (or makes some other decision) based on information obtained from that bank, as well as parameters from other sources or entities (e.g., generated using data A and B and stored in model parameter database201). Compared to the system shown inFIG. 1, information from multiple sources (and thus more information) is used in making a decision or model which in general improves the performance of the system.

One benefit to the system shown inFIGS. 2A and 2Bis that the system can estimate model parameters or make a decision based on multiple sources of data even when the original data (e.g., data202aand202b) is no longer available. This occurs in real-life for a variety of reasons. In one example, a data set includes patient records or patient information. The owner of the patient information (e.g., a hospital) may be unwilling to provided access to the information for an unlimited period of time because of patient confidentiality laws or liability issues. In another example, there is a contractual agreement to only have access to information for a certain amount of time or until a task is completed. Multi-source Bayesian modeling system200overcomes these issues since only parameters are stored in model parameter database (which are anonymous and do not include identifying information such as names) and access to data202aand202bcan be cut off after the parameters have been obtained and stored. Also, since parameters are different from the “raw” or original data, they can be kept and used later even if there is a contractual obligation to return or delete the “raw” or original data. That is, the parameters are the property of the entity that generated the parameters, not the entity that owns or manages the raw data itself.

Multi-source Bayesian modeling system200is also different from a modeling system in which data from multiple sources is crudely combined together (e.g., source set of data (104) ∪ data A (202a) ∪ data B (202b)). Data A and B (202aand202b) may contain millions of transactions or other pieces of information. The parameters determined and stored in model parameter database201range (in some embodiments) from one or two parameters to hundreds of parameters. Much less information needs to be stored and processed. In contrast, data which is merely combined together would be quite large, making it difficult to store and manage. Also, data sets do not always have corresponding or similar data and crudely combining data together provides little or no instruction on how to handle situations in which there are different types of data, one set has no information but another set does, etc. For example, if information from two different medical studies are combined, different techniques may have been used or different patient information may have been recorded (e.g., one study asked a patient's ethnic background and the other did not record this information). Multi-source Bayesian modeling system200includes techniques to address this issue of processing data from multiple sources. In some embodiments described in further detail below, a mapping is used to correlate or indicate which pieces of data in the source set of data (104) correspond to which parameter(s) (if any) stored in model parameter database201. In some embodiments described herein, parameters are used (as opposed to the “raw” data such as data202aand202b) and therefore a mapping (or other technique) is between a source set of data (104) and parameters in database201, not between the source set of data (104) and data A and B (202aand202b).

The examples shown inFIGS. 2A and 2Bare merely illustrative of the techniques which are sought to be patented. In some cases, data202a,202b, and104may be connected to network102at the same time. In some embodiments, the ordering of processing is reversed. For example, in some configurations the source set of data104is processed first, and then data202aand202b. Any number or amount of data can be processed. For example, additional sets of data owned or managed by some other entity (or alternatively, source data owned or managed by the entity for which the modeling is being performed) can be processed after the state shown inFIG. 2B.

FIG. 3is a flowchart illustrating an embodiment of a process for managing a model parameter database and performing Bayesian modeling using a model parameter database. In the example shown, information from multiple sources is used, for example from a plurality of financial institutions, medical studies, etc. In some embodiments, multi-source Bayesian modeling system200shown inFIGS. 2A and 2Bis managed using example process300.

At302it is determined whether new data is available. For example, inFIG. 2Anew data202aand202bbecome available. If new data is available, the new data is accessed and an updated set of model parameters is generated and is stored in a model parameter database at304. InFIG. 2Afor example, data A and B (202aand202b) are accessed, an updated set of model parameters is generated by parameter generator203and is then stored in model parameter database201. In some embodiments, generating and storing an updated set of model parameters includes (re)generating an entire set of information and overwriting old information stored in the database. Alternatively, in some other embodiments, information is generated and stored in the database without writing over previously generated and previously stored information.

After updating at304(or, if there is no new data available, at306) it is decided whether to perform modeling. If so, the model parameter database is accessed and Bayesian modeling using stored model parameters is performed at308. For example, the parameters stored in database201are accessed and are used in Bayesian modeling. In some embodiments, some other information is used in performing Bayesian modeling (e.g., source set of data104associated with the customer inFIG. 2B).

After performing modeling at308or if no modeling was performed, at310it is determined whether the process is over. For example, a model parameter database may be shut down. If so, the process ends. Otherwise, it is determined at302whether new data is available.

FIG. 4is a flowchart illustrating an embodiment of a process for updating a model parameter database. In some embodiments, step304ofFIG. 3is implemented as shown. In some embodiments, a model parameter database is updated in some other manner.

At402, a source set of data is received where the source set of data comprises data representing a plurality of transactions stored in a source transaction database. For example, inFIG. 2B, source set of data104is received.

At404, an estimation model is received for modeling data for a transaction. For example, a multi-source Bayesian modeling system may be capable of generating and storing different parameters for different types of models (e.g., linear model, hazard model, etc.) and step404is used to specify which type of model (and thus, which parameters are to be used) for the source set of data received at402. In some embodiments, the estimation model is received from or otherwise specified by a user (e.g., a modeling expert). In some embodiments, the estimation model is determined or selected automatically For example, in modeling loan take-up probability for unsecured personal loans an example of a pre-defined model could have the form Probability(Take-up)=Logit(β0+β1Loan Rate+β2Probability of Default).

At406, a mapping between the source set of data and a model parameter database is received, where the model parameter database is used to store a plurality of model parameters for the estimation model. One embodiment of a mapping is described in further detail below.

At408, parameters extracted from the model parameter database and the source set of data are combined in a Bayesian framework using a parameter estimation engine to obtain an updated set of model parameters. For example, this includes using the mapping obtained at406and the source set of data obtained at402.

FIG. 5is a diagram showing a posterior distribution and associated graphs. In the example shown, a linear function y=β0+β1x+ε is output by a multi-source Bayesian modeling system. For this particular model (i.e., a linear model) the values of the parameters β0and β1are determined. Graph500shows an example line and corresponding data points from which the line is determined.

Graph502shows a distribution for the parameter β1. The x-coordinate of graph502shows possible values of β1and the y-coordinate shows the probability of a given value of β1. β1′ is the mean of the distribution shown in graph502and the value of β1selected by a multi-source Bayesian system will be in the range of β1′.

The particular distribution obtained or realized for β1will depend on the hyperparameters of the distribution. In this particular example, the hyperparameter σ corresponds to the variance of the distribution shown in graph502. Some other models or parameters may depend upon different hyperparameters. Graph504shows a distribution for the hyperparameter σ. The x-coordinate of graph504shows the possible values of σ and the y-coordinate shows the probability for a given value. The distribution of σ shown in graph504affects the distribution of β1shown in graph506which in turn affects the value of β1output by a multi-source Bayesian modeler.

Equation506shows a posterior distribution which comprises of two conditional probabilities which are multiplied together. Conditional probability508is the probability of data (such as the data points shown in graph500), given parameter(s) (such as β0and β1) and hyperparameter(s) (such as σ). Conditional probability510is the probability of the parameters given the hyperparameter(s). Probability512is the probability of a hyperparameter. Other Bayesian systems which do not use multiple sources of data typically “freeze” conditional probability510and/or probability512. For example, they may pick a value for a hyperparameter (such as a) and use that value without updating it. One problem with this is that a hyperparameter (and thus the value of a corresponding parameter determined by a system) may change over time. For example, in the case of a medical study of an antibiotic, the antibiotic may become less effective over time as the organisms evolve and develop a resistance to the drug. Freezing conditional probability510may prevent a Bayesian system from having as accurate an estimate as would be desired. In contrast, a multi-source Bayesian system does not freeze conditional probability510.

As used herein, a hyperparameter controls or otherwise affects the particular shape or realization of a given distribution. A distribution has one or more hyperparameters associated with it and one distribution may not necessarily have the same number or type of hyperparameters as another distribution.

Parameters are used to propose a mathematical model for an observed phenomena in terms of observed effects. A hyperparameter is used to formulate the distribution of likely values for a parameter of a model. For example, if it is assumed the values of a model parameter are randomly distributed about a mean value, then parameter values would follow a Gaussian distribution. In order to fully characterize a Gaussian distribution both the mean and variance would be required. The variance in this case would be a hyperparameter for any model parameter that follows a Gaussian distribution.

Parameter generator203in this embodiment includes model specification602. In this system, data600is accessed or otherwise passed to model specification602(e.g., via a network) where one or more models are specified for a particular effect being modeled (e.g., conversion, number of loans accepted in particular time period, utilization of a line of credit, etc.) and model specification602specifies a model to use. In one example, block602may specify to use a linear model having corresponding parameters. Data600in this example can be any data. For example, at the point in time shown inFIG. 2A, data600corresponds to data A and/or data B (202aand/or202b). At the point in time shown inFIG. 2B, data600corresponds to source set of data104.

The historical data included in data600is passed via model specification602to map604. As used herein, a map is a mapping which describes which variables in the samples or pieces of historical data map to which parameters. For example, in the case of a medical study, the variables may be age, gender, smoker/non-smoker, number of hours person works out in a week, etc. An example of a map is described in further detail below.

Based on which parameters are being mapped to (i.e., known from map604), stored values for those parameters are accessed from model parameter database201and passed to parameter estimation engine606via map604. In this example, parameter estimation engine606outputs a single value or function for a given parameter from multiple, stored values or functions. For example, a plurality of stored values or functions for the parameter λ may be passed to parameter estimation engine606and a single value or function for the parameter λ is output. In some embodiments, there are multiple parameters and a single one is output by parameter estimation engine606for each parameter. The end parameter will be a weighted average based on the hyperparameter(s) of the assumed distributions.

The parameter(s) output by parameter estimation engine606are passed to multi-source Bayesian modeler608, which also receives information from model specification602. In this embodiment, multi-source Bayesian modeler is configured to be able to operate or otherwise process a variety of models and so model specification602identifies which model to use. Model specification602also passes information from data600to multi-source Bayesian modeler608. For example, ifFIG. 6corresponded to the system shown inFIG. 2B, multi-source Bayesian modeler608would use information from data A and B (stored in model parameter database201and passed to modeler608via map604and parameter estimation engine606) and information from source set of data104(i.e., data600and passed to modeler608via model specification602).

In this example, multi-source Bayesian modeler608also inputs constraints from parameter space constraints610. Multi-source Bayesian modeler608uses these constraints (if any) to alter the probability distribution of the parameters in a way that would not otherwise come about based solely on the data passed to modeler608. Some examples of constraints include trending, shifting, rounding, applying a floor or ceiling value, etc. In some embodiments, parameter space constraints610includes expert knowledge. For example, there may be some knowledge an expert user has about the system and/or what the model that is output should resemble and this expert knowledge is input to the system using parameter space constraints610. In some embodiments, parameter space constraints610is optional and is not included in a parameter generator.

FIG. 7is a diagram showing an embodiment of a map between variables in data and parameters stored in a model parameter database. In some embodiments, variable map604is implemented as shown.

As described above, “raw” data in general can be expressed as Si(n, {right arrow over (y)}, X) where n is the number of observations for that particular source set of data, {right arrow over (y)} is a vector of responses, and X which is a n by k matrix of variables. The arrows between data variables700and model parameter database702(which include repository model parameters704and repository dimension parameters706) comprise map604. In this example, variable X0maps to M1of repository model parameters704, variable X1maps to Dd-1of repository dimension parameters706, and variable Xk-1maps to Km-1of repository model parameters704.

Some variables, such as variable Xkmay not map to anything. In such cases, this means there is currently no parameter stored in the model parameter database which maps to that variable. In some embodiments, a new parameter is created and stored. In some embodiments, information associated with that variable is not used.

Some variables, such as variable X1, map to a dimension in repository dimension parameters706. Dimensions are typically used to segment data into like groups in terms of response. For example, in modeling the number of vehicle loans originated by a particular financial institution, the data would typically be segmented along dimensions of vehicle age. As another example, age and gender would be typical dimensions in a medical study of drug effectiveness.

FIG. 8is a diagram showing an embodiment of a multi-source Bayesian system that processes financial data. In the example shown, model parameter database800and source set of data812include financial data. The columns in model parameter database800include a source (e.g., source1corresponds to loans from one bank, source2corresponds to loans from a second bank, and source3corresponds to loans from third bank), loan amount, FICO score, covariate, parameter, variance of the parameter, and min, max, and mean of the covariate.

Three groups of data801a-801care accessed from model parameter database800and are shown in graph802. Data801a-801ccorrespond to sources1-3, respectively and are shown as curves803a-803cin graph802. In this particular example, data801a-801cinclude loan information for people with FICO scores of 650 and are shown as parameters β1,1, β1,2and β1,3, respectively.

In some embodiments, the particular data or parameters that are obtained from model parameter database800depends upon which model is selected to be used. For example, in the system ofFIG. 6, model specification602specifies which model to use (and thus, which parameters are applicable).

In addition to the parameter β1priorwhich is output by parameter estimation engine804, multi-source Bayesian modeler808also receives information from source set of data812and general linear model810. In some embodiments, multiple models are available and some entity (e.g., model specification602in the embodiment ofFIG. 6) selects or specifies the model to be used and passes it to multi-source Bayesian modeler808. In this particular example, a linear model is used and the model is associated with whether or not a loan is booked (i.e., accepted by a customer in the event a loan is offered) and is β0+β2×Rate+β2×FICO+β3×Amount+β4×Rate×FICO. Multi-source Bayesian modeler also inputs source set of data812. In some embodiments, the entity that owns or manages source set of data812is some other entity besides sources1-3. In some embodiments, modeling is being performed for the entity that owns or manages source set of data812.