Abstract:
A database analysis system comprising a store for storing databases, a user input and a user output, and a processor. The store includes a first database having a plurality of records of prospective customers each record having predictor variables such as information about the prospective customer including contact details and customer characteristics, the predictor variables being discretely identifiable in separate fields, each variable being categorized into two or more categories, such as household income level, the store also includes a second database of responders comprising a plurality of records of predictor variables of selected customers such as actual respondents to an earlier survey. The processor is a d apted to compare the predictor variables for each of the records in the first and second database and to generate an output to the user output to enable the user to view the comparative result, such as in the form of a profile report.

Description:
FIELD OF THE INVENTION 
   The invention relates to a system for analyzing databases and in particular a system for enabling enhanced direct marketing campaigns by weighted selection of potential customers from a prospect database. 
   BACKGROUND 
   It is known to conduct analysis of customer databases in order to try to enhance the results of marketing campaigns such as direct mail shot campaigns. However, such known systems are not very sophisticated, very time consuming in their analysis and or do not provide sufficiently successful results. 
   SUMMARY 
   Accordingly, an aspect of the invention seeks to provide improvements in such analysis systems and preferably to provide enhanced processing times as well as improved marketing campaigns. 
   According to an aspect of the invention there is provided a database analysis system comprising a store for storing databases, a user input and a user output, and a processor wherein the store comprises a first database having a plurality of records of prospective customers each record comprising predictor variables such as information about the prospective customer including contact details and customer characteristics, the predictor variables being discretely identifiable in separate fields, each predictor variable being categorised into two or more categories, such as household income level, the store further comprising a second database of responders comprising a plurality of records of predictor variables of selected customers such as actual respondents to an earlier survey, wherein the processor is a d apted to compare the predictor variables for each of the records in the first and second database and to generate an output to the user output to enable the user to view the comparative result, such as in the form of a profile report. 
   Preferably the user output is configured to display the user viewable comparative result of the first and second database by different categories. 
   Beneficially, the processor can calculate a statistical significance for the comparative results of the first and second database and preferably wherein the statistical significance forms part of the user viewable display at the user output. 
   Also, the records of the first database are preferably compared to the records of the second database to create a comparative (or profile) report. Preferably the process of creating a profile report on the processor or analysis server is optimised to involve a maximum of two passes through the first, prospect database, independent of the number of variables. This can be achieved by the program maintaining a matrix of counts for every category of every variable for both analysis (responder database) and base sets (prospects database). The program should preferably ensure that transactional variables are correctly aggregated to the related person. This can be achieved by the program maintaining a matrix of Boolean flags for every category of every included variable. That is records related to one person may relate to one or more actual transactions for example in relation to a holiday company prospect database, an individual may have taken one, two or more holidays with that company, but the persons characteristics and therefore the primary predictor variables remain the same and in its simplest level, the individual should not have multiple entries for each transaction. 
   Preferably the user viewable comparative result comprises a category index value being a representation of a ratio of the percentage of the number of records for a given category in the second database over the total records for all categories for the associated variable, compared with the percentage of the number of records in the first database for the same category over the total number of records for all categories for the associated variable in the first database. 
   Also, the processor preferably drives the user output to provide a user viewable display comprising information representative of the relative penetration of a given category, preferably the representation is graphical, such as in the form of a bar chart representative, the size of which graphical representation is representative of the categories index value, the direction of which graphical is representative of, and or the colour of which graphical report is representative of the statistical significance. 
   Each record in the first database preferably has a unique identifier which preferably is in the form of an alphanumeric identifier and more preferably in the form of a unique reference number. 
   The second database is preferably a subset of the first database and each record in the second database comprises at least a unique identifier for each record in common with the same record in the first database, and preferably wherein the second database comprises records consisting only of a unique identifier common to records in the first database. 
   Additionally, the processor is preferably a d apted to use weighted modeling analysis of characteristics of predictor variables and results from the comparison of the first and second database to enable selection of records in the first database. 
   The processor preferably scores each of the records in the first database based on the weighted modeling analysis and preferably wherein the score is a numerical value having for example three significant figures. Also, the processor can be a d apted to band records in the first database into bands of scores, and or more preferably the processor is a d apted automatically to band the records according to predetermined criteria, such as twenty four bands between −12 to −11, and 11 to 12. Alternatively, a user interface enables a user to determine the banding of the records. 
   Preferably, the system is a d apted to hold out a proportion of the first database for analysis, thereby to prevent analysis using the weighted modeling analysis technique, and preferably wherein the proportion is set as a default such as 20%. 
   Preferably, the processor is a d apted to assess the overall information gained for each variable based on a weight average of the significant weighted modeling analysis measures for all categories of a variable, and more preferably the processor is a d apted to rank variables according to the information gain and enable selection of only the most important variables, and preferably wherein selection is automatic. 
   Also, a user interface can be a d apted to enable a user to exclude one or more variables from the weighted modeling analysis of the data. 
   The processor can be configured automatically to select variables for use in the weighted modeling analysis based on at least one of the overall information gain, and maximum and minimum index values for the categories of a variable. 
   Also, the weighted modeling analysis is preferably hybrid model using both information theory and bayesian probability theory. 
   Preferably, the processor is a d apted to conduct a comparison of the first and second databases to generate a comparative report and to conduct a weight of evidence analysis thereby to enable enhanced selection of individual records from the first database, and or wherein the processor is a d apted to conduct a comparison of the first and second databases to generate a mathematical model to enable enhanced selection of individual records from the first database. 
   Moreover, the processor can be a d apted to omit weighted modeling analysis calculations for categories with a statistical significance or z-score below a threshold value, preferably the threshold value is variable, and or the threshold value is 3 and or selectable by a user. 
   The system can comprise a server in communication with a user client computer system such as in a local area, wide area, or virtual private network, or other internet arrangement. 
   Preferably the system is a d apted to generate an output to the user of selected records from the first database, and more preferably the processor is a d apted to remove any records from the second database from the selected results database. 
   Another aspect of the invention provides a method of transferring marketing data between first (e.g. server) and second (e.g. client) computers comprising the steps of assigning a unique identifier to each record in a first marketing database, creating a second database of selected unique identifiers, and transferring the second database between the fist and second computers. 
   Preferably, the method also comprises the steps of enabling access to the first marketing database by one of the first and second computers and enabling extraction of records from the first marketing database using the unique identifiers from the second database. Preferably unique identifier comprises an alphanumeric string and preferably is a unique reference number. 

   
     BRIEF DESCRIPTION OF THE DRAWINGS 
     An example of a system according to the invention will now be described, by way of example only, with reference to the according schematic drawings, in which: 
       FIG. 1  is an overview of the architecture of a system according to the invention, 
       FIG. 2  is a table representing part of a prospects database, 
       FIG. 3  is a table representing a responder database, 
       FIG. 4  represents the process of creating a profile report, 
       FIG. 5  is a screen shot of a profile report, 
       FIG. 6  represents the process of applying the PWE model to records in the prospect database, 
       FIG. 7  is a screen shot of the variables (also known as “columns”, or “data fields”) of the responder database ranked by Information Gain, 
       FIG. 8  is a screen shot of a profile report with an a d ded column for the calculated PWE score and ZdExp, 
       FIG. 9  is a table representing the prospect database with assigned PWE scores for each record, 
       FIG. 10  is a screen shot of selection of responders, 
       FIG. 11  represents the process of selecting prospects by PWE score, 
       FIG. 12  is a screen shot of a response chart, 
       FIG. 13  is a screen shot of a revenue chart, 
       FIG. 14  is a screen shot of part of a segment gains table, 
       FIG. 15  is a screen shot of part of a cumulative gains table, and 
       FIG. 16  is a graph of profit versus segment. 
   

   DETAILED DESCRIPTION 
   Referring to  FIG. 1  there is shown a system  10  according to the invention comprising a user services section  12 , a web services section  14 , a business services section  16  and a data services section  18 . These sections are described for ease of understanding the invention which provides, in this form, web client access via the web services section  14  to a data analysis system, here in the business section  16  and enables user manipulation and selection of data. 
   The user services section  12  comprises web client PCs  20  running a client software application  22 , such as a Java applet that provides selection, analysis and reporting capabilities in a user-friendly graphical user interface environment. Preferably the software application  22  runs in a web browser and requires minimal or zero a d ministration on the client PC, the latest application software and data installing and updating procedures can occur automatically from the web server. 
   The web services section  14  comprises a web server (WS)  24  and a web server filter  26 . The WS  24  manages the communication and delivery of web pages to the browser of the client PCs  20 . A suitable WS  24  is, for example, Microsoft Internet Information Server (IIS) which is the standard web server software application delivered as part of Windows 2000 and Windows NT. 
   The web server filter  26  provides routing and scheduling of selection and analysis requests to the application server  16 . The web server filter  26  could be a plug-in ISAPI filter for the IIS  24  for example. 
   The business services section  16  comprises an application server  28  at the core of the present invention for managing data analysis requests. 
   The data services section  18  comprises an analysis server  30  an analysis data store  32 , a selection store  34  and a user a d min database  36 . 
   The application server  28  processes requests for analysis and selection by calling the analysis server  30 . User a d ministration data is accessed from the user a d min database  36 . The results are formatted rea d y for transmission and display in the web client PCs  20 . For efficient loa d balancing and easy a d ministration the application Server can run as a service under Windows 2000 and NT for example. 
   The analysis server  30  is an analysis engine that uses optimised selection from the data store  32  to deliver extremely fast count, cross tab and profile results. The analysis server preferably uses a standard server activation model such as COM (Component Object Model) or SOAP(Simple Object Activation Protocol). 
   The analysis data store  32  is preferably a compressed and indexed, column orientated data store optimised to deliver extremely fast count, cross tab and profile results. 
   The user a d min database can be, for example, a standard ODBC (Open Database Connectivity) database (such as Microsoft SQL Server or Access) that stores user a d ministration and system usage data. In data vending systems the database also stores the pricing model and purchase details. 
   The selection store  34  is a simple file database or file directory storage area on the server that can be used to hold saved selection criteria, copies of exported data and possibly uploa d ed files from a user. Typically saved selection criteria require minimal storage, exported data and user files have storage requirements dependent on the number of records and size of user information or exported fields. 
   There are various physical configurations of the software application running on the client PCs  20 . The Web Services  14 , application server  16  and data services  18  may be installed on the same physical server or on separate machines linked by a fast network connection. It is also quite possible to install both client and server components on a standalone PC. It will be appreciated that each of the sections  12 ,  14 ,  16  and  18  comprise a computer in the form of a PC or a server having a processor capable of data analysis. 
   Referring to  FIG. 2  there is shown a prospect database  40 . The prospect database is a large set of prospect records  42  that may potentially be used for direct marketing. The prospect records  42  may be consumers or businesses, for instance. Examples of consumer prospect databases  40  include: customers of a financial services company e.g. bank, building society, insurer; customers of a utility or telecommunications company; comprehensive consumer lists built from electoral role, credit and questionnaire data and supplied by marketing data providers such as Experian and Equifax. Examples of business prospect databases include: business customers of a financial services company e.g. bank, insurer; business customers of a utility or telecommunications company, comprehensive business lists built from Companies House, credit and questionnaire data and supplied by marketing data providers such as Experian and Dun &amp; Bra d street. 
   For each record  42  in the prospect database  40  certain information is held such as: a unique reference or entry marker such as an alphanumeric string of characters and/or numbers, or simply a unique reference number (URN)  44  identifying each record  42  e.g. customer ID, account number; name and a d dress details  46 ; transactional data gleaned from operational systems e.g. start dates, billing amounts, credit history, contact history; and a d ditional data  48  such as postal sector, age, income, household type, house type etc. for consumers or postal sector, industry type, turnover, number of employees for businesses. This information is categorised into a number of fields  50  and preferably each record has an entry in each field although the entry may be missing. 
   The a d ditional data  48  may be gathered from application forms or questionnaires or purchased from marketing data providers. Data protection legislation may limit the extent of information that may be held on individuals. The fields  50  created from the transactional data and a d ditional data  48  provide the predictor variables that are analysed later. All variables&#39; values can preferably be divided into a number of categories; numeric data is preferably “banded” into discrete categories. 
   In some cases the prospect database  40  is only a subset of the total data available, for example if a service is not available in a certain geographic regions or to certain age groups then these may be omitted from the prospect database  40 . The following records  42  are an example of an excerpt from a prospect database  40  sourced from a fictional cable telecoms company and includes: Unique reference number (URN)  44 , Name  52 , Region  54  (full a d dress details omitted for this example), months since start  56 , subscription type  58  and the original application source  60 , year of birth  62  (YOB), employment type  64  (Job), house type  66  (House) and banded income  68  (Income Band). In a data set there are often many more transactional data and a d ditional data  42  fields  50 . 
   Referring to  FIG. 3  there is shown a responders database  70  representing records of responders  72 . The responders  72  could be new records which are not in the prospect database  40  but are usually a subset of the prospect database  40  preferably represented by a set of unique reference numbers (URNs)  44  or other unique reference. Examples of responders  72  are: consumers or businesses who are customers for a certain product or service; consumers or businesses that have replied positively to a marketing promotion; consumers or businesses that form an important customer segment e.g. high spending, long-term, lapsed subscribers. The responders&#39; subset  70  can be identified by a marker for their records  42  such as a set of URNs  44 . The URNs  44  may be identified directly by running a query on the prospect database  40  or the URNs  44  may be captured as the replies are processed depending in the nature of the responders  72 . If an externally sourced prospect database  40  is used it is necessary to use a name and a d dress matching process to link the responders to URNs  44  in the prospect database. Data providers such as Experian and Dun &amp; Bra d street provide name and a d dress matching as part of their services. 
   An a d vantage of marking each record with a URN  44  or other similar unique reference is that a set of URNs  44 , or similar, makes a relatively small data file that uniquely identifies the responders  72 . Because no name and a d dress or other data is attached the file is safe to transfer across the Internet. The URNs  44  uniquely identify the responders  72  in the prospect database  40 . 
   The software application running on the client PCs  20  preferably includes a function to import URN files. This function allows the user to identify the location of the Responders on their PC  20  or selection store  34  and if necessary uploa d the file across the internet. 
   Referring to  FIG. 4  there is shown a block diagram  80  of the process of comparing data in the responder and prospects databases to create a profile report. The software application “profiles” by comparing the responders  72  (the Analysis set) against the backdrop of the whole prospect database  40 . The software application  28  generates a profile report  82  that highlights market sectors in which the responders  72  are over or under represented relative to the prospect database  40 . This information can be used directly to guide marketing strategy and in particular more accurately determine prospect list selection. To generate a profile report  82  in the software application program, the user identifies the responders  70  as the Analysis set and requests the PC  20  to generate a profile report  82 . 
   The process of creating a profile report  82  on the analysis server involves only a maximum of two passes through the prospect database, independent of the number of variables, typically requiring only a few seconds of processing time. This is possible since the program maintains a matrix of counts for every category of every variable for both analysis and base sets. The program also maintains a matrix of Boolean flags for every category of every included variable to ensure that transactional variables are correctly aggregated to the related person. 
   An example of a profile report  82  can be see in more detail in  FIG. 5 . The profile report  82  has a section for each included variable with a row for each category  84 . The report shown was generated from the same source data as the prospect database  40  examples. For each category  84  can be rea d the number and percentage of records  34  in the responders  70  set (Analysis) and the prospect database  40  (Base). The remaining columns  86 , comprising columns for the index  88 , z-score  90  and Penetration  92  show the market penetration. 
   The Index  88  is the ratio Analysis %/Base % multiplied by one hundred. An Index  88  of over one hundred shows over representation in the category; less than one hundred shows under representation. 
   The standard z-score  90  indicates the statistical significance. The z-score  90  of a proportion is a standard statistical result. The magnitude of the z-score  90  can be interpreted as the number of standard deviations of the Index  88  away from the expected value of the proportion if the Analysis set was a random sample. The sign of the z-score  90  indicates the direction of the deviation from the expected value, above or below. 
   The Index  88  is illustrated by the size and direction of the Penetration graphic  92  and the z-score  90  is illustrated by the colour, such as from yellow (low significance i.e. low z-score) to red (high significance i.e. high z-score). 
   In this example it can be seen that responders  72  are over represented in the 10–15 k and 15–20 k income bands but under represented in higher income bands relative to the base population. 
   It is difficult to aggregate the information in the profile report to rank the prospect database and hence select the best new prospects for marketing. For example, just choosing the intersection of the best categories is not usually a good strategy, as often very few prospects will have all of these attributes. Instea d the invention produces a predictive weight of evidence (PWE) modelling technique, described later, which uses the characteristics of the predictor variables (e.g. House Type, Income Band) and the results of the profile report  82  to develop a balanced model and score the prospect database  40 . 
   Referring to  FIG. 6  there is shown a block diagram  94  representing the process of applying the PWE model  96  to records in the prospect database  40 . The software application  28  uses the predictive weight of evidence (PWE) technique to extract a predictive classification model directly from the profile report  82 . The technique is fast, automatic and requires a minimum of user input. The profile report  82  is easily rea d and therefore the marketing analyst can understand the factors included in the model. The PWE model  96  is used to score all the records  42  in the prospect database  40  based on the classification information “learned” from the profile report  82 . The score is used to rank and pick off the best prospects for marketing and can displayed as an extra column  98  in both the prospect database  40  and the profile report  82 . 
   Model Reports  100  to illustrate the potential gains ma d e by using the PWE model  96  can be generated by measuring how well the scores predict the known responders  72  and are described in detail later. 
   Before applying the PWE model  96  a hold out sample can be specified to reserve some data so that a later test and report on the model using a sample of data that was not used to build the model can be conducted. The default is to build the model using 80% of the data and report using the remaining 20%. 
   The software application  22  displays a table  102  of all the variables analysed in the profile report  32  as is shown in  FIG. 7 . The information gain  104  ranks the variables by their power in predicting whether a record is a Responder or not. 
   The overall variable information gain  104  is calculated as the weight average of the significant PWE measures for all categories of the variable. The weights are preferably the category proportions in the prospect (second) database. The information gain  104  and the maximum and minimum index  106  and  108  can be used to decide which variables to include in the model  96 . At this point the user may also apply background knowledge of the data and business to exclude variables from a group that represent the same information e.g. to not include both postal code and town; to include only the business classification with the highest information gain. This manual intervention is optional—the PWE technique automatically gives most weight to the most important variables. Once the variables have been chosen the user simply inputs a request to the PC  12  to build the model  96 . 
   The Predictive Weight of Evidence (PWE) model  96  is a hybrid model building technique based on information theory and Bayesian probability, to extract predictive information from the profile report  82  to create a score for each prospect record in the prospect database  40  indicating the propensity for the record  42  to be a responder. For each record  42  in the prospect database  40  the category for each predictor variable can be rea d . With knowledge of this category the corresponding predictive weight of evidence (PWE)  98  measure from the corresponding category row in the profile report  82 A as shown in  FIG. 8  can be rea d also. If the category is not known (missing or invalid) for the prospect record a PWE measure of zero is assigned. 
   As an example and continuing to refer to  FIG. 8 , if the prospect record has Income Band “10–15 k”  110  the PWE is 0.60. This is interpreted as evidence of 0.60 for the record being a responder. If however the prospect record has Income Band “40–50 k”  112  the PWE is −0.58. This is interpreted as evidence of 0.58 against the record being a responder. The examples from the prospect database  40  and responders database  70  given earlier seem to suggest that responders  72  come from the lower income bands. The PWE method tests the relationship statistically (using the full files) and produces the PWE measures  98  that indicate the strength of the relationship relative to the other predictor variables. 
   The PWE category score is derived from the numbers in the profile report  82  by using information theory to consider the evidence for and against membership of the responders&#39; set  70  given membership of the category. 
   The PWE score  98  is based on the predictive weight of evidence that membership of a variable category A i  gives that a record should be classified as a Responder. In probabilistic terms the interest is in the probability of the “classification to Y (Responder)” or “classification to N (Not Responder)” event occurring. 
   The PWE category score is developed (initially for one variable, A) by considering the difference in the gain of information when a record characterised by category A i  is assigned to Y (“Responder”) as opposed to the alternative class N (“Not Responder”). 
   Initially the PWE can be calculated for one variable A.
 
 PWE ( Y/N|A   i )= G ( Y|A   i )− G ( N|A   i )
 
Where i is the index of the category e.g. 15–20 k and G is the information gain.
 
   For example, if we know that a prospect has income band 10–15 k this provides an amount of “evidence” that the record may be a Responder and similarly “evidence” that the record is Not Responder. The PWE is the difference between these two values. 
   To calculate G(Y|A i ) the Information Gain on classification to class Y based on membership of category A i  we need:
         the information value (for classification to class Y) of knowing Y is I (Y) where
 
 I ( Y )=log 2 {1 /P ( Y )} bits (from Shannon&#39;s Information Theory)
   where P(Y) is the probability of Y i.e. probability that any prospect is a responder;   −360   and the information value (for classification to class Y) of knowing Y|A i  is I (Y|A i ) where
 
 I ( Y|A   i )=log 2  {1 /P ( Y|A   i )} bits (from Shannon&#39;s Information Theory)
   where P(Y|A i ) is the probability of Y given membership of category A i  i.e. probability that any prospect with category value of variable A=A i  is a Responder (e.g. probability that prospect is a Responder given that Income Band=10–15 k.).   G(Y|A i )=information gain on classification of Y provided by knowledge that the category value of variable A is =A i  
 
 G ( Y|A   i )=value of information of  Y|A   i −value of information of  Y  
 
 G ( Y|A   i )= I ( Y|A   i )− I ( Y )
       

   Note that knowledge that the category value of variable A is =A i  (e.g. knowledge that income band is 10–15 k) will at worst give us no information therefore I (Y|A i ) will always be greater than I(Y) hence G(Y|A i ) is always positive.
 
 G ( Y|A   i )=log 2 {1 /P ( Y|A   i )}−log 2 {1 /P ( Y )}
 
 G ( Y|A   i )=log 2   {P ( Y|A   i )/ P ( Y )}
 
and similarly for classification to class N,
 
 G ( N|A   i )=log 2   {P ( N|A   i )/ P ( N )}
 
The probabilities are estimated using the observed frequencies in the profile report  82 .
         P(Y) and P(N) are estimated from the totals.
 
 P ( Y )= y+/t+ 
 
 P ( N )= n   +   t   + =( t   +   −y   + )/ t   + 
   Where y +  is the sum of Yes (Analysis) records and t +  is the sum of total (Base) records.   P(Y|A i ) and P(N|A i ) are estimated from the ith row counts.
 
 P ( Y|A )= y   i   /t   i  
 
 P ( N|A   i )= n   i   /t   i =( t   i   −y   i )/ t   i  
   Where y i  is the Yes (“Analysis”) count of category i and t i  is the total (“Base”) count of category i.       

   Thus the weight of evidence provided by knowledge that predictor variable A taking value A i  for or against the assignment of the record to class Y is given by:
 
 PWE ( Y/N|A   i )= G ( Y|A   i )− G ( N|A   i )
 
 PWE ( Y/N|A   i )=log 2   {P ( Y|A   i )/ P ( Y )}−log 2   {P ( N|A   i )/ P ( N )}
 
 PWE ( Y/N|A )=log 2   {P ( N ) P ( Y|A   i )/ P ( Y ) P ( N|A   i )}
 
Substituting the probability estimates gives
 
 PWE ( Y/N|A   i )=log 2   {y   i ( t   +   −y   + )/ y   + ( t   i   −y   i )}
 
Example: Income Band 10–15 k in the above profile report:
 
−360  PWE ( Y/N|“ 15–20  k ”)=log 2   {y   i ( t   +   −y   + )/ y   + ( t   i   −y   i )}
 
 PWE ( Y/N|“ 15–20  k ”)=log 2 {4496×(313694−19262)/19262(57668−4496)}
 
 PWE ( Y/N|“ 15–20  k ”)=0.37 ( to  2 dp )
 
   The weight of evidence measure is displayed in the PWE column of the profile report  82 A. 
   The weight of evidence in this case is positive; knowledge that value (Income Band)=“15–20 k” gives positive evidence of magnitude 0.37 that the record should be classified under Y. 
   In general PWE (Y/N|A i ) is positive if the information that value (A)=A i  provides positive support to predicting a Y classification (Responder). PWE (Y/N|A i ) is negative if the information that value (A)=A i  provides evidence against predicting a Y classification. 
   The overall predictive classification score (PWE Score) for a record is calculated by summing significant PWE contributions from all included predictor variables:
         PWE Score=Sum ([PWE] v,i ) over all included predictor variables v.   Where [PWE] v,i  is the evidence provided by knowledge of the category i for variable v given by:   [PWE] v,i =PWE for variable v, category i, if |ZdExp i |&gt;Z crit .   [PWE] v,i =0, if |ZdExp i |&lt;=Z crit .   [PWE] v,i =0, if category unknown for variable v       

   ZdExp is a z-score measure of statistical significance relevant to the category results used to calculate the category PWE. 
   Zcrit is the Threshold Zscore Value for significance that is specified by the user. The default value is 3 that corresponds to a confidence level of over 99.5% 
   Notice that PWE contribution is omitted from the sum if the corresponding ZdExp is below the critical value i.e. the category does not give a significant deviation from the expected value. No PWE contribution is a d ded if the category is “unknown”. 
   Notice that the only user inputs to the modelling process are the list of included variables and Z crit  the ZdExp threshold for inclusion. 
   The scoring model is improved by omitting PWE contributions that arise from categories that are not statistically significant. The a d justed residual ZdExp can be used to test the significance of the deviation of the observed frequency from the expected frequency. 
   The method is based on the properties of the standardised residual z ij  for a cross tab given by:
 
 Z   ij =( f   ij   −e   ij )/ sqrt ( e   ij )
         Where f ij  is the observed frequency and the expected frequency is estimated as e ij =f i+ f +j /N where f i+  and f +j  are the cross tab marginal totals for the ith row and the jth column respectively and N is the grand total.       

   Haberman (1973) investigated the sampling properties of z ij . The maximum likelihood estimate of the asymptotic variance, n ij , of z ij  is:
 
 n   ij =(1 −f   i+   /N )(1 −f   +j   /N )
 
   Under independence, the a d justed residual d ij  has an approximately standard normal distribution.
 
 d   ij   =z   ij   /sqrt ( n   ij )
 
   So, d ij  is a special z-score showing the significance of the deviation from the expected value, hence the designation ZdExp. 
   Now taking the more specific case of the Responder (Analysis) column and the i&#39;th category row in the profile report:
 
[ zd exp] i   =z   i   /sqrt ( n   i )
         where   z i =(y i −e i )sqrt(e i ) where e i =t i y + /t +     and n i =(1−t i /t + )(1−y + /t + ).       

   Large positive values of [ZdExp] i  indicate that the observed frequency is significantly greater than the expected value. Large negative values of [ZdExp] i  indicate that the observed frequency is significantly less than the expected value. 
   For a chosen significance level a d  there are three cases depending on the value of [ZdExp] i  relative to the standard normal deviate Za d :
 
[ Zd Exp] i   &gt;+Za   d/2  
 
   A significant proportion of the records classified Y have classification A i . It is more likely that a record with value (A)=A i  will belong to Y. In other words P(Y|A i )&gt;P(Y) with (1−a d )×100% confidence. 
   For example [ZdExp] 3 =18.34 so P(Y|″ 15–20 k″)&gt;P(Y) 
   i.e. “the probability of Y given “15–20 k” Income Band is significantly greater than the overall probability of Y.”
 
[ Zd Exp] i   &lt;−Za   d/2  
 
   A significant proportion of the records classified B j  have values of A other than A i . It is less likely that a record with value (A)=A i  will belong to Y. In other words P(Y|A i )&lt;P(Y) with (1−a d )×100% confidence. 
   For example [ZdExp] 5 =−4.60 so P(Y|″30–40 k″)&lt;P(Y)
 
− Za   d/2   &lt;[Zd Exp] i   &lt;+Za   d/2  
 
   The a d justed residual [ZdExp] i  is not significant. Knowledge that a record is characterized by A i  does not provide significant information about the classification to Y or N. 
   For example [ZdExp] 4 =1.68 so we conclude that P(Y|″20–30 k″)&gt;P(Y) but the result is not significant at a=0.05 (since z&lt;1.96) 
   The criteria used for selecting category rows that provide significant information for classification of the target variable is therefore:
 
|[ Zd Exp] i   |&gt;Z   crit  
         where Z crit  is a parameter expressing the significance required in order to include a category contribution into the model. For example with significance a d =0.05 the critical value of Z crit  is 1.96.       

   The PWE score is calculated and banded and stored for each record in the prospect database using a single pass through and prospect database. This is achieved, for every record, by looking up the category for each included predictor variable, rea d ing the corresponding PWE value and aggregating to give the PWE score for the record. 
   Referring to  FIG. 9  there is shown the prospect database  40  with a d ded PWE score  98  column. The result of running the PWE model  96  is a score for every prospect record in the prospect database  40  calculated according to each individual&#39;s characteristics. The PWE score is initially calculated as a real number (e.g. 5.678) but is then also automatically banded (e.g. 5.0 to 6.0) to provide a segmentation of the data set by propensity to be classified as a Responder. The PWE score band is then appended to the prospect database  40  as a new data field  50  in the analysis data store or a related database. The whole process of scoring, banding, appending occurs automatically and rapidly controlled from the software application  22  on the clients PC  20  using the analysis server  30  without user intervention. The size of the bands, however, can be a d justed by the user so, for example, the score of 5.678 could be banded as 5.0 to 5.5 or 4.0 to 6.0. 
   Bands with high positive scores are usually the best segments for marketing, as they include prospect records with the highest propensity to be a responder. Bands with high negative scores are usually the worst segments for marketing as they include prospect records with the lowest propensity to be responders  72 . 
   The best prospects with the highest scores can be selected by simply selecting the top ranking bands  114 —in  FIG. 10  the bands for scores greater than 2.0 have been selected. There are more sophisticated methods of choosing the optimum number of bands using the model reports  100 . 
   In some cases the original responders  72  will be excluded from the final selection. This would be the case if the responders  72  were alrea d y customers for the product to be marketed. Responders  72  can be easily excluded by referencing the URN  44  file or other relevant marker as an exclusion criterion in the application  22  selection. 
     FIG. 11  shows a block diagram  116  of the process of selecting by score segment. After the selected prospects  118  are chosen an export function  120  of the software application  22  is then used to extract the selected prospect records as a file (in a variety of standard formats). The export file can be compressed and password encrypted and transferred to the client PC  20  file system. 
   If the prospect database  40  is provided by an external data supplier (e.g. Dun &amp; Bra d street, Experian) the prospect records to be extracted may have to be licensed for use. In this case the software application  22  invokes a “buy data” function to process the data order and collect payment against a credit card, credit account or other suitable means. 
   The software application  22  can display model reports  100  [created by the analysis server] using gains tables and charts to explore the validity and potential of using the PWE model  96  for marketing. These model reports  100  can also be used to select the score bands to extract for marketing. 
   In  FIG. 12  there can be seen a response chart  122  that shows how well the model  98  predicts the Analysis set (the original Responders)  70  from the base population. 
   Line  124  shows the effect of marketing using the best scoring segments first. In this case marketing to the top 20% of the total base selection captures around 90% of the responders  72  (labelled % Yes). 
   Typically the user will concentrate marketing efforts on the best 5–30% of the base population where the model curve is steepest. 
   Without the model  96 , just picking prospects at random the plot would follow line  128  A user marketing to 20% of the prospects would expect to get 20% of the responders  70 . 
   Line  130  shows the theoretical best model (hindsight) that would be obtained only by apriori knowledge of the responders. 
   All three lines meet at 0% and 100% since, 0% represents marketing to nobody and consequently getting no responses, while 100% represents marketing to all prospects and consequently getting the total number of responders  70 . 
   By taking into account the costs of marketing and expected conversion rates, that can be entered using the client software application  22 , profit and revenue charts can be calculated to show the possible financial gains. Such a revenue chart  132  is shown in  FIG. 13 . 
   All three lines meet at 0% and 100% since, 0% represents marketing to nobody and consequently getting no responses, while 100% represents marketing to all prospects and consequently getting the total number of responders  70 . 
   By taking into account the costs of marketing and expected conversion rates, that can be entered using the client software application  22 , profit and revenue charts can be calculated to show the possible financial gains. Such a revenue chart  132  is shown in  FIG. 13 . 
   The revenue chart  132  shows the result of marketing to successive segments from best to worst in terms of expected costs and revenues. The estimated fixed and per record costs used to calculate financial projections are user specified for the report. 
   The revenue chart  132  shown in  FIG. 13 , as an example, has the following properties: that as more segments are marketed the cost line  134  increases continuously; the revenue line  136  initially increases but then flattens off after the majority of responders  72  (and therefore revenue) have been captured; the profit line  138  is the difference between the revenue and cost increasing initially with the revenue produced by the best segments but later dragged down dramatically by the increasing costs of marketing to the worst segments resulting in a profit curve with a single maximum value, indicating the optimum number of segments to include; the point at which the revenue and cost lines meet is the maximum revenue that can be achieved without making a loss; and the best segments contain relatively few prospects and therefore although the gain is high they produce little revenue. 
   The black cross hair lines  140  here indicate the selection of segments that optimizes the Profit. A user can alter the selection by clicking on a point to move the cross hair  140  to that segment. A lower or higher than optimum profit may be preferred due to, for example, restrictions on cost or requirements to maximize revenue respectively. 
   In this case the cross hair can be repositioned with a mouse click to change the segment selection to meet a profit or revenue objective. The cross hairs on all the charts are linked. Repositioning the cross hair on the revenue chart  132 , moves the cross hair to the corresponding point on the profit chart shown in  FIG. 16  and response chart  122  (and vice versa). 
   Any selection that is ma d e with the cross hair  140  is highlighted on the cumulative gains table shown in  FIG. 15  and can be automatically translated to a selection on the banded score. 
   The Segment Gains Table shown in  FIG. 14  shows the result of marketing to each segment ranked by score. In the segment gains table: 
   Model Gra d e is the ranking of the segment from best to worst. 
   Yes is the actual number of Yes responses in the segment. This is the number of records in the Analysis Selection with scores in this segment. 
   −360No is the actual number of No responses in the segment. This is the number of records not in the Analysis Selection but with scores in this segment. 
   Total is the total number of prospects in the segment (Yes+No). This is the number of records in the Base Selection with scores in this segment. 
   Yes % is the percentage of Yes responses in the Total number of prospects in the segment. 
   No % is the percentage of No responses in the Total number of prospects in the segment. 
   % Yes is the percentage of Yes responses out of the total number of Yes responses in the Base Selection. 
   % No is the percentage of No responses out of the total number of No responses in the Base Selection. 
   % Total is the percentage of records in this segment compared to the whole Base Selection. This measure shows the relative size of the segment. 
   The Gain indicates the performance of the segment and is the ratio of the Yes % for this segment with the overall Yes %. 
   The Revenue is the number of Yes responses multiplied by the per-record revenue (from Options). It is the total gross revenue from selling to the Yes responders. 
   The Cost is the expense of marketing to the Total number of prospects in this segment using the per-record and fixed costs from the Options page. 
   Profit is the Revenue—Cost and is your estimated projected profit for marketing to this segment. 
   Revenue ROI is the revenue return on investment estimate for marketing to this segment. If the Revenue ROI is 10.0 then you make $10 of gross revenue for every $1 spent on marketing to this segment. 
   Profit ROI is the profit return on investment estimate for marketing to this segment. If the Profit ROI is 5.0 then you make $5 of profit for every $1 spent on marketing to this segment. 
   The Cumulative Gains Table [ FIG. 15 ] shows the cumulative result of marketing to each successive segment in the segment gains table. The columns of the cumulative gains table is the source data that is graphed in the response chart  122 , profit chart, and revenue chart  132 . In the cumulative gains table: 
   Model Gra d e is the ranking of the segment from best to worst. 
   Yes is the actual accumulated number of Yes responses up to and including the segment. 
   No is the actual accumulated number of No responses up to and including the segment. 
   Total is the total accumulated number of prospects up to and including the segment (Yes+No). 
   Yes % is the percentage of Yes responses in the Total number of prospects up to and including the segment. 
   −360No % is the percentage of No responses in the Total number of prospects up to and including the segment. 
   0% Yes is the percentage of Yes responses up to and including the segment out of the total number of Yes responses in the Base Selection. 
   % No is the percentage of No responses up to and including the segment out of the total number of No responses in the Base Selection. 
   % Total is the percentage of records up to and including the segment compared to the whole Base Selection. This measure shows the relative size of the segments selected so far. 
   The Gain indicates the performance of the selection up to and including the segment. It is the ratio of the Yes % up to and including the segment with the overall Yes %. 
   The Revenue is the number of Yes responses multiplied by the per-record revenue (from Options). It is the total gross revenue from selling to the Yes responders up to and including the segment. 
   The Cost is the expense of marketing to the Total number of prospects up to and including the segment using the per-record and fixed costs from the Options page. 
   Profit is the Revenue—Cost and is your estimated projected profit for marketing to all prospects up to and including the segment. 
   Revenue ROI is the revenue return on investment estimate for marketing to all prospects up to and including the segment. If the Revenue ROI is 10.0 then you make $10 of gross revenue for every $1 spent on marketing. 
   Profit ROI is the profit return on investment estimate for marketing all prospects up to and including the segment. If the Profit ROI is 5.0 then you make $5 of profit for every $1 spent on marketing. 
   The revenue chart  132  and profit chart plot the segment number on the horizontal axis. An alternative offered by the client software application  22  is to plot the percentage of the total prospect records on the horizontal axis. 
   A further option for these charts is to limit the extent of the horizontal scale to a percentage of the total prospect records. The default is to display only up to 75% but this can be changed dynamically by changing the “display to” number on the report display. Often the last section of the horizontal scale is not interesting and by excluding it the chart is ma d e more rea d able in the area covering the most profitable segments.