Forecasting model generation for sample biased data set

A method, apparatus, system, and computer program product for creating a forecasting model for payroll records. Payroll records are received for a group of employers. The payroll records comprise granular data parameters about employees of the group of employers. A forecasting model is created that aligns the payroll records to high-level employment data. Creating the forecasting model includes identifying predictor variables from the granular data parameters of the payroll records. Creating the forecasting model includes generating a set of basis functions from the predictor variables. Creating the forecasting model includes combining the set of basis functions to create the forecasting model.

BACKGROUND

The disclosure relates generally to an improved computer system and, more specifically, to a method, apparatus, system, and computer program product for creating forecasting model that take into account sampling bias in a data set.

2. Description of the Related Art

Forecasting is a process of making predictions of the future based on past and present data. Forecasting has many applications in business, and can aid in making decisions such as allocating funds for employee compensation. A forecast that is marginally more accurate can give a business the edge to maximize profitability, while also attracting and retaining talented employees.

In the area of human resource management, an organization may want to forecast employment data to facilitate evaluation and comparison of wage patterns within and between different companies, industry sectors, and geographical regions. Examples of forecasted employment data include average, median, and percentiles of annual base salary, hourly wage rates, etc.

Forecasting is typically performed using aggregated data. However, depending on the sample sources and sample sizes, aggregation raises several potential difficulties. A common disadvantage of aggregated data is a small number of records in a group that can lead to wrong inferences. Furthermore, contextual anomalies can cause data outliers to become normal by adding more dimensions to the data, thereby affecting the reliability of the benchmarks. This can be exacerbated by missing dimension values and client base bias.

SUMMARY

According to one embodiment of the present invention, a method creating a forecasting model from a sampling-biased data set. Payroll records for a group of employers are received by a computer system. The payroll records comprise granular data parameters about employees of the group of employers. A forecasting model is created that aligns the payroll records to high-level employment data. Creating the forecasting model includes: identifying predictor variables from the granular data parameters of the payroll records; generating a set of basis functions from the predictor variables; and combining the set of basis functions to create the forecasting model.

According to another embodiment of the present invention, a forecasting modeling system comprising a computer system that operates to receive payroll records for a group of employers. The payroll records comprise granular data parameters about employees of the group of employers. The computer system creates the forecasting model that aligns the payroll records to high-level employment data. In creating the forecasting model, the computer system further operates to: identify predictor variables from the granular data parameters of the payroll records; generate a set of basis functions from the predictor variables; and combine the set of basis functions to create the forecasting model.

According to yet another embodiment of the present invention, a computer program product for creating a forecasting model from a sampling-biased data set comprises a computer-readable-storage media and program code stored on the computer-readable storage media. The program code includes program code for receiving payroll records for a group of employers. The payroll records comprise granular data parameters about employees of the group of employers. The program code includes program code for creating the forecasting model that aligns the payroll records to high-level employment data. The program code for creating the forecasting model comprises: code for identifying predictor variables from the granular data parameters of the payroll records; code for generating a set of basis functions from the predictor variables; and code for combining the set of basis functions to create the forecasting model.

DETAILED DESCRIPTION

The illustrative embodiments recognize and take into account one or more different considerations. For example, the illustrative embodiments recognize and take into account that sampling-bias in payroll records in a group can create unreliable inferences when predicting employment data. The illustrative embodiments further recognize and take into account that contextual anomalies in aggregated data can allow data outliers to become normal by the addition of dimensions. Ignoring the sampling-bias in payroll records when creating forecasting model can result in forecasting model that are less accurate than desired.

Therefore, it would be desirable to have a method, apparatus, system, and computer program product that take into account at least some of the issues discussed above, as well as other possible issues. For example, it would be desirable to have a method and apparatus that overcome an issue with creating a forecasting model from a sampling-biased data set to accurately predict employment data at a higher level of granularity.

Thus, illustrative embodiments provide a method, apparatus, system, and computer program product that create a forecasting model from a sampling-biased data set. In one illustrative example, payroll records are received for a group of employers. The payroll records comprise granular data parameters about employees of the group of employers. A forecasting model is created the that aligns the payroll records to high-level employment data. Creating the forecasting model includes identifying predictor variables from the granular data parameters of the payroll records. Creating the forecasting model includes generating a set of basis functions from the predictor variables. Creating the forecasting model includes combining the set of basis functions to create the forecasting model.

According to an illustrative example, the forecasting model is used to predict employment data. Using the forecasting model enables predictions at a higher level of granularity than reported in the high-level employment data.

According to an illustrative example, payroll records are classified into of a plurality of cells at an industry level and a size level. A time series is generated time series from the payroll records for each of these cells. Each time series is then seasonally adjusted, and outliers are removed.

According to an illustrative example, the time series within each of the plurality of cells is adjusted to match the distributions of industry and employer size reported in the high-level employment data. high-level employment data is interpolated to determine weight values at the industry level. The weight values are extrapolated at the size level based on the time series of a corresponding one of the plurality of cells. A weighted average of the seasonally-adjusted time series is computed within each industry based on the extrapolated weight values. Th cells are aggregated, and the forecasting model is generated therefrom.

With reference now to the figures and, in particular, with reference toFIG.1, a block diagram of a forecasting environment is depicted in accordance with an illustrative embodiment. In this illustrative example, forecasting environment100is an environment in which forecasting model102can be used to make predictions such as employment forecasting. In this illustrative example, modeling system104operates to create forecasting model102.

As depicted, modeling system104comprises computer system106and model manager108. Model manager108is located in computer system106.

As used herein, a “set of,” a “group of,” or a “number of,” when used with reference to items, means one or more items. For example, a “set of items” is one or more items. Likewise, a “group of items” or a “number of items” is one or more items. For example, “a number of different types of networks” is one or more different types of networks.

In the illustrative example, model manager108in computer system106operates as a forecasting model system to create forecasting model102from payroll records110. Forecasting model102can be used to predict granular employment data112.

As depicted, model manager108can operate to receive payroll records110for a group of employers114. In the illustrative example, payroll records110comprise granular data parameters116about employees118of the group of employers114.

Payroll records110are data generated in performing payroll operations for employers114. payroll records110may include various data parameters about employees118. Data records that are specific to individual employees are more highly granular than generalized observations about a company or industry.

As used herein, “granular” data is detailed data, or the lowest level that data can be in a target set. It refers to the size that data fields are divided into, in short how detail-oriented a single field is. As the data becomes more subdivided and specific, it is also considered more granular.

In the illustrative example, model manager108can create forecasting model102from payroll records110. Using payroll records110to create forecasting model102enables predictions of employment data at a higher level of granularity.

In the illustrative example, forecasting model102comprises a regression analysis122that employs multivariate adaptive regression splines124. Multivariate adaptive regression splines125is a non-parametric regression technique that models nonlinearities and interactions between variables.

When model manager108uses multivariate adaptive regression splines124to create forecasting model102, forecasting model102takes the form of:

Each basis function Bi(x) is an element of a particular basis for the function space k. Each basis function Bi(x) is either a constant, a hinge function, or a product of two or more hinge functions.

A hinge function takes the form:
max(0,x−c)
or
max(0,c−x)

Hinge functions partition the data into disjoint regions, each of which can be treated independently. Two or more hinge functions can be multiplied together to form a basis function that is non-linear.

Starting with just the intercept term, i.e., the mean of the response values, basis functions are repeatedly added in mirrored-pairs to the model. At each step, the mirrored-pair of basis functions is selected to produce a maximum reduction in sum-of-squares residual error. Model manager108continues adding terms until the change in residual error is too small to continue or until a maximum number of terms is reached.

Individual basis functions are then removed one-by-one from the model, until an optimal submodel is determined. the performance of model subsets, i.e., a model that excludes a basis function, are compared using generalized cross validation (GCV) to in order to identify which basis function should be removed. Generalized cross-validation uses a formula to approximate the error that would be determined by leave-one-out validation. the formula penalizes model complexity to avoid model overfit.

In this illustrative example, creating forecasting model102includes identifying predictor variables126from the granular data parameters116of the payroll records110.

Predictor variables126are independent variables (x) used by model manager108to create forecasting model102. Each of predictor variables126corresponds to one of granular data parameters116. To create forecasting model102, model manager108identifies predictor variables126that best predict high-level employment data120. In other words, instead of determining f(x) from payroll records110, model manager108sets f(x) based on high-level employment data120, and then identifies ones of granular data parameters116that that best predict high-level employment data120.

High-level employment data120comprise statistics about employment. high-level employment data120is reported at a lower level of granularity than payroll records110. For example, high-level employment data120can include statistics about employment reported at an industry level.

High-level employment data120can be generated by an entity different than an entity that provides payroll records110. In one specific example, high-level employment data120comprises the Current Employment Statistics produced by the Bureau of Labor and Statistics (BLS).

The Bureau of Labor and Statistics (BLS) Current Employment Statistics (CES) program, also known as the payroll survey or the establishment survey, is a monthly survey of approximately 145,000 businesses and government agencies representing approximately 697,000 worksites throughout the United States. From the sample, CES produces and publishes employment, hours, and earnings estimates for the nation, states, and metropolitan areas at detailed industry levels.

In the illustrative example, creating the forecasting model102comprises a regression analysis using multivariate adaptive regression splines. Creating forecasting model102includes generating a set of basis functions128from the predictor variables126. creating forecasting model102includes combining the set of basis functions128to create forecasting model102. In the illustrative example, the identifying, generating, and combining steps are performed using multivariate adaptive regression splines.

Using the forecasting model102generated from payroll records110enables model manager108to predict employment data at a higher level of granularity than reported in the high-level employment data120. In this manner, model manager108in computer system106provides a practical application for creating forecasting model102such that the functioning of computer system106is improved in forecasting granular employment data112.

In one illustrative example, one or more solutions are present that overcome an issue with creating forecasting model102that predict employment data at a higher level of granularity. In other words, these forecasting model102include aligning the payroll records to high-level employment data. As a result, one or more solutions may provide an effect of enabling the creation of forecasting model102that have increased accuracy in predicting employment data at a higher level of granularity as compared to current techniques for creating forecasting models to forecast employment data.

Computer system106can be configured to perform at least one of the steps, operations, or actions described in the different illustrative examples using software, hardware, firmware, or a combination thereof. As a result, computer system106operates as a special purpose computer system in which model manager108in computer system106enables creating forecasting model102that predict employment data at a higher level of granularity as compared to currently used techniques. In particular, model manager108transforms computer system106into a special purpose computer system as compared to currently available general computer systems that do not have model manager108.

In the illustrative example, the use of model manager108in computer system106integrates processes into a practical application for creating forecasting model102that increases the performance of computer system106. In other words, model manager108in computer system106is directed to a practical application of processes integrated into model manager108in computer system106that creates forecasting model102from payroll records110that predicts employment data at a higher level of granularity. In other words, computer system106or some other computer system has increased accuracy in predicting employment data at a higher level of granularity when using forecasting model102as created by model manager108.

In this illustrative example, model manager108in computer system106enables creating forecasting model102with increased accuracy as compared to forecasting models created by currently used techniques. In this illustrative example, model manager108receives payroll records for a group of employers. The payroll records comprise granular data parameters about employees of the group of employers. In this illustrative example, model manager108creates forecasting model102that aligns the payroll records to high-level employment data. Creating the forecasting model102includes identifying predictor variables from the granular data parameters of the payroll records. Creating the forecasting model102also includes generating a set of basis functions from the predictor variables. Additionally, creating the forecasting model102includes combining the set of basis functions to create the forecasting model102.

Turning now toFIG.2, a block diagram illustrating aligning the payroll records to high-level employment data is depicted in accordance with an illustrative embodiment. The data flow illustrated in this figure can be used to create a forecasting model that aligns the payroll records to high-level employment data inFIG.1.

As depicted, model manager108can operate to classifying the payroll records110into of a plurality of cells202at an industry level204and a size level206. model manager108then aligns the plurality of cells202to distributions of industry210and employer size212reported in the high-level employment data120. By breaking data into different categories, model manager108can provides a more compelling, in depth look at employment at a higher level of granularity.

In one illustrative example, model manager108can operate to classifying payroll records110at an industry level204based on the North American Industrial Classification system (NAICS)

In one illustrative example, model manager108assigns each payroll records110to cells202at industry level204according to the North American Industrial Classification System used by the BLS. Alternatively, model manager108assigns each payroll records110to cells202at industry level204according using the Standard Industrial Classification, or use a NAICS-SIC mapping to reclassify those payroll records110.

Model manager108can operate to classifying payroll records110at a size level206based on employer size212. The companies commonly classified at industry level204are then sub-grouped into cells202according to their employer size212. Cells202at size level206can include one or more different cells based on the number of employees employed by the Employer. For example, size level206can include cells202for an employer size of 1-19 employees, 20-49 employees, 50-499 employees, 500-999 employees, and 1,000 or more employees, well as other suitable sizes.

In one illustrative example, payroll records110are aggregated up to the “parent” company based on a common Federal Employer Identification Number (EIN). Such aggregates are then classified by the size of parent company rather than the size of the individual establishment. For example, the aggregation of all of a company's different locations may now appear in a larger business category, rather than as many “small” or “medium” businesses.

If a company includes divisions or subsidiaries from more than one industry, each unit is allocated to the appropriate industry at industry level204. However, the company size to which the unit is allocated is the company size of the combined control keys. For example, if a company is made up of two units, each with 15 employees in two industries, the company is allocated to company size 20-49, but the number of employees allocated in this company size to each industry is still15.

In one illustrative example, model manager108can generate a time series208from the payroll records110within each of the plurality of cells202. time series208can be an employment growth for each of the plurality of cells202.

In this illustrative example, model manager108can creates matched pairs of establishments that have reported payroll records110in two consecutive months. Each month's data include only the matched pairs available in that month. Matched pairs are aggregated and matched-pair growth rates of employment are computed into the different cells202.

The CES measures the number of people on payrolls during the pay period that includes the 12th of the month (the reference period). A pay period can be any length of time; the most common pay frequency is weekly. But a pay period can also cover two weeks; it can be bimonthly, monthly, etc. If payroll records110provide pay dates rather than pay periods, matched pairs are constructed using interpolation.

If there is no recorded employment for the pay period that includes the 12th, but a record exists for either a later or earlier pay period during the month, Model manager108can estimate employment for the reference period by linearly interpolating between the level of employment on the prior record and the record for the later pay period. The maximum time range for linear interpolation to capture missing employment on the 12th can depend on the payment frequency of an establishment.

After calculating time series208at each industry level204by size level206, model manager108can operate to seasonally adjusting the time series208. In one illustrative example, model manager108applies an autoregressive integrated moving average (ARIMA) and seasonal adjustment decomposition (SEATS) to seasonally adjusting the time series208. Deseasonalized trends can then be recalculated at industry level204with each new month of data.

In this illustrative example, model manager108can removing outliers216from the seasonally-adjusted time series214. For example, for each payroll records110commonly classified at industry level204, model manager108compares the record with the trend value. Model manager108identifies and removes outliers from payroll records110using the same autoregressive integrated moving average (ARIMA) and seasonal adjustment decomposition (SEATS). Model manager108can then recalculate time series208at size level206using the cleaned data.

As depicted, model manager108can operate to adjusting the time series within each of the plurality of cells to match the distributions of industry and employer size reported in the high-level employment data. For example, time series208at industry level204can be are computed by taking a weighted average within each industry level204of time series208at size level206.

In this illustrative example, adjusting the time series within each of the plurality of cells includes interpolating the high-level employment data to determine weight values218at the industry level. The weight values are extrapolated at the size level based on the time series of a corresponding one of the plurality of cells. a weighted average220of the seasonally-adjusted time series is then computed within each industry level204based on the extrapolated weight values.

This method allows different trends of employment by size of payroll within industries, while assuming that the other industry-wide relationships implied by the regressions hold for all size classes within an industry. The monthly industry distribution of company cells and the forecasted industry employment growth are then combined to produce the forecasted employment growth for company cells in the NER

The illustration of forecasting environment100inFIG.1is not meant to imply physical or architectural limitations to the manner in which an illustrative embodiment can be implemented. Other components in addition to or in place of the ones illustrated may be used. Some components may be unnecessary. Also, the blocks are presented to illustrate some functional components. One or more of these blocks may be combined, divided, or combined and divided into different blocks when implemented in an illustrative embodiment. For example, forecasting model102can operate to create one or more forecasting model in addition to or in place of forecasting model102.

Turning next toFIG.3, a flowchart of a process for creating a forecasting model from a sampling-biased data set is depicted in accordance with an illustrative embodiment. The process inFIG.3can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program code that is run by one or more processor units located in one or more hardware devices in one or more computer systems. For example, the process can be implemented in model manager108in computer system106inFIG.1.

The process begins by receiving payroll records for a group of employers (step310). The payroll records comprise granular data parameters about employees of the group of employers.

The process creates the forecasting model that aligns the payroll records to high-level employment data, (step320). As depicted, creating the forecasting model comprises identifying predictor variables from the granular data parameters of the payroll records (step330), generating a set of basis functions from the predictor variables (step340), and combining the set of basis functions to create the forecasting model (step350). The process terminates thereafter.

With reference next toFIG.4, a flowchart of a process for predicting employment data is depicted in accordance with an illustrative embodiment. The process inFIG.4is an example of one manner in which the process ofFIG.3can be implemented.

Continuing from step320ofFIG.3, the process using the forecasting model to predict employment data (step410). The process terminates thereafter. Using the forecasting model generated by the process ofFIG.3enables the prediction of employment data at a higher level of granularity.

Turning now toFIG.5, a flowchart of a process for forecasting model that aligns the payroll records to high-level employment data is depicted in accordance with an illustrative embodiment. The flowchart inFIG.5illustrate additional processing steps that can be used to create a forecasting model that aligns the payroll records to high-level employment data.

Continuing from step310ofFIG.3, the process classifies the payroll records into of a plurality of cells at an industry level and a size level (step510). The process generates a time series from the payroll records within each of the plurality of cells (step520). In this manner, the forecasting model aligns the plurality of cells to distributions of industry and employer size reported in the high-level employment data.

The process seasonally adjusts the time series (step530). The process removes outliers from the seasonally-adjusted time series (step540). The seasonal adjustment and outlier removal can be are performed using an autoregressive integrated moving average (ARIMA) and seasonal adjustment decomposition (SEATS) of the time series.

The process adjusts the time series within each of the plurality of cells to match the distributions of industry an employer size reported in the high-level employment data (step550). According to an illustrative example, adjusting the time series comprises interpolating the high level employment data to determine weight values at the industry level (step560), extrapolating the weight values at the size level based on the time series of a corresponding one of the plurality of cells (step570), and computing a weighted average of the seasonally adjusted time series within each industry based on the extrapolated weight values (step580). Thereafter, the process continues to step320ofFIG.3, where the forecasting model is built from the cleaned time series.

Turning now toFIG.6, a block diagram of a data processing system is depicted in accordance with an illustrative embodiment. Data processing system600can be used to implement one or more data processing systems in computer system106. In this illustrative example, data processing system600includes communications framework602, which provides communications between processor unit604, memory606, persistent storage608, communications unit610, input/output (I/O) unit612, and display614. In this example, communications framework602takes the form of a bus system.

Processor unit604serves to execute instructions for software that can be loaded into memory606. Processor unit604includes one or more processors. For example, processor unit604can be selected from at least one of a multicore processor, a central processing unit (CPU), a graphics processing unit (GPU), a physics processing unit (PPU), a digital signal processor (DSP), a network processor, or some other suitable type of processor. Further, processor unit604can may be implemented using one or more heterogeneous processor systems in which a main processor is present with secondary processors on a single chip. As another illustrative example, processor unit604can be a symmetric multi-processor system containing multiple processors of the same type on a single chip.

Memory606and persistent storage608are examples of storage devices616. A storage device is any piece of hardware that is capable of storing information, such as, for example, without limitation, at least one of data, program code in functional form, or other suitable information either on a temporary basis, a permanent basis, or both on a temporary basis and a permanent basis. Storage devices616may also be referred to as computer-readable storage devices in these illustrative examples. Memory606, in these examples, can be, for example, a random-access memory or any other suitable volatile or non-volatile storage device. Persistent storage608may take various forms, depending on the particular implementation.

For example, persistent storage608may contain one or more components or devices. For example, persistent storage608can be a hard drive, a solid-state drive (SSD), a flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination of the above. The media used by persistent storage608also can be removable. For example, a removable hard drive can be used for persistent storage608.

Communications unit610, in these illustrative examples, provides for communications with other data processing systems or devices. In these illustrative examples, communications unit610is a network interface card.

Input/output unit612allows for input and output of data with other devices that can be connected to data processing system600. For example, input/output unit612may provide a connection for user input through at least one of a keyboard, a mouse, or some other suitable input device. Further, input/output unit612may send output to a printer. Display614provides a mechanism to display information to a user.

Instructions for at least one of the operating system, applications, or programs can be located in storage devices616, which are in communication with processor unit604through communications framework602. The processes of the different embodiments can be performed by processor unit604using computer-implemented instructions, which may be located in a memory, such as memory606.

These instructions are referred to as program code, computer usable program code, or computer-readable program code that can be read and executed by a processor in processor unit604. The program code in the different embodiments can be embodied on different physical or computer-readable storage media, such as memory606or persistent storage608.

Program code618is located in a functional form on computer-readable media620that is selectively removable and can be loaded onto or transferred to data processing system600for execution by processor unit604. Program code618and computer-readable media620form computer program granular employment data622in these illustrative examples. In the illustrative example, computer-readable media620is computer-readable storage media624.

In these illustrative examples, computer-readable storage media624is a physical or tangible storage device used to store program code618rather than a medium that propagates or transmits program code618.

Alternatively, program code618can be transferred to data processing system600using a computer-readable signal media. The computer-readable signal media can be, for example, a propagated data signal containing program code618. For example, the computer-readable signal media can be at least one of an electromagnetic signal, an optical signal, or any other suitable type of signal. These signals can be transmitted over connections, such as wireless connections, optical fiber cable, coaxial cable, a wire, or any other suitable type of connection.

Further, as used herein, “computer-readable media620” can be singular or plural. For example, program code618can be located in computer-readable media620in the form of a single storage device or system. In another example, program code618can be located in computer-readable media620that is distributed in multiple data processing systems. In other words, some instructions in program code618can be located in one data processing system while other instructions in in program code618can be located in one data processing system. For example, a portion of program code618can be located in computer-readable media620in a server computer while another portion of program code618can be located in computer-readable media620located in a set of client computers.

The different components illustrated for data processing system600are not meant to provide architectural limitations to the manner in which different embodiments can be implemented. In some illustrative examples, one or more of the components may be incorporated in or otherwise form a portion of, another component. For example, memory606, or portions thereof, may be incorporated in processor unit604in some illustrative examples. The different illustrative embodiments can be implemented in a data processing system including components in addition to or in place of those illustrated for data processing system600. Other components shown inFIG.6can be varied from the illustrative examples shown. The different embodiments can be implemented using any hardware device or system capable of running program code618.

Thus, illustrative embodiments provide a method, apparatus, system, and computer program product that create a forecasting model from a sampling-biased data set. In one illustrative example, payroll records are received for a group of employers. The payroll records comprise granular data parameters about employees of the group of employers. A forecasting model is created the that aligns the payroll records to high-level employment data. Creating the forecasting model includes identifying predictor variables from the granular data parameters of the payroll records. Creating the forecasting model includes generating a set of basis functions from the predictor variables. Creating the forecasting model includes combining the set of basis functions to create the forecasting model.

One or more illustrative examples overcome a problem with creating forecasting model that predict employment data at a higher level of granularity. In other words, these forecasting model include aligning the payroll records to high-level employment data. As a result, one or more of the illustrative examples may enable the creation of forecasting model that have increased accuracy in predicting employment data at a higher level of granularity as compared to current techniques for creating forecasting model to forecast employment data.