INCREMENTAL UPDATES OF CONFLATED DATA RECORDS

According to examples, an apparatus may include a processor and a memory on which are stored machine-readable instructions that when executed by the processor, may cause the processor to receive an updated data record from a data source and may determine a first conflated data record. The first conflated data record may be associated with the updated data record and include data records in a first grouping from among a plurality of data sources. The processor may identify the data records included in the first conflated data record and may generate a second conflated data record that updates conflations among the updated data record and the identified data records. The second conflated data record may include a second grouping of data records. The processor may replace the first conflated data record with the second conflated data record to incrementally update a set of conflated data records.

BACKGROUND

Datasets that include data records from multiple data sources may be maintained on computing devices. The computing devices may update the data records based on updates received from the data sources.

DETAILED DESCRIPTION

For simplicity and illustrative purposes, the principles of the present disclosure are described by referring mainly to embodiments and examples thereof. In the following description, numerous specific details are set forth in order to provide an understanding of the embodiments and examples. It will be apparent, however, to one of ordinary skill in the art, that the embodiments and examples may be practiced without limitation to these specific details. In some instances, well known methods and/or structures have not been described in detail so as not to unnecessarily obscure the description of the embodiments and examples. Furthermore, the embodiments and examples may be used together in various combinations.

A computing device may maintain and store data records received from a data source or from multiple data sources as a set of conflated data records. A conflated data record may be a data record in which a plurality of data records, which may be from multiple data sources, may be conflated, e.g., grouped together, unified, and/or the like into a single dataset. In addition, the set of conflated data records may be a conflation, unification, or grouping of data from multiple data sources to form the single dataset. Conflation may be defined as a mechanism that may relate or link the data records from the multiple data sources to build relationships between the data records. Data records having a similar feature, such as a common entity name, a common entity type, and/or the like, may be conflated into a single conflated data record. For instance, a data record directed to a user's name may be conflated with a data record directed to the user's spouse, the user's address, the user's email address, and/or the like, as these data records may each be directed to the same user (entity). In the present disclosure, the set of conflated data records may also be referred to as a set of grouped data records.

The computing device may receive an updated data record from a data source among a plurality of data sources and this updated data record may be incorporated in a corresponding conflated data record in the set of conflated data records. In some instances, the incorporation of the updated data record may affect one or more other data records in the set of conflated data records. By way of particular example in which the updated data record is a name of an individual, a first conflated data record may include conflated data records that include an individual's name that is the same or similar to the name as it appears in the updated data record. In some examples, the first conflated data record may include those conflated data records whose source primary keys match the ones that may be included in the updated data record.

A concern associated with maintaining a set of conflated data records may be that an update to a data record in a conflated data record may change conflations (e.g., correlations, groupings, links, etc.) among the updated data record and other data records included in the conflated data record. Such data records may be referred to as being “dirty”, and in some instances, not all of the “dirty” data records may be processed. In this regard, “clean” data records may refer to remaining data records in the set of conflated data records, other than the “dirty” data records, that may not need to be re-processed. In some instances, it may not be possible to accurately update conflated data records directly by undoing a previous conflation on “dirty” data records and redoing a conflation with updated data records. For instance, while applying updates to particular data records in the set of conflated data records may be possible, in many of these instances, the updates may not be applied to all associated data records in the set of conflated data records and, in some instances, may require recursively regrouping the data records due to the changes and how conflation of data records may be defined, which may result in inaccuracies in the updated set of conflated data records. In these instances, the resulting set of conflated data records may not be equivalent to a set of conflated data records resulting from a full refresh of the set of conflated data records.

In some instances, a full refresh may be performed to ensure accuracy of the set of conflated data records. However, a full refresh may be time and/or computing resource intensive, particularly in instances in which the set of conflated data records includes a large volume of data. In some instances, data sources may have a relatively small number of updates in a given period of time when compared to the size of the entire dataset. In these instances, it may be relatively inefficient to perform a full refresh of the set of conflated data records, which may involve a relatively large amount of time and/or computing resources.

Disclosed herein are apparatuses, systems, methods, and computer-readable media that may enable efficient updates to the set of conflated data records by, for instance, enabling incremental updates to the set of conflated data records while accurately conflating the data records in the set of conflated data records. As discussed herein, a processor may receive an updated data record from a data source, in which the updated data record may be associated with a first conflated data record among the set of conflated data records. The first conflated data record may be a conflation or unification of data records in a first grouping from a plurality of data sources. The processor may determine the first conflated data record among the set of conflated data records. As also discussed herein, incorporation of the updated data record may change conflations among the updated data record and the other data records in the first conflated data record, which may cause the first conflated data record to become “dirty”.

In some examples, the processor may generate an updated conflated data record to replace the “dirty” conflated data record. The processor may identify, from information corresponding to the plurality of data sources, the data records included in the first conflated data record. The processor may generate a second conflated data record that updates the conflations among the updated data record and the identified data records. The second conflated data record may have a second grouping of data records. For instance, the second conflated data record may have a different or updated grouping of data records from the plurality of data sources, may include different data records than those included in the first conflated data record, and/or the like. The processor may replace the first conflated data record with the second conflated data record in the set of conflated data records to incrementally update the set of conflated data records.

Through implementation of the features of the present disclosure, a processor may enable improved updates to the conflated dataset, which may reduce latency and consumption of processing resources, for instance, by use of incremental updates rather than a full refresh of the conflated dataset. In some examples, the improved updates may increase the accuracy and the consistency associated with incremental processing of updates by leveraging other data records that may be relevant to or associated with the updated data record and re-processing conflated data records that include these relevant or associated data records together with the current update.

Reference is made toFIGS. 1, 2, and 3.FIG. 1shows a block diagram of an apparatus100that may receive an updated data record, determine a first conflated data record among a set of conflated data records, generate a second conflated data record that updates conflations of data records, and replace the first conflated data record with the second conflated data record to incrementally update the set of conflated data records, in accordance with an embodiment of the present disclosure.FIG. 2shows a block diagram of an example system200that may include the apparatus100depicted inFIG. 1, in accordance with an embodiment of the present disclosure.FIG. 3shows a block diagram of example data records300, which may be implemented in the system200depicted inFIG. 2, including an updated data record, a set of conflated data records including a first conflated data record, and a second conflated data record to incrementally update the set of conflated data records, in accordance with an embodiment of the present disclosure. It should be understood that the apparatus100depicted inFIG. 1, the system200depicted inFIG. 2, and/or the data records300depicted inFIG. 3may include additional features and that some of the features described herein may be removed and/or modified without departing from the scopes of the apparatus100, the system200, and/or the data records300.

The apparatus100may include a processor102and a memory110. The apparatus100may be a computing device, including a server, a node in a network (such as a data center or a cloud computing resource), a desktop computer, a laptop computer, a tablet computer, a smartphone, an electronic device such as Internet of Things (IoT) device, and/or the like. The processor102may include a semiconductor-based microprocessor, a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), and/or other hardware device. In some examples, the apparatus100may include multiple processors and/or cores without departing from a scope of the apparatus. In this regard, references to a single processor as well as to a single memory may be understood to additionally or alternatively pertain to multiple processors and multiple memories.

The memory110may be an electronic, magnetic, optical, or other physical storage device that contains or stores executable instructions. The memory110may be, for example, Read Only Memory (ROM), flash memory, solid state drive, Random Access memory (RAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a storage device, an optical disc, or the like. The memory110may be a non-transitory computer-readable medium. The term “non-transitory” does not encompass transitory propagating signals.

As shown inFIG. 1, the processor102may execute instructions112-120to update a set of conflated data records based on an updated data record received from a data source. The instructions112-120may be machine-readable instructions, e.g., non-transitory computer-readable instructions. In other examples, the apparatus100may include hardware logic blocks or a combination of instructions and hardware logic blocks to implement or execute functions corresponding to the instructions112-120.

The apparatus100may be connected via a network202, which may be the Internet, a local area network, and/or the like, to a server204. In addition, a data store206may be connected to the server204. In some examples, the server204may maintain a data source (not shown), such as a cloud-based data warehouse, data center, and/or the like. In addition, the apparatus100may maintain a table of data records that may be a conflation of the data from multiple data sources.

The processor102may fetch, decode, and execute the instructions112to receive an update208including an updated data record210from a data source (not shown). The update208may include instructions to insert a new data record, modify an existing data record, or delete a data record. The update208may be an update for a predetermined time period, a periodic update, or the like.

The processor102may generate and store snapshots214ato214nof the data sources, which may be a record of a state of the data source at a particular point in time. The snapshots214ato214nof the data sources may include a plurality of snapshots214ato214ncorresponding to the plurality of data sources. The snapshots214ato214nmay be full snapshots of the data sources, which may be formed using incremental updates received from the data sources over time. The processor102may append incremental updates from the data sources to the snapshots214ato214nto maintain up-to-date copies of the data sources at the apparatus100. In some examples, the update208may be an incremental update that is received from a data source after a corresponding snapshot214aof the data source is generated.

The processor102may maintain both the snapshots214ato214nof the data sources and previous updates from the data sources and may apply the updates to the set of conflated data records212. The processor102may store the snapshots214ato214nand the previous updates in a predetermined folder structure at the apparatus100.

The processor102may track different updates via timestamps. For instance, the processor102may maintain a timestamp for the update208, which may track the time period associated with the current update, such as at a time t. The processors102may also maintain time stamps for the snapshots214ato214n, in which the time stamps may identify when the snapshots214ato214nwere generated, for instance, a period up to the current update, such as at time t−1. Based on the timestamps, the processor102may track and append each incremental update to the snapshots214ato214nto keep the snapshots214ato214nup to date.

In some examples, a data source that is maintained at the server204may be one among a plurality of data sources. The processor102may conflate, e.g., bring together, group, combine, unify, or the like, the data from the multiple data sources into a set of conflated data records, for instance the set of conflated data records212, via updates from each of the multiple data sources.

By way of particular example and for purposes of illustration, the processor102may use a folder structure400as depicted inFIG. 4to maintain both full snapshots of the data sources and incremental updates to implement incremental ingestion of updates. In this example, the {yyyy}-{MM}-{dd}-{hh} may indicate all partition files within that folder having an incremental time stamp (ITS)>={yyyy}-{MM}-{dd}-{hh}, where {yyyy}, {MM}, {dd}, and {hh} represent year, month, day, and hour, respectively. The processor102may maintain folders to the {hh} level, for instance, for cases in which the highest frequency for incremental refresh is hourly.

In some examples, the incremental partition files may have a LastUpdateTimestamp (ETS) column to indicate when an update, insert, or delete operation was performed. In some examples, in the Delete partition folder, the processor102may set the files to have the same schema as the files in the {hh} partition folder. By way of particular example, for manifest.json, the partition section may point to just the {hh} partition folder with filenames. The processor102may consider these to be update and insert (upserts) partitions. In some instances, if there exists a Delete folder in {hh}, the processor102may consider all files in the Delete folder as delete actions.

The processor102may fetch, decode, and execute the instructions114to determine a conflated data record216among the set of conflated data records212that may be associated with the updated data record210. As shown inFIG. 3, the conflated data record216may include multiple conflated data records216a,216b, and216cas described in further detail herein. In some examples, the processor102may use the updated data record210to update, insert, or delete a first data record210ain the first conflated data record216a. The first data record210amay be a data record in the set of conflated data records212that matches the updated data record210. In some examples, the processor102may filter the set of conflated data records212to find the first conflated data record216athat includes the first data record210athat matches the updated data record210, as depicted by arrow302inFIG. 3.

In some examples, the processor102may also determine data records among the set of conflated data records212that may be associated with, and thus may be affected by the updated data record210, such as an associated conflated data record216b. By way of particular example in which the update208is an update to change an individual's name from “Jonathan Smith” to “Jon Smith”, the first conflated data record216amay include the first data record210ahaving the updated individual's name “Jon Smith”. In some examples, the processor102may determine other data records, such as the associated conflated data record216bthat may include a second data record210bhaving a variation of the individual's name, such as “Jonathan S.”, “Jon S.”, “Jonathan M. Smith”, and/or the like.

In some examples, the processor102may apply a filter to the set of conflated data records212to determine the associated conflated data record216bthat may include the second data record210bassociated with the first data record210a. For instance, the processor102may apply a user-defined rule to control how the filter may identify data records that are considered to be associated with the updated data record210. In some examples, the processor102may display a user interface (UI) for setting a variety of rules. A rule for the filter may be associated with a level of precision of a match between the first data record210aand the second data record210bin the associated conflated data record216b. By way of particular example and for purposes of illustration, the level of precision for a match to the first data record210amay be a direct match, in which the processor102may identify only the first data record210athat may be a direct match with the updated data record210. Alternatively or additionally, the processor102may apply a level of precision for a match across different formats of the updated data record210and the associated data records210bin the associated conflated data record216b, such as different spellings, inclusion of initials, and/or the like. In some examples, the processor102may apply fuzzy matching techniques between the updated data record210and other data records included in the set of conflated data records212to filter the set of conflated data records212.

Continuing with the example in which the update208is to update an individual's name, for a level of precision for an exact match, the processor102may filter the set of conflated data records212to determine only the first conflated data record216a, which may include an exact match of the updated name “Jon Smith”. Alternatively or additionally, for a different level of precision for the filter, the processor102may filter the set of conflated data records212to identify content of data records having different formats or variations of the individual's name as being associated with the updated data record210, such as the data record210bwhich may include “Jonathan S.”, “Jon S.”, “Jonathan M. Smith”, and/or the like. As such, based on the set rule and the associated level of precision of a match with the updated data record210, the fidelity or sensitivity of the filter may be changed, and thus a number of conflated data records identified as being associated with the updated data record210may be varied. It should be understood that, while a particular example for a rule for filtering different formats and variations of an individual's name is described herein for illustration purposes, the processor102may define various types of rules, which may be associated with various types and characteristics of the data in the data records.

The processor102may fetch, decode, and execute the instructions116to identify, from information corresponding to the plurality of data sources, the data records220aincluded in the conflated data record216. In some examples, the updated data record210and the other data records220aincluded in the first conflated data record216amay have a predetermined correlation when the data from the data sources is conflated to form the set of conflated data records212. In some examples, the data records220athat are included in the first conflated data record216amay be data records from a plurality of different data sources, which may be conflated together to form the first conflated data record216a. The correlation between the data records210a,220amay be determined based on a conflation plan and/or the conflation algorithm implemented for conflation. In this regard, once the updated data record210is updated, the correlation between the data records210aand220ain the first conflated data record216amay no longer be valid, and the processor102may identify the first conflated data record216aas being “dirty”. As such, in order to generate an updated conflated data record to replace the “dirty” conflated data record, the processor102may identify each data record220aincluded in the conflated data record216from the data source for re-processing for conflation, as depicted by arrow304inFIG. 3. In this regard, the information corresponding to the plurality of data sources for identifying the source data records may include snapshots214ato214nof previous updates received from the plurality of data sources, which may be stored on the apparatus100. Alternatively or additionally, the information corresponding to the plurality of data sources for identifying the data records220amay be obtained directly from the data sources, such as the server204, which may be hosting the data sources.

In some examples, the processor102may use primary keys (not shown) associated with each data record220ato identify the data records220ain the snapshots214ato214n. In this regard, each data record220a,210aincluded in the first conflated data record216amay include a primary key. The primary key may be unique to each data record and may be used to identify the data record in the snapshots214ato214n. In this regard, the processor102may apply a primary key based filter to the snapshots214ato214nof the plurality of data sources to identify the data records220aincluded in the first conflated data record216a. In some examples, the data records220aincluded in the first conflated data record216amay be identified from one or more than one of the snapshots214ato214n.

In some examples, the processor102may fetch, decode, and execute the instructions118to generate a second conflated data record218that updates conflations among the updated data record210and the identified data records220a. The second conflated data record218may include the updated data record210and a second grouping of data records220b. In this regard, the data records220bin the second grouping may include one or more of the data records220ain the first grouping, which may be selected according to the updated conflation. For instance, the processor102may generate the second conflated data record218to include the updated data record210a, which may be the same as the updated data record210, and a group of data records220d, which may be a selected set of data records among the identified data records220aor220b, as depicted by arrow306inFIG. 3. In some examples, the data records220aor data records220bmay be data records whose source data records are from multiple data sources, and which may be identified using multiples ones of the snapshots214ato214n. The processor102may group the updated data record210aand selected ones of the identified data records220ain the second grouping based on the updates to the conflations among the updated data record210and the identified data records220ato generate the second conflated data record218. The processor102may determine the updated conflation by re-processing the identified data records220a, including the updated data record210a, according to a conflation algorithm and/or a conflation plan. As such, the updated data record210amay cause the second grouping for the second conflated data record218to be different than the first grouping for the first conflated data record216.

In some examples, the processor102may repeat the process for gathering associated conflated data records in the set of conflated data records212in order to identify additional data records to recursively re-process for conflation. The processor102may determine an additional conflated data record216cin the set of conflated data records212, which may be affected by the updated data record210a. For instance, the accuracy of the incremental updates to the set of conflated data records212may be improved with greater numbers of data records that are re-processed, up to a full refresh, at the cost of speed and efficiency of the updates. In this regard, the processor102may identify data records220aincluded in the first conflated data record216a, and may determine the additional conflated data record216cfrom the plurality of data sources based on the identified data records220a. In this regard, the processor102may generate the second conflated data record218to also include the identified additional data records220cfrom the additional conflated data record216c.

The processor102may fetch, decode, and execute the instructions118to replace the first conflated data record216with the second conflated data record218in the set of conflated data records212to incrementally update the set of conflated data records212, as depicted by arrow308inFIG. 3. In some examples, the processor102may designate the conflated data records216a,216b, and/or216cas “dirty”, and may delete these data records from the set of conflated data records212. The processor102may replace the “dirty” conflated data records216a,216b, and/or216cwith updated conflated data records, for instance the second conflated data record218that updates the correlations among the updated data record210and the identified data records220b. While the various data records in the present description, such as the updated data record210, the conflated data records216a,216b,216c, the second conflated data record218, and/or the like, are described as a single data record simply for ease of description and clarity, it should be understood that the various data records may include a large number of data records.

As described herein, the set of conflated data records212may be a unification or a conflation of data records from a plurality of data sources. By way of particular example and for purposes of illustration, the processor102may incrementally update a conflation match pair table (CMP), such as the set of conflated data records212, based on a current delta input for conflation (ΔEi), such as the update208, using an incremental update algorithm500depicted inFIG. 5.

In this example, the processor102may apply the Filter operation to identify an extended list of data records to reprocess. Given ΔEi, the processor102may have a list of primary keys for data records from the data source to be processed. Based on CMP, AM, NM, the processor102may compute the extended list of such primary keys {ids}, such as the first conflated data record216depicted inFIGS. 2 and 3. The processor102may use the extended list of primary keys to identify the candidate list of data records for which the processor102may redo the match.

In some examples, the processor102may use the filter to control to what extent to redo the match. For instance, the processor102may use the filter to control a tradeoff between quality of conflation vs. speed/efficiency. The Filter operation may rely on both primary keys in {ΔEi}, CMP, AM, NM, as well as a fuzzy match of data records that may have new primary keys against existing CMP. With this additional fuzzy match step, the processor102may handle the data records that may include new primary keys, but which should be clustered with existing records.

Various manners in which a processor implemented on the apparatus100may operate are discussed in greater detail with respect to the method600depicted inFIG. 6.FIG. 6depicts a flow diagram of a method600for replacing a first conflated data record in a set of conflated data records with a second conflated data record to incrementally update the set of conflated data records, in accordance with an embodiment of the present disclosure. It should be understood that the method600depicted inFIG. 6may include additional operations and that some of the operations described therein may be removed and/or modified without departing from the scope of the method600. The description of the method600is made with reference to the features depicted inFIGS. 1, 2, and 3for purposes of illustration.

At block602, the processor102may receive an updated data record, such as the updated data record210in the update208depicted inFIG. 3, from a data source among a plurality of data sources from which the processor102is to receive data records.

At block604, the processor102may filter a set of conflated data records, such as the set of conflated data records212depicted inFIG. 3, to determine a first conflated data record216ahaving data that matches data in the updated data record210. In this regard, the data record210ain the first conflated data record216amay be the same as the updated data record210.

The first conflated data record216amay include data records220afrom among the plurality of data sources arranged in a first grouping. At block606, the processor102may filter snapshots of the plurality of data sources, such as snapshots214ato214ndepicted inFIG. 3, to identify the data records220aincluded in the first conflated data record216a.

At block608, the processor102may generate a second conflated data record, such as the second conflated data record218depicted inFIG. 3, that may update correlations among the updated data record210aand the identified data records220a. In this regard, the processor102may re-process the updated data record210aand the identified data records220aaccording to a conflation algorithm to update the correlations. The generated second conflated data record218may include the updated data record210aand data records220b, which may be selected ones of the identified data records220aaccording to the updated correlations.

In some examples, the processor102may filter the set of conflated data records212based on a rule to determine associated conflated data records for re-processing, such as the first conflated data record216aand/or associated conflated data record216b. The rule may be a user-defined rule that is associated with a level of precision of a match between the updated data record210aand the data records220a,220bincluded in the first conflated data record216aand the associated conflated data record216b.

In some examples, the matched data in the associated conflated data record216b, such as data in data record210b, may have a format that is different from data in the first data record210a. In this regard, the processor102may filter the set of conflated data records212based on a level of precision of a match between the different formats of the data in the first data record210aand the data in the data record210bin the associated conflated data record216b.

In some examples, the processor102may determine the associated conflated data record216bamong the set of conflated data records212based on fuzzy matching between the updated data record210and the data records210bincluded in the set of conflated data records212.

In some examples, the snapshots214ato214nof the plurality of data sources may include previous updates received from the plurality of data sources. The processor102may generate and maintain a snapshot214ato214nfor each of the plurality of data sources. For each of the data sources, the processor102may update the corresponding snapshot214ato214nby appending new incremental updates to the previous incremental updates from a particular data source. The processor102may use the updated snapshots214ato214nto process subsequent updates from respective ones of the plurality of data sources.

In some examples, the processor102may determine primary keys associated with the data records220aincluded in the first conflated data record216a. The processor102may identify the data records220aincluded in the first conflated data record216ain the snapshots214ato214nbased on the primary keys.

In some examples, the processor102may group the updated data record210aand the identified data records220a,220bin a second grouping based on the updates to the correlations among the updated data record210aand the identified data records220a,220bto generate the second conflated data record218. In this regard, the second conflated data record218may include the updated data record210a, which may be the same as the updated data record210, and data records220d, which may be selective ones of the identified data records220a,220bthat may be grouped based on an updated correlation for conflation due to the updated data record210. The second grouping in the second conflated data record218may be different from the first grouping in the first conflated data record216aand the associated conflated data record216bbased on the updated correlation.

In some examples, based on the identified data records220a,220bincluded in the first conflated data record216aand the associated conflated data record216b, the processor102may determine an additional conflated data record216cin the set of conflated data records212. For instance, the processor102may use the data records220a,220bidentified in the snapshots214ato214nto determine, in the set of conflated data records212, additional data records for re-processing, such as data records220cincluded in the additional conflated data record216c. The processor102may then identify the additional data records220cincluded in the additional conflated data record216cfrom the snapshots214ato214nof the plurality of data sources and may generate the second conflated data record218to include selective ones of the identified additional data records220cin addition to the identified data records220aand220b.

Some or all of the operations set forth in the method600may be included as utilities, programs, or subprograms, in any desired computer accessible medium. In addition, the method600may be embodied by computer programs, which may exist in a variety of forms both active and inactive. For example, they may exist as machine-readable instructions, including source code, object code, executable code or other formats. Any of the above may be embodied on a non-transitory computer-readable storage medium.

Examples of non-transitory computer-readable storage media include computer system RAM, ROM, EPROM, EEPROM, and magnetic or optical disks or tapes. It is therefore to be understood that any electronic device capable of executing the above-described functions may perform those functions enumerated above.

Turning now toFIG. 7, there is shown a block diagram of a computer-readable medium700that may have stored thereon computer-readable instructions to generate snapshots of a plurality of data sources, determine a first grouped data record that is associated with an updated data record, generate a second grouped data record based on identified source data records from the generated snapshots, and update a set of grouped data records to replace the first grouped data record with the second grouped data record to incrementally update the set of grouped data records, in accordance with an embodiment of the present disclosure. It should be understood that the computer-readable medium700depicted inFIG. 7may include additional instructions and that some of the instructions described herein may be removed and/or modified without departing from the scope of the computer-readable medium700disclosed herein. The description of the computer-readable medium700is made with reference to the features depicted inFIGS. 1, 2, and 3for purposes of illustration. The computer-readable medium700may be a non-transitory computer-readable medium. The term “non-transitory” does not encompass transitory propagating signals.

The computer-readable medium700may have stored thereon machine-readable instructions702-714that a processor disposed in an apparatus100may execute. The computer-readable medium700may be an electronic, magnetic, optical, or other physical storage device that contains or stores executable instructions. The computer-readable medium700may be, for example, Random Access memory (RAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a storage device, an optical disc, and the like.

The processor may fetch, decode, and execute the instructions702to generate snapshots214ato214nof a plurality of data sources that may include previous updates to a set of grouped data records, such as the set of conflated data records212depicted inFIG. 3, from the plurality of data sources. The processor may fetch, decode, and execute the instructions704to store the generated snapshots214ato214nof the plurality of data sources. The snapshots214ato214nmay be generated and stored locally on the apparatus100.

The processor may fetch, decode, and execute the instructions706to receive an update208from a data source among the plurality of data sources. The update may include an updated data record210.

The processor may fetch, decode, and execute the instructions708to determine, through application of a filter on the set of grouped data records212, a first grouped data record, such as the first conflated data record216adepicted inFIG. 3. The first grouped data record216amay be associated with the updated data record210and may include other data records, such as data records220adepicted inFIG. 3, from among the plurality of data sources. The first grouped data record216amay include an updated data record210a, which may be the same as the updated data record210.

The processor may fetch, decode, and execute the instructions710to identify source data records for the first grouped data record216afrom the generated snapshots214ato214n, such as data records220a,220bin snapshots214ato214ndepicted inFIG. 3.

The processor may fetch, decode, and execute the instructions712to generate a second grouped data record, such as the second conflated data record218, that may update conflations among the updated data record210aand the identified source data records220a,220bin the snapshots214ato214n.

The processor may fetch, decode, and execute the instructions714to update the set of grouped data records212to replace the first grouped data record216awith the second grouped data record218to incrementally update the set of grouped data records212.

In some examples, the processor may filter the set of grouped data records212based on a rule to determine the first grouped data record216aand/or an associated grouped data record216b. The rule may be a user-defined rule that may be associated with a level of precision of a match between the updated data record210aand the data records220a,220bincluded in the grouped data records216a,216b.

The processor may determine primary keys associated with the data records220a,220bincluded in the grouped data records216a,216b. The processor may identify the source data records220a,220bfor the grouped data records216a,216bfrom the snapshots214ato214nbased on the primary keys.

In some examples, the processor may generate the second grouped data record218by grouping the updated data record210aand the identified source data records220a,220bbased on the updates to the correlations among the updated data record210aand the identified data records220a,220bto generate the second grouped data record218. The second grouped data record218may include the updated data record210aand selective ones of the identified source data records220a,220bbased on the updated correlation. In this regard, a grouping in the second grouped data record218may be different from a grouping of the data records220a,220bin the grouped data records216a,216bbased on the updated correlation due to the updated data record210a.