Secure data replication in distributed data storage environments

A described method includes receiving, by a database system, an instruction to change a first data element in a table in a database, which includes a first copy and a second copy of the table. A first entry is created in a first change-table. The first entry includes an updated value for a first data element. A second entry is created in a second change-table. Creating the second entry includes, changing the updated value into a ciphertext if the first data element is secured, and storing the ciphertext into the second entry. If the first data element is non-secured, the updated value is stored into the second entry as is. The second copy of the table is modified using the second change-table. The second copy of the table is used to respond to subsequent queries.

BACKGROUND

The present invention generally relates to computing technology, and more particularly, to a database management system that manages storage of electronic data in a distributed database system in a secured manner.

Data replication is the frequent electronic copying of data records stored on a source data store to a replica data store, either for data recovery or to allow users on multiple computing devices to access data relevant to their tasks without interfering with the work of others. In data storage systems, it is often desirable to have stored data replicated in multiple locations so that the data is available locally in each of the locations. Each location will have a local data storage device, which can satisfy requests to read data on its own, i.e., without needing to query other data storage devices of the data storage system.

SUMMARY

According to one or more embodiments of the present invention, a computer-implemented method for secure data replication in data storage environments includes receiving, by a database system, an instruction to change a first data element in a table that is in a database. The database includes a first copy of the table and a second copy of the table. The method further includes, in response to receiving the instruction, creating a first entry in a first change-table. The first entry includes data elements including an updated value for the first data element. The updated value is provided by the instruction. Further, the method includes creating a second entry in a second change-table. Creating the second entry includes, in response to determining that the first data element is a secured data element, changing the updated value into a ciphertext using a security algorithm, and storing the ciphertext into the second entry as content of the first data element in the second change-table. Further, in response to determining that the first data element is a non-secured data element, the updated value is stored into the second entry as content of the first data element of the second change-table. The method further includes modifying, by the database system, the second copy of the table according to the instruction that is received using the second change-table, wherein the second table is used to respond to subsequent queries.

According to one or more embodiments of the present invention, a database system includes a memory device, and one or more processors coupled with the memory device. The one or more processors perform a method for secure data replication in a database, wherein the database includes a first copy of a table and a second copy of the table. The method includes receiving an instruction to change a first data element in the table. The method further includes, in response to receiving the instruction, creating a first entry in a first change-table. The first entry includes data elements including an updated value for the first data element. The updated value is provided by the instruction. Further, the method includes creating a second entry in a second change-table. Creating the second entry includes, in response to determining that the first data element is a secured data element, changing the updated value into a ciphertext using a security algorithm, and storing the ciphertext into the second entry as content of the first data element in the second change-table. Further, in response to determining that the first data element is a non-secured data element, the updated value is stored into the second entry as content of the first data element of the second change-table. The method further includes modifying, by the database system, the second copy of the table according to the instruction that is received using the second change-table, wherein the second table is used to respond to subsequent queries.

According to one or more embodiments of the present invention, a computer program product includes a storage medium readable by one or more processing circuits. The storage medium includes instructions executable by the one or more processing circuits to perform a method for secure data replication in a database, wherein the database includes a first copy of a table and a second copy of the table. The method includes receiving an instruction to change a first data element in the table. The method further includes, in response to receiving the instruction, creating a first entry in a first change-table. The first entry includes data elements including an updated value for the first data element. The updated value is provided by the instruction. Further, the method includes creating a second entry in a second change-table. Creating the second entry includes, in response to determining that the first data element is a secured data element, changing the updated value into a ciphertext using a security algorithm, and storing the ciphertext into the second entry as content of the first data element in the second change-table. Further, in response to determining that the first data element is a non-secured data element, the updated value is stored into the second entry as content of the first data element of the second change-table. The method further includes modifying, by the database system, the second copy of the table according to the instruction that is received using the second change-table, wherein the second table is used to respond to subsequent queries.

DETAILED DESCRIPTION

Example embodiments of the present invention relate to, among other things, devices, systems, methods, computer-readable media, techniques, and methodologies for improving a database system. Change data capture technology is used in order to keep a database system synchronized with low latency. For such change data capture, the database system replicates instructions to change one or more data elements in the database. For example, such instructions to change a data element can include instructions to INSERT, UPDATE, and DELETE a data element. In a database system that implements change data capture, such instructions change the data element(s) between a target database and a replica database. Embodiments of the present invention further improve such distributed database systems that have replicas of the stored data by facilitating secure updates to the stored data when using the change data capture. One or more embodiments of the present invention facilitate transforming a field being updated into secured data, which can then be applied transparently through the existing replication process. During the application of the changes, a source change table and a target change table are synchronized to ensure that the secure data is stored in the database.

Change data capture includes capturing data from a database update log and capturing the update-records into a change-table with updated records as well as metadata. The change-table is then used to apply only the updated records into a replica of the original database.

FIG. 1depicts a database system that implements change data capture according to one or more embodiments of the present invention. The database system100includes at site A, a first data storage device102and host devices130A and130B, amongst others, in communication with the first data storage device102. The host devices130A and130B make requests to read data from and write data to the first data storage device102. The database system100further includes, at a site B, a second data storage device104and host devices150A and150B, amongst others, in communication with the second data storage device104. Similarly, the host devices150A and150B make requests to read data from and write data to the second data storage device104.

The first data storage device102and second data storage device104are in communication with each other so that the data they store, including any updates to the data made by requests to write data, is replicated at each of sites A and B.

The database system100further includes, at a site C, a third data storage device106, and, at a site D, a fourth data storage device107. Unlike the first data storage device102at site A and the second data storage device104at site B, the third data storage device106and fourth data storage device107are not in communication with any host devices. As a result, the third data storage device106and fourth data storage device107will not receive any requests to read or write data. Such sites with data storage devices that are not themselves receiving write requests from host devices, and so are merely acting as copies of data stored elsewhere, are described herein as non-active sites.

However, the third data storage device106and fourth data storage device107are each in communication with the first data storage device102and the second data storage device104, and each replicates the data stored at each of site A and B. In use, the third data storage device106and fourth data storage device107might, for example, be maintained while the migration is being done from one site to another (e.g., from site A to site C), while site A is remaining in active use; or to provide a backup for use in the case one of the first data storage device102or second data storage device104fails.

While in the embodiment shown inFIG. 1the third data storage device106and fourth data storage device107are located at different sites from the first data storage device102and the second data storage device104, in alternative embodiments one or both may be located at the same site as first data storage device102or the second data storage device104. Further, while in the embodiment shown inFIG. 1, site A and site B, each includes only a single data storage device in communication with host devices, in other alternative embodiments sites may include multiple data storage devices in communication with one or more host devices.

The first data storage device102and second data storage device104can replicate data between themselves.

It is understood that the shown data storage devices and sites are just one example embodiment of the present invention and that in other embodiments of the present invention, the number of sites, storage devices, and their organization can vary from the depicted example.

FIG. 2depicts a block diagram and operational flow for changing a data element in a data storage device. In the depicted example, the first data storage device102is illustrated; however, it is understood that any other storage device can be operated in the same manner. Further, it is understood that although, in the examples described herein, the data storage device stores data using a table data structure, the data storage device can store the data using other types of data structures, which can also be updated in substantially the same manner as a table. A data element can be a particular data field, i.e., a cell represented by a particular row-column combination. Alternatively, or in addition, a data element can be an entire row or an entire column in the table.

It should be noted that figures herein depict a target table being changed per example requests. It is understood that the changes are also being made in a source table. While the target table is shown with the change, the change is not shown in the source table so as to depict to the reader, the two states of the data—before the request (in the source table), and after the request completes (in the target table). The database system100implements the changes in the target table if the request is first hardened in the source table. Accordingly, the figures herein depict the mechanics of an update to the source table that triggers the replication in the target table, however, changes to the source table itself are not depicted herein, and it should be understood that such changes are made by the database system without affecting the technical solutions provided by one or more embodiments of the present invention.

Consider, in the example scenario ofFIG. 2, that the source table202includes data elements, and that one or more data elements are to be updated by an instruction210. In the example scenario herein, the data element201being changed is the field represented by row-1, col-2, such that the existing value of “yyy” is being changed to “ppp.” It should be further noted that the data values can be different in other embodiments of the present invention. Also, the number of rows and columns in table202is exemplary and that in one or more embodiments of the present invention the table202can include a different number of rows and columns. Further yet, while the example scenario depicts an “update” operation that changes an existing value, in one or more embodiments of the present invention, the change can include inserting a new value.

Referring to the example scenario, the instruction210changes the source table202to a target table204. For changing the data per the instruction210, by using data change capture techniques, the database system100creates a change-table220in response to the instruction210. The change-table220is an intermediate data structure that stores update-records. The update-records that are stored in the change-table220contain the values that are going to be applied to the target table204. The database system100subsequently uses an apply process that transforms the data elements that are to be updated; in this case, the data element201. In one or more embodiments of the present invention, the data elements that are to be updated are noted in a metadata portion222of the change-table220. In one or more embodiments of the present invention, the metadata portion222can further indicate an operation that is to be performed for the change to be applied. The change-table220can store several such changes that are to be applied to the target table204. The database system100executes one or more computer-executable instructions to apply these changes at a later time. The changes can be applied in a sequential manner in one or more embodiments of the present invention. Alternatively, in one or more embodiments of the present invention, the database system100applies the changes in a more efficient out-of-order manner, by analyzing the changes being made, and skipping any redundant changes.

However, technical challenges exist with such change capture techniques when applied to a distributed database system100when the data elements are to be secured, such as by encryption, masking, or any other technique to secure the values stored in the database system100. For example, when a data element is to be updated, the change-table220can include values for the data element that is to be applied to the target table204. These values are not secured in existing systems, and hence, are vulnerable. Here, “value” of the data element can also be referred to as the “content” of the data element.

Such technical challenges are addressed by one or more embodiments of the present invention. Embodiments of the present invention facilitate protecting each data element in the change-tables that are going to be applied to the target table204. In one or more embodiments of the present invention, the data elements are transformed into secured data, and such secured data is then applied to the target table204using existing replication techniques. The replication can apply the changes to the multiple sites in the database system100.

FIG. 3depicts a block diagram and operational flow for changing a data element in a data storage device in a secure manner according to one or more embodiments of the present invention. Implementing techniques described inFIG. 3improves the security of the data stored in the database system100. Consider the same example scenario again, where the data element201is a field being updated by the change instruction210from “yyy” to “ppp.”

Embodiments of the present invention create a copy of the change-table220; the copy is referred to as a “secondary change-table”320herein; however, the copy can be referred by any other term in other embodiments of the present invention. The secondary change-table320only includes those update-records from the first change-table220that have not yet been applied. In one or more embodiments of the present invention, the secondary change-table320is created periodically, at a predetermined frequency, such as every 10 seconds, every 2 minutes, or any other frequency.

Creating the secondary change-table320can include deleting an existing instance of the secondary change-table320and creating a new instance of the secondary change-table320. In one or more embodiments of the present invention, when creating an instance of the secondary change-table320, the database system100checks the timestamp of the update record in the first change-table220. The timestamp in an update record indicates when that update record was created. Using the timestamp, only those update-records that have been created within a predetermined duration from the present time, when the instance is being created, are copied into the instance of the secondary change-table320.

When the instance of the secondary change-table320is being made, the data element identifiers, for example, column names, are examined to determine if the data that is in the first change-table220is to be secured. A user/administrator can specify which data elements are to be secured. For example, such specifications can be stored in the user settings of the database system100. Accordingly, the database system100checks the settings to determine if any of the data elements in the first change-table220are to be secured. If there are any data elements to be secured, for example, the data element201, while making the copy of the data elements into the secondary change-table320, the identified data elements, are secured, and the secured data is stored in a data element301in the secondary change-table320. The secondary change-table320is then used to apply changes to the data elements in the target table204.

There can be several approaches for generating secure data when copying values of the data element from the update-records in the first change-table220to the update-records in the secondary change-table320. For example, the secure data can be created with a security operation that is reversible. In this case, the data value can be encrypted, and the resulting ciphertext can include security metadata that facilitates reverting the ciphertext to the original data value. The security metadata is stored in the metadata322in the secondary change-table320. Alternatively, or in addition, the database system100stores the security metadata in other locations on the servers. The database system100tracks such related information with the secondary change-table320, so that the data value can be interpreted at a later point to restore it from the ciphertext that is generated by the security function. To facilitate storing the ciphertext, in one or more embodiments of the present invention, the database system100changes the schema of the data elements between the first change-table220and the secondary change-table320. The schema change is required because the securing operation changes the format of the data value that is stored in the original field to the type of data value of the ciphertext. The “format” of data can be a field-type associated with the data element, for example, text, number, date, etc. The format is selected from a list of data types provided by the database system100.

Other examples of securing the data value can include format preserving methods, such as masking, redactions, randomization, etc. In these cases, the schema of the data element being secured remains unchanged between the first change-table220and the secondary change-table320. These securing techniques do not change the format of the data value, rather change the value of the data that is stored using a specific ciphering formula. Deciphering the original data from the secured data can include applying a reverse of the ciphering formula.

After such securing, the secondary change-table320has a potentially modified schema and secured data rather than clear, i.e., non-secured data, in the original form. Updating the data in the database includes performing an “apply process” that pulls changes from the secondary change-table320(rather than the first change-table220) and applies the changes to the target table204. The target table204now contains secure data from the secondary change-table320. When users request a read from the target table204, the secured data is retrieved, which then is processed by deciphering the retrieved data. The deciphering can be performed by a central system, such as the database system100, or another security server (not shown). Alternatively, the deciphering can be performed locally by a client device (not shown) at the user's end.

FIG. 4depicts a flowchart of a method400for securing data replication in distributed data storage environments according to one or more embodiments of the present invention. The method400can be executed by one or more processing units that are part of the database system100. The method400includes receiving an instruction210to change a data element in the first data storage device102, at block402. The instruction210can be in the form of a computer-executable instruction, such as using a structured query language (SQL) or other such programming languages. Further description of the flowchart is provided using the example scenario that is described herein with respect to the change instruction210inFIG. 2andFIG. 3.

An entry in the first change-table220is created in response to the instruction210that is received, at block404. The entries in the first change-table220include update-records that are to be applied to the source table202. Each entry includes data elements that are to be updated from the source table202. The database also includes a target table204in which the changes from the first change-table220are applied. The target table204is then used for responding to subsequent queries.

The method400further includes creating a copy of the entry from the first change-table220in the secondary change-table320, at block406. The copy is created only for an entry that is new, i.e., with a timestamp that indicates that the entry is created after the last iteration of copying entries from the first change-table220. As noted earlier, entries from the first change-table220are copied into the secondary change-table320at a predetermined frequency. In one or more embodiments of the present invention, making such copies can be manually initiated by an administrator/user.

Creating the copy includes determining whether the data element201that is being changed is a secured data element, at block414. For example, the data element201, such as a row, or a field, or a column, can be secured by using an encryption key, a redaction (e.g., a mask), a randomization algorithm, or any other securing algorithm. Alternatively, the data element201may not be secured. The user settings associated with the first data storage device102can include information on whether the data element201is secured. Alternatively, or in addition, the user settings can be associated with a database that is stored in the first data storage device102, the database including the source table202of which the data element201is a part. The database system100can check whether the data element201is secured based on the user settings. Alternatively, or in addition, the database system100can determine the security of the data element201based on the metadata of the data element201.

If the data element201is not secured, a copy of the data element301is made in the entry in the secondary change-table320as is, i.e., without securing the content of the data element, at block416. Alternatively, if the data element201is secured, the content of the data element301is secured using a securing algorithm, which results in ciphertext, at block418. The ciphertext is then stored in the entry in the secondary change-table320, at block420. The copy of the data element301in the secondary change-table320is created in this manner by checking and copying the content of each data element in the first entry from the first change-table220, at block412.

FIG. 5depicts a flowchart of a method for securing a data element when copying from first entry to a second entry according to one or more embodiments of the present invention. The method500includes reading the content of the data element from the first entry from the first change-table220, at block502. Once it has been determined that the content is to be secured, the method500further includes determining whether the securing includes a schema/format change of the content or not, at block504.

If format-preserving security is to be applied, the content of the data element from the first entry is transformed into ciphertext using a schema-preserving securing algorithm, at block506. Examples of format-preserving security algorithms can include masking, redacting, randomizing characters/elements of the content, or any other such algorithms. In this case, the schema remains unchanged between the content of the data element from the first entry and the ciphertext that is stored in the secondary change-table320. In turn, the schema of the first change-table220and that of the secondary change-table320remain unchanged.

In the case where format-preserving security is not applied, the content of the data element from the first entry is transformed into ciphertext using a securing algorithm that may not preserve the format, at block508. Accordingly, the ciphertext created in this case has a different, i.e., distinct schema from that of the content in the data element from the first entry. For example, the securing algorithm, in this case, can be an encryption algorithm that generates the ciphertext that can include character types that may not be included in the original content in the first entry.

In one or more embodiments of the present invention, the ciphertext can be created where the operation is reversible. In this case, the original content is encrypted, and the resulting ciphertext can include security-metadata. Alternatively, the database system100tracks the security-metadata, such that the ciphertext can be interpreted at a later point to restore the original content. In turn, the database system100changes the schema between the first change-table220and the secondary change-table320, as the securing operation changes the type of the data element field.

Embodiments of the present invention facilitate creating secure data in a replicated table based on policy and database schema by having a secondary change capture table, which contains secured data. Further, embodiments of the present invention facilitate creating the secondary change capture table based on the original change capture table. A copy method is performed that includes checking data elements, such as column names, to identify fields for which secure elements are to be created in the secondary change table. Further yet, embodiments of the present invention facilitate applying changes from the secondary change capture table, with the secured data, to a new table, where queries to the database system are run against the new table, and in turn against secure data.

Turning now toFIG. 6, a computer system600is generally shown in accordance with an embodiment. The computer system600is part of the database system100and the database system100and facilitates executing methods described herein. The computer system600is responsible for handling/providing the various functionalities of the database system100or the database system100. The computer system600can be an electronic, computer framework comprising and/or employing any number and combination of computing devices and networks utilizing various communication technologies, as described herein. The computer system600can be easily scalable, extensible, and modular, with the ability to change to different services or reconfigure some features independently of others. The computer system600may be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, computer system600may be a cloud computing node. Computer system600may be described in the general context of computer system executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. Computer system600may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, including memory storage devices.

As shown inFIG. 6, the computer system600has one or more central processing units (CPU(s))601a,601b,601c, etc. (collectively or generically referred to as processor(s)601). The processors601can be a single-core processor, multi-core processor, computing cluster, or any number of other configurations. The processors601, also referred to as processing circuits, are coupled via a system bus602to system memory603and various other components. The system memory603can include a read-only memory (ROM)604and a random access memory (RAM)605. The ROM604is coupled to the system bus602and may include a basic input/output system (BIOS), which controls certain basic functions of the computer system600. The RAM is read-write memory coupled to the system bus602for use by the processors601. The system memory603provides temporary memory space for operations of said instructions during operation. The system memory603can include random access memory (RAM), read-only memory, flash memory, or any other suitable memory systems.

The computer system600comprises an input/output (I/O) adapter606and a communications adapter607coupled to the system bus602. The I/O adapter606may be a small computer system interface (SCSI) adapter that communicates with a hard disk608and/or any other similar component. The I/O adapter606and the hard disk608are collectively referred to herein as a mass storage610.

Software611for execution on the computer system600may be stored in the mass storage610. The mass storage610is an example of a tangible storage medium readable by the processors601, where the software611is stored as instructions for execution by the processors601to cause the computer system600to operate, such as is described hereinbelow with respect to the various Figures. Examples of computer program product and the execution of such instruction is discussed herein in more detail. The communications adapter607interconnects the system bus602with a network612, which may be an outside network, enabling the computer system600to communicate with other such systems. In one embodiment, a portion of the system memory603and the mass storage610collectively store an operating system, which may be any appropriate operating system, such as the z/OS or AIX operating system from IBM Corporation, to coordinate the functions of the various components shown inFIG. 6.

Additional input/output devices are shown as connected to the system bus602via a display adapter615and an interface adapter616and. In one embodiment, the adapters606,607,615, and616may be connected to one or more I/O buses that are connected to the system bus602via an intermediate bus bridge (not shown). A display619(e.g., a screen or a display monitor) is connected to the system bus602by a display adapter615, which may include a graphics controller to improve the performance of graphics-intensive applications and a video controller. A keyboard621, a mouse622, a speaker623, etc. can be interconnected to the system bus602via the interface adapter616, which may include, for example, a Super I/O chip integrating multiple device adapters into a single integrated circuit. Suitable I/O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include common protocols, such as the Peripheral Component Interconnect (PCI). Thus, as configured inFIG. 6, the computer system600includes processing capability in the form of the processors601, and, storage capability including the system memory603and the mass storage610, input means such as the keyboard621and the mouse622, and output capability including the speaker623and the display619.

In some embodiments, the communications adapter607can transmit data using any suitable interface or protocol, such as the internet small computer system interface, among others. The network612may be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, among others. An external computing device may connect to the computer system600through the network612. In some examples, an external computing device may be an external web server or a cloud computing node.

It is to be understood that the block diagram ofFIG. 6is not intended to indicate that the computer system600is to include all of the components shown inFIG. 6. Rather, the computer system600can include any appropriate fewer or additional components not illustrated inFIG. 6(e.g., additional memory components, embedded controllers, modules, additional network interfaces, etc.). Further, the embodiments described herein with respect to computer system600may be implemented with any appropriate logic, wherein the logic, as referred to herein, can include any suitable hardware (e.g., a processor, an embedded controller, or an application-specific integrated circuit, among others), software (e.g., an application, among others), firmware, or any suitable combination of hardware, software, and firmware, in various embodiments.

Characteristics are as follows:

Service Models are as follows:

Deployment Models are as follows:

The present invention may be a system, a method, and/or a computer program product. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to carry out aspects of the present invention.