Data synchronisation across multiple data storages when processing transactions received on a network

A synchronization unit examines the packets on a network to determine the changes being caused to a database server, and propagates the same changes to corresponding backup server(s). As the synchronization is implemented in a separate unit, the throughput performance of database servers may not be impeded due to the synchronization requirement.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to data storage systems such as databases, and more specifically to a method and apparatus for synchronizing data across multiple data storage systems when processing transactions received on a network.

2. Related Art

Data storage systems, such as database systems, are generally used to store data. The stored data can then be later accessed and used for various purposes, as is well known in the relevant arts.

The same data is often stored on multiple data storage systems. Such an approach provides several advantages such as having a backup storage system in case the other data storage systems fail for whatever reason. An another example, each data storage system can be used to serve different sets of users, thereby providing a superior throughput performance.

At least to maintain such same data on multiple data storage systems, there is a general need to synchronize the data across the data storage systems. Synchronization generally refers to effecting a change on all the data storage systems if the change is performed on one of the data storage systems. For convenience, the data storage system on which the change is to be performed (first) is referred to as a primary storage system, and the remaining data storage systems are referred to as backup storage systems.

One source of the changes are transaction requests (such as insert or update statements in SQL when the data storage system corresponds to a database system) received on networks. In such a scenario, the changes effected to the data in the primary storage system need to be propagated to the backup storage system as well.

In one prior approach, the primary storage system operates to propagate each change to the backup storage systems. One problem with such an approach is that the overhead to propagate changes may negatively impact the performance of the primary data server, which may be unacceptable in several situations.

Accordingly, what is needed is a method and apparatus which overcomes at least some of the problems noted above when synchronizing data across multiple data storage systems.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A synchronization unit provided according to an aspect of the present invention is designed to examine packets on a network and determine whether a transaction corresponding to a packet causes a change in a primary server. The synchronization unit then propagates the changes to any backup servers automatically. Since the primary server is relieved (at least substantially) of the task of propagating the changes, the throughput performance of primary server can be enhanced. In addition, the synchronization operations may also be performed quickly due to the use of a separate system for synchronization unit.

2. Example Environment

FIG. 1is a block diagram illustrating an example environment in which various aspects of the present invention can be implemented. The environment is shown containing client systems110A and110B, Internet120, firewalls130A and130B, local area networks140A and140B, database servers150A and150B, synchronization unit170and backup server180A and180B. Each component is described below in further detail.

Internet120generally refers to a conglomeration of networks connecting various systems, and is implemented using protocols such as Internet Protocol (IP). Local area networks (LANs)140A and140B can be implemented using protocols such as IP, ATM, frame relay, Ethernet, and is generally owned or operated by an organization.

Firewall130A is implemented using routers/switches, and provides connectivity between systems connected to LAN140-A and those accessible via Internet120(in addition to tasks such as preventing unauthorized access). Firewall130B is implemented similarly. Client systems110A and110B establish database connections with database servers150A and150B, and send transaction requests causing the data stored in the servers to be changed.

Database servers150A and150B represent example data storage systems, in which data is stored. Database servers allow storing, retrieval and modifications of/to data using structured queries, as is well known in the relevant arts. Backup servers180A and180B respectively store the same data as database servers150A and150B, and thus there is a general need to synchronize backup server180A with database server150A, and backup server180B with database server150B.

Synchronization unit170, provided according to an aspect of the present invention, ensures such synchronization based on packets transmitted on LAN140A. In particular, synchronization unit170monitors the packets on LAN140A to determine the changes to database server150A/B, and causes the changes to be replicated on (or propagated to) corresponding backup servers.

As may be appreciated, synchronization unit170is provided external to the data storage systems, thereby avoiding degradation of throughput performance of the storage systems. The manner in which synchronization unit170may be implemented is described below with examples.

3. Operation of Synchronization Unit

FIG. 2is a flow-chart illustrating the manner in which a synchronization unit operates in an embodiment of the present invention. The flow chart is described with reference toFIG. 1(and also with respect to one pair of database server and backup server) merely for illustration. However, the features can be implemented in other environments (and with more pairs of database servers and backups servers) as well. The flow chart begins in step201, in which control immediately passes to step210.

In step210, synchronization unit170receives data indicating a backup server corresponding to a database server of interest. In an embodiment, a database administrator provides configuration data indicating backup servers corresponding to database servers of interest. Thus, in the scenario ofFIG. 1, synchronization unit170is configured to indicate that backup servers180A and180B are respectively associated with database servers150A and150B.

In step230, synchronization unit170determines the connections on which updates occur to database server150A. The connections are determined according to the protocols using which client systems communicate with the database servers. The manner in which connections are determined in an example environment is described in a section below in further detail.

In step250, synchronization unit170monitors the connections on LAN140to determine a transaction request causing a change in data stored in database server150A. The determination generally needs to take into consideration the packet formats and protocol.

In step270, synchronization unit170generates a transaction request to cause similar change in the data stored in backup server150A. Accordingly, synchronization unit170needs to operate similar to client systems in initiating and completing (i.e., replicating the change) the transaction request on backup server150A. Therefore, synchronization unit170may need to be provided the authorization to effect changes on the backup servers.

Control then passes to step250to process the next transaction on the determined connections. It should be further appreciated that additional connections (step230) could be formed while the loop of steps250and270is being executed, and steps250and270need to be executed for each presently active connection (though not shown in the Figure). Similarly, the loop of steps250and270needs to end when a connection terminates.

From the above, it may be appreciated that the implementation of synchronization unit170needs to be consistent with the protocol and packet formats using which database updates (insert, change, delete) are performed. Accordingly, the implementation considerations with respect to an example protocol and packet format are described below.

4. Protocol Consideration

In TCP/IP environment, a transport connection is defined by the source IP address, destination IP address, source port and destination port, as is well known in the relevant arts. In general, to establish a database connection, client systems110A sends a connection setup request to server system150A at a pre-specified (destination) port. In one embodiment, the pre-specified port number equals 1521 (in decimal).

Server system150A and client systems110A can then be designed to setup a database connection on the same transport connection (as on which the connection setup request is received), or alternatively a new transport connection can be setup to support the desired database connection. The implementation of both approaches will be apparent to one skilled in the relevant arts, and different server systems can be implemented to setup database connections using different ones of the noted approaches.

Accordingly, it should be appreciated that synchronization unit170needs to be implemented to monitor packets on LAN140A, consistent with the specific approach of database connection setup. One embodiment described below is implemented according to Net8 Protocol, which is described in further detail in a book entitled, “Oracle Net8: Configuration and troubleshooting”, by Toledo et al, Publisher: O'Reilly; 1 edition (December, 2000), ISBN Number: 1565927532.

Once the connection is setup, synchronization unit170again needs to examine the packets consistent with the format using which transaction requests are presented on the database connections. Accordingly, the packet format in an example scenario is described below in further detail.

5. Packet Format

FIG. 3contains an example packet format using which database servers and client systems communicate in one embodiment. The packet format is shown with respect to TCP/IP for illustration, however other protocols can be used consistent with the approaches illustrated herein. For illustration, it is assumed that all database connection requests and transactions (and control packets) are on port1521of server system150A/B.

IP header310indicates the source IP address and destination IP address in the corresponding fields. IP protocol field311contains a value representing TCP. Accordingly, fields321and322respectively contain source port number and destination port number.

Length field330is of 2 bytes long, and indicates the length of the data (in octets) from that point to the end of the packet. Flags field350contains flags, which can be used for various controls, and data field360stores data as appropriate for specific packet types.

Type field340indicates the specific purpose for which the present packet is directed. The type field can take on values 1 (database connection setup request), 2 (connection accept from the database server), 3 (refuse connection), and 6 (transaction request contained in the data field360).

Thus, when a packet with type field340equaling 1 is followed by a packet with type field equaling 2 on LAN140A, synchronization unit170determines that a database connection is setup between the corresponding client system and the database server. Then, synchronization unit170needs to examine packets on the same database connection with the type field set to value 6. The data in such packets indicates whether the packet contains a transaction request which would cause a change in database server150A.

Synchronization unit170parses such packets to determine the specific change that is being caused in database server150A, and causes the change to be propagated to backup server180A. For example, the SQL query contained in the packet may be sent to backup server180A to effect the same change, and thus achieve the desired synchronization. The packets between client systems and server systems may be continued to be monitored until the database connection is closed (e.g., by appropriate value in type field340).

It should be understood that synchronization unit170can be implemented using a desired combination of hardware, software and firmware, as suited for a specific situation. The description is continued with respect to an embodiment in which various features are operative by execution of software instructions in a digital computer system.

6. Digital Processing System

FIG. 4is a block diagram illustrating the details of synchronization unit170in which various aspects of the present invention are operative by execution of appropriate software instructions. Synchronization unit170may contain one or more processors such as central processing unit (CPU)410, random access memory (RAM)420, secondary memory430, graphics controller460, display unit470, network interface480, and input interface490. All the components except display unit470may communicate with each other over communication path450, which may contain several buses as is well known in the relevant arts. The components ofFIG. 4are described below in further detail.

CPU410may execute instructions stored in RAM420to provide several features of the present invention. CPU410may contain multiple processing units, with each processing unit potentially being designed for a specific task. Alternatively, CPU410may contain only a single general purpose processing unit. RAM420may receive instructions from secondary memory430using communication path450.

Graphics controller460generates display signals (e.g., in RGB format) to display unit470based on data/instructions received from CPU410. Display unit470contains a display screen to display the images defined by the display signals. Input interface490may correspond to a keyboard and/or mouse. Network interface480provides connectivity to a network (e.g., using Internet Protocol), and may be used both for monitoring packets on LAN140A, as well as to send/receive IP packets.

Secondary memory430may contain hard drive435, flash memory436and removable storage drive437. Secondary memory430may store the data and software instructions, which enable server system150to provide several features in accordance with the present invention. Some or all of the data and instructions may be provided on removable storage unit440, and the data and instructions may be read and provided by removable storage drive437to CPU410. Floppy drive, magnetic tape drive, CD_ROM drive, DVD Drive, Flash memory, removable memory chip (PCMCIA Card, EPROM) are examples of such removable storage drive437.

Removable storage unit440may be implemented using medium and storage format compatible with removable storage drive437such that removable storage drive437can read the data and instructions. Thus, removable storage unit440includes a computer readable storage medium having stored therein computer software and/or data.

In this document, the term “computer program product” is used to generally refer to removable storage unit440or hard disk installed in hard drive435. These computer program products are means for providing software to server system150. CPU410may retrieve the software instructions, and execute the instructions to provide various features of the present invention described above.

While various embodiments of the present invention have been described above, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of the present invention should not be limited by any of the above described exemplary embodiments, but should be defined only in accordance with the following claims and their equivalents. Also, the various aspects, features, components and/or embodiments of the present invention described above may be embodied singly or in any combination in a data storage system such as database.