Method and apparatus for routing a data stream through a plurality of data movers independent of a network interface type

A method and apparatus for routing a data stream through a plurality of data movers independent of a network interface type is provided. In one embodiment, the method for routing the data stream to a destination with indifference to network interface type includes segregating the data stream into a plurality of data blocks at an application layer, wherein the plurality of data blocks are to be routed to a destination through the plurality of data movers and coordinating data path selection for communicating the plurality of data blocks to the plurality of data movers over a plurality of data paths.

BACKGROUND

1. Field of the Invention

Embodiments of the present invention generally relate to backup techniques and, more particularly, to a method and apparatus for routing a data stream through a plurality of data movers independent of a network interface type to optimize load balancing.

2. Description of the Related Art

In a typical computing environment, small to large sized organizations utilize various technologies, such as a data storage system, to store and protect mission critical data. The data storage system, generally, includes a plurality of data movers and an array of physical disk drives (e.g., ATA disks, Fibre channel disks, a magnetic tape library and any other data storage device) that facilitate data backup and/or restoration. A data mover, in any type of the data storage system, refers to the function (e.g., a process) that is able to push or pull (e.g., send or receive, respectively) data over a plurality of data paths between various computing environments (e.g., various platforms, protocols, systems and the like).

The data movers, generally, include data transfer systems, devices and/or software that utilize the capabilities of the data storage system (e.g., data backup, duplication and/or restoration processes) to quickly and reliably route the mission critical data from one location (e.g., a client computer, a database and the like) to another location (e.g., tape library, disk drives and the like) through a network interface. For example, a data movers may read the mission critical data from one data storage device and then, transfer the mission critical data to another data storage device.

The mission critical data may be lost and/or corrupted due to various system failures or a virus attack. As such, the mission critical data may be backed up on a regular basis (e.g., continuously) to the one or more storage devices (e.g., a tape drive, a hard disk drive and/or the like). In conventional backup techniques, the mission critical data is routed through a single network interface or data path. In other words, each data block of the mission critical data is transmitted over the same data path regardless of an input/output (I/O) load and/or another better performing data path. Consequently, the single data path is congested and becomes a bottleneck for routing the mission critical data from a computer to the one or more storage devices.

There are one or more technologies that leverage two data paths to communicate a data stream between the client and a single data mover for a backup process. Such technologies, however, operate at a network layer (e.g., the network layer of Open System Interconnection (OSI) or Internet layer of TCP/IP). If the single data mover fails during transmission, the data stream is lost. Furthermore, the backup process also fails and must be restarted. Additionally, if any of the two data paths fail during transmission, the data stream is also lost if the backup process cannot be failed over to the other data path and/or cannot be retried. For example, the backup process may employ a data transmission protocol that does not permit retries after such a failure.

Unfortunately, error recovery solutions are limited to coarse-grain checkpoint restart mechanisms, which locate a point-in-time at which the backup process was interrupted and restarts the backup process from that point-in-time. Moreover, such technologies cannot enable fine granularity for the error recovery solutions if the data stream is sent as a completely separate archiving (.TAR) file. As a result, the conventional backup techniques are unable to provide a reliable and efficient backup of the data stream over multiple data paths and suffer from network bandwidth and throughput constraints.

Therefore, there is a need in the art for a method and apparatus for routing a data stream through a plurality of data movers over a plurality of data paths independent of a network interface type to optimize load balancing.

SUMMARY OF THE INVENTION

Embodiments of the present invention comprise a method and apparatus for routing a data stream through a plurality of data movers independent of a network interface type. In one embodiment, a method for routing a data stream to a destination through a plurality of data movers with indifference to network interface type includes segregating the data stream into a plurality of data blocks at an application layer, wherein the plurality of data blocks are to be routed to a destination through the plurality of data movers and coordinating data path selection for communicating the plurality of data blocks to the plurality of data movers over a plurality of data paths.

DETAILED DESCRIPTION

FIG. 1is a block diagram of a system100for routing a data stream to a destination through a plurality of data movers with indifference to network interface type according to one or more embodiments of the present invention. In one embodiment, the system100comprises a client102, a plurality of data movers104, a server106and a destination128where each is coupled to the other through a network108.

The client102is a type of computing device (e.g., a laptop, a desktop, a Personal Digital Assistant (PDA), a mobile phone and/or the like), such as those generally known in the art. The client102includes a Central Processing Unit (CPU)110, various support circuits112and a memory114. The CPU110may comprise one or more commercially available microprocessors or microcontrollers that facilitate data processing and storage. The various support circuits112facilitate the operation of the CPU110and include one or more clock circuits, power supplies, cache, input/output circuits and the like. The memory114comprises at least one of Read Only Memory (ROM), Random Access Memory (RAM), disk drive storage, optical storage, removable storage and the like. The memory114includes various software packages, such as an agent116. The memory114further includes various data, such as a data stream117.

The plurality of data movers104are illustrated as a data mover1041. . . a data mover104n. Generally, the data movers104include processes that serve as an interface between the client102and the destination128. In one embodiment, the data movers104are abstract machines running on one or more computing devices. For example, the data movers104may be instances of middleware (e.g., software comprising a collection of algorithms for communicating data for a variety of applications, such as data backup, recovery and duplication tasks) executed by the one or more computing devices (e.g., a media server). In another embodiment, the data movers104are physical devices with embedded software for routing the data stream117to the destination128. In operation, the data movers104receives the plurality of data blocks that form the data stream117from the client102through the network108in accordance with various communication protocols. The data stream117may be communicated to the data movers104via various application layer protocols, such as File Transfer Protocol (FTP), Network File System (NFS), Common Internet File System (CIFS) and/or the like.

The server106is a type of computing device (e.g., a laptop, a desktop, a Personal Digital Assistant (PDA), a mobile phone and/or the like), such as those generally known in the art. The server106includes a Central Processing Unit (CPU)118, various support circuits120and a memory122. The CPU118may comprise one or more commercially available microprocessors or microcontrollers that facilitate data processing and storage. The support circuits120facilitate the operation of the CPU118and include one or more clock circuits, power supplies, cache, input/output circuits and the like. The memory122comprises at least one of Read Only Memory (ROM), Random Access Memory (RAM), disk drive storage, optical storage, removable storage and the like. The memory122includes various software packages, such as backup software126. The memory112further includes various data, such as policy information124.

The network108comprises a communication system that connects computers by wire, cable, fiber optic and/or wireless link facilitated by various types of well-known network elements, such as hubs, switches, routers and the like. The network108may employ various well-known protocols to communicate information amongst the network resources. For example, the network108may be a part of the Internet or intranet using various communications infrastructure, such as Ethernet, WiFi, WiMax, General Packet Radio Service (GPRS) and the like. Furthermore, the network106may form a portion of a Storage Network Area (SAN) using various communications infrastructure such as Ethernet, Fibre Channel, InfiniBand, SCSI (Small Computer System Interface) and/or the like.

According to various embodiments, the agent116includes software code that is configured to segregate the data stream117into a plurality of data blocks at an application layer. For example, the application layer of a networking architecture, such as TCP/IP, provides one or more application-level network services. In one embodiment, the agent116coordinates data path selection for communicating the plurality of data blocks to the data movers104over the plurality of data paths. The agent116communicates the plurality of data blocks over a plurality of data paths to one or more data movers of the data movers104. In one embodiment, the agent116selects a data path of the plurality of data paths in accordance with the policy information124. For example, the agent116may select a data path to a data mover having a lowest Input/Output (I/O) load. In another embodiment, the agent116assigns a number to each data block of the plurality of data blocks. For example, each number may correspond with a position in the data stream117.

In one embodiment, the data blocks are asynchronously reassembled at a destination128volume regardless of how the data blocks are received at the one or more data movers of the data movers104. In one embodiment, the agent116identifies one or more data blocks of the data blocks that are not present at the destination128. For example, if the agent116receives a transmission error associated with a data path, then the one or more data blocks were not transmitted correctly over the data path or received at the destination128. In response, the agent116resends the identified one or more data block through another data path of the plurality of data paths.

According to various embodiments, the policy information124defines one or more restrictions to data path selection by the agent116. For example, the policy information124may indicate that the agent116may not select a data path having a particular load. In another embodiment, the policy information124may permit the agent116an unrestricted use of the plurality of data paths in order to route the data blocks to the destination128. In one embodiment, the policy information124defines one or more load balancing techniques for the data path selection, such as round robin, next available path, weight-based decision and/or the like. For example, the policy information124may assign a plurality of user-defined weights to the plurality of data paths in which the agent116selects an available data path having a lowest weighted input/output load value (e.g., weighted average).

The backup software126may be enterprise backup software (e.g., SYMANTEC NetBackup products, SYMANTEC BackupExec products and/or the like). According to various embodiments, the backup software126is designed to facilitate storage (i.e., a backup) of various client data. The backup software126cooperates with the agent116to create a backup (e.g., an image) of the various client data in the form of the data stream117. For example, the data stream117may include a volume-level backup image that is routed to the destination128.

The destination128generally includes one or more storage devices, such as sequential storage devices (i.e., magnetic tape drives), optical storage devices (e.g., hard disk drives, a disk array) and/or the like. In one embodiment, the data stream117may be routed to the destination128using various data transmission protocols, such as Ethernet using IP (Internet Protocol), iSCSI (Internet Small Computer System Interface) and/or the like. According to one or more embodiments, the data movers104store the data stream117in the destination128. In one embodiment, the data movers104write each data block to appropriate locations in the destination128. As an example, the data movers104may write a data block to a location that corresponds with a position (e.g., a number) amongst the data blocks that form the data stream117. As such, the data stream117is reassembled at the destination128in a correct order.

FIG. 2is a functional block diagram that illustrates a multi-path system200for routing a data stream to a destination through a plurality of data movers with indifference to network interface type according to one or more embodiments of the present invention.

In one embodiment, the multi-path system200includes the client102that is coupled with a data mover202a data mover204and a data mover206through a data path208, a data path210and a data path212, respectively. The data mover202and the data mover204are coupled with a storage server214through a data path218and a data path220, respectively. The data mover204is coupled with a storage server216through a data path222. The storage server214and the storage server216are coupled with a disk224through a data path226and a data path228, respectively.

The storage server214and the storage server216generally include various components (i.e., hardware and software) that are configured to manage storage resources (e.g., storage devices) within a data storage system. For example, the storage server214and the storage server216process write requests from the client102and write data to the disk224accordingly.

In operation, a data stream is segregated at the client102into a plurality of data blocks at an application-layer according to one or more embodiments. Because the plurality of data blocks is segregated at the application-layer, any network interface type may be used as a data path to a data mover. For example, a data block a plurality of data blocks may be communicated to the data mover202over Ethernet using IP (i.e., Internet Protocol) as the data path208. Then, another data block a plurality of data blocks may be communicated to the data mover204over Fibre Channel using an SYMANTEC NetBackup SANClient implementation as the data path210. Accordingly, the data mover202and the data mover204write the data block and the another data block to the disk224through the storage server214. Even though the data path208differs from the data path210, the data block and the another data block are assembled at the storage server214and written to appropriate locations at the disk224.

Furthermore, the plurality of data blocks are numbered in an sequential order according to one or more embodiments. For example, a number for a particular data block corresponds with a position within the data stream. In addition, because each data block of the data stream is assigned a number, the plurality of data blocks may be reassembled at the disk224asynchronously. As such, the each data block is written to a location on the disk224that corresponds with the assigned number. In other words, the plurality of data blocks may be written to the disk224regardless of an order in which the plurality of data blocks arrive at the data mover202, the data mover204and/or the data mover206.

Additionally, in response to a transmission error, each and every lost data block may be resent over a different network interface as a data path. For example, if there is a transmission error at the data path208, the data block of the plurality of data blocks is resent to the data mover206over iSCSI (Internet Small Computer System Interface) as the data path212. Accordingly, the data mover206writes the data block to the disk224through the storage server216. Because the plurality of data blocks are numbered, the data mover206writes the data block to a location that corresponds with an assigned number. Hence, the another data block may be written to the disk224after the data block even through the data block was communicated before the another data block. Moreover, the data block and the another data block may be written to correct locations in the disk224using different storage servers.

FIG. 3is a flow diagram of a method300for routing a data stream to a destination through a plurality of data movers with indifference to network interface type according to one or more embodiments. The method300starts at step302and proceeds to step304, at which an instruction to perform a backup process is received. In one embodiment, a client (e.g., the client102ofFIG. 1) receives the instruction from a server (e.g., the server106ofFIG. 1).

At step306, a data stream (e.g., the data stream117ofFIG. 1) is created. At step308, the data stream is segregated at an application layer of a network architecture. In one embodiment, an agent (e.g., the agent116ofFIG. 1) segregates the data stream into one or more data blocks. At step310, policy information (e.g., the policy information124ofFIG. 1) is accessed. At step312, a data path is selected for routing a data block. At step314, the data block is communicated to a data mover. At step316, a determination is made as to whether there are more data blocks to be routed. If it is determined that there are more data blocks to be routed (option “YES”), then the method300returns to step312. If, at step316it is determined that there are no more data blocks to be routed (option “NO”), then the method300proceeds to step318, at which the method300ends.

FIG. 4is a flow diagram of a method400for processing an error message associated with a data path according to one or more embodiments. The method400starts at step402and proceeds to step404, at which a plurality of data blocks are routed to a destination (e.g., the destination128ofFIG. 1). As described herein, the plurality of data blocks form a data stream (e.g., the data stream117ofFIG. 1)

At step406, an error message associated with a data path is received. At step408, a determination is made as to whether the error message indicates that a transmission error occurred while routing the data blocks. If it is determined that the error message indicates a transmission error (option “YES”), then the method400proceeds to step410. If, at step408it is determined that the error message does not indicate a transmission error (option “NO”), then the method400proceeds to step416. At step410, one or more lost data blocks are identified. In one embodiment, an agent (e.g., the agent116ofFIG. 1) identifies the lost data blocks. At step412, another data path is selected. At step414, the lost data blocks are communicated on the another data path. The method400proceeds to step416, at which the method400ends.