Data identification system

Disclosed is a method of operating a data storage system. The method comprises identifying changed segments of a primary storage volume, receiving a data request for a plurality of data items in a secondary storage volume, identifying changed data items of the plurality of data items in the secondary storage volume based on a correspondence between the plurality of data items in the secondary storage volume and the changed segments of the primary storage volume, and transferring the changed data items in response to the data request.

TECHNICAL BACKGROUND

In the field of computer hardware and software technology, a virtual machine is a software implementation of a machine (computer) that executes program instructions like a real machine. Virtual machine technology allows for the sharing of, between multiple virtual machines, the physical resources underlying the virtual machines.

In virtual machine environments, storage volumes within the virtual machines contain data items that need to be accessed and scanned. Unfortunately, accessing the underlying contents of a storage volume can be very resource intensive, reducing the performance of a virtual machine and other operations within a virtual machine environment.

OVERVIEW

Disclosed are data identification systems and methods of operating data identification systems. In an embodiment, a method comprises identifying changed segments of a primary storage volume, receiving a data request for a plurality of data items in a secondary storage volume, identifying changed data items of the plurality of data items in the secondary storage volume based on a correspondence between the plurality of data items in the secondary storage volume and the changed segments of the primary storage volume, and transferring the changed data items in response to the data request.

In an embodiment, identifying the changed segments of the primary storage volume further comprises generating a request for a changed segment list, transferring the request for the changed segment list, receiving the changed segment list, and processing the changed segment list to determine the changed segments of the primary storage volume.

In an embodiment, the data request comprises a scan request to scan the plurality of data items.

In an embodiment, receiving the changed data items and responsively scanning the changed data items for a virus.

In an embodiment, transferring unchanged data items in response to detecting a virus in one of the changed data items, wherein the unchanged data items represent all data items except the changed data items.

In an embodiment, the primary storage volume comprises a virtual disk file.

In an embodiment, the secondary storage volume comprises a virtual storage device, and wherein the virtual disk file contains the first virtual storage device.

In an embodiment, transferring a request for the changed data items to the virtual storage device and receiving the changed data items in response to the request.

In an embodiment, the data request further comprises a request for a plurality of data items in a second secondary storage volume.

In an embodiment, identifying second changed blocks of a second primary storage volume, identifying second changed data items of the plurality of data items in the second secondary storage volume based on a correspondence between the plurality of data items in the second secondary storage volume and the second changed blocks of the second primary storage volume, and transferring the second changed data items in response to the data request.

DETAILED DESCRIPTION

In virtual machine environments, accessing the underlying contents of a storage volume can be very resource intensive, reducing the performance of a virtual machine and other operations within a virtual machine environment. Nonetheless, individual data items on a storage volume need to be accessed.

For example, when scanning data files for viruses, anti-virus software must walk the contents of a file system and scan files looking for infections. However, as with many processes, the scan process scales with the total number of files that need to be scanned. Thus, if the number of files that need scanning can be reduced, then the time to do the scan process can be reduced.

One way in which the number of data items on which a process needs to operate can be reduced is by operating only on those data items that have changed. Several software processes use this method.

An improved technique involves identifying changed data items in a storage volume based on the correspondence between the data items and the changed segments.

In an example, a guest operating system may be executed within a virtual machine that contains a virtual disk drive. The guest operating system may further include a file system for storage of the volume data. In this case, data items may be requested by a user via a data item request. It should be understood that the user may be a human operator, as well as other software applications, hardware elements, or the like.

In this case, changed segments of the file system are first determined. Data items in the virtual disk drive that have changed are then identified based on the correspondence between the changed data items and the changed segments of the file system.

Referring now toFIG. 1, data identification system100is illustrated in an embodiment whereby a data identification (DI) module102is implemented in the data identification system in order to identify data items in a secondary storage volume. Data identification system100includes processing system101, DI module102, secondary storage volume103, and primary storage volume105.

Processing system101comprises any system or collection of systems capable of executing DI module102to identify data items in secondary storage volume103. Processing system101may be a micro-processor, an application specific integrated circuit, a general purpose computer, a server computer, or any combination or variation thereof. Data identification module102may be program instructions executable by processing system101.

Primary and secondary storage volumes105and103, respectively, may be any storage volumes capable of storing a volume of data. Primary storage volume105comprises segments106. Secondary storage volume comprises data items104. Data items104comprise the volume of data in storage volume103. Segments106comprise sections of a data volume in storage volume105.

In an example, storage volume103may be a virtual storage volume. In this case, data items104may be the virtual storage contents of a data volume in storage volume103. The data volume may itself be stored within another storage volume, such as storage volume105. Segments106may then comprise sections of the data volume in storage volume105. In this case, processing system101and/or primary storage volume105may additionally track segments106of the data volume in storage volume105that have changed.

FIG. 2illustrates process200describing the operation of data identification system100. To begin, a volume of data is generated and stored. Processing system101identifies changed segments of segments106on storage volume105(Step202) and receives a data item request for files in storage volume103(Step203). A data item may be, for example, a file. Processing system101, executing DI module102, then identifies changed data items in storage volume103based on the correspondence between the data items104and the changed segments106(Step204).

Lastly, processing system101transfers the changed data items (Step205). Those skilled in the art will appreciate that data item and file transfers, as referred to herein, are typically not literally transfers. Rather, a version of the data item or file may be transferred or copied. However, in some embodiments, the files may literally be transferred.

Those skilled in art will also appreciate that data item requests may be used for a variety of applications. In fact, a data item request may be used by any user that benefits from identifying only the changed data items. For example, a data item request can be used to identify changed data items for scanning. Similarly, a data item request may be used to identify changed data items for computer or data forensics, compliance needs, or in order to log system changes. It should be understood that the user may be a human operator, as well as other software applications, hardware elements, or the like.

Communication interface311comprises a network card, network interface, port, or interface circuitry that allows data storage system300to communicate with a storage volume. Communication interface311may also include a memory device, software, processing circuitry, or some other communication device. Communication interface311may use various protocols, such as host bus adapters (HBA), SCSI, SATA, Fibre Channel, iSCI, WiFi, Ethernet, TCP/IP, or the like to communicate with a storage volume.

User interface312comprises components that interact with a user to receive user inputs and to present media and/or information. User interface312may include a speaker, microphone, buttons, lights, display screen, mouse, keyboard, or some other user input/output apparatus—including combinations thereof. User interface312may be omitted in some examples.

Processing system313may comprise a microprocessor and other circuitry that retrieves and executes software315, including DI module302, from storage system314. Storage system314comprises a disk drive, flash drive, data storage circuitry, or some other memory apparatus. Storage system314may further comprise a primary and secondary storage volume. The primary storage volume may comprise segments or portions. The secondary storage volume may comprise data items. Processing system313is typically mounted on a circuit board that may also hold storage system314and portions of communication interface311and user interface314.

Software315comprises computer programs, firmware, or some other form of machine-readable processing instructions. Software315may include an operating system, utilities, drivers, network interfaces, applications, virtual machines, or some other type of software. When executed by processing system313, software315directs processing system313to operate data storage system300in order to identify changed data items in the secondary storage volume.

In operation, a data item request for data items stored on the secondary storage volume may be received by data identification system300via communication interface311or user interface312. Processing system313executes software315to processes the request by first identifying changed segments of the primary storage volume. Software315then identifies changed data items in the secondary storage volume based on the correspondence between the data items in the secondary storage volume and the changed segments of the primary storage volume. Lastly, software315transfers the identified changed data items in response to the request.

FIG. 4illustrates data identification system400in another embodiment whereby DI module402is implemented in order to identify files404in a storage volume403and scan module407is implemented to incrementally scan files404for viruses. Data identification system400includes processing system401, DI module402, scan module407, storage volume403, and underlying storage volume405. As shown, scan module407is separate from DI module402. However, those skilled in the art will appreciate that scan module407may be contained within DI module402.

Processing system401comprises any system or collection of systems capable of executing DI module402and scan module407to identify files in storage volume403and scan the files. Processing system401may be a micro-processor, an application specific integrated circuit, a general purpose computer, a server computer, or any combination or variation thereof. DI module402and scan module407may be program instructions executable by processing system401.

Storage volume403may be a virtual storage volume. Storage volume403comprises files404. Files404(shown as files X, Y, and Z) comprise the virtual storage contents of a data volume in storage volume403. Files404comprise the volume of data in storage volume403. Underlying storage volume405comprises blocks406. Blocks406(shown as A, B, C, and D) comprise blocks or sections of the data volume in underlying storage volume405.

Processing system401and/or underlying storage volume105may track blocks406of the data volume in storage volume405that have changed. In operation, processing system401, executing DI module402, uses the changed blocks in order to determine files that have changed on a virtual storage volume.

In this example, files404are requested by a user via scan request. Alternatively and/or additionally, requests for other applications (e.g., such as data forensic applications) may be made. Again, it should be understood that the user may be a human operator, as well as other software applications, hardware elements, or the like.

FIG. 5illustrates a sequence diagram that illustrates operation of data identification system400according to an embodiment for scanning data files. In this example, data identification system400receives a scan request to scan files X, Y, and Z on storage volume403.

Referring toFIG. 5, DI module402receives a scan request to scan files X, Y, and Z on storage volume403. In order to reduce the number of files that need to be accessed from storage volume403, DI module attempts to determine which of the files404have changed. To this end DI module402transfers a changed block list request to storage volume405. In response, storage volume405transfers the changed block list to DI module402. As shown, the changed block list is maintained by storage volume405. Those skilled in the art will appreciate that the changed block list may be maintained processing system401or another element—or combination of elements—within data identification system400.

Once DI module402has obtained the changed block list, DI module402then identifies changed files in the virtual machine storage volume403. For example,FIG. 6shows changed block list614. Changed block list614indicates that blocks B and D have been changed. Corresponding the blocks606to files604indicates that block A corresponds to file X, block B corresponds to file Y (which also corresponds to block C), and block D corresponds to file Z. Changed block list614can be used to reliably determine when a file's content has changed. Based on the changed blocks B and D, DI module402can identify the files that have changed. In this case, files Y and Z have changed. It is clear fromFIG. 6that some files (i.e., file Y) may occupy multiple blocks.

After the changed files have been identified, DI module402then transfers a request to storage volume403for the changed files. Those skilled in the art will appreciate that the request may be directed to the guest operating system for the virtual machine. In response, storage volume403returns the changed filed to DI module402. DI module402responsively transfers the changed files to scan module407which scans the changed files and transmits a response to DI module402. Those skilled in the art will appreciate that other process or operations—in lieu of or in addition to scanning—may be performed on the changed files. For example, DI module402may be used to access changed data files to be used in conjunction with data forensics. Lastly, DI module402then transfers a scan response which may indicate whether a virus was detected and which file or files are affected.

FIG. 7illustrates the operation of data identification system400when executing DI module402for incrementally scanning a virtual storage volume on a virtual machine according to another embodiment. In this example, data identification system400receives a scan request to scan files on storage volume403. Data identification system400responds with a scan response indicating whether or not a virus is detected.

In this case, DI module402first determines whether scan module407contains a virus signature file that is up to date (Step703). Processing system401and/or scan module407may persistently store a unique identifier for the version of the virus software's signature file. Typically, the signature file is used to determine whether the virus software's signature has changed (i.e., whether there is an updated version of the software available). If a full scan has never been run on the target file system or the signature data has been updated since the last full scan then all files are requested for scanning (Step713). In some embodiments, requests for files (i.e., Step713, Step711, and Step710) can be omitted.

Alternatively, if the signature file is up-to-date then DI module402retrieves a changed block list from a virtual machine (Step705). The changed block list may, for example, be retrieved from a guest O/S, a hypervisor, virtual hardware, or accessed via a V-disk file. Typically, the changed block list is generated by the hypervisor, the guest O/S, or the storage volume (i.e., underlying storage volume405); however, other. Once the changed block list is obtained, DI module402identifies changed blocks on the virtual storage disk (Step707) and the changed files (Step709) that correspond to the changed blocks.

DI module402then requests the changed files from storage volume403for scanning (Step711). Again, in some embodiments, requests for files may be omitted. UsingFIG. 6as an example, DI module402would request changed files Y and Z corresponding to changed blocks B and D, respectively.

Once requested files have been received, DI module402transfers the requested files to scan module407. Those skilled in the art will appreciate that typically a file is not literally copied or transferred. Rather, a version can be transferred or copied. However, in some embodiments, the files may literally be transferred. Scan module407scans the files for viruses (Step715). Those skilled in the art will appreciate that scan module407and DI module402may be a single software module. If no viruses are found, DI module402transfers a scan response indicating successful completion of scan with no detected virus (Step721).

Alternatively, if a virus is found, DI module402determines whether the remaining files (those files on the virtual storage disk that have not changed) have been scanned (Step717). Again, usingFIG. 6as an example, if either changed file Y or Z, or both, is found to have a virus, then the remaining files (i.e., file X) is requested for scanning.

If the remaining files have not been scanned then those files are requested (Step719) and scanned (Step715). DI module402then transfers a scan response indicating that the scan was completed successfully and indicates the virus(es) that were detected along with the affected files.

FIG. 8illustrates data storage system800in another embodiment. In this environment, data storage system800includes processing system801, storage system803, hypervisor805, and DI module802. As shown, DI module802is stored on hypervisor805. However, those skilled in the art will appreciate that DI module802may be stored on storage system803or anywhere within hypervisor805. Processing system801executes software including DI module802to identify data items on the virtual hardware.

Hypervisor805includes virtual machine809. Virtual machine809includes guest application811, guest operating system813, virtual hardware815, and virtual disk file819. As shown, hypervisor805includes block change list814and virtual disk file819includes block mapping table820. Virtual hardware815includes virtual storage volume816, virtual processor817, and virtual peripheral818.

As shown, guest hypervisor805generates block change list814. Those skilled in the art will appreciate that block change list814may be generated by an entity within virtual machine809(i.e., guest operating system813), processing system801, or storage system803. In particular, changed block list814may be generated by replication software, continuous data protection (CDP) software, or virtual disk change block tracking software running on virtual machine809, hypervisor805, or processing system801.

Block change list814describes the blocks that have changed on virtual disk file819. Block mapping table820describes the storage of the data volume in virtual disk file819. For example, block mapping table820may describe the correspondence between files on storage volume816and underlying virtual disk file819.

In operation, DI module802is executed by processing system801to identify changed data items identified by virtual storage volume816stored in virtual disk file819, using block change list814and block mapping table820.

To begin, DI module802retrieves block change list814and, identifies changed blocks. The underlying data items contents can then be determined by, for example, using the block mapping table820to determine which data items correspond to the changed blocks. For example, block change list814may indicate that block B has changed and block mapping table820may indicate that block B corresponds to file Y.

Once the changed data items are determined, those changed data items can then be requested from virtual machine809. For example, once DI module802determines that file Y changed, DI module802can then request file Y from storage volume816. DI module802then transfers the changed file for further processing by a user requesting the changed file or for other use. Again, it should be understood that the user may be a human operator, as well as other software applications, hardware elements, or the like.

FIG. 9illustrates data storage system900in another embodiment. Data storage system900is similar to data storage system800, shown inFIG. 8, except that DI module902and scan module907are located within guest operating system913. In this embodiment, DI module902comprises program instructions executable within an operating system environment, such as guest operating system913. Scan module907comprises program instructions executable on guest operating system913.

In operation, DI module902is executed by processing system901within guest operating system913to identify changed data items on the data volume stored in virtual disk file919, using block change list914and block mapping table920. Scan module907is executed by processing system901and/or processor917to scan the changed data items.

DI module902retrieves block change list914and, identifies changed blocks. In this example, block B has changed. The underlying data items contents can then be determined by, for example, using the block mapping table920to determine which data items correspond to the changed blocks. In this example, data item Y corresponds to changed block B.

Once the changed data items are determined, those changed data items can be requested by DI module902and transferred to scan module907for scanning.

In other embodiments, DI module902may simply transfer a list of the changed data items. In this case, scan module907can request the changed data items from storage volume916and scan the data items. Again, those skilled in the art will appreciate that the files may not be literally transferred. Rather, versions of the files can be copied. Once scanned, scan module907can communicate with DI module902and transfer a scan response.

FIG. 10illustrates data storage system1000in another embodiment. Data storage system1000is similar to data storage system800and900, shown inFIGS. 8 and 9, except that multiple virtual machines are shown, DI module1002is within hypervisor1005, and scan module1007is within DI module1002. In this embodiment, DI module1002comprises program instructions executable within hypervisor1005for identifying changed data items on storage volume1016and storage volume1036.

Those skilled in the art will appreciate that scan module1007and DI module1002can reside in different locations anywhere on virtual machine1009and/or virtual machine1029. For example, DI module1002may be located within virtual machine1009and executed to determine changed data items on virtual machine1029. Similarly, scan module1007may be located on virtual machine1009and executed to scan changed data items from virtual machine1029.

In operation, DI module1002is executed by processing system1001to identify changed data items on the data volume stored in virtual disk file1019and changed data items on the data volume stored in virtual disk file1039. Changed data items on the data volume stored in virtual disk file1019are determined using block change list1014and block mapping table1020. Changed data items on the data volume stored in virtual disk file1039are determined using block change list1034and block mapping table1040. Scan module1007is executed by processing system1001to scan the changed data items.

DI module1002operates similarly to the DI module902and802. However, in some embodiments, multiple instances of DI module1002(not shown for simplicity) may be executed in parallel by processing system1001to identify and scan changed data items on the data volume stored in virtual disk file1019and the data volume stored in virtual disk file1039.