Deduplicated storage with multiple storage domains

A method, apparatus, system, and computer program product for deduplicating data. Data is stored by a computer system in storage domains in a storage system. Deduplication is performed by the computer system within each storage domain in the storage domains using a hash table that includes entries for all of the storage domains. Cross-deduplication between the storage domains is avoided in the storage system.

BACKGROUND

The disclosure relates generally to an improved computer system and more specifically to a method, apparatus, system, and computer program product for deduplicating data in a storage system with multiple storage domains.

2. Description of the Related Art

Reducing the size of data in storage systems is desirable with the large amounts of data that are present in computer systems. Data deduplication is a data compression technique used to reduce storage overhead. Data deduplication eliminates duplicate data to obtain a smaller footprint of data in a storage system. The storage system can be a disk drive, a backup server, a group of backup servers, a storage area network (SAN), or some other type of storage system.

With deduplication, the process eliminates redundant copies of data to reduce the amount of storage space used. The deduplication process removes copies of data such that one copy of the data is retained in the storage system. Redundant data is replaced with a pointer to the remaining copy of the data.

For example, in an email system, multiple copies of the same attachment may be present. A hundred copies of an attachment, such as a presentation file that is 2 megabytes in size, can be stored in the email system. When a backup is performed, archiving all of these copies of the presentation file uses 200 megabytes of storage for this attachment. With deduplication, the amount of storage space needed drops to 2 megabytes for this particular attachment.

SUMMARY

According to one embodiment of the present invention, a method deduplicates data. Data is stored by a computer system in storage domains in a storage system. Deduplication is performed by the computer system within each storage domain in the storage domains using a hash table that includes entries for all of the storage domains. Cross-deduplication between the storage domains is avoided in the storage system.

According to another embodiment of the present invention, a data deduplication system comprises a computer system. The computer system stores data in storage domains in a storage system and performs deduplication within each storage domain in the storage domains using a hash table that includes entries for all of the storage domains, wherein cross-deduplication between the storage domains is avoided in the storage system.

According to yet another embodiment of the present invention, a computer program product for deduplicating data comprises a computer-readable-storage media, first program code, and second program code stored on the computer-readable storage media. The first program code is run to store the data in storage domains in a storage system. The second program code is run to perform deduplication within each storage domain in the storage domains using a hash table that includes entries for all of the storage domains, wherein cross-deduplication between the storage domains is avoided in the storage system.

DETAILED DESCRIPTION

The illustrative embodiments recognize and take into account one or more different considerations. For example, the illustrative embodiments recognize and take into account that the manner in which some organizations or users store data in storage systems can reduce the feasibility of deduplication. The illustrative embodiments recognize and take into account that organizations or users may place primary data such as production data, development data, and test data on the same storage system as backup data. The illustrative embodiments also recognize and take into account that currently deduplication is a global process for a storage system and when formed on the storage system, only one copy of a piece of data is present. As result, a backup copy of the piece of data is absent. As a result, the illustrative embodiments recognize and take into account that deduplication of the stored system can result in a loss of redundancy in data. For example, deduplication can remove data from a backup of the primary data. As a result, a corruption of the primary data results in an inability to restore the primary data from the backup.

Thus, the illustrative embodiments provide a method, an apparatus, a computer system, and a computer program product for deduplicating data in which actively used data is grouped into one storage domain and the backup of data is grouped into another storage domain. A storage domain is a grouping of data from which deduplication can be performed. For example, one grouping of the data can be servers that store data for used in production, development, testing, or some combination thereof. Another grouping of the data can be servers that are used to backup data.

In one illustrative example, data is stored by a computer system in storage domains in a storage system. Deduplication is performed by the computer system within each storage domain in the storage domains using a hash table that includes entries for all of the storage domains. Cross-deduplication between the storage domains is avoided in the storage system. In other words, a copy of a piece of data stored in one storage domain is not removed because the piece of data is also stored in another storage domain in the storage system.

With reference now to the figures and, in particular, with reference toFIG. 1, a pictorial representation of a network of data processing systems is depicted in which illustrative embodiments can be implemented. Network data processing system100is a network of computers in which the illustrative embodiments may be implemented. Network data processing system100contains network102, which is the medium used to provide communications links between various devices and computers connected together within network data processing system100. Network102may include connections, such as wire, wireless communication links, or fiber optic cables.

In the depicted example, server computer104and server computer106connect to network102along with storage system108. In addition, client devices110connect to network102. As depicted, client devices110include client computer112, client computer114, and client computer116. Client devices110can be, for example, computers, workstations, or network computers. In the depicted example, server computer104provides information, such as boot files, operating system images, and applications to client devices110. Further, client devices110can also include other types of client devices such as mobile phone118, tablet computer120, and smart glasses122. In this illustrative example, server computer104, server computer106, storage system108, and client devices110are network devices that connect to network102in which network102is the communications media for these network devices. Some or all of client devices110may form an Internet of things (IoT) in which these physical devices can connect to network102and exchange information with each other over network102.

Client devices110are clients to server computer104in this example. Network data processing system100may include additional server computers, client computers, and other devices not shown. Client devices110connect to network102utilizing at least one of wired, optical fiber, or wireless connections.

In this illustrative example, storage system108is connected to network102. Storage system108can be selected from at least one of a network attached storage, a storage area network,

Program code located in network data processing system100can be stored on a computer-recordable storage medium and downloaded to a data processing system or other device for use. For example, program code can be stored on a computer-recordable storage medium on server computer104and downloaded to client devices110over network102for use on client devices110.

In this illustrative example, storage system108can store data that has multiple types of roles. For example, storage system108can store primary data130and backup data132. Primary data130can include at least one of production data, development data, or testing data. This data can include at least one of program code, configuration files, data used in day to day tasks, data used to perform tests, spreadsheets, computer aided design files, images, video, audio, or other types of data. Backup data132can be copies some or all of primary data130. Backup data132can be used to restore primary data130in case of corruption of primary data130.

As depicted in this example, deduplication controller134is configured to control deduplication of primary data130and backup data132in storage system108. In this example, primary data130and backup data132are separated into two storage domains in storage system108. Primary data130is located in primary storage domain136and backup data132is located in backup storage domain138.

Hash table140contains hashes142for data in both primary data130in primary storage domain136and backup data132in backup storage domain138. In other words, hashes are present in hash table140for primary data130and backup data132.

In this illustrative example, deduplication controller134performs deduplication of data in primary storage domain136and backup storage domain138separately. In other words, deduplication is segregated by storage domains in storage system108. In this illustrative example, the storage domains store the data based on the roles of the data, primary storage domain136for primary data130which is data that is actively used in tasks and operations performed by a user and backup storage domain138for backup data132that is maintained for recovery of primary data130in case of corruption of primary data130.

When new data144is received for storage in storage system108, deduplication controller134generates new hash146and compares new hash146to hashes142to determine whether existing hash148is present in hashes142that matches new hash146. If a match is found, that means that new data144is the same as existing data150that corresponds to existing hash148.

When a match is present, deduplication controller134determines whether new data144is in the same storage domain as existing data150. For example, if existing data150is in primary storage domain136and new data144is designated for backup storage domain138, new data144is a backup of existing data150and is stored in backup storage domain138. Deduplication controller134adds new hash146to hash table140to reflect that new data144is now present in backup storage domain138.

On the other hand, if new data144is designated for storage in primary storage domain136, new data144is a duplicate of existing data150and is not stored in primary storage domain136. In this case, deduplication controller134updates hash table140to increment a reference count to reflect new data144in primary storage domain136.

As depicted, hash table140with hashes142for primary data130and backup data132links the two storage domains, primary storage domain136and backup storage domain138. The use of a single hash table for multiple storage domains reduces the amount of storage resources needed and reduces complexity. Further, hash table140with hashes142for both primary storage domain136and backup storage domain138can be used by deduplication controller134to perform actions used for at least one of data validation or repair within the storage system108.

The illustration of deduplication of data in network data processing system100is intended as one illustrative example and not meant to limit the manner in which other illustrative examples can be implemented. For example, storage system108is shown as attached to network102. In other illustrative examples, storage system108can be a disk drive or solid-state drive in server computer104. In other illustrative examples, storage system108can be storage devices in at least one a group of server computers or group of client devices that are configured as a primary domain and a backup domain.

With reference now toFIG. 2, a block diagram of a deduplication environment is depicted in accordance with an illustrative embodiment. In this illustrative example, deduplication environment200includes components that can be implemented in hardware such as the hardware shown in network data processing system100inFIG. 1.

As depicted, deduplication system202operates in deduplication environment200to manage data204stored in storage system206. Storage system206is a physical hardware system and can take a number of different forms. For example, storage system206can include at least one of a disk drive, a solid state drive, a network attached storage (NAS), a storage area network (SAN), or other types of storage devices.

In this illustrative example, storage system206can be divided or partitioned into storage domains208. The different storage domains can be based on physical hardware or logical constructs. When disk drives are present, computers in which the disk drives are located can be considered part of storage system206. For example, a set of servers with hard disks can be a storage domain in storage domains208. As another example, a logical unit number for a logical unit in a storage area network (SAN) can be a storage domain in storage domains208.

As depicted, deduplication system202comprises computer system210and deduplication controller212. As depicted, deduplication controller212is located in computer system210. Computer system210is a physical hardware system and includes one or more data processing systems. When more than one data processing system is present in computer system210, those data processing systems are in communication with each other using a communications medium. The communications medium can be a network. The data processing systems can be selected from at least one of a computer, a server computer, a tablet computer, or some other suitable data processing system.

In this illustrative example, deduplication controller212can control the creation of storage domains208in storage system206. A storage domain is a grouping of data204for deduplication. For example, deduplication controller212can form to storage domains, such as primary storage domain214and backup storage domain216.

Primary storage domain214can be used to store primary data218in data204. In this illustrative example, primary data218is active data that is used for at least one of production, development, testing, or other purposes. For example, primary storage domain214can be at least one of a production storage domain, a development storage domain, a test storage domain, or some other suitable type of storage domain. Backup storage domain216stores backup data220in data204. Backup data220is used to restore primary data218that may be corrupted, accidentally deleted, or otherwise unusable or unavailable.

As depicted, deduplication controller212performs deduplication on each storage domain in storage domains208separately from other storage domains in storage domains208. In other words, deduplication is not performed across two or more storage domains.

In this illustrative example, deduplication is performed using hash table222, which contains hashes224for storage domains208. As depicted, hashes224are stored in entries226in hash table222. As depicted, hash table222links the different storage domains in storage domains208.

As a result, the deduplication performed by deduplication controller212in deduplication system202avoids cross-deduplication between storage domains208in storage system206. In other words, duplicate copies of data204between different storage domains are not removed and replaced with a pointer or reference to removed data.

Thus, the different roles played by different storage domains in storage domains208can remain unaffected when an actual copy of data in a particular storage domain is needed. For example, copy238of piece of data240in backup data220in backup storage domain216and piece of data240in primary data218in primary storage domain214are not affected by deduplication performed individually on storage domains208. Both copy238of piece of data240and piece of data240remain in storage domains208.

Consequently, the role of copy238of piece of data240in backup storage domain216can be fulfilled if piece of240in primary storage domain214becomes corrupted, missing, or otherwise unusable. In that situation, piece of data240in primary storage domain214can be recovered using copy238of piece of data240in backup storage domain216.

When new data228is received for storage, deduplication controller212determines whether a match is present between new hash230for new data228received for storage and existing hash232for existing data234in hashes224in hash table222. Responsive to the match being present, deduplication controller212determines whether the match is for a same storage domain in storage domains208as new data228. In other words, deduplication controller212identifies the storage domain in storage domains208for existing data234and determines whether new data228is designated for the same storage domain. The identification of the storage domain for new data228can be determined based on a request received to store new data228. The source originating the request or metadata in the request can be used to identify the storage domain.

Responsive to the match being present for a different storage domain in storage domains208, deduplication controller212stores new data228in a particular storage domain in storage domains208designated for new data228. Further, responsive to the match being for the different storage domain in storage domains208, deduplication controller212updates hash table222to reflect that new data228exists the particular storage domain in storage domains208in addition to existing data234in another storage domain in storage domains208in storage system206.

Responsive to the match being for the same storage domain, deduplication controller212updates hash table222to increment a reference count to reflect new data228being present in the another storage domain in the storage domains208. In this instance, new data228is copy238of existing data234with both being in the same storage domain. As result, only one copy is saved in the storage domain as part of the deduplication process performed by deduplication controller212.

In this illustrative example, the identification of common data236between storage domains208can be performed using hash table222. As depicted, common data236can be used to perform at least one of data validation, repair of data, or other actions in primary storage domain214in the storage system206.

For example, common data236can be located in both primary storage domain214and backup storage domain216. In this example, common data236can be used to replace at least one of corrupted or deleted data in data204in primary storage domain214.

For example, when corruption is present, deduplication controller212determines whether copy238of piece of data240in primary storage domain214is present in backup storage domain216in response to piece of data240in primary storage domain214being corrupt. In this illustrative example, copy238of piece of data240and piece of data240are examples of common data236. Deduplication controller212replaces piece of data240in primary storage domain214with copy238of piece of data240in response to copy238of piece of data240being present in backup storage domain216.

In replacing piece of data240, deduplication controller212determines whether copy238of piece of data240in backup storage domain216is corrupt in response to copy238of piece of data240being present in backup storage domain216. Further, deduplication controller212replaces piece of data240in primary storage domain214with copy238of piece of data240in response to copy238of piece of data240not being corrupt.

With reference next toFIG. 3, a block diagram of an entry is depicted in accordance with an illustrative embodiment. Entry300is an example of one implementation for an entry in entries226in hash table222inFIG. 2. Entry300is used to represent piece of data302in a storage system. Piece of data302is an example of piece of data240inFIG. 2or other data in data204inFIG. 2.

As depicted, entry300includes hash304, storage domain306, location308, and reference count310. In this illustrative example, entry300is present for each piece of data in storage domain. If same piece of data is present in two different storage domains, two entries are used to represent those two pieces of data.

Hash304is a value generated from piece of data302. Hash304can be generated using any function that can map data of arbitrary size to data of a fixed size.

Storage domain306identifies the storage domain in which piece of data302is located. Location308identifies where piece of data302can be found in storage domain306. For example, location308can be a pointer, a logical unit number (LUN), a volume identifier and Internet protocol address, or some other suitable type of location information. Reference count310identifies how many copies of piece of data302are present in storage domain306.

In one illustrative example, one or more technical solutions are present that overcome a technical problem with global deduplication being performed in a storage system. As a result, one or more technical solutions can provide a technical effect dividing a storage system into storage domains in which each storage domain deduplicated separately from other storage domains. Further, one or more technical solutions include a hash table that contains information for all of the data in the different storage domains. This common hash table enables reducing the amount of storage and processing resources used to perform deduplication and manage the storage domains in the storage system.

Computer system210can be configured to perform at least one of the steps, operations, or actions described in the different illustrative examples using software, hardware, firmware or a combination thereof. As a result, computer system210operates as a special purpose computer system in which deduplication controller212in computer system210enables deduplicating of data204across storage domains208such that deduplications performed within storage domains208and not across storage domains208in storage system206in a manner that maintains the role of data204in particular storage domains. In particular, deduplication controller212transforms computer system210into a special purpose computer system as compared to currently available general computer systems that do not have deduplication controller212.

Turning next toFIG. 4, a flowchart of a process for deduplicating data is depicted in accordance with an illustrative embodiment. The process inFIG. 4can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program code that is run by one of more processor units located in one or more hardware devices in one or more computer systems. For example, the process can be implemented in deduplication controller212in computer system210inFIG. 2.

The process begins by storing data in storage domains in a storage system (step400). The process performs deduplication within each storage domain in the storage domains using a hash table that includes entries for all the storage domains (step402). The process terminates thereafter.

With the process inFIG. 4, cross-deduplication between the storage domains is avoided in the storage system. In other words, copies of the same piece of data in different storage domains are not compressed through the deduplication process. The deduplication process results in a copy of the data being the present in each storage domain in the storage domains in which the copy of the data is stored within the storage system. In this manner, the function of a particular piece of data in a particular storage domain can be maintained with the presence of particular piece of data in the particular storage domain.

Turning toFIG. 5, a flowchart of a process for storing new data in storage domains is depicted in accordance with an illustrative embodiment. The process inFIG. 5can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program code that is run by one of more processor units located in one or more hardware devices in one or more computer systems. For example, the process can be implemented in deduplication controller212in computer system210inFIG. 2.

The process begins by receiving new data for storage in the storage system (step500). The process generates a new hash from the new data (step502).

The process determines whether a match is present between the new hash for the new data received for storage and an existing hash for existing data in the hash table (step504). Responsive to the match being present, the process determines, whether the match is for a same storage domain in the storage domains as the new data (step506). The storage domain designated for the new data can be determined from the request to store the new data. For example, the request can specify a particular storage domain. In another example, the source of the new data can be used to determine what particular storage domain is used to store the new data.

Responsive to the match being present for a different storage domain in the storage domains, the process stores the new data in a particular storage domain in the storage domains designated for the new data (step508). The process updates the hash table to reflect that the new data exists in the particular storage domain in the storage domains in addition to the existing data in another storage domain in the storage domains in the storage system (step510). The process terminates thereafter.

With reference again to step506, responsive to the match being for the same storage domain, the process updates the hash table to increment a reference count to reflect the new data in the another storage domain in the storage domains (step512). The process terminates thereafter. In this case, the new data is a duplicate of existing data in the same storage domain and is not stored.

With reference again to step504, if a match is not present between the new hash for new data received for storage and the existing hash for existing data in the hash table, the process proceeds to step508as described above.

With reference now toFIG. 6, another flowchart of a process for storing new data in a storage domain is depicted in accordance with an illustrative embodiment. When implemented in software, the process can take the form of program code that is run by one of more processor units located in one or more hardware devices in one or more computer systems. For example, the process can be implemented in deduplication controller212in computer system210inFIG. 2. This example describes receiving data from storage from a production process for a backup process.

The process begins by receiving a write request to write new data to a storage domain in a storage system (step600). A determination is made as to whether the new data originates from a backup process or a primary process (step602). For example, the data may be for a request to write data for a primary process such as a production process, a development process, or a testing process. As another example, the data may be for a request to write data for a backup process. This identification can be made from metadata in the write request. For example, a tag may be included in the metadata with the new data to indicate at least one of the source of the new data or the storage domain designated to store new data.

If the data is received from a primary process, a determination is made as to whether the new data is new to the primary storage domain (step604). The determination in step602can be made by creating a hash from the new data and comparing the hash to existing hashes for the primary storage domain.

If the data is new to the primary domain, the process stores the new data in the primary storage domain (step606). The process updates a hash table to include an entry identifying the new data in the primary storage domain (step608). The process terminates thereafter.

With reference again to step604, if the new data is not new to the primary storage domain, the process increments a reference count in an entry for the hash for the new data (step610).

With reference again to step602, if the data is received from a backup process, a determination is made as to whether the new data is new to the backup storage domain (step612). If the new data is new to the back of storage domain, the process stores the new data in the backup storage domain (step614). The process updates a hash table to include an entry identifying the new data in the backup storage domain (step616). The process terminates thereafter. With reference again to step612, if the new data is not new to the backup storage domain, the process proceeds to step610.

Turning toFIG. 7, a flowchart of a process for storing data using tags is depicted in accordance with an illustrative embodiment. The process inFIG. 7can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program code that is run by one of more processor units located in one or more hardware devices in one or more computer systems. For example, the process can be implemented in deduplication controller212in computer system210inFIG. 2. This process can be used to identify a storage domain for new data received for storage in a storage system with multiple storage domains. For example, tags can be used in write requests to identify the source originating the data for storage.

The process beings by receiving a write request to write new data to a storage domain (step700). A determination is made as to whether a tag in the write request indicates that the new data originates from a data source in the form of a backup process (step702). In this illustrative example, the metadata in the write request can include a tag that identifies the source of the new data. Further, the metadata includes a destination for the new data. This destination can be, for example, a volume or logical unit number is this illustrative example.

If the tag indicates that the new data originates from a backup process, a determination is made as to whether the destination is marked as being in a backup storage domain (step704). If the destination is marked as being in a backup storage domain, the process stores the new data if the new data is unique in the backup storage domain (step706). The process terminates thereafter.

With reference again to step704, if the destination is not marked as being in a backup storage domain, the process generates an alert (step708). The process terminates thereafter. In this case, a mismatch is present between the tag and the destination in the metadata.

Turning back to step702, if a tag is not present that indicates that the new data originates from a data source in the form of a backup process, the process determines whether the destination is in a primary domain (step710). If the destination is in the primary domain the process stores the new data if the new data is unique in the primary storage domain (step712). The process terminates thereafter. Otherwise, the process proceeds to step708. Step706and step712can be implemented using steps from the flowchart inFIG. 6.

With reference next toFIG. 8, a flowchart of a process for restoring data is depicted in accordance with an illustrative embodiment. The process inFIG. 8can be implemented in hardware, software, or both. When implemented in software, the process can take the form of program code that is run by one of more processor units located in one or more hardware devices in one or more computer systems. For example, the process can be implemented in deduplication controller212in computer system210inFIG. 2.

The process begins by determining whether a copy of a piece of data in the primary storage domain is present in the backup storage domain in response to the piece of data in the primary storage domain being corrupt (step800). The piece of data can be identified based on a segment identifier in this illustrative example. The segment identifier can be a range of physical locations such as a segment in a disk drive.

If a copy of the piece of data is present in the backup storage domain, the process determines whether the data is corrupt (step802). If the copy of the piece of data is not corrupt in the backup storage domain, the process replaces the piece of data in the primary storage domain with the copy of the piece of data (step804). The process terminates thereafter.

The replacement in step804is an actual physical replacement of data. In other words, the pointer to the copy of the piece of data is not used in the primary storage domain. Instead, the copy of the piece data is actually placed into the primary storage domain to replace the corrupted piece of data in the primary storage domain.

With reference back to step802, if the copy of the piece of data in the backup storage domain is corrupt, the process generates an indicator that the piece of data cannot be restored (step806). The process terminates thereafter. With again to step800, if the copy of the piece of data is not present in the backup storage domain, the process proceeds to step806as described above.

With reference toFIGS. 9-13, illustrations of pseudo code for writing new data to storage domains are depicted in accordance with an illustrative embodiment. In this illustrative example, pseudo code900can be implemented in deduplication controller212in computer system210inFIG. 2for a process that handles an incoming write request to write data to storage domains such as a primary storage domain in the form of a production storage domain and a backup storage domain.

As depicted, section902pseudo code900defines lists for tracking environmental constructs. In this example, environmental constructs are for control structures in the process.

In this illustrative example, section904in pseudo code900defines lookup functions for searching constructs. Section906defines functions for performing operations on data and tracking constructs.

Section908pseudo code900processes an incoming write request. Sub-section910in section908checks the source of the write request and the destination. The source can be, for example, a production host (PROD_HOST) or a backup host (BKP_HOST). The destination is a volume/logical unit number (LUN) in this example. Sub-section912in section908performs integrity checks and repairs as needed on data in the production domain. Pseudo code900in this section writes the data in a manner that avoids cross-deduplication.

Section914in pseudo code900writes data to the backup domain separately from production. The writing of data in section914avoids cross-deduplication and makes the associated metadata updates in the hash table for the storage domains.

Turning now toFIG. 14, a block diagram of a data processing system is depicted in accordance with an illustrative embodiment. Data processing system1400can be used to implement server computer104, server computer106, client devices110, inFIG. 1. Data processing system1400can also be used to implement computer system210inFIG. 2. In this illustrative example, data processing system1400includes communications framework1402, which provides communications between processor unit1404, memory1406, persistent storage1408, communications unit1410, input/output (I/O) unit1412, and display1414. In this example, communications framework1402takes the form of a bus system.

Processor unit1404serves to process instructions for software that can be loaded into memory1406. Processor unit1404includes one or more processors. For example, processor unit1404can be selected from at least one of a multicore processor, a central processing unit (CPU), a graphics processing unit (GPU), a physics processing unit (PPU), a digital signal processor (DSP), a network processor, or some other suitable type of processor.

Memory1406and persistent storage1408are examples of storage devices1416. A storage device is any piece of hardware that is capable of storing information, such as, for example, without limitation, at least one of data, program code in functional form, or other suitable information either on a temporary basis, a permanent basis, or both on a temporary basis and a permanent basis. Storage devices1416may also be referred to as computer-readable storage devices in these illustrative examples. Memory1406, in these examples, can be, for example, a random-access memory or any other suitable volatile or non-volatile storage device. Persistent storage1408may take various forms, depending on the particular implementation.

For example, persistent storage1408may contain one or more components or devices. For example, persistent storage1408can be a hard drive, a solid-state drive (SSD), a flash memory, a rewritable optical disk, a rewritable magnetic tape, or some combination of the above. The media used by persistent storage1408also can be removable. For example, a removable hard drive can be used for persistent storage1408.

Communications unit1410, in these illustrative examples, provides for communications with other data processing systems or devices. In these illustrative examples, communications unit1410is a network interface card.

Input/output unit1412allows for input and output of data with other devices that can be connected to data processing system1400. For example, input/output unit1412may provide a connection for user input through at least one of a keyboard, a mouse, or some other suitable input device. Further, input/output unit1412may send output to a printer. Display1414provides a mechanism to display information to a user.

Instructions for at least one of the operating system, applications, or programs can be located in storage devices1416, which are in communication with processor unit1404through communications framework1402. The processes of the different embodiments can be performed by processor unit1404using computer-implemented instructions, which may be located in a memory, such as memory1406.

These instructions are referred to as program code, computer usable program code, or computer-readable program code that can be read and processed by a processor in processor unit1404. The program code in the different embodiments can be embodied on different physical or computer-readable storage media, such as memory1406or persistent storage1408.

Program code1418is located in a functional form on computer-readable media1420that is selectively removable and can be loaded onto or transferred to data processing system1400for processing by processor unit1404. Program code1418and computer-readable media1420form computer program product1422in these illustrative examples. In the illustrative example, computer-readable media1420is computer-readable storage media1424.

In these illustrative examples, computer-readable storage media1424is a physical or tangible storage device used to store program code1418rather than a medium that propagates or transmits program code1418.

Alternatively, program code1418can be transferred to data processing system1400using a computer-readable signal media. The computer-readable signal media can be, for example, a propagated data signal containing program code1418. For example, the computer-readable signal media can be at least one of an electromagnetic signal, an optical signal, or any other suitable type of signal. These signals can be transmitted over connections, such as wireless connections, optical fiber cable, coaxial cable, a wire, or any other suitable type of connection.

The different components illustrated for data processing system1400are not meant to provide architectural limitations to the manner in which different embodiments can be implemented. In some illustrative examples, one or more of the components may be incorporated in or otherwise form a portion of, another component. For example, memory1406, or portions thereof, may be incorporated in processor unit1404in some illustrative examples. The different illustrative embodiments can be implemented in a data processing system including components in addition to or in place of those illustrated for data processing system1400. Other components shown inFIG. 14can be varied from the illustrative examples shown. The different embodiments can be implemented using any hardware device or system capable of running program code1418.

Thus, illustrative embodiments of the present invention provide a computer implemented method, an apparatus, a computer system, and a computer program product for deduplicating data. Data is stored by a computer system in storage domains in a storage system. Deduplication is performed by the computer system within each storage domain in the storage domains using a hash table that includes entries for all of the storage domains. Cross-deduplication between the storage domains is avoided in the storage system.

As a result, one or more illustrative examples are present that overcome a technical problem with global deduplication being performed in a storage system. As a result, one or more illustrative examples can provide a technical effect providing a store system into storage domains in which each storage domain is deduplicated separately from other storage domains. Further, one or more illustrative examples employ a hash table that contains information for all of the data in the different storage domains. This common hash table enables reducing the amount of storage and processing resources used to perform deduplication and enables managing the storage domains in the store system.