DEVICE FOR MANAGING DISTRIBUTED STORAGE RESOURCES AND METHOD FOR MANAGING SUCH STORAGE RESOURCES

A device for managing storage resources includes a plurality of servers with storage devices, a setting module, a first establishing module, and a second establishing module. The setting module includes a plurality of first storage devices in each server to be a virtual hard disk. When any other storage device of a server is damaged, the storage managing device maps the virtual hard disk with a new storage device and establishes a logical storage device, to perform data access operations on the logical storage device. A related method and a related non-transitory storage medium are also provided.

FIELD

The subject matter herein generally relates to data storage.

BACKGROUND

Mass-storage servers have evolved from a single mass-storage server to a distributed system which is composed of numerous, discrete, storage servers networked together. In order to maintain the high availability of the data or to avoid data loss through hard disk damage, a copy of the data is copied and stored on the hard disks of different servers. When these hard disks or servers are damaged, the number of backup copies will be reduced. When the distributed data storage system detects this situation, it will trigger the data backfill action.

DETAILED DESCRIPTION

FIG. 1illustrates a storage resource managing device100in accordance with an embodiment of the present disclosure. The storage resource managing device100is connected to a plurality of servers200via a communication network. The storage resource managing device100is configured to manage a plurality of storage devices of the servers200. In the embodiment, the storage resource managing device100is a management server.

The storage resource managing device100can include, but is not limited to, a processor10and a storage unit20. The storage resource managing device100can be connected to each server200via a plurality of wires or be connected to each server200via a wireless network, for example, a WI-FI, a wireless local area network, or the like.

In the embodiment, the storage unit20can be a read only memory (ROM) or a random access memory (RAM). The storage resource managing device100and the servers200can be arranged in an environment of a machine room.

FIGS. 2-4illustrate that the server200includes a plurality of first storage devices210and a plurality of second storage devices220. The first storage devices210and the second storage devices220are used to store data. In the embodiment, the first storage device210is a memory, and the second storage device220is a hard disk drive (HDD). The stored data is program code and/or software data. Therefore, the storage resource managing device100can connect the HDDs on the server to each other through a network to form a large-scale storage system, that is, the HDDs on the server are connected to each other to form distributed data access system400.

As shown inFIG. 2, the storage resource managing device100may include a setting module101, a first establishing module102, a second establishing module103, a detecting module104, a flash cache module105, and an adjusting module106. In the embodiment, the aforementioned modules can be a set of programmable software instructions stored in the storage unit20and executed by the processor10or a set of programmable software instructions or firmware in the processor10.

The setting module101is used to form the first storage devices210in each server200into a hard disk (not shown in the figure) providing emulation.

In the embodiment, each server has 10 HDDs as an example. The number of servers and their included HDDs can be adjusted according to actual needs.

For example, a storage server usually does not need much memory space. There are a total of 16 memory slots on the server200, and usually only four 32 GB memories (128 GB in total) are inserted to save hardware costs. The remaining 12 memory slots are also filled with 32 GB of memory (a total of 384 GB). The 384 GB of memory will be reserved for subsequent data backfill. Suppose there are 20 in number of a server200, and each server200has ten 10 TB hard drives for the storage system. Since the memory of each server200is full, each server200has an additional 384 GB of memory space. Therefore, the setting module101can build these memory spaces into a memory emulation hard disk (RAM Disk) with a storage capacity of 384 GB. Thus, these 20 servers200have a total of 20 of 384 GB emulation hard disks.

In the embodiment, the first establishing module102is used to map each emulation hard disk to establish a virtual hard disk230.

For example, the storage resource managing device100may use distributed storage tools to establish the 20 emulation hard disks into a distributed storage system. The first establishing module102further creates a virtual hard disk230with a storage capacity of 7680 GB from the system.

When any one of the second storage devices220of the server200is damaged, the second establishing module103is used to map the virtual hard disk230to a new second storage device220to establish a logical storage device300, so as to perform data access operations on the newly created logical storage device300. The newly created logical storage device300will use the virtual hard disk as a read-write cache space, and the newly created logical storage devices are used to replace HDD as the basic storage device of the distributed data access system, the logical storage device300can greatly improve the access speed.

In the embodiment, when the first storage devices210in each server are in an idle state, the first storage devices210in each server may be formed into an emulation hard disk or part of an emulation hard disk.

The emulation hard disk has a first storage capacity, the virtual hard disk has a second storage capacity, and the second storage capacity is greater than the first storage capacity.

In the embodiment, the second establishing module103preferably maps the virtual hard disk230with the new second storage device220through a flash cache module105to establish the logical storage device300. The flash cache module105may include a BCACHE or FLASHCACHE software package.

If a hard disk is damaged and replaced with a new hard disk, the data backfill can be performed by the first establishing module102and the second establishing module103.

The logical storage device300uses the virtual hard disk230as a cache device for the new hard disk, that is, the virtual hard disk230is a cache device310in the logical storage device300, and the new hard disk is the backing device320in the logical storage device300. When the logical storage device300is established, the adjusting module106adjusts the cache mode to the write back mode. When data is written to the logical storage device300, as long as the data is written to the cache device310, the write operation is completed.

The data backfilled by the remaining hard disks will start to be written to the cache device310of the logical storage device300, and the virtual hard disk will be virtualized by the memory space reserved by all servers. When all the data that needs to be backfilled into the new hard disk is written to the cache device310, the data backfill action will end.

After the backfill action ends, the adjusting module106converts the cache mode to a write around mode. New write requirements are directly written to the backing device320, the storage function of the cache is released from the logical storage device300, and only the original backing device is left to provide the storage service of the distributed data access system.

When the storage of the cache device310is released, the data stored in the cache device310will be flushed into the backing device320, and the action will be executed in the operating system of the server. After flushing, the backing device320can operate independently in the distributed storage system.

In the embodiment, all the memory space reserved by the server200is used as the cache space of the new replacement hard disk, the cache mode is set to the write back mode, and the backfill data from other hard disks can be first stored in the cache space virtualized by the memory. Because the data transmission of the memory is through electronic signals, not the same as the hard disk, it is generally limited by the speed of the physical hard disk rotation. Therefore, the cache space virtualized by the memory is at least 100 times faster than the hard disk.

For example, if a user is only relying on the IO performance of the new hard disk, it takes approximately 167 hours to backfill all the data to the new hard disk, and the data backfilling operation of the distributed storage system is ended. If the storage resource management method is used, the data can be written to the cache space virtualized by the memory in about 1.67 hours. At this time, the data backfilling action of the distributed storage system is ended. The remaining part of the cache device310to write data back to the backing device320is executed by the operating system of the server to which the new hard disk belongs, and the write speed of the new hard disk can reach 100 MB/s or more.

During the backfilling data process of the comparative embodiment 1 and the embodiment 1, obtaining and recording parameters are shown in Table 1

Table 1 shows, the embodiment 1 can make the required data backfill time 9 times faster than the data backfill method in the comparative embodiment 1.

The memory space (7680 GB) reserved by the 20 servers is just large enough to completely store 6 TB of data written by the backfill of the other 199 hard disks.

When the distributed storage system cluster is large enough, the remaining empty memory slots are largely vacant (each server has 12 empty slots), if all the slots can be filled with memory and used as a cache for data backfill, it can make the utilization rate of the server higher and the use of the computer room more efficient.

In the embodiment, if the first storage device210of the server200is damaged, the storage resource managing device100can repair the server according to the following operations.

First, the detecting module104detects the damaged first storage device210and confirms its position on the server200, and then replaces the damaged first storage device210with a new first storage device210. The detecting module104may include a memory test software package.

Next, the setting module101will use the new first storage device210and the undamaged first storage device210to recreate an emulation hard disk.

Further, the first establishing module102recreates a virtual hard disk and adds it back to the distributed data access system. The second establishing module103will use tools (such as BCACHE or FLASHCACHE) to recreate the logical storage device with the new hard disk to be backfilled, and finally add it back to the original distributed data access system to execute the data backfill.

In the embodiment, if the new hard disk is damaged, the storage resource managing device100can repair the server according to the following operations.

First, users can replace the damaged hard disk with a new hard disk. The detecting module104performs a smart control check on the new hard disk, to confirm that the new hard disk is satisfactory.

Next, the second establishing module103will use tools (such as BCACHE or FLASHCACHE) to recreate the logical storage device with the virtual hard disk, and finally add the logical storage device back to the original distributed data access system to execute the data backfill.

In the embodiment, if the server where the new hard drive is located is damaged, the storage resource managing device100can repair the server according to the following operations.

After shutdown, the detecting module104detects damaged components of the server and replaces related components, and then activates them on to confirm that they are normal. The first establishing module102recreates a virtual hard disk from the reserved memory space of the server, and finally adds the virtual hard disk back to the original distributed data access system.

Next, the second establishing module103will use tools (such as BCACHE or FLASHCACHE) to recreate the logical storage device with the new hard disk to be backfilled, add it back to the original distributed data access system to execute the data backfill, and finally add the logical storage device back to the original distributed data access system to execute the data backfill.

The storage resource managing device100reduces the risk of data loss in the backfill process and improves the security of the data.

FIG. 5illustrates a flowchart of a method for managing storage resources. The method for managing storage resources may include the following steps.

In block S501, the first storage devices of the server are formed into an emulation hard disk.

In block S502, mapping the emulation hard disks to establish a virtual hard disk.

In block S503, mapping the virtual hard disk with the new second storage device to establish a logical storage device, to perform data access operations on the logical storage device.

When any one of the plurality of second storage devices of the server200is damaged, the storage resource managing device100maps the virtual hard disk with the new second storage device and establishes a logical storage device, to perform data access operations on the logical storage device. Therefore, the storage resource managing device and method greatly reduce the risk of data in the backfill process and improve the security of the data.