Computer system with virtualization mechanism and management table, cache control method and computer program

At least one of a first application program and a second application program sends an access request to a second cache management module, which receives the access request, and references the second cache management table to identify the storage location of the access-target data conforming to the access request. When access-target data exists in first cache area, the second cache management module sends a request to the first cache management module storing the access-target data, and where access-target data does not exist in the first cache area, acquires the access-target data from the second storage device. When the access-target data is in first cache area, the first cache management module acquires the access-target data conforming to the request from the relevant first cache area, and sends access-target data to the second cache management module.

TECHNICAL FIELD

The present invention relates to cache control.

BACKGROUND ART

For the purpose of utilizing the big data, attention to the high-speed analysis technology of big data is increasing. An IT platform capable of accessing large amounts of data at high speeds is needed to realize this high-speed analysis technology.

To meet this requirement, for example, Patent Literature 1 discloses a parallel processor having multiple processors, a distributed shared memory (cache memory) provided for each processor, and an information transfer line therebetween. Specifically, each processor is able to store data, which is being held in a distributed shared memory of another processor, in its own distributed shared memory as needed. That is, each processor disclosed in Patent Literature 1 is able to share data with another processor using the distributed shared memory. Thus, the entire parallel processor is able to access large amounts of data at high speeds without increasing the capacity of the distributed shared memory.

However, in a case where the capacity of the distributed shared memory is large enough to be able to store all the data targeted by an application pursuant to processing large amounts of data, the distributed shared memory, although fast, becomes very expensive. For this reason, it is conceivable that an I/O (Input/Output) node, which comprises a nonvolatile storage device and a cache memory for temporarily storing data to be input/output to/from this storage device, be coupled to a processor (computation node) comprising a cache memory, frequently used data be stored in the cache area of the processor, and infrequently used data be stored in the I/O node storage device.

For example, Patent Literature 2 discloses a technology for hierarchizing a storage area, which includes the computation node cache memory, the I/O node cache memory, and the I/O node storage device, linking this hierarchized storage area to the processing of an application, and arranging data required by the application in a storage area belonging to a prescribed tier.

CITATION LIST

Patent Literature

SUMMARY OF INVENTION

Technical Problem

However, in Patent Literature 2, the management of the cache memory in the computation node and the management of the cache memory in the I/O node are independent of one another. The management of a cache memory is optimized for the node (the I/O node or the computation node) comprising the cache memory. Therefore, cache memory utilization efficiency deteriorates in a computer system, which comprises multiple computation nodes and I/O nodes (I/O nodes, which are examples of computers) coupled thereto. For example, the same data could remain in both the computation node cache memory and the I/O node cache memory, or infrequently used data could be left in the cache memory.

Solution to Problem

There are multiple first computers, and a second computer, which comprises a second storage device. Each of the multiple first computers comprises a first application program, a first cache management module, and a first cache area. The second computer comprises a virtualization mechanism, and multiple OSs run on the virtualization mechanism. The multiple OSs comprise an OS on which a second application program runs, and at least one OS comprising a second cache management module and a second cache management table. The first application program and/or the second application program send(s) an access request to the second cache management module. The second cache management module receives the access request from the first application program and/or the second application program, and references the second cache management table to identify the storage location of the access-target data conforming to the access request. In a case where the access-target data exists in the first cache area, the second cache management module sends a data transfer request to the first cache management module storing the access-target data, and in a case where the access-target data does not exist in the first cache area, acquires the access-target data from the second storage device. In the case where the access-target data exists in the first cache area, the first cache management module acquires the access-target data conforming to the data transfer request from the relevant first cache area, and sends the access-target data to the second cache management module. The second cache management module sends the acquired access-target data or the received access target data to the first application program and/or the second application program.

DESCRIPTION OF EMBODIMENTS

A number of examples will be explained.

In the following explanation, various types of information may be explained using the expression “xxx table”, but the various information may also be expressed using a data structure other than a table. To show that the various information is not dependent on the data structure, “xxx table” can be called “xxx information”.

Furthermore, in the following explanation, there may be cases where processing is explained having a computer program as the doer of the action, but since stipulated processing is performed in accordance with a program being executed by a controller-equipped processor (for example, a CPU (Central Processing Unit)) while using a storage resource (for example, a memory) and a communication control device (for example, a communication port) as needed, the processor may also be regarded as the doer of the processing. A process, which is explained having the program as the doer of the action, may be regarded as a process performed by a controller, which comprises the processor executing this program, or an apparatus comprising this controller (for example, a computation node or an I/O node). Furthermore, either part or all of a program may be realized using dedicated hardware. The controller may comprise the dedicated hardware in addition to a processor. A computer program may be installed in respective computers from a program source. The program source, for example, may be either a program delivery server or a storage medium.

In the following explanation, in a case where like elements “ttt” are explained by distinguishing therebetween, these like elements may be expressed by combining reference signs like “tttA” and “tttB”, but when no particular distinction is made between these elements, the elements are described simply as “ttt”.

FIG. 1is a block diagram showing an example of the entire configuration of a computer system1related to Example 1.

A computer system1comprises a computation node2and an I/O node3, and these components (the computation node2and the I/O node3) are installed in the same enclosure. The computation node2and the I/O node3are coupled inside the enclosure using a prescribed interface (for example, a PCI (Peripheral Components Interconnect bus))23,33. The computation node2, for example, is a host computer. The I/O node3, for example, is a storage apparatus to which the host computer is coupled.

The computation node2and the I/O node3do not have to be disposed in the same enclosure. The computation node2and the I/O node3may also be coupled via a prescribed communication network (for example, a SAN (Storage Area Network)).

The computation node2, for example, is a type of computer. The computation node2stores a control program group211, a management table group212, and an application program4in a memory21. The application program4is software, which has been designed for a certain purpose (for example, software for enabling a computer to function as a word processor). The control program group211and the management table group212will be explained further below.

The I/O node3, for example, is a type of input/output device, and specifically, as was explained hereinabove, is a storage apparatus. The I/O node3comprises a nonvolatile storage device34and a memory31, and stores a control program group211and a management table group212in the memory31.

FIG. 2is a block diagram showing an example of the configuration of the computation node2.

The computation node2comprises a memory21, an interface23, and a processor (for example, a CPU (Central Processing Unit))22coupled thereto.

The processor22is responsible for controlling the operation of the computation node2, and executes the necessary processing based on the control program group211and management table group212stored in the memory21. The memory21, in addition to being used to store the control program group211and the management table group212, is also used as working memory for the processor22. The interface23is a communication interface, and controls the protocol when the computation node2communicates with the I/O node3.

The control program group211comprises a configuration information collection program2111, a configuration information notification program2112, and a cache management program2113. The programs2111through2113in the drawing may be realized using a single program, or may be divided into either two or less, or four or more programs depending on the implementation.

The configuration information collection program2111, for example, operates as an interface for configuring a configuration management table2121, which will be explained further below. The configuration information collection program2111, for example, configures and/or updates the configuration management table2121by invoking a setting API (Application Program Interface).

The configuration information notification program2112notifies the I/O node3of information being held in the configuration management table2121and information being stored in a cache management table2122, which will be explained further below, when a not-shown OS (Operating System) of the computation node2is booted up, and in accordance with a request from a configuration information collection program3111and a shared cache management program3112of the I/O node3, which will be explained further below.

The cache management program2113stores data from the I/O node3(the memory31or the storage device34) in a local cache area215, and updates the data stored in the local cache area215in accordance with a request from the application program4. Specifically, for example, the cache management program2113adds an entry to a cache management table2122and updates the entry.

The management table group212comprises the configuration management table2121and the cache management table2122. These management tables2121and2122will be explained further below.

FIG. 3is a block diagram showing an example of the configuration of the I/O node3.

As shown inFIG. 3, the I/O node3comprises a memory31, a processor32, an interface33, and a storage device34.

The memory31, processor32and interface33respectively comprise substantially the same functions as the memory21, processor22, and interface33of the computation node2, and as such, detailed explanations thereof will be omitted.

The storage device34, for example, may by multiple physical storage devices configured as a RAID (Redundant Array of Inexpensive Disks), or may be a logical storage device based on either an internal or external physical storage device. For example, data, which is used by the computation node2application program4, is stored in the storage device34, and this data is stored in the memory21and used when needed by the application program4.

The control program group311stored in the memory31includes a configuration information collection program3111and a shared cache management program3112. The programs3111and3112may be realized by being consolidated into a single program, or may be divided into three or more programs depending on the implementation.

The configuration information collection program3111updates a configuration management table3121and an address management table3123in accordance with a request from the information notification part2112of the computation node2.

The shared cache management program3112, in accordance with a request from the cache management program2113of the computation node2, acquires data from the storage device34, reads/writes the data from/to the cache area of the memory31, and updates a shared cache management table3122, which will be explained further below.

The management table group312stored in the memory31comprises a configuration management table3121, a cache management table3122, and an address management table3123. These tables3121through3123will be explained further below.

FIG. 4is a schematic diagram for illustrating a method for using the memory21of the computation node2.

The computation node2allocates and uses the storage area of the memory21with respect to the above-mentioned management table group212and the respective programs (4,211), and, in addition, logically partitions and uses the memory21storage area as an OS allocation area213, a local cache area214, and a control command storage area215.

The OS allocation area213is an area used as a working area by the OS, the application program4, and the control program groups211. The local cache area214is used as a cache area by the control program group211. The control command storage area215is for storing the contents of a processing request (command) from the local cache area214and the I/O node3.

FIG. 5is a schematic diagram for illustrating a method for using the memory31of the I/O node3.

The I/O node3allocates the storage area of the memory31to the above-mentioned management table group312and the control program group311, and, in addition, allocates the memory31storage area to a control program allocation area313and a cache area314, and logically partitions and uses the control program allocation area313and the cache area314.

Of these, the I/O node3logically partitions and manages the cache area314as a normal cache area3144, a local extended cache area3141, and a shared cache area3142, and, in addition, manages multiple virtual local cache areas3143respectively corresponding to the multiple local cache areas214.

The normal cache area3144is for temporarily storing data inputted/outputted to/from the storage device34.

The local extended cache area3141stores data from the computation node2local cache area214. The shard cache area3142stores data, which is referenced by multiple computation nodes2.

The virtual local cache area3143is associated with the local cache areas214of the respective computation nodes2. That is, the virtual local cache area3143exists in proportion to the number of computation nodes2communicating with the I/O node3. For example, the drawing shows a case in which the I/O node3is communicating with three computation nodes2A through2C, and virtual local cache areas3143A through3143C, which respectively correspond to local cache areas214A through214C of the respective computation nodes2A through2C, exist in the memory31. A single virtual local cache area3143A will be given as an example of the virtual local cache area as deemed appropriate hereinbelow.

The I/O node3associates the addresses (physical addresses, which will be explained further below) of the local cache areas214A through214C of the computation nodes2A through2C with the address (virtual addresses, which will be explained further below) of the corresponding virtual local cache areas3143A through3143C, and, in addition, manages the utilization statuses of the local cache areas214A through214C. This makes it possible for the I/O node3to integratively manage the computation node2local cache areas214A through214C and the I/O node3cache area314.

The virtual local cache area3143A is associated with the local cache area214A, but does not actually store data (for example, data used by the application program4). The data in the virtual local cache area3143A is actually stored in the corresponding local cache area214A. The I/O node3can access data in the corresponding local cache area214A by referencing the virtual local cache area3143A. The virtual local cache area3143A does not store the same data as the data stored in the local cache area214A, thereby making it possible to conserve the capacity of the I/O node3cache area314.

There is one local extended cache area3141, and this local extended cache area3141is logically partitioned, and a partitioned area is provided to each computation node2. This approach promises to keep the capacity of the local extended cache area3141small. However, a local extended cache area3141may also be provided to each computation node2.

FIG. 6shows an example of the configuration management table2121of the computation node2.

The configuration management table2121comprises a capacity21211, a used capacity21212, and a physical address21213. The capacity21211shows the capacity of the local cache area214. The used capacity21212shows the amount of data actually used by the computation node2with respect to the capacity of the local cache area214. The physical address21213shows multiple physical addresses belonging to the local cache area214.

FIG. 7shows an example of the cache management table2122of the computation node2.

The cache management table2122comprises a page number21221, a physical address21222, a data21223, and a valid bit21224.

The storage device34comprises multiple pages (storage areas), and the page number21221is the number of a page of the storage device34. The physical address21222is the same as the physical address21213of the configuration management table2121, that is, it shows the physical address of the local cache area214. Based on a combination of the page number21221and the physical address21222, it is clear where data, which is stored in a certain page of the storage device34, is stored in the local cache area214. The data21223shows the type of data being stored in an area (an area in the local cache area214) identified from the physical address21222.

The valid bit21224is information showing whether the data stored in the area identified using the physical address21222is “valid” or “invalid”. “Invalid” shows that the data stored in the area identified in accordance with the physical address21222is different from data stored in a page of the storage device34associated with the relevant area. Alternatively, “valid” shows that the data stored in the area identified using the physical address21222is the same as data stored in the page associated with the relevant area.

FIG. 8shows an example of the configuration management table3121of the I/O node3.

The configuration management table3121correspondingly manages a node identifier31211, a communication mode31212, a capacity31213, and a used capacity31214for each computation node2or I/O node3.

The node identifier31211is information for identifying the computation nodes2A through2C and the I/O node3. The node identifier “0” denotes the I/O node3. The node identifiers “A through C” respectively denote the computation nodes2A through2C.

The communication mode31212shows the mode by which the I/O node3accesses data, which is associated with the cache area314, and data, which is stored in the cache area314. The modes for accessing data, for example, are RDMA (Remote Direct Memory Access), and memory reference. “RDMA” shows that the I/O node3accesses data stored in the local cache area214, which is associated with a virtual local cache area3143. “Memory reference” shows that the I/O node3accesses data stored in the local extended cache area3141and the shared cache area3142, which are areas of the cache area314other than the virtual local cache area3143.

The capacity31213shows the capacity of both the local extended cache area3141and the shared cache area3142in a case where the node identifier31211is “0”. In a case where the node identifier31211is “A through C”, the capacity31213shows the capacity of each of the local cache areas214A through214C.

The used capacity31214shows the capacity, which is actually being used with respect to the capacity31213.

In a case where the node identifier31211is “0”, the capacity31213and the used capacity31214are divided into two rows. The upper row shows the capacity31213and the used capacity31214of the local extended cache area3141, and the lower row shows the capacity31213and the used capacity31214of the shared cache area3142. In addition, in a case where the node identifier31211is “A through C”, the capacity31213and the used capacity31214of the virtual local cache area3143corresponding to the local cache areas214A through214C of the respective computation nodes2A through2C are shown.

FIG. 9shows an example of the cache management table3122of the I/O node3.

The cache management table3122shows whether or not data stored in the cache area314and data stored in the storage device34are redundant.

The cache management table3122comprises a page number31221, a virtual address31222, and a valid bit31223.

The page number31221is a number of a page in the storage device34. The virtual address31222shows multiple virtual addresses, which belong to the cache area314. For example, in the example shown in the drawing, the page number31221“0” is associated with the virtual address “3”, and this shows that data stored in the page of the page number “0” is stored in the area (the area in the cache area314) belonging to virtual address “3”. Meanwhile, an invalid value “-” is associated as the virtual address31222with the page number31221“1”. This shows that data stored in the storage device34is not stored in the cache area314.

The valid bit31223is information showing whether data stored in the storage device34is “valid” or “invalid”. The valid bit31223is substantially the same as the valid bit21224of the cache management table2122. For example, in the drawing, the page number “2” of the storage device34is associated with the virtual address “5” of the cache area314, but the valid bit31223is “invalid”, thereby indicating that the data stored in the area (the area in the cache area314) of the virtual address “5” differs from the data being stored in the page of page number “2”.

FIG. 10shows an example of the address management table3133of the I/O node3.

The address management table3123comprises a virtual address31231, an attribute value31232, a physical location31233, a physical address31234, and a utilization status31235for each area in the cache area314. The information of the address management table3123will be explained below by taking one area in the cache area314(referred to as “target area” in the explanation ofFIG. 10) as an example.

The virtual address31231shows the virtual address of the target area.

The attribute value31232shows the type of area, which comprises the target area. The attribute value31232“extended” shows that the target area is being used as the local extended cache area3141. The attribute value31232“shared” shows that the target area is being used as the shared cache area3142. Also, the attribute value31233“virtual A” shows that the target area is being used as the virtual local cache area3143A. This attribute value31232may be omitted.

The physical location31233shows the place where the data stored in the target area is actually stored. “Local memory” shows that the data stored in the target area is actually being stored in the memory31of the I/O node3. In addition, “Computation node A” shows that the data stored in the target area is actually being stored in the memory21of the computation node2A.

The physical address31234shows the physical address of the target area. The utilization status31235shows whether data is being stored in the area indicated by the physical address31234of the target area. “Used” shows that data is being stored in the area indicated by the physical address31234of the target area. “Unused” shows that data is not being stored in the area indicated by the physical address31234of the target area.

How data is migrated between caches will be explained next.

FIG. 22is a schematic diagram for illustrating the migration of data between caches. Furthermore, in the explanation ofFIG. 22, it is supposed that the computer system1is as described in (1) through (5) below.

(1) The local cache area of the computation node A is an area for storing data acquired at the time of an application program A (the application program of the computation node A) read request. The same also holds true for the local cache area of the computation node B, which is used at the time of an application program B (the application program of the computation node B) read request.

(2) The virtual local cache areas A and B are area names, which appear in the cache management table3122of the I/O node3for determining whether or not the I/O node3writes data to the local cache areas of computation nodes A and B, and are not for using the I/O node3memory area to actually store data. In a case where the I/O node3decides to read/write data from/to the virtual local cache area A (B), the I/O node3issues a read/write request to the local cache area A (B) of the computation node A (B).

(3) The local extended cache area is for use as a migration destination for data prior to an overwrite when a data overwrite has been generated in the local cache area A (B) of the computation node A (B).

(4) The shared cache area3142is for storing data, which has been referenced by both application programs A and B, and migrates data with respect to a read request from the application program A (B) in a case where a hit occurred for either the computation node B (A) local cache area B (A) or the local extended cache area.

(5) Both of the application programs A and B can use the data in the local cache areas of the computation nodes A and B, and in the I/O node memory by going through the I/O node.

A process for migrating data from the local cache area A (virtual local cache area A) to the shared cache area will be explained here.

In a case where there has been a read request from the application program A to read data X from a page of the storage device34, the computation node A sends the read request to the I/O node3without determining whether or not the data X is in the local cache area A. The I/O node3reads the data X from the storage device34to the normal cache area3144, and sends the read data X to the computation node2A, which is the source of the read request. The computation node2A stores the received data X in the local cache area A.

Then, in a case where there is a data X read request from the application program B, the computation node B sends this read request to the I/O node3without determining whether or not the data X is in the local cache area B. The I/O node3receives the read request, reads data X from the local cache area A corresponding to the virtual local cache area A, which corresponds to the source of the read request, stores the read data X in the normal cache area3144, and sends the data X from the normal cache area3144to the computation node B. The computation node B stores the received data X in the local cache area A. Thereafter, the I/O node3migrates the data X stored in the normal cache area3144(the above-mentioned data X, which has been stored in the normal cache area3144from the local cache area A) from the normal cache area3144to the shared cache area3142. That is, in a case where the application B once again targets data X for a read, the data X will generate a hit in the shared cache area3142even, for example, when the data X has been overwritten and has disappeared from the local cache area B. In a case where the I/O node3has received a read request from the computation node B targeting the data X for a read, this makes it possible to rapidly send the data X without placing a load on the computation node A, and, in addition, without relying on the cache update frequency of the application A.

A process for migrating data from the local cache area A (virtual local cache area A) to the extended local cache area will be explained here.

The I/O node3checks the remaining capacity (or, the free area ratio, which is the percentage of remaining capacity relative to the capacity of the local cache area A) of the local cache area A of the computation node A at cache update time. Then, in a case where the remaining capacity of the computation node A local cache area A is equal to or smaller than the size of the update data, the I/O node3migrates, from among data stored in the local cache area A, the data, which corresponds to a prescribed condition (for example, infrequently used data) X, to the local extended cache area.

Thus, there may be cases where storing the data, which conforms to a prescribed condition (for example, infrequently used data) X, in the local extended cache area A temporarily rather than suddenly swapping this data out of the storage device34makes it possible to reduce the frequency with which the data X must be read from the storage device34. Furthermore, the data X, which has been stored in the local extended cache area A, is migrated to the shared cache area3142when targeted for a read by the computation node B.

A process for migrating data from the local extended cache area to the shared cache area3142will be explained here.

In a case where there is a read request for the data X from the application program B after data migration process (2) has been performed, the computation node B sends the read request to the I/O node3. The I/O node3receives this read request, stores the data X, which is being stored in the local extended cache area, in the normal cache area3144for a time, and sends the data X from the normal cache area3144to the computation node B. The computation node B stores the received data X in the local cache area B. Thereafter, the I/O node3migrates the data X from the normal cache area3144to the shared cache area3142.

The flow of processing of each program will be explained next.

FIG. 11is an example of a flowchart of the processing of the configuration information collection program2111of the computation node2.

The configuration information collection program2111detects the execution of a setting API from the application program4(S101).

In a case where the execution of the setting API has not been detected (S101: No), the configuration information collection program2111performs the processing of S101.

In a case where the configuration information collection program2111has detected the execution of the setting API (S101: Yes), the configuration information collection program2111uses the setting API to collect the configuration information to be notified (S102), and writes the collected configuration information to the configuration management table2121(S103).

The configuration management table2121is either created or updated in accordance therewith.

FIG. 12is an example of a flowchart of the processing of the information notification program2112of the computation node2.

The information notification program2112determines whether or not there has been a configuration information acquisition request from the I/O node3(S201). As used here, the configuration information is information, which is managed by the configuration management table2121and the cache management table2122of the computation node2.

In a case where a configuration information acquisition request from the I/O node3has not been detected (S201: No), the information notification program2112checks whether or not a computation node2OS boot-up process has been executed (S202).

In a case where the execution of the OS boot-up process has not been detected (S202: No), the information notification program2112returns to the processing of S201.

In a case where the execution of the OS boot-up process has been detected (S202: Yes), the information notification program2112collects information related to the local cache area214from the configuration management table2121and the cache management table2122(S203), and sends the collected information to the I/O node3as configuration information (S204). Thereafter, the information notification program2112once again performs the processing of S201.

In a case where a configuration information acquisition request from the I/O node3has been detected (S201: Yes), the information notification program2112determines the type of the request (read or write) from the I/O node3(S205).

In a case where the determination of S205is that the request is a read request, the information notification program2112performs the processing of S206. In a case where the request is a write request, the information notification program2112performs the processing of S204.

In a case where the determination of S206is that the read request is a request to acquire configuration information, the information notification program2112performs the processing of S203.

In a case where the determination of S206is that the read request is a request to acquire cache data (the data in the local cache area214), a physical address will be specified in this read request. The configuration information notification program2112uses the physical address of the read request to reference the cache management table2122, identifies the storage location of the read-target data (S207), and acquires the read-target data from the identified storage location (an area in the local cache area214) (S208). Thereafter, the information notification program2112performs the processing of S204.

FIG. 13is an example of a flowchart of the processing of the cache management program2113of the computation node2.

The cache management program2113checks whether or not an I/O request has been generated in accordance with a certain computer program (assumed to be the application program4in the explanation ofFIG. 13) being executed (S301).

In a case where there is no I/O request from the application program4(S301: No), the cache management program2113performs the processing of S301.

In a case where there is an I/O request from the application program4(S301: Yes), the cache management program2113sends the I/O request from the application program4to the I/O node3without determining whether or not data conforming to this I/O request is in the local cache area214(S302).

The cache management program2113receives a reply (hereinafter referred to as I/O reply) with respect to the I/O request from the I/O node3(S303). Specifically, for example, the cache management program2113, in accordance with the I/O reply from the I/O node3, stores the data conforming to the I/O request in the area (the area in the local cache area214) belonging to the physical address specified in the I/O reply.

The cache management program2113updates the cache management table2122. For example, the cache management program2113adds an entry conforming to the I/O request to the cache management table2122, stores a page number, a physical address, and data included in the received reply in the added entry, and, in addition, configures the valid bit21223in this entry to “valid”. The cache management program2113updates the value of the used capacity21212of the configuration management table2121to the value specified in the I/O reply from the I/O node3(S304).

The cache management program2113replies to the application program4that the I/O processing has been completed (S305).

FIG. 14is an example of a flowchart of the processing of the configuration information collection program3111of the I/O node3.

The I/O node3determines whether or not there is a configuration information notification request from the computation node2(S401).

In a case where there is no configuration information notification request from the computation node2(S401: No), the configuration information collection program3111performs the processing of S401.

In a case where there is a configuration information notification request from the computation node2(S401: Yes), the configuration information collection program3111determines whether or not the notification request-source computation node2is a new computation node2(S402). A new computation node2is a computation node, which is not registered in the address management table3123. Specifically, for example, the configuration information collection program3111identifies a PCI pin number from which the communication was generated, and determines whether the identified computation node2is a new computation node2by determining whether or not the identified computation node2is registered in the address management table3123.

The configuration information collection program3111stores information conforming to the configuration information notification request in the memory31of the I/O node3as the configuration information, and/or updates the memory31with the information conforming to the configuration information notification request (S403).

The configuration information collection program3111, in a case where the computation node2is a new computation node2, for example, configures the PCI pin number as the identification number of this computation node2, and based on the capacity of the usable cache memory21(local cache memory214) of this computation node2, allocates a virtual address3143, which is an area constituting a portion of the cache area314, to the cache memory21(S404).

The configuration information collection program3111stores configuration information related to the computation node2registered in the address management table3123and/or configuration information related to the new computation node2in the cache management table3122and the address management table3123(S405). Thereafter, the configuration information collection program3111performs the processing of S401once again.

According to the processing shown inFIG. 14, the I/O node3is able to integratively manage the information being managed by the computation node2configuration management table2121and the cache management table2122of each computation node2.

FIG. 15is an example of a flowchart of the processing of the shared cache management program3112of the I/O node3.

The shared cache management program3112determines whether or not there is an I/O request from the computation node2(S501).

In a case where there is no I/O request from the computation node2(S501: No), the shared cache management program3112once again performs the processing of S501.

In a case where there is an I/O request from the computation node2(S501: Yes), the shared cache management program3112determines whether the I/O request is a read request or a write request (S502).

In a case where the I/O request is a read request, the shared cache management program3112identifies the storage location of data conforming to the read request from the cache management table3122(S503). The data storage location here shows the page number31221of the storage device34. Initially, the data conforming to the read request is not stored in any of the local cache areas214associated with the local extended cache area3141, shared cache area3142, and virtual local cache area3143.

The shared cache management program3112reads the data from the read-source page (the page to which the read request-specified address belongs) of the storage device34in which the data conforming to the read request is stored, and stores the read data in the normal cache area3144(S504).

The shared cache management program3112sends a reply (response) including this data to the request-source (the source of the read request) computation node2(S505).

Alternatively, in a case where the I/O request is a write request, the shared cache management program3112writes the data conforming to the write request (the write-target data) to the write-destination page (the page to which the write request-specified address belongs) of the storage device34(S506). The shared cache management program3112does not write the write-target data to the normal cache area3144at this point, but may temporarily store this data in the normal cache area3144, and thereafter write this data to the write-destination page from the normal cache area3144.

The shared cache management program3112configures the valid bit31223corresponding to the write-destination page to “invalid” in the cache management table3122(S507). This process, for example, is for prohibiting the application program4from using pre-update data stored in the cache areas214and314. Processing such as this is performed because data read to the cache areas214and314from the storage device34prior to updating will differ from the post-update data stored in the storage device34as a result of data stored in the storage device34having been updated.

The shared cache management program3112references the address management table3123, and sends a cache management table2122update request to the computation node2corresponding to the virtual address for which the valid bit31223was configured to “invalid” in S507(S508). This update request specifies the physical address corresponding to the virtual address for which the valid bit31223was configured to “invalid” in S507.

The computation node2, which is the destination of the update request of S508, receives this update request, and configures the valid bit21223corresponding to the physical address specified in this update request to “invalid” in the cache management table2122(S509). Thereafter, the shared cache management program3112once again performs the processing of S505.

FIG. 16is a detailed example data acquisition processing (S504ofFIG. 15).

The shared cache management program3112references the cache management table3122and determines whether or not there is a cache hit (S601). The “presence or absence of a cache hit” here refers to whether or not the read-target data is stored in the virtual local cache area corresponding to the computation node, which is the source of the read request, and specifically to whether or not the association between the read-source page number31221and virtual address31222exists in the cache management table3122.

In a case where there is no cache hit (S601: No), the shared cache management program3112reads the read-target data from the read-source page (the storage device34) (S602), stores the read data in the normal cache area3144, includes this data in a reply to the request-source computation node2, and, in addition, configures the valid bit31223corresponding to the read-source page to “valid” in the cache management table3122(S603).

Thereafter, the shared cache management program3112determines whether or not a prescribed condition regarding the virtual local cache area3143corresponding to the request-source computation node2has been satisfied (S604). Specifically, for example, the shared cache management program3112determines whether or not the free space (or the free capacity ratio) for the virtual local cache area3143(the local cache area214) corresponding to the request-source computation node2is equal to or smaller than a prescribed value. This determination can be made by referencing the configuration management table3121.

In a case where the prescribed condition for the virtual local cache area3143corresponding to the request-source computation node2has been satisfied (S604: Yes), the shared cache management program3112executes a swap-out process (S605). As used here, “swap-out process”, for example, refers to moving infrequently used data from among the data stored in the virtual local cache area3143(local cache area214) to the storage device34. This swap-out process increases the remaining capacity of the virtual local cache area3143(local cache area214).

In a case where there has been a cache hit (S601: Yes), the shared cache management program3112references the address management table3123and determines in which area of the cache area314(that is, which of the local extended cache area3141, the shared cache area3142, and the virtual local cache area3143) the cache hit data is located (the area storing the read-target data) (S606).

In a case where the determination of S606is that the cache hit data location is the virtual local cache area3143, the shared cache management program3112uses, for example, a RDMA transfer process to acquire the read-target data from the local cache area214, which is associated with the virtual local cache area3143(S607), and determines whether or not the data request-source computation node and the data acquisition-destination computation node are the same (S608). This determination processing is implemented by comparing the node identifiers31211in the configuration management table3121.

In a case where the determination of S608is that it is not the same node (S608: No), the shared cache management program3112performs the processing of S603.

In a case where the determination of S608is that it is the same node (S608: Yes), the shared cache management program3112writes the acquired data to the shared cache area3142(S609). Thereafter, the shared cache management program3112performs the processing of S603.

In a case where the determination of S606is that the cache hit location is the local extended cache area3141, the shared cache management program3112acquires the read-target data from the local extended cache area3141(S610), and write this data to the shared cache area3142(S609). Thereafter, the shared cache management program3112performs the processing of S603.

In a case where the determination of S606is that the cache hit location is the shared cache area3142, the shared cache management program3112acquires the read-target data from the shared cache area3142(S611). Thereafter, the shared cache management program3112performs the processing of S603.

The determination as to whether or not there is a cache hit may be performed in order from the shared cache area, the virtual local cache area, and the local extended cache area. Specifically, for example, the shared cache management program3112may perform the cache hit determination for the shared cache area, and in a case where the result of this determination is negative, may perform the cache hit determination for the virtual local cache area, and in a case where the result of this determination is negative, may perform the cache hit determination for the local extended cache area. The reason the initial target of the cache hit determination is the shared cache area is because the preferential use of data, which has been stored in the shared cache area, avoids the frequently accesses to the computation node2, thereby making it possible to hold the performance degradation of the application program4in check.

In addition, the shared cache management program3112of the I/O node3may comprise a setting API for controlling the order of cache hit determinations, and may change the determination order in accordance with a specification from the application program4.

The swap-out process, for example, is performed based on the amount of remaining capacity of the local cache area214associated with the virtual local cache area3143. Specifically, for example, the shared cache management program3112references the configuration management table3121and determines whether or not the capacity31213of the request-source computation node2is equal to or larger than the sum of the used capacity31214and the write capacity (the size of the write-target data) (S701).

In a case where the capacity31213is equal to or larger than the sum of the used capacity31214and the write capacity (S701: Yes), the shared cache management program3112writes the write-target data to the local cache area214associated with the virtual local cache area3143, and adds the value of the write capacity to the used capacity31214of the request-source computation node2(S702).

Alternatively, in a case where the capacity31213is less than the sum of the used capacity31214and the write capacity (S701: No), the shared cache management program3112decides migration-target data from among the data stored in the local cache area214of the request-source computation node2based, for example, on a LRU (Least Recently Used) algorithm (S703). The shared cache management program3112reads the migration-target data from the local cache area214and temporarily stores this migration-target data in the normal cache area3144.

The shared cache management program3112, based on the cache management table3122and the address management table3123, uses the page number corresponding to the migration-target data to retrieve the migration-target data and redundant data (matching data) from the local extended cache area3141and the shared cache area3142(S704).

In a case where the redundant data is not being stored in either of the local extended cache area3141or the shared cache area3142(S705: No), the shared cache management program3112references the configuration management table3121(the capacity31213of the node having the node identifier31211of “0”), and determines whether or not the capacity31213of the local extended cache area3141is equal to or larger than the sum of the used capacity31214and the migration capacity (the size of the migration-target data) (S706).

In a case where the capacity31213is equal to or larger than the sum of the used capacity31214and the migration capacity (S706: Yes), the shared cache management program3112writes the migration-target data to a free area in the local extended cache area, and adds the migration capacity to the used capacity31214of the local extended cache area in the configuration management table3121(S707).

Alternatively, in a case where the capacity31213is less than the sum of the used capacity31214and the migration capacity (S706: No), the shared cache management program3112decides the data to be swapped out from among the data stored in the local extended cache area3141(for example, makes a decision based on the LRU algorithm), swaps out the decided data from the local extended cache area3141to the storage device34, and writes the migration-target data to the free area of the local extended cache area3141(the area in which the swapped out data had been stored) (S708). In addition, the shared cache management program3112updates the cache management table3122with respect to the area in which the swapped out data had been stored (the write-destination area of the migration-target data) (S709). Specifically, for example, the page number31211for the relevant area is changed from the page number of the swapped out data to the page number of the migration-target data. In this example, the size of the swapped out data (decided data) matches the size of the migration-target data, and as such, the updating of the used capacity31214as in S707need not be performed. The size of the swapped out data (the decided data) may also be larger than the migration-target data.

In a case where the migration-target data is stored in either the local extended cache area3141or the shared cache area3142(S705: Yes), the shared cache management program3112determines whether the location in which the migration-target data and the redundant data are being stored is the local extended cache area3141or the shared cache area3142(S710).

In a case where the determination of S710is that the redundant data is stored in the local extended cache area3141, the shared cache management program3112references the configuration management table3121and determines whether or not the capacity31213of the shared cache area3142is equal to or larger than the sum of the used capacity31214and the redundant capacity (the size of the redundant data) (S711).

In a case where the capacity31213is equal to or larger than the sum of the used capacity31214and the redundant capacity (S711: Yes), the shared cache management program3112writes the redundant data to the shared cache area3142, and adds the size of the redundant data to the used capacity31214of the shared cache area in the configuration management table2131(S712).

In a case where the capacity31213is less than the sum of the used capacity31214and the redundant capacity (S711: No), the shared cache management program3112decides the data to be swapped out of the shared cache area3142(for example, makes a decision based on the LRU algorithm), and stores the decided data in the storage device34. The shared cache management program3112writes the redundant data to the free area of the shared cache area3142(the area in which the swapped out data had been stored) (S713). In addition, the shared cache management program3112deletes this redundant data from the local extended cache area3141(S714). Thereafter, the shared cache management program3112performs the processing of S709. The size of the data decided in S713matches the size of the redundant data, but may be larger than the size of the redundant data.

In a case where the determination of S710is that the redundant data is stored in the shared cache area3142, the shared cache management program3112ends the processing.

FIG. 18is an example of a flowchart showing the flow of processing of the computation node2boot-up process.

When the computation node2is powered up (S801), the computation node2(the configuration information collection program2111) executes an initialization process to delete all the data being managed in the cache management table2122(S802), and sends the configuration management table2121and the cache management table2122to the I/O node3(S803).

The I/O node3(the configuration information collection program3111) receives the configuration management table2121and the cache management table2122from the computation node2, and determines whether information based on these tables2121and2122is already registered in the address management table3123(S804).

In a case where this information is not registered (S804: No), the I/O node3(the configuration information collection program3111) references the unregistered configuration management table2121of the computation node2, creates the same number of virtual addresses31231as there are physical addresses21213, and registers the virtual addresses31231in the address management table3123(S805). This processing associates the virtual local cache area3143of the I/O node3with the local cache area214of the computation node2. Thereafter, the I/O node3determines whether or not this processing was a success (S806).

In a case where the processing was a success (S806: Yes), the I/O node3ends the processing. Alternatively, in a case where the processing failed (S806: No), the I/O node3performs the processing of S803.

In a case where the determination of S804is that the information is registered (S804: Yes), the I/O node3configures the utilization statuses31235of all the computation nodes2corresponding to the address management table3123to unused (S807). Thereafter, the I/O node3performs the processing of S806.

FIG. 19is an example of a flowchart showing the flow of processing an I/O node3boot-up process.

When the I/O node3is powered up (S901), the I/O node3(the configuration information collection program3111) uses the RDMA function, sets a data notification bit in the control command storage area215of each computation node2memory (S902), and sends an interrupt notification to each computation node2(S903).

The computation node2(the configuration information collection program2111) receives the interrupt notification from the I/O node3(S904), checks that a data notification bit is stored in the control command storage area215of the memory21(S905), and sends the cache management table2122information held by this computation node2to the I/O node3(S906).

The I/O node3receives the cache management table2122information from the computation node2, and registers information based on this information in the cache management table3122(S907). The I/O node3determines whether or not the processing of S907was a success (S908).

In a case where the processing was a success (S908: Yes), the I/O node3ends the processing. Alternatively, in a case where the processing failed (S908: No), the I/O node3performs the processing of S902.

According to Example 1, in a computer system1in which an I/O node3is coupled to multiple computation nodes2, it is possible to share data in a cache area without duplicating this data.

In addition, Example 1 also makes it possible to share frequently used data in a cache area between computation nodes2by forming a cache area (a shared cache area), which stores data referenced by multiple computation nodes2. Therefore, Example 1 makes it possible to enhance the processing performance of an application program running on the computation node2without increasing the capacity of the cache area of each computation node2, and, in addition, also enables I/O requests from multiple computation nodes to be concentrated in the local cache area of a computation node to avoid the deterioration of application program processing performance.

Example 2 will be explained hereinbelow. In so doing, the differences with Example 1 will mainly be explained, and explanations of points in common with Example 1 will either be simplified or omitted.

In Example 2, in a case where there is an I/O request from the application program4, each computation node2determines whether or not the request data is stored in the local cache area214of the computation node2, and in a case where the data is being stored, returns the stored data to the application program4without sending the I/O request to the I/O node3.

FIG. 20is an example of a flowchart showing the flow of processing of a cache management program2113related to Example 2.

The cache management program2113determines whether or not there is an I/O request from the application program4(S1001).

In a case where there is no I/O request (S1001: No), the cache management program2113performs the processing of S1001.

Alternatively, in a case where there is an I/O request (S1001: Yes), the cache management program2113references the cache management table2122, and determines whether the requested data is stored in the local cache area214(S1002).

In a case where the requested data is not being stored in the local cache area214(S1002: No), the cache management program2113performs the same processing as the processing of S302through S305ofFIG. 13(S1003through S1006).

Alternatively, in a case where the requested data is stored in the local cache area214(S1002: Yes), the cache management program2113acquires the data from the local cache area214rather than from the cache area314of the I/O node3(S1007). The cache management program2113creates a response message for the application program4(S1008). Thereafter, the cache management program2113performs the processing of S1006.

Example 3 will be explained. In so doing, the points of difference with Example 1 and Example 2 will mainly be explained, and explanations of the points in common with Example 1 and Example 2 will either be simplified or omitted.

In Example 3, the computation nodes2include a virtual node (a virtual computer) in addition to a physical node. There is no I/O node3in Example 3, and in place of the I/O node3, any virtual node performs the same processing as the I/O node3.

FIG. 21shows the overall configuration of a computer system1related to Example 3.

There is a computation node2D, which has a storage device34. The computation node2D creates and manages one or more virtual servers51by executing a virtualization mechanism5(for example, a hypervisor). The virtual servers51include a virtual server51A, which is a virtual computation node2, and a virtual server51B, which performs the same processing as the I/O node3.

The processor22executes processing as a computation node by executing the required processing based on a control program group211D and a management table group212D, which are stored in a memory area (omitted from the drawing) allocated to the virtual server51A from within the memory21. In addition, the processor22executes processing as an I/O node by executing the required processing based on a control program group311and a management table group312, which are stored in a memory area (omitted from the drawing) allocated to the virtual server51B from within the memory21.

This example employs a configuration, which partitions processing using a virtualization mechanism, but either an OS thread partition or processing partition function may be employed without using the virtualization mechanism.

A number of examples have been explained hereinabove, but the present invention is not limited to these examples.

For example, in the examples described above, in the processing of S603(refer toFIG. 16) of the computation node2cache management part2113, data was written to an area corresponding to a request source, but a hash value or the like may be used to uniquely decide a computation node2for storing data so as not to duplicate data between computation nodes2.

In the examples described above, the cache data of the local cache area214, the local extended cache area3141, and the shared cache area3142are updated at the time of a read request from the application program4. However, the shared cache management program3112can comprise a control API, and in accordance with a specification from the application program4, the control API can fixedly arrange data used by the application program4in any of the cache areas, i.e. the local cache area214, the local extended cache area3141, and the shared cache area3142. In accordance with this, the processing of S603, S604, and S605in the processing of the shared cache management program3112are omitted.

In the examples described above, in the shared cache management program3112of the I/O node3, data was migrated from the one computation node2(for example, computation node2A) to the other computation node (for example, computation node2B) via the I/O node3in accordance with a RDMA communication in a case where a cache hit occurred in any virtual local cache3143, but a direct RDMA communication process may be performed between the computation nodes2(for example, computation nodes2A and2B) without going through the cache area314of the I/O node3.

In the examples described above, the application program4sends an I/O request, which specifies a page number, but an I/O request may be sent using a virtual address specification.

In the examples described above, processing, which specifies the capacity of the local cache area214, was performed by the configuration information collection program2111of the computation node2, but this processing may be performed by the configuration information collection program3111of the I/O node3.

In the examples described above, one local extended cache area3141was formed in the memory21, but multiple local extended cache areas3141may be formed in each computation node.

In the examples described above, the local extended cache area3141and the shared cache area3142were logically partitioned and managed in the memory31, but the two areas may be managed as a single area.

In the examples described above, to ensure that data in the local extended cache area3141and the shared cache area3142was not duplicated, a check was performed to determine whether or not the data being stored in this two cache areas was redundant, but this processing may be omitted.

In the examples described above, the configuration is such that the interfaces23and33of the computation node2and the I/O node3are coupled without going through a prescribed apparatus, but, for example, the computation node2interface22and the I/O node3interface33can be coupled via a switch. In so doing, for example, in a case where there are multiple computation nodes2and I/O nodes3, when data is to be sent from a certain computation node2to a specific I/O node3, the computation node2may send information identifying the I/O node3to the switch apparatus at the same time as the I/O request. This makes it possible for the switch to identify a specific computation node2, and to distribute an I/O request from the computation node2to the specific I/O node3.

The storage device may comprise logical areas, and each logical area may an element comprising a stripe based on a RAID group. Each logical area group may be a component of a logical unit provided to an apparatus, which is the source of a write request (for example, either a host computer or another storage system), or may be an area group (an area group allocated to the write-destination virtual segment corresponding to the write to the virtual segment), which is dynamically allocated to any of multiple virtual segments (virtual storage areas) comprising a virtual logical unit (for example, a logical unit conforming to Thin Provisioning) provided to the write request source apparatus. In the case of the latter, a storage area pool may be configured using multiple logical segments, and may be allocated to the virtual segment in logical segment units. The storage area pool may comprise multiple logical units, and in accordance with this, each logical unit may comprise two or more logical segments.

The swap-out process may be performed at a different time either instead of or in addition to being performed during a series of processes (S504ofFIG. 15) performed after receiving the I/O request.

In this specification, a storage system of the following (Wording 1) and (Wording 2) has been explained.

A computer system comprising:

multiple first computers; and

a second computer comprising a second storage device,

wherein each of the above-mentioned multiple first computers comprises a first application program, a first cache management module, and a first cache area,

the above-mentioned second computer comprises a virtualization mechanism,

multiple OSs run on the above-mentioned virtualization mechanism, and the above-mentioned multiple OSs comprise an OS for running a second application program, and at least one OS comprising a second cache management module and a second cache management table,

the above-mentioned first application program and/or the above-mentioned second application program send(s) an access request to the above-mentioned second cache management module,

the above-mentioned second cache management module:

(A) receives the above-mentioned access request from the above-mentioned first application program and/or the above-mentioned second application program;

(B) references the above-mentioned second cache management table to identify a storage location of an access-target data conforming to the above-mentioned access request;

(C) sends a data transfer request to a first cache management module which stores the above-mentioned access-target data in a case where the above-mentioned access-target data exists in the above-mentioned first cache area; and

(D) acquires the above-mentioned access-target data from the above-mentioned second storage device in a case where the above-mentioned access-target data does not exist in the above-mentioned first cache area,

in the case of the above-mentioned (C), the above-mentioned first cache management module acquires the above-mentioned access-target data conforming to the above-mentioned data transfer request from a relevant first cache area, and sends the above-mentioned access-target data to the above-mentioned second cache management module, and

the above-mentioned second cache management module sends the above-mentioned acquired access-target data or the received access-target data to the above-mentioned first application program and/or the above-mentioned second application program, which are/is the source of the above-mentioned access request.

A computer system, comprising:

multiple first computers, which each comprise a first cache area; and

a second computer, which comprises a second storage device,

wherein the above-mentioned second computer comprises a function for managing the above-mentioned first cache areas of the multiple first computers,

each of the above-mentioned multiple first computers comprises a first application program, a first cache management module, and a first cache area,

the above-mentioned second computer comprises a second cache management module, multiple second cache areas, a third cache area, and a second cache management table,

each of the above-mentioned multiple first cache areas is associated with one of the above-mentioned second cache areas,

the above-mentioned first cache area and the above-mentioned third cache area are physical cache areas,

the above-mentioned second cache area is a virtual cache area,

the above-mentioned target first computer is any first computer of the above-mentioned multiple first computers, and

a target first application program of the above-mentioned target first computer sends an access request to the above-mentioned second computer via the first cache management module of the above-mentioned target first computer,

the above-mentioned second cache management module:

(A) receives the above-mentioned access request from the above-mentioned first application program;

(B) references the above-mentioned second cache management table to identify a storage location of access-target data conforming to the above-mentioned access request; and

(C) in a case where the above-mentioned access-target data exists in the above-mentioned second cache area, sends a data transfer request to a relevant first cache management module of the above-mentioned target first computer, which comprises a target first cache area corresponding to the above-mentioned second cache area,

the above-mentioned relevant first cache management module acquires the above-mentioned access-target data conforming to the above-mentioned data transfer request from the above-mentioned target first cache area, and sends the above-mentioned access-target data to the above-mentioned second cache management module, and

the above-mentioned second cache management module sends the received access-target data to the above-mentioned first application program.

In these wordings, the first application program and the second application program, for example, correspond to application programs4A through4D. The OS, for example, corresponds to either the control program group211A through211D or the control program group311. The first cache management module, for example, corresponds to the control program group211and the management table group212(excluding cache management table2122), and the second cache management module, for example, corresponds to the control program group311and the management table group312(excluding the cache management table3122).

REFERENCE SIGNS LIST