PATENT ABSTRACT
The present invention provides a computer system capable of providing the latest statistics on a file system in real time. The computer system includes a memory having a program for executing statistics on a file system, and a controller for executing the program. When the file system is accessed, the controller compares first statistic information before access processing on the file system with second statistic information after the access processing; and if it is determined that any difference exists between the first statistic information and the second statistic information, the controller updates the statistic result of the file system based on the difference.

PATENT DESCRIPTION
TECHNICAL FIELD 
       [0001]    The present invention relates to a computer system. Particularly, the invention relates to a computer system and file system management method suited for use in acquisition of statistic information of file systems. 
       BACKGROUND ART 
       [0002]    Recently, the difficulty of storage capacity management and stored file management has been increasing due to diversification of files/content stored in storage systems. If a storage administrator becomes aware of a tendency of stored files according to, for example, types and size of files to be stored, and update dates and times, and a file increase/decrease tendency the storage administrator can efficiently manage storage capacity plans and file migration. So, an efficient method for obtaining the tendency of files stored in file systems, that is, statistic information, is required. 
         [0003]    Main types of the statistic information to find the tendency of files stored in the file storage system are as follows. A first type is size distribution of each of pluralities of files and their chronological variation. A second type is distribution of extensions of each file and their chronological variation. A third type is distribution of update dates and times and access dates and times of each file and their chronological variation. A fourth type is a combination of these types, for example, how access dates and times of files with a certain extension are distributed. The statistic information of files is stored in, for example, a database or a table and users can refer to the statistic information via a GUI. 
         [0004]    An administrative user of a storage system needs to refer to metadata of files and collect them in order to obtain the statistic information. Regarding the conventional technology, there is a method, which is implemented by a statistic information collection program, for collecting metadata of the entire file system as a target, storing the collected information in, for example, a database or a table, and calculating the statistic information. If this method is employed, the statistic information collection program needs to periodically refer to the metadata of all files existing in the file system and it takes a lot of time to collect the metadata, which is not efficient. 
         [0005]    Therefore, there is a method executed by a metadata collection system for reporting a change of metadata to a system of a storage user when the metadata is changed, as a method for efficiently collecting the metadata (U.S. Pat. No. 7,890,551). According to this method, the latest metadata is efficiently collected by reporting an event of the metadata change to the user. 
         [0006]    Another conventional technique is a system for efficiently collecting any change of metadata from snapshots of the metadata (U.S. Pat. No. 7,693,864). According to this system, snapshots of a file system are periodically obtained and any difference between metadata before the change and metadata after the change can be obtained by comparing the metadata in the past in the snapshots with the present metadata. 
       CITATION LIST 
     Patent Literature 
       [0007]    PTL 1: U.S. Pat. No. 7,890,551 
         [0008]    PTL 2: U.S. Pat. No. 7,693,864 
       SUMMARY OF INVENTION 
     Technical Problem 
       [0009]    With the first conventional technique, changes of the metadata can be recognized, but how the metadata has changed cannot be recognized. So, it cannot determine how the changes of the metadata would influence the statistic information table. Therefore, every time the metadata changes, the system needs to refer to the metadata of the entire file system in order to update the statistic information table. 
         [0010]    Then, with the second conventional technique, the metadata can be obtained only when obtaining snapshots. So, it is difficult to collect the statistic information of the file system in real time. 
         [0011]    Therefore, it is an object of the present invention to provide a computer system capable of providing the latest statistics of a file system in real time. Another object of the present invention is to provide a computer system and file system management method capable of updating the statistics of the file system in real time without periodically collecting the statistic information of all files in the file system. 
       Solution To Problem 
       [0012]    So, the present invention is characterized in that when the file system is accessed, first statistic information before access processing on the file system is compared with second statistic information after the access processing; and if any difference exists between the first statistic information and the second statistic information, statistic results of the file system are updated based on the difference. 
       Advantageous Effects of Invention 
       [0013]    According to the present invention, a computer system capable of providing the latest statistics of the file system in real time can be provided. Particularly, it is possible to provide a computer system capable of updating the statistics of the file system in real time without periodically collecting the statistic information of all files in the file system. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0014]      FIG. 1  shows a block configuration diagram of a computer system according to an embodiment of the present invention. 
           [0015]      FIG. 2  is a block diagram showing the detailed configuration of a file storage system. 
           [0016]      FIG. 3  is an example of a policy table that is set by statistic information collection policy setting processing. 
           [0017]      FIG. 4  shows an example of a statistic information table in which the statistic information is of a type classified by file extensions. 
           [0018]      FIG. 5  shows an example of a statistic information table in which the statistic information is of a type classified by last access time to files. 
           [0019]      FIG. 6  shows an example of a statistic information table in which the statistic information is of a type classified by file sizes. 
           [0020]      FIG. 7  is a block diagram showing a data structure of metadata. 
           [0021]      FIG. 8  is an example of a metadata collection table. 
           [0022]      FIG. 9  is a policy setting GUI displayed on a screen of an input/output unit shown in  FIG. 1 . 
           [0023]      FIG. 10  is a display example of statistic results of a file system. 
           [0024]      FIG. 11A  is a flowchart for explaining statistic information collection processing. 
           [0025]      FIG. 11B  is a flowchart for explaining statistic information collection processing which follows the processing in  FIG. 11A . 
           [0026]      FIG. 12A  is a flowchart for explaining metadata collection processing. 
           [0027]      FIG. 12B  is a flowchart for explaining metadata collection processing which follows the processing in  FIG. 12A . 
           [0028]      FIG. 12C  is a flowchart for explaining metadata collection processing which follows the processing in  FIG. 12B . 
       
    
    
     DESCRIPTION OF EMBODIMENTS 
       [0029]    Next, an embodiment of the present invention will be explained.  FIG. 1  shows a block configuration diagram of a computer system according to an embodiment of the present invention. The computer system includes, as main components, a file storage system  20  and a management computer  11  for the file storage system  20 . One or more client computers  12  are connected to the file storage system. 
         [0030]    The file storage system  20  includes a management interface  201  and a data interface  202 . An interface  113  of the management computer  11  is connected to the management interface  201 . The client computer  12  is connected to the data interface  202 . The management computer  11  includes an arithmetic unit (CPU)  111 , a memory  112  having a management program  1121  of the file storage system  20 , and an I/O device  140  such as a screen for an administrative user. 
         [0031]      FIG. 2  is a block diagram showing the detailed configuration of the file storage system  20 . The file storage system includes a memory  204 , an arithmetic unit  203 , and a disk array system  205  in which a plurality of hard disk drives  2051  are arranged in an array. The memory  204  stores an OS of the file storage system, an application program, and file systems  2045  for files  70  stored in the disk array system. 
         [0032]    The application program is constructed as a changed metadata collection program  2044  for collecting changed metadata. The metadata is management information of files constituting a file system. The computer system uses the management information of the files as statistic information of the file system. Incidentally, the changed metadata collection program  2044  may be constructed as part of functions of the OS  204  or the file system  2045 . 
         [0033]    The changed metadata collection program  2044  has a function  20441  as an application program interface (API), a function  20443  collecting metadata, a function  20444  temporarily storing the metadata, and a function  20445  reporting updates to the file(s). Incidentally, for the sake of convenience, the function as a changed metadata collection application program interface (API) will be described as a changed metadata API ( 20441 ), the function collecting the metadata will be described as a metadata collection unit  20443 , the function temporarily storing the metadata will be described as a temporary metadata storage unit  20444 , and the function reporting updates to the files will be described as a file change notification unit  20445 . 
         [0034]    The memory  204  has a metadata collection target table  80  for specifying the types of metadata, which are targets to be collected, and file systems which are metadata collection targets, as an information management table that is necessary for the changed metadata collection program  2044  to fulfill the above-described functions. 
         [0035]    The memory  204  has a statistic information processing program as an application program of a different tier from that of the changed metadata collection program  2044  or as a function of the OS. The statistic information processing program executes various processing for, for example, generating and updating statistics of the file systems. For example, the statistic information processing program has a function setting a policy for collecting the statistic information of the file systems (a statistic information collection policy setting unit  30 ), a function collecting the statistic information (a statistic information collection unit  2042 ), a function collecting the statistic information and executing processing on the collected statistic information (the statistic information collection processing unit  2042 ), and a statistic information table  40 . 
         [0036]      FIG. 3  is an example of a policy table  30 A that is set by the statistic information collection policy setting unit  30 . The policy table  30 A is constituted from a file system name setting area  301  for specifying a file system (FS) which is a target to collect the statistic information from among file systems belonging to a storage area of the disk array system  205 , a statistic information type setting area  302 , and a definition information setting area  303 . 
         [0037]      FIG. 3  shows that examples of the types of the statistic information include file size classification, extension classification, and last access time classification. These pieces of information are provided from the metadata of files. 
         [0038]    Furthermore,  FIG. 3  shows that in the case of the file size classification, the file sizes are classified into less than 1 KB, 1 KB or more and less than 10 KB, and 10 KB or more; and in the case of the file extension classification, extensions are classified into Pdf, doc, and xls.  FIG. 3  also shows that in the case of the last access time classification, accesses to the files are managed daily. 
         [0039]      FIG. 4  shows an example of the statistic information table  40  for which the statistic information type is the file extension classification. The statistic information table  40 , including those hereinafter explained, is set for each file system which is a target to obtain the statistic information. The statistic information table  40  may integrate a plurality of file systems. An area for setting the present statistic results and an area for the statistic results in the past points in time (for example, 2011-07-30 12:00 and 2011-07-29 12:00) are set to the statistic information table  40 . The number of file counts of files with each extension and a total file size (total capacity) of files with the same extension are recorded at each management timing. 
         [0040]      FIG. 5  shows an example of the statistic information table  40  for which the statistic information type is classification by the last access time to files. Present and past statistic areas are set to the statistic information table  40  in the same manner as in  FIG. 4 . Then, the last file access time is managed for each of the present point in time (2011-07-31 at present) and the past points in time (for example, 2011-07-30 12:00 and 2011-07-29 12:00) at each management timing. This management is executed based on the number of counts of a plurality of files belonging to the classified last file access time, and the total capacity of the plurality of files. Assuming that the present date is Jul. 31, 2011, the number of counts [201] on [2011-07-31] which belongs to the [present] area is an accumulated value of files accessed by a user computer(s) after 2011-07-31 00:00 until the present time. The number of counts [145] on [2011-07-30] is an accumulated value of files accessed by the user computer(s) after 2011-07-30 00:00 until 2011-07-31 00:00. Incidentally, if a file accessed at present was also accessed one day before, one is subtracted from the number of counts [145] on [2011-07-30] and one is added to the number of counts [201] on [2011-07-31]. Values in vertical columns for [2011-07-30 12:00] and [2011-07-31 12:00] in the table are the number of counts and the total capacity for the last file access time at each point in time in the statistic information table  40 . 
         [0041]      FIG. 6  shows an example of the statistic information table  40  for which the statistic information type is the file size classification. An area for setting the statistic information at present and an area for the statistic information at the past points in time (for example, 2011-07-30 12:00 and 2011-07-29 12:00) are set to the statistic information table  40 . Then, a plurality of file size classification types of files belonging to the relevant file system which is the target to collect the statistic information are recorded in each area. The number of files and the total capacity are recorded for each file size classification type. 
         [0042]      FIG. 7  is a block diagram showing a data structure of metadata. The file storage system  20  uses the metadata, which are attributes of files, as necessary file statistic information (management information) to obtain the statistics of the file system. A file  70  has a metadata  701  area and a data body area  702 . The metadata includes a file name (extension)  7011 , file capacity  7012 , last file status update time  7013 , last file access time  7014 , last data update time  7015  (these three types of times corresponding to ctime, atime, and mtime, respectively, in Unix OS), a user ID  7016  of a file owner, and a group ID  7017  of the file owner. Incidentally, the statistic information of the file is not limited to the above-mentioned metadata, and information such as security information which is set to the relevant file (such as an encryption form, an encryption level, access limitations, and a RAID level), information about the use of the file (profile of a file user), and hierarchical information of the file may be included as metadata and any and all pieces of information about the attributes of a file can be used as the statistic information. 
         [0043]      FIG. 8  is an example of the metadata collection table  80 . The metadata collection table is constituted from an area  801  for setting a file system which is a target to collect the metadata, and an area  802  for setting the type of metadata which is a metadata target to be collected. This metadata collection table is set by the metadata collection unit  20443  based on the policy table  30 A. The metadata collection unit  20443  sets the type of metadata to be obtained from the file to the metadata collection table  80  based on the statistic information type which is set to the policy table with respect to the file system which is the target to collect the statistic information. For example, if the statistic information type in the policy table is the extension, the metadata to be obtained from the file is a file name. 
         [0044]    The aforementioned policy table  30 A ( FIG. 3 ) is set by the management program  1121  of the management computer  11  via the statistic information collection unit  30  for the file storage system  30  to the statistic information collection policy setting unit  30 . The policy is set by the administrative user to the management program  1121  of the management computer  11  via the input/output unit  140 . 
         [0045]      FIG. 9  is a policy setting GUI displayed on a screen of the input/output unit  140 . The reference numeral  1401  in  FIG. 9  indicates an area for inputting a file system which is a target to collect the statistic information and the reference numeral  1402  indicates an area for the administrative user to select whether the statistic information (file extension) should be displayed or not. The reference numeral  1403  indicates a check box for selecting the file extension as the statistic information. The reference numeral  1404  indicates an area for selecting whether the statistic information (last access time) should be displayed or not. The reference numeral  1405  indicates a check box for selecting the timing of the last access time. The reference numeral  1406  indicates an area for selecting whether the statistic information (file size) should be displayed or not. The reference numeral  1407  indicates an area for selecting a file size distribution border. 
         [0046]    The statistic results are displayed as shown in  FIG. 10 , for example, in a form of a pie chart for each statistic result (classification by the file extension and classification by the last access time) as proportions of the number of files with respect to each of the plurality of pieces of statistic information to the total number of files in the relevant file system. The statistic information is displayed on the screen of the input/output unit  140  for the management computer  11  after receiving the results of the statistic information collection unit  2042  for the file storage system. The reference numeral  1501  indicates a statistic display example when the statistic information type is the file extension; and the reference numeral  1502  indicates a statistic display example when the statistic information type is the last access time to files. 
         [0047]    Next, the operation of the computer system according to the present invention will be explained.  FIG. 11A  and  FIG. 11B  are flowcharts illustrating the operation of the statistic information collection unit  2042 . These flowcharts are actually executed by the CPU  203 ; however, the flowcharts will be explained by referring to the statistic information collection unit  2042 , which is a subject of the function of the statistic information collection program, as an execution subject in order to make the explanation of the flowcharts easier to understand. 
         [0048]    Firstly, as shown in  FIG. 11A , the statistic information collection unit  2042  refers to the statistic information collection policy table  30 A ( FIG. 3 ) which is set by the user (S 1101 ). Next, the statistic information collection unit  2042  either newly defines the statistic information table  40  for storing the statistic information to be collected or updates the definitions of the existing statistic information table, based on the statistic information collection policy which is set to the statistic information collection policy table  30 A (S 1102 ). 
         [0049]    The statistic information collection unit  2042  notifies the changed metadata collection API ( 20441 ) of a file system(s) as a target(s) to collect the statistic information, which is set to the statistic information collection policy table  30 A, as well as the type of the related metadata corresponding to a pattern of the statistic information to be collected from each file system (S 1103 ). In step S 1104 , changed metadata collection processing by the changed metadata collection program is executed. The details of this processing will be explained below with reference to the other flowchart. 
         [0050]    If any change of the related metadata happens with respect to a file belonging to the file system which is a statistic information collection target, the changed metadata collection API ( 20441 ) sends the related metadata before and after the change for the file, whose related metadata has been changed, to the statistic information collection unit  2042  as shown in  FIG. 11B  (S 1105 ). If the file is newly created, the related metadata before the change becomes “Null.” If the file is deleted, the related metadata after the change becomes “Null.” Incidentally, the notification of the related metadata change is made every time the related metadata of a file is changed, for example, every time the file is update, created, or deleted; however, changes during a specified interval may be collectively reported. 
         [0051]    The statistic information collection unit  2042  compares the related metadata before the change (old metadata) with the related metadata after the change (new metadata) and extracts necessary information to update the statistic information table (S 1106 ). Then, the statistic information collection unit updates the statistic information table based on the extracted information (S 1107 ). For example, if data is written to a certain file and its file size is changed (size before the change: 5 KB; and size after the change: 15 KB), the statistic information collection unit subtracts one from the number of file counts, which is 2340, for the file size (1 KB&lt;file size&lt;10 KB) belonging to the record [present] in the statistic information table ( FIG. 6 ), subtracts 5 KB from the total capacity of 10.2 MB, adds one to the number of file counts, which is 3224, for the file size (10 KB&lt;file size&lt;50 KB), and adds 15 KB to the total capacity of 77.2 MB. 
         [0052]    Furthermore, if the change of the related metadata is based on the creation of a new file, the statistic information collection unit  2042  adds one to the number of counts of files belonging to a record of that extension and adds the relevant file size to the total capacity based on the extension of the newly created file (see  FIG. 4 ). On the other hand, if the change of the related metadata is based on a file deletion, the statistic information collection unit  2042  subtracts one from the number of counts of files belonging to a record of the deleted file and subtracts the relevant file size from the total capacity based on the extension of the deleted file. 
         [0053]    Furthermore, if the related metadata is the last file access time, the statistic information collection unit  2042  refers to the last file access time before the change and updates the statistic information collection table  40 . For example, referring to  FIG. 5 , when receiving a change of the related metadata in a case where the present date is Jul.  31 ,  2011 , the statistic information collection unit  2042  adds one to [Number of Counts] in the area of [2011-07-31] at [Present] and adds the capacity of the file, whose metadata has been changed, to [Total Capacity]. Assuming that the previous access time for the relevant file was between [2011-07-30 00:00] and [2011-07-30 23:59], the statistic information collection unit  2042  subtracts one from [Number of Counts] for [2011-07-30] belonging to [present] and subtracts the file capacity from [Total Capacity]. 
         [0054]    In step S 1108 , the statistic information collection unit checks whether or not an instruction to halt the statistic information collection has been issued from the administrative user; and if an affirmative judgment is returned, the instruction to halt the metadata collection is issued to the changed metadata collection API (S 1109 ). If a negative judgment is returned, the processing returns to step S 1104  in  FIG. 11A . The halt instruction is executed by the management computer  11  when, for example, maintaining the file systems or the file storage system or changing the statistic information collection policy. When the processing of the flowchart is halted and then later resumed, the statistic information collection unit  2042  starts from step S 1101 . 
         [0055]    Next, processing for collecting the changed metadata will be explained.  FIG. 12A  to  FIG. 12C  are flowcharts for explaining the operation of the changed metadata collection processing program  2044  executing the metadata collection processing. Firstly, in S 1201 , the metadata collection unit  20443  receives information sent from the statistic information collection unit  2042  via the metadata collection API ( 20441 ), that is, a file system name, which is a statistic information collection target, and the type of the related metadata to be collected; and records them in the metadata collection target table  80 . 
         [0056]    The metadata collection unit  20443  refers to the metadata collection target table  80  and sets the file change notification unit  20445  so that the statistic information collection target file system will be a monitor target (S 1202 ). Next, the file change notification unit  20445  starts monitoring the statistic information collection target file system (S 1203 ). This monitoring is to check whether or not access from the client computer  12  is made to a file belonging to the monitor target file system. 
         [0057]    Then, the metadata collection unit  20443  checks, via the changed metadata collection API ( 20441 ), whether an instruction to halt the collection has been issued or not (S 1204 , S 1109 ). If the halt instruction has been issued, the metadata collection unit  20443  terminates the processing of the flowchart. If the halt instruction has not been issued, the file change notification unit  20445  checks the file system storage information area of the memory and checks whether or not the monitor target file system exists in the file storage system, in step S 1205 . If the file system does not exist, the file change notification unit  20445  terminates the processing of the flowchart. 
         [0058]    If a negative judgment is returned in step S 1204  and an affirmative judgment is returned in step S 1205 , the processing of the flowchart proceeds to step S 1206  ( FIG. 12B ); and the file change notification unit  20445  always analyzes accesses from the client computer  12  and judges whether a file which is a file access request target belongs to the monitor target file system and whether an access command causes a change in the metadata, for example, whether or not a write command has occurred when the file size is a collection target. Furthermore, if the last access time is the collection target, the file change notification unit  20445  judges whether or not a read command has occurred. 
         [0059]    If the file change notification unit  20445  returns an affirmative judgment in this step, the file change notification unit  20445  issues a command to the OS to have the file system temporarily stop writing/reading data to/from the file and notifies the metadata collection unit  20443  that the write/read request to write/read data to/from the file in the monitor target file system has occurred (S 1207 ). If the file change notification unit  20445  returns a negative judgment in step S 1206 , the processing returns to S 1204   
         [0060]    After receiving the notification from the file change notification unit  20445 , the metadata collection unit  20443  refers to the metadata collection target table  80  and identifies the type of the collection target metadata that is set for the file system to which the write/read request target file belongs. Then, the metadata collection unit  20443  reads the identified type of the related metadata from the file and saves and records it to a temporary storage area of the temporary metadata storage unit  20444  (S 1208 ). Furthermore, the metadata collection unit  20443  issues a command to the OS to cancel the file writing/reading halt. The related metadata recorded in the temporary metadata storage area is metadata before data was written to or read from the file according to the write/read command. 
         [0061]    The file change notification unit  20445  judges whether writing/reading of data to/from the file for which the write/read request was made has been completed or not (S 1209 ). If writing/reading of data to/from the file has been completed, the file change notification unit  20445  notifies the metadata collection unit  20443  that writing/reading of data to/from the file has been completed (S 1210 ). The file change notification unit  20445  monitors a write/read completion response from the OS, which controls commands from the client computer, to the client computer  12 . 
         [0062]    After receiving the notification, the metadata collection unit  20443  reads updated metadata from the file, to or from which data was written or read, reads old metadata, which has been saved to the temporary metadata storage area, and then compares the updated metadata with the old metadata (S 1211 :  FIG. 12C ). 
         [0063]    Subsequently, the metadata collection unit  20443  refers to the metadata collection target table  80  and judges whether any difference exists between the updated metadata and the pre-update metadata with respect to the metadata of the type defined as the metadata collection target (S 1212 ). If any difference exists, the metadata collection unit  20443  outputs the updated metadata and the old metadata via the metadata collection API ( 20441 ) to the statistic information collection unit  2042  (S 1213 ). If a negative judgment is returned in this step, the processing returns to S 1204 . 
         [0064]    If the file system is accessed and the statistic information about the file system changes before and after the access, the computer system always monitors the changes of the file system and updates the statistic results of the file system according to the changes as the result of the above-described processing. Therefore, it is possible to always maintain the statistics of the file system in the latest state. When this is performed, the load imposed by the updates on the computer system is not so large. 
         [0065]    In the aforementioned embodiment, the changed metadata collection program and the statistic information processing program are located in the file storage system, but they may be located in the management computer. Furthermore, the statistic information table  40  is used to record the statistic results relating to one type of metadata, but it may be used to record the statistic results relating to a plurality of types of metadata, for example, in a case where distribution of last file access time is recorded with respect to one extension. 
         [0066]    Furthermore, in the aforementioned embodiment, the difference in the metadata before and after the file access is checked by the changed metadata collection program; however, this check may be performed by the statistic information collection unit. Furthermore, the statistic information table is updated when a file is accessed; however, this update may be performed at certain time intervals, for example, every several hours. 
       REFERENCE SIGNS LIST 
       [0067]      11  Management computer 
         [0068]      12  Client computer 
         [0069]      20  File storage system 
         [0070]      70  File 
         [0071]      203  Arithmetic unit (controller) 
         [0072]      205  Disk array system 
         [0073]      2044  Changed metadata collection program 
         [0074]      2045  File system