Abstract:
This invention presents a method of constructing a storage network system that generates and stores information at the adoptive rate that matches the wired/wireless network data transfer rate, and automatically recovers lost data due to physical/functional failure of storages. This storage network system parses data and distributes parallelly to multiple storages for the purpose of reducing the storage access time. The amount of the storage access time reduction is inversely proportion to the number of storages that are accessed simultaneously. 
     This proposed storage network system also recovers lost data by utilizing the error correction information in the parsed data.

Description:
RELATED APPLICATIONS 
       [0001]    The present application is a continuation application of U.S. provisional patent application, Ser. No. US60/776,762, filed Feb. 24, 2006, included by reference herein and for which benefit of the priority date is hereby claimed. 
     
    
     FIELD OF THE INVENTION 
       [0002]    The present invention relates to the field of the wired/wireless networks where the instantaneous data transfer rate can vary without any prior notice, and, more particularly, to the field of Distributed Storage, Distributed Processing, and Parallel Processing. 
       BACKGROUND OF THE INVENTION 
       [0003]    Any network has bottlenecks that limit the overall performance, and this bottleneck is usually associated with storage access, and the interface between the Physical layer and the Data Link layer. Therefore, although the latest networking technology enables the data transfer between wireless channels with a multiple giga bytes per second rate, the overall system performance would be limited by the storage access time. When the particular storage holds the majority of data that a number of devices in the network system need to access simultaneously, the system overall performance degrades even more. 
         [0004]    One other deficiency of the current storage system is that it would require back-up files to restore any corrupted or lost data. The file backup/restoration task usually requires some technical knowledge that a general homeowner lacks. 
         [0005]    Currently, to improve the overall data transfer rate, network developers have put their efforts in reducing the storage access time, and the data transfer time between the storage units and the physical layer interface unit. This resulted in introducing fast I/O storage units, such as SATA Hard Drive, that reached the peak data rate of 300 MBps (or 2.4 Giga bits per second). 
         [0006]    In addition, the storage manufactures produce simple backup devices, such as CDROM and external hard drive to ease the file backup/restoration task for the general users. However, these devices are mainly to backup PC data, and currently there is no general method that would allow a homeowner to backup the files stored in various devices in a home. 
         [0007]    The general shortcoming of the current network developers&#39; solutions is that these solutions do not resolve the basic disparity between the storage access rate and data transfer rate of the physical medium (channels). This is because the average data access rate of the SATA Hard Drive is less than ½, actually close to ⅓, of the peak data access rate due to the burstiness of the storage access pattern, which is caused by the physical/logical partition of the data sectors, the size of the caches, and the overhead in the storage access protocol including the additional seek time. The seek time becomes a dominating factor if a system executes a multiple read operations from a single storage. Furthermore, since there are other physical limitations that associate with various mechanical components in the storage units, the access rate cannot improve indefinitely. 
         [0008]    It is therefore an object of the invention to create a wireless network that sustains the maximum data generation and consumption rate that matches the overall data transfer rate of the network 
         [0009]    It is another object of the invention to create a wireless network whose data generation and consumption rate that are not degraded by the slowest storage device. 
         [0010]    It is another object of the invention to create a wireless network that reduces/minimizes/annihilates the network system performance degradation due to the bottleneck on the storage access by partitioning a complete set of data into a multiple sections, and then distributing these sections into different storages/memories 
         [0011]    It is another object of the invention to create a wireless network system that can recover the data that were lost due to physical/logical failures. 
         [0012]    It is therefore an object of the invention to create a wireless network system that can recover last updated data that were lost due to physical/logical failures. 
       SUMMARY OF THE INVENTION 
       [0013]    In accordance with the present invention, there is provided a method of achieving the optimum data generation and consumption rate that would match the data transfer rate of the network system, and a method of constructing a storage network system that automatically recovers lost data due to physical/functional storage failures. 
         [0014]    This invention also presents a way of constructing a storage network system that parses data and distributes parallel to multiple storages for the purpose of reducing the storage access time. 
         [0015]    The amount of the storage access time reduction is inversely proportional to the number of storages that are accessed simultaneously. 
         [0016]    Furthermore, this proposed storage network system recovers lost data by utilizing the error correction information in the parsed data. 
         [0017]    When the data is parallelly retrieved from multiple storages, the storage network system reconstructs the original data by assembling the parsed data followed by performing an error correction task, which recovers any lost parsed data due to storage failures. 
         [0018]    The amount of lost parsed data that the system recovers is dependent on the amount of error recovery redundancy in the parsed data. 
         [0019]    After the storage network system recovers the lost date, it informs the user about the failed storage units, which the user can replace or remove later. 
         [0020]    Thus, unlike the traditional backup system that may hold stale data, since this storage network system recovers the lost data that is the last written data, there would be no loss of information. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0021]    A complete understanding of the present invention may be obtained by reference to the accompanying drawings, when considered in conjunction with the subsequent, detailed description, in which: 
           [0022]      FIG. 1  is a detail view of a partition and distribution (p&amp;d) processor block where the data is parsed and parallelly distributed to multiple storage units; 
           [0023]      FIG. 2  is a detail view of a wireless connection for the partition and distributing (p&amp;d) processor for both storing and retrieving data; 
           [0024]      FIG. 3  is a detail view of the normalized access time of each storage unit that the partition and distribution (p&amp;d) processor may access. This table is referred as the storage capability table (sct); and 
           [0025]      FIG. 4  is a detail view of a storage mapping table (smt), which contains information on how the data have been partitioned and distributed. 
       
    
    
       [0026]    For purposes of clarity and brevity, like elements and components will bear the same designations and numbering throughout the Figures. 
       DESCRIPTION OF THE PREFERRED EMBODIMENT 
       [0027]    When the data generation rate and the consumption rate of network devices match the communication channel data transfer rate in the network, the overall performance of the network should be able to achieve the optimal point. 
         [0028]    However, in many cases, even when the data generation rate matches with the data transfer rate in the network, the optimum performance level would not be achievable unless the system can coordinate the activities of the data to transmitters and the receivers, the availability the communication channels, and the accessibility of the data storage units. 
         [0029]    The recent development in the wired/wireless PHY technologies (such as HDMI, UWB, 802.11n) enables transfer of data from 10 Mbps to near 5 Gbps, and any network that contains devices with these variety of PHY technologies needs to support the data generation and consumption rates that is comparable to the channel bandwidth to achieve the optimal performance. The data generation and consumption rates depend on the end devices that provide visual/audio/data display  201  units, or storage units. Since the visual/audio/data display  201  units are output only devices that usually perform a single function, the data consumption rate is fixed. However, storage units need to support the variable data access rate that matches not only the consumption/generation rate of the end units, but also the data transfer rate of the physical channels. 
         [0030]    Previously, the distributed storage or the distributed memory indicates a way of storing information such that a complete set of information is stored in a single physical unit that may also hold multiple sets of complete information. The distributed storage/memory also means that a file server has the table that maps where data/information/software are stored in which storage. 
         [0031]    Therefore, all information retrieve requires the file server to behave as the gatekeeper while the information passes through the file server, and the file server becomes the system bottleneck. 
         [0032]    This invention presents a method that shows how to construct a network system that dynamically adjusts the overall storage access rate, so that the data access rate matches the data rate required for optimum operation of the visual, audio, and data display  201  units in the network. 
         [0033]    This method optimizes the network operation by reducing/minimizing/annihilating the network system performance degradation due to the bottleneck on the storage access by applying the Partition &amp; Distribution (P&amp;D) of data method. 
         [0034]    The P&amp;D method simply means to partition a complete set of data into a multiple sections, and distribute these sections into different storages/memories. 
         [0035]    There are three advantages of applying this method: 
         [0036]    1) When a network looses storage due to a physical/logical failure, the network system can recover the data. 
         [0037]    2) Since each storage operates independently, and since there are multiple storages that operate simultaneously, the data generation and consumption rate is computed by: 
         [0000]      Max Data generation/consumption rate= S  (data rate of each storage unit). 
         [0038]    3) Since, the data generation/consumption task is distributed, the computation is also distributed. Therefore, the slowest storage or the slowest file server does not degrade the overall network system performance. In general, the slowest device in the system dictates the worst case system performance. However, the worst case system performance with the P&amp;D method is the average of the performance of all the devices in the network. 
         [0039]    The P&amp;D method can be implemented in software, hardware or in both. The important point is the selection of the Partition algorithm. The purpose of partitioning the data is to be able to store the data into many storage elements simultaneously, and to retrieve the data from the same storage elements simultaneously. The Distribution algorithm is based on the access rate and the size of each storage. 
         [0040]    The P&amp;D process consists of 4 steps: 
         [0041]    Step 1) Error Correction (ex. Reed-Solomon) encoding 
         [0042]    Step 2) Byte-wise or Word-wise or Block-wise permutation 
         [0043]    Step 3) Byte-wise or Word-wise or Block-wise partition 
         [0044]    Step 4) Distribution of the data to multiple storage units 
         [0045]    In Step 1: 
         [0046]    The data are broken into a manageable size (ex. 64 Bytes), and coded with an error correction code. This coding allows the system to recover the data when storage failures (defects) occur. 
         [0047]    In Step 2: 
         [0048]    The RS encoded data is permutated as in the following examples. The permutation algorithm needs to match the partition algorithm in Step 3. 
         [0049]    In Step 3: 
         [0050]    The data partition should be done in such a way that the system could recover the lost data when a storage failure occurs. The system should use two data recovery algorithms: an error correction method or an error erasure method. 
         [0051]    The system applies the error correction method for the general error correction when there is burst error due to momentary transfer failures due to noise, interferences, or bad memory sectors in the storage units. 
         [0052]    The system applies the error erasure method when the system detects physically failed storage units. Since, when the error locations are known, the error correction capability with the RS error erasure method allows recovery of twice of the data compared to the error correction method, the system would be able to tolerate multiple storage failures without loosing data. 
         [0053]    In Step 4: 
         [0054]    The system determines the size of the distributed data for each storage unit based on the speed and the capacity of each storage unit so that the access time of all the storage units for storing/retrieving the portion of the distributed data are the same. 
         [0055]    To retrieve data, the process works in reverse order that was described in the Partition and Distribution (P&amp;D) steps. The system retrieves data from the storage units in parallel, assembles the data, and executes either the error-correction code and/or the error-erasure code. 
         [0056]    This description is an example of how to implement the Partition and Distribution (P&amp;D) function. 
         [0057]      FIG. 1  ( 100 ) shows the block diagram of P&amp;D. 
         [0058]    In  FIG. 1  ( 100 ), the first block ( 101 ) is the error-correction block. This block receives or supplies data to the Wired/Wireless Interface block ( 203 ) in  FIG. 2  ( 200 ). The data coming into this block ( 100 ) is encoded with an error correcting codes ( 101 ). The error-correction encoded data goes into the Byte Interleave ( 102 ) to be able to recover data if any storage units experience physical failures. 
         [0059]    The byte-interleaved data is partitioned ( 103 ) into sections to support high-speed data generation and consumption rate by means of parallel/simultaneous access of multiple Storage Units ( 105 ). The partitioned data ( 103 ) is distributed by the Distribution ( 104 ) according to the discussion presented in the previous section. 
         [0060]    The P&amp;D retrieves the data from the multiple Storage Units ( 105 ) and processes the data in the reverse order to provide information to the Wired/Wireless Interface block ( 203 ). The P&amp;D Collects ( 104 ) the data, assembles ( 103 ) the data, and conducts the reverse byte interleaving process ( 102 ). The RS Encoder/Decode ( 101 ) block performs either an error-correction operation or an error-erasure-correction operation on the data it receives from the Byte Interleave ( 102 ) block. The system software instructs the RS Encoder/Decoder ( 101 ) to perform an error-correction operation when there are no hard failed (physically damaged or unusable) Storage Units ( 105 ), or instructs to perform an error-erasure-correction operation if there are any damaged Storage Units ( 105 ). This is because the system can inform the RS Encoder/Decoder ( 101 ) with the location of errors that are related to the damaged Storage Units, and the RS Encoder/Decoder ( 101 ) can double the data recovery efficiency by utilizing the information on which bytes or bits are expected to fail due to hard failures on the Storage Units ( 105 ). 
         [0061]      FIG. 2  ( 200 ) shows the Partition and Distribution (P&amp;D) process for both storing and retrieving data. 
         [0062]    In  FIG. 2  ( 200 ), the Display  201  ( 201 ) represents where the data is consumed, and the SetTop-Box ( 202 ) represents where the data is generated. The Display  201  ( 201 ) may be more than a single unit where total data demand rate over-exceeds the access rate of the fastest storage unit, and also the data generation rate may over-exceed the access rate of the fastest storage unit. 
         [0063]    The Wired/Wireless Network Unit  210  (NU) ( 203 ) is the gateway to the Storage Units ( 205 ). The Storage Units ( 205 ) are shared amongst all Network Units (NU) ( 210 ), and each NU ( 210 ) consists of the Wired/Wireless Interface ( 203 ) and P&amp;D ( 204 ), and a network system may comprise of multiple Nus ( 210 ). The Wired/Wireless Interface ( 203 ) functions as a protocol converter that may link between UWB and USB, or 820.11n and 1394 etc. 
         [0064]    The P&amp;D holds the Storage Capability Table  300  (SCT) ( 300 ) and the Storage Mapping Table  400  (SMT) ( 400 ) along with the Partition &amp; Distribution function that is described previously. The SMT ( 400 ) contains information on how the data has been partitioned and distributed. The SCT ( 300 ) indicates the normalized access time of each storage unit that the P&amp;D ( 100 ) may access. Thus the P&amp;D ( 100 ) can synchronize the operation of the storage units for the maximum performance. 
         [0065]      FIG. 3  shows an example of the SCT table ( 300 ). 
         [0066]    In  FIG. 3  ( 300 ), the Normalized Speed  302  ( 302 ) of XYZ- 3  ( 312 ) is 1 since XYZ- 3  ( 312 ) is the slowest storage unit. The Normalized Speed  302  ( 302 ) of XYZ- 1  ( 310 ) is 4, which indicates that the access time of this storage unit is 4 times faster than the XYZ- 3  ( 312 ). The Normalized Speed  302  ( 302 ) for each storage unit stays the same unless the system adds a new storage unit that is slower than the slowest storage unit that was in the system previously. 
         [0067]    The Space Available ( 303 ) indicates how many Kbytes of memory space has not been used. In this table, XYZ- 3  ( 312 ) has 6 giga bytes available memory space. 
         [0068]    The # of NU Serving ( 304 ) indicates how many Wired/Wireless Network Units are connected to the particular storage unit. 
         [0069]    The Effective Speed  305  ( 305 ) is computed by dividing the Normalized Speed  302  ( 302 ) by the # of NU Serving ( 304 ). 
         [0000]      Effective Speed  305  ( 305 )=Normalized Speed  302  ( 302 )/# of NU Serving ( 304 ). 
         [0070]    The NU ( 210 ) makes the Distribution decision based on this table for the maximum performance, for the NU ( 210 ) with this table recognizes that the storage unit XYZ- 3  ( 312 ) is its own dedicated storage unit with largest available space, but has the slowest access time. However, since this storage unit is not accessed by any other NU ( 210 ), the effective speed  305  is faster than the storage unit XYZ- 1  ( 310 ). 
         [0071]    The storage unit XYZ- 2  ( 311 ) has the fastest Effective Speed  305 , but it has the smallest space available. Thus the NU ( 210 ) may decide to distribute the majority of its data to XYZ- 1  ( 310 ) and XYZ- 3  ( 312 ), and the most timing critical data to XYZ- 2  ( 311 ). The timing critical data in XYZ- 2  ( 311 ) may be the first 100 kilo bytes of the information that needs to start the process immediately, and the NU ( 210 ) retrieves subsequent information while the Wired/Wireless Interface processes the first 100 kilo bytes of data. 
         [0072]    According to SCT Table ( 300 ), the effective access time of data is 5/3 of the normalized speed  302  since the storage units XYZ- 1  ( 311 ) and XYZ- 3  ( 312 ) are accessed simultaneously. 
         [0073]    Therefore, this arrangement supports the network data transfer rate that is 60% faster than the slowest storage unit XYZ- 3  ( 312 ). 
         [0074]    The P&amp;D ( 100 ), with the instruction from the software, may increase the SCT ( 300 ) and SMT ( 400 ) tables to accommodate more storage units to improve the overall storage access speed via simultaneous and parallel operations on more storage units. The P&amp;D ( 100 ) optimizes the overall access time by preserving the space in XYZ- 2  ( 312 ) for the future high-speed access. 
         [0075]      FIG. 4  is an example of the SMT table ( 400 ). 
         [0076]    In SMT table ( 400 ), the system address  401  is the reference address  401  that maps to the physical storage addresses. The total data size is 105 mega bytes, and the first 5 mega bytes are stored in XYZ- 2  ( 402 ) at address  401  A 0 - 2  ( 412 ) for fast access as it was discussed previously. The majority of data is stored in the XYZ- 1  ( 401 ) and XYZ- 3  ( 403 ) at the address  401  location A 1 - 1  ( 411 ) and A 1 - 3  ( 413 ). The 100 mega-byte information may be stored in multiple sectors in each storage units, but the address  401  mapping to the storage unit is handled by the DMA function in the system. 
         [0077]    Since other modifications and changes varied to fit particular operating requirements and environments will be apparent to those skilled in the art, the invention is not considered limited to the example chosen for purposes of disclosure, and covers all changes and modifications which do not constitute departures from the true spirit and scope of this invention. 
         [0078]    Having thus described the invention, what is desired to be protected by Letters Patent is presented in the subsequently appended claims.