Source: https://patents.google.com/patent/US10044807B2/en
Timestamp: 2019-04-24 16:09:05+00:00

Document:
2017-12-12 Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION reassignment INTERNATIONAL BUSINESS MACHINES CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: DHUSE, GREG R.
A computing device includes an interface configured to interface and communicate with a dispersed storage network (DSN), a memory that stores operational instructions, and processing circuitry operably coupled to the interface and to the memory. The processing circuitry is configured to execute the operational instructions to perform various operations and functions. The computing device dispersed error encodes a data segment to produce a set of encoded data slices (EDSs) and selects a subset of a set of storage units (SUs). Then, the computing device transmits list slice requests to the subset of the set of SUs and receive list slice responses from at least some of them. The computing device determines a highest current revision level based on list slice responses and select a revision level for the set of EDSs. The computing device generates a set of checked revision slice requests and transmit them to the set of SUs.
The present U.S. Utility Patent Application claims priority pursuant to 35 U.S.C. § 120 as a continuation-in-part (CIP) of U.S. Utility patent application Ser. No. 15/673,978, entitled “STORING DATA IN A DISPERSED STORAGE NETWORK,” filed Aug. 10, 2017, pending, which claims priority pursuant to 35 U.S.C. § 120 as a continuation of U.S. Utility application Ser. No. 14/876,154, entitled “STORING DATA IN A DISPERSED STORAGE NETWORK,” filed Oct. 6, 2015, issued as U.S. Pat. No. 9,774,684 on Sep. 26, 2017, which claims priority pursuant to 35 U.S.C. § 119(e) to U.S. Provisional Application No. 62/086,611, entitled “MAINTAINING DATA INTEGRITY IN A DISPERSED STORAGE NETWORK” filed Dec. 2, 2014, all of which are hereby incorporated herein by reference in their entirety and made part of the present U.S. Utility Patent Application for all purposes.
U.S. Utility application Ser. No. 14/876,154 also claims priority pursuant to 35 U.S.C. § 120 as a continuation-in-part (CIP) of U.S. Utility application Ser. No. 14/792,577, entitled “DISPERSED STORAGE WRITE PROCESS,” filed Jul. 6, 2015, issued as U.S. Pat. No. 9,354,974 on May 31, 2016, which is a continuation of U.S. Utility application Ser. No. 13/863,475, entitled “DISPERSED STORAGE WRITE PROCESS,” filed Apr. 16, 2013, issued as U.S. Pat. No. 9,092,140 on Jul. 28, 2015, which is a continuation of U.S. Utility application Ser. No. 12/797,025, entitled “DISPERSED STORAGE WRITE PROCESS,” filed Jun. 9, 2010, issued as U.S. Pat. No. 8,595,435 on Nov. 26, 2013, which claims priority pursuant to 35 U.S.C. § 119(e) to U.S. Provisional Application No. 61/230,038, entitled “DISPERSED STORAGE NETWORK VERSION SYNCHRONIZATION,” filed Jul. 30, 2009, all of which are hereby incorporated herein by reference in their entirety and made part of the present U.S. Utility Patent Application for all purposes.
U.S. Utility application Ser. No. 13/863,475 also claims priority pursuant to 35 U.S.C. § 120 as a continuation-in-part (CIP) patent application of U.S. Utility application Ser. No. 12/080,042, entitled, “REBUILDING DATA ON A DISPERSED STORAGE NETWORK,” filed Mar. 31, 2008, issued as U.S. Pat. No. 8,880,799 on Nov. 4, 2014, which is a continuation-in-part (CIP) of U.S. Utility application Ser. No. 11/973,542, entitled “ENSURING DATA INTEGRITY ON A DISPERSED STORAGE GRID,” filed Oct. 9, 2007; and is a continuation-in-part (CIP) of U.S. Utility application Ser. No. 11/403,391, entitled “SYSTEM FOR REBUILDING DISPERSED DATA,” filed Apr. 13, 2006, issued as U.S. Pat. No. 7,546,427 on Jun. 9, 2009, which is a continuation-in-part (CIP) of U.S. Utility application Ser. No. 11/241,555, entitled “SYSTEMS, METHODS, AND APPARATUS FOR SUBDIVIDING DATA FOR STORAGE IN A DISPERSED DATA STORAGE GRID,” filed Sep. 30, 2005, issued as U.S. Pat. No. 7,953,937 on May 31, 2011, all of which are hereby incorporated herein by reference in their entirety and made part of the present U.S. Utility Patent Application for all purposes.
Within prior art data storage systems, sometimes when data (e.g., updated data) is to be written to storage, a check may be performed to determine a current version of prior corresponding data. the prior art does not provide adequate means by which a determination of the current version of that prior corresponding data without significant overhead, consumption of network resources, etc.
FIG. 12 is a diagram illustrating an embodiment of a method for execution by one or more computing devices in accordance with the present invention.
In some examples, note that dispersed or distributed storage network (DSN) memory includes one or more of a plurality of storage units (SUs) such as SUs 36 (e.g., that may alternatively be referred to a distributed storage and/or task network (DSTN) module that includes a plurality of distributed storage and/or task (DST) execution units 36 that may be located at geographically different sites (e.g., one in Chicago, one in Milwaukee, etc.). Each of the SUs (e.g., alternatively referred to as DST execution units in some examples) is operable to store dispersed error encoded data and/or to execute, in a distributed manner, one or more tasks on data. The tasks may be a simple function (e.g., a mathematical function, a logic function, an identify function, a find function, a search engine function, a replace function, etc.), a complex function (e.g., compression, human and/or computer language translation, text-to-voice conversion, voice-to-text conversion, etc.), multiple simple and/or complex functions, one or more algorithms, one or more applications, etc.
FIG. 9 is a diagram of an example of a distributed storage and task processing in accordance with the present invention. This diagram includes a diagram of an example of the distributed computing system performing a dispersed or distributed storage network (DSN) operation. The distributed computing system includes a DS (distributed storage) client module 34 (which may be in user device 14 and/or in computing device 16 of FIG. 1), a network 24, a plurality of SUs 1-n that includes two or more SUs 36 of FIG. 1 (which form at least a portion of DSN memory 22 of FIG. 1), a managing unit (not shown), and an integrity processing unit (not shown). The DS client module 34 includes an outbound DS processing section 80 and an inbound DS processing section 82. Each of the SUs 1-n includes a controller 86, a processing module 84, memory 88, a DT (distributed task) execution module 90, and a DS client module 34.
In an example of operation, the DS client module 34 receives data 92 and one or more tasks 94 to be performed upon the data 92. The data 92 may be of any size and of any content, where, due to the size (e.g., greater than a few Terabytes), the content (e.g., secure data, etc.), and/or task(s) (e.g., MIPS intensive), distributed processing of the task(s) on the data is desired. For example, the data 92 may be one or more digital books, a copy of a company's emails, a large-scale Internet search, a video security file, one or more entertainment video files (e.g., television programs, movies, etc.), data files, and/or any other large amount of data (e.g., greater than a few Terabytes).
Within the DS client module 34, the outbound DS processing section 80 receives the data 92 and the task(s) 94. The outbound DS processing section 80 processes the data 92 to produce slice groupings 96. As an example of such processing, the outbound DS processing section 80 partitions the data 92 into a plurality of data partitions. For each data partition, the outbound DS processing section 80 dispersed storage (DS) error encodes the data partition to produce encoded data slices and groups the encoded data slices into a slice grouping 96. In addition, the outbound DS processing section 80 partitions the task 94 into partial tasks 98, where the number of partial tasks 98 may correspond to the number of slice groupings 96.
The outbound DS processing section 80 then sends, via the network 24, the slice groupings 96 and the partial tasks 98 to the SUs 1-n of the DSN memory 22 of FIG. 1. For example, the outbound DS processing section 80 sends slice group 1 and partial task 1 to SU 1. As another example, the outbound DS processing section 80 sends slice group #n and partial task #n to SU #n.
Each SU performs its partial task 98 upon its slice group 96 to produce partial results 102. For example, SU #1 performs partial task #1 on slice group #1 to produce a partial result #1, for results. As a more specific example, slice group #1 corresponds to a data partition of a series of digital books and the partial task #1 corresponds to searching for specific phrases, recording where the phrase is found, and establishing a phrase count. In this more specific example, the partial result #1 includes information as to where the phrase was found and includes the phrase count.
Upon completion of generating their respective partial results 102, the SUs send, via the network 24, their partial results 102 to the inbound DS processing section 82 of the DS client module 34. The inbound DS processing section 82 processes the received partial results 102 to produce a result 104. Continuing with the specific example of the preceding paragraph, the inbound DS processing section 82 combines the phrase count from each of the SUs 36 to produce a total phrase count. In addition, the inbound DS processing section 82 combines the ‘where the phrase was found’ information from each of the SUs 36 within their respective data partitions to produce ‘where the phrase was found’ information for the series of digital books.
In another example of operation, the DS client module 34 requests retrieval of stored data within the memory of the SUs 36 (e.g., memory of the DSN memory 22). In this example, the task 94 is retrieve data stored in the memory of the DSN memory. Accordingly, the outbound DS processing section 80 converts the task 94 into a plurality of partial tasks 98 and sends the partial tasks 98 to the respective SUs 1-n.
In response to the partial task 98 of retrieving stored data, a SU 36 identifies the corresponding encoded data slices 100 and retrieves them. For example, SU #1 receives partial task #1 and retrieves, in response thereto, retrieved slices #1. The SUs 36 send their respective retrieved slices 100 to the inbound DS processing section 82 via the network 24.
The inbound DS processing section 82 converts the retrieved slices 100 into data 92. For example, the inbound DS processing section 82 de-groups the retrieved slices 100 to produce encoded slices per data partition. The inbound DS processing section 82 then DS error decodes the encoded slices per data partition to produce data partitions. The inbound DS processing section 82 de-partitions the data partitions to recapture the data 92.
FIG. 10A-B are a schematic block diagram of another embodiment of a dispersed storage network that includes a set of storage units (SUs) 1-n, the network 24 of FIG. 1, and the outbound distributed storage (DS) processing 80 of FIG. 9 and/or computing device 16 of FIG. 1. Note that such operations, functions, etc. as described herein as being performed by such outbound DS processing 80 may alternatively be performed by computing device 16. Each SU may be implemented utilizing the SU 36 of FIG. 1. The DSN functions to store multiple revisions of a data object, where each revision of the data object is dispersed storage error encoded to produce a plurality of sets of encoded data slices that are stored in the set of SUs 1-n.
FIG. 10A includes initial steps of an example of operation of the storing of the multiple revisions of the data object, where the outbound DS processing 80 receives a store request 1010, where the store request 1010 includes one or more of the data object for storage, an identifier of the data object, a revision level of the data object, and a requesting entity identifier (ID). Having received the data object for storage, the outbound DS processing 80 divides the data into one or more data segments in accordance with a data segmentation scheme. The data segmentation scheme includes at least one of evenly dividing the data object to produce a fixed number of data segments and/or dividing the data object by a predetermined data segment size.
For each data segment, the outbound DS processing 80 dispersed storage error encodes the data segment to produce a set of encoded data slices (e.g., slices 1-n). Having produced the set of encoded data slices, the outbound DS processing 80 selects a subset of the set of SUs. For example, the outbound DS processing 80 selects SUs associated with a most favorable level of storage reliability. As another example, the outbound DS processing 80 selects a number of SUs of the subset of the set of SUs based on a desired level of confidence for existing revision level determination. The number may be at least one of a predetermined number, a received number, a determined number based on the desired level of confidence, and/or a number recovered via a look up.
Having selected the subset of the set of SUs, the outbound DS processing 80 issues, via the network 24, list slice requests to the subset of the set of SUs, where each list slice request includes a slice name associated with the set of encoded data slices. For example, the outbound DS processing 80 issues list slice requests 2-3 to the SUs 2-3 when two SUs meet the desired level of confidence.
Having issued the list slice request, the outbound DS processing 80 receives list slice responses from the subset of the set of SUs, where each list slice response indicates a slice name and revision level corresponding to each stored encoded data slice of the requested slice name. Having received the list slice responses, the outbound DS processing 80 determines a highest current revision level based on the received list slice responses. The determining includes selecting a highest revision level of all responses and selecting a highest common revision level. For example, the outbound DS processing 80 indicates that revision level 3 is the highest revision level when receiving a list slice responses 2-3 that both indicate that revision level 3 is the highest revision level.
FIG. 10B includes illustrates further steps of the example of operation of the storing of the multiple revisions of the data object, where the outbound DS processing 80 selects a revision level for the set of encoded data slices. The selecting includes one of selecting one higher than the highest current revision level when storing new data and selecting a current revision level when overwriting data. Alternatively, the outbound DS processing 80 selects any revision level that is not currently in use when storing the new data.
Having selected the revision level, the outbound DS processing 80 generates a set of checked revision write slice requests, where each request includes one or more of a transaction number, and encoded data slice of the set of encoded data slices, a slice name associated with the encoded data slice, the highest revision level, and the selected revision level. Having generated the set of checked revision write slice requests, the outbound DS processing 80 sends, via the network 24, the set of checked revision write slice requests to the set of SUs 1-n. Alternatively, or in addition to, the outbound DS processing 80 issues a commit request when receiving a sufficient number of favorable checked revision write slice responses.
In an example of operation and implementation, a computing device (e.g., computing device 16, which may be implemented to include outbound DS processing 80) includes an interface configured to interface and communicate with a dispersed or distributed storage network (DSN), a memory that stores operational instructions, and a processing module, processor, and/or processing circuitry operably coupled to the interface and memory. The processing module, processor, and/or processing circuitry is configured to execute the operational instructions to perform various operations, functions, etc. In some examples, the processing module, processor, and/or processing circuitry, when operable within the computing device based on the operational instructions, is configured to perform various operations, functions, etc. In certain examples, the processing module, processor, and/or processing circuitry, when operable within the computing device is configured to perform one or more functions that may include generation of one or more signals, processing of one or more signals, receiving of one or more signals, transmission of one or more signals, interpreting of one or more signals, etc. and/or any other operations as described herein and/or their equivalents.
In an example of operation and implementation, the computing device is configured to dispersed error encode a data segment associated with a data object to produce a set of encoded data slices (EDSs). Note that the data object is segmented into a plurality of data segments, and the data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of EDSs. Also, note that the set of EDSs to be distributedly stored among a plurality of storage units (SUs) within the DSN. The computing device is also configured to select a subset of a set of storage units (SUs). note that the subset of the set of SUs having a most favorable reliability level among the set of SUs. The computing device is also configured to transmit list slice requests to the subset of the set of SUs and to receive list slice responses from at least some of the set of SUs. Then, the computing device is also configured to determine a highest current revision level based on list slice responses and to select a revision level for the set of EDSs. The computing device is also configured to generate a set of checked revision slice requests that includes the highest current revision level based on list slice responses and the revision level for the set of EDSs. Also, the computing device is also configured to transmit the set of checked revision slice requests to the set of SUs.
In some examples, the computing device is also configured to generate the list slice requests to include slice names associated with corresponding EDSs of the set of EDS and to transmit the list slice requests to the subset of the set of SUs.
In some other examples, the computing device is also configured to extract revision levels from the list slice responses. The computing device is also configured to determine the highest current revision level of the set of EDSs in accordance with an approach that is based on a predetermination, a historical storage reliability level, a performance goal, selecting the highest revision level, selecting a common highest revision level, and/or selecting a majority highest revision level.
In even other examples, the computing device is also configured to select the revision level for the set of EDSs to be greater than the highest current revision level when storing new data. In even other examples, the computing device is also configured to generate a checked revision write slice request of the set of checked revision slice requests to include a transaction number, an EDS of the set of EDSs a slice name associated with the EDS of the set of EDSs, the highest current revision level, and/or the revision level.
In some examples, with respect to a data object, the data object is segmented into a plurality of data segments, and a data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce a set of encoded data slices (EDSs) (e.g., in some instances, the set of EDSs are distributedly stored in a plurality of storage units (SUs) within the DSN). In some examples, the set of EDSs is of pillar width. Also, with respect to certain implementations, note that the decode threshold number of EDSs are needed to recover the data segment, and a read threshold number of EDSs provides for reconstruction of the data segment. Also, a write threshold number of EDSs provides for a successful transfer of the set of EDSs from a first at least one location in the DSN to a second at least one location in the DSN. The set of EDSs is of pillar width and includes a pillar number of EDSs. Also, in some examples, each of the decode threshold, the read threshold, and the write threshold is less than the pillar number. Also, in some particular examples, the write threshold number is greater than or equal to the read threshold number that is greater than or equal to the decode threshold number.
Note that the computing device as described herein may be located at a first premises that is remotely located from a second premises associated with at least one other SU, dispersed storage (DS) unit, computing device, at least one SU of a plurality of SUs within the DSN (e.g., such as a plurality of SUs that are implemented to store distributedly a set of EDSs), etc. In addition, note that such a computing device as described herein may be implemented as any of a number of different devices including a managing unit that is remotely located from another SU, DS unit, computing device, etc. within the DSN and/or other device within the DSN, an integrity processing unit that is remotely located from another computing device and/or other device within the DSN, a scheduling unit that is remotely located from another computing device and/or SU within the DSN, and/or other device. Also, note that such a computing device as described herein may be of any of a variety of types of devices as described herein and/or their equivalents including a DS unit and/or SU included within any group and/or set of DS units and/or SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, and/or a video game device, and/or any type of computing device or communication device. Also, note also that the DSN may be implemented to include and/or be based on any of a number of different types of communication systems including a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), and/or a wide area network (WAN). Also, in some examples, any device configured to support communications within such a DSN may be also be configured to and/or specifically implemented to support communications within a satellite communication system, a wireless communication system, a wired communication system, a fiber-optic communication system, and/or a mobile communication system (and/or any other type of communication system implemented using any type of communication medium or media).
FIG. 11 is a flowchart illustrating another example of storing data in accordance with the present invention. The method 1100 begins or continues at the step 1110 where a processing module (e.g., of a distributed storage (DS) client module and/or computing device) dispersed storage error encodes a data segment to produce a set of encoded data slices. The method 1100 continues at the step 1120 where the processing module selects a subset of a set of storage units. For example, the processing module identifies storage units associated with a most favorable storage reliability level.
The method 1100 continues at the step 1130 where the processing module issues list slice requests to the selected subset of the set of storage units. For example, the processing module generates the list slice requests to include slice names associated with corresponding encoded data slices of the set of encoded data slices and sends the generated list slice requests to the selected subset of the set of storage units.
The method 1100 continues at the step 1140 where the processing module determines a highest current revision level based on received list slice responses. For example, the processing module extracts revision levels from the received list slice responses and selects an approach based on one or more of a predetermination, historical storage reliability levels, and a performance goal. The approaches include selecting a highest revision level, selecting a common highest revision level, and selecting a majority highest revision level.
The method 1100 continues at the step 1150 where the processing module selects a revision level for the set of encoded data slices. For example, the processing module selects a revision level greater than the highest current revision level when storing new data. As another example, the processing module selects a revision level associated with data to be overwritten when overwriting previously stored data.
The method 1100 continues at the step 1160 where the processing module generates a set of checked revision write slice requests that includes the selected revision level and the highest current revision level. For example, the processing module generates each of the set of checked revision write slice request to include a transaction number, an encoded data slice of the set of encoded data slices, a slice name associated with the encoded data slice, the highest current revision level, and the selected revision level.
The method 1100 continues at the step 1170 where the processing module sends the set of checked revision write slice requests to the set of storage units. For example, the processing module identifies a set of storage units and outputs the set of checked revision write slice requests to the identified set of storage units. Alternatively, or in addition to, the processing module issues a subsequent set of checked revision write slice requests with a different highest current revision level and/or selected revision level based on received check provision write slice responses, when the responses include an updated highest current revision level.
The method 1200 operates in step 1210 by dispersed error encoding a data segment associated with a data object to produce a set of encoded data slices (EDSs). Note that the data object is segmented into a plurality of data segments, and the data segment of the plurality of data segments is dispersed error encoded in accordance with dispersed error encoding parameters to produce the set of EDSs. Note that the set of EDSs to be distributedly stored among a plurality of storage units (SUs) within the dispersed or distributed storage network (DSN).
The method 1200 then continues in step 1220 by selecting a subset of a set of storage units (SUs). Note that the subset of the set of SUs having a most favorable reliability level among the set of SUs. The method 1200 operates in step 1230 by transmitting (e.g., via an interface of the computing device that is configured to interface and communicate with the DSN) list slice requests to the subset of the set of SUs.
The method 1200 then continues in step 1240 by receiving (e.g., via the interface) list slice responses from at least some of the set of SUs. The method 1200 then operates in step 1250 by determining a highest current revision level based on list slice responses. The method 1200 operates in step 1260 by selecting a revision level for the set of EDSs.
The method 1200 then operates in step 1270 by generating a set of checked revision slice requests that includes the highest current revision level based on list slice responses and the revision level for the set of EDSs. The method 1200 then continues in step 1280 by transmitting (e.g., via the interface) the set of checked revision slice requests to the set of SUs.
This disclosure presents, among other things, various novel solutions that provides for, among other things, optimistic checked writes. For example, when performing checked operations, the computing device needs to be aware of the most recent current revision on each storage unit (SU) in order to formulate a checked write request with the correct expected revision such that the request succeeds. However, in order to determine the current version on a given SU requires a prior read, check, or listing request. This results in double the number of inputs/outputs (IO's) and increased round-trip time to perform a checked write. To bypass this limitation, the computing device makes the optimization of choosing a smaller number of SUs to issue reads/checks/listing operations against. For example, in a 10-of-16 system (e.g., 10 being a decode, read, and/or write threshold), the computing device may issue check requests to only 1 or 2 SUs, rather than 10 or 16, and assume that the highest result among that limited set is the correct revision. It may optimistically assume that it is and issue a checked write using the assumed version, or alternatively, fall back and issue additional checks to reach a more confident or definite quorum.
transmit the set of checked revision slice requests to the set of SUs.
transmit the list slice requests to the subset of the set of SUs.
determine the highest current revision level of the set of EDSs in accordance with an approach that is based on at least one of a predetermination, a historical storage reliability level, a performance goal, selecting the highest revision level, selecting a common highest revision level, or selecting a majority highest revision level.
select the revision level for the set of EDSs to be greater than the highest current revision level when storing new data.
generate a checked revision write slice request of the set of checked revision slice requests to include at least one of a transaction number, an EDS of the set of EDSs a slice name associated with the EDS of the set of EDSs, the highest current revision level, or the revision level.
the write threshold number is greater than or equal to the read threshold number that is greater than or equal to the decode threshold number.
a SU of the set of SUs or another set of SUs within the DSN, a wireless smart phone, a laptop, a tablet, a personal computers (PC), a work station, or a video game device.
8. The computing device of claim 1, wherein the DSN includes at least one of a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).
13. The computing device of claim 9, wherein the DSN includes at least one of a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).
transmitting, via the interface, the set of checked revision slice requests to the set of SUs.
transmitting, via the interface, the list slice requests to the subset of the set of SUs.
determining the highest current revision level of the set of EDSs in accordance with an approach that is based on at least one of a predetermination, a historical storage reliability level, a performance goal, selecting the highest revision level, selecting a common highest revision level, or selecting a majority highest revision level.
selecting the revision level for the set of EDSs to be greater than the highest current revision level when storing new data.
generating a checked revision write slice request of the set of checked revision slice requests to include at least one of a transaction number, an EDS of the set of EDSs a slice name associated with the EDS of the set of EDSs, the highest current revision level, or the revision level.
20. The method of claim 14, wherein the DSN includes at least one of a wireless communication system, a wire lined communication system, a non-public intranet system, a public internet system, a local area network (LAN), or a wide area network (WAN).

References: § 120
 § 120
 § 119
 Application No. 62
 § 120
 § 119
 Application No. 61
 § 120