Adapting resiliency of enterprise object storage systems

Various implementations herein enable managing a resiliency factor of an object stored in an enterprise object storage system. For example, in various implementations, a method of adjusting a realized resiliency factor of an object based on a target resiliency factor for the object is performed by an ingest entity of a storage system that includes a cluster of storage entities. The ingest entity includes a non-transitory computer readable storage medium, and one or more processors. In various implementations, the method includes obtaining a target resiliency factor for an object. In various implementations, the method includes determining whether or not to adjust a realized resiliency factor of the object based on the target resiliency factor. In various implementations, the method includes adjusting the realized resiliency factor of the object to an adjusted resiliency factor in response to determining to adjust the realized resiliency factor.

TECHNICAL FIELD

The present disclosure relates generally to enterprise object storage systems, and in particular, to managing data loss resiliency of enterprise object storage systems.

BACKGROUND

Some previously available storage systems provide fault tolerance through data mirroring. With data mirroring, multiple copies of an object are stored on a vault disk and again on different drives, so that a drive failure can only damage at most one copy of the data. The downside of data mirroring is that it is expensive due to being resource-intensive. For example, to be resilient to one failure, a storage system that utilizes data mirroring has to double the disk space available. Similarly, to be resilient to two failures, a storage system that utilizes data mirroring has to triple the disk space available. Another particular problem with data mirroring is that it results in a policy conflict for recording various media programs. Specifically, when recording a program on behalf of a customer, the Digital Millennium Copyright Act (DMCA) provides that one and only one unique instance of the data may be created for the customer. Therefore in this situation, data mirroring for the sake of providing fault tolerance violates copyright and associated fair use restrictions.

DESCRIPTION OF EXAMPLE EMBODIMENTS

Numerous details are described herein in order to provide a thorough understanding of the illustrative implementations shown in the accompanying drawings. However, the accompanying drawings merely show some example aspects of the present disclosure and are therefore not to be considered limiting. Those of ordinary skill in the art will appreciate from the present disclosure that other effective aspects and/or variants do not include all of the specific details of the example implementations described herein. While pertinent features are shown and described, those of ordinary skill in the art will appreciate from the present disclosure that various other features, including well-known systems, methods, components, devices, and circuits, have not been illustrated or described in exhaustive detail for the sake of brevity and so as not to obscure more pertinent aspects of the example implementations disclosed herein.

Overview

Some previously available storage systems provide fault tolerance through erasure coding. Typically, a storage system that utilizes erasure coding stores an object across various storage entities. In addition to storing object data, some previously available storage systems synthesize and store parity information for the object. Such storage systems are typically capable of detecting a loss of object data. Upon detecting the loss of object data, the storage system utilizes the parity information and the remaining object data to reconstruct the lost object data. Some previously available storage systems operate according to a predefined resiliency overhead that indicates a percentage of storage space that the storage system allocates to store parity information. Typically, a higher resiliency overhead makes the storage system more resilient to failures that result in data loss. However, a higher resiliency overhead tends to increase the cost of storing objects because maintaining a higher resiliency overhead is more resource-intensive.

Various implementations disclosed herein enable adjusting a realized resiliency factor associated with an object based on a target resiliency factor for the object. For example, in various implementations, a method of adjusting a realized resiliency factor of an object is performed by an ingest entity of a fault-tolerant enterprise object storage system (“storage system”, hereinafter for the sake of brevity). In some implementations, the storage system is configured to synthesize parity data for an object in order to protect the object from a data loss event. In various implementations, the storage system includes a plurality of storage entities, and one or more computing processors. In some implementations, each storage entity is configured to store object data and/or parity data on a block basis. In various implementations, the method includes obtaining a target resiliency factor for an object or the storage system as a whole. In some implementations, the target resiliency factor indicates a target number of data storage entities for storing object data (e.g., a target data storage entity count) and a target number of parity storage entities for storing parity information (e.g., a target parity storage entity count).

In various implementations, the method includes determining whether or not to adjust a realized resiliency factor of the object based on the target resiliency factor. In some implementations, the realized resiliency factor indicates a first number of data storage entities configured to store object data (e.g., a realized data storage entity count) and a first number of parity storage entities configured to store parity information (e.g., a realized parity storage entity count). In various implementations, the method includes adjusting the realized resiliency factor to an adjusted resiliency factor. In some implementations, the adjusted resiliency factor includes at least one of a second number of data storage entities (e.g., an adjusted data storage entity count) and a second number of parity storage entities (e.g., an adjusted parity storage entity count) in order to provide fault-tolerance for the object data within a threshold range of the target resiliency factor.

Example Embodiments

FIGS. 1A and 1Bare schematic diagrams of a storage system environment10in accordance with some implementations. While pertinent features are shown, those of ordinary skill in the art will appreciate from the present disclosure that various other features have not been illustrated for the sake of brevity and so as not to obscure more pertinent aspects of the example implementations disclosed herein. To that end, as a non-limiting example, the storage system environment10includes a client device20, and a fault-tolerant enterprise object storage system100(storage system100, hereinafter). In various implementations, the client device20and the storage system100communicate via a network (not shown), such as portions of the Internet and/or a private network.

In operation, the storage system100is utilized to store various objects. In some implementations, an object refers to a data asset. In some implementations, an object includes a data asset that is presentable to a user via the client device20. For example, the object includes a video file that represents a movie or a TV show, an audio file that represents a song, a text file, etc. In various implementations, the object includes a file of any file type (e.g., .mov, .wma, .mp4, .avi, .mp3, .jpg, .txt, .doc, .docx, .xls, .ppt, etc.) In some implementations, an object includes a data asset that represents a set of computer-readable instructions that are executable at the client device20. For example, in some implementations, the object includes a native application that is downloaded and installed at the client device20, a browser plugin, etc.

In various implementations, the storage system100includes a cluster of storage entities110-1,110-2. . .110-10(collectively, storage entities110), and an ingest entity120. In the example ofFIGS. 1A-1B, the storage system100includes ten storage entities110. However, in some examples, the storage system100includes a fewer number of storage entities110, or a greater number of storage entities110. The storage entities110store objects. For example, as illustrated inFIG. 1A, some of the storage entities110collectively store an object112. In various implementations, some of the storage entities110are configured to store object data for the object112, and some of the remaining storage entities110are configured to store parity data for the object112. In some implementations, the storage entities110that store object data for the object112are referred to as data storage entities, and the storage entities110that store parity data for the object112are referred to as one or more parity storage entities. The storage system100utilizes any suitable combination of methods and systems for synthesizing the parity data. In various implementations, the storage system100utilizes the parity data to recover (e.g., rebuild, reconstruct, restore, and/or repair) the object112in the event of a data loss. In some implementations, the storage entities110and the ingest entity120are network entities such as servers.

In some implementations, a storage entity110includes one or more computer readable storage mediums. For example, the storage entity110includes solid state memory devices, hard disk memory devices, optical disk drives, read-only memory and/or nanotube-based storage devices. In some implementations, the storage entities110includes servers that execute computer-readable instructions. In various implementations, a storage entity110includes various blocks (not shown). For example, in some implementations, a storage entity110that stores object data (e.g., a data storage entity such as the storage entities110-1,110-2. . .110-4) includes data blocks for storing the object data. Similarly, a storage entity110that stores parity data (e.g., a parity storage entity such as the storage entity110-5) includes parity blocks for storing the parity data. As described herein, in various implementations, a block refers to the smallest addressable block of memory (e.g., the smallest allocation unit of data) in a storage entity110. In some implementations, the blocks are identically-sized (e.g., 2 MB each) for processing convenience.

In various implementations, the ingest entity120serves as an interface for the storage system100. The ingest entity120receives/transmits data from/to any device that is external to the storage system100. Specifically, the ingest entity120receives/transmits data from/to the client device20. In various implementations, receiving/transmitting data includes receiving/transmitting the objects. Alternatively and/or additionally, receiving/transmitting data includes receiving/transmitting instructions. In some implementations, the instructions include operations that are performed in relation to the objects. Example instructions include writing an object, reading an object, deleting an object, copying an object, etc. In some implementations, the ingest entity120includes hardware and/or software that enables the ingest entity120to perform various operations described herein. In some examples, the ingest entity120is implemented by a server system (e.g., as illustrated inFIG. 5). In some implementations, the ingest entity120is configured to operate as one of the storage entities110. Put another way, in some implementations, one of the storage entities110is configured to operate as the ingest entity120.

In various implementations, the storage system100utilizes various methods and systems associated with distributed erasure coding. In some implementations, the storage system100distributes an object across multiple storage entities110. For example, the storage system100stores the first 2 MB of the object data at storage entity110-1, the next 2 MB of the object data at storage entity110-2, etc. In some implementations, the storage system100distributes the object across multiple storage entities110even if the object is small enough to be stored at a single storage entity110. Distributing the object data and the parity data across multiple storage entities110reduces the risk of losing the entire object in the event of a data loss. To that end, in various implementations, the object112is a data asset (e.g., a data item) that is stored in accordance with distributed erasure coding.

In various implementations, the object112is associated with a realized resiliency factor130. In various implementations, the realized resiliency factor130indicates a number of storage entities110that the storage system100has allocated to store the object data, and a number of storage entities110that the storage system100has allocated to store the parity data for the object112. In some implementations, the number of storage entities110that store the object data is referred to as a realized data storage entity count130d, and the number of storage entities110that store parity data is referred to as a realized parity storage entity count130p. In the example ofFIG. 1A, the object112is stored across five storage entities (e.g., storage entities110-1,110-2. . .110-5). In this example, four of the five storage entities110store object data, and one of the five storage entities110stores parity data. For example, as illustrated, the storage entities110-1,110-2. . .110-4store object data and the storage entity110-5stores parity data. Thus, in the example ofFIG. 1A, the realized data storage entity count130dfor the object112is four, and the realized parity storage entity count130pis one. In some scenarios, the realized resiliency factor130is expressed as a ratio of the realized data storage entity count130dand the realized parity storage entity count130p(e.g., 4:1 for the object112shown inFIG. 1A).

In various implementations, the realized resiliency factor130differs for various objects that are stored in the storage system100. In other words, in various implementations, the storage system100stores objects according to different realized resiliency factors130. In various implementations, the realized resiliency factor130indicates a realized resiliency overhead associated with the object112. In various implementations, the realized resiliency overhead is determined (e.g., computed) by dividing the realized parity storage entity count130pfor the object by the realized data storage entity count130dfor the object. In some implementations, the realized resiliency overhead is expressed as a percentage that represents the percentage of storage entities110allocated for storing parity data for the object112. In the example ofFIG. 1A, the realized resiliency overhead for the object112is 0.25 (¼) or 25%.

In various implementations, the storage system100(e.g., the ingest entity120) obtains a target resiliency factor30for the object112. For example, as illustrated inFIG. 1A, in some implementations, the storage system100receives the target resiliency factor30from the client device20. In some implementations, the target resiliency factor30indicates a target data storage entity count30d(e.g., a target number of data storage entities for storing the object data), and a target parity storage entity count30p(e.g., a target number of parity storage entities for storing the parity data). In some examples, the storage system100receives the target resiliency factor30for the object112as part of a write request to write the object112into the storage system100. Additionally and/or alternatively, in some examples, the storage system100receives the target resiliency factor30for the object112after the object112is written into the storage system100. Moreover, in some examples, the storage system100obtains the target resiliency factor30by retrieving the target resiliency factor30from a non-transitory memory.

In some implementations, the target resiliency factor30indicates a target number of data storage allocations, and/or a target number of parity storage allocations. In some examples, the target number of data storage allocations refers to a target number of storage disks to store the object data, and the target number of parity storage allocations refers to a target number of storage disks to store the parity data. In some examples, the target number of data storage allocations refers to a target number of data blocks to store the object data, and the target number of parity storage allocations refers to a target number of parity blocks to store the parity data.

As illustrated inFIG. 1B, in various implementations, the storage system100adjusts the realized resiliency factor130to an adjusted resiliency factor230based on the target resiliency factor30. In some implementations, the adjusted resiliency factor230includes an adjusted data storage entity count230d(e.g., an adjusted number of data storage entities for storing the object data), and/or an adjusted parity storage entity count230p(e.g., an adjusted number of parity storage entities for storing the parity data). In the example ofFIG. 1B, the adjusted resiliency factor230is 6:2. Put another way, in the example ofFIG. 1B, the adjusted data storage entity count230dis six, and the adjusted parity storage entity count230pis two. Moreover, in the example ofFIG. 1B, the adjusted resiliency overhead is 0.33 ( 2/6) or 33%. A person of ordinary skill in the art will appreciate that since the adjusted parity storage entity count230pis greater than the realized parity storage entity count130p(between the two examples), the object112has become more resilient to data loss events. In various implementations, the storage system100adjusts the realized resiliency factor130to the adjusted resiliency factor230in order to provide fault-tolerance for the object112within a threshold range of the target resiliency factor30. For example, in some implementations, the storage system100sets the adjusted resiliency factor230equal to the target resiliency factor30. In some implementations, the storage system100adjusts the realized resiliency factor130to the adjusted resiliency factor230in order to attain a resiliency overhead that is closer to a target resiliency overhead indicated by the target resiliency factor30.

In some implementations, the storage system100transmits an adjustment confirmation232to the client device20upon adjusting the realized resiliency factor130to the adjusted resiliency factor230. In some examples, the adjustment confirmation232indicates the adjusted resiliency factor230. For example, as illustrated inFIG. 1B, the adjustment confirmation232includes the adjusted data storage entity count230dand the adjusted parity storage entity count230p. In some implementations, the adjustment confirmation232includes other information. For example, in some scenarios, the adjustment confirmation232includes the adjusted resiliency overhead for the object112.

In some implementations, the storage entities110are implemented on the same computing device. For example, in some implementations, the storage entities110are storage disks. In such implementations, multiple storage entities110are enclosed in a single housing. Alternatively, in some implementations, the storage entities110are implemented on different computing devices. In some implementations, the storage system100is a distributed storage system including multiple computing devices networked over multiple locations. In some implementations, the storage system100is configured to store video data associated with multicast (e.g., broadcast) content. In other words, the storage system100serves as a digital video recorder (DVR). In some implementations, the storage system100serves as a cloud-based DVR, since the storage system100is capable of servicing read requests and write requests that the storage system100receives.

In various implementations, the client device20includes any suitable computing device, such as a computer, a laptop computer, a tablet device, a netbook, an internet kiosk, a personal digital assistant, a mobile phone, a smartphone, a wearable, a gaming device, a computer server, etc. In some implementations, the client device20includes one or more processors, one or more types of memory, a display and/or other user interface components such as a keyboard, a touch screen display, a mouse, a track-pad, a digital camera and/or any number of supplemental devices to add functionality. In some implementations, a client device20includes a suitable combination of hardware, software and firmware configured to provide at least some of protocol processing, modulation, demodulation, data buffering, power control, routing, switching, clock recovery, amplification, decoding, and error control.

FIG. 2is a block diagram of the ingest entity120in accordance with some implementations. In various implementations, the ingest entity120includes a resiliency management module122, a resiliency determination module124, a resiliency adjustment module126, and a data store128. In various implementations, the resiliency management module122, the resiliency determination module124, and the resiliency adjustment module126are implemented in hardware (e.g., as one or more application specific integrated circuits (ASICs)) and/or in software (e.g., as one or more sets of computer readable instructions that are executed by one or more central processing units).

In various implementations, the resiliency determination module124determines the realized resiliency factor130for the object112. For example, in some implementations, the resiliency determination module124determines the realized data storage entity count130d, and the realized parity storage entity count130pfor the object112. In some implementations, the resiliency determination module124queries the storage entities110to determine where various data segments (e.g., one or more data blocks) and parity segments (e.g., one or more parity blocks) of the object112are stored. In some examples, the resiliency determination module124transmits locate requests to the storage entities110, and receives locate responses that indicate the locations of various data segments and parity segments. In some scenarios, the locate requests include an object identifier (ID) for the object112. In various implementations, the resiliency determination module124determines the realized data storage entity count130dby counting the number of storage entities110that store data segments of the object112. Similarly, the resiliency determination module124determines the realized parity storage entity count130pby counting the number of storage entities110that store parity segments of the object112. In some implementations, the resiliency determination module124provides the realized resiliency factor130to the resiliency management module122. Additionally and/or alternatively, the resiliency determination module124stores the realized resiliency factor130in the data store128. In various implementations, the resiliency determination module124determines the realized resiliency factor130periodically, and/or in response to a request (e.g., from the resiliency management module122).

In various implementations, the resiliency management module122manages a resiliency factor of the object112. In various implementations, the resiliency management module122obtains the target resiliency factor30for the object112, and determines whether or not to adjust the realized resiliency factor130of the object112based on the target resiliency factor30. In some implementations, the resiliency management module122determines a number of available storage entities110in the storage system100. Upon determining the number of available storage entities110, the resiliency management module122determines whether the number of available storage entities110is sufficient to realize the target resiliency factor30. For example, the resiliency management module122determines whether the number of available storage entities110is greater than or equal to a sum of the target data storage entity count30dand the target parity storage entity count30p. If the number of available storage entities110is sufficient to realize the target resiliency factor30, then the resiliency management module122determines to adjust the realized resiliency factor130to an adjusted resiliency factor230. Moreover, if the number of available storage entities110is sufficient to realize the target resiliency factor30, then the resiliency management module122sets the adjusted resiliency factor230equal to the target resiliency factor30.

In some implementations, the resiliency management module122determines that the number of available storage entities110is not sufficient to realize the target resiliency factor30. For example, the resiliency management module122determines that the number of available storage entities110is less than the sum of the target data storage entity count30dand the target parity storage entity count30p. In such implementations, the resiliency management module122determines whether the number of available storage entities110is sufficient to adjust the realized resiliency factor130to an adjusted resiliency factor230that is closer to the target resiliency factor30. If the number of available storage entities110is sufficient to adjust the realized resiliency factor130closer to the target resiliency factor30, then the resiliency management module122determines an adjusted resiliency factor230that is closer to the target resiliency factor30. In some implementations, the resiliency management module122determines that the number of available storage entities110is not sufficient to adjust the realized resiliency factor130equal to or closer to the target resiliency factor30. In such implementations, the resiliency management module122determines not to adjust the realized resiliency factor130. In various implementations, the resiliency management module122transmits the adjusted resiliency factor230to the resiliency adjustment module126. In various implementations, the resiliency management module122selects an adjusted resiliency factor230that is within a threshold range of the target resiliency factor30. In other words, in various implementations, the resiliency management module122determines to adjust the realized resiliency factor130in order to provide fault-tolerance for the object112within a threshold range of the target resiliency factor30.

In various implementations, the resiliency adjustment module126adjusts the realized resiliency factor130to the adjusted resiliency factor230. In some implementations, the resiliency adjustment module126re-distributes the object data for the object112across a number of storage entities110indicated by the adjusted data storage entity count230d. In some implementations, the resiliency adjustment module126synthesizes parity information for the object112based on the re-distributed object data and the adjusted resiliency factor230. Upon synthesizing the parity information, the resiliency adjustment module126stores the parity information across a number of storage entities110indicated by the adjusted parity storage entity count230p. In some implementations, the resiliency adjustment module126transmits the adjustment confirmation232to the client device20. In some examples, the adjustment confirmation232indicates the adjusted resiliency factor230for the object112. In some implementations, the resiliency adjustment module126determines whether adjusting the realized resiliency factor130to the adjusted resiliency factor230would result in the object being mirrored. In such implementations, the resiliency adjustment module126adjusts the realized resiliency factor130to the adjusted resiliency factor230upon determining that the adjustment would not result in the object112being mirrored. However, if adjusting the realized resiliency factor130to the adjusted resiliency factor230would result in the object112being mirrored, then the resiliency adjustment module126determines not to adjust the realized resiliency factor130in order to avoid violation of copyright law. In various implementations, the resiliency adjustment module126adjusts the realized resiliency factor130to the adjusted resiliency factor230.

In various implementations, the data store128stores information (e.g., metadata) associated with objects that are stored in the storage system100. For example, in some implementations, the data store128stores object identifiers (IDs) for objects that are stored in the storage entities110. In some implementations, the data store128stores the target resiliency factor30for the object112. In such implementations, the resiliency management module122obtains the target resiliency factor30by retrieving the target resiliency factor30from the data store128. In some implementations, the data store128stores the realized resiliency factor130for the object112. In such implementations, the resiliency determination module124and/or the resiliency management module122retrieve the realized resiliency factor130from the data store128. In some implementations, the resiliency management module122periodically obtains the target resiliency factor30from the data store128and determines whether or not to adjust the realized resiliency factor130based on the target resiliency factor30. To that end, the data store128includes one or more databases, tables (e.g., look-up tables), indices (e.g., inverted indices), and/or any other suitable data structure.

FIG. 3is a flowchart representation of a method300of adjusting a realized resiliency factor of an object based on a target resiliency factor for the object in accordance with some implementations. In various implementations, the method300is implemented as a set of computer readable instructions that are executed at a storage system (e.g., the storage system100shown inFIGS. 1A-1B). For example, in various implementations, the method300is performed by an ingest entity of the storage system (e.g., the ingest entity120shown inFIGS. 1A, 1B and 2). Briefly, the method300includes obtaining a target resiliency factor for an object, determining whether or not to adjust a realized resiliency factor of the object based on the target resiliency factor, and adjusting the realized resiliency factor to an adjusted resiliency factor upon determining to adjust the realized resiliency factor.

As represented by block310, in various implementations, the method300includes obtaining a target resiliency factor for an object (e.g., the target resiliency factor30shown inFIGS. 1A and 2). As represented by block310a, in some implementations, the method300includes receiving the target resiliency factor as part of a write request to write the object into the storage system. As represented by block310b, in some implementations, the method300includes receiving the target resiliency factor after the object is stored in the storage system. In some examples, the object is stored according to a first target resiliency factor, and the method300includes receiving a second target resiliency factor that is different from the first target resiliency factor. As represented by block310c, in some implementations, the method300includes retrieving the target resiliency factor from a non-transitory memory (e.g., the data store128shown inFIG. 2). In some examples, an object is stored according to a resiliency factor that is different from the target resiliency factor for the object (e.g., due to a lack of available storage entities). In such examples, obtaining the target resiliency factor includes periodically retrieving the target resiliency factor from the memory in order to determine whether the target resiliency factor is realizable (e.g., due to a change in the number of available storage entities). In various implementations, obtaining the target resiliency factor includes obtaining a target data storage entity count, and a target parity storage entity count (e.g., the target data storage entity count30d, and the target parity storage entity count30p, respectively, shown inFIGS. 1A and 2).

As represented by block320, in various implementations, the method300includes determining whether or not to adjust a realized resiliency factor of the object based on the target resiliency factor for the object. As represented by block322, in some implementations, the method300includes determining whether a number of available storage entities is greater than or equal to a sum of the target data storage entity count and the target parity storage entity count. In other words, in some implementations, the method300includes determining whether the number of available storage entities is sufficient to realize the target resiliency factor. In some examples, determining the number of available storage entities includes identifying the storage entities that are online (e.g., in service, and ready to store object data and/or parity data). In some implementations, if the number of available storage entities is greater than or equal to the sum then the method300proceeds to block324, otherwise the method300proceeds to block326. Put another way, if the number of available storage entities is sufficient to realize the target resiliency factor then the method300proceeds to block324, else the method300proceeds to block326. As represented by block324, in various implementations, the method300includes determining to adjust the realized resiliency factor to an adjusted resiliency factor that is equal to the target resiliency factor.

As represented by block326, in some implementations, the method300includes determining whether the number of available storage entities is sufficient to adjust the realized resiliency factor to an adjusted resiliency factor that is closer to the target resiliency factor. In some examples, the method300includes determining whether the number of available storage entities is sufficient to realize a resiliency factor that is within a threshold range of the target resiliency factor (e.g., within 10% of the target resiliency factor). In other words, in some implementations, the method300includes determining whether there are enough storage entities available to adjust the realized resiliency factor to an adjusted resiliency factor that is within a threshold range of the target resiliency factor. In various implementations, if an adjusted resiliency factor that is closer to the target resiliency factor is attainable, then the method300proceeds to block328, otherwise the method300proceeds to block330.

As represented by block328, in various implementations, the method300includes determining to adjust the realized resiliency factor to an adjusted resiliency factor that is closer to the target resiliency factor (e.g., within a threshold range of the target resiliency factor). In various implementations, the adjusted resiliency factor being within a threshold range of the target resiliency factor refers to an adjusted resiliency overhead being within a threshold range of a target resiliency overhead. In various implementations, upon determining to adjust the realized resiliency factor, the method300includes determining the adjusted resiliency factor. In some implementations, determining the adjusted resiliency factor includes selecting a resiliency factor that is as close as possible to the target resiliency factor given the number of available storage entities. In various implementations, the adjusted resiliency factor is a function of the target resiliency factor and the number of available storage entities. In some implementations, the adjusted resiliency factor includes an adjusted data storage entity count that is different from (e.g., greater than, or less than) the realized data storage entity count. Additionally and/or alternatively, in some implementations, the adjusted resiliency factor includes an adjusted parity storage entity count that is different from (e.g., greater than, or less than) the realized parity storage entity count.

As represented by block330, in various implementations, the method300includes determining not to adjust the realized resiliency factor. In some implementations, the method300includes determining not to adjust the realized resiliency factor if the number of the available storage entities is not sufficient to realize an adjusted resiliency factor that is closer to the target resiliency factor. In some implementations, the method300includes determining not to adjust the realized resiliency factor if the number of available storage entities is not sufficient to realize an adjusted resiliency factor that is within a threshold range of the target resiliency factor. In some implementations, the method300includes determining not to adjust the realized resiliency factor if a difference between the adjusted resiliency factor and the realized resiliency factor is less than a threshold difference (e.g., less than 2% of the realized resiliency factor).

As represented by block360, in various implementations, the method300includes adjusting the realized resiliency factor to an adjusted resiliency factor upon determining to adjust the realized resiliency factor. As described herein, in some implementations, the method300includes adjusting the realized resiliency factor to an adjusted resiliency factor that is equal to the target resiliency factor. In some implementations, the method300includes adjusting the realized resiliency factor to an adjusted resiliency factor that is closer to the target resiliency factor. In some implementations, the method300includes adjusting the realized resiliency factor to an adjusted resiliency factor that is within a threshold range of the target resiliency factor.

As represented by block362, in various implementations, the method300includes re-distributing object data based on the adjusted resiliency factor. In some implementations, the method300includes storing the object data across a number of storage entities that is equal to the adjusted data storage entity count. In various implementations, the adjusted data storage entity count is greater than the realized data storage entity count. In such implementations, the method300includes moving some of the object data to storage entities that were not previously storing the object data, so that the object data is spread across a number of storage entities that is equal to the adjusted data storage entity count. In some implementations, the adjusted data storage entity count is less than the realized data storage entity count. In such implementations, the method300includes removing the object data from the storage entities that are in excess of the adjusted data storage entity count, and limiting the object data to a number of storage entities that is equal to the adjusted data storage entity count.

As represented by block364, in various implementations, the method300includes synthesizing parity information based on the re-distributed data and the adjusted resiliency factor. In some implementations, the method300includes storing the parity information in a number of storage entities that is equal to the adjusted parity storage entity count. A person of ordinary skill in the art will appreciate that, in some implementations, a combination of various methods and systems are utilized to synthesize the parity information. In some implementations, the method300includes transmitting an adjustment confirmation to the client device (e.g., the adjustment confirmation232shown inFIG. 1B). In some examples, the adjustment confirmation indicates the adjusted resiliency factor. For example, in some scenarios, the adjustment confirmation includes the adjusted data storage entity count and/or the adjusted parity storage entity count.

FIG. 4Ais a flowchart representation of a method320aof determining whether or not to adjust a realized resiliency factor of an object upon detecting a decrease in a number of available storage entities in accordance with some implementations. In various implementations, the method320ais implemented as a set of computer readable instructions that are executed at a storage system (e.g., the storage system100shown inFIGS. 1A-1B). For example, in various implementations, the method320ais performed by an ingest entity of the storage system (e.g., the ingest entity120shown inFIGS. 1A, 1B and 2). Briefly, the method320aincludes detecting a decrease in a number of available storage entities, and determining whether or not to adjust a realized resiliency factor of an object based on the decrease in the number of available storage entities.

As represented by block332, in various implementations, the method320aincludes storing an object according to a realized resiliency factor that is equal to a target resiliency factor for the object. As represented by block334, in various implementations, the method320aincludes detecting a decrease in a number of available storage entities. For example, in some implementations, the method320aincludes detecting that one or more storage entities has become unavailable. In some examples, the method320aincludes detecting the decrease in the number of available storage entities when one or more storage entities goes offline (e.g., due to a power outage, or a scheduled maintenance). In some scenarios, the method320aincludes detecting the decrease in the number of available storage entities when one or more storage entities fails (e.g., due to a mechanical failure, or a software failure). In some implementations, the method320aincludes detecting that the number of available storage entities has decreased from a first number that is above a sum of the target data storage entity count and the target parity storage entity count to a second number that is below the sum.

As represented by block336, in various implementations, the method320aincludes determining whether the decreased number of available storage entities is sufficient to maintain a realized resiliency factor of an object that is equal to the target resiliency factor for the object. For example, in some implementations, the method320aincludes determining whether the decreased number of available storage entities is greater than or equal to a sum of the realized data storage entity count and the realized parity storage entity count. If the decreased number of available storage entities is sufficient to maintain the realized resiliency factor then the method320aproceeds to block338, otherwise the method320aproceeds to block340. As represented by block338, if the decreased number of available storage entities are sufficient to maintain the target resiliency factor, then the method320aincludes determining not to adjust the realized resiliency factor. As represented by block340, if the decreased number of available storage entities are not sufficient to maintain the target resiliency factor, then the method320aincludes determining to adjust the realized resiliency factor to an adjusted realized factor that is different from the target resiliency factor. In some implementations, the method320aincludes determining an adjusted resiliency factor that is as close to the target resiliency factor as possible given the decreased number of available storage entities.

FIG. 4Bis a flowchart representation of a method320bof determining whether or not to adjust a realized resiliency factor of an object upon detecting an increase in a number of available storage entities in accordance with some implementations. In various implementations, the method320bis implemented as a set of computer readable instructions that are executed at a storage system (e.g., the storage system100shown inFIGS. 1A and 1B). For example, in various implementations, the method320bis performed by an ingest entity of the storage system (e.g., the ingest entity120shown inFIGS. 1A, 1B and 2). Briefly, the method320bincludes detecting an increase in a number of available storage entities, and determining whether or not to adjust a realized resiliency factor of an object based on the increase in the number of available storage entities.

As represented by block342, in various implementations, the method320bincludes storing an object according to a realized resiliency factor that is less than a target resiliency factor for the object. As represented by block344, in various implementations, the method320bincludes detecting an increase in a number of available storage entities. For example, in some implementations, the method320bincludes detecting that one or more storage entities that was previously unavailable has become available. In some examples, the method320bincludes detecting the increase in the number of available storage entities when one or more storage entities comes online (e.g., after suffering a power outage, or after undergoing a scheduled maintenance). In some scenarios, the method320bincludes detecting the increase in the number of available storage entities when one or more storage entities that had previously failed is restored (e.g., via a mechanical upgrade, or a software patch). In some implementations, the method320bincludes detecting an increase in the number of available storage entities from a first number that is below a sum of the target data storage entity count and the target parity storage entity count to a second number that is above the sum.

As represented by block346, in various implementations, the method320bincludes determining whether the increased number of available storage entities is sufficient to adjust the realized resiliency factor to an adjusted resiliency factor that is closer to the target resiliency factor. If the increased number of available storage entities is sufficient to attain an adjusted resiliency factor that is closer to the target resiliency factor then the method320bproceeds to block348, otherwise the method320bproceeds to block350. As represented by block348, in some implementations, the method320bincludes determining to adjust the realized resiliency factor to an adjusted resiliency factor that is closer to the target resiliency factor than the realized resiliency factor. In some examples, the adjusted resiliency factor is equal to the target resiliency factor. In some examples, the adjusted resiliency factor is within a threshold range of the target resiliency factor. In some implementations, the method320bincludes determining an adjusted resiliency factor that is a function of the increased number of available storage entities and the target resiliency factor. As represented by block350, if the increased number of storage entities is not sufficient to attain an adjusted resiliency factor that is closer to the target resiliency factor, then the method320bincludes determining not to adjust the realized resiliency factor.

FIG. 5is a block diagram of a server system500enabled with various modules that are provided to adjust a realized resiliency factor of an object based on a target resiliency factor for the object in accordance with some implementations. For example, in various implementations, the server system500is enabled with one or more components of an ingest entity of a storage system (e.g., the ingest entity120shown inFIGS. 1A, 1B and 2) according to some implementations. While certain specific features are illustrated, those of ordinary skill in the art will appreciate from the present disclosure that various other features have not been illustrated for the sake of brevity, and so as not to obscure more pertinent aspects of the implementations disclosed herein. To that end, as a non-limiting example, in some implementations the server system500includes one or more processing units (CPUs)502, a network interface503, a memory510, a programming interface508, and one or more communication buses504for interconnecting these and various other components.

In some implementations, the network interface503is provided to, among other uses, establish and maintain a metadata tunnel between a cloud hosted network management system and at least one private network including one or more compliant devices. In some implementations, the communication buses504include circuitry that interconnects and controls communications between system components. The memory510includes high-speed random access memory, such as DRAM, SRAM, DDR RAM or other random access solid state memory devices; and may include non-volatile memory, such as one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, or other non-volatile solid state storage devices. The memory510optionally includes one or more storage devices remotely located from the CPU(s)502. The memory510comprises a non-transitory computer readable storage medium.

In some implementations, the memory510or the non-transitory computer readable storage medium of the memory510stores the following programs, modules and data structures, or a subset thereof including an optional operating system520, a resiliency management module522, a resiliency determination module524, a resiliency adjustment module526, and a data store528. In various implementations, the resiliency management module522, the resiliency determination module524, the resiliency adjustment module526and the data store528are similar to the resiliency management module122, the resiliency determination module124, the resiliency adjustment module126and the data store128, respectively, shown inFIG. 2.

The operating system520includes procedures for handling various basic system services and for performing hardware dependent tasks.

In various implementations, the resiliency determination module524determines a realized resiliency factor for an object (e.g., the realized resiliency factor130shown inFIGS. 1A and 2). To that end, in various implementations, the resiliency determination module524includes instructions and/or logic524a, and heuristics and metadata524b. In various implementations, the resiliency management module522determines whether or not to adjust the realized resiliency factor of an object to an adjusted resiliency factor (e.g., the adjusted resiliency factor230shown inFIGS. 1B and 2) based on a target resiliency factor (e.g., the target resiliency factor30shown inFIGS. 1A and 2). In various implementations, the resiliency management module522manages a resiliency factor of an object. To that end, in various implementations, the resiliency management module522includes instructions and/or logic522a, and heuristics and metadata522b. In various implementations, the resiliency adjustment module526adjusts the realized resiliency module to the adjusted resiliency factor. In some implementations, the adjusted resiliency factor is equal to the target resiliency factor. In some implementations, the adjusted resiliency factor is closer to the target resiliency factor than the realized resiliency factor. In various implementations, the adjusted resiliency factor is a function of the target resiliency factor and a number of available storage entities in the storage system. To that end, in various implementations, the resiliency adjustment module526includes instructions and/or logic526a, and heuristics and metadata526b. In various implementations, the server system500performs the method300shown inFIG. 3, the method320ashown inFIG. 4A, and/or the method320bshown inFIG. 4B.