Selective caching of erasure coded fragments in a distributed storage system

Methods and apparatus related to framework and/or methodology for selective caching of Erasure Coded fragments in a distributed storage system are described. In one embodiment, a plurality of fragments of a data object are generated. Each of the plurality of fragments is Erasure Coded (EC) prior to storage at a storage node of a plurality of storage nodes. Each of the plurality of fragments is transmitted with a caching hint to indicate whether that fragment is to be cached at the storage node. Other embodiments are also disclosed and claimed.

FIELD

The present disclosure generally relates to the field of electronics. More particularly, some embodiments generally relate to framework and/or methodology for selective caching of Erasure Coded (EC) fragments in a distributed storage system.

BACKGROUND

Generally, Erasure coding (EC) is a method of data protection in which data is split into fragments, expanded, and encoded with redundant data pieces and stored across a set of different storage nodes or storage media.

Moreover, EC is similar to RAID (Redundant Array of Inexpensive Drives) in that parity fragments are created and stored along with data fragments. In the event of a storage device failure, lost data fragments can be reconstructed from the remaining data and parity fragments. However, some current EC implementations can be inefficient.

DETAILED DESCRIPTION

In the following description, numerous specific details are set forth in order to provide a thorough understanding of various embodiments. However, various embodiments may be practiced without the specific details. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to obscure the particular embodiments. Further, various aspects of embodiments may be performed using various means, such as integrated semiconductor circuits (“hardware”), computer-readable instructions organized into one or more programs (“software”), or some combination of hardware and software. For the purposes of this disclosure reference to “logic” shall mean either hardware, software, firmware, or some combination thereof.

EC is generally implemented as a “k-of-n” scheme, where user data is decomposed into k data fragments and m parity fragments. When accessing data, any k of the n=k+m total fragments need to be accessible, otherwise data is lost. In other words, in a k-of-n scheme, a system can suffer up to m device failures. This makes EC very well suited to distributed storage systems, such as Ceph™ and Swift™, where there are multiple backend storage nodes (any of which can fail).

However, EC creates a complication for any caching solution in the backend storage nodes, since any given node will only see a fragment of the object data. In order to retrieve the data to service a GET (i.e., read) request, the minimal subset of fragments (e.g., from k of the n backend storage nodes) are retrieved. Consequently, the retrieval of the data is gated by the slowest responder amongst the minimal fragment subset. This implies that caching a fragment in a particular node will not provide any benefit unless each node holding the other fragments also caches its respective fragment. There is little benefit to caching only a subset of the fragments. Ideally, all n nodes should agree whether to cache the data (and parity) fragments for a given object. However, for this to occur, the backend storage nodes need to possess cluster-level information; namely, the caching policy for each fragment stored by the storage nodes.

Some embodiments relate to framework and/or methodology for selective caching of Erasure Coded (EC) fragments in a distributed storage system. Moreover, existing backend storage node caching solutions may operate in isolation and fail to leverage any cluster-level context, making them ineffective for caching when EC is used. To this end, one or more embodiments enable effective caching in an EC distributed data store via a framework and methodology to coordinate caching of fragments across the storage system cluster. For example, caching “hints” (which may include one or more bits of data) may be sent to storage nodes of the cluster in order to inform them of the caching policy of their EC fragments. In an embodiment, logic (e.g., logic160shown in the figures) is able to select whether to cache EC fragments in a distributed storage system. Also, while logic160is illustrated in various locations in the figures, embodiments are not limited to the locations shown and logic160may be provided in other locations.

As discussed herein, object store (or object storage) generally refers to a storage architecture that manages data as objects, e.g., as opposed to storage architectures such as file systems (that manage data as a file hierarchy) or block storage (which manages data as blocks). Object storage systems allow relatively inexpensive, scalable and self-healing retention of massive amounts of unstructured data. The object stores may be distributed at various near and/or far storage nodes or storage devices Additionally, as discussed herein, a far (or remote) node generally refers to a node that is reachable over a network link (or even across one or more network switches or hubs), whereas a near node may be reached over a local communication channel or interconnect (such as those discussed herein with reference toFIGS. 3, 4, and/or5, e.g., including a Peripheral Component Interconnect (PCI), PCI express (PCIe), etc.). Also, a node generally refers to a computing device (e.g., including a storage device or memory).

Furthermore, one or more embodiments discussed herein may be applied to any type of memory including Volatile Memory (VM) and/or Non-Volatile Memory (NVM). Also, embodiments are not limited to a single type of NVM and non-volatile memory of any type or combinations of different NVM types (e.g., including NAND and/or NOR type of memory cells) or other formats usable for memory) may be used. The memory media (whether used in DIMM (Dual Inline Memory Module) format or otherwise) can be any type of memory media including, for example, one or more of: nanowire memory, Ferro-electric Transistor Random Access Memory (FeTRAM), Magnetoresistive Random Access Memory (MRAM), multi-threshold level NAND flash memory, NOR flash memory, Spin Torque Transfer Random Access Memory (STTRAM), Resistive Random Access Memory, byte addressable 3-Dimensional Cross Point Memory, single or multi-level PCM (Phase Change Memory), memory devices that use chalcogenide phase change material (e.g., chalcogenide glass) or “write in place” non-volatile memory. Also, any type of Random Access Memory (RAM) such as Dynamic RAM (DRAM), backed by a power reserve (such as a battery or capacitance) to retain the data, may provide an NV memory solution. Volatile memory can include Synchronous DRAM (SDRAM). Hence, even volatile memory capable of retaining data during power failure or power disruption(s) may be used for memory in various embodiments.

The techniques discussed herein may be provided in various computing systems (e.g., including a non-mobile computing device such as a desktop, workstation, server, rack system, etc. and a mobile computing device such as a smartphone, tablet, UMPC (Ultra-Mobile Personal Computer), laptop computer, Ultrabook™ computing device, smart watch, smart glasses, smart bracelet, etc.), including those discussed with reference toFIGS. 1-5. More particularly,FIG. 1illustrates a block diagram of a computing system100, according to an embodiment. The system100may include one or more processors102-1through102-N (generally referred to herein as “processors102” or “processor102”). The processors102may communicate via an interconnection or bus104. Each processor may include various components some of which are only discussed with reference to processor102-1for clarity. Accordingly, each of the remaining processors102-2through102-N may include the same or similar components discussed with reference to the processor102-1.

In an embodiment, the processor102-1may include one or more processor cores106-1through106-M (referred to herein as “cores106,” or more generally as “core106”), a processor cache108(which may be a shared cache or a private cache in various embodiments), and/or a router110. The processor cores106may be implemented on a single integrated circuit (IC) chip. Moreover, the chip may include one or more shared and/or private caches (such as processor cache108), buses or interconnections (such as a bus or interconnection112), logic120, memory controllers (such as those discussed with reference toFIGS. 3-5), or other components.

In one embodiment, the router110may be used to communicate between various components of the processor102-1and/or system100. Moreover, the processor102-1may include more than one router110. Furthermore, the multitude of routers110may be in communication to enable data routing between various components inside or outside of the processor102-1.

The processor cache108may store data (e.g., including instructions) that are utilized by one or more components of the processor102-1, such as the cores106. For example, the processor cache108may locally cache data stored in a memory114for faster access by the components of the processor102. As shown inFIG. 1, the memory114may be in communication with the processors102via the interconnection104. In an embodiment, the processor cache108(that may be shared) may have various levels, for example, the processor cache108may be a mid-level cache and/or a last-level cache (LLC). Also, each of the cores106may include a level 1 (L1) processor cache (116-1) (generally referred to herein as “L1 processor cache116”). Various components of the processor102-1may communicate with the processor cache108directly, through a bus (e.g., the bus112), and/or a memory controller or hub.

As shown inFIG. 1, memory114may be coupled to other components of system100through a memory controller120. Memory114includes volatile memory and may be interchangeably referred to as main memory. Even though the memory controller120is shown to be coupled between the interconnection104and the memory114, the memory controller120may be located elsewhere in system100. For example, memory controller120or portions of it may be provided within one of the processors102in some embodiments.

System100also includes NV memory130(or Non-Volatile Memory (NVM), e.g., compliant with NVMe (NVM express)) coupled to the interconnect104via NV controller logic125. Hence, logic125may control access by various components of system100to the NVM130. Furthermore, even though logic125is shown to be directly coupled to the interconnection104inFIG. 1, logic125may communicate via a storage bus/interconnect (such as the SATA (Serial Advanced Technology Attachment) bus, Peripheral Component Interconnect (PCI) (or PCI express (PCIe) interface), etc.) with one or more other components of system100(for example where the storage bus is coupled to interconnect104via some other logic like a bus bridge, chipset (such as discussed with reference toFIGS. 3, 4, and/or5), etc.). Additionally, logic125may be incorporated into memory controller logic (such as those discussed with reference toFIGS. 3-5) or provided on a same Integrated Circuit (IC) device in various embodiments (e.g., on the same IC device as the NVM130or in the same enclosure as the NVM130). System100may also include other types of non-volatile memory such as those discussed with reference toFIGS. 3-5, including for example a hard drive, etc.

Some embodiments use cluster-level context to coordinate caching amongst storage cluster nodes that hold fragments of a given piece of data. The consistency processes and existing communication paths in the distributed store may be used to pass caching hints from entities possessing cluster level state to the backend storage node where the fragments are stored. These entities may be referred to as proxy nodes (in Swift™ object stores) or gateway nodes (in Ceph™ object stores). Specifically, a proxy or gateway node is aware of each object being stored, understands how objects are decomposed into EC fragments, and/or can communicate the caching policy of these fragments to the backend storage nodes. Based on the above principles, one or more embodiments utilize a few different coordination schemes including: (1) when sending fragments of data to different nodes, a hint is also sent to each node in the cluster whether to cache (or not cache) the respective fragment (e.g., cache all fragments or none); and/or (2) when a node evicts a fragment from its cache, that node transmits the eviction information to the other nodes holding fragments of the original data (e.g., evict all when evicting one). Alternatively, the evicting node could report the eviction to logic160for communication to other nodes holding fragments of the original data object.

By contrast, current distributed stores fail to make any attempt to coordinate amongst different storage nodes when it comes to caching of EC fragments. While prior work may have used technology within each backend storage node to direct the storage traffic appropriately between storage devices to improve performance, it uses context available locally (metadata vs data, size of the data) to determine where the data should be directed. Such techniques do not work in an erasure coded store where each node only receives a fragment of the entire data uploaded in a request and, as a consequence, every fragment may be of the same size. Techniques that only rely on information at the backend storage node do not have sufficient cluster-level context to recognize that certain fragments are part of the same data. Without this context, as mentioned, caching only some of the fragments will not typically provide improved performance.

Furthermore, distributed stores may implement EC in a manner that is transparent to the end user accessing objects. The storage system uploads and downloads complete objects with one of the components of the distributed system (like the Swift proxy) responsible for “sharding” an uploaded object into EC fragments or reassembling fragments into complete objects to return to the end user. Generally, sharding is a type of data partitioning that separates data into smaller, more easily managed parts called data shards. The word shard means a small part of a whole. An embodiment utilizes this architecture to generate caching hints at a cluster-level entity responsible for creating the object fragments or shards as shown inFIG. 2A.

More particularly,FIG. 2Aillustrates a block diagram of a storage system to provide coordinated EC caching, according to an embodiment. The system (e.g., through logic160) may provide support for caching of all fragments of a data object (or more generally a piece of data) or none.FIG. 2Billustrates a flow diagram for selective caching of EC fragments in a distributed storage system, according to an embodiment.

Referring toFIGS. 2A and 2B, clients220may communicate through an access network222(which could be a wired or wireless network such as network303discussed with reference toFIGS. 3 and/or 4) with storage cluster (or network)224via distributed storage system logic or middleware160. Location of logic160is not limited to being between the storage cluster224and clients220, and in various embodiments logic160may be located in locations shown inFIG. 2A, as well as locations not shown, as long as logic160performs its operations as described herein. In one embodiment, a policy driven framework is defined for logic160to choose which object's fragments should be cached250, e.g., based on object size, object source, object destination, etc.

For example, an operator or user may specify a size-range based policy to cache all the fragments of objects whose size falls in specified ranges. Another example policy could be to cache all fragments of objects uploaded by a specific source/user or for a specific destination. Once an object has been erasure coded, the appropriate cache hint is generated252by the storage middleware (or logic160) when sending an individual fragment to the chosen backend storage nodes254. In an embodiment, the storage middleware may be implemented in a server (which may be referred to as the “proxy” or “gateway” in a distributed storage system) since the process of creating EC fragments is generally transparent to the client. If the policy dictates that an object be cached, a hint to cache the fragments of the object are sent along with each fragment to the storage cluster (or network)224(which may be implemented via a network of servers in an embodiment, e.g., where the network of servers may communicate via a computer network (such as those discussed herein with reference toFIGS. 3 and/or 4), including one or more types of network connections such as Ethernet, fiber optics, etc.). Conversely, if an object is not to be cached, hints to ensure none of the fragments are cached are sent (or alternatively no hints are sent at all). The cache hints are then extracted256at the backend storage nodes of the storage network and the hints are added to (e.g., file system) write operation258. In an embodiment, the cache/caching hint may also be used by a receiving storage node to determine where to store an EC fragment in the storage hierarchy of the storage node (or other backend nodes coupled to the receiving node). For example, in one embodiment, the receiving storage node could examine if an incoming EC fragment includes a hint, and if it does, store it in a faster tier in the storage hierarchy. Moreover, to form the storage hierarchy, the backend storage servers may have multiple different storage mediums (e.g., SSD, HDD, etc.) with different characteristics that are organized as tiers. For instance, the slowest storage medium may be the largest in size. A subset of the data stored in the slowest tier(s) could also be stored in the faster tier(s). All these operations may occur transparently to the user of the storage hierarchy. Hence, one goal is to keep data that is most relevant stored in the faster storage mediums. In another embodiment, the incoming hint could be used as an indication that the receiving node apply its local storage policy to choose the appropriate storage tier (whereas EC fragments with no hint may bypass the faster tiers entirely and be placed in the slowest tier without consulting the local storage policy).

As shown inFIG. 2A, the backend storage nodes may include a storage server (and/or other types of storage devices). The storage node may include one or more consistency processes226to ensure consistency of data stored, e.g., by passing an eviction hint from the object storage node to its (e.g., closest one or more (or two)) neighbors. The eviction process may repeat until all of the fragments for the given object are evicted. The storage node may also include a file system, cache controller (e.g., to cache or evict fragments per the cache hint), and/or one or more storage devices (such as a Hard Disk Drive (HDD), a Solid State Drive (SSD), combinations thereof (e.g., a hybrid drive with both an SSD and an HDD), any other non-volatile memory (such as those discussed herein)). Hence, as discussed herein, a node may include a server as well as one or more storage devices. The SSD portion of the storage node may be used for general data storage and/or caching of EC fragments in some embodiments. Also, the caching hints discussed herein may be used for both object read and write operations. Furthermore, in cases where there is limited cache space, some object fragments may still get evicted (even if they had a hint to cache on WRITE). In cases such as these, a subsequent READ can carry a hint that the fragments should be “read allocated in the cache” if they are not already there.

FIG. 2Cillustrates an object storage system utilizing EC fragments, according to an embodiment. The example ofFIG. 2Chighlights how the EC fragment handling may be performed in a distributed object store like Swift, but embodiments are not limited to Swift and other object stores may be utilized. As shown, individual EC fragments may include metadata, CRC (Cyclical Redundancy Code), and fragment data. Each object segment is encoded and sent to object nodes as EC fragments. The EC fragments may be appended to the previous fragment to build an EC fragment archive at object nodes. In an embodiment, the EC fragment transfer may end with an object etag (or entity tag, which is an opaque identifier assigned to a specific version of the object at the given URL (Uniform Resource Locator)) (which may be 16 bytes long in at least one embodiment) and a content length (“len”) which captures the length of the original object before fragmenting. Moreover, the fragment archive names may include a fragment index.

In one embodiment, the caching hints are added to the messages exchanged between the storage system components. For example, in Swift, the communication between the proxy and the object servers takes place via HTTP (HyperText Transfer Protocol) verbs. One embodiment extends the proxy to pass a hint using an HTTP header along with the HTTP verb for a given request, to indicate that the fragment be cached. An object server (or logic such as160) in the backend storage node may extract this hint and pass the hint to the cache controller. In the case of Ceph, message header extensions to the Ceph protocol can be used to pass the cache hints between the Ceph gateway and the backend Object Storage Daemon (OSD). In an embodiment, HTTP messages are sent between a proxy and backend storage nodes. The body of the message may contain the fragment. The headers of the message may be extended to carry the hint.

In one embodiment, when an object fragment is evicted from a backend node, the eviction may be coordinated from (e.g., every) other node(s) holding a fragment of the related object. One embodiment of this coordination for Swift involves the backend node sending an eviction hint to the Swift proxy which uses the Swift ring structure to identify the location of every other fragment of this object among the backend storage nodes to forward the eviction hint to each target node. Another embodiment uses one of the background consistency processes226(e.g., a Swift reconstructor) to pass the eviction hint from the object storage node to its closest two neighbors. This process repeats until all of the fragments for the given object are evicted. These two schemes may ensure that cache space is utilized efficiently in each of the nodes.

These schemes can be further improved using the principles of Erasure Coding. Namely, EC involves creating k data fragments which contain chunks of the original data and m code fragments which contain information to recreate the original data if one of the data fragments is lost, where k+m=n. First, caching hints are sent to all n nodes holding fragments (data or code) of the original data. This scheme aims to maximize performance. Second, caching hints are only sent to the k nodes holding data fragments. This allows for speedy retrieval of the data fragments to return the original data, while saving the cache space that would otherwise be used up to cache the m code fragments that are only required if one of the data fragments is lost (presumably not a common occurrence).

Furthermore, the decision for when to send eviction hints can similarly be modified depending on whether an operator/user prefers to aggressively clean up cache space (send hint when any fragment, code, or data, is evicted) or prefers to delay cleanup for potentially increasing the cache hits (send hint only when data fragment is evicted).

Additionally, referring toFIG. 2C, one might erroneously assume that simply deploying traditional LRU (Least Recently Used) caches on backend nodes should provide efficient caching of erasure coded data. For example, since consistent hashing distributes data equally amongst nodes, the EC fragments on each node would naturally enter the cache concurrently and also be evicted simultaneously. Per the below overview of a specific implementation of EC in Swift, it can be explained why LRU without any cluster-level coordination cannot utilize cache space effectively.

More particularly, on receiving an object to upload, the Swift Proxy first breaks the object into fixed sized ‘segments’. Each segment is then Erasure Coded into a specified (via a policy) number of fragments, n. The Swift ring is then consulted to determine the target nodes for the given object URL (Uniform Resource Locator). The proxy stripes the fragments across the identified target nodes such that the n fragments for each segment are sent to the same n target nodes. As the fragments are received at the target object storage node, they are appended to a fragment archive, which is stored on the object storage node. This approach is taken to reduce the number of files to store on the storage media (there would be an object explosion if each fragment were treated as its own object).

A consequence of this approach of appending fragments into an archive for a given fragment index is that a really large object would result in larger fragment archives to be placed on the target nodes for the uploaded object. Since the consistent hashing is done at the object level, ‘objects’ are distributed equally amongst the storage nodes in the cluster, but a large object would cause more data to be stored on a subset of the nodes. Thus, if caches are deployed on each of the nodes, fragments of a given piece of data would be evicted from the caches at different times. In other words, without coordination, cache space would be wasted since only some fragments of an object would be retrieved from the cache, whereas the overall object reconstruction would be gated by the slower responding nodes that retrieve their respective fragments from the slower storage medium that backs the cache tier.

FIG. 3illustrates a block diagram of a computing system300in accordance with an embodiment. The computing system300may include one or more central processing unit(s) (CPUs)302or processors that communicate via an interconnection network (or bus)304. The processors302may include a general purpose processor, a network processor (that processes data communicated over a computer network303), an application processor (such as those used in cell phones, smart phones, etc.), or other types of a processor (including a reduced instruction set computer (RISC) processor or a complex instruction set computer (CISC)).

Various types of computer networks303may be utilized including wired (e.g., Ethernet, Gigabit, Fiber, etc.) or wireless networks (such as cellular, including 3G (Third-Generation Cell-Phone Technology or 3rd Generation Wireless Format (UWCC)), 4G (Fourth-Generation Cell-Phone Technology), 4G Advanced, Low Power Embedded (LPE), Long Term Evolution (LTE), LTE advanced, etc.). Moreover, the processors302may have a single or multiple core design. The processors302with a multiple core design may integrate different types of processor cores on the same integrated circuit (IC) die. Also, the processors302with a multiple core design may be implemented as symmetrical or asymmetrical multiprocessors.

In an embodiment, one or more of the processors302may be the same or similar to the processors102ofFIG. 1. For example, one or more of the processors302may include one or more of the cores106and/or processor cache108. Also, the operations discussed with reference toFIGS. 1-2Cmay be performed by one or more components of the system300.

A chipset306may also communicate with the interconnection network304. The chipset306may include a graphics and memory control hub (GMCH)308. The GMCH308may include a memory controller310(which may be the same or similar to the memory controller120ofFIG. 1in an embodiment) that communicates with the memory114. The memory114may store data, including sequences of instructions that are executed by the CPU302, or any other device included in the computing system300. Also, system300includes logic125/160and/or NVM130in various locations such as shown or not shown. In one embodiment, the memory114may include one or more volatile memory devices such as random access memory (RAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), static RAM (SRAM), or other types of memory devices. Nonvolatile memory may also be utilized such as a hard disk drive, flash, etc., including any NVM discussed herein. Additional devices may communicate via the interconnection network304, such as multiple CPUs and/or multiple system memories.

The GMCH308may also include a graphics interface314that communicates with a graphics accelerator316. In one embodiment, the graphics interface314may communicate with the graphics accelerator316via an accelerated graphics port (AGP) or Peripheral Component Interconnect (PCI) (or PCI express (PCIe) interface). In an embodiment, a display317(such as a flat panel display, touch screen, etc.) may communicate with the graphics interface314through, for example, a signal converter that translates a digital representation of an image stored in a memory device such as video memory or system memory into display signals that are interpreted and displayed by the display. The display signals produced by the display device may pass through various control devices before being interpreted by and subsequently displayed on the display317.

A hub interface318may allow the GMCH308and an input/output control hub (ICH)320to communicate. The ICH320may provide an interface to I/O devices that communicate with the computing system300. The ICH320may communicate with a bus322through a peripheral bridge (or controller)324, such as a peripheral component interconnect (PCI) bridge, a universal serial bus (USB) controller, or other types of peripheral bridges or controllers. The bridge324may provide a data path between the CPU302and peripheral devices. Other types of topologies may be utilized. Also, multiple buses may communicate with the ICH320, e.g., through multiple bridges or controllers. Moreover, other peripherals in communication with the ICH320may include, in various embodiments, integrated drive electronics (IDE) or small computer system interface (SCSI) hard drive(s), USB port(s), a keyboard, a mouse, parallel port(s), serial port(s), floppy disk drive(s), digital output support (e.g., digital video interface (DVI)), or other devices.

The bus322may communicate with an audio device326, one or more disk drive(s)328, and a network interface device330(which is in communication with the computer network303, e.g., via a wired or wireless interface). As shown, the network interface device330may be coupled to an antenna331to wirelessly (e.g., via an Institute of Electrical and Electronics Engineers (IEEE) 802.11 interface (including IEEE 802.11a/b/g/n/ac, etc.), cellular interface, 3G, 4G, LPE, etc.) communicate with the network303. Other devices may communicate via the bus322. Also, various components (such as the network interface device330) may communicate with the GMCH308in some embodiments. In addition, the processor302and the GMCH308may be combined to form a single chip. Furthermore, the graphics accelerator316may be included within the GMCH308in other embodiments.

Furthermore, the computing system300may include volatile and/or nonvolatile memory. For example, nonvolatile memory may include one or more of the following: read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically EPROM (EEPROM), a disk drive (e.g.,328), a floppy disk, a compact disk ROM (CD-ROM), a digital versatile disk (DVD), flash memory, a magneto-optical disk, or other types of nonvolatile machine-readable media that are capable of storing electronic data (e.g., including instructions).

FIG. 4illustrates a computing system400that is arranged in a point-to-point (PtP) configuration, according to an embodiment. In particular,FIG. 4shows a system where processors, memory, and input/output devices are interconnected by a number of point-to-point interfaces. The operations discussed with reference toFIGS. 1-3may be performed by one or more components of the system400.

As illustrated inFIG. 4, the system400may include several processors, of which only two, processors402and404are shown for clarity. The processors402and404may each include a local memory controller hub (MCH)406and408to enable communication with memories410and412. The memories410and/or412may store various data such as those discussed with reference to the memory114ofFIGS. 1 and/or 3. Also, MCH406and408may include the memory controller120in some embodiments. Furthermore, system400includes logic125/160and/or NVM130in various locations such as shown or not shown. The logic125/160and/or NVM130may be coupled to system400via bus440or444, via other point-to-point connections to the processor(s)402or404or chipset420, etc. in various embodiments.

In an embodiment, the processors402and404may be one of the processors302discussed with reference toFIG. 3. The processors402and404may exchange data via a point-to-point (PtP) interface414using PtP interface circuits416and418, respectively. Also, the processors402and404may each exchange data with a chipset420via individual PtP interfaces422and424using point-to-point interface circuits426,428,430, and432. The chipset420may further exchange data with a high-performance graphics circuit434via a high-performance graphics interface436, e.g., using a PtP interface circuit437. As discussed with reference toFIG. 3, the graphics interface436may be coupled to a display device (e.g., display317) in some embodiments.

In one embodiment, one or more of the cores106and/or processor cache108ofFIG. 1may be located within the processors402and404(not shown). Other embodiments, however, may exist in other circuits, logic units, or devices within the system400ofFIG. 4. Furthermore, other embodiments may be distributed throughout several circuits, logic units, or devices illustrated inFIG. 4.

The chipset420may communicate with a bus440using a PtP interface circuit441. The bus440may have one or more devices that communicate with it, such as a bus bridge442and I/O devices443. Via a bus444, the bus bridge442may communicate with other devices such as a keyboard/mouse445, communication devices446(such as modems, network interface devices, or other communication devices that may communicate with the computer network303, as discussed with reference to network interface device330for example, including via antenna331), audio I/O device, and/or a data storage device448. The data storage device448may store code449that may be executed by the processors402and/or404.

In some embodiments, one or more of the components discussed herein can be embodied as a System On Chip (SOC) device.FIG. 5illustrates a block diagram of an SOC package in accordance with an embodiment. As illustrated inFIG. 5, SOC502includes one or more Central Processing Unit (CPU) cores520, one or more Graphics Processor Unit (GPU) cores530, an Input/Output (I/O) interface540, and a memory controller542. Various components of the SOC package502may be coupled to an interconnect or bus such as discussed herein with reference to the other figures. Also, the SOC package502may include more or less components, such as those discussed herein with reference to the other figures. Further, each component of the SOC package520may include one or more other components, e.g., as discussed with reference to the other figures herein. In one embodiment, SOC package502(and its components) is provided on one or more Integrated Circuit (IC) die, e.g., which are packaged onto a single semiconductor device.

As illustrated inFIG. 5, SOC package502is coupled to a memory560(which may be similar to or the same as memory discussed herein with reference to the other figures) via the memory controller542. In an embodiment, the memory560(or a portion of it) can be integrated on the SOC package502.

The I/O interface540may be coupled to one or more I/O devices570, e.g., via an interconnect and/or bus such as discussed herein with reference to other figures. I/O device(s)570may include one or more of a keyboard, a mouse, a touchpad, a display, an image/video capture device (such as a camera or camcorder/video recorder), a touch screen, a speaker, or the like. Furthermore, SOC package502may include/integrate items125,130, and/or160in an embodiment. Alternatively, items125,130, and/or160may be provided outside of the SOC package502(i.e., as a discrete logic).

Embodiments described herein can be powered by a battery, wireless charging, a renewal energy source (e.g., solar power or motion-based charging), or when connected to a charging port or wall outlet.

The following examples pertain to further embodiments. Example 1 may optionally include an apparatus comprising: logic to generate a plurality of fragments of a data object to be Erasure Coded (EC) prior to storage at a storage node of a plurality of storage nodes, wherein each of the plurality of fragments is to be transmitted with a caching hint to indicate whether that fragment is to be cached at the storage node. Example 2 may optionally include the apparatus of example 1, comprising logic to determine whether to cache the plurality of fragments based at least in part on one or more of: a size of the data object, a source of the data object, and a destination of the data object. Example 3 may optionally include the apparatus of example 1, comprising logic to determine where to store an EC fragment in a storage hierarchy of the storage node based at least in part on the caching hint. Example 4 may optionally include the apparatus of example 1, comprising logic to pass an eviction hint from the storage node to one or more neighboring storage nodes. Example 5 may optionally include the apparatus of example 1, wherein the logic is to pass an eviction hint to all storage that store the plurality of fragments. Example 6 may optionally include the apparatus of example 1, wherein the storage node is to comprise the logic. Example 7 may optionally include the apparatus of example 1, comprising logic to extract the caching hint and to add the caching hint to a write operation to a memory coupled to the storage node. Example 8 may optionally include the apparatus of example 1, wherein the plurality of storage nodes is to comprise a near storage node and/or a far storage node. Example 9 may optionally include the apparatus of example 1, wherein the plurality of storage nodes is to communicate via a network. Example 10 may optionally include the apparatus of example 9, wherein the network is to comprise a wired and/or a wireless network. Example 11 may optionally include the apparatus of example 1, wherein each of the plurality of storage nodes is to comprise one or more of: a hard disk drive, a solid state drive, and a hybrid drive. Example 12 may optionally include the apparatus of example 1, wherein each of the plurality of storage nodes is to comprise volatile memory and/or non-volatile memory. Example 13 may optionally include the apparatus of example 12, wherein the non-volatile memory is to comprise one or more of: nanowire memory, Ferro-electric Transistor Random Access Memory (FeTRAM), Magnetoresistive Random Access Memory (MRAM), flash memory, Spin Torque Transfer Random Access Memory (STTRAM), Resistive Random Access Memory, byte addressable 3-Dimensional Cross Point Memory, PCM (Phase Change Memory), write-in-place non-volatile memory, and volatile memory backed by a power reserve to retain data during power failure or power disruption. Example 14 may optionally include the apparatus of example 1, further comprising one or more of: at least one processor, having one or more processor cores, communicatively coupled to the storage node, a battery communicatively coupled to the apparatus, or a network interface communicatively coupled to the apparatus.

Example 15 may optionally include a method comprising: generating a plurality of fragments of a data object to be Erasure Coded (EC) prior to storage at a storage node of a plurality of storage nodes, wherein each of the plurality of fragments is transmitted with a caching hint to indicate whether that fragment is to be cached at the storage node. Example 16 may optionally include the method of example 15, further comprising determining whether to cache the plurality of fragments based at least in part on one or more of: a size of the data object, a source of the data object, and a destination of the data object. Example 17 may optionally include the method of example 15, further comprising determining where to store an EC fragment in a storage hierarchy of the storage node based at least in part on the caching hint. Example 18 may optionally include the method of example 15, further comprising passing an eviction hint from the storage node to one or more neighboring storage nodes. Example 19 may optionally include the method of example 15, further comprising extracting the caching hint and to add the caching hint to a write operation to a memory coupled to the storage node. Example 20 may optionally include the method of example 15, further comprising the plurality of storage nodes communicating via a network.

Example 21 may optionally include one or more computer-readable medium comprising one or more instructions that when executed on at least one processor configure the at least one processor to perform one or more operations to: generate a plurality of fragments of a data object to be Erasure Coded (EC) prior to storage at a storage node of a plurality of storage nodes, wherein each of the plurality of fragments is to be transmitted with a caching hint to indicate whether that fragment is to be cached at the storage node. Example 22 may optionally include the one or more computer-readable medium of example 21, further comprising one or more instructions that when executed on the processor configure the processor to perform one or more operations to determine whether to cache the plurality of fragments based at least in part on one or more of: a size of the data object, a source of the data object, and a destination of the data object. Example 23 may optionally include the one or more computer-readable medium of example 21, further comprising one or more instructions that when executed on the processor configure the processor to perform one or more operations to determine where to store an EC fragment in a storage hierarchy of the storage node based at least in part on the caching hint. Example 24 may optionally include the one or more computer-readable medium of example 21, further comprising one or more instructions that when executed on the processor configure the processor to perform one or more operations to pass an eviction hint from the storage node to one or more neighboring storage nodes.

Example 25 may optionally include a computing system comprising: a processor; memory, coupled to the processor, to store data corresponding to object stores; and logic to generate a plurality of fragments of a data object to be Erasure Coded (EC) prior to storage at a storage node of a plurality of storage nodes, wherein each of the plurality of fragments is to be transmitted with a caching hint to indicate whether that fragment is to be cached at the storage node. Example 26 may optionally include the system of example 25, comprising logic to determine whether to cache the plurality of fragments based at least in part on one or more of: a size of the data object, a source of the data object, and a destination of the data object. Example 27 may optionally include the system of example 25, comprising logic to determine where to store an EC fragment in a storage hierarchy of the storage node based at least in part on the caching hint. Example 28 may optionally include the system of example 25, comprising logic to pass an eviction hint from the storage node to one or more neighboring storage nodes. Example 29 may optionally include the system of example 25, wherein the logic is to pass an eviction hint to all storage that store the plurality of fragments. Example 30 may optionally include the system of example 25, wherein the storage node is to comprise the logic. Example 31 may optionally include the system of example 25, comprising logic to extract the caching hint and to add the caching hint to a write operation to the memory coupled to the storage node. Example 32 may optionally include the system of example 25, wherein the plurality of storage nodes is to comprise a near storage node and/or a far storage node. Example 33 may optionally include the system of example 25, wherein the plurality of storage nodes is to communicate via a network. Example 34 may optionally include the system of example 25, wherein each of the plurality of storage nodes is to comprise one or more of: a hard disk drive, a solid state drive, and a hybrid drive. Example 35 may optionally include the system of example 25, wherein each of the plurality of storage nodes is to comprise volatile memory and/or non-volatile memory. Example 36 may optionally include the system of example 25, further comprising one or more of: the processor, having one or more processor cores, communicatively coupled to the storage node, a battery communicatively coupled to the apparatus, or a network interface communicatively coupled to the apparatus.

Example 37 may optionally include an apparatus comprising means to perform a method as set forth in any preceding example. Example 38 comprises machine-readable storage including machine-readable instructions, when executed, to implement a method or realize an apparatus as set forth in any preceding example.

Additionally, such tangible computer-readable media may be downloaded as a computer program product, wherein the program may be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of data signals (such as in a carrier wave or other propagation medium) via a communication link (e.g., a bus, a modem, or a network connection).

Reference in the specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least an implementation. The appearances of the phrase “in one embodiment” in various places in the specification may or may not be all referring to the same embodiment.

Thus, although embodiments have been described in language specific to structural features, numerical values, and/or methodological acts, it is to be understood that claimed subject matter may not be limited to the specific features, numerical values, or acts described. Rather, the specific features, numerical values, and acts are disclosed as sample forms of implementing the claimed subject matter.