Patent ID: 12222929

DETAILED DESCRIPTION

The subject matter of embodiments of the present disclosure is described herein with specificity to meet statutory requirements, but this description is not intended to limit the scope of the claims. The claimed subject matter may be embodied in other ways, may include different elements or steps, and may be used in conjunction with other existing or later developed technologies. This description should not be interpreted as implying any required order or arrangement among or between various steps or elements except when the order of individual steps or arrangement of elements is explicitly noted as being required.

Embodiments of the disclosure will be described more fully herein with reference to the accompanying drawings, which form a part hereof, and which show, by way of illustration, exemplary embodiments by which the disclosure may be practiced. The disclosure may, however, be embodied in different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy the statutory requirements and convey the scope of the disclosure to those skilled in the art.

Among other things, the present disclosure may be embodied in whole or in part as a system, as one or more methods, or as one or more devices. Embodiments of the disclosure may take the form of a hardware implemented embodiment, a software implemented embodiment, or an embodiment combining software and hardware aspects. For example, in some embodiments, one or more of the operations, functions, processes, or methods described herein may be implemented by one or more suitable processing elements (such as a processor, microprocessor, CPU, GPU, TPU, controller, etc.) that is part of a client device, server, network element, remote platform (such as a SaaS platform), an “in the cloud” service, or other form of computing or data processing system, device, or platform.

The processing element or elements may be programmed with a set of executable instructions (e.g., software instructions), where the instructions may be stored on (or in) one or more suitable non-transitory data storage elements. In some embodiments, the set of instructions may be conveyed to a user through a transfer of instructions or an application that executes a set of instructions (such as over a network, e.g., the Internet). In some embodiments, a set of instructions or an application may be utilized by an end-user through access to a SaaS platform or a service provided through such a platform.

In some embodiments, one or more of the operations, functions, processes, or methods described herein may be implemented by a specialized form of hardware, such as a programmable gate array, application specific integrated circuit (ASIC), or the like. Note that an embodiment of the inventive methods may be implemented in the form of an application, a sub-routine that is part of a larger application, a “plug-in”, an extension to the functionality of a data processing system or platform, or other suitable form. The following detailed description is, therefore, not to be taken in a limiting sense.

The disclosed system and accompanying methods for handling Implicit transactions in a system incorporating a hybrid cloud cache may utilize a portion of the hybrid cloud cache, termed an “orphanage” herein. The orphanage may comprise a separate internal meta-namespace for objects that are associated with Implicit metadata transactions occurring in the cloud. In some embodiments, objects sent to the orphanage stay in the orphanage until all Explicit transactions on them (or that may impact them) are completed in the hybrid cloud cache. In some embodiments, the system and methods also introduce an orphanage processor, which at regular intervals queries the cloud and transfers these objects (those associated with Implicit metadata transactions) to a “purgatory” (temporary storage location) portion of the hybrid cloud cache in a situation where the system and methods determine that the objects have been deleted in the cloud. The objects that may have moved to a new location in the cloud may be reclaimed from the orphanage if (or when) the new location information is cached by the hybrid cloud cache.

In some embodiments, the hybrid cloud cache may keep folders in a metadata file system (MDFS), which may be a structured hierarchy or tree of cached folders. At each level in the MDFS, the hybrid cloud cache may maintain a meta-file that contains metadata for folders and files that are within that folder. In some embodiments, and in some cases triggered by a specific event (examples of which are described herein), a cache refresh manager of the hybrid cloud cache may fetch a version of a meta-file from the cloud and compare the version with a current version of the meta-file stored in the hybrid cloud cache. The comparison of the two meta-files may be used to indicate which objects are to be selected as candidates for transfer to the orphanage.

Objects may be candidates for transfer to the orphanage if they meet one or more of the following criteria, provided the situation is not caused by the cache pushing its own Explicit transaction to the cloud:1. The object is present only in the meta-file version from the hybrid cloud cache;2. The object is present in both meta-files, but the name does not match; or3. The object is present only in the meta-file version from the cloud and the object is also cached by the hybrid cloud cache. As an example of how this situation might occur, assume that an object path is “/Shared/Design/Documents” and its identifier is ID1. The object is also cached. The following Implicit transaction happens in cloud: Move “/Shared/Design/Documents” to “/Shared/Construction/Documents”. The hybrid cloud cache fetches the meta-file for “/Shared/Construction” and finds that the object “Documents” with ID1 is present in the new meta-file. The hybrid cloud cache knows that the object with ID1 is cached (from #1) but the location is different.
When an object is moved to the orphanage using the disclosed system and methods, its unique identifiers are used as its name to avoid name conflicts that may be caused by the presence of objects with the same name at different levels in the MDFS hierarchy.

The orphanage processor described herein is an entity that operates at a regular interval (which is configurable, but may be set to a default value of, for example, 1 hour) and performs a bottom-up traversal of the orphanage (internal) namespace, during which it performs the following processes or “checks” on each leaf object in the namespace hierarchy:1. Determines if there are there any outstanding Explicit Transactions involving the object; and2. Determines if the object still exists in the cloud namespace.
If both determinations return a negative response, then the orphanage processor moves the leaf object to the purgatory of the hybrid cloud cache. Otherwise, the object is left as is in the orphanage and the bottom-up traversal is aborted.

From one perspective, the functioning of the orphanage processor can be represented as:if outstanding Explicit transactions involving the object are present, or the object still exists in the cloud namespace, then abort the bottom-up-traversal; elseif the object does not exist in cloud namespace, then move the object to the purgatory of the hybrid cloud cache.

In some embodiments, a meta-data cache manager that is used as part of the process for caching new objects may check for an object's existence in the orphanage. If the object is found in the orphanage, then the system and methods may reclaim the object from the orphanage and transfer it to its new location in the hybrid cloud cache that matches with its location in the cloud.

The described system and methods provide a concept of an orphanage as a separate internal namespace for objects whose existence or location in the cloud namespace is not known to a hybrid cloud cache. Additionally, the system and methods provide the ability to reclaim objects from the orphanage and transfer the reclaimed objects to the correct location when needed. Further, the system and methods provide an orphanage processor which determines the correct placement for orphaned objects if such objects do not exist in the cloud. The system and methods also provide the ability to move objects back and forth between the orphanage and a user-visible namespace in the presence of Explicit transactions as well as the ability to complete those transactions without violating the transactional integrity of the hybrid cloud cache.

With reference toFIGS.1and2, the system (element100inFIG.1) and methods handle Implicit transactions in a hybrid cloud cache using a portion of the hybrid cloud cache, referred to as an orphanage (element212ofFIG.2). The orphanage212of the hybrid cloud cache may comprise a separate internal meta-namespace for objects that are associated with Implicit metadata transactions of the cloud250. In some embodiments, objects sent to the orphanage212stay in the orphanage until all Explicit transactions involving them are completed in the hybrid cloud cache.

The system100and methods also introduce an orphanage processor, which at regular intervals queries the cloud and transfers these objects to a purgatory (element214ofFIG.2) of the hybrid cloud cache if the system100and methods determine that the objects have been deleted in the cloud250. Additionally, the objects that may have moved to a new location in the cloud250may be reclaimed from the orphanage212if (or when) the new location is cached by the hybrid cloud cache.

As an example of this situation, assume an object ID1 is in the orphanage. The object ID1 exists in the cloud in a location “/Shared/Documents/Reservations”. A user tries to perform an operation on “/Shared/Documents/Reservation”. The hybrid cloud cache finds that it has cached “/Shared/Documents” but not “/Shared/Documents/Reservation”. The hybrid cloud cache tries to cache “/Shared/Documents/Reservation” and finds that it already has an object with ID1 in the orphanage and moves it from Orphanage to the new-location “/Shared/Documents/Reservation. This step is referred as a “reclaim”.

A hybrid cloud cache may keep folders in the MDFS (e.g., MDFS element204inFIG.2), which may be a structured hierarchy or tree of cached folders. At each level in the MDFS, the hybrid cloud cache may maintain a meta-file (e.g., meta-file210ofFIG.2) that contains metadata for folders and files that are within that folder. In some embodiments, a cache refresh manager (element118ofFIG.1) of the hybrid cloud cache may fetch a new copy of meta-file from the cloud250and compare the new copy with a current copy in the hybrid cloud cache. The cache refresh (fetch) operation may occur at scheduled times, regularly at an interval set in advance or due to execution of specific logic, or in response to a specific event or set of events (examples of which are described herein), for example.

As mentioned, the comparison of current and new meta-files is used to identity which objects are selected for transfer to the orphanage212. An object may be a candidate for transfer to the orphanage if they meet any of the following criteria, provided the situation is not caused by a cache pushing its own Explicit transaction to the cloud250:the object is present only in the current copy of the meta-file from the hybrid cloud cache; orthe object is present in both meta-file copies, but the name does not match; orthe object is present only in the new copy from the cloud250and the object is also cached by the hybrid cloud cache.

When the object(s) are moved to the orphanage212using the described system and methods, their unique identifiers may be used as their name(s) to avoid name conflicts that may be caused by the presence of objects with the same name at different levels in the hierarchy.

The orphanage processor may be an element or process that operates at a regular interval and performs a bottom-up traversal of the orphanage namespace. The orphanage processor may cause the following checks or evaluations to be performed for each leaf object in the namespace tree or hierarchy:are there any outstanding Explicit transactions involving the object?; anddoes the object still exist in the cloud namespace?
If both inquiries (checks) return a negative response, then the orphanage processor moves the leaf object to the purgatory214of the hybrid cloud cache. Otherwise, the object is left as is in the orphanage212and the bottom-up traversal is aborted. As mentioned, the function of the orphanage processor can also be represented as:if an outstanding Explicit transaction is present or the object still exists in the cloud namespace, then abort the bottom-up-traversal; elseif the object does not exist in cloud namespace, then move the object to the purgatory214of the hybrid cloud cache.
In some embodiments, a metadata cache manager108used as part of a process for caching new objects may check for an object's existence in the orphanage212. If the object is found, then the system and methods may reclaim the object from the orphanage212and transfer the object to its new location in the hybrid cloud cache, where that new location matches its location in the cloud250.

FIG.1is a schematic diagram illustrating a system including components for supporting functionality for handling implicit transactions in a hybrid cloud cache, in accordance with some embodiments. The system100may be configured to perform any of the functionality described herein. The system100may be configured to support, but is not limited to supporting, caching services, hybrid cloud services, cloud services, transaction system services, content collaboration services, error handling services, content delivery services, monitoring services, cloud computing services, satellite services, telephone services, voice-over-internet protocol services (VoIP), software as a service (SaaS) applications, platform as a service (PaaS) applications, gaming applications and services, social media applications and services, operations management applications and services, productivity applications and services, mobile applications and services, and any other computing applications and services.

In some embodiments, the system100may be included within another system, may be a separate system from another system, and/or may be a subsystem of another system. System100may be implemented, at least in part, as a set of computer-executable instructions that when executed by a suitably programmed processor, cause the processor or a server or device of which the processor is a part, to perform one or more of the methods, functions, operations, or processes described herein.

The system100may include, but is not limited to including, a REST Application Programming Interface (API)102(or other API), a smart cache API layer104(or other API layer), a journaling system106(which may include any number of journals), a metadata cache manager108, a data cache manager110, a metadata store112, a data store114, a policy enforcer116, a cache refresh manager118, a cloud-file-storage (CFS) client layer120, a recovery manager122, and a policy engine124. The system100may include a hybrid cloud cache that may utilize the various components of the system100and may communicatively link and upload data to and/or download data from a cloud system. The REST API102may serve as the interface between user devices and/or other devices and applications supporting the functionality of the system100. The REST API102may be configured to receive API calls from clients (e.g., to access a cache and/or to perform other functions), such as from user devices. The smart cache API layer104may serve as an intermediary between the REST API102and the journaling system106and the other components of the system100, such as, but not limited to, the cloud-file-storage system and/or other components or applications of the system100.

The journaling system106may include one or more journals. One or more of the journals may be configured to record transactions associated with operations requested by a user (including, for example, data and metadata associated with the operations). The metadata may be information that describes the data and/or operations, what is in the data, and/or the type of operation. In some embodiments, the journals may be a circular log, buffer, and/or other data structure.

In some embodiments, the journals may transfer records containing information associated with the operations to the cloud, such as to a cloud-based platform or system. Once the records are transferred to the cloud, the records may be deleted from (or overwritten in) the journal(s). The journal(s) may be utilized to ensure that the operations requested by users/clients are carried out and performed, even if the system100crashes or suffers another type of interruption. Data and metadata associated with the operations may be managed by the data cache manager110and the metadata cache manager108, respectively. In some embodiments, the records including the data and metadata may be stored in the data store114and the metadata store112, respectively.

The system100may include a policy enforcer116, which may be configured to enforce the policies and rules associated with the system100. The cache refresh manager118may be configured to refresh any cache in the system100. For example, the cache refresh manager118may be configured to ensure that data and/or metadata recently stored in a particular cache is current and/or accurate. The system100may also include a cloud-file-storage system client layer120, which may be utilized to facilitate the providing of the records associated with the operations from the journal(s) to a cloud-file-storage system (e.g., the cloud system). Additionally, the system100may include a recovery manager122, which may be configured to recover lost data and/or metadata and ensure that the integrity of the data in the journals and/or caches of the system100is preserved. The system100may further include a policy engine124, which may be configured to generate and/or conduct various operations associated with policies and/or rules to be utilized with the system100.

With regards to policy engine124, examples of policies that may be implemented by the engine include but are not limited to, or required to include the following:a) Storage Management policies (e.g., how much disk space should be used by the cache);b) Caching policies—what should be cached, what part of cache should be updated and when; andc) Eviction policies—What should be evicted/removed from cache if there is a need to do so.
In some embodiments, one or more policies may be implemented in whole or in part by a rule-base, a trained machine learning model, or other decision process.

Referring now toFIG.2, which is a schematic diagram illustrating a system including additional components for supporting the functionality of the system ofFIG.1, in accordance with some embodiments.FIG.2illustrates an example architecture for a metadata file system200for use with the system100. The system200may be configured to perform the functionality described herein and may represent an on-disk structure illustrative of how the hybrid cloud cache system100ofFIG.1stores its data and/or metadata.

The system200may include a cache202for storing data, files and/or folders, a DCFS (data cache file system)208of the cache202for storing files and/or data, a MDFS (metadata file system)204of the cache202for storing metadata (for all objects in the cache, except those in the Lost+Found), a meta-file210of the MDFS204for storing metadata associated with files and/or data, an orphanage212for providing a separate internal meta-namespace for objects that are associated with Implicit metadata transactions, a purgatory214for providing a location to which are transferred objects deleted from the cloud, a transient area216for data for files not yet transferred to the cloud, a Lost+Found subsystem206, a DCFS218of the Lost+Found subsystem206, a MDFS220of the Lost+Found subsystem206(for metadata for the objects in the Lost+Found), a meta-file221of the Lost+Found subsystem206, a shared portion222, a meta-file224of the shared portion222, documents226of the shared portion222, a meta-file228of the documents226, general information230, a meta-file232of the general information230, design documents234, a meta-file236of the design documents234, a private portion238, a meta-file240of the private portion238, a user1242, a meta-file244associated with the user1242, a user2246, a meta-file248associated with the user2246, and a cloud250(i.e., a cloud-based platform or data storage).

Folders222,226,230,234,238,242, and246are examples of user folders in the namespace. There can be any number of such folders arranged in a hierarchy. The figure shows them as examples to demonstrate that for each user folder a meta-file is created in the hybrid cloud cache which stores the metadata associated with that folder.

It should be noted that the elements, components, or processes illustrated inFIG.2as being in the shared portion222and in the private portion238are designated as such for purposes of an example, and are not intended to be limited to the specific files, documents, and/or other components associated with each shown inFIG.2. In other examples, the shared portion222and/or the private portion238may have any number of user-desired folder-hierarchies inside the shared portion222and/or the private portion238.

User1242and User2246may be humans, computing devices, programs, processes, clients, robots, and/or other types of users. The meta-files210,221,224,228,232,236,240,244, and248may serve as files that describe data, files and/or folders associated with the corresponding component of the system200to which they are connected. In some embodiments, the meta-files210,221,224,228,232,236,240,244, and248may include attributes, such as, but not limited to, name, size, user, number of versions, upload-time, another attribute, or a combination thereof.

In some embodiments, the white boxes to the left of the black boxes inFIG.2may form a part of an internal meta-namespace of the hybrid cloud cache of the system100that is not visible to users/clients. In some embodiments, the black boxes inFIG.2may form a part of a user-visible namespace that is visible to users/clients.

Operatively, the system100and/or system200may operate as described with reference to the method300shown inFIG.3. A combination of the components, devices, programs, or networks of the system100and/or system200may execute and/or implement the operations or functions described herein. As shown inFIG.3, an exemplary method300for handling Implicit transactions in a hybrid cloud cache may include one or more of the steps or stages shown, with one or more of the steps being optional in some examples.

In some embodiments, the method300may proceed as follows:At step302, the method300may include maintaining folders in a MDFS of a hybrid cloud cache and maintaining a meta-file at each level of a hierarchy of cached folders maintained in the MDFS of the hybrid cloud cache;The meta-file may include metadata for folders, files, and/or other objects within a particular folder associated with a level of the hierarchy;At step304, the method300may include fetching, from a cloud (e.g. cloud250), a new copy of a particular meta-file corresponding to a particular level in the hierarchy of folders;In some embodiments, the cache refresh manager118of the hybrid cloud cache may perform the fetching of the new copy of the meta-file;In some embodiments, the fetching may be performed at selected time intervals and/or in response to a particular event;At step306, the method300may include comparing the new copy of the meta-file from the cloud to the current copy of the meta-file from the hybrid cloud cache;The comparison may involve comparing the information in each of the meta-files pertaining to one or more objects referenced in the meta-files;At step308, the method300may include determining if an object associated with an Implicit metadata transaction is present only in the current copy of the meta-file, or if the object is present in both the new copy of the meta-file and the current copy of the meta-file but the names of the objects do not match, or if the object is present only in the new copy of the meta-file and is also cached in the hybrid cloud cache;If the outcome of the determination in step308is the “No” branch, then the method300may proceed back to step302and again to step304;If the outcome of the determination in step308is the “Yes” branch, then the method300may proceed to step310, which may include transferring the object to an orphanage212of the hybrid cloud cache;In some embodiments, the orphanage212may be a separate internal-meta namespace for objects that are associated with Implicit metadata transactions;when moving an object (and meta-file) to the orphanage212, a unique identifier may be used to replace the name of the object to avoid name conflicts that may be caused by the presence of objects with the same name at different levels in the cache hierarchy;The method300may then proceed to step312, which may include conducting a bottom-up traversal of the orphanage namespace, such as by utilizing the orphanage processor element or process described herein;At step314, the method300may include determining if an outstanding Explicit transaction involving the object is present, or if the object still exists in the cloud namespace;If there is no outstanding explicit transaction for the object and the object does not exist or no longer exists in the cloud (the “No” branch of step314), then the method300may proceed to step316, which may include moving the object to a purgatory214of the hybrid cloud cache;In some embodiments, the purgatory214may serve as a resting place for objects that have been permanently deleted in the cloud;If at step314it is determined that there is an outstanding Explicit transaction to be conducted for the object that is present in the cache, or that the object still exists in the cloud namespace (the “Yes” branch of step314), then the method300may proceed to step318, which may include aborting the bottom-up traversal of the orphanage namespace for that subtree and leaving the object in the orphanage212.

In some embodiments, steps or stages302-310may be performed by the cache refresh manager118as described herein, and steps or stages312-318may be performed by the orphanage processor as described herein. Further, the method300may continuously operate as additional transactions are received and/or processed by system100or system200, and/or as new meta-files are generated in the systems and as updates are made in the systems. The method300may further incorporate any of the features and functionality described for the system100, the system200, another method disclosed, or as otherwise described herein.

FIG.4is a flow chart or flow diagram illustrating an example method400for reclaiming an object from an orphanage, in accordance with some embodiments. A combination of the components, devices, programs, or networks of the system100and/or system200may execute and/or implement the operations or functions described in the following with reference toFIG.4:At step402, the method400may include receiving a request form a user to perform an operation on an object, such as a file or folder;At step404, the method400may include determining if the object is cached in the hybrid cloud cache;If the object is not cached (corresponding to the “No” branch of step404), then the method400may proceed to step406, which may include caching the object from the cloud250(i.e., obtaining the object from the cloud-based platform);If at step404, the object is determined to be cached in the hybrid cloud cache (corresponding to the “Yes” branch of step404), then the method400may proceed to step408, where at step408, the method400may include determining if the object is in the orphanage212of the hybrid cloud cache;In some embodiments, the metadata cache manager108, as part of a process of caching new objects in the hybrid cloud cache, may check for their existence in the orphanage212;If the object is determined to be in the orphanage212of the hybrid cloud cache (the “Yes” branch of step408), then the method400may proceed to step410, where at step410, the method400may include reclaiming the object from the orphanage212and moving the object to its new location in the hybrid cloud cache that matches its corresponding location in the cloud250;If at step408, the object is determined to not be in the orphanage212(the “No” branch of step408), then the method400may proceed to step412, where at step412, the method400may include performing the requested operation on the object that the user requested in step402.

In some embodiments, the metadata cache manager108may perform steps406and410. The method400may continuously operate as additional user requests are received and/or processed by the system100,200. Further, the method400may incorporate any of the features and functionality described for the system100, the system200, the method300, another method disclosed, or as otherwise described herein.

The illustrations of arrangements described herein are intended to provide a general understanding of the structure of various embodiments, and they are not intended to serve as a complete description of all the elements and features of apparatus and systems that might make use of the structures described herein. Other arrangements may be utilized and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. Figures are also merely representational and may not be drawn to scale. Certain proportions thereof may be exaggerated, while others may be minimized. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

Thus, although specific arrangements have been illustrated and described herein, it should be appreciated that any arrangement calculated to achieve the same purpose may be substituted for the specific arrangement shown. This disclosure is intended to cover all adaptations or variations of various embodiments and arrangements of the system and methods described. Combinations of the above arrangements, and other arrangements not specifically described herein, will be apparent to those of skill in the art upon reviewing the above description. Therefore, it is intended that the disclosure not be limited to the particular arrangement(s) or embodiments disclosed, but include all embodiments and arrangements falling within the scope of the appended claims.

The foregoing is provided for purposes of illustrating, explaining, and describing one or more embodiments of the disclosure. Modifications and adaptations to these embodiments will be apparent to those skilled in the art and may be made without departing from the scope or spirit of this invention. Upon reviewing the embodiments, it would be evident to an artisan with ordinary skill in the art that said embodiments can be modified, reduced, or enhanced without departing from the scope and spirit of the claims described below.