Context-aware durability of file variants

Disclosed are various embodiments for context-aware durability of variants of files. A first data store is associated with a first level of data durability, and a second data store is associated with a second level of data durability that is lower than the first level of data durability. A data object is stored in the second data store when the data object is a derivative of another data object stored in the first data store. A data object is stored in the first data store when the data object is not a derivative of another data object stored in the first data store.

CROSS REFERENCE TO RELATED APPLICATIONS

The following application is related to co-pending U.S. patent application entitled “VARIANTS OF FILES IN A FILE SYSTEM” filed on even date Ser. No. 13,047,343, which is incorporated herein by reference in its entirety.

BACKGROUND

Network connections for different computing devices may have different capabilities. For example, a network connection for a home laptop or workstation may be a reliable, high speed connection. By contrast, a network connection for a smartphone may be a relatively low speed connection with a relatively high probability of packet loss. In addition, different computing devices may have different physical characteristics. For example, a home workstation may have a large display, a laptop may have a relatively smaller display, and a smartphone may have a handheld display.

DETAILED DESCRIPTION

The present disclosure relates to providing a file system that includes variants of files. With the ubiquitous availability of network connectivity for computing devices, users are increasingly seeking ubiquitous availability for their data files. For example, users may want to listen to an audio file from their smartphone, from their office workstation, from their laptop at home, and/or from other computing devices. However, users may be inconvenienced by having to carry around portable storage devices and connect them to their computing devices. In addition, it may be difficult to access a file stored on a portable storage device from multiple computing devices.

Various embodiments of the present disclosure relate to remote storage of data files and variants of files according to a utility computing model. A file system may be employed to provide access to files stored in a cloud computing resource comprising a networked plurality of computing devices. In some embodiments, the file system may correspond to a virtual file system where the actual data objects of the files are stored in a separate data storage system.

Variants of files may correspond to version of files that are related to other files or derived or generated from other files. Such variants may be customized for particular client computing devices of a customer. In various embodiments, variants may exist transparently to the customer such that a customer may be sent a file variant instead of the original, for example, when the customer is using a particular type of client computing device to request the file. In some embodiments, context-aware durability of variants may be employed such that data that correspond to the variants are stored in data stores that offer reduced durability of data. In the following discussion, a general description of the system and its components is provided, followed by a discussion of the operation of the same.

With reference toFIG. 1, shown is a networked environment100according to various embodiments. The networked environment100includes one or more computing devices103in data communication with one or more computing devices106and one or more clients109by way of a network112. The network112includes, for example, the Internet, intranets, extranets, wide area networks (WANs), local area networks (LANs), wired networks, wireless networks, or other suitable networks, etc., or any combination of two or more such networks.

The computing device103may comprise, for example, a server computer or any other system providing computing capability. Alternatively, a plurality of computing devices103may be employed that are arranged, for example, in one or more server banks or computer banks or other arrangements. For example, a plurality of computing devices103together may comprise a cloud computing resource, a grid computing resource, and/or any other distributed computing arrangement. Such computing devices103may be located in a single installation or may be distributed among many different geographical locations. For purposes of convenience, the computing device103is referred to herein in the singular. Even though the computing device103is referred to in the singular, it is understood that a plurality of computing devices103may be employed in the various arrangements as described above.

Various applications and/or other functionality may be executed in the computing device103according to various embodiments. Also, various data is stored in a data store115that is accessible to the computing device103. The data store115may be representative of a plurality of data stores115as can be appreciated. The data stored in the data store115, for example, is associated with the operation of the various applications and/or functional entities described below.

The components executed on the computing device103, for example, include a file service118, a variant service121, one or more variant encoders122, one or more applications124, and other applications, services, processes, systems, engines, or functionality not discussed in detail herein. The file service118is executed to maintain a file hierarchy of files and folders in based file systems for users. To this end, the file service118may support various file-related operations such as, for example, creating files, deleting files, modifying files, setting permissions for files, downloading files, and/or other operations.

The variant service121is executed to create and maintain variants of files in the file systems. To this end, the variant service121may be configured to detect situations in which file variants are to be generated, for example, for particular types of clients, for particular applications124, and/or for other contexts. The variant service121may be executed in conjunction with one or more variant encoders122to generate the variants. The variant encoders122may correspond, for example, to audio codecs, video codecs, image resizing applications, data compression applications, data transformation applications, data import applications, and/or other encoders of data. The variant service121may also be configured to delete unnecessary variants periodically.

The applications124correspond to hosted applications that may access the data stored in the file systems. Various applications124may, for example, have a web-based interface and may serve up network pages, such as web pages or other forms of network content, to facilitate user interaction. Other applications124may be internal applications and may not have a web-based interface. Non-limiting examples of applications124may include a photo organizing tool, a music file organizer and playback tool, a word processor, a spreadsheet, an email application, and so on.

The data stored in the data store115includes, for example, file systems130, files133, folders136, file variants137, client information138, and potentially other data. The file systems130correspond to logical file hierarchies that are maintained within the computing device103on behalf of one or more users. In various embodiments, the logical file hierarchies embodied in the file systems130are abstractions that do not directly correspond to any particular physical storage arrangements such as disk drives, portable storage media, etc. Each account-level user may have its own file system130, or the file systems130may be shared by multiple account-level users.

Each file system130may be associated with account data131, one or more client associations132, and/or other file system metadata. The account data131includes various data regarding authorized users of the computing device103. Such account data131may include, for example, usernames, passwords, security credentials, file management permissions, storage quotas and limitations, authorized applications, billing information, and/or other data. The client associations132describe various clients that are associated with the file system130. The clients may correspond, for example, to client computing devices such as clients109or applications/services such as applications124. As a non-limiting example, a user may use a tablet computing device, a desktop workstation, and a smartphone to access the file system130. The client associations132may also describe various capabilities of clients109, such as types of browser applications, plug-ins present, email applications, and so on. Accordingly, the client associations132may permit the variant service121and/or other applications to determine types of clients that are associated with each file system130. The client associations132may be detected automatically and/or may be manually configured.

Each file system130may include one or more files133and one or more folders136. A file133may include a file name142, a unique identifier145, a data object reference148, a parent object reference151, one or more permissions, a file creation date, a file modification date, and/or other data. The file name142may correspond to a human-readable character string that identifies the contents of the file133. In some embodiments, the file name142may be limited in character length and may be limited in terms of allowed characters. In some cases, such limitations may be enforced for compatibility reasons. In addition, the file name142may be restricted to be unique for a folder136and/or for a file system130in various embodiments.

The unique identifier145corresponds to a number, character string, or other identifier that uniquely identifies the file133across the file system130and/or all of the file systems130. In one embodiment, a unique identifier145of a file133coupled with an identifier of a file system130may correspond to an identifier that is unique across all of the file systems130. In another embodiment, the unique identifier145of a file133may itself be unique across all of the file systems130.

The data object reference148corresponds to a reference that may identify a data object associated with the file133that is stored in a data store such as the data store115or another data store. In one embodiment, the data object reference148may include a uniform resource identifier (URI). In another embodiment, the data object reference148may include a key value for obtaining the data object. The data store from which to obtain the data object may be determined implicitly or explicitly, for example, from a definition in the data object reference148.

The parent object reference151corresponds to a reference that may identify a folder136or file133that is considered to be the parent of the file133. In this way, folders136are associated with files133. In some situations, the parent object reference151may include a special identifier (e.g., NULL,0, etc.) that identifies the root folder136of the file system130.

Each folder136may include, for example, a folder name157, a unique identifier160, a parent object reference163, permissions, and/or other data. It is noted that, in some embodiments, a folder136may be implemented using the same data structure as a file133. In other embodiments, a folder136may be implemented using a data structure that is different from the file133.

The unique identifier160corresponds to a number, character string, or other identifier that uniquely identifies the folder136across the file system130and/or all of the file systems130. In one embodiment, a unique identifier160of a folder136coupled with an identifier of a file system130may correspond to an identifier that is unique across all of the file systems130. In another embodiment, the unique identifier160of a folder136may itself be unique across all of the file systems130. In some embodiments, the unique identifier160may also be unique with respect to the unique identifiers145of the files133.

The parent object reference163corresponds to a reference that may identify a folder136or file133that is considered to be the parent of the folder136. In this way, folders136are associated with other folders136. In some situations, the parent object reference163may include a special identifier (e.g., NULL,0, etc.) that identifies the root folder136of the file system130.

The file variants137are variants of files133or other file variants137. The file variants137may be data-reduced versions of the files133or other file variants137. As a non-limiting example, where the file133corresponds to an image, a file variant137associated with the file133may correspond to a reduced-size version of the image, such as a thumbnail or other image. As another non-limiting example, where the file133corresponds to an audio file133, a file variant137associated with the file133may correspond to a downsampled version of the audio file133, with a lower sample rate, lower bitrate, more aggressive compression, and/or other changed characteristics. In some cases, the file variants137may be customized or optimized for a particular type of client computing device, such as, for example, a mobile computing device like a smartphone. The file variants137may also be customized for delivery for particular types or categories of client109configurations.

Each file variant137may include, for example, a file name164, a unique identifier165, a data object reference166, a parent object reference167, a description of one or more client associations168, and/or other data. The file name164may correspond to a human-readable character string that identifies the contents of the file variant137. In some embodiments, the file name164may be limited in character length and may be limited in terms of allowed characters. In some cases, such limitations may be enforced for compatibility reasons. In addition, the file name164may be restricted to be unique for a folder136and/or for a file system130in various embodiments. In one embodiment, the file name164corresponds to the file name142of the corresponding file133associated with the file variant137, with or without a distinguishing identifier. The file name164may be generated automatically by the variant service121. The file name164may be hidden or absent in embodiments where the file variants137are ordinarily hidden from users.

The unique identifier165corresponds to a number, character string, or other identifier that uniquely identifies the file variant137across all of the file variants137of a particular file133or across a file system130and/or all of the file systems130. In one embodiment, a unique identifier165of a file variant137coupled with an identifier of a file system130may correspond to an identifier that is unique across all of the file systems130. In one embodiment, a unique identifier165of a file variant137coupled with a unique identifier145of a corresponding file133may correspond to an identifier that is unique across all of the file systems130. In another embodiment, the unique identifier165of a file variant137may itself be unique across all of the file systems130.

The data object reference166corresponds to a reference that may identify a data object associated with the file variant137that is stored in a data store such as the data store115or another data store. In one embodiment, the data object reference166may include a uniform resource identifier (URI). In another embodiment, the data object reference166may include a key value for obtaining the data object. The data store from which to obtain the data object may be determined implicitly or explicitly, for example, from a definition in the data object reference166. In some embodiments, the data objects corresponding to file variants137may be stored in a different data store from the data objects corresponding to the files133. As a non-limiting example, the data objects corresponding to file variants137may be stored in a data store that offers reduced data durability.

The parent object reference167corresponds to a reference that may identify a file133or a file variant137that is considered to be the parent of the file variant137. In this way, a file variant137is tied to a file133or another file variant137to which the file variant137is related. The client associations168may refer to types of configurations of clients109with which the file variant137is associated. As a non-limiting example, a file variant137might be customized for streaming to mobile devices, such as smartphones. Therefore, the client associations168may identify one or more smartphones associated with the user for which the file variant137is customized. In some cases, a file variant137, such as a thumbnail image, may be applicable to all clients109and configurations.

The client information138includes information relating to capabilities of clients, such as, for example, clients109, applications124, and/or other types of clients. The client information138may describe the capabilities of clients, such as whether they are able to decode an audio file encoded with a certain codec, the display screen size, network112bandwidth limitations, and so on. In one embodiment, the client information138may include associations of types of clients with types of file variants137.

The computing device106may comprise, for example, a server computer or any other system providing computing capability. Alternatively, a plurality of computing devices106may be employed that are arranged, for example, in one or more server banks or computer banks or other arrangements. For example, a plurality of computing devices106together may comprise a cloud computing resource, a grid computing resource, and/or any other distributed computing arrangement. Such computing devices106may be located in a single installation or may be distributed among many different geographical locations.

For purposes of convenience, the computing device106is referred to herein in the singular. Even though the computing device106is referred to in the singular, it is understood that a plurality of computing devices106may be employed in the various arrangements as described above. Further, though the computing device106is described separately from the computing device103, it is understood that the applications and data described in connection with the computing device106may be provided in the computing device103in some embodiments. In some cases, one or more of the computing devices106may be operated by an entity that is different from the entity that operates the computing devices103.

Various applications and/or other functionality may be executed in the computing device106according to various embodiments. Also, various data is stored in a data store178and a reduced durability data store179that are accessible to the computing device106. The data store178and the reduced durability data store179may each be representative of a plurality of data stores178and/or reduced durability data stores179as can be appreciated. In one embodiment, the data store178and/or reduced durability data store179may correspond to cloud storage resources where data storage is provided according to a utility computing model. In one embodiment, the data store178and/or reduced durability data store179may provide eventually consistent storage.

The reduced durability data store179is associated with a lower level of data durability than the data store178. That is to say, there may be a higher probability that data stored in the reduced durability data store179may be lost, unavailable, or otherwise inaccessible in comparison to the data stored in the data store178. The reduced durability data store179may, for example, store fewer redundant copies of data or have a lower level of hardware redundancy in comparison to the data store178. In one embodiment, the data store178may include data that can be processed to regenerate other data stored in the data store178, while the reduced durability data store179may exclude data that can be processed to regenerate other data stored in the reduced durability data store179. The reduced durability data store179may be less geographically distributed, have a less reliable power supply, have fewer redundant connections to the network112, and/or other resource constraints in comparison to the data store178.

As a non-limiting example, the data store178may be associated with a 99.9999% probability that no data item will be lost for a user within a year, while the reduced durability data store179may be associated with a 99.99% probability that no data item will be lost for a user within a year. Such percentages are given merely as examples to illustrate the reduced level of durability associated with the reduced durability data store179. It is noted that the reduced durability data store179may be absent in some embodiments, and the data may instead be stored in the data store178. The data stored in the data store178and the reduced durability data store179, for example, are associated with the operation of the various applications and/or functional entities described below.

The components executed on the computing device106, for example, include a data storage service181and other applications, services, processes, systems, engines, or functionality not discussed in detail herein. The data storage service181is executed to provide access to store and retrieve data from the data stores178. The data storage service181may store and retrieve data based upon a unique identifier such as a key, which may be a character string, number, or other identifier. The data storage service181may function to abstract the underlying storage layer so that users of the data storage service181have no knowledge as to how their data is actually being stored. For example, users may not know on what disk drives, in what computing device106, in what data center, etc. that their data is being stored. In various embodiments, a user may be able to specify performance characteristics or types of hardware to be used in storing the data. For example, a user may be able to request that a data object be stored in the reduced durability data store179instead of the data store178.

In embodiments that include the reduced durability data store179, the data storage service181may be configured to direct storage of data to either the data store178or the reduced durability data store179based on whether the data is a file variant137or otherwise derived from other data that is stored in the data store178. Thus, if data is lost or otherwise unavailable from the reduced durability data store179, the data may be regenerated from data stored in the data store178. In some embodiments, the file service118, the variant service121, and/or another application may direct the storage of data to either the data store178or the reduced durability data store179.

The data stored in the data store178includes, for example, data objects184and potentially other data. The data objects184correspond to the data stored for the files133. Such data may be any type of data object, such as, for example, text data, binary data, multimedia data, and so on. In some embodiments, the data objects184may also correspond to the data stored for the file variants137. The data stored in the reduced durability data store179includes, for example, data objects185and potentially other data. The data objects185correspond to the data stored for the file variants137. Such data may be any type of data object, such as, for example, text data, binary data, multimedia data, and so on.

It is noted that in some configurations, the data objects184and185may merely be references to another data object184and185. For example, multiple users may have copies of a music file in their file system130. Because the copies are identical, the data objects184and185may merely be addresses or pointers to the shared data. Such associations may be updated with a copy-on-write system when the shared data is modified. In some cases, the shared data may be stored externally in a content delivery network.

The client109is representative of a plurality of client devices that may be coupled to the network112. The client109may comprise, for example, a processor-based system such as a computer system. Such a computer system may be embodied in the form of a desktop computer, a laptop computer, mobile devices, personal digital assistants, cellular telephones, smartphones, set-top boxes, music players, web pads, tablet computer systems, game consoles, electronic book readers, or other devices with like capability. The client109may include, for example, one or more display devices such as cathode ray tubes (CRTs), liquid crystal display (LCD) screens, gas plasma-based flat panel displays, LCD projectors, or other types of display devices, etc.

The client109may be configured to execute various applications such as a client application187and/or other applications. The client application187is executed to store, retrieve, process, and otherwise use files133and folders136in one or more file systems130in the computing device103. In one embodiment, the client application187corresponds to a browser application that renders network pages provided by the computing device103that facilitate interaction with the file service118, the variant service121, and/or the applications124. In one embodiment, the client application187is integrated with an operating system of the client109to provide access to a file system130similarly to any mounted file system of the client109. The client109may be configured to execute applications beyond client application187such as, for example, browser applications, email applications, instant message applications, and/or other applications.

Next, a general description of the operation of the various components of the networked environment100is provided. To begin, a user may establish an account through the client application187or another application124to create or access one or more file systems130. Where the user is described as performing an action, it is understood that the user may be interacting with at least the client application187to perform the action.

The user may specify various parameters, such as, for example, a maximum storage requirement, performance requirements, a rate plan, access permissions, security credentials, and so on. The user may provide billing information and/or existing billing information may be correlated with the user. Accordingly, the account data131is populated and a file system130may be created. In some embodiments, the user may also specify various types of clients109which will be accessing the file system130. This information may be stored, for example, in the client associations132.

The user may transfer files to the file system130by way of the applications124or the client applications187. Such a transfer may involve the user uploading a file from the client109to the computing device103or the computing device106. In other cases, a user may purchase a file (e.g., a music file) from an online retailer associated with the computing device103, and the file may be automatically saved to the file system130by way of an application124.

When a file is transferred to the file system130, a file133is created in the file system130by the file service118. The file name142may correspond to the original file name associated with the source file, or the file name142may be explicitly specified by the user or an application124. The unique identifier145is generated for the file133. A data object184corresponding to the data associated with the source file is stored in the data store178by the file service118interacting with the data storage service181. A data object reference148corresponding to the stored data object184is then recorded for the file133. A parent object reference151may optionally be specified for the file133. If no parent object reference151is specified, the file133may be considered to be in the root folder136for the file system130.

Folders136may also be created or imported by users and/or applications124. The folder name157may correspond to the original name associated with a source folder, or the folder name157may be explicitly specified by the user or the application124. The unique identifier160is generated for the folder136. A parent object reference163may optionally be specified for the folder136. If no parent object reference163is specified, the folder136may be considered to be in the root folder136for the file system130.

It is noted that the various data associated with the files133, the folders136, or the file systems130need not be recorded in inodes, vnodes, file allocation tables, or any other data structure associated with physical file systems. In various embodiments, the data associated with the files133, the folders136, and the file systems130may be maintained as data within a relational database management system (RDBMS) or similar database management systems.

The file service118may support various operations related to files133and folders136such as, for example, renaming, copying, moving, deleting, recycling, uploading, downloading, and so on. When a file133or folder136is renamed, the file name142or folder name157is updated, but no change occurs to the data object184. When a file133or folder136is copied, a duplicate file133or folder136(and contents) may be created to point to the same data object(s)184. The data object184itself may be duplicated when necessary according to copy-on-write or another methodology.

When a file133or folder136is moved, the parent object reference151or163may be updated, without any changes to the underlying data object(s)184. Similarly, when a change is made concerning where the underlying data objects184are stored, the file133or folder136may remain unchanged. When a file133or folder136is deleted, data object(s)184may be removed if no other files133point to the data object(s)184. In some cases, uploading and downloading functionality may involve the client application187interfacing directly with the data storage service181. The implementations of the operations described above limit modifications to the data objects184in the data store(s)178, which may be expensive in terms of time or cost.

When the files133are created in the file system130, when the files133are accessed, and/or at another time, file variants137may be created for various files133. The file variants137correspond to variants of the files133that may be data reduced and/or customized for delivery to certain types of clients109or applications124. In some cases, the file variants137may be said to be derivatives of the files133, at least in part. In other cases, the file variants137may merely be related to the files133. Further, some file variants137may be applicable to all clients109and applications124.

In some cases, the variant service121may be responsible for maintaining the file variants137for the files133. Where the file variants137are customized for a particular characteristic of a client109or application124, they may be data reduced to facilitate mobile streaming, they may be downsampled or resized for display on a client109, they may be pre-processed for an application platform associated with the client109, or otherwise encoded. Various variant encoders122may be employed to perform the encoding.

As a non-limiting example, a user may have a library of audio files133in a file system130. Such audio files133may initially be placed in the file system130in a high-bitrate format, such as Moving Pictures Experts Group (MPEG) Layer3(MP3) at a 256 kilobits per second bitrate. Such a format may be considered lossless or near lossless. However, the user may wish to listen to the audio file133through a mobile computing device, with a connection to the network112that has relatively low bandwidth and may be susceptible to periods of connectivity loss.

Accordingly, the variant service121may generate a file variant137for the file133that corresponds to a version of the file133that is customized for delivery to the mobile computing device. For example, a bitrate of 64 kilobits per second may be considered optimal for the type of mobile computing device. The variant encoder122may correspond to an MP3 audio codec and may be configured to encode the high-bitrate audio file133into a low-bitrate file variant137.

When the user accesses the audio file133from the mobile computing device, the appropriate file variant137may be automatically substituted. Such a substitution may be transparent or non-transparent. For example, a user may wish to download the original high-bitrate audio file133as a progressive download rather than merely a stream. The variant service121and/or the file service118may permit the user an option to download the original audio file133. Such an option may be selected automatically according to the context of the request for the file133.

As another non-limiting example, a user may wish to download a document on a tablet computing device. The document may have originally been created for display on a large widescreen display monitor. The variant service121may be configured to detect that the user is requesting the file133from the tablet computing device and to encode a file variant137appropriately. The file variant137may be a resized version of the document that is customized for the display of the tablet computing device.

As yet another non-limiting example, a user may upload multiple high-resolution image files133to the file system130. The variant service121may create file variants137corresponding to lower resolution, reduced size thumbnail images of the image files133automatically, regardless of the types of clients109associated with the file system130. The thumbnail images that are file variants137may be automatically detected by an application124that is a photo album and used to render album pages containing thumbnails, with the option to access the original high-resolution image files133.

As yet another non-limiting example, a user may upload a music file133to the file system130. Using external data, the variant service121may identify the music file133and create a file variant137that corresponds to album cover art that is related to the music file133. One or more images corresponding to the album cover art may be stored in a data store178or179and linked to one or more file variants137of the music file133. In one example, an initial file variant137may be a high resolution version of the cover art, while a variant of the file variant137may be a thumbnail version of the cover art.

In some embodiments, the data underlying the file variants137may be stored in a reduced durability data store179, which has a higher likelihood of the data being lost or inaccessible. The reduced durability may not be a problem for the file variants137because they may be capable of regeneration from data objects184that are stored in a data store178having a higher level of durability. Reduced durability data storage may have a lower cost and/or other advantages over high durability data storage.

Further, the variant service121may be configured to periodically delete file variants137or their underlying data when they have not been used within a predetermined length of time or are not predicted to be used. For example, several file variants137may be generated for a tablet computing device, but the user may have sold the tablet computing device and will never access the file variants137again. Thus, the applicable file variants137may be safely deleted. Even if the user were to access the files133from another tablet computing device, new file variants137may be regenerated from the original files133on demand or when otherwise appropriate.

The variant service121may be configured to identify file variants137that are customized for capabilities of a client109or application124according to the client information138. Upon knowing what type of client109or application124is requesting a file133, the variant service121may refer to the capabilities of the client109or application124in the client information138. The variant service121may determine whether the capabilities match the characteristics of one or more file variants137associated with a requested file133. In one embodiment, the file variant137that is the best match may be returned. In another embodiment, all of the file variants137for the file133that are compatible may be returned.

In some embodiments, the variant service121may be exposed to a user at a client109to allow the user to obtain a listing of file variants137for a particular file133and request a file variant137from the listing. Such file variants137may or may not exist. If the requested file variant137does not exist, it may be generated on demand by the variant service121. In one example, the user may provide a description of the client109to the variant service121, which may then determine the file variants137that pertain to the particular client109.

Referring next toFIG. 2, shown is a flowchart that provides one example of the operation of a portion of the file service118according to various embodiments. In particular,FIG. 2relates to storing a file133(FIG. 1) in a file system130(FIG. 1) and creating file variants137(FIG. 1) if desired. It is understood that the flowchart ofFIG. 2provides merely an example of the many different types of functional arrangements that may be employed to implement the operation of the portion of the file service118as described herein. As an alternative, the flowchart ofFIG. 2may be viewed as depicting an example of steps of a method implemented in the computing device103(FIG. 1) according to one or more embodiments. Although the tasks ofFIG. 2are described as being performed by the file service118, one or more of the tasks ofFIG. 2may be performed by the variant service121(FIG. 1), the data storage service181(FIG. 1), and/or other applications in other embodiments.

Beginning with box203, the file service118obtains a request to store a file133in a file system130. Such a request may originate with a client application187(FIG. 1) or an application124(FIG. 1). In box206, the file service118obtains a data object184(FIG. 1) that corresponds to the file133from the client application187or the application124. In box209, the file service118stores the data object184in the data store178(FIG. 1).

In box212, the file service118creates the file133in the file system130. To this end, the file service118may set a file name142(FIG. 1) according to the request, generate a unique identifier145(FIG. 1) for the file133, set a reference to the data object184stored in the data store178as a data object reference148(FIG. 1), set a unique identifier160(FIG. 1) of a parent folder136(FIG. 1) as a parent object reference163(FIG. 1), and set permissions and/or other metadata for the file133.

In box215, the file service118determines clients109that are associated with the file system130and/or the file133. As a non-limiting example, a user associated with the file system130may have used previously, or explicitly configured, clients109such as an electronic book reader, a laptop computer, and a smartphone. Further, the user may have used certain client applications187, such as browser applications, music players, document processing applications, etc. that may impact what versions of files133are used.

In box218, the file service118determines whether one or more file variants137are to be generated for the file133. In some cases, such a determination may depend on the clients109associated with the file system130and/or the file133. In other cases, such a determination may be independent of the clients109. If no file variant137is to be generated for the file133, the portion of the file service118ends. Otherwise, the file service118proceeds to box221.

In box221, the file service118encodes the data object184using one or more appropriate variant encoders122(FIG. 1) to obtain one or more encoded data objects. The encoded data objects are stored in a data store in box224. In one embodiment, the encoded data objects are stored as data objects185(FIG. 1) in the reduced durability data store179(FIG. 1). In another embodiment, the encoded data objects are stored as data objects184in the data store178.

In box227, the file service118creates one or more file variants137referring to the encoded data object(s) in the file system130. To this end, the file service118may generate respective file name(s)164(FIG. 1) and unique identifiers165(FIG. 1) for the file variants137. The file service118may set the data object reference166(FIG. 1) for the file variants137to point to an identifier of the respective encoded data object stored in the data store178or the reduced durability data store179. The file service118may set the parent object reference167(FIG. 1) for each of the file variants137to be the unique identifier145of the file133. Thereafter, the portion of the file service118ends.

Moving on toFIG. 3, shown is a flowchart that provides one example of the operation of another portion of the file service118according to various embodiments. In particular,FIG. 3relates to accessing a file133(FIG. 1) in a file system130(FIG. 1). It is understood that the flowchart ofFIG. 3provides merely an example of the many different types of functional arrangements that may be employed to implement the operation of the portion of the file service118as described herein. As an alternative, the flowchart ofFIG. 3may be viewed as depicting an example of steps of a method implemented in the computing device103(FIG. 1) according to one or more embodiments. Although the tasks ofFIG. 3are described as being performed by the file service118, one or more of the tasks ofFIG. 3may be performed by the variant service121(FIG. 1), the data storage service181(FIG. 1), and/or other applications in other embodiments.

Beginning with box303, the file service118obtains a request from a client109(FIG. 1) to access a file133. The request may be obtained directly from a client application187(FIG. 1) or through an application124(FIG. 1). In box306, the file service118determines whether a file variant137(FIG. 1) should be provided instead of the file133according to the type of client109and/or the type of file133. If no file variant137is to be provided, the file service118continues to box309and obtains a data object184(FIG. 1) corresponding to the requested file133from the data store178(FIG. 1) and sends the data object184to the client109. It is noted that the data store178may correspond to a high durability data store178in some embodiments. Thereafter, the portion of the file service118ends.

Otherwise, if a file variant137is to be provided, the file service118moves from box306to box312. In box312, the file service118determines whether the file variant137exists in the file system130. More specifically, the file service118may determine whether a corresponding encoded data object185(FIG. 1) exists in some embodiments. If the appropriate file variant137for the client109exists, the file service118obtains the data object185(FIG. 1) from the reduced durability data store179(FIG. 1) and sends the data object185to the client109. In some embodiments, the file variant137may instead point to a data object184stored in the data store178. In such cases, the corresponding data object184is obtained and sent to the client109. Thereafter, the portion of the file service118ends.

Otherwise, if the file variant137does not exist, the file service118moves from box312to box318. In box318, the file service118generates an encoded data object corresponding to the file variant137from the data object184corresponding to the file133. Various variant encoders122(FIG. 1) may be employed, and a variant service121may coordinate this task in some embodiments. In box321, the file service118stores the encoded data object corresponding to the file variant137as a data object185in the reduced durability data store179. In other embodiments, the file service118may instead store the encoded data object corresponding to the file variant137as a data object184in the data store178.

In box324, the file service118creates the file variant137in the file system130. To this end, the file service118may generate respective file name(s)164(FIG. 1) and unique identifiers165(FIG. 1) for the file variants137. The file service118may set the data object reference166(FIG. 1) for the file variants137to point to an identifier of the respective encoded data object stored in the data store178or the reduced durability data store179. The file service118may set the parent object reference167(FIG. 1) for each of the file variants137to be the unique identifier145(FIG. 1) of the file133. In box327, the file service118sends the encoded data object that corresponds to the file variant137to the client109. In one embodiment, the encoded data object may be sent to the client109while it is being generated and stored. Such an optimization may reduce the delay associated with encoding the data object and storing it before sending. Thereafter, the portion of the file service118ends.

Turning now toFIG. 4, shown is a flowchart that provides one example of the operation of yet another portion of the file service118according to various embodiments. In particular,FIG. 4relates to deleting a file133(FIG. 1) in a file system130(FIG. 1). It is understood that the flowchart ofFIG. 4provides merely an example of the many different types of functional arrangements that may be employed to implement the operation of the portion of the file service118as described herein. As an alternative, the flowchart ofFIG. 4may be viewed as depicting an example of steps of a method implemented in the computing device103(FIG. 1) according to one or more embodiments. Although the tasks ofFIG. 4are described as being performed by the file service118, one or more of the tasks ofFIG. 4may be performed by the variant service121(FIG. 1), the data storage service181(FIG. 1), and/or other applications in other embodiments.

Beginning with box403, the file service118obtains a request to delete a file133from a client109. In particular, the request may be obtained from a client application187(FIG. 1) or from an application124(FIG. 1). In box406, the file service118initiates deletion of the data object184(FIG. 1) that corresponds to the file133as identified by the data object reference148(FIG. 1). In one embodiment, a delay or wait time may be employed to avoid a race condition when deleting the data object184.

In box409, the file service118determines whether any file variants137(FIG. 1) are associated with the file133. In other words, the file service118determines whether any file variants137have a parent object reference167(FIG. 1) set to be the unique identifier145(FIG. 1) of the file133. If no file variants137are associated with the file133, the file service118proceeds to box412and deletes the file133from the file system130. Thereafter, the portion of the file service118ends. However, if file variants137are associated with the file133, the file service118instead continues to box415.

In box415, the file service118initiates deletion of the data objects185(FIG. 1) and/or data objects184that correspond to the file variants137. This may include variants of file variants137. In one embodiment, a delay or wait time may be employed to avoid a race condition when deleting the data objects184,185. In box418, the file service118deletes the file variants137(and any variants thereof) from the file system130. In box412, the file service118deletes the file133from the file system130. Thereafter, the portion of the file service118ends.

Referring next toFIG. 5, shown is a flowchart that provides one example of the operation of a portion of the data storage service181according to various embodiments. It is understood that the flowchart ofFIG. 5provides merely an example of the many different types of functional arrangements that may be employed to implement the operation of the portion of the data storage service181as described herein. As an alternative, the flowchart ofFIG. 5may be viewed as depicting an example of steps of a method implemented in the computing device106(FIG. 1) according to one or more embodiments. Although the tasks ofFIG. 5are described as being performed by the data storage service181, one or more of the tasks ofFIG. 5may be performed by the variant service121(FIG. 1), the file service118(FIG. 1), and/or other applications in other embodiments.

Beginning with box503, the data storage service181obtains a request to store a data object. Such a request may be obtained, for example, from the file service118, the variant service121, or another application. In box506, the data storage service181determines whether the data object corresponds to a file variant137(FIG. 1). If the data object corresponds to a file variant137, the data storage service181proceeds to box509and stores the data object as a data object185(FIG. 1) in the reduced durability data store179(FIG. 1). In one embodiment, the data storage service181may also return an identifier of the data object185to the requestor. Thereafter, the portion of the data storage service181ends.

Otherwise, if the data storage service181determines that the data object does not correspond to a file variant137, the data storage service181continues to box512. In box512, the data storage service181stores the data object as a data object184(FIG. 1) in the data store178(FIG. 1), which is associated with a higher level of data durability than the reduced durability data store179. In one embodiment, the data storage service181may also return an identifier of the data object184to the requestor. Thereafter, the portion of the data storage service181ends.

Continuing on toFIG. 6, shown is a flowchart that provides one example of the operation of a portion of the variant service121according to various embodiments. It is understood that the flowchart ofFIG. 6provides merely an example of the many different types of functional arrangements that may be employed to implement the operation of the portion of the variant service121as described herein. As an alternative, the flowchart ofFIG. 6may be viewed as depicting an example of steps of a method implemented in the computing device103(FIG. 1) according to one or more embodiments. Although the tasks ofFIG. 6are described as being performed by the data storage service181, one or more of the tasks ofFIG. 6may be performed by the file service118(FIG. 1), and/or other applications in other embodiments.

Beginning with box603, the variant service121determines clients109(FIG. 1) that are associated with the file system130(FIG. 1) of a user. Although clients109are referenced in connection withFIG. 6, it is understood that the principles ofFIG. 6may be applied to applications124(FIG. 1) with defined capabilities and preferences as well. As a non-limiting example, a user associated with the file system130may have used previously, or explicitly configured, clients109such as an electronic book reader, a laptop computer, and a smartphone. Further, the user may have used certain client applications187(FIG. 1), such as browser applications, music players, document processing applications, etc. that may impact what versions of files133(FIG. 1) are used. In box606, the variant service121determines the file variants137(FIG. 1) that are associated with the file system130of the user.

In box609, for one of the file variants137, the variant service121determines whether the file variant137is associated with a client109that is no longer associated with the user. If the file variant137is associated with a client109that is no longer associated with the user (or the file system130), the variant service121moves to box612and deletes the data object185(FIG. 1) corresponding to the file variant137from the reduced durability data store179(FIG. 1). The file variant137may subsequently be deleted from the file system130. In various embodiments, some time may elapse between when a client109is disassociated and when the deletion is initiated. Also, the variant service121may determine that the file variant137may be usable for another client109, in which case the variant service121may decide not to delete the file variant137. The variant service121continues to box615.

If the variant service121determines, in box609, that the file variant137is not for a client109that is no longer associated with the user, the variant service121proceeds to box618. In box618, the variant service121determines whether the file variant137has been accessed within a predetermined length of time or otherwise meets retention criteria. If the variant service121does not meet retention criteria, the variant service121continues to box612and deletes the data object185corresponding to the file variant137from the reduced durability data store179. The file variant137may subsequently be deleted from the file system130. The variant service121continues to box615.

In box615, the variant service121determines whether another file variant137remains to be processed. If so, the variant service121returns to box609and begins evaluating the other file variant137. If no other file variant137remains to be processed, the portion of the variant service121ends.

With reference toFIG. 7, shown is a schematic block diagram of the computing device103according to an embodiment of the present disclosure. The computing device103includes at least one processor circuit, for example, having a processor703and a memory706, both of which are coupled to a local interface709. To this end, the computing device103may comprise, for example, at least one server computer or like device. The local interface709may comprise, for example, a data bus with an accompanying address/control bus or other bus structure as can be appreciated. Although the computing device103is depicted inFIG. 7, it is understood that the computing device106may be of similar construction.

Stored in the memory706are both data and several components that are executable by the processor703. In particular, stored in the memory706and executable by the processor703are the file service118, the variant service121, the variant encoders122, the applications124, and potentially other applications. Also stored in the memory706may be a data store115and other data. In addition, an operating system may be stored in the memory706and executable by the processor703.

Also, the processor703may represent multiple processors703and the memory706may represent multiple memories706that operate in parallel processing circuits, respectively. In such a case, the local interface709may be an appropriate network112(FIG. 1) that facilitates communication between any two of the multiple processors703, between any processor703and any of the memories706, or between any two of the memories706, etc. The local interface709may comprise additional systems designed to coordinate this communication, including, for example, performing load balancing. The processor703may be of electrical or of some other available construction.