Source: https://patents.google.com/patent/US9251201B2/en
Timestamp: 2019-03-19 20:22:23
Document Index: 377421342

Matched Legal Cases: ['Application No. 2011305839', 'Application No. 2011305839', 'Application No. 201110285468', 'Application No. 201110343154', 'Application No. 201110285468', 'Application No. 201110343154', 'Application No. 201110343154', 'Application No. 12821531', 'Application No. 2013']

US9251201B2 - Compatibly extending offload token size - Google Patents
Compatibly extending offload token size Download PDF
US9251201B2
US9251201B2 US13/714,413 US201213714413A US9251201B2 US 9251201 B2 US9251201 B2 US 9251201B2 US 201213714413 A US201213714413 A US 201213714413A US 9251201 B2 US9251201 B2 US 9251201B2
US13/714,413
US20140172811A1 (en
2012-12-14 Assigned to MICROSOFT CORPORATION reassignment MICROSOFT CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GREEN, DUSTIN L.
2012-12-14 Priority to US13/714,413 priority Critical patent/US9251201B2/en
2014-06-19 Publication of US20140172811A1 publication Critical patent/US20140172811A1/en
2016-02-02 Publication of US9251201B2 publication Critical patent/US9251201B2/en
Aspects of the subject matter described herein relate to offload technology. In aspects, a mechanism is described that allows an offload provider to use larger tokens. The larger token may be physical or virtual. In response to an offload read command, a larger token may be created and data from the larger token may be split or injected into multiple tokens of a smaller size. In response to an offload write command, data from the multiple tokens may be combined into a larger token and/or extracted and used to obtain bulk data.
Briefly, aspects of the subject matter described herein relate to offload technology. In aspects, a mechanism is described that allows an offload provider to use larger tokens. The larger token may be physical or virtual. In response to an offload read command, a larger token may be created and data from the larger token may be split or injected into multiple tokens of a smaller size. In response to an offload write command, data from the multiple tokens may be combined into a larger token and/or extracted and used to obtain bulk data.
FIGS. 2-4 are block diagrams that represent exemplary arrangements of components of systems in which aspects of the subject matter described herein may operate;
FIG. 5 is a diagram that illustrates one exemplary scheme for representing one larger token with one or more smaller subtokens in accordance with aspects of the subject matter described herein;
FIG. 6 is a block diagram that represents an exemplary arrangement of components of a system in which aspects of the subject matter described herein may operate; and
FIGS. 7-9 are flow diagrams that generally represent exemplary actions that may occur in accordance with aspects of the subject matter described herein
FIGS. 2-4 and 6 are block diagrams that represent exemplary arrangements of components of systems in which aspects of the subject matter described herein may operate. The components illustrated in FIGS. 2-4 and 6 are exemplary and are not meant to be all-inclusive of components that may be needed or included. In other embodiments, the components and/or functions described in conjunction with FIGS. 2-4 and 6 may be included in other components (shown or not shown) or placed in subcomponents without departing from the spirit or scope of aspects of the subject matter described herein. In some embodiments, the components and/or functions described in conjunction with FIGS. 2-4 and 6 may be distributed across multiple devices.
Turning to FIG. 2, the system 205 may include an initiator 210, data access components 215, token provider(s) 225, a store 220, and other components (not shown). The system 205 may be implemented via one or more computing devices. Such devices may include, for example, personal computers, server computers, hand-held or laptop devices, multiprocessor systems, microcontroller-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, cell phones, personal digital assistants (PDAs), gaming devices, printers, appliances including set-top, media center, or other appliances, automobile-embedded or attached computing devices, other mobile devices, distributed computing environments that include any of the above systems or devices, and the like.
In one embodiment, a data access component may comprise any component that is given an opportunity to examine I/O between the initiator 210 and the store 220 and that is capable of changing, completing, or failing the I/O or performing other or no actions based thereon. For example, where the system 205 resides on a single device, the data access components 215 may include any object in an I/O stack between the initiator 210 and the store 220. Where the system 205 is implemented by multiple devices, the data access components 215 may include components on a device that hosts the initiator 210, components on a device that provides access to the store 220, and/or components on other devices and the like. In another embodiment, the data access components 215 may include any components (e.g., such as a service, database, or the like) used by a component through which the I/O passes even if the data does not flow through the used components.
As used herein, the term computer code is to be read to include instructions that dictate actions a computer is to take. These instructions may be included in any computer-readable media volatile or nonvolatile.
One or more of the data access components 215 may reside on an apparatus that hosts the initiator 210 while one or more other of the data access components 215 may reside on an apparatus that hosts or provides access to the store 220. For example, if the initiator 210 is an application that executes on a personal computer, one or more of the data access components 215 may reside in an operating system hosted on the personal computer. An example of this is illustrated in FIG. 3.
As another example, if the store 220 is implemented by a storage area network (SAN), one or more of the data access components 215 may implement a storage operating system that manages and/or provides access to the store 220. When the initiator 210 and the store 220 are hosted in a single apparatus, all or many of the data access components 215 may also reside on the apparatus.
An offload read allows an initiator to obtain a token that represents data of a store. Using this token, the initiator or another initiator may request an offload write. An offload write allows an initiator to cause an offload provider to write some or all of the data represented by the token.
In one embodiment, a token includes a cryptographically secure number that is obtained via a successful offload read. At the present time, one example of a cryptographically secure number is a 256 bit number generated in an appropriate way (e.g., via creating a random number by sampling some random physical phenomena). Some exemplary procedures for generating cryptographically secure numbers are described in Request for Comments (RFC) 1750. With advances in technology both the length of the secure number and the procedure used to generate a cryptographically secure number may change without departing from the spirit or scope of aspects of the subject matter described herein.
A token represents data that is immutable as long as the token is valid. The data a token represents is sometimes referred to as bulk data.
In some implementations, an offload provider may initiate sharing of data storage locations among all tokens and storage containers already sharing the data, and in addition, another storage container or token. For example, to service an offload read, an offload provider may initiate sharing between a source storage container and a token. Then, to service an offload write using the token, the offload provider may initiate sharing among the source storage container, the token, and the destination storage container. If the token is later invalidated, sharing with the token is stopped, but the sharing between source and destination storage containers may continue (e.g., until a write is received that is directed at that data).
As used herein, in one implementation, a token provider is part of an offload provider. In this implementation, where a token provider is described as performing actions, it is to be understood that the offload provider that includes the token provider is performing those actions. In another implementation, a token provider may be separate from the offload provider.
To initiate an offload read of data of the store 220, the initiator 210 may send a request to obtain a token representing the data using a predefined command (e.g., via an API). In response, one or more of the data access components 215 may respond to the initiator 210 by providing one or more tokens that represents the data or a subset thereof. A token may be represented by a sequence of bytes which are used to represent immutable data. The size of the immutable data may be larger, smaller, or the same size as the token.
With a token, the initiator 210 may request that all or portions of the data represented by the token be logically written. Sometimes herein this operation is called an offload write. The initiator 210 may do this by sending the token together with one or more offsets and lengths to the data access components 215.
One or more layers of the stack may be associated with a token provider. A token provider may include one or more components that may generate or obtain tokens that represent portions of the data of the store 220 and provide these tokens to an initiator.
If a data access component 215 fails an offload read or write, an error code may be returned that allows another data access component or the initiator to attempt another mechanism for reading or writing the data.
FIG. 3 is a block diagram that generally represents an exemplary arrangement of components of systems in which a token provider is hosted by the device that hosts the store. As illustrated, the system 305 includes the initiator 210 and the store 220 of FIG. 2. The data access components 215 of FIG. 3 are divided between the data access components 310 that reside on the device 330 that hosts the initiator 210 and the data access components 315 that reside on the device 335 that hosts the store 220. In another embodiment, where the store 220 is external to the device 335, there may be additional data access components that provide access to the store 220.
The device 335 may be considered to be one example of an offload provider as this device includes components for performing offload reads and writes and managing tokens.
The token provider 320 may generate, validate, and invalidate tokens. For example, when the initiator 210 asks for a token for data on the store 220, the token provider 320 may generate a token that represents the data. This token may then be sent back to the initiator 210 via the data access components 310 and 315.
When the initiator 210 or any other entity provides the token to the token provider 320, the token provider 320 may perform a lookup in the token store 325 to determine whether the token exists. If the token exists and is valid, the token provider 320 may provide location information to the data access components 315 so that these components may logically read or write or logically perform other operations with the data as requested.
In another exemplary arrangement similar to FIG. 3, the token provider 320 and token store 325 may be included in the device 330, and the data access components 310 connected to token provider 320. For example, an operating system (OS) of the device 330 may include the token provider 320 and the token store 325. In this example, the initiator 210 may assume the existence of a token provider and token store for all copying performed by the initiator 210. With this assumption, the initiator 210 may be implemented to omit code that falls back to normal read and write.
In the example above, the OS may implement offload read by logically reading the requested data from the data access components 315 and storing the data in storage (volatile or non-volatile) of device 330, creating a new token value, and associating the newly created token value with the read data. The OS may implement offload write by copying (e.g., logically writing) the data associated with the token to the destination specified by initiator 210. In this example, the initiator 210 may need to re-attempt a copy at the offload read step in some scenarios, but this re-attempt may be less burdensome for the initiator than falling back to normal read and write.
FIG. 4 is a block diagram that generally represents another exemplary environment in which aspects of the subject matter described herein may be implemented. As illustrated, the environment includes a source initiator 405, a destination initiator 406, a source storage container 410, a destination storage container 411, a source physical store 415, a destination physical store 416, an offload provider 420, and may include other components (not shown).
The source initiator 405 and the destination initiator may be implemented similarly to the initiator 210 of FIG. 2. The source initiator 405 and the destination initiator 406 may be two separate entities or a single entity.
Although illustrated as only having one storage container between the initiator and the physical store, as mentioned previously, in other embodiments there may be multiple layers of storage containers between the initiator and the physical store.
The source initiator 405 may obtain a token by issuing an offload read. In response, the offload provider 420 may generate a token and provide it to the source initiator 405.
If the source initiator 405 and the destination initiator 406 are separate entities, the source initiator 405 may provide the token to the destination initiator 406. The destination initiator 406 may then use the token to issue an offload write to the destination storage container 411.
Extending Token Size
With offload technology, a standard or industry may dictate a certain fixed size of the token. For various reasons, some implementers may desire a size that is larger than the standardized fixed size.
To accommodate larger sized tokens, the standard may be modified to allow multiple tokens. A token larger than the fixed size may then be represented by multiple smaller subtokens of the fixed size. For example, one standard requires a token to be 512 bytes. In an implementation for this standard, the subtokens may each be exactly 512 bytes while the larger token may be larger than 512 bytes (e.g. 995, 2000, 4096, or some other number of bytes).
FIG. 5 is a diagram that illustrates one exemplary scheme for representing one larger token with one or more smaller subtokens in accordance with aspects of the subject matter described herein. As illustrated, an exemplary large token 505 may have standard required fields H, a provider ID P, random data R, vendor data V, and other data X.
The standard required field H may include any fields required or otherwise specified by a standard. For example, the standard required fields H may include one or more of: data that indicates when a token was generated, data that indicates when the token is supposed to expire, data that indicates where a token came from, or other data specified by a standard.
The provider ID P may indicate an instance of an offload provider that generated the token. The provider ID P may be used in a threshold test to determine whether the token is to be ignored or not. If the provider ID P is not a provider ID that would have been provided by the offload provider, the offload provider may reject the token altogether. Otherwise, the offload provider may take additional actions to validate the token.
The vendor data V may include any data that a vendor implementing an offload provider may desire. As one example, a vendor may include addressing information that indicates an address of an offload provider that provided the token 505. As other examples, the vendor data V may include a hash key, a digest, a lookup key, metadata, data related to the bulk data, data the helps identify or locate portions of the bulk data, other data, or the like.
The other data X may include any other data that is included in the token 505.
The subtokens may be transmitted via virtually any protocol. For example, in one example, the subtokens may be transmitted via a Small Computer System Interface (SCSI) protocol. In another example, the subtokens may be transmitted via a file sharing protocol that transfers file data via server message blocks. One exemplary file sharing protocol includes the Server Message Block (SMB) protocol. In another example, the subtokens may be transmitted via a distributed file system protocol that is based on remote procedure calls to access files. One example protocol based on remote procedure calls includes the Network File System (NFS) protocol.
The examples above are not intended to be all-inclusive or exhaustive of protocols that may be used. Indeed, based on the teachings herein, those skilled in the art may recognize many other protocols that may be used without departing from the spirit or scope of aspects of the subject matter described herein.
The subtokens 510-515 may be tokens of a fixed size (e.g., dictated by standard) that represent the token 505. The subtokens 510-515 may include various fields. For example, a subtoken may include fields (H0, HN1, H . . . ) required by a standard, a provider ID field (P), a token ID (T), sequence data (S0, SN1, S . . . ), a number that indicates how many subtokens represent the token 505, and data corresponding to the data of the token 505. This other data is represented by H, R0 to RN1, V0 to VN2, and X0 to XN3, where H corresponds to the standard required fields H in the token 505, R0 to RN1 correspond to the random data R in the token 505, V0 to VN2 correspond to the vendor data V in the token 505, and X0 to XN3 correspond to the other data X in the token 505.
The fields (H0, HN1, H . . . ) may include any data required or otherwise specified by a standard. This may include, for example, header or other fields specified by any version of the SCSI protocol. The fields (H0, HN1, H . . . ) may occur prior to and/or after any other fields indicated in FIG. 5.
Where the SCSI protocol is used, the fields (H0, HN1, H . . . ) may include, for example, header or other fields specified by any version of the SCSI protocol. Some exemplary fields include: timestamp of token creation, token type (e.g., point in time copy), address of source, data identifying a representation of data token type, data that identifies each of the subtokens as a token for transferring bulk data without requiring the bulk data to pass through an initiator of a command that requested the transferring, other fields specified by the SCSI protocol, or the like.
Where other protocols are used, the fields (H0, HN1, H . . . ) may include, for example, fields required or allowed by those protocols. In one implementation, the fields (H0, HN1, H . . . ) may be omitted altogether.
In some implementations, the fields (H0, HN1, H . . . ) may also include, for example, types of data indicated above with respect to the standard required fields H of the token 505.
The provider ID P may indicate an instance of an offload provider that generated the token and may be used in the same manner as indicated above.
The token ID T may be data that identifies a subtoken as belonging to a group of subtokens that represent a larger token. For example, a token ID of “ABCD” in each of the T fields of the subtokens 510-515 may identify the subtokens 510-515 as belonging to a group of subtokens that represent the token 505. If a subtoken has a different token ID, the offload provider may determine that the subtoken is not part of the group of subtokens that represent the token 505.
The sequence data (S0, SN1, S . . . ) may include data that indicates an ordering of the subtokens. For example, the sequence data may include an increasing number (e.g., 1, 2, 3, 4, etc.) that indicates an order of the subtokens. The order may be used to combine the subtokens 510-515 to reconstruct the token 505 or portions thereof.
In one implementation, data from the fields of the subtokens 510-515 may be combined to construct all the data included in the token 505. For example, in this implementation, the combined data of the subtokens 510-515 may include at least the data included in the token 505.
In another implementation, the subtokens 510-515 do not include all the data that is included in the token 505. For example, the subtokens 510-515 may include enough data to identify (e.g., through a lookup table or other data structure) data in the token 505. For example, the subtokens 510-515 may be combined to obtain R and address information. R and the address information may then be used by an offload provider to look up the other information included in the token 505.
In another example, one or more of the subtokens 510-515 may include data that may be used to map to the random data R. In this example, the random data R cannot be reconstructed from the data found solely in the subtokens 510-515, but the random data R may be found (e.g., in a lookup table) from the random data included in one or more of the subtokens 510-515. In this example, other data (e.g., the address data of the token 505) may be included in one or more of the subtokens 510-515. The address data may then be used to locate a mapping table, for example, that may be used to locate the other data included the token 505.
A similar mechanism may also be used to find other omitted data that is not physically found in the subtokens 510-515 but that may be found using data that maps to the omitted data.
In one example, one of the subtokens (sometimes referred to herein as the master subtoken) may include all of the random data R while the other subtokens may not include any data that corresponds to the random data R. In another example, the subtokens 510-515 may each include data that corresponds to the random data R.
Various mechanisms may be used to validate the token 505. In one example, after the token 505 is reconstructed from the subtokens 510-515, a bitwise comparison is performed to determine whether the token 505 is exactly a token generated by the offload provider. If the bits in the token equal the bits found for a token having R in the token store, the token 505 may be determined to be valid.
In another example, a digest of the token 505 may be computed and the digest may be compared to digests of tokens generated by the offload provider. In this example, a digest may be selected that has a low or no possibility of colliding with other digests. In this example, if the digest is equal to a digest of a token generated by the offload provider, the token 505 may be determined to be valid.
In another example, the token 505 may be determined to be valid if R is equal to an R of a token stored by the offload provider.
In one implementation, the larger token 505 is a token that is provided and actually exists and is implemented as one or more data structures. The larger token 505 may be physically divided into the multiple subtokens 510 which may also be recombined to form the larger token 505.
In another implementation, the larger token 505 comprises a virtual offload token that logically includes the fields illustrated for the token 505, but where all the fields may not actually be in the same data structure. In this implementation, the token 505 does not go through a period where a single chunk of data includes all the fields of the token 505. Instead, the subtokens 510-515 include data corresponding to the token 505 (or data usable to find the data of the token 505) but the subtokens 510-515 are not actually combined to form a monolithic chunk of data that includes the fields of the token 505. Likewise, in this implementation, the token 505 is not first created and then divided into the subtokens 510-515. The larger token 505 is referred to as a virtual offload token because it does not exist physically and independently of the subtokens 510-515 but exists virtually in the data of the subtokens 510-515. It is to be understood that when the token 505 is described herein that both implementations are contemplated.
FIG. 6 is a block diagram that represents an exemplary arrangement of components of a system in which aspects of the subject matter described herein may operate. As illustrated, the system includes an initiator 605, a source storage stack 610, a destination storage stack 611, a spittler/injector 615, a combiner/extractor 616, and an offload provider 630.
The offload provider 630 as illustrated is separated into a source offload provider 635 and a destination offload provider 636 to indicate that components of the offload provider 630 may be on different machines that communicate with each other to perform the functions of the offload provider 630. In another example, however, the source offload provider 635 and the destination offload provider 636 may be merged and placed on a single computer. In one implementation, the source offload provider 635 and the destination offload provider 636 are different offload providers altogether that may negotiate the transmission of offload data in response to an offload write command.
The initiator 605 initiates an offload read or write. In one example, the initiator 605 may be separated into a source initiator and destination initiator (as illustrated in FIG. 4) where the source initiator initiates an offload read and obtains multiple subtokens in response thereto and then provides the subtokens to the destination initiator which later initiates an offload write. In another example, the initiator 605 may directly initiate both the offload read and the offload write.
It is to be understood that an offload write is an offload write regardless of form. For example, forwarding a token to a different machine that in turn issues an offload write is really just a different way for an offload read initiator to initiate on offload write.
The source storage stack 610 and the destination storage stack 611 may each be implemented by one or more components arranged in layers where each layer may perform a different function.
The spittler/injector 615 may include one or more components. The spittler/injector 615 may receive an offload read command from the source storage stack 610. In response, the spittler/injector 615 may send an offload read command to the source offload provider 635. In response to the offload read command, the source offload provider 635 may provide a large token. After receiving the large token, the spittler/injector 615 may split the token into a plurality of smaller tokens and provide these smaller tokens to the source storage stack 610. The subtokens may, for example, be of a fixed standardized size as mentioned previously.
In one implementation, an offload read command may include a number that indicates how many subtokens may be provided in response to the offload read. This number may originate from the initiator 605 or a component of the source storage stack 610.
If the spittler/injector 615 determines that the number is large enough, the spittler/injector 615 may provide the subtokens as requested by the source storage stack 610. Otherwise, in one example, the spittler/injector 615 may return a message that indicates how many subtokens are needed to respond to the offload read request. In another example, the spittler/injector 615 may return an error that indicates that the number is not large enough and may allow the initiator 605 to try a larger number(s) if the initiator 605 determines to do so.
In another implementation, an offload read command may omit a number that indicates how many subtokens may be provided in response to the offload read. In this implementation, the component sending the offload read command may request subtokens until the spittler/injector 615 indicates that all subtokens for the offload read command have been provided.
In another implementation, the spittler/injector 615 may indicate a number of subtokens that were generated in response to the offload read command. The component that sent the offload read command may then be responsible for obtaining the subtokens from the spittler/injector 615.
In an offload write command with multiple subtokens, the initiator 605 may send the subtokens to the destination storage stack 611 which may send the subtokens to the combiner/extractor 616. The combiner/extractor 616 may then combine the subtokens into a single large token and provide the single large token to the destination offload provider 636.
The subtokens may be provided in a single message or in multiple messages depending on implementation.
In one implementation, the spittler/injector 615 may be combined with the source offload provider 635 and the combiner/extractor 616 may be combined with the destination offload provider 636. In at least this implementation, the spittler/injector 615 may inject data of a virtual offload token into the subtokens while the combiner/extractor 616 may extract the data from the subtokens without the larger token ever existing as a physical data structure.
FIGS. 7-9 are flow diagrams that generally represent exemplary actions that may occur in accordance with aspects of the subject matter described herein. For simplicity of explanation, the methodology described in conjunction with FIGS. 7-9 is depicted and described as a series of acts. It is to be understood and appreciated that aspects of the subject matter described herein are not limited by the acts illustrated and/or by the order of acts. In one embodiment, the acts occur in an order as described below. In other embodiments, however, the acts may occur in parallel, in another order, and/or with other acts not presented and described herein. Furthermore, not all illustrated acts may be required to implement the methodology in accordance with aspects of the subject matter described herein. In addition, those skilled in the art will understand and appreciate that the methodology could alternatively be represented as a series of interrelated states via a state diagram or as events.
FIG. 7 is a flow diagram that generally represents exemplary actions that may occur at a destination offload provider in accordance with aspects of the subject matter described herein. At block 705, the actions begin.
At block 710, a message is received that indicates that two or more subtokens represent a larger token. The subtokens are each of a fixed size (e.g., a size specified by a standard). The larger token has a size larger than the fixed size. This means that the data included in the larger token is more than can fit into one of the subtokens. The data corresponding to the larger token is maintained by an offload provider. The data may be maintained in a single data structure corresponding to the larger token or in multiple data structures (e.g., that are not combined). The larger token represents data that is immutable as long as the data is larger token is valid.
For example, referring to FIG. 6, the combiner/extractor 616 may receive subtokens from the destination storage stack 611. The subtokens may be provided by the initiator 605 in conjunction with an offload write directed to the destination storage stack 611.
At block 715, data is extracted from the subtokens. Extracting data may include, for example, combining the subtokens into a larger token prior to obtaining the data or obtaining the data from the subtokens without combining the subtokens into the larger token. For example, referring to FIG. 6, the combiner/extractor 616 may combine/extract data from the subtokens. For example, some of the data extracted may include a number that associates the token with the data the token represents. This number is sometimes referred to as a key.
At block 720, the key is obtained from one or more of the subtokens. For example, referring to FIG. 6, after the combiner/extractor 616 combines the subtokens to form a larger token, the destination offload provider 636 may obtain the key from the larger token. As another example, without physically combining the data of the subtokens, the combiner/extractor 616 may extract the key from a virtual token (e.g., one or more of the subtokens) without physically combining all the data of the subtokens.
At block 725, evidence of the key is provided to a component of the offload provider. Using the evidence, the key may be validated as part of the actions of block 725 or as a separate set of actions. Providing evidence of the key may include, for example:
1. Providing the key itself;
2. Providing the key and other data (one or more fields) of the larger token;
3. Providing a digest (e.g., a hash function) of the key;
4. Providing a digest derived from the key and other data (one or more fields) of the larger token; or
5. Providing other evidence of the key and/or larger token.
FIG. 8 is a flow diagram that generally represents exemplary actions that may occur at a source offload provider in accordance with aspects of the subject matter described herein. At block 805, the actions begin.
At block 810, an offload read request is received. For example, referring to FIG. 6, the source offload provider 635 receives an offload read request initiated by the initiator 605.
At block 815, in response to the offload read message, a key is generated to return in response to the offload read message. The key in placed in a token (physical or virtual), the data of which will be placed into subtokens to return in response to the offload read message. For example, referring to FIG. 6, the source offload provider 635 may generate a token that includes the key.
At block 820, the data of the token is divided/injected into subtokens. For example, referring to FIG. 6, the splitter/injector 615 takes the data from the token generated at block 815 and splits/injects the data into subtokens that are provided to the source storage stack 610 for delivery to the initiator 605.
At block 825, evidence of the key is received. For example, referring to FIG. 6, a component of the source offload provider 635 receives evidence of the key. In one example, this evidence may be received as the offload provider 630 obtains the key from the subtokens received from the combiner/extractor 616. In another example, a component of the offload provider at a destination of an offload write (e.g., the destination offload provider 636) may obtain the key included in the subtokens, read an address contained therein, use the address to contact a component (e.g., the source offload provider 635) of the offload provider that generated the key, and provide the key to the component. In another example, a destination offload provider that is a different offload provider from the source offload provider may receive the key and addressing information, contact the source offload provider, and provide the key. Using the evidence, the key may be validated as part of the actions of block 825 or as a separate set of actions.
At block 830, bulk data corresponding to the token is provided. For example, referring to FIG. 6, the source offload provider 635 may provide a portion or all of the bulk data corresponding to the token to the destination offload provider 636.
FIG. 9 is a flow diagram that generally represents exemplary actions that may occur at an offload initiator in accordance with aspects of the subject matter described herein. At block 905, the actions begin.
At block 910, an offload read request is initiated by communicating with a component of a source storage stack. For example, referring to FIG. 6, the initiator 605 may send an offload read request to the source storage stack 610. In conjunction with the offload read request, a number may be send that indicates a maximum number of subtokens that are allowed to be returned in response to the offload read request.
At block 915, in response to the message, subtokens are received. The subtokens represent a token (physical or virtual) that is larger than any of the subtokens individually. The larger token represents data that is immutable as long as the larger token is valid. For example, referring to FIG. 6, in response to the offload read request, the initiator receives multiple subtokens. In conjunction with receiving the subtokens, a number may be received that indicates how many subtokens were generated in response to the offload read request.
At block 920, the initiator provides the subtokens to a component of a destination storage stack. For example, referring to FIG. 6, the initiator 605 provides the subtokens to a component of the destination storage stack 611.
At block 925, other actions, if any, may be performed.
As can be seen from the foregoing detailed description, aspects have been described related to offload technology. While aspects of the subject matter described herein are susceptible to various modifications and alternative constructions, certain illustrated embodiments thereof are shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit aspects of the claimed subject matter to the specific forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions, and equivalents falling within the spirit and scope of various aspects of the subject matter described herein.
receiving, by the computing device, two or more subtokens, each of a fixed size, the subtokens together representing a larger token of a size larger than the fixed size, the larger token representing data that is immutable as long as the larger token is valid;
obtaining, by the computing device from one or more of the subtokens, a cryptographically secure key and information that identifies a source from which the data represented by the lamer token is obtainable; and
providing, by the computing device to an offload provider that comprises at least one other computing device that is separate from the computing device, evidence of the key to obtain a portion of the data represented by the larger token without the portion of the data passing through an initiator that provided the subtokens.
2. The method of claim 1, further comprising obtaining information additional to the addressing information from the one or more of the subtokens.
3. The method of claim 1, further comprising combining data from the two or more subtokens to create the larger token, the two or more subtokens including at least all data included in the larger token.
4. The method of claim 1, where the providing the evidence of the key comprises providing the key.
5. The method of claim 1, where the providing the evidence of the key comprises providing a digest of the key.
6. The method of claim 1, where the providing the evidence of the key comprises providing a digest derived from the key and other data of the larger token.
7. The method of claim 1, where the larger token comprises a virtual token that is represented by data of the subtokens.
8. The method of claim 1, where the subtokens are each exactly 512 bytes, or where one or more of the subtokens includes a field required by a standard, the field identifying a representation of data token type.
9. The method of claim 1, where the subtokens are transmitted via a file sharing protocol that transfers file data via server message blocks.
10. The method of claim 1, where the subtokens are transmitted via a distributed file system protocol that accesses files via remote procedure calls.
a computing device that includes at least one processor and memory; and
at least one program module that, based on execution by the at least one processor, configures the computing device to receive an offload read message initiated by an initiator;
generate in response to the received offload read message, a cryptographically secure key; and
provide, to a separate computing device, subtokens, each of a fixed size, the subtokens representing a larger token of a size larger than the fixed size data corresponding to the larger token maintained by an offload provider, the subtokens comprising information that identifies a source from which the data represented by the lamer token is obtainable, the larger token representing data that is immutable as long as the larger token is valid, the larger token including the cryptographically secure key.
12. The system of claim 11, where the larger token is a physical token and wherein splitter/injector splits a physical token into the subtokens.
13. The system of claim 11, where the larger token is a virtual token that is embodied in the subtokens but that is not embodied as a single token separate from the subtokens, and where portions of the virtual token are injected into the subtokens.
14. The system of claim 11, the computing device further configured to receive an offload write message that includes data corresponding to the subtokens.
15. At least one hardware computer storage medium having computer-executable instructions that, based on execution by at least one processor of at least one computing device that include at least one memory, configure the computing device to perform actions, comprising:
initiating, by the at least one computing device, an offload read request by communicating with a component of a source storage stack;
receiving, by the at least one computing device from at least one separate computing device in response to the offload read request, subtokens, the subtokens representing a larger token that is larger than any of the subtokens individually, the larger token representing data that is immutable as long as the larger token is valid, where the received subtokens comprise information that identifies a source from which the data represented by the lamer token is obtainable; and
initiating, by the at least one computing device, an offload write request by providing the subtokens to a destination storage stack.
16. The at least one computer storage medium of claim 15, the actions further comprising: in conjunction with the offload read request, sending a number that indicates a maximum number of subtokens that are allowed to be returned in response to the offload read request.
17. The at least one computer storage medium of claim 15, the actions further comprising: in response to offload read request, receiving a number that indicates how many subtokens were generated in response to the offload read request.
US13/714,413 2012-12-14 2012-12-14 Compatibly extending offload token size Active 2033-03-28 US9251201B2 (en)
US13/714,413 US9251201B2 (en) 2012-12-14 2012-12-14 Compatibly extending offload token size
PCT/US2013/075212 WO2014093952A1 (en) 2012-12-14 2013-12-14 Compatibly extending offload token size
RU2015122660A RU2672789C2 (en) 2012-12-14 2013-12-14 Compatibility extending offload token size
BR112015011935A BR112015011935A2 (en) 2012-12-14 2013-12-14 compatibly extending offload token size
JP2015548028A JP6420253B2 (en) 2012-12-14 2013-12-14 The expansion of off-road token size compatibility
EP13821245.1A EP2932692B1 (en) 2012-12-14 2013-12-14 Compatibly extending offload token size
CN201380065507.9A CN104995895A (en) 2012-12-14 2013-12-14 Compatibly extending offload token size
US20140172811A1 US20140172811A1 (en) 2014-06-19
US9251201B2 true US9251201B2 (en) 2016-02-02
ID=49956364
US13/714,413 Active 2033-03-28 US9251201B2 (en) 2012-12-14 2012-12-14 Compatibly extending offload token size
US (1) US9251201B2 (en)
EP (1) EP2932692B1 (en)
JP (1) JP6420253B2 (en)
CN (1) CN104995895A (en)
BR (1) BR112015011935A2 (en)
RU (1) RU2672789C2 (en)
WO (1) WO2014093952A1 (en)
US20020198788A1 (en) 2001-06-20 2002-12-26 International Business Machines Corporation System and method for product evaluation
US6785743B1 (en) 2000-03-22 2004-08-31 University Of Washington Template data transfer coprocessor
US20060230222A1 (en) 2004-02-18 2006-10-12 Kenji Yamagami Storage control system and control method for the same
CN100343793C (en) 2004-11-19 2007-10-17 国际商业机器公司 Autonomic data caching and copying on a storage area network aware file system using copy services
US20080065835A1 (en) 2006-09-11 2008-03-13 Sun Microsystems, Inc. Offloading operations for maintaining data coherence across a plurality of nodes
US20080128484A1 (en) 2002-09-13 2008-06-05 Paul Spaeth Method and system for managing token image replacement
WO2008070811A2 (en) 2006-12-06 2008-06-12 Fusion Multisystems, Inc. (Dba Fusion-Io) Apparatus, system, and method for managing data in a storage device with an empty data token directive
US20080147755A1 (en) 2002-10-10 2008-06-19 Chapman Dennis E System and method for file system snapshot of a virtual logical disk
CN101278270A (en) 2005-10-07 2008-10-01 国际商业机器公司 Apparatus and method for handling DMA requests in a virtual memory environment
US20090172665A1 (en) 2005-06-24 2009-07-02 Azul Systems, Inc. Reducing latency in a segmented virtual machine
US20090248835A1 (en) 2008-03-31 2009-10-01 Subhankar Panda Offloading data transfers between a local and remote network
JP2010033206A (en) 2008-07-28 2010-02-12 Fujitsu Ltd Virtual machine monitor device, program, and memory sharing management method between virtual machines
US20100083276A1 (en) 2008-09-30 2010-04-01 Microsoft Corporation On-the-fly replacement of physical hardware with emulation
US20100115184A1 (en) 2008-11-04 2010-05-06 Phison Electronics Corp. Flash memory storage system and controller and data protection method thereof
US20100115208A1 (en) 2008-10-31 2010-05-06 John Gifford Logan Control i/o offload in a split-path storage virtualization system
EP2262164A1 (en) 2008-02-18 2010-12-15 Microelectronica Española, S.A.U. Secure data transfer
US7890717B2 (en) 2005-08-31 2011-02-15 Hitachi, Ltd. Storage system, data transfer method, and program
US20110055406A1 (en) 2005-12-02 2011-03-03 Piper Scott A Maintaining session states within virtual machine environments
US20110214172A1 (en) 2004-10-22 2011-09-01 Eckehard Hermann Authentication Over a Network Using One-Way Tokens
WO2012039939A2 (en) 2010-09-23 2012-03-29 Microsoft Corporation Offload reads and writes
US20120233434A1 (en) 2011-03-11 2012-09-13 Microsoft Corporation Virtual Disk Storage Techniques
US20120324560A1 (en) 2011-06-17 2012-12-20 Microsoft Corporation Token data operations
US20130041985A1 (en) 2011-08-10 2013-02-14 Microsoft Corporation Token based file operations
US20130179649A1 (en) 2012-01-09 2013-07-11 Microsoft Corporation Offload Read and Write Offload Provider
US20130179959A1 (en) 2012-01-05 2013-07-11 Microsoft Corporation Zero Token
US20140164571A1 (en) 2012-12-12 2014-06-12 Microsoft Corporation Copy offload for disparate offload providers
US9147081B2 (en) * 2010-07-27 2015-09-29 Infinidat Ltd. Method of access control to stored information and system thereof
2012-12-14 US US13/714,413 patent/US9251201B2/en active Active
2013-12-14 JP JP2015548028A patent/JP6420253B2/en active Active
2013-12-14 WO PCT/US2013/075212 patent/WO2014093952A1/en active Application Filing
2013-12-14 BR BR112015011935A patent/BR112015011935A2/en active Search and Examination
2013-12-14 CN CN201380065507.9A patent/CN104995895A/en active Search and Examination
2013-12-14 EP EP13821245.1A patent/EP2932692B1/en active Active
2013-12-14 RU RU2015122660A patent/RU2672789C2/en active
US20120233682A1 (en) 2006-11-22 2012-09-13 Verizon Patent And Licensing Inc. Secure access to restricted resource
US8261267B2 (en) 2008-07-28 2012-09-04 Fujitsu Limited Virtual machine monitor having mapping data generator for mapping virtual page of the virtual memory to a physical memory
US8250267B2 (en) 2008-10-31 2012-08-21 Netapp, Inc. Control I/O offload in a split-path storage virtualization system
"ControlSphere Token data structure", as archived on web.archive.org on Jul. 5, 2007.
"Copy Offload for Disparate Offload Providers", U.S. Appl. No. 13/711,637, filed Dec. 12, 2012, pp. 70.
"Offloaded Data Transfer (ODX) with Intelligent Storage Arrays", Retrieved at <<http://feishare.com/attachments/article/297/windows-offloaded-data-transfer.pdf, Feb. 28, 2012, pp. 14.
"Offloaded Data Transfer (ODX) with Intelligent Storage Arrays", Retrieved at <<http://feishare.com/attachments/article/297/windows-offloaded-data-transfer.pdf>>, Feb. 28, 2012, 14 pages.
"Offloaded Data Transfer (ODX) with Intelligent Storage Arrays", Retrieved at >, Feb. 28, 2012, 14 pages.
"What pv technology means in vSphere", Jan. 30, 2010.
12/888,433, filed Sep. 23, 2010, Neal R. Christiansen.
12/938,383, filed Nov. 3, 2010, Dustin L. Green.
13/162,592, filed Jun. 17, 2011, Bryan Matthew.
13/207,014, filed Aug. 10, 2011, Neal R. Christiansen.
13/343,718, filed Jan. 5, 2012, Dustin L. Green.
13/345,753, filed Jan. 9, 2012, Dustin L. Green.
Agarwal, "Distributed Checkpointing of Virtual Machines in Xen Framework", May 2008.
AU Patent Examination Report No. 1 for Application No. 2011305839, Apr. 4, 2014.
AU Patent Examination Report No. 2 for Application No. 2011305839, Jul. 2, 2014.
Bandulet, "Object Based Storage Devices", Jul. 2007.
Campbell, "Hyper-V Scalability Evident in Its New Virtual Disk Format", Jul. 9, 2012.
CN First Office Action for Application No. 201110285468.6, Jan. 30, 2014.
CN First Office Action for Application No. 201110343154.7, Jan. 22, 2014.
CN Second Office Action for Application No. 201110285468.6, Sep. 30, 2014.
CN Second Office Action for Application No. 201110343154.7, Sep. 23, 2014.
CN Third Office Action for Application No. 201110343154.7, Mar. 24, 2015.
Dean, "Data Movement in Kernelized Systems", Proceedings of the Workshop on Micro-kernels and Other Kernel Architectures, Apr. 27-28, 1992.
Dell, "Auto-Snapshot Manager/VMware Edition: Automated, Integrated, and Scalable Protection of Virtual Machines", 2009.
Devulapalli, "Design of an Intelligent Object-based Storage device", originally retrieved Jul. 5, 2010.
Didier Van Hoye, "TRIM/UNMAP Support In Windows Server 2012 & Hyper-V/VHDX", Working Hard in It, May 23, 2012.
Elnozahy, "The Performance of Consistent Checkpointing", Proceedings of the 11th Symposium on Reliable Distributed Systems, Oct. 5-7, 1992.
EP Communication for Application No. 12821531.6, Feb. 13, 2015.
Eshel, "Panache: A Parallel File System Cache for Global File Access", Proceedings of the 8th USENIX Conference on File and Storage Technologies (FAST'10), Feb. 23-26, 2010.
Faulkner, "How to Resize a Microsoft Virtual Hard Drive (VHD) File", How-To Geek, Jul. 26, 2010.
IBM, "Disk cache infrastructure enhancements", IBM Knowledge Center, Jun. 15, 2010.
Jarvinen, "Embedded SFE: Offloading Server and Network using Hardware Tokens", Proceedings of the 14th International Conference on Financial Cryptography and Data Security (FC'10), Jan. 25-28, 2010.
JP Notice of Rejection for Application No. 2013-530171, Apr. 16, 2015.
Kerr, "Windows With C++: The Virtual Disk API In Windows 7", MSDN Magazine, Apr. 2009.
Lachaize, "A Distributed Shared Buffer Space for Data-intensive Applications", vol. 2, Proceedings of the 5th International Symposium on Cluster Computing and the Grid (CCGrid 2005), May 9-12, 2005.
Lai, "Recovering VMware snapshot after parent changed", edgylogic, Nov. 6, 2007.
LászlóCzap et al., Secure Key Exchange in Wireless Networks, 2011, IEEE, 6 pages. *
Leser, "Towards a Worldwide Distributed File System: The OSF DCE File System as an example", DCE Evaluation Team, Open Software Foundation(TM), Sep. 27, 1990.
Leser, "Towards a Worldwide Distributed File System: The OSF DCE File System as an example", DCE Evaluation Team, Open Software Foundation™, Sep. 27, 1990.
Microsoft, "Deploying Virtual Hard Disk Images", Jan. 6, 2009.
Microsoft, "FSCTL-FILE-LEVEL-TRIM control code", Windows Dev Center, Jul. 19, 2012.
Microsoft, "STORAGE-OFFLOAD-TOKEN structure", Hardware Dev Center, originally retrieved Sep. 7, 2011.
Microsoft, "Using differencing disks", TechNet, as archived on web.archive.org on Apr. 4, 2011.
Microsoft, "Virtual Hard Disk Image Format Specification", Version 1.0, Oct. 11, 2006.
Narayanan, "Write Off-Loading: Practical Power Management for Enterprise Storage", Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST'08), Feb. 26-29, 2008.
PCT International Search Report and Written Opinion for Application No. PCT/US2011/050739, Apr. 9, 2012.
PCT International Search Report and Written Opinion for Application No. PCT/US2012/047261, Jan. 31, 2013.
PCT International Search Report and Written Opinion for Application No. PCT/US2013/074509, Mar. 28, 2014.
PCT International Search Report and Written Opinion for Application No. PCT/US2013/075212, Apr. 25, 2014, all pages.
PCT Notification of Transmittal of the International Preliminary Report on Patentability for Application No. PCT/US2013/075212, Mar. 9, 2015.
PCT Written Opinion of the International Preliminary Examining Authority for Application No. PCT/US2013/075212, Oct. 22, 2014.
Sun Microsystems, Inc., "Saving and Restoring ZFS Data: Chapter 6. Working With ZFS Snapshots and Clones", Solaris ZFS Administration Guide, May 2006.
VMware, "Troubleshooting parent virtual disk errors in Fusion", Knowledge Base, originally retrieved Feb. 23, 2011.
Walla, "Kerberos Explained", Windows 2000 Advantage magazine, Microsoft TechNet, May 2000, all pages.
Wang, "A computation offloading scheme on handheld devices", Journal of Parallel Distributed Computing, Elsevier, Jun. 2004.
Wang, "Cooperative Cache Management in S2FS", Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA), Jun. 28-Jul. 1, 1999.
Wu, et al., "Distributed Runtime Load-Balancing for Software Routers on Homogeneous Many-Core Processors", Retrieved at <<http://conferences.sigcomm.org/co-next/2010/Workshops/PRESTO/PRESTO-papers/01-Wu.pdf>>, Proc. of the ACM context workshop on programmable routers for extensible services of tomorrow (PRESTO), Nov. 30, 2011, pp. 6.
Wu, et al., "Distributed Runtime Load-Balancing for Software Routers on Homogeneous Many-Core Processors", Retrieved at >, Proc. of the ACM context workshop on programmable routers for extensible services of tomorrow (PRESTO), Nov. 30, 2011, pp. 6.
Yang, "Virtual Hard Disk Performance: Windows Server 2008/Windows Server 2008 R2/Windows 7", A Microsoft White Paper, Published: Mar. 2010.
RU2015122660A (en) 2016-12-27
JP6420253B2 (en) 2018-11-07
JP2016505960A (en) 2016-02-25
EP2932692A1 (en) 2015-10-21
BR112015011935A2 (en) 2017-07-11
CN104995895A (en) 2015-10-21
RU2672789C2 (en) 2018-11-19
EP2932692B1 (en) 2017-02-01
WO2014093952A1 (en) 2014-06-19
US20140172811A1 (en) 2014-06-19
CN101233514A (en) 2008-07-30 A method for managing I/O
US20140108617A1 (en) 2014-04-17 Data storage in cloud computing
US8311225B2 (en) 2012-11-13 Scalable key archival
US7409519B2 (en) 2008-08-05 Synchronizing logical systems
RU2672789C2 (en) 2018-11-19 Compatibility extending offload token size
US9606747B2 (en) 2017-03-28 Importing pre-existing data of a prior storage solution into a storage pool for use with a new storage solution
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:GREEN, DUSTIN L.;REEL/FRAME:029467/0456