Source: http://www.google.de/patents/US8082397
Timestamp: 2013-12-12 06:13:18
Document Index: 549834125

Matched Legal Cases: ['Application No. 60', 'art 450', 'art 450', 'art 500', 'art 500', 'art 700', 'art 700', 'art 800', 'art 800', 'art 250']

Patent US8082397 - Private slot - Google PatenteSuche Bilder Maps Play YouTube News Gmail Drive Mehr » Erweiterte Patentsuche | Anmelden Erweiterte Patentsuche PatenteDescribed are techniques and criteria used in connection with cache management. The cache may be organized as a plurality of memory banks in which each memory bank includes a plurality of slots. Each memory bank has an associated control slot that includes groups of extents of tags. Each cache slot has...http://www.google.de/patents/US8082397?utm_source=gb-gplus-sharePatent US8082397 - Private slot Ver�ffentlichungsnummerUS8082397 B1PublikationstypErteilung AnmeldenummerUS 10/955,136 Ver�ffentlichungsdatum20. Dez. 2011Eingetragen30. Sept. 2004 Priorit�tsdatum13. Aug. 2004 Ver�ffentlichungsnummer10955136, 955136, US 8082397 B1, US 8082397B1, US-B1-8082397, US8082397 B1, US8082397B1 ErfinderJosef Ezra, Adi OferUrspr�nglich Bevollm�chtigterEmc CorporationZitat exportierenBiBTeX, EndNote, RefManPatentzitate (10), Nichtpatentzitate (3), Referenziert von (1), Klassifizierungen (8), Juristische Ereignisse (1) Externe Links: USPTO, USPTO-Zuordnung, EspacenetPrivate slotUS 8082397 B1 Zusammenfassung Described are techniques and criteria used in connection with cache management. The cache may be organized as a plurality of memory banks in which each memory bank includes a plurality of slots. Each memory bank has an associated control slot that includes groups of extents of tags. Each cache slot has a corresponding tag that includes a bit value indicating the availability of the associated cache slot, and a time stamp indicating the last time the data in the slot was used. The cache may be shared by multiple processors. Exclusive access of the cache slots is implemented using an atomic compare and swap instruction. The time stamp of slots in the cache may be adjusted to indicate ages of slots affecting the amount of time a particular portion of data remains in the cache. Each director may obtain a cache slot from a private stack of nondata cache slots in addition to accessing a shared cache used by all directors.
providing for each processor a private cache including only nondata cache slots;
providing a shared cache including cache slots accessible by a plurality of processors;
receiving a request to allocate a cache slot for use by said each processor;
selecting a nondata cache slot from said private cache for allocation for data and nondata uses in connection with said request if said private cache includes any nondata cache slots; and
otherwise selecting a cache slot from said shared cache for allocation in connection with said request.
2. The method of claim 1, wherein the method for cache management is used a data storage system, each processor being a director included in the data storage system, and said cache slot is selected in connection with processing an I/O operation.
3. The method of claim 1, further comprising performing by said processor:
determining whether a first cache slot is a data cache slot or a non-data cache slot;
if said first cache slot is a non-data cache slot, designating said first cache slot as being included in said private cache of said processor; and
if said first cache slot is a data cache slot, indicating that said first cache slot is available for use by all of said plurality of processors.
determining if said private cache of said processor is full; and
indicating that said first cache slot is available for use by all of said plurality of processors if said private cache is full.
5. The method of claim 4, wherein said first cache slot is indicated as being included in said private cache or said shared cache in accordance with a first indicator associated with said first cache slot.
6. The method of claim 5, wherein said first cache slot is included in said private cache and includes a unique processor identifier identifying the particular processor that is associated with said private cache.
7. The method of claim 6, wherein said first cache slot is indicated as unavailable for use by any processor other than said particular processor by a second indicator in said cache slot.
updating said first indicator to indicate that said first slot is included in said shared cache after obtaining a lock on said first slot for exclusive access.
retrying to obtain said lock a plurality of times if previous attempts to obtain said lock indicate that another processor currently has said lock.
10. The method of claim 5, wherein said first cache slot is indicated as being a private cache slot and is physically located in a memory next to a second cache slot indicated as being included in said shared cache available for use by a plurality of processors.
11. The method of claim 1, wherein said selected cache slot is selected from said private cache slot and the method further comprising:
determining whether said cache slot is to be included in said shared cache in accordance with an I/O operation associated with said selected cache slot.
12. The method of claim 11, wherein said cache slot remains not available for reuse by said plurality of processors if data included in said cache slot is associated with a pending write operation.
13. The method of claim 11, wherein said cache slot is designated as available for reuse by said plurality of processors if said cache slot is associated with a read operation.
14. A computer program product comprising a computer readable medium with code stored thereon for cache management, the computer readable medium comprising code that:
provides for each processor a private cache including only nondata cache slots;
provides a shared cache including cache slots accessible by a plurality of processors;
receives a request to allocate a cache slot for use by said each processor;
selects a nondata cache slot from said private cache for allocation for data and nondata uses in connection with said request if said private cache includes any nondata cache slots; and
otherwise selects a cache slot from said shared cache for allocation in connection with said request.
15. The computer program product of claim 14, wherein the cache management is used a data storage system, each processor being a director included in the data storage system, and said cache slot is selected in connection with processing an I/O operation.
16. The computer program product of claim 14, further comprising code that causes said processor to perform:
17. The computer program product of claim 16, further comprising code that:
determines if said private cache of said processor is full; and
indicates that said first cache slot is available for use by all of said plurality of processors if said private cache is full.
18. The computer program product of claim 17, wherein said first cache slot is indicated as being included in said private cache or said shared cache in accordance with a first indicator associated with said first cache slot.
19. The computer program product of claim 18, wherein said first cache slot is included in said private cache and includes a unique processor identifier identifying the particular processor that is associated with said private cache.
20. The computer program product of claim 19, wherein said first cache slot is indicated as unavailable for use by any processor other than said particular processor by a second indicator in said cache slot.
21. The computer program product of claim 18, further comprising code that:
updates said first indicator to indicate that said first slot is included in said shared cache after obtaining a lock on said first slot for exclusive access.
22. The computer program product of claim 21, further comprising code that:
retries to obtain said lock a plurality of times if previous attempts to obtain said lock indicate that another processor currently has said lock.
23. The computer program product of claim 18, wherein said first cache slot is indicated as being a private cache slot and is physically located in a memory next to a second cache slot indicated as being included in said shared cache available for use by a plurality of processors.
24. The computer program product of claim 14, wherein said selected cache slot is selected from said private cache slot and the computer program product further comprising code that:
determines whether said cache slot is to be included in said shared cache in accordance with an I/O operation associated with said selected cache slot.
25. The computer program product of claim 24, wherein said cache slot remains not available for reuse by said plurality of processors if data included in said cache slot is associated with a pending write operation.
26. The computer program product of claim 24, wherein said cache slot is designated as available for reuse by said plurality of processors if said cache slot is associated with a read operation. Beschreibung
RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 60/601,397, filed on Aug. 13, 2004, which is incorporated by reference herein.
This application generally relates to caching, and more particularly to cache management as may be used in a data storage system.
Different techniques may be used to manage the cache. In some implementations, cache management operations may be costly in terms of computer resources, such as execution time and memory. For example, it may costly to determine an available slot for use. An executing processor may make multiple attempts at different slots before locating one which can be used to store new data in the cache. There are also costs associated with inserting and removing an element from the cache that may vary with the particular cache implementation and associated data structures.
Thus, it may be desirable to provide an efficient cache management technique which reduces the costs associated with cache management. The reduction may be produced as a result of more efficient processing of one or more individual operations. The reduction may also be produced by reducing the frequency with which any one or more cache management operations are performed in cache operation.
SUMMARY OF THE INVENTION In accordance with one aspect of the invention is a method for cache management comprising: providing for each processor a private cache including only nondata cache slots; providing a shared cache including cache slots accessible by a plurality of processors; and wherein a cache slot for use by a processor is determined by selecting a cache slot from said private cache of said processor if said private cache is not empty, and wherein a cache slot is selected from said shared cache otherwise. The method for cache management may be used in a data storage system, each processor being a director included in the data storage system, and said cache slot may be selected in connection with processing an I/O operation. The method may also include performing by said processor: determining whether a first cache slot is a data cache slot or a non-data cache slot; if said first cache slot is a non-data cache slot, designating said first cache slot as being included in said private cache of said processor; and if said first cache slot is a data cache slot, indicating that said first cache slot is available for use by all of said plurality of processors. The method may include determining if said private cache of said processor is full; and indicating that said first cache slot is available for use by all of said plurality of processors if said private cache is full. The first cache slot may be indicated as being included in said private cache or said shared cache in accordance with a first indicator associated with said first cache slot. The first cache slot may be included in said private cache and may include a unique processor identifier identifying the particular processor that is associated with said private cache. The first cache slot may be indicated as unavailable for use by any processor other than said particular processor by a second indicator in said cache slot. The method may include updating said first indicator to indicate that said first slot is included in said shared cache after obtaining a lock on said first slot for exclusive access. The method may include retrying to obtain said lock a plurality of times if previous attempts to obtain said lock indicate that another processor currently has said lock. The first cache slot may be indicated as being a private cache slot and may be physically located in a memory next to a second cache slot indicated as being included in said shared cache available for use by a plurality of processors. The selected cache slot may be selected from said private cache slot and the method may further comprise determining whether said cache slot is to be included in said shared cache in accordance with an I/O operation associated with said selected cache slot. The cache slot may remain not available for reuse by said plurality of processors if data included in said cache slot is associated with a pending write operation. The cache slot may be designated as available for reuse by said plurality of processors if said cache slot is associated with a read operation.
In accordance with another aspect of the invention is a computer program product for cache management comprising code that: provides for each processor a private cache including only nondata cache slots; provides a shared cache including cache slots accessible by a plurality of processors; and wherein a cache slot for use by a processor is determined by selecting a cache slot from said private cache of said processor if said private cache is not empty, and wherein a cache slot is selected from said shared cache otherwise. The cache management may be used a data storage system, each processor being a director included in the data storage system, and said cache slot may be selected in connection with processing an I/O operation. The computer program product may include code that causes said processor to perform: determining whether a first cache slot is a data cache slot or a non-data cache slot; if said first cache slot is a non-data cache slot, designating said first cache slot as being included in said private cache of said processor; and if said first cache slot is a data cache slot, indicating that said first cache slot is available for use by all of said plurality of processors. The computer program product may include code that: determines if said private cache of said processor is full; and indicates that said first cache slot is available for use by all of said plurality of processors if said private cache is full. The first cache slot may be indicated as being included in said private cache or said shared cache in accordance with a first indicator associated with said first cache slot. The first cache slot may be included in said private cache and may include a unique processor identifier identifying the particular processor that is associated with said private cache. The first cache slot may be indicated as unavailable for use by any processor other than said particular processor by a second indicator in said cache slot. The computer program product may also include code that: updates said first indicator to indicate that said first slot is included in said shared cache after obtaining a lock on said first slot for exclusive access. The computer program product may also include code that: retries to obtain said lock a plurality of times if previous attempts to obtain said lock indicate that another processor currently has said lock. The first cache slot may be indicated as being a private cache slot and may be physically located in a memory next to a second cache slot indicated as being included in said shared cache available for use by a plurality of processors. The selected cache slot may be selected from said private cache slot and the computer program product may further comprise code that determines whether said cache slot is to be included in said shared cache in accordance with an I/O operation associated with said selected cache slot. The cache slot may remain not available for reuse by said plurality of processors if data included in said cache slot is associated with a pending write operation. The cache slot may be designated as available for reuse by said plurality of processors if said cache slot is associated with a read operation.
FIG. 9 is an example of an embodiment of a private stack structure as may be used by each director;
FIG. 10 is a flowchart of processing steps in one embodiment for obtaining a new cache slot;
FIG. 11 is a flowchart of processing steps in one embodiment for handling a cache slot when a director determines that the cache slot may be reused;
FIG. 12 is an example representation of an embodiment of a heartbeat table;
FIGS. 13 and 14 are example illustrations of embodiments of cache structures;
FIG. 15 is a flowchart of more detailed processing steps in one embodiment for performing a push operation;
FIG. 16 is a flowchart of more detailed processing steps in one embodiment for performing a pop operation;
FIGS. 17-20 are flowcharts of processing steps of an embodiment for obtaining a cache slot from the shared cache;
FIG. 21 is an example of an embodiment of a tag-based cache with multiple memory banks.
DETAILED DESCRIPTION OF EMBODIMENT(S) Referring now to FIG. 1, shown is an example of an embodiment of a computer system. The computer system 10 includes a data storage area 12 connected to host systems 22 a-22 c through communication medium 18. In this embodiment of the computer system 10, the N hosts 22 a-22 c may access the data storage area 12, for example, in performing input/output (I/O) operations or data requests. The communication medium 18 may be any one of a variety of networks or other type of communication connections as known to those skilled in the art. The communication medium 18 may be a network connection, bus, and/or other type of data link, such as a hardwire or other connections known in the art. For example, the communication medium 18 may be the Internet, an intranet, network or other connection(s) by which the host systems 22 a-22 c may access and communicate with the data storage area 12, and may also communicate with each other and other components included in the computer system 10.
Referring now to FIG. 2A, shown is a diagram 20 illustrating additional detail of one embodiment of the system 10 of FIG. 1. The plurality of hosts 22 a-22 c are coupled to a data storage system 24. The data storage system 24 may be one of a plurality of data storage systems included in the data storage area 12. The data storage system 24 includes an internal memory 26 that facilitates operation of the storage system 24 as described elsewhere herein. The data storage system also includes a plurality of host adaptors (HA's) 28 a-28 c that handle reading and writing of data between the hosts 22 a-22 c and the storage system 24. Although the diagram 20 shows each of the hosts 22 a-22 c coupled to each of the HA's 28 a-28 c, it will be appreciated by one of ordinary skill in the art that one or more of the HA's 28 a-28 c may be coupled to other hosts.
Referring now to FIG. 2B, a diagram 50 illustrates an embodiment of the storage system 24 where each of a plurality of directors 52 a-52 c are coupled to the memory 26. Each of the directors 52 a-52 c represents one of the HA's 28 a-28 c, RA's 32 a-32 c, or DA's 38 a-38 c. In an embodiment disclosed herein, there may be up to sixteen directors coupled to the memory 26. Of course, for other embodiments, there may be a higher or lower maximum number of directors that may be used.
The diagram 50 also shows an optional communication module (CM) 54 that provides an alternative communication path between the directors 52 a-52 c. Each of the directors 52 a-52 c may be coupled to the CM 54 so that any one of the directors 52 a-52 c may send a message and/or data to any other one of the directors 52 a-52 c without needing to go through the memory 26. The CM 54 may be implemented using conventional MUX/router technology where a sending one of the directors 52 a-52 c provides an appropriate address to cause a message and/or data to be received by an intended receiving one of the directors 52 a-52 c. As described above, an embodiment may include a cache in the global memory portion 25 b of FIG. 2A. An embodiment may include a single or multiple replacement queue arrangement in the cache. An example of an embodiment that includes a cache using multiple replacement queues is described in pending U.S. patent application Ser. No. 09/535,134, entitled �Segmenting Cache to Provide Varying Service Levels�, filed Mar. 24, 2000, and assigned to EMC Corporation of Hopkinton, Mass. An example of a system with a single cache memory is described in issued U.S. Pat. No. 5,381,539, Yanai et al., entitled �System and Method for Dynamically Controlling Cache Management�, and also assigned to EMC Corporation of Hopkinton, Mass.
Referring now to FIG. 3, shown is an example of an embodiment 60 of a cache arrangement as a queue. Shown in the representation 60 is a circular structure in which each of the elements, such as 62, corresponds to a cache slot. Each cache slot may correspond to a portion of memory, such as one or more memory blocks. Each memory block may correspond to, for example, a track on one of the drives shown in connection with FIG. 2A. In this representation, each of the slots are connected to other slots by forward and backward pointers, such as 62 a and 62 b, in a doubly linked list arrangement. Additionally, the head or beginning of the queue is designated by a head pointer 64.
It should be noted that as described herein, an embodiment may include a cache which is in the form of the foregoing queue using doubly linked list or other data structures known to those of ordinary skill in the art. The queue described herein should not be construed as a limitation to the techniques described herein. Additionally, it should be noted that an embodiment may use a least-recently-used or other technique in determining which slots remain in the cache and which ones are removed.
Referring now to FIG. 4, shown is an equivalent representation 70 of the previously described queue 60 in connection with FIG. 3. The representation shown in FIG. 4 is a logical equivalent of the representation shown in FIG. 3. The representation 70 of FIG. 4 logically corresponds to that in FIG. 3 such that, for example, element 72 corresponds to the beginning cache slot as noted by the head of the queue pointer 64 in connection with the previously described figure. Similarly, the last element of the queue is denoted by slot 78 which in this example is labeled also as the tail of the queue. Elements or slots may be inserted into the list at the head of the queue and exit or leave the cache at the tail of the queue. For example, when an element is deposited into the cache, it may be placed at the head of the queue in slot location denoted by 72 in connection with a read operation. Additional elements may be progressively added to the head portion or other location within the queue 72. As elements are added to the queue, subsequent elements progress toward the tail of the list. When another slot is added to the replacement queue at position 72, the slot currently at position 72 moves to that slot designated as position 73 and the newly added element falls into the position of element 72.
An element may be placed in the queue, for example, when an element is referenced in connection with an I/O operation such as a cache miss for a read operation, or in connection with processing pending write operations, for example. Once in the queue, an element progresses through the queue from the head 72 towards the tail 78 of the queue.
Referring now to FIG. 8, shown is a more detailed representation of a tag 112 a as included in FIG. 7. The 2 byte tag 112 a includes an L-bit 92 and a 15 bit time stamp value 94. The L-bit, which may be the upper bit in the 2-byte tag arrangement, may be used to indicate the availability of a cache slot associated with the particular tag. This L-bit may be used in performing operations in which a processing step may be to obtain a cache slot. Associated processing operations are described in more detail elsewhere herein in following paragraphs. The time stamp value indicates, within a particular resolution, such as � second, when the associated slot was last used. For example, when there is a cache �hit� to a particular slot, the associated time stamp is updated with a new time stamp value.
Data may be stored in the cache in connection with performing data operations. Different processing steps may be performed using the cache in connection with performing different data operations. For example, when a read request is received from a host computer, a determination may be made as to whether the requested data is in the cache. If so, the data is returned. Otherwise, the data may be read from the particular data storage device, stored in the cache and then sent to the host system. A slot from the cache is determined in which to store the data. When a write operation is performed, an embodiment may store the data in the cache as a pending write which is actually written to a physical device, such as a disk, at some later point in time in accordance with system specific policies. After the data is written to the device, a cache slot may be made available for reuse. What will now be described are processing steps that may be performed in an embodiment in connection with cache management operations.
In one embodiment, data associated with a write I/O operation is first stored in cache and then later destaged or actually written out to the storage device. While the data is included in the cache but not yet written out to the device, the cache slot includes data associated with a write pending, and the slot is not available for replacement or reuse. It should be noted that a slot may be �reused� when the slot's existing data is invalidated by reuse of the slot for another subsequent purpose as described herein such as, for example, in connection with a subsequent I/O operation, or other nondata purpose. In the tag-based cache embodiment, a slot may be indicated as unavailable for reuse or replacement when the L-bit is set. In an embodiment using the cache structure of FIG. 3, for example, a slot that is unavailable for reuse or replacement may be removed from the cache chain structure. Once the data included in the cache slot has been destaged, an indicator in the cache slot may be set to indicate the slot as available for reuse, as described elsewhere herein in connection with the L-bit being cleared. When an embodiment makes a request for a cache slot such as in connection with processing a new I/O operation, an embodiment may select a cache slot from those indicated as available, such as selecting from those cache slots having an L-bit=0.
In one embodiment using the tag-based cache, a cache slot may be a candidate for reuse if the L-bit=0. Candidates (L-bit=0) may also be characterized as including meaningful user data, or, alternatively, nondata. In one embodiment, nondata candidate slots are not associated with user data on a data storage device and may be designated as a �free� slot with a unique or special timestamp value, such as zero. Candidate cache slots which are designated as data cache slots associated with user data on a data storage device may have a non-zero timestamp value to differentiate available data cache slots from the nondata available cache slots. Any cache slot have an L-bit=1 is not available for use or is otherwise not a current candidate for reuse.
An embodiment may use any one or more different criteria in connection with selection of an available cache slot. For example, an indicator, such as an L-bit setting, may be used in connection with a time stamp value to select a particular cache slot. Additionally, an embodiment may select a particular cache slot for use in accordance with whether the slot has been designated as a data cache slot or a nondata cache slot. Such an indication may be made using a bit flag or other indicator in a cache slot. For example, after data is destaged from a cache slot including data for a pending write, the cache slot may still include user data that can be used in connection with subsequent I/O operations. Even though the cache slot may be indicated as available for use, the data in the cache slot may be reused for a subsequent I/O operation. Thus, an embodiment may select a cache slot that does not contain data (or includes nondata) prior to selecting an available cache slot that includes data. Accordingly, an embodiment may have different policies in connection with selecting a particular cache slot from those possible cache slot candidates indicated as free or available for use. An embodiment may preferably select a slot which is both indicated as available, such as in accordance with an L-bit setting, and additionally is a nondata cache slot, as may be indicated using a unique or special timestamp value.
It should be noted that an embodiment may use cache slots for nondata or data uses that may vary in accordance with each embodiment. For example, in one embodiment, nondata cache slots may be used as scratch cache slots, for testing, in RAID processing, XRC (Extended Remote Copy), and the like. Such uses of cache slots may be characterized as nondata because the cache slots do not include valid user data associated with a data storage device that may be used in connection with subsequent I/O operations. When the nondata use of a cache slot completes, the slot does not contain any valid useful data for subsequent I/O operations. For example, in RAID processing, a cache slot includes parity information. Once such information is written out to a device, the data included in the cache slot is not associated with, for example, a valid portion of user data such as track of user data on a data storage device. In XRC processing, the cache slot may be used to store host commands. After the commands are processed, the cache slot does not include data that may be used in connection with subsequent I/O operations. Such cache slots as used in XRC and RAID operations are examples of nondata cache slots such that after the XRC and RAID operations are complete, the data storage system does not have any subsequent use for the information in the nondata cache slot(s).
An embodiment may also use one or more different cache management structures in addition to the shared cache that may be used by one or more directors as described herein in a data storage system. In one embodiment, each director may also manage and maintain for its own use a stack of private cache slots. It should be noted that the shared cache structure may be, for example, the cache structure previously described in connection with FIGS. 3 and 4 as well as the tag-based cache also described herein. In addition to the foregoing shared cache, each director may maintain what may be characterized as a private cache structure. In the embodiment that will be described in following paragraphs, the private cache structure used by each director is the stack of private cache slot pointers. The cache slots included in the private cache maintained by each director may include only nondata cache slots.
In the tag-based cache embodiment described in following paragraphs, the L-bit of a cache slot with a value of 1 indicates that the cache slot is not available to be allocated as a new slot. Cache slots may be designated as �unavailable� for one or more reasons that may vary with each embodiment. In the embodiment described herein, cache slots included in the private cache maintained by each director are considered �unavailable� and have the L-bit=1. Cache slots having an L-bit=1 may indicate an unavailable status for other reasons such as, for example, write pending data that has not been written out to a device.
Referring now to FIG. 9, shown is an example of an embodiment of a data structure 400 that may be used in connection with implementing a director's private cache. As illustrated in 400, an embodiment may use a stack data structure of private cache slots. The representation 400 includes an illustration of a stack data structure as known to those of ordinary skill in the art. Entries are pushed or placed on top of the stack such as entry 402 as indicated by arrow 404 a. An entry such as entry 402 may be removed as indicated by arrow 404 b by popping the top entry 402 from the stack.
In one embodiment, the stack structure 400 may be characterized as a logical representation of the private slot pointers maintained by each of the directors. A slot may be indicated as private in any one of a variety of different ways that may vary with each cache implementation. For example, each cache slot may indicated as private by setting a flag included in the flag word of the cache slot and associated control slot in the tag-based cache implementation described elsewhere herein. With the tag-based cache, those cache slots designated as private for each director may be included within the same physical and/or logical memory unit(s) as those cache slots within the shared cache for use by all of the directors with a particular indicator for which of those slots are private versus non private for use by a particular director. Additionally, an embodiment of the tag-based cache may include an identifier indicating which director or processor has designated this particular cache slot as private for its own use. The particular use of the director identifier and the private flag will be described in connection with processing steps in following paragraphs. In connection with the queue cache structure of FIGS. 3 and 4, pointers may be used to include each of the cache slots as an entry in the appropriate shared or private cache. Thus, in an embodiment using a linked list queue structure for the private and shared caches, a particular cache slot may be indicated as private for a particular director or as nonprivate for use by one or more directors in accordance with the particular data structure into which the cache slot is included utilizing, for example, pointers.
It should be noted that the foregoing are examples of data structures that may be used in connection with implementing the techniques described herein. One of ordinary skill in the art will appreciate that any one of a variety of different data structures as well as indicators may be used in connection with implementing the techniques described herein.
In an embodiment using the private stack structure per director and the shared cache, when a director determines it is able to make a slot free and available as a candidate for reuse, the director may take steps to possibly return the cache slot to its private stack rather than indicating the slot as a candidate for reuse by all directors. The director attempts to push the cache slot's pointer to the director's private stack. If the director is unable to successfully push a nondata cache slot to its private stack, or if a �sanity� check processing fails (i.e., the cache slot's flags are different than expected indicating an error condition needing recovery processing), the director then makes the slot available for reuse by other directors. In other words, an individual director keeps within its private stack a nondata cache slot if possible. In other instances, the director may return the cache slot for reuse to the shared cache for reuse by all directors.
It should be noted that a director may be unable to push a cache slot to its private stack, for example, if the stack is full. In one embodiment, each director maintains a stack of a predetermined size in accordance with the particular traffic and I/O operations tuned in accordance with a particular system. When the maximum number of stack entries has been exceeded, the director is unable to store any further cache slots within its private stack.
When there is a request to obtain a slot for storing data or nondata, a director looking for the cache slot first checks its private stack to determine if there are any cache slot entries within its stack. If so, the director pops the top entry from its stack for use. Otherwise, if there are no entries remaining in the director's private stack, the director attempts to locate and obtain an available cache slot from the shared cache using normal or regular cache management processing.
The particular size of the private cache structure of each director, such as the size of the stack, may be vary in accordance with the number and type of I/O operations, and the like, in an embodiment. In one embodiment, for example, the private stack maintained by each director may have 10 or 16 entries.
Referring now to FIG. 10, shown is a flowchart 450 summarizing the processing steps that may be used in one embodiment in connection with processing a request for a cache slot by a particular director. The steps of 450 may be performed by each director when a new cache slot is needed such as, for example, in connection with storing data for an I/O operation. At step 452, a director determines that a new cache slot is needed such as, for example, in connection with using a cache slot for a write or read operation (use as a data cache slot), or in connection with nondata operations and uses, such as for XRC and RAID operations, use as a scratch cache slot, and the like. At step 454, an attempt is made to pop or remove a top element from the director's private stack. It should be noted that step 454 may be unsuccessful for any one or more reasons that may vary in accordance with an embodiment such as, for example, the stack may be empty, the director may not be able to obtain a slot lock, or other unexpected conditions indicating an error (such as a slot not including properly set indicators designating this slot as �private�).
If step 454 is successful, processing proceeds to step 460 where the director pops the top element from its private stack of cache slots. At step 462, the cache slot removed from the private cache has its information updated as needed to maintain cache coherency and also in accordance with the particular use for which the new slot is being requested. The steps performed in connection with updating the cache slot vary in accordance with embodiment. For example, in one embodiment, the cache slot information update may include updating the cache slot's flags and other fields in accordance with whether the cache slot is being used as a data or nondata cache slot, updating shared cache management structures if the cache slot is being removed from the private stack and included in the shared cache, and the like. For example, if the new cache slot is associated with a write request and write pending data, the cache slot may not be included in the shared cache and may be initially indicated as unavailable (L-bit=1 in tag-based cache; not included in the cache structure for queue-based cache of FIG. 3). If the new cache slot is associated with a read request, the cache slot may be indicated as available (L-bit=0 in tag-based cache; included in the queue-based cache structure of FIG. 3) and also including valid user data (non-zero timestamp for a data cache slot). The cache slot is then returned to the caller of flowchart 450 processing as the new cache slot for use.
If, at step 454, the pop is unsuccessful, control proceeds to step 456 to perform normal or regular processing that may otherwise be performed in an embodiment to select an available slot candidate. In one embodiment, this may include selecting a candidate from those available slots (L-bit=0) with preference for selection to those nondata or free cache slots (zero timestamp) from the shared cache. Control proceeds to step 458 where the cache slot selected has its slot information updated as needed to maintain cache coherency and in accordance with the use for which the cache slot is being requested. Processing performed at step 458 in connection with maintaining cache coherency and updating the cache slot and related structures is similar to that as described in connection with step 462. The updated slot is then returned as part of step 458 processing. It should be noted that processing of step 456 may include what may be characterized as �normal� cache processing to select a cache slot from the shared cache in connection with an embodiment that does not utilize private caches for directors. Processing associated with step 456 that may be included in one embodiment is described elsewhere herein in more detail in connection with FIGS. 17-21.
In the foregoing processing steps of FIG. 10, the director requesting a cache slot first looks to its stack of private slots. If unable to obtain one from its private cache, the director proceeds with what may be characterized as �normal cache processing� to obtain an available cache slot from the shared cache that may be accessed by multiple directors.
Referring now to FIG. 11, shown is a flowchart 500 of processing steps that may be performed in an embodiment in connection with returning a cache slot for reuse to one of the existing logical cache structures. It should be noted that the processing steps of flowchart 500 of FIG. 11 summarize the processing steps previously described in connection with returning a slot to either the private stack of cache slots for reuse by a single director, or the shared cache used by multiple directors.
At step 502, a director has completed processing an existing cache slot and the existing cache slot may now be reused. In one embodiment, a cache slot may be returned to either the director's private cache or the shared cache for reuse in connection with any one or more different processing operations. For example, cache slot return processing as described in connection with FIG. 11 may be performed as part of a garbage collection process, post processing performed when an operation (such as a destaging operation) is complete, and the like. Once a director has completed processing of an existing cache slot, control proceeds to step 504 where a determination is made as to whether the cache slot is indicated as a data or nondata cache slot. If a determination is made that the cache slot is not a nondata cache slot, control proceeds to step 506 where the cache slot is returned to the shared cache. In the embodiment described herein, processing at step 506 includes updating the cache slot's information as needed to maintain cache coherency. For example, this may include updating the cache slot's information to indicate this slot as an available (L-bit=0) cache slot including data (non-zero timestamp).
If at step 504 it is determined that the cache slot to be returned has been designated as a nondata cache slot, control proceeds to step 508 where a determination is made as to whether the director's stack of private cache slots available for use is full. If the stack is full, control proceeds from step 508 to step 506 processing. Step 506 processing for this particular slot includes updating the cache slot's information to indicate that this cache slot is available (L-bit=0) and free including nondata (timestamp=0) so that the cache slot may be reused by any of the directors in the data storage system. Otherwise, if the existing directors' stack is not full, control proceeds to step 510 where the cache slot determined at step 502 is pushed to the top of the stack of private cache slots for that particular director. At step 510, the cache slot's information is also updated as needed to maintain cache coherency including, for example, indicating the slot as a private slot (L-bit=1, private flag or indicator set, zero timestamp indicating a nondata slot).
It should be noted that an embodiment may also test for different additional conditions in step 508 of FIG. 11 in determining whether a cache slot is added as to a director's private stack. For example, in one embodiment, a cache slot included in a memory unit indicated as disabled may not be added to the private stack. A logical or physical memory unit may be indicated as disabled, using a hardware setting for example, if one or more physical memory units are in the process of being removed. Setting a disable option or bit for the memory unit may be performed as a step prior to removal. Prior to physically removing a memory unit, processing steps may also be performed to remove or �drain� all cache slots included in this unit from each director's private stack. This may be done by returning to the shared cache each cache slot of the disabled unit to be removed that is included in a director's private stack. Prior to selecting a cache slot candidate for use, a director may first check to ensure that the cache slot is not included in a memory unit designated as disabled.
In the event that one of the directors fails with a non-empty private stack, the cache slots included in the failed director's private stack may be released. The detection of a failed director or processor may be performed by another director. In order to determine whether a particular director is dead or alive, an embodiment may use any one or more of a variety of different techniques. In one embodiment, each of the directors, including the DAs, and other directors within a data storage system, may update a particular location in global memory at predetermined time intervals. The foregoing may be characterized as a heartbeat of each of the different directors. In the event that a heartbeat is not detected for a first director as expected by a second director, the second director may conclude that the first director is in a dead state.
Referring now to FIG. 12, shown is an example representation 590 of a heartbeat table that may be stored in global memory on a data storage system. Each director, including DAs, RAs, and the like, may be associated with a unique row in the representation 590 as indicated by the director identifier in column 92. Each director may be expected to report or update the time stamp value in column 596 at each particular interval as indicated by column 594. The repeated recordation and reporting of the time stamp in column 596 at each of the particular time period intervals as indicated by column 594 may be characterized as the heartbeat of the associated director as indicated in column 592. In the event that the current time advances past the last time stamp value plus the interval for a first director, other directors in this embodiment assume that the first director is in the dead state. In addition to each director being expected to update or write its corresponding time stamp value into the global memory at pre-determined time intervals, each director also reads the values in the representation 590 in order to determine which other director may have entered the dead state. For example, as part of normal processing, each director may check the heartbeat status of every other director once a second, or other time period that may vary with each embodiment. In the event that a particular director detects that another director is dead because a heartbeat has not been received within an expected time period, that particular director may then update other values as may be maintained within the global memory to indicate that this particular director is now dead rather than alive. An embodiment may also include a status of which directors have been determined by another to be in the dead state.
In the event that each of the directors has the same time interval or heartbeat period within which a time stamp update is expected, the table 590 may omit the intervals 94.
It should be noted that, as will be appreciated by one of ordinary skill in the art, access to the heartbeat table or other structure is synchronized since it is accessed by multiple directors for modification. Any one of a variety of techniques that may vary with each embodiment may be used in performed this synchronization.
At predetermined time intervals, each director may update its own heartbeat information in the foregoing heartbeat table as well as read the heartbeat information of other directors to determine if any director is in a dead or unavailable state and return any of the dead director's private cache slot(s) to the shared cache. Once a director has been characterized as �dead�, the cache slots may be scanned to locate the dead director's private slots and also to locate and release other slots that were in use by the dead director.
Referring now to FIG. 13, shown is an illustration 600 of cache structures in one embodiment. The illustration 600 shows only some elements of the data storage system without interconnections as included in FIGS. 2A and 2B for the purposes of illustrating the caching technique. Included in 600 is a global memory 610, a tag-based cache structure 620 including the cache slots, a private stack 614 for the director DA1 630, and TOS (top of stack indicator) for 614. The particulars of 600 may be characterized as representing a snapshot of the state of the caching structures at a point in time. The structure 620 includes a control slot and multiple cache slots, such as 620 a-620 c. The private stack 614 in this example is an array of pointers to cache slots for DA1. The TOS indicator 612 is 2 indicating the current top of the stack. Each entry in 614, such as 614 a, identifies a corresponding entry in the cache structure 620 which is included in the private stack for the director DA1. Each cache slot included in the private stack 614, such as 620 a, may include an indicator designating that this cache slot is PRIVATE and for a specific director, such as DA1. In one embodiment, the PRIVATE indicator may be a bit included in a flag word and the director identifier, such as DA1, may be an alphanumeric or other unique identifier designating an owner director. The director identifier and PRIVATE designation may be stored in a portion of the cache slot and/or also included the control slot. As also described herein, a slot included in a private cache has the L-bit=1. Note that additional details about a tag-based cache are described elsewhere herein.
It should be noted that cache slots of 620 may be designated as being in one or more private stacks and also included in the shared cache available for use by all directors. Structure 620 may logically represent one or more physical memory units. In this example, all the structures are shown as being included in global memory. However, an embodiment may alternatively include some or all of the structures used by each director using other memory, such as may be local to a director.
In connection with accessing shared resources, such as portions of global memory including the cache structures described herein, even though not included in some processing steps herein, it will be appreciated by one of ordinary skill in the art that any one or more different techniques may be used to synchronize access among multiple directors. For example, an embodiment may have a per slot lock that is acquired and set prior to a director accessing a cache slot in a tag-based cache implementation. The locks are released when any necessary cache slot updates have completed. The particular techniques may vary with embodiment.
Referring now to FIG. 14, shown is an illustration 650 of cache structures in another embodiment. The global memory 660 includes a cache structure 670, a private stack structure 680 for director DA1 675, and a TOS 672 (similar to element 612 of FIG. 13). In this embodiment, those cache slots that are included in the private cache 680 may be connected by pointers as illustrated separate from those included in the structure 670. In this example, all the structures are shown as being included in global memory. However, an embodiment may alternatively include some or all of the structures used by each director using other memory, such as may be local to a director.
What will now be described are more detailed processing steps in connection with the pop and push operations of, respectively, FIGS. 10 and 11.
Referring now to FIG. 15, shown is a flowchart 700 of processing steps that may be performed in an embodiment as part of a push operation for pushing an element on the private stack of a director. The processing steps of FIG. 15 may be performed by each director using code included in, and executed by, each director. As known to those of ordinary skill in the art, the code may be stored in any one of a variety of different forms readable by a processor such as a processor of the director.
Prior to beginning execution of flowchart 700, it is assumed that any necessary lock(s) for the cache slot being pushed have been acquired and accordingly indicate the appropriate state. Additionally, the cache slot being pushed on the private stack is a nondata cache slot being returned for reuse. Thus, the director executing the steps of 700 has already acquired the cache slot and indicated the cache slot as unavailable for use by other directors, such as by setting the L-bit to 1. At step 702, a compare and swap instruction may be performed with the expected flags. If the flag word of the cache slot to be pushed has the expected flag word, it means that another director has not updated the cache slot since the expected flag word has last been obtained. This may be performed as a �sanity check� to detect the possible race condition that may occur when a cache slot is being accessed by multiple directors. An example of a particular race condition is described elsewhere herein. As also described elsewhere herein, the compare and swap in one embodiment may be used to perform a conditional update if a condition is true. In step 702, the private cache slot is indicated as private only if the flag word of the cache slot is the expected set of flag conditions. At step 704, a determination is made as to whether the compare and swap succeeded. If not, control proceeds to step 706 where error processing may be performed. The particular error processing may vary with each embodiment. Otherwise, if step 704 evaluates to yes, control proceeds to step 708 where the cache slot's director identifier is updated. At step 710, any private cache state variables, such as pointers, top of stack counters, and the like, may be updated.
Referring now to FIG. 16, shown is a flowchart 800 of processing steps that may be performed in an embodiment as part of a pop operation for popping or removing a top element on the private stack of a director. The processing steps of FIG. 16 may be performed by each director using code included in each director. As part of performing the processing steps of 800 in this example, it is assumed any necessary locks are acquired and released when modifying the data in the cache slot currently being popped or removed from the stack for use. The slot currently being popped should have an L-bit with a value of 1 indicating that the current slot is unavailable and is not part of the shared cache for use by another director, and a private indicator should be set from a previous push. Generally, the processing of flowchart 800 sets the appropriate indicators and updates the appropriate cache management structures, variables, and the like in accordance with the use of the cache slot request.
At step 802, private cache state variables may be updated. Step 802 modifies those variables used in maintaining the private cache structure, such as pointers and/or counters, in an embodiment as in step 810. At step 804, a determination is made as to whether the current cache slot is to be made available for reuse by all directors once the data for the associated I/O operation for which this pop is being performed has been placed in the cache slot. If so, control proceeds to step 806 where the cache slot is indicated as available for use and inclusion in the shared cache by setting the L-bit to 0. Control proceeds to step 808 where the appropriate flag bits are set to indicate the cache slot is not private. Step 808 may also include other processing as needed to maintain cache coherency and any cache management structures in an embodiment as also described elsewhere herein. Such processing may include, for example, updating other information about the cache slot (i.e., nondata cache slot having zero timestamp, data cache slot having nonzero timestamp), updating cache management structures (such as updating pointers used in managing the cache structures of FIG. 3), and the like.
If step 804 determines that the cache slot is not being made available for reuse by the directors and not returned to the shared cache, control proceeds to step 810 where the L-bit is set to indicate this cache slot as unavailable. Control proceeds to step 812 where additional cache slot information may be updated to maintain cache coherency and cache structures that may be included in an embodiment as also described elsewhere herein. For example, in one embodiment, the timestamp may be updated to the current timestamp and the cache flags set to indicate the cache slot as non-private.
It should be noted that if an I/O operation is a write operation, for example, the data may be placed into the cache slot, marked as write pending, and actually written out to the device at a later time. If the I/O operation is a write, the cache slot is not returned to the shared cache until a later undetermined time after the destaging of the write pending data. In connection with a cache slot request for write operation data, processing steps may be performed for which the condition at step 804 evaluates to NO, to not place the cache slot in the shared cache. If the operation is a read operation, in contrast to the write, the cache slot may be reused and is returned to the shared cache since there is no destaging of data. In connection with a cache slot request for read operation data, processing steps may be performed for which the condition at step 804 evaluates to YES to place the cache slot in the shared cache. The cache slot may be immediately available for reuse since there is no waiting for data in the cache slot to be destaged prior to reuse of the cache slot.
Note that in connection with performing the processing steps for modifying and accessing an element of the private cache, steps may be performed to attempt to acquire the necessary lock(s). In one embodiment, a cache slot lock may be obtained prior to modifying data included in the cache slot. The lock may then be released after modification is complete. It may be possible to have a race condition in which two writers are attempting to access and modify the same cache slot. This may cause a temporary locking error to occur. Subsequent attempts to acquire the lock may succeed. Thus, an embodiment may perform a specified number of retries in the event that a locking failure occurs. The following illustrates an example of when such as condition may occur for directors A and B.
Reads extent X including slot S with L-bit = 0
Locks S, updates L-bit = 1
return S by pushing to B's private stack, releases lock
looking for available slots in cache and locks S as part of finding a new available slot processing (failed to obtain slot from private cache and use �normal� processing to locate an available slot).
tries to lock slot S as part of pop processing and fails to lock
finds mismatch with compare and swap with L-bit value and releases lock on S
In an embodiment with a specified number of retries, B retries to acquire lock on S and now succeeds. Otherwise, if no specified number of retries, the �pop failed� from step B1 causes processing to continue with �normal� processing to locate an available slot.
Although the foregoing processing steps for the race condition and additional detail processing for a push operation (FIG. 15) and pop operation (FIG. 16) are described with reference to the tag-based cache, one of ordinary skill in the art will appreciate that these steps may be adapted and performed in connection with other cache implementations.
What will now be described in FIGS. 17-20 are processing steps that may be performed in an embodiment in connection with obtaining a cache slot from the shared cache in connection with what is characterized as �normal processing� to obtain a cache slot as in step 456 of FIG. 10 when there are no available slots in the private stack as may be maintained for each director included in the data storage system. Note that an L-bit setting of 1 indicates that a cache slot is not available for use and may mean that the cache slot is included in the private cache of a director or is not included in a private cache but is otherwise in use. It should be noted that the processing of FIGS. 17-20 set forth herein is also described in pending U.S. patent application Ser. No. 10/080,321 filed Feb. 21, 2002, entitled CACHE MANAGEMENT VIA STATISTICALLY ADJUSTED TIME STAMP QUEUE, which is incorporated by reference herein.
Referring now to FIG. 17, shown is a flowchart of steps of an embodiment for obtaining a slot from the cache. Generally, the technique searches for an available slot or displaces the oldest slot. These steps may be performed by each DA or other processor, for example, within a system such as described in connection with FIG. 2A. It should be noted that although these processing steps are described with reference to a tag-based cache, one of ordinary skill in the art will appreciate that these processing steps may be extended for use in connection with other cache implementations, such as the embodiment of the cache structure described elsewhere herein FIGS. 3 and 4.
Additionally, when there are multiple processors each attempting to locate an available slot, techniques may be used in connection with determining the next subsequent extent of tags for each processor in order to minimize clustering. In other words, techniques may be used such that each processor attempts to locate an available slot from different extents of tags to minimize the likelihood that a first and a second processor look in the same extent of tags. Accordingly, these techniques may also minimize the likelihood that any two processors may be attempting to access the same available slot.
Referring now to FIG. 18, shown is a flowchart 250 of processing steps performed in connection with the FIND_SLOT routine. At step 252, ptr is assigned to point to the first tag in the current extent of tags. Additionally, the num_swap_fails tracking variable is initialized to 0. num_swap_fails counts the number of failed swaps as described in following paragraphs. At step 254, a determination is made as to whether num_swap_fails exceeds a predetermined maximum. In one embodiment, MAX_FAILS may be 4. Other embodiments may have other values for MAX_FAILS that may vary from that described herein. It should be noted that each DA, director or processor has its own unique pointer (ptr) such that each DA, for example, may attempt to obtain a slot from locations different than that of other DAs. If a determination is made at step 254 that the maximum number of failed swap attempts has been exceeded, control proceeds to step 266 where failure is returned. Otherwise, control proceeds to step 256.
At step 256, a determination is made as to whether processing is complete for all tags in this extent. If so, control proceeds to step 300 in FIG. 20 where a determination is made as to whether there is an �oldest� slot. If so, this slot is used as the available slot, as in step 304, and control proceeds to step 260. Otherwise, control proceeds to step 302 where failure is returned.
then FAIL and unlock shared resource else /*SUCCESS*/ swap in new values as in step 260 unlock shared resource The foregoing may be implemented used different mechanisms and techniques included in a system for providing exclusive access to a shared resource, such as the shared memory used as the cache in this instance.
If a determination is made at step 258 that the current tag is not free, control proceeds to step 280 which is continued in FIG. 19. At step 280, the current time stamp is updated and the temporary variable age is assigned the current tag's time stamp value. It should be noted that the processing step of updating the current time stamp may be performed in any one of a variety of different increment units. For example, in one embodiment, current time stamp may be updated in increments of 4 units. In this example, multiple processors may be using the same cache in which each of the processors has its own clock and associated time used in connection with time stamps. Each of the processor clocks may have time synchronization differences such that at a particular point in time, time stamps produced by any two of the clocks may differ. A time stamp increment, such as 4 units, may be selected in accordance with any such synchronization differences when comparing or using time stamp values as in processing herein. In one embodiment, the increment is 4 units=2 seconds, each unit being � second. This increment amount may vary in accordance with embodiment.
In particular, an embodiment may adjust the time stamp value of an associated slot in accordance with the Fall Through Time (FTT). Generally, the FTT refers to the average amount of time it takes for an unpromoted slot once it is in the queue to exit the queue. In other words, it is the average amount of time it takes a slot to pass through or �fall� through the queue from the head position and then exit out of the queue through the tail position, for example, referencing the illustration of FIG. 4. A slot may be added to the head position or at another position in accordance with the relative time stamps of those in the queue. The FTT is described in issued U.S. Pat. No. 5,592,432, Vishlitzky et al, which is incorporated herein by reference, and in pending U.S. patent application Ser. No. 10/853,035, filed May 25, 2004, entitled CACHE FALL THROUGH TIME ESTIMATION, which is incorporated by reference herein.
for I = 1 to max for all processors { current_proc_id = identifier of processor I; initial_extent_value_processor_pointer[I] = (number of extents in all banks * current_proc_id)/ (max number of processors) I = I + 1 } where I is an index over the range of all processors and each processor has an associated unique processor identifier. The initial value of a starting extent for each processor is selected in accordance with the unique processor identifier. In this embodiment, the memory may be organized into banks and number of extents in all banks refers to the total number of extents in all of the memory banks. As described elsewhere herein, each memory bank may include a particular number of extents that may vary in accordance with each embodiment. Another embodiment may use the processor identifier in connection with determining a random number used in selecting an initial value for each processor's starting extent.
The various parameters, such as the number of extent increments, �n�, the director or processor identifiers, the particular array element associated with each processor or director, and the like may be specified as part of initialization processing. Values for these parameters may be specified, for example, as part of system configuration data which is read upon initializing the data storage system, such as the Symmetrix data storage system. Similarly, values for these parameters may also be modified by updating the system configuration file and reloading the data stored therein, or through other utilities that may be included in an embodiment, such as a utility providing for dynamic updating of parameter values which may or may not modify the definitions stored within a configuration file. The particular techniques used in connection with specifying and/or modifying values described herein may vary in accordance with each embodiment.
Referring now to FIG. 21, shown is an example representation of a tag-based cache structure comprising multiple memory banks or units. In one embodiment, the cache slots in each memory bank may be referenced sequentially so that the memory bank boundaries and mapping of a cache slot number to a particular physical cache slot is transparent to the director. Included in 450 are n-memory banks represented a continuum of cache slots from 1 to MM. An embodiment may map a logical cache slot number, such as 100, to a particular bank number and cache slot within the bank, such as cache slot 1 in bank 2, in order to access and modify the necessary portions in performing the processing steps described herein.
Patentzitate Zitiertes PatentEingetragen Ver�ffentlichungsdatum Antragsteller TitelUS520693924. Sept. 199027. Apr. 1993Emc CorporationSystem and method for disk mapping and data retrievalUS53815394. Juni 199210. Jan. 1995Emc CorporationSystem and method for dynamically controlling cache managementUS55924325. Sept. 19957. Jan. 1997Emc CorpCache management system using time stamping for replacement queueUS577839423. Dez. 19967. Juli 1998Emc CorporationSpace reclamation system and method for use in connection with tape logging systemUS584514719. M�rz 19961. Dez. 1998Emc CorporationSingle lock command for an I/O storage system that performs both locking and I/O data operationUS585720831. Mai 19965. Jan. 1999Emc CorporationMethod and apparatus for performing point in time backup operation in a computer systemUS7136883 *9. Sept. 200214. Nov. 2006Siemens Medial Solutions Health Services CorporationSystem for managing object storage and retrieval in partitioned storage mediaUS20020032844 *25. Juli 200114. M�rz 2002West Karlon K.Distributed shared memory managementUS20020046324 *8. Juni 200118. Apr. 2002Barroso Luiz AndreScalable architecture based on single-chip multiprocessingUS20030140209 *13. Aug. 200224. Juli 2003Richard TestardiFast path caching* Vom Pr�fer zitiertNichtpatentzitateReferenz1U.S. Appl. No. 09/535,134, filed Mar. 24, 2000, Lambright, et al.2U.S. Appl. No. 10/080,321, filed Feb. 21, 2002, Ezra, et al.3U.S. Appl. No. 10/853,035, filed May 25, 2004, Levin-Michael et al. Referenziert von Zitiert von PatentEingetragen Ver�ffentlichungsdatum Antragsteller TitelUS20120254548 *4. Apr. 20114. Okt. 2012International Business Machines CorporationAllocating cache for use as a dedicated local storage* Vom Pr�fer zitiertKlassifizierungen US-Klassifikation711/130, 711/121, 711/118, 711/119, 711/123Internationale KlassifikationG06F12/02 UnternehmensklassifikationG06F12/084 Europ�ische KlassifikationG06F12/08B4SJuristische Ereignisse DatumCodeEreignisBeschreibung30. Sept. 2004ASAssignmentFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:EZRA, JOSEF;OFER, ADI;REEL/FRAME:015862/0808Owner name: EMC CORPORATION, MASSACHUSETTSEffective date: 20040928DrehenOriginalbildGoogle-Startseite - Sitemap - USPTO-Bulk-Downloads - Datenschutzerkl�rung - Nutzungsbedingungen - �ber Google Patente - Feedback gebenDaten bereitgestellt von IFI CLAIMS Patent Services.© 2012 Google