Source: https://patents.google.com/patent/CN108351832A/en
Timestamp: 2019-12-10 04:46:38
Document Index: 414343814

Matched Legal Cases: ['art 356', 'arts 366', 'arts 376', 'art 444', 'art 454', 'arts 516', 'arts 6', 'art) 1230']

CN108351832A - Realize the hardware accelerator for storing write cache management - Google Patents
Realize the hardware accelerator for storing write cache management Download PDF
CN108351832A
CN108351832A CN201680065411.6A CN201680065411A CN108351832A CN 108351832 A CN108351832 A CN 108351832A CN 201680065411 A CN201680065411 A CN 201680065411A CN 108351832 A CN108351832 A CN 108351832A
CN201680065411.6A
R.加尔布雷思
D.莫特尔
A.格哈德
J.爱德华兹
R.韦克沃斯
B.巴克
G.拉达克里什南
2015-11-09 Priority to US62/252,749 priority
2016-11-07 Application filed by 国际商业机器公司 filed Critical 国际商业机器公司
2016-11-07 Priority to PCT/IB2016/056677 priority patent/WO2017081593A1/en
2018-07-31 Publication of CN108351832A publication Critical patent/CN108351832A/en
Provide the design structure resided over for realizing the method and controller and main body controller circuit of enhancing storage adapter write cache management.The controller includes hardware write cache engine, is realized for storing the hardware-accelerated of write cache management.The controller manages write cache data and metadata in a manner of minimum or do not need firmware participation, substantially increases performance.
Realize the hardware accelerator for storing write cache management
Present invention relates in general to data processing fields, and more particularly relate to realize that enhancing storage adapter is write Enter the method and the resident design structure thereon of controller and main body controller circuit of cache management.
Storage adapter is used to mainframe computer system being connected to Peripheral storage I/O equipment, such as hard disk drive, and solid-state is driven Dynamic device, tape drive, CD drive etc..Currently, mainframe computer system is connected to storage and fitted by various High Speed System interconnection Orchestration, and storage adapter is connected to storage I/O equipment, such as quick peripheral component interconnection (PCIe), Serial Attached SCSI (SAS) (SAS), optical-fibre channel and InfiniBand.
Storage adapter and storage subsystem generally comprise write cache to improve performance.Write cache is usual It is non-volatile, for example, using the DRAM for supporting flash memory, is used for masking redundancy array of independent disks (RAID) (such as RAID-5 And RAID-6) introduce write-in loss.Write cache can also be grasped by the multiple main frames that will be placed in write-in caching It is merged into single (destage) operation that degrades, is then handled by RAID layers and disk unit to improve storage performance.For Write cache data and catalogue or metadata, can be mirrored to second or double adapters, in adapter by redundancy It is advantageously utilized in the case of failure.
The complete association for cache users or host data or other complicated caches in storage subsystem In, it usually needs many cpu cycles update required metadata to place data into write cache.This metadata includes height Fast cache lines (CL) structure, for hash (hash) table of fast search and for searching the least recently used of oldest data (LRU) queue.Metadata be used for being consistent property, make cache directory keep it is non-volatile, and enable searching data from It is deleted in cache.
Need a kind of effective ways and controller of the storage write cache management for realizing enhancing.It needs to provide Such method and controller provide hardware-accelerated design, including hardware write cache engine, with minimum or do not have Firmware participates in managing write cache data and metadata to greatly improve performance.
As used in following described and claimed, term controller and controller circuitry should be by broadly It is interpreted as including input/output (IO) adapter (IOA) and including connection mainframe computer system and peripheral storage I/O equipment Various arrangements IO RAID adapters, peripheral storage I/O equipment includes hard disk drive, solid state drive, magnetic tape drive Device, CD drive etc..
As used in following described and claimed, term：Firmware is not used substantially, and there is minimum, It is minimum or participated in without firmware, it should to be broadly understood to include the operation executed by hardware and function, and may include The operation or control of firmware are transmitted or passed to from firmware.
The main aspect of the present invention is will to provide a kind of write cache management enhanced for realizing storage adapter Method and the resident design structure thereon of controller and main body controller circuit.The present invention other importances be Such method that there is no negative effect, controller and design structure will be provided, and overcome prior art arrangement Many disadvantages.
In brief, the method and controller for realizing enhancing storage adapter write cache management are provided, And the resident design structure thereon of main body controller circuit.The controller includes hardware write cache engine, in fact It is current in storage write cache management it is hardware-accelerated.The controller with it is minimum or do not need firmware participation write to manage Enter cached data and metadata, substantially increases performance.
The present invention can be from following to preferred reality of the invention shown in attached drawing together with above and other objects and advantages It applies and is best understood in the detailed description of example, wherein：
Figure 1A and Figure 1B is to be shown respectively to enhance write-in height according to having for preferred embodiment for realizing storage adapter The schematic diagram and block diagram of the example system of the dual controller of fast cache management and input/output (IO) adapter.
Fig. 2A and 2B is respectively illustrated to be enhanced according to the storage adapter for realizing Figure 1A and Figure 1B of preferred embodiment Example logic block address (LBA) included by the controller of write cache management converts RAID storehouses, in fig. 2, write-in Cache is placed in high-order RAID storehouses, at Dynamic Tier (Easy Tier), Striping data and odd even band Change on (RAID) layer, cache is executed on the basis of array ID/ array LBA.
Fig. 3 shows the example read operation according to the hardware supported of preferred embodiment, including is adapted to for realizing storage Device enhances example logic block address (LBA) conversion of Fig. 2A and 2B included by the controller of write cache management.
Fig. 4 shows the example write operation according to the hardware supported of preferred embodiment, including for realizing Figure 1A and figure The example logic block address of Fig. 2A and 2B included by the controller of the storage adapter enhancing write cache management of 1B (LBA) it converts, write cache is placed in high-order RAID storehouses, in dynamic layer (Dynamic Tier) (Easy Tier), on Striping data and odd even striping (RAID) layer.
Fig. 5 shows the example degraded operation according to the hardware supported of preferred embodiment, including is adapted to for realizing storage Device enhances example logic block address (LBA) conversion of Fig. 2A and 2B included by the controller of write cache management.
Fig. 6 shows the control for enhancing write cache management for realizing storage adapter according to preferred embodiment Example cache line (CL) structure in control storage (CS) included by device.
Fig. 7 shows the control for enhancing write cache management for realizing storage adapter according to preferred embodiment Example cache row (CL) structure in the control storage (CS) for storing (DS) for automatic mirror-image to data included by device.
Fig. 8 shows the control for enhancing write cache management for realizing storage adapter according to preferred embodiment Example cache row (CL) structure in data storage (DS) included by device.
Fig. 9 shows the control for enhancing write cache management for realizing storage adapter according to preferred embodiment Example local SAS and long-range SAS cache lines (CL) control storage (CS) mirror image or SAS mirror images included by device are deleted to number According to storage (DS).
Figure 10 A and 10B are shown enhances write cache pipe according to preferred embodiment for realizing storage adapter The example transactions in cache line (CL) state in hardware included by the controller of reason.
Figure 11 A and Figure 11 B are shown enhances write cache according to preferred embodiment for realizing storage adapter Example cache row (CL) chain with identical array ID/LBA in hardware included by the controller of management.
Figure 12 A, 12B, 12C and 12D are shown respectively to be enhanced for realizing storage adapter according to preferred embodiment and be written It is used to be written included by the controller of cache management, unoptimizable or interim write-in, the hardware of reading and degraded operation The exemplary operations chain based on controller chassis (CB) in engine.
Figure 13 is the control for enhancing write cache management for realizing storage adapter shown according to preferred embodiment The flow chart of the exemplary operations of ablation process included by device processed.
Figure 14 shows the control for enhancing write cache management for realizing storage adapter according to preferred embodiment The example XOR on the no write de-lay with superposition included by device processed merges.
Figure 15 is the control for enhancing write cache management for realizing storage adapter shown according to preferred embodiment The flow chart of the exemplary operations of reading process included by device processed.
Figure 16 is the control for enhancing write cache management for realizing storage adapter shown according to preferred embodiment The flow chart of the exemplary operations of degradation structure processing included by device processed.
Figure 17 is the control for enhancing write cache management for realizing storage adapter shown according to preferred embodiment The flow chart of the exemplary operations of the single processing that degrades included by device processed.
Figure 18 is the control for enhancing write cache management for realizing storage adapter shown according to preferred embodiment The flow chart of the exemplary operations of multiple degradations processing included by device processed.
Figure 19 is the control for enhancing write cache management for realizing storage adapter shown according to preferred embodiment The flow chart of the exemplary operations around (BYPASS) processing included by device processed.
Figure 20 shows the control for enhancing write cache management for realizing storage adapter according to preferred embodiment Removing hardware auxiliary tool packet of the example based on register included by device processed.
Figure 21 and Figure 22 is to show to enhance write cache pipe according to preferred embodiment for realizing storage adapter The flow chart of the exemplary operations for removing engine processing included by the controller of reason.
Figure 23 is shown enhances the control of write cache management according to preferred embodiment for implementing storage adapter Exemplary hardware counter included by device processed and statistics.
Figure 24 and Figure 25 is shown enhances write cache pipe according to preferred embodiment for realizing storage adapter Example CL limitations included by the controller of reason and threshold value.
Figure 26 is the control for enhancing write cache management for realizing storage adapter shown according to preferred embodiment The flow chart of the exemplary operations of clobal processing included by device processed.
Figure 27 is the control for enhancing write cache management for realizing storage adapter shown according to preferred embodiment The flow chart of the exemplary step of every ARRAY PROCESSING included by device processed.
Figure 28 and Figure 29, which respectively illustrates to enhance to be written for realizing storage adapter according to preferred embodiment, to be delayed at a high speed Deposit the example global trigger included by the controller of management and every array trigger；And
Figure 30 is designed in semiconductor, the flow chart of the design process used in manufacture and/or test.
The embodiment of the present invention it is described in detail below in, refer to the attached drawing, attached drawing show can implement the present invention Example embodiment.It should be understood that using other embodiment and structure change can be carried out without departing from the present invention Range.
Term used herein is used only for the purpose of describing specific embodiments, and is not intended to limit the present invention.Such as this paper institutes It uses, unless the context clearly indicates otherwise, otherwise singulative " one ", "one" and "the" are intended to also include plural shape Formula.It will be further appreciated that when used in this manual, specified the stated spy of term " include " and or " include " Sign, entirety, step, operation, the presence of element and/or component, but do not preclude the presence or addition of other one or more features, whole Body, step, operation, component, assembly unit and/or a combination thereof.
Feature according to the present invention provides a kind of side enhancing write cache management for realizing storage adapter Method and controller and the resident design structure thereon of main body controller circuit.The method and controller of the present invention includes pipe Reason write cache and the newer write cache hardware engine of ground atom that cache directory is provided.Use write-in Cache hardware engine, which is executed, both to be written to write cache and is read from write cache, without the use of firmware, Hardware engine is linked together.
Referring now to the drawings, Figure 1A and 1B, which is respectively illustrated, enhances according to preferred embodiment for realizing storage adapter Write cache management by reference character 100 is totally specified there is dual controller and is referred generally to by reference character 102 The example system of fixed input/output (IO) adapter.System 100 includes that system 100 includes with the write-in for being mirrored to double IOA The first input/output adapter (IOA) or controller 102 of cached data and catalogue or metadata, IOA#1 and second are defeated Enter/o adapter (IOA) or controller 102, IOA#2, this can be used in the case of adapter fails.System 100 is wrapped The host system 104 with IOA#1 and #2 controllers 102 is included, IOA#1 and #2 controllers 102 are via peripheral component interconnection high speed (PCIE) bus is directly coupled to host system 104 or is optionally coupled to host system 104 via PCIE interchangers 106.System System 100 include such as hard disk drive (HDDs) or rotating driver 108 multiple storage devices 108 and including for example independently The solid state drive (SSD) 108 of the redundant array (RAID) of driver, via with IOA#1 and #2, between controller 102 SAS connections one or more Serial Attached SCSI (SAS) connection, be coupled to alternately through corresponding expander 110 IOA#1 and #2, controller 102.
In fig. ib, the IOA#1 and #2 in the data-storage system 100 according to preferred embodiment, controller 102 are shown Other exemplary components of each.Controller 102 includes one or more processors or central processor unit (CPUs) 120, include at least the write cache 122 of data storage (DS) 124, and such as dynamic random access memory (DRAM) Control store (CS) 126.Controller 102 includes flash memory 128 and nonvolatile RAM (NVRAM) 130.
According to preferred embodiment, controller 102 includes write cache engine 132.Controller 102 includes via PCIE Bus is connected to quick (PCIE) interface 134 of peripheral component interconnection of host system and is connected to the every of multiple storage devices 108 One Serial Attached SCSI (SAS) controls (SAS CTLR) 136.
IOA#1 and #2, controller 102 and write cache engine 132 comprising multiple hardware engines, such as It realizes the method for linking together hardware engine, is write without using firmware to execute to be written or cover in write cache Enter；And for realizing the method for linking together hardware engine, the reading from write cache is executed without using firmware It takes.
Feature according to the present invention, including enhancing for realizing storage adapter for write cache engine 132 are written The method and controller 102 of cache management generate operational chain using different hardware engine, to execute writing without using firmware Enter or cover to be written in write cache.Controller 102 and write cache engine 132 by execute following steps come Realize this method：Non-volatile (NV) buffer is distributed, for the direct memory access (DMA) from host dma data, is determined In more new cache line (CL) state, if covered the available data in cache, started in write operation Merging non-4K data at the end of, DMA data also updates the CL in local and remote NV memories to remote adapter, The CL for covering data in local and remote NV memories is removed, final updated to CL states is cancelled and is used for any overlapping number According to NV Buffer allocations, and to Host Command send respond.
Feature according to the present invention, including enhancing for realizing storage adapter for write cache engine 132 are written The method and controller 102 of cache management generate operational chain using different hardware engine, high from write-in not use firmware Speed caching executes reading.Controller 102 and write cache engine 132 execute this method by executing following steps：Point With Vol buffering areas, search, which is read, in write cache hits, and to host dma data, redistributes Vol buffering areas, finally CL states are updated, are that any data concomitantly to degrade in the slave cache being just read redistribute NV buffering areas, with And it sends and responds to Host Command.
Feature according to the present invention, including enhancing for realizing storage adapter for write cache engine 132 are written The method and controller 102 of cache management generate operational chain using different hardware engine, high from write-in not use firmware Speed caching execution part reads hit.Controller 102 and write cache engine 132 execute this by executing following steps Method：Vol buffers are distributed, search, which is read, in write cache hits, and part is used for from HDDs/SSDs readings/merging The data for reading hit deallocate Vol buffering areas to host dma data, and final updated is just read to CL states Any data concomitantly to degrade from cache cancel distribution NV buffering areas, and send and respond to Host Command.
Feature according to the present invention, including enhancing for realizing storage adapter for write cache engine 132 are written The method and controller 102 of cache management generate operational chain, to use minimum firmware from write-in using different hardware engine Cache executes degraded operation.Controller 102 and write cache engine 132 help to generate the degradation behaviour with option Make, to start at LRU or specified array LBA, and with the end of size/span of largest request or 256K band boundaries End stops.
Feature according to the present invention, including enhancing for realizing storage adapter for write cache engine 132 are written The method and controller 102 of cache management realize that be allowed for the host of identical CLs to read degrades together with cache A group policy of Shi Fasheng.The conflict of generation is by hardware queue and assigns.
Feature according to the present invention, the method for realizing the enhanced write-in cache management of storage adapter and control Device 102 includes write cache engine 132, uses operation structure number (Op Build Number) or sequence in (CL) Row number with can ensure correctly identify for identical array logic block address (LBA) cache in relatively legacy data and compared with New data.
Feature according to the present invention, including enhancing for realizing storage adapter for write cache engine 132 are written The method and controller 102 of cache management are realized for being written, read and complete or cancel CL updates with degraded operation The middle hardware used is closed.
Feature according to the present invention, the method for realizing the enhanced write-in cache management of storage adapter and control Device 102 includes write cache engine 132, which realizes for identification with update for will be by The unique hardware engine of the CLs for the array LBA range removed from cache.
Feature according to the present invention, the method for realizing the enhanced write-in cache management of storage adapter and control Device 102 includes that write cache engine 132 realizes XOR (exclusive or) pooling function, such as is carrying out no write de-lay using superposition When.
Feature according to the present invention, including enhancing for realizing storage adapter for write cache engine 132 are written The method and controller 102 of cache management realize the method based on trigger with dynamic optimization Cache degradation rate And adjust the threshold value of data and limitation in the cache for each array, the input (register) and defeated provided using hardware Go out (register and event) to assist firmware.
Feature according to the present invention, including enhancing for realizing storage adapter for write cache engine 132 are written The method and controller 102 of cache management realize the high level architecture of hardware-accelerated design, are participated in or are not had with minimum firmware There is firmware to participate in, to manage write cache data and metadata (catalogue), greatly improving performance.
Feature according to the present invention, including enhancing for realizing storage adapter for write cache engine 132 are written The method and controller 102 of cache management, provide CLs (cache line), Hash table and hard per array LRU queues Part manipulates.This be completed with pseudo- atomic way, or to remain unchanged to the update of these structures or to execute every time It is all updated completely when hardware engine.For example, all entries just changed all are checked out into caching, change is primary or more It is secondary, the then check-in (Checked-In) suddenly if success.Only CL keeps non-volatile, and adapter fails are occurring When can change the subset of CL.
Feature according to the present invention, the method for realizing the enhanced write-in cache management of storage adapter and control Device 102 includes write cache engine 132, which includes allowing the CL for tracking following state fixed Justice：During covering carries out (OIP), covering operation address period is being used；It is read during read operation in carrying out (RIP)；And degradation During operation in degradation (destage) operation (DIP).Allow RIP and DIP simultaneously to reduce collision, this is in traditional design It is impossible.
Feature according to the present invention, the method for realizing the enhanced write-in cache management of storage adapter and control Device 102 realizes write-in, unoptimizable or interim write-in, reading and degraded operation including write cache engine 132, provides enhancing Performance.
It is as shown in Figure 2 A for realizing the enhanced write-in cache management of storage adapter referring now to Fig. 2A and 2B Controller 102 realize the example logic block address (LBA) usually specified by reference character 200 and translate RAID storehouses.Fig. 2 B Show the master for (VSet) the volume collection (that is, the logical device that can be read and write by host) usually specified by reference character 220 Machine LBA conversions.
In fig. 2, controller 102 realizes host LBA and AFDASD (the advanced work(being used together with IBM i operating systems Energy DASD) conversion 202.AFDASD represents one and can be set by the physics for the support RAID that IBM i operating systems read and write It is standby.Host LBA and AFDASD conversion 202 is carried with starting in the AFDASD of frame 204 to array or reverse data striping 206 For array LBA and array ID, array LBA and array ID is provided in 210 read/writes caching 208.By read/write cache 208 The high position being placed in RAID storehouses 200, and higher than on (RAID) layer data striping 212, provide logical device LBAs With LDNs (logic device number) 214 and odd even band 216, physical equipment LBA and LDN 218 is provided, wherein according to preferably Embodiment executes cache on the basis of array ID/ array LBA.Logical block address (LBA) is converted RAID storehouses 200 and is dropped Low complexity, and ratio previously reduced the existing design that write cache is placed under dynamic layer and Striping data Delay, wherein completing cache on the basis of equipment LBA.
Fig. 2 B show the RAID storehouses 220 of the host LBA conversions with VSet 222, are started with VSet to array 224, 226 provide array LBA and array ID, and array LBA and array ID, VSet are provided with read/write cache 228 230 222 execute (AIX) with advanced interactive, and Unix, Linus are used together with other common operating systems.RAID storehouses 220 include The dynamic layer for remapping array LBA (s)/ID is provided at 234 or simple layer remaps 232, followed by Striping data 212 Common code (RAID) layer, provide logical device LBAs and LDNs (logic device number) 214 and provide physical equipment LBA and The parity stripization 216 of LDN 218.
Feature according to the present invention, a kind of method and control enhancing write cache management for realizing storage adapter Device 102 processed, including write cache engine 132 realize that automatic reading to be written to cache, is usually joined in no firmware With in the case of or until storage device 108, executes complete or part from write cache and reads hit.
With reference to figure 3,4 and 5, read there is illustrated illustrative, write-in and degraded operation, wherein having in frame turning The frame of point indicates that hardware HW passes control to firmware FW with the position of processing operation.
Referring now to Figure 3, there is illustrated specified by reference character 300 according to the usual of hardware supported of preferred embodiment Example read operation, the hardware include Fig. 2A and Fig. 2 B respective examples logical block address (LBA) conversion RAID storehouses, quilt Included in the controller 102 for enhancing write cache management for implementing storage adapter.For host LBA and AFDASD RAID storehouses show to read LBA length at frame 302,304, and show to skip mask in frame 306,308.As shown, FDDASD provides the data strip frame for being applied to write-in/reading cache 328 to array or reverse data striping frame 310 312、314、316、318、320、324、326.For host LBA and Vset RAID storehouses, LBA length is read in frame 330,332 Place is shown, is skipped mask and is shown more than 32 in frame 334,336 and at LBA 338.VSet provides unbiased to array frame 340 Frame 342 is moved, the past of non-zero offset frame 344, Vset frames 346 and is applied to being less than for write-in/reading cache 328 at end 256K length frame 348.As indicated, write cache 328 provides whole hits, 4K alignment 350 all hits, skips 352, All hit rather than 4K are aligned beginning and end 354, and all hit rather than 4K alignment start 356, and all hits, without It is that 4K alignment terminates 358.As shown, write-in read cache 328 provide recklessly in, 4K alignment 360, recklessly in, jump Cross 362, recklessly in, non-4K is aligned beginning and end 364, recklessly in, non-4K alignment starts 366, and recklessly in, non-4K alignment Terminate 368.As shown, write-in reads cache 328 and provides partial hit, and 4K alignment 370, partial hit is skipped 372, partial hit, the beginning and end 374 of non-4K alignment, partial hit, non-4K alignment starts 376 and partial hit, non- 4K alignment terminates 378.
As shown, write-in reads cache 328 and provides ongoing reading (RIP) at frame 380, ongoing Mirror image (MIP) and ongoing superposition (OIP) provide 65 cache lines (CLs) at frame 382, are less than 4K in frame 384 Partial hit is provided in CL and partial hit chicken switchings are provided at frame 386, to adapt to abnormal conditions.Such as without high speed It shown by the frame 388 of the frame 387 of caching and not dynamic layer, reads and enters Striping data frame 389, data strip is provided Band frame is as shown in 312,314,316,318,320,322,324,326,327 and 329.Otherwise, dynamic layer frame 390 frame 391 not It provides and exchanges, provided in frame 392 and exchange identical redundancy group (RG), do not provided in frame 393 and exchange different RG, carried in frame 394 Halved belt is provided for afoot exchange, and/or in frame 395.Data strip frame 389 is provided in frame 396 intersects subarray, And/or the driver of the exposure of hit is provided in frame 398.
Feature according to the present invention, including enhancing for realizing storage adapter for write cache engine 132 are written The method and controller 102 of cache management realize that is supported in hardware automatically writes operation, wherein the hardware is not using Same controller chassis or hardware engine generates operational chain and is written or covers write-in usually to be executed in the case where no firmware participates in To said write cache.
Referring now to Figure 4, there is illustrated specified by reference character 400 according to the usual of hardware supported of preferred embodiment Example write operation, the hardware include Fig. 2A and Fig. 2 B respective examples logical block address (LBA) conversion RAID storehouses, quilt Included in the controller 102 for enhancing write cache management for implementing storage adapter.For host LBA and AFDASD RAID storehouses, write-in LBA length is shown at frame 402,404, and is skipped mask and shown at frame 406,408.As shown , AFDASD to array or reverse data striping frame 410 provide the data strip for being applied to write-in/reading cache 415 Frame 412,414.For host LBA and Vset RAID storehouses, show to read LBA length at frame 416,418, in frame 420,422 Place shows to skip mask, and the LBA more than 32 is shown at frame 424.VSet is provided to array frame 426 without offset block 428, 430, non-zero offset frame 432, Vset frames 432 past end and be applied to write-in/reading cache 415 be more than 256K The frame 434 of length.As shown, write cache 415 provides no write de-lay, 4K alignment 438, no write de-lay skips 440, No write de-lay, the beginning and end 442 of non-4K alignment, no write de-lay, non-4K alignment start 444, and no write de-lay, non-4K alignment is tied Beam 446.As shown, cache 415 is read in write-in provides the no write de-lay (FW) with covering (Ov), 4K alignment 448, tool The no write de-lay (FW) for having covering (Ov), skips 450, has the no write de-lay (FW) of covering (Ov), non-4K alignment starts and knot There is beam 452 no write de-lay (FW) of covering (Ov), non-4K alignment to start 454, have the no write de-lay (FW) of covering (Ov), Non- 4K alignment terminates 456.As shown, write-in reads cache 415 and provides ongoing reading (RIP) at frame 458, Ongoing mirror image (MIP) and ongoing superposition (OIP).As shown, there is no cache in frame 460, and in frame 462 do not have dynamic layer, and in frame 464 there is non-single driver RAID-0, said write to proceed to Striping data frame 466, data R0-1 bands frame 468 is provided.
Referring now to Figure 5, there is illustrated the usual hardware branch by specified by reference character 500 according to preferred embodiment The exemplary degraded operation held, includes example logic block address (LBA) conversion of Fig. 2A and Fig. 2 B, Fig. 2A and Fig. 2 B are included in In the controller 102 for enhancing write cache management for implementing storage adapter.As represented by block 502, degrade search or Person starts in LRU (chained list used in an array) or in array offset (array LBA)；And the search that degrades is in largest request The end of size or span or 256K band boundaries (64 CLs) is terminated.CLs with DIP/MIP/OIP is noted, otherwise by Ignore.As illustrated in box 504, degraded operation is provided in the case where being with or without dual controller 102.It is operated with reading and writing 300,400 is different, and firmware is for generating hardware operation chain to execute overall degradation.This is because many different due to supporting RAID level and other complexity and caused by degrade variation number it is too many, hardware can not be in the case where no firmware be assisted Processing.Show that there is 4K LBA/ length frame 506 and write-in/reading caching 505 more than 32 LBA frames 508.Degraded operation packet Degradation is included, 4K alignment 510, degradation skips 512, and degrade non-4K alignment beginning and end 514, degrades, and non-4K alignment starts 516 Hes Degrade, non-4K alignment terminates 518.The dynamic layer frame 520 that remaps does not provide exchange at frame 522, is provided in frame 524 and exchanges phase Same RG is provided in frame 526 and is exchanged different RG, provides ongoing exchange in frame 528, and/or provide intersection in frame 530 Band.As shown in the not frame 532 of dynamic layer, degradation proceeds to Striping data frame 534, provides data strip as shown in the figure Frame 536,538,540,542,544,546,548,550,552 and 554.Data strip frame 534 is provided in frame 556 intersects submatrix Row, and/or provide the driver for the exposure hit in frame 558.
Referring now to Figure 6, it illustrates the exemplary cache rows stored according to the control of preferred embodiment in (CS) (CL) structure is usually specified by reference character 600, and is included in for realizing storage adapter enhancing write cache In the controller 102 of management.Cache line (CL) structure 600 includes the byte 0-7 that there is CS states to define, as follows：7 The CL of=distribution, is arranged by distribution engine
The positions 6=position
5=PIP (removing in carrying out) is opened and count-up counter (is passed cancelling the when of distributing when CL is marked for removing Down counter)
4=MIP (mirror image is underway), in HASH, in LRU
3=HASH links are effectively (may or may not be in LRU)
2=RIP (reading underway) may or can not possibly be in LRU
1=DIP (degrading underway) is not in LRU
0=OIP (covering being removed after mirror image, it is underway to merge mask) is not in LRU.
The definition of CL states allows to track following state：Write-in when data/catalogue is mirrored to local/remote nv memory During operation, mirror image is underway (MIP)；Using operation progress address period is covered, cover underway (OIP)；It is reading During extract operation, read underway (RIP)；During degraded operation, degrade underway (DIP).Operate (OP) structure number It can increase with each cache entries.For the addresses the CS Up pointers of LRU, relatively low 6=0, and for the addresses CS of LRU Down or next pointers, relatively low 6=0.For the addresses the CS previous pointer of hash, relatively low 6=0, and for hash The next pointer in the addresses CS, relatively low 6=0.Cache line (CL) structure 600 includes compressed index value.
In NV batch operations, position position be set and for all CLs after first CL count-up counter.It is right In each CL, counter successively decreases, and cancels for NV and is provided with position position in distribution.When FW determinations should before starting degraded operation When write cache being allowed how complete to reach, FW will use this counter.Higher position counting allows more complete WC.
Referring now to Figure 7, there is illustrated showing in the control storage (CS) for storing (DS) for automatic mirror-image to data Example property cache line (CL) structure, is usually specified by reference character 700, and be included in and be used for according to preferred embodiment In the controller for realizing storage adapter enhancing write cache management.Data storage (DS) is arrived for automatic mirror-image 700 Control memory (CS) in cache line (CL) structure include array ID, mask, the offset of 6B arrays, 0x80 or C0, behaviour Make (OP) structure number and compressed index value.
Referring now to Figure 8, it illustrates the exemplary cache rows stored according to the data of preferred embodiment in (DS) (CL) structure is usually specified by reference character 800, and is included in for realizing storage adapter enhancing write cache In the controller of management.As shown, it is the minimum letter needed for effective (distributing) that the CL in DS 800, which has mark CL, Breath, data are used for which array ID and LBA, op structures number and compression relevant information.
Referring now to Figure 9, there is illustrated according to preferred embodiment example local SAS and long-range SAS cache lines (CL) control storage (CS) mirror image or SAS mirror images, which are deleted to data, stores (DS), is usually specified by reference character 900, and by It is included in the controller for enhancing write cache management for realizing storage adapter.Local SAS cache lines (CL) Control storage (CS) 902 includes local SAS mirror images or SAS mirror images the deletion path 904 to local datastore DS 906.It will SAS interconnection 908 is supplied to long-range SAS cache lines (CL) control storage (CS) 910 and remote data storage DS 912.From this Ground CS 902 provides long-range SAS mirror images to remote data storage DS 912 or SAS mirror images delete path 914.
Referring now to Figure 10 A and 10B, show in cache line (CL) state in the hardware according to preferred embodiment Example transactions, usually specified by reference character 1000, and be included in for realizing storage adapter enhance write-in high speed In the controller of cache management.Cache line CL status transactions 1000 include having the effective free pools 1002 of HASH and LRU, Distribution 1004, it is effective with the effective mirror images 1006 of HASH and LRU and with HASH lists and LRU list is effectively idle 1008.As shown, each in free pool 1002, distribution 1004, mirror image 1006 and free time 1008 includes the CL of 7=distribution CL states, be arranged by distribution engine, the parts 6=position, 5=removes state, and 4=MIP, 3=HASH link are effective, 2=RIP, 1 =DIP, 0=OIP.Cache line CL status transactions 1000 include HASHV＆OIP＆LRU=NULL 1010, HASHV ,s ＆DIP＆ LRU=NULL 1012 and HASHV＆DIP＆LRU=VALID 1014, HASHV＆RIP＆DIP＆LRU=NULL 1016, with And RIP＆LRU/HASH=NULL1018.1020 row is being indicated in the following, HASHV, RIP, DIP, OIP are only arranged when being arranged Go out.
Referring also to Figure 10 B together with the corresponding CL status transactions row 1A-9B in Figure 10 A, if 1B has found MIP/RIP/ DIP/OIP is set, then it includes recycling the controller chassis that cache line CL status transactions 1000, which are included in the installation at 1022, (CB).As shown in 1A rows, a page table is read, and be opened for each CL, MIP and HASHV, be added to LRU together In HASH.As shown in 1B rows, next CL is searched in HASH to obtain identical array ID and array offset, for The each CL found opens OIP, is removed from LRU, is put into SAS and deletes CL Duplicated pages inventories.As shown in 1024, MIP＆ is closed OIP.As shown in 2A rows, page inventory is read, MIP is closed for each CL.As shown in 2B rows, if OIP settings and phase Next CL is read in same array ID＆ arrays offset, and the CL each found closes OIP, closes HASHV, deletes and is placed on from HASH Cancel on configuration page inventory.
As shown at 1026, installation is cancelled.As shown in 3A rows, page inventory is read, for each CL, MIP is closed, closes HASHV, and removed from HASH.As shown in 3B rows, next CL is read, closes OIP, PURGE is closed, then rejoins LRU.As shown in 1028, if " 4 " find MIP/RIP/OIP, reads search and recycle CB to the WQ.As shown in the 4th row, search For HASH to obtain matched array ID and array offset, the CL each found opens RIP, and it is clear to place the combination HDMA2h pages It is single；Otherwise, volume index is written to combination page inventory.
As shown in 1030, RIP is closed；Combination page inventory is read, using skipping mask.As shown in 5A rows, each look for To CL opened with HASHV, close RIP.As shown in 5B rows, each CL is found in the case where HASHV is closed, and closes RIP, It puts and cancels assignment page inventory.As shown in 1032, degrades, ignore DIP/MIP/OIP.As shown in the 6th row, searched in HASH The array offset of one 256KB range finds open DIP for each CL, deletes, be placed in page inventory from LRU.Such as Shown in 1034, DIP, reading page inventory are closed.As shown in 7A rows, for each CL found in the case where RIP is closed, DIP is closed, HASHV is opened, is removed from HASH, is put into and cancels assignment page inventory.As shown in 7B rows, RIP is opened In the case of each CL for finding, close DIP, close HASHV, removed from HASH.
As shown in 1036, degradation is cancelled.As shown in eighth row, page inventory is read, closes DIP, closes removing state, and Added back LRU.As shown in 1038, removes, ignore MIP or distribution=0 or 4：0=0 or if remove state= 1.As shown in 9A rows, for each CL found in the case where OIP/DIP is closed and HASHV is opened, removing state is opened, It opens DIP and is removed from LRU, is placed on degradation page inventory.As shown in 9B rows, for being closed in OIP/DIP openings or HASHV The each CL found when closing opens removing state.
Referring now to Figure 11 A and 11B, there is illustrated write for realizing storage adapter enhancing according to preferred embodiment Enter example cache row (CL) chain with identical array ID/LBA included by the controller of cache management, it is general It is specified respectively by reference character 1100,1120.In Figure 11 A, as shown, the control with identical array ID/LBA stores (CS) the example CL chains 1100 in include the free time 1102 comprising distribution=1, LOC=0/1, HASHV=1 (LRU=is effective)； Including Allocate=1, LOC=0/1, HASHV=1 (LRU=is effective), the RIP 1104 of RIP=1；Including distributing=1, LOC The DIP 1106 of=0/1, HASHV=1, DIP=1；Including Allocate=1, LOC=0/1, HASHV=1, RIP=1, DIP =1 RIP/DIP 1108；Including Allocate=1 and including Allocate=1, (LRU=has by LOC=0/1, HASHV=1 Effect) free time 1112 ALLOCATED 1110；And including Allocate=1, LOC=0/1, MIP=1, HASHV=1 The MIP 1114 of (LRU=is effective) and including distribute=1, LOC=0/1, the OIP 1116 of HASHV=1.
In Figure 11 B, the example CL chains 1100 in the data storage (DS) with identical array ID/LBA include distribution 1122, the 1122 of distribution includes that=1 and Op of distribution builds #=N+X+Y；The 1124 of distribution include Allocate=1 and Op Build#=N+X；1126 with distribution include Allocate=1 and Op build#=N.
Referring now to Figure 12 A, Figure 12 B, Figure 12 C and Figure 12 D are shown respectively according to preferred embodiment for realizing storage Included by the controller of adapter enhancing write cache management for being written, unoptimizable or interim write-in, read and The exemplary operations chain based on controller chassis (CB) in the hardware engine of degraded operation.
In fig. 12, the example write-in usually specified by reference character 1200 is handled：NV pages 1202 of distribution, distribution Data store A pages of inventory in (DS)；Host dma is used for data to DS compressions 1204 from host dma to A page listings；It writes Enter caching (WC) covering search and installation 1026, determine whether covered when updating CL states it is existing in cache Data (open MIP and generate OIP inventories or B pages of inventory, it is determined whether need sector IO)；The sectors XOR I/O merges 1208 and is writing Enter and merges non-4K data at the beginning and end of operation；SAS mirror images 1210 by page inventory new data be mirrored to local NV DS and Long-range NV DS；SAS mirror images delete 1212 mirror images and delete B page inventories；For example, if SAS mirror images or SAS mirror images delete failure, MIP＆OIP 1214 is closed, the MIP for page inventory is closed and generates the new inventory or C page inventories of OIP, and closes MIP＆ OIP is cancelled；NV cancels distribution 1216 and cancels the distribution overlapping page or C page inventories；It sends 1218 with SIS and is sent to host and ring It answers.
In fig. 12, the example unoptimizable usually specified by reference character 1220 or interim write-in processing, which are included in, does not press XOR Vol to NV copies 1222 in the case of contracting.
In fig. 12 c, the example reading process usually specified by reference character 1224 includes Vol distribution 1126；WC is read Search 1228；SAS reads (or hit is read in part) 1230；HDMA 1232 is decompressed from DS；VOL cancels distribution 1234；It closes RIP 1236；NV cancels distribution 1238, for example, the data to have degraded；1240 are sent with SIS.
In fig. 12d, the exemplary degradation processing usually specified by reference character 1242 includes the search 1244 that degrades；XOR Decompression 1246；The various assistance that RAID degradations participate in are executed, such as Vol distribution/cancellation distributes, PSL lock locking/unlockings, setting/ Update/removing odd even update footprint (PUFP), SAS operations；SAS mirror images delete 1250；For example, if Destage fails, close DIP 1252, and close DIP and cancel；Cancel distribution 1254 with NV.
Referring now to Figure 13, slow shown here as write-in high speed is enhanced according to preferred embodiment for realizing storage adapter The flow chart of the exemplary operations of the usual ablation process specified by reference character 1300 included by the controller 102 of management is deposited, Start from frame 1302.The inspection for the volatibility page (x2) provided is executed as shown in decision box 1304.When offer volatibility It when page, then bypasses as shown in frame 1306, and is written by AMGR as shown in frame 1308.Otherwise, when not providing volatibility page, The inspection of possible data in cache is executed as shown in decision box 1310.When identifying the possibility in cache When data, then the inspection for skipping or WC coverings are abnormal is executed as shown in decision box 1312.When identification skip or WC covering When abnormal, then as shown in frame 1316, volatibility distribution (x2) be provided or is pre-allocated, the host as shown in frame 1318 directly stores Device accesses (HDMA) and is such as written by AMGR shown in frame 1320, and Vol cancellation distribution as indicated by block 1322.Such as box 1323 It is shown, then SIS transmissions are provided, sends and responds to host.
Otherwise, when skipping or WC coverings are abnormal unrecognized, then according to preferred embodiment by executing hardware chaining step It is rapid to go to the write-in of write cache：Distribute the NV pages 1324 of the distribution A page inventories in data storage (DS)；It is main Machine DMA to DS compressions 1226, for from host dma data to A page inventories；Write-in caching (WC) coating search and installation 1328 determine that the available data whether covered when updating CL states in caching (opens MIP and generates OIP inventories or the B pages Inventory, it is determined whether need sector IO)；The sectors XOR I/O merge 1330 write operation at the beginning and end of merge non-4K numbers According to；1332 Duplicated pages inventory new data of SAS mirror images is to local NV DS and/or arrives long-range NV DS；SAS mirror images delete 1334 mirrors As deleting B page inventories；MIP＆OIP 1336 is closed to close the MIP for A page inventories and generate new OIP inventories or C pages Face inventory；NV cancels distribution 1338 and redistributes the overlapping page or C page inventories；1323, which are sent, with SIS sends response to host.
Referring now to Figure 14, slow it illustrates write-in high speed is enhanced according to preferred embodiment for realizing storage adapter Deposit the example on the usual no write de-lay with superposition specified by reference character 1400 included by the controller 102 of management XOR merges.XOR merges on no write de-lay with coating 1400 including including the mirror image (MIP) and new data 1402 in processing Buffering area A and buffering area B including the coating (OIP) in processing and be merged into slow including MIP and merging data 1406 Rush the legacy data 1404 in area A.
Referring now to Figure 15, there is illustrated enhance write-in high speed according to preferred embodiment for realizing storage adapter The flow of the usual exemplary operations specified by reference character 1500 of reading process included by the controller 102 of cache management Figure, starts from frame 1502.The inspection for the volatibility page (x2) provided is executed as shown in decision box 1504.It is volatile when providing Property page when, then as read via AMGR shown in frame 1508 shown in frame 1506.Otherwise, when not providing volatibility page When, the inspection in cache for possible data is executed as shown in decision box 1510.It may in cache when identifying Data when, then as executed the corresponding inspection for skipping or reading search exception shown in decision box 1512.When identification is skipped or is read When search is abnormal, then providing volatibility distribution (x2) as shown in frame 1516 or pre-allocating, AMGR is read as shown in frame 1518, such as Host direct memory access (HDMA) shown in box 1520, and the Vol as shown in frame 1522 cancels distribution.Such as 1523 institute of box Show, then SIS transmissions are provided, sends and responds to host.
Otherwise, when skip or read search it is abnormal unrecognized when, then passed through according to preferred embodiment and execute HW link steps It is rapid to execute the reading from write cache：Vol Allocate 1524 provide A-Vol；WC reads search 1526, wherein B Including the Vol indexes from A and the NV indexes for the data in caching, also generate and skip position；(2 friendships are read by AMGR Pitch 256KB boundary op) 1528；B-NVv is received from DS decompressions HDMA1530；Vol cancels 1532 (or without op) of distribution and receives A- Vol；It closes RIP 1534 and receives B-NVv and for the C-CLr that the index of jump position provides closing RIP has been arranged；NV cancels distribution 1536, such as C-CLr is provided, due to being removed by degrading, so only needing to cancel distribution NV indexes；It is sent with SIS 1523。
Referring now to Figure 16, there is illustrated enhance write-in high speed according to preferred embodiment for realizing storage adapter The stream of the usual exemplary operations specified by reference character 1600 of degradation structure processing included by the controller 102 of cache management Cheng Tu starts from frame 1602.
Feature according to the present invention, hardware engine, which assists to generate to have, to be started in either LRU or specified array LBA The degraded operation of option, and stop in size/span of largest request or in the end of 256K band boundaries.Such as side Shown in frame 1604, searched for by hardware auxiliary to call to degrade, hardware auxiliary will pass through hash (hash) table and/or LRU teams Row search write cache degrades to generate and (opens DIP), as shown, has either in LRU or specified The options that start of array LBA, or start in LRU and (there is the gap for HDD, be not used for the gap of SSD).Such as decision box Shown in 1606, the inspection for the complete 256K of discovery and the complete band of permission being written (FSW) is executed.It is complete when finding 256K and when not finding the FSW of permission, as shown in decision box 1608, executing cannot be complete as single op drivings for Op At inspection.When Op cannot be completed as single op drivings to be found, overlapping or serialization are executed as shown in frame 1610 Degrade, for example, due to the gap when driver does not support skip operations or CL includes the sector I/O with possible gap. When Op cannot be completed as single op drivings not to be found, then single degradation is executed as shown in frame 1612.
When the FSW of the complete 256K and permission that find are identified, then such as decision box 1614 is shown executes main odd even school Test the inspection that band starts.At the beginning of not finding primary parity band, then single degrade is executed as shown in frame 1616. At the beginning of finding primary parity band, as shown in box 1618, start to call the search that degrades at array offset.Such as Shown in decision box 1620, execute for all complete 256 or the inspection of no additional search.If it is, such as frame Multiple degradations are executed shown in 1622.If it is not, then as shown in box 1624, cancels additional degradation or arranged for later work Team.Then single degrade is executed as shown in box 1626.
It is to show to enhance write cache according to preferred embodiment for realizing storage adapter referring now to Figure 17 The flow of the usual exemplary operations specified by reference character 1700 of single degradation processing included by the controller 102 of management Figure, starts from frame 1702.Such as RAID 5 or 6 is checked in indicated execute of decision box 1704.If identifying RAID 5/6, hold The following HW operating procedures gone according to preferred embodiment, including：Vol distributes (2X buffers) 1706, XOR decompressions 1708, leads to The write-ins of AMGR 1710 are crossed, Vol cancels distribution 1712, SAS mirror images and deletes (locally and/or remotely) 1714, receives B-DIP and (degrade It is underway), it closes DIP 1716 and receives B-DIP, and C-CLr (being cancelled for example, if degrading and unsuccessfully closing DIP) is provided, NV takes The distribution 1718 that disappears receives C-CLr, and operation is completed as shown in frame 1720.
If RAID 5/6 is unrecognized, the following HW operating procedures according to preferred embodiment are executed, including：Read NO DEST (XOR decompressions) 1722, is written by AMGR 1724, and is continued SAS mirror images and deleted (locally and/or remotely) 1714, B-DIP (degrading underway) is received, the DIP 1716 for receiving B-DIP is closed, and provides C-CLr (for example, if drop Grade failure is then closed DIP and is cancelled), NV cancels distribution 1718 and receives C-CLr, and completes to operate in frame 1720.
Referring now to Figure 18, there is illustrated enhance write-in high speed according to preferred embodiment for realizing storage adapter The stream of the usual exemplary operations specified by reference character 1800 of multiple degradations processing included by the controller 102 of cache management Cheng Tu starts from frame 1802.Executing following HW operating procedures according to preferred embodiment includes：Vol distributes (2X buffers) 1804, 1806, Vol is written via the band of AMGR and cancels distribution 1808, SAS mirror images deletion (locally and/or remotely) 1810, receives B- DIP closes DIP*N 1812 and receives B-DIP, and provides C-CLr, and NV cancels distribution * N 1814 and receives C-CLr, and such as frame Operation is completed shown in 1816.
Referring now to Figure 19, there is illustrated enhance write-in high speed according to preferred embodiment for realizing storage adapter The flow chart of the exemplary operations around processing included by the controller 102 of cache management, is usually specified by reference character 1900, Start from frame 1902.As shown in frame 1904, start to call the search that degrades at array offset, to determine size/span of OP.Such as Shown in decision box 1906, the inspection of the OR=0 and write-in of index #=0 to(for) mode bit is executed.If it is, operation is It is completed as shown in frame 1908.Otherwise, if it is not, then as shown in decision box 1910, execute write-in index # whether=0 inspection. If the index #=0 of write-in, delay is such as provided shown in frame 1912, and operate and return to frame 1904, call and degrade search simultaneously Continue.If the # of the index of write-in is not equal to 0, such as execute shown in frame 1914 whether op cannot be by drive as single op The inspection of completion, such as due to gap, but driver does not support that skip operations or CL include the sector I/ with possible gap O.If op such as executes the degradation of overlapping or serialization as cannot individually be completed to drive shown in frame 1916.If Op can be completed as single op to drive, then single degrade is executed as shown in 1918.Operation calls drop back to frame 1904 Grade is searched for and is continued.
Referring now to Figure 20, slow it illustrates write-in high speed is enhanced according to preferred embodiment for realizing storage adapter The exemplary removing hardware auxiliary tool packet based on register included by the controller 102 of management is deposited, usually by reference character 2000 is specified.As shown in frame 2002, WC hardware reads all CL, and the following occurs by any general for matching this range：If =1, PIP=0, MIP=0 and HASHV=1 are distributed, then opens PIP and count-up counter.If count-up counter, DIP= 0, OIP=0, then DIP is opened, is removed from LRU, and CL is placed in page table inventory.WC engines can handle CB during this period, This function will interlock with CB processing.FW loads register 2004：
Array ID (7：0)2006
Due to that must be 4KB alignment, start array offset (44：0), position (2：0)=02008, due to that must be that 4KB is more It is a, terminate array bias size (44：0), position (2：0)=02010,
Page table inventory pointer (31：0)2012
Page table size (11：0), highest 4K-1 entries 2014
Page table current size (11：0)2016
Current CL indexes (24：0), 2018 (zero can be set as when starting, keep after the dwell)
Max CL indexes (24：0)2020
Significance bit, page table interrupt bit 2022；With
It is current to remove counter (24：0) 2024, for provided with PIP any CL, cancelling distribution and will decrement by.
CL 2026 (is removing) position 2028 including PIP and (is incrementally removing counting undetermined when setting, removing of successively decreasing, which is cancelled, to divide The counting undetermined matched).Event Q entries 2030 are complete in page table list and include when sending removing page listings event entries 2032 Above-mentioned pause；When Purge completes scanning and page table inventory is not empty, sends and remove page inventory EventQ entries 2034 (11：10=type=11b), pay attention to：It must restart；And it after removing completion and counter and being zero, sends clear Except completion EventQ entries 2036 (11：10=types=10b).It is worth noting that FW is removed using search is degraded for small The cached data of the LBAs of range.Larger range of LBA should be used for by removing engine.
It is to show to enhance write-in high speed according to preferred embodiment for realizing storage adapter referring now to Figure 21 and 22 The flow chart of the exemplary operations for removing engine processing included by the controller 102 of cache management, usually by reference character 2100 It is specified.In figure 21, it removes engine processing and starts from frame 2102, as shown in decision box 2104, execute WC engines and be in the free time It checks.When WC engines, which are in idle, to be identified, the inspection to the CL of all processing is executed as shown in decision box 2106.If no It is CL then to be handled as shown in frame 2108, and execute page inventory as shown in decision box 2110 and completely check.If the page is clear It is single imperfect, then as shown in decision box 2112, execution whether the boundaries 256K inspection.If it is, operation returns to judgement Frame 2104 checks that WC engines are idle and continue.Otherwise, it if page inventory is complete, as shown in frame 2114, provides with having The index # of effect carrys out indicating section removing.When handling all CL, is executed as shown in decision box 2116 and check blank page inventory. If page inventory is not empty, the effective index # provided used in frame 2114 carrys out indicating section removing.If page inventory For sky, then executing zero inspection for removing counter as shown in decision box 2118.If it is, as shown in box 2120, instruction It removes and completes.Otherwise the operation as shown in box 2122 is completed.
With reference to shown in figure 22, the removing request started at box 2202 includes checking as shown in decision box 2204 currently The reset procedure being executed.It is lined up as shown in box 2205 if it is, removing request.Otherwise it as shown in box 2206, uses It is called in the removing engine routine of array ID and LBA range.As shown in decision box 2208, execute for the index that is found and The inspection of the removing engine of pause.If it is, executing SAS mirror images as shown in box 2210 deletes (locally and/or remotely).Such as DIP is closed shown in frame 2212, and is provided NV as shown in frame 2214 and cancelled distribution.As shown in frame 2216, engine is removed again Start (not suspending), and check the removing engine of the index and pause that find, back to decision box 2208 and continues.When finding Index and pause removing engine it is unrecognized when, as shown in frame 2218 remove complete.Then as shown in decision box 2220 Execute the inspection for the removing request for whether having queuing.If it is, operation at frame 2206 return with call be directed to array ID and The removing engine routine of LBA range simultaneously continues.Otherwise, operation is completed as shown in frame 2222.
Referring now to Figure 23, show enhances write cache according to preferred embodiment for implementing storage adapter Exemplary hardware counter included by the controller 102 of management and statistics, are usually specified by reference character 2300.Hardware counter It is safeguarded on the basis of each array ID by HW with statistical data 2300, including：
Present counter value CL 2302
Present counter value position position 2304
The positions HWM position 2308
The positions LWM position 2312
The upper oldest CL entries of LRU UP, LRU, zero=sky 2314
LRU DOWN, next position will place new CL, zero=sky 2316
Present counter value WC installations amount to 2318
Present counter value WC carries out installation 2320 using overlapping
Present counter value WC, which is read, amounts to 2322；With
Present counter value WC reads hit with all cachings and reads 2324
Feature according to the present invention, for the cache management under most of hardware controls, firmware is difficult to help to manage Each array cache threshold value (that is, data-level needed for the cache of each array) and each array cache (maximum amount of data i.e. in each array cache) is limited to help without hardware.Also overall (non-array) threshold value and Management limitation.Hardware is provided from firmware and output (register and event) to the input (register) of firmware, to help to manage height Fast cache threshold and limitation.
Feature according to the present invention, provide it is a kind of it is new based on the method for trigger come dynamic optimization degradation rate.Newly Based on the method for trigger using one group of register come cache page height trigger and cache CLs height trigger Device.The counter of each in these countings of hardware maintenance, and when hardware counter is crossed in these high or low triggers Any one when, it generate firmware event to be processed.When these trigger values are crossed and these events are handled, Gu Part then correspondingly adjusts degradation queue depth.If cache page or cache cls high water marks (water Mark it) is crossed, then can increase degradation queue depth.Meanwhile the high and low cache page and/or cache CL are posted Storage value will increase to detect next crosspoint.If the cache page or CL low-water marks are crossed, degrade speed Rate may be reduced with high-order and low-water mark setting.Degradation rate will be by the cache page or cache CLs is determined.To avoid multiple affair, when either the cache page or cache CLs are more than high water mark, the two High water mark can all move up.But the low-water mark of the highest resource of mobile utilization rate is very important.From essence On say, it is that must monitor low water level to detect when can to reduce the unique resource of degradation rate to cause the resource of higher degradation rate.
Feature according to the present invention, when the cache page or CL low-water marks are crossed, if it is relatively low money Source, then degradation rate will not change, and the low water level of the only resource must be turned down.If it is higher resource, then It has to check for lower with resource class to determine whether it has become the resource of higher utilization rate.Then degradation rate will be based on compared with High resource is set.Moreover, two high water marks are all based on higher resource setting.Next, higher resource Low-water mark be necessarily placed at the trigger point of next relatively low degradation rate.Finally, if compared with low-resource low water level compared with The new of high resource is arranged in higher point, then it must also be reduced.
Feature according to the present invention, each based on array trigger method dynamic adjust each resource constraint.This is every A resource cache CLs height trigger and each resource location count height trigger and use one group of register.Hardware Safeguard the counter of each in these countings, and when hardware counter crosses over any one in these high or low triggers When a, it generates firmware event to be processed.Substantially these triggers are used to monitor ratio and the adjustment of CLs and position counting Each resource constraint.If high water mark is crossed in CL countings, the limit should be reduced.Meanwhile the height water of CLs should be increased Position.If position counting crosses high water mark, limitation should be increased, and increase the height and low water level mark for position counting Note.If CL countings have been more than low-water mark, the limitation should be increased, and the height and low water level of CLs should be reduced Label.If local count has been more than low-water mark, this boundary should be reduced, and position counting should be reduced High-low water level.
Referring now to Figure 24 and Figure 25, it illustrates enhanced to be written for realizing storage adapter according to preferred embodiment Example CL limitations included by the controller 102 of cache management and threshold value, are usually referred to by reference character 2400,2500 respectively It is fixed.
In fig. 24, the overall constraint for NV 4K indexes (size of CACHE DIRECTORY) is shown at 2402 and is used for NV The CL limitations of the overall threshold (nn% of limitation) of 4K indexes and threshold value 2400, and use example high-water mark 2404 (HWM) overall constraint (size of actual cache) indexed for NV 528 is shown and is used for low water mark (LWM) NV 528 indexes the overall threshold of (n% of the limit).Such as array #1,2406, array #2,2408, array #3,2410 and array # Shown in 4,2412, per array, limitation is shown in dotted line and is shown with dotted line relative to HWM and LWM per array threshold for respective examples Go out.
In fig. 25, CL limitations and threshold value 2500 include the input from HW 2502 from register 2504, including
CL is counted (per array ID)
CL position countings (per array ID)
NV 4K freely index (by head/tail pointer) and
NV 528 is freely indexed (by head/tail pointer).The input from HW 2502 from event 2506 includes：
NV 4K are in HWM or more
NV 4K are in HWM or less
NV 528 is in HWM or more
NV 528 is in HWM or less
Array CL is counted in HWM or more
Array CL is counted in HWM or less
Array CL position countings are in HWM or more
Array CL position countings are in LWM or less
FW is limited and threshold calculations function 1508 receives the input from HW 2502 and provides output to HW 2510, is wrapped Include the update of CL limitation (each arrays) 2512.This waits for Q to update adding deduct for port and completes by using FW arrays.To HW 2510 output includes for counting the update 2514 with the HWMs/LWMs of the array of CL position countings based on CL.It is to be noted that The HWMs/LWMs that NV 4K and NV 528 is freely indexed infrequently changes.Other outputs 2516 include degradation rate 2518.
Referring now in Figure 26, enhance write-in high speed shown here as according to preferred embodiment for realizing storage adapter The flow chart of the exemplary operations of clobal processing included by the controller 102 of cache management, usually by reference character 2600 It is specified.As shown in decision box 2602, HWM or LWM is identified.For HWM, as shown in decision box 2604, the page or CL are identified.Such as frame Shown in 2606, the page and CL HWM increase for the identified page.As shown in frame 2608, page LWM increases.Such as 2610 institute of frame Show, the page and CL HWM increase for identified CL.As shown in frame 2612, CL LWM increase.As shown in frame 2614, degradation rate Increase.For LWM, as shown in decision box 2615, the page or CL are identified.As shown in frame 2616, executed for the page identified Whether the page is more than the inspection of CL.If the page as shown in frame 2620 is more than CL, the page and CL HWM reduce.When the page is little When CL, page LWM is zeroed out, and the CL LWM as shown in frame 2622 are set.As shown in decision box 2624 execute check with Identification reduces the needs of degradation rate.When needing reduction degradation rate unrecognized, then at the clobal as shown in frame 2626 Reason is completed.The top resource LWM as shown in frame 2628 is reduced.The degradation rate as shown in frame 2630 is reduced, and then clobal is handled It is completed in frame 2626.As shown in decision box 2632, the inspection for whether being less than CLs for the page of identified CL executed.Work as page When face is less than CLs, the page and CL HWM reduce in frame 2620.When the page is not less than CL, CL LWM are set to zero, and the page LWM is arranged to as shown in frame 2634.It executes and is checked to identify the needs for reducing degradation rate as shown in decision box 2636.When need Reduce degradation rate be not identified when, then frame 2626 complete clobal processing.As shown in frame 2630, then degradation rate It reduces, then degradation clobal processing is completed at frame 2626.
It is to show that enhancing write-in high speed according to preferred embodiment for realizing storage adapter delays referring now to Figure 27 The flow chart for depositing the exemplary step of every ARRAY PROCESSING included by the controller 102 of management, is usually specified by reference character 2700. As shown in decision box 2702, HWM or LWM are identified.For HWM, as shown in decision box 2704, place or CL are identified.Such as frame Shown in 2706, CL HWM and LWM increase for identified CL.As shown in frame 2708, array CL limitations are reduced.Such as frame 2710 It is shown, increase place HWM and LWM for the place identified.As shown in frame 2712, array CL limitations increase.Then, as judged Shown in frame 2714, executes and check to determine whether CL countings need higher degradation rate.If CL, which is counted, needs higher degradation Rate then increases array degradation rate as shown in box 2716, and each ARRAY PROCESSING as shown in frame 2718 terminates.For LWM, such as Shown in determination block 2720, position or CL are identified.As shown in frame 2722, reduced for the CL identified, CL HWM and LWM. As shown in frame 2724, array CL limitations are increased.As shown in frame 2726, for the position identified, position HWM and LWM are reduced. As shown in frame 2728, array CL limitations reduce.Then operation returns to decision box 2714, executes and checks whether to determine CL countings Higher degradation rate is needed, and is continued.
Referring now to Figure 28 and Figure 29, respectively illustrates increased for realizing storage adapter according to preferred embodiment here The usual example global trigger specified by reference character 2800 included by the controller 102 of strong write cache management and The every array trigger usually specified by reference character 2900.In Figure 28, global trigger 2800 includes relative to vertical axis The degradation rate and relative percentage all shown, the time is shown relative to trunnion axis.In Figure 29, wrapped per array trigger 2900 Include relative to shown in vertical axis degradation rate and CL quantity/position/limitation, relative to shown in trunnion axis time and it is big/ The region of the Ops of sequence and random Ops.In Figure 29, as shown, CL countings are shown in solid lines, and position counting is empty Line shows that CL limitations are shown in broken lines.
Figure 30 shows the block diagram of example design flow 3000.Design cycle 3000 can be according to the class of the IC designed Type and change.For example, the design cycle 3000 for building application specific IC (ASIC) can be different from for design standard group The design cycle 3000 of part.Design structure 3002 is preferably the input to design process 3004 and can come from IP and provides Quotient, core developer or other Chevron Research Company (CRC)s can either generate by the operator of design cycle or from other sources.Design Structure 3002 includes schematic diagram or the IOA or controller 102, hardware description language, such as Verilog of HDL forms, VHDL, C Deng.Design structure 3002 may be embodied on one or more machine readable medias.For example, design structure 3002 can be control The text file or graphical representation of device 102 and performance state machine 200.Design process 3004 preferably controller 102 is synthesized or It is converted into netlist 3006, wherein netlist 3006 is, for example, conducting wire, transistor, logic gate, control circuit, I/O, the row of model etc. Table describes the connection with other elements in IC design and circuit and is recorded in at least one of machine readable media On.This may be an iterative process, wherein netlist 3006 according to the design specification and parameter of circuit recombined it is primary or Repeatedly.
Design process 3004 may include using various inputs；Such as the input from library element 3008, the library element 3008 can accommodate one group of common component for giving manufacturing technology, circuit and equipment, including model, layout and symbol table Show, such as different technologies node, 32nm, 45nm, 90nm etc., design specification 3010, characteristic 3012, verify data 3014, Design rule 3016 and test data file 3018 may include test pattern and other test information.Design process 3004 Such as standard circuit design process, such as timing analysis is may further include, is verified, design rule check, place and route Operation etc..The those of ordinary skill of IC design field is appreciated that the possible electronics used in design process 504 The degree of design automation tool and application is made without departing from the scope of the present invention and spiritual.The design structure of the present invention is not limited to Any specific design cycle.
Design process 3004 is preferably converted together with any additional IC design or data (if applicable) In the embodiment of the present invention as shown in Figure 1A and 1B to the second design structure 3020.Design structure 3020 is resided in for exchanging On the storage medium of the data format of the topology data of integrated circuit, for example, with GDSII (GDS2), GL1, OASIS or for depositing Store up the information of any other suitable format storage of this design structure.Design structure 3020 may include such as testing Data file, design content file, manufaturing data, layout parameter, conducting wire, metal rank, through-hole, shape, for passing through manufacture The data of circuit line and by manufacture the embodiment of the present invention as shown in Figure 1A and 1B semiconductor maker need it is any its The information of his data.Then design structure 3020 may proceed to the stage 3022, in the stage 3022, for example, design structure 3020 proceed to out band, are released to manufacture, are released to mask room, are sent to another design office, are sent back to client, Etc..
Although by reference to the datail description present invention of the embodiment of the present invention shown in attached drawing, these details It is not intended to limit the range such as appended claims the claimed invention.
1. a kind of data-storage system, including：
Realize that the storage adapter for storing write cache enhances the controller of write cache management, including：
Realize the hardware-accelerated hardware write cache engine of storage write cache；And
The hardware write cache engine substantially manages write cache data and metadata without using firmware.
2. data-storage system as described in claim 1, wherein hardware write cache engine management write-in high speed Data cached and metadata, which is included in Standalone Drives redundant array (RAID) storehouse, provides the storage write cache, It is wherein based on array ID/ arrays LBA (logical block address) and executes write cache.
3. data-storage system as claimed in claim 2, wherein the hardware write cache engine provides CLs (high speeds Cache lines) hardware operation.
4. data-storage system as claimed in claim 2, wherein the hardware write cache engine is read in write operation It is defined using CL (cache line) to track CL states during extract operation and degraded operation.
5. data-storage system as described in claim 1, wherein the hardware write cache engine is executed and deposited to described Store up write cache write-in, including the hardware write cache engine from write cache control storage (CS) to Write cache data store the mirror image of (DS).
6. data-storage system according to claim 5, wherein the hardware write cache engine is from write-in high speed The mirror image that buffer control stores (CS) to write cache data storage (DS) includes local mirror image in the controller And the Remote Switched Port Analyzer to dual controller.
7. data-storage system as described in claim 1, wherein the hardware write cache engine is write from the storage Enter cache and execute reading, includes reading hit from all or part of of storage write cache.
8. data-storage system as claimed in claim 6, wherein the hardware write cache engine is write from the storage Enter cache and execute degradation, including the hardware write cache engine uses hash table and least recently used (LRU) One in queue generates degraded operation search from the storage write cache.
9. a kind of method for realizing storage adapter write cache management in data-storage system, including：
Offer includes the controller of hardware write cache engine；
It provides for realizing the hardware-accelerated hardware write cache engine of storage write cache；With
It provides for high speed substantially to be written to manage the hardware of write cache data and metadata without using firmware Caching engine.
10. method as claimed in claim 9 is included in Standalone Drives redundant array (RAID) storehouse and provides the storage Write cache, wherein executing write cache based on array ID/ arrays LBA (logical block address).
11. method as claimed in claim 9, including the hardware write cache engine provide CLs (cache line) Hardware operation.
12. method as claimed in claim 9, including the hardware write cache engine is in write operation, read operation With during degraded operation, is defined using CL (cache line) and track CL states.
13. method as claimed in claim 9, including the hardware write cache engine execute to the storage and height are written The write-in of speed caching, including the hardware write cache engine control storage (CS) to the control from write cache The mirror image of write cache data storage (DS) in device and the Remote Switched Port Analyzer to dual controller.
14. method as claimed in claim 9, including the hardware write cache engine are slow from storage write-in high speed Counter foil row is read, and includes reading hit from all or part of of storage write cache.
15. method as claimed in claim 9, including the hardware write cache engine are slow from storage write-in high speed Counter foil row degrades, including the hardware write cache engine uses in hash table and least recently used (LRU) queue One generates degraded operation search from the storage write cache.
16. a kind of design structure embodied in the machine readable media used in the design process, the design structure include：
The controller circuitry being tangibly embodied in the machine readable media used in design process, the controller circuitry are used for Realize that storage adapter enhances write cache management in data-storage system, the controller circuitry includes：Realization is deposited Store up the hardware-accelerated hardware write cache engine of write cache；With
The hardware write cache engine substantially manages write cache data and metadata without using firmware, Described in design structure be read and used for semiconductor chip manufacture when generate include the controller circuitry chip.
17. such as the design structure of claim 16, wherein the design structure is as the topology data for switched integrated circuit Data format it is resident on a storage medium.
18. such as the design structure of claim 16, wherein design structure includes test data file, characteristic, verify data Or at least one of design specification.
19. such as the design structure of claim 16, wherein design structure includes describing the netlist of the controller circuitry.
20. as claim 16 design structure, wherein the hardware cache engine management write cache data and Metadata, including the hardware cache engine is to the write-in of storage write cache and from the storage write cache It reads.
CN201680065411.6A 2015-11-09 2016-11-07 Realize the hardware accelerator for storing write cache management CN108351832A (en)
US62/252,749 2015-11-09
PCT/IB2016/056677 WO2017081593A1 (en) 2015-11-09 2016-11-07 Implementing hardware accelerator for storage write cache management
CN108351832A true CN108351832A (en) 2018-07-31
CN201680065411.6A CN108351832A (en) 2015-11-09 2016-11-07 Realize the hardware accelerator for storing write cache management
CN103324584B (en) 2016-08-10 The system and method for non-uniform cache in polycaryon processor
US20070260846A1 (en) 2007-11-08 Methods of prefetching data in data storage systems
US20120017054A1 (en) 2012-01-19 Storage sub-system and method for controlling the same
CN101231572B (en) 2012-06-27 Storage system