Source: https://patents.google.com/patent/US7958388?oq=7%2C958%2C388
Timestamp: 2018-03-24 19:54:48
Document Index: 90008220

Matched Legal Cases: ['Application No. 03', 'Application No. 03', 'Application No. 03', 'Application No. 03', 'Art. 96', 'Application No. 38', 'Application No. 03', 'Application No. 09153548', 'Application No. 03']

US7958388B2 - Methods and systems for a storage system - Google Patents
Methods and systems for a storage system Download PDF
US7958388B2
US7958388B2 US12476212 US47621209A US7958388B2 US 7958388 B2 US7958388 B2 US 7958388B2 US 12476212 US12476212 US 12476212 US 47621209 A US47621209 A US 47621209A US 7958388 B2 US7958388 B2 US 7958388B2
Active - Reinstated, expires 2022-12-24
US12476212
US20090240976A1 (en )
Melvin James Bullen
Steven Louis Dodd
David James Herbison
Mobile Networking Solutions LLC
Parallel Iron LLC
A storage system that may include one or more memory sections, one or more switches, and a management system. The memory sections include memory devices and a section controller capable of detecting faults with the memory section and transmitting messages to the management system regarding detected faults. The storage system may include a management system capable of receiving fault messages from the section controllers and removing from service the faulty memory sections. Additionally, the management system may determine routing algorithms for the one or more switches.
The present application is a continuation of and claims benefit of U.S. patent application Ser. No. 11/710,407, filed Feb. 26, 2007 now U.S. Pat. No. 7,543,177, which is a continuation of and claims benefit of U.S. patent application Ser. No. 10/284,199, filed Oct. 31, 2002, (now U.S. Pat. No. 7,197,662) and relates to the U.S. patent application Ser. No. 10/284,278, filed Oct. 31, 2002, by M. James Bullen, Steven L. Dodd, David J. Herbison, and William T. Lynch, entitled “Methods and Systems for a Storage System Including an Improved Switch,” and the U.S. patent application Ser. No. 10/284,268, filed Oct. 31, 2002, by M. James Bullen, Steven L. Dodd, David J. Herbison, and William T. Lynch, entitled “Methods and Systems for a Memory Section,” all of which are incorporated by reference herein in their entireties.
The present invention relates to data storage, and more particularly, to methods and systems for a high throughput storage device.
Accordingly, the present invention is directed to methods and systems that address the problems of prior art.
In accordance with the purposes of the invention, as embodied and broadly described herein, methods and systems for an apparatus are provided including one or more memory sections, one or more switches, and a management system. The one or memory sections include one or more memory devices capable of storing data in storage locations, and a memory section controller capable of detecting faults in the memory section and transmitting a fault message in response to the detected faults. The one or more switches include one or more interfaces for connecting to one or more external devices, and a switch fabric connected to one or more memory sections and the external device interfaces and interconnecting the memory sections and the external device interfaces based on an algorithm. A management system is provided capable of receiving fault messages from the memory section controllers and removing from service the memory section from which the fault message was received, and wherein the management system is further capable of determining an algorithm for use by a switch fabric in interconnecting the memory sections and the external device interfaces, and instructing the switch to execute the determined algorithm.
FIG. 1 is a block diagram of a storage hub environment, in accordance with methods and systems provided;
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.
The memory sections 30 preferably include the storage for the storage hub 10 along with other hardware for accessing the storage. As used herein, the term “memory section” refers to any subsystem including one or more memory devices that may be used for storing information. This architecture is applicable to any device that can store data. Thus, when the storage hub 10 receives a request to store data, the data is forwarded to a memory section 30, which stores the data. Likewise, when a request for data is received by the storage hub 10, the request is directed to the memory section 30 storing the requested information. The memory section 30 then reads the requested data, after which it is sent to the server 12 requesting the data. More detailed descriptions of exemplary memory sections 30 and their operations are presented below.
The management complex 26 of the storage hub 10 performs management-type functions for the storage hub 10 and connects the storage hub 10 with the external management system 14. As used herein the term “management complex” refers to any software and/or hardware for performing management of the storage hub 10. A more detailed description of the management complex 26 is presented below.
The switches 22 may be any type of switch using any type of switch fabric, such as, for example, a time division multiplexed fabric or a space division multiplexed fabric. As used herein, the term “switch fabric” the physical interconnection architecture that directs data from an incoming interface to an outgoing interface. For example, the switches 22 may be a Fibre Channel switch, an ATM switch, a switched fast Ethernet switch, a switched FDDI switch, or any other type of switch. The switches 22 may also include a controller (not shown) for controlling the switch.
Management Complex
FIG. 3 illustrates a logical architecture for a management complex 26, in accordance with methods and systems provided. As illustrated, the management complex 26 may include functions that manage administrative processes 32 and functions that manage control processes 34. These management functions can include one or more central processing units (CPUs) for executing their respective processes. Additionally, the management complex 26 may use one or more application program interfaces (APIs) for communications between these functions.
The control processors 34 perform configuration management functions for the memory sections 30, I/O controllers 24, switches 22, and the device drivers 28 of the servers 12. As used herein, the term “configuration” is a broad term that encompasses the various possible operating states of each component of the storage hub. As used herein, an “operating state” refers to a possible way in which the storage hub or one of its components operates as defined by parameter values. These parameter values, for example, may be set by a user of the storage hub, such as, for example, a system administrator, through, for example, an external management system 14. Operating states may include, for example, how often a component (e.g., a memory section 30) sends performance statistics to the management complex 26, the list of events that causes a component (e.g., a memory section, etc.) to report an alarm, and/or the type of alarm reported (e.g., catastrophic failure of component, minor fault with component, etc.). Further, as used herein, the term “configuration management” means the understanding of the current operating states of the storage hub's components and the capability to react to changes in the states of those components as defined by software running in the control processors 34. For example, the control processors 34 may control in real time the number of active memory sections 30 in the storage hub 10, the switches 22, and the device drivers 28 of the servers 12, if any, and any external servers 22 connected to the storage hub.
The control processors 34 may also, for example, be able to perform fault management for the storage hub 10. The term “fault management” as used herein means attempting to detect faults and take corrective action in response to the detection of a fault. For example, the control processors may recognize an operational failure of a memory section 30 or part of a memory section 30 and re-map data to working memory sections 30. Then, the control processors 34 may communicate this re-mapping to the external management system 14 and the device drivers 28 running on servers 12 attached to the storage hub 10.
The control processors 34 may also manage “bad-block” remapping functions when a memory section fails 30 and the writing of data to the magnetic storage device 38 in the event of power failures. Bad block remapping is a process wherein data blocks discovered by the section controller 54 or management complex 26 to be in a damaged memory device are, if possible, recovered.
For example, if the control processors 34 discover that block 65,000 in memory section 30-2 does not read correctly, the control processor 34 may decide to remap block 65,000 in memory section 30-2 to block location 1,999,998 in memory section 30-2. The control processor 34 may then direct the CDA 16 to read the data block and cause it to be written in location 1,999,998 in memory section 30-2. Once completed, the control processor 34 may inform the switches 22 and memory section 30-2 that block 65,000 may now be read from location 1,999,998.
The control processors 34 may also store a copy of the software (i.e., a software image) run by the switches 22. A more thorough description of the switches 22 is present below. If the need arises, it can reload the switch software to one or more of the switches. As discussed below, the switch 22 may include one or more switch controllers (not shown) for executing this software to control the switch 22. In the event the switch 22 uses multiple controllers configured in a master-slave architecture, the control processor 34 may determine which of the controllers in the switch is (are) the master(s) and which is (are) the slave(s).
Additionally, the control processors 34 may determine the status (active, idle, out-of-service) of ports (not shown) on the switch 22, whether the ports are used to connect to servers 12 or to memory sections 30. The control processors 34 may also provide configuration management data to the switches 22. Examples of configuration management data include the date, the time, a routing algorithm to use, an interval for a status check, the identity of active server ports, etc. Further, the control processors 34 may instruct the switch to use different “hunt” algorithms to find idle ports that may be used in establishing connections. These algorithms may be included in the software executed by the switch controller, examples of which include rotary hunt, skip route, and least-used.
FIG. 5 is a block diagram of an exemplary memory section 30, in accordance with methods and systems provided. As illustrated, the memory section 30 may include a switch portal (“S-portal”) 42, a section controller 54, a read only memory (ROM) 56, a temporary storage 58, a temporary storage interface device 60, a temporary store selector (“T-selector”) 62, a synchronizer 68, one or more memory interface devices 64-1 thru 64-8, and one or more memory devices 66-1 to 66-n.
The section controller 54 may also include memory device control circuitry 55 for controlling the memory devices 66. This memory device control circuitry 55 may, for example include a memory latching circuit for controlling the state of the memory devices 66 through the binary states of the memory latch. A further description of memory latching is presented below. The section controller 54 may further include test circuitry 59 for testing the memory section 30. A more detailed description of an exemplary test procedure is presented below. Additionally, the section controller may include a header/test interface 63 for providing header type information (e.g., a data block identifier, destination address, etc.) and testing the memory section 30. Also, the section controller 54 may include timing circuitry 61 that may provide master and slave clock signal and other timing signals, such as start and stop read or write signals, etc. for use by the memory section.
The temporary storage interface device 60 is any type of device capable of accessing the temporary storage device 58. For example, the temporary storage interface device 60 may include one or more shift register arrays (not shown), including a plurality of shift registers interconnected in series, such that the data may be serially clocked through the shift register arrays. For a further description of shift register arrays and their use in accessing storage media such as memory devices, see the patent application by William T. Lynch and David J. Herbison, entitled “Methods and Systems for Improved Memory Access,” filed on the same day as this application, which is incorporated by reference herein in its entirety.
FIG. 6 illustrates a functional diagram of a switch 22, in accordance with methods and system consistent with the invention. As illustrated, the switch 22 includes a switch/server communications interface 204 for interfacing with a server 12, a switch/memory section communications interface 208, a switch fabric 206, and a switch controller 202. The switch/server communications interface 204 and switch/memory section communications interface 208 may be standard switch interfaces found in commercially available switches and the terms memory section and server are used to indicate the devices to which the connections leaving the switch 22 preferably connect. The switch fabric 22 may be any type of switch fabric, such as an IP switch fabric, an FDDI switch fabric, an ATM switch fabric, an Ethernet switch fabric, an OC-x type switch fabric, or a Fibre channel switch fabric. Thus, the switch 22 may be any type of commercially available switch.
In an alternative embodiment to that of FIGS. 6 and 7, the selector 44 need not be included and all memory interface devices 64 may be connected to the switch fabric 206.
Memory Interface Device
FIG. 11 includes a more detailed block diagram of an embodiment of a memory interface device 64, in accordance with methods and systems provided. As shown in FIG. 11, the memory interface devices 64-1 and 64-2 may each include a write shift register array 72 and a read shift register array 74. Both the read and write shift register arrays can include a plurality of shift registers 76 interconnected in series. Each shift register 76 of the shift register array (72 and 74) is connected to a connector circuitry 77 which connects the shift register 76 to a corresponding I/O pin of the memory device 66.
As used herein, the term “shift register” refers to any register, device, stage or anything else with one or more selectable inputs that allows a signal to be received at an input and then output on the occurrence of some event, such as, for example, a control or clock signal. Although the term shift register sometimes refers to not just a single register stage, but also to a series of such registers, as used herein the term shift register refers to a single stage. A series of these shift registers is referred to herein as either a shift register chain, or a shift register string. The series set of registers is also sometime referred to as a “series array of (shift) registers” or shift register array that may be either a single chain of shift registers or parallel chains of shift registers. For example, the shift registers may be any type of shift register, whether dynamic or latching, whether single clock or master/slave clock, whether sampling or edge trigger, whether data (D), RS, or JK, or a stage of a charge coupled device (CCD), or any other type of device that shifts its input to an output on the basis of clock signal. The shift register arrays/chains may include any number of shift registers without departing from the scope of the invention.
The memory interface device 64 may also receive control and timing signals from the timing circuitry 61 of the section controller 54. These control and timing pulses may be timed such that the data read from or written into a memory device 66 using the respective pulses are read or written in such a manner that the shift registers 76 of the memory interface device maintain their shifting as if only a shift was taking place. For a further description of memory interface devices incorporating shift registers, please see the aforementioned U.S. patent application Ser. No. 10/284,198 by William T. Lynch and David J. Herbison, entitled “Methods and Apparatus for Improved Memory Access,” which was incorporated by reference herein in its entirety. Additionally, data transmitted by a memory interface device 64 may, for example, be transmitted in common mode, differential mode, or in any other manner as deemed appropriate by the system designers.
Exemplary Writing Operation
FIG. 12 illustrates a flow chart for an exemplary writing operation for the memory section 30 of FIG. 5 with reference to FIGS. 1, 6, and 11, in accordance with methods and systems provided. This flow chart illustrates but one example of a writing operation, and other methods may be used for implementing a write operation without departing from the scope of the invention. In this example, the memory devices 66 of the memory section 30 are partitioned into data blocks, where each data block is identifiable by a data block identifier. A more thorough description of data blocks and partitioning of memory devices are presented below.
The switch 22 then may use the data block identifier (DBI) to direct the data request to the memory section 30 that is to store the data block by, for example, determining, based on the DBI, an address for the memory section that the switch 22 uses to route the data request to the memory section (Step 1204). For example, when a data request arrives at an SSCI 204 of a switch 22, the SSCI may, for example, direct the data request to the switch controller 202, which may then, use a table to look up the address corresponding to the DBI, use an algorithm to compute the address from the DBI, or use some other method.
When the memory device 66 becomes available, the memory device 66 signals the microprocessor 51 in the section controller 54 via the memory section control circuitry 55. (Step 1230). This may, for example, be accomplished by the memory device 66 sending an interrupt signal to the microprocessor 51 via the memory device control circuitry 55. Then, the microprocessor 51 sends a message to the memory device 66 through the memory device control circuitry 55 to ready itself for storing the data (Step 1232). When the memory device is ready, the temporary storage memory interface device 60 passes the data to the T-selector 62, which, because this is a write operation, passes the data to the memory interface device 60. For example, if the memory device 66 were available at Step 1222, the data need not be stored in the temporary memory storage device 58. The data is then clocked into the shift register array 76 of the first memory interface device 64-1 where it is clocked through the write chain of shift register arrays 76 until it is loaded into the memory interface device 64 corresponding to the memory device 66 to which the data is to be written (Step 1234). The data is then written to the memory device at an address supplied by the section controller 54 of the memory section 30 (Step 1236).
A more detailed description of the connections between the shift register arrays and the memory devices and a method for writing the data is presented in the aforementioned U.S. patent application Ser. No. 10/284,198 by William T. Lynch and David J. Herbison entitled “Methods and Apparatus of Memory Access” filed on the same day as the present application
Exemplary Reading Operation
FIG. 13 illustrates a flow chart for an exemplary reading operation for the memory section of FIG. 5 with reference to FIGS. 1,6, and 11, in accordance with methods and systems provided. This flow chart illustrates but one example of a read operation and other methods may be used for implementing a read operation without departing from the scope of the invention. In this example, the memory devices 66 of the memory section 30 are partitioned into data blocks, where each data block is identifiable by a data block identifier. A more thorough description of data blocks and partitioning of memory devices is presented below. A read request may originate from a user connected to a server 12 via a network. A device driver 28 in the server then sends the request, including a data block identifier (DBI), to a switch 22 in the storage hub 10 (Step 1302). The device driver 28 may, for example, determine the data block identifier using standard methods. For example, the server 12 may be executing Oracle or a similar type application, which may be used to determine the data block identifier.
If the requested data block is not stored in the memory section 30, the section controller 54 sends a negative acknowledgement (NAK) message (Step 1314), through the CCI 46 and switch 22 to the requesting server 12. After receiving the NAK (Step 1316), the requesting server 12 may attempt to re-read (Step 1318) the data block from the memory section, may attempt to read the device from another device (not shown), or may inform the application. If the section controller 54 verifies that the memory section stores the requested data block, the microprocessor 51 in the section controller 54 determines which memory device 66 stores the data and checks its state to determine if the memory device 66 is busy or available (Step 1320). For example, as discussed above, the microprocessor 52 may store in its internal memory 52 a status code for each memory device 66 in the memory section 30 that the microprocessor 51 may consult to determine the availability of the memory device 66.
If the memory device 66 at step 10 was busy, the read request may be queued in the internal memory 52 of the microprocessor 51 (Step 1322). When the memory device becomes available, it may send an interrupt signal to the section controller 54 (Step 1324), which then executes the read request as described beginning at Step 1326.
A test operation for the embodiment of FIG. 5 will now be described. In certain instances, it may be desirable to test the system using known data. When testing the system, test data and a control signal are sent from the section controller 54 to the T-selector 62 such that T-selector sends the test data to the memory interface devices 64 and the test data may be passed through the system. The test data after being written to and read from the memory interface devices 64 may then be sent to the selector 44, which may be, for example, instructed by the test circuitry 59 to direct the test data to the test circuitry 59. The test circuitry 59 may then check the data using error detection and correction capabilities, such as, for example, parity checks and/or bit level comparisons. If the data are correct, no action is required. If the data are not correct, the test data may be resent. If the data are then correct, no action is required. If not, the section controller 54 may notify the management complex 26 that a fault has occurred and begin to isolate the fault through the error recovery capabilities present in the software it executes. In parallel, the management complex 26 may then execute fault management procedures, such as those discussed above in the section on the management complex 26.
Parallelism and Scalability of Storage Hub
The storage hub 10 may exhibit hierarchical parallelism. The phrase “hierarchical parallelism” as used herein refers to parallelism in the memory section, parallelism between the memory section and the switch fabric, and parallelism among all the memory sections through the switch fabric's connections to servers.
In this example, so long as the requested data are resident in different memory devices 66, the memory section 30 itself may support N simultaneous reads and one write, where N is the number of communications channel connections available to the memory section 30 and preferably does not exceed the number of memory devices 66. For example, as illustrated in FIG. 5, the communications channel interface 46 has 4 communications channels connections for transmitting and receiving information. The switch 22 preferably can handle simultaneous read requests and write requests that it can fulfill. The section controller 54 of the memory section 30 preferably manages the reading and writing of data to the memory devices 66 and manages any conflicts. The section controller 54 of the memory section 30 manages conflicts through the capabilities present in the software it executes. For example, the section controller 54 may direct that write requests have a priority than read requests with the lower priority requests being queued. For example, as previously discussed, the data for write requests may be queued in the temporary storage device 58, and that write and read requests may be queued in the internal memory 52 of the section controller 54. The management complex 26 may direct the section controller 54 to resolve conflicts using other methods such as, for example, a first-to-arrive/a-first-to-be-processed algorithm.
In addition, the storage hub 10, in this example, may also be scalable. More particularly, if increased capacity is demanded from the storage hub 10, this increased capacity may be handled by, for example, adding additional memory sections 30 and cabinets to house them, including backup power, higher capacity and/or additional switches 22, and/or increasing the number and/or capacity of the connections to the storage hub 10. Additionally, as the capacity of the storage hub 10 is increased, the capacity and/or number of management complex processors (32 and 34) may be increased, if necessary, to ensure that the management complex 26 has sufficient capacity to monitor the respective states of the storage hub's memory sections 30
Additionally, in this example, the memory sections 30 may also be scalable. For example, increased capacity of a memory section 30 may be obtained by, for example, adding additional memory devices 66, memory interface devices 64, and/or communications channel interfaces to the memory section. Preferably, the section controller 54 for each memory section includes a sufficiently large resident memory for holding a “map” of the location of each data block resident in its memory section 30. Commercially available microprocessors may be used by the section controller for storing this map.
FIG. 14 illustrates a logical diagram of N memory devices 66, in accordance with methods and systems provided. As discussed above, the memory devices 66 may be solid state memory devices that can store B bits, and the number of bits that each memory device can store may be different. In one example, the management complex 26 may send a command to the section controller 54 of a memory section 30, instructing the section controller 54 to reconfigure a memory device 66 in the memory section 30 into a number of partitions including one or more data blocks each having a particular block size, where the block size is the number of bits in the block. Use of differently sized data blocks and data block partitions allows the storage hub to suit its storage structure to different applications whose data are stored on the same storage hub. For example, an application like an on-line catalog that always stored text and an image or images should preferably prefer a larger block size than an application like a customer reservation system that stored only text. Since the partitions and block sizes can be reconfigured at will through commands from the management complex, new data for new applications may be loaded into the storage hub without have to stop its operations or affect data in other memory sections that are not being changed.
Alternative Memory Interface Device
FIG. 15 illustrates an embodiment wherein the memory interface devices use a common shift register array for reading from and writing to the memory device, in accordance with methods and systems provided. As illustrated, each memory interface device 64 includes a shift register array 78 of a plurality of shift registers 76 interconnected in series. Further, like the embodiment of FIG. 11, these memory interface devices are connected in pairs to form a chain.
The signal is then passed to the read selector 76 of the first memory interface device 64-1 (in this example) in the chain. Because, this is a write operation, the data is clocked into the shift register array 78 and is clocked through the shift registers until it is loaded into the shift register array 78 corresponding to the memory device 66 to which the data is to be written. The data is then written to the memory device 66. Methods and systems for writing data from a shift register array 78 to a memory device 66 are presented in more detail below.
An exemplary reading operation for the embodiment of FIG. 15 will now be described. First, an identifier (e.g., destination address, data block identifier, etc.) for the data is supplied to the read selector 84 from the section controller 54. The destination address is then clocked through the chain of shift register arrays.
As will be obvious to one of skill in the art, other embodiments of the memory interface device are possible, without departing from the scope of the invention. For example, although each memory interface device is described as only having one read or write shift register array, the memory interface device may include any number of write or read chains of shift register arrays. Further, the shift registers 76 rather than being 1 bit shift registers may be of any depth. For example, the shift register arrays could be, for example, N×M arrays such as, for example, 2×8, 4×32, 8×16, etc. arrays as determined by the system designers for their particular implementation. Additionally, the shift registers arrays may be configured in a ring, such that the data once loaded into a chain circulates synchronously in the chain. A more detailed description of memory access using shift register arrays is set forth in the aforementioned U.S. patent application Ser. No. 10/284,198 by William T. Lynch and David J. Herbison entitled “Methods and Apparatus for Improved Memory Access” filed on the same day as the present application.
Alternative Memory Section
FIG. 16 illustrates an alternative memory section 30, in accordance with methods and systems provided. These memory sections are provided as exemplary mechanisms that could be employed to practice the invention, and are not intended to limit the scope of the claims.
The following provides a brief overview of an example for a reading operation for the memory section of FIG. 14. In this example, the communications channel interfaces 46 are preferably fibre channel I/O components. When a data request arrives at the communications channel interface 46, the communication channel interface 46 detects it and sends an interrupt signal to the section controller 54. This interrupt signal preferably includes information regarding the data block to be read from the memory devices. The section controller 54 then maps this data block information to an address in the memory devices. That is, the section controller determines from this data block information the memory devices 66 storing the requested data along with the addresses for this data on those memory devices. The section controller 54 then loads this address information into its internal memory, such that the addresses are transferred to the memory devices as in the above-describe memory latching example.
2. A method for use in a storage system, comprising:
storing data in storage locations in a memory device, the memory device included in a memory section;
determining, by a management system, a routing algorithm for use by a switch controller that executes software, including the routing algorithm;
providing, by the management system, the routing algorithm to the switch controller;
executing, by the switch controller, the routing algorithm, to configure a configurable switch connecting the memory section to an interface;
detecting a fault associated with the data in the storage locations in the memory device;
determining, by the management system in response to the detecting, a new routing algorithm that redirects data for the memory device to a replacement memory device; and
providing the new routing algorithm to the switch controller.
transmitting, by the management system, information regarding the fault and the new routing algorithm to an external management system.
storing a copy of the data in the memory device in a non-volatile storage device;
instructing, by the management system, the non-volatile storage device to load the copy of the data into the replacement memory device via the configurable switch; and
storing the copy of the data from the non-volatile storage device into the replacement memory device.
receiving, by a memory interface device in the memory section, data read from the memory device;
receiving, by the memory interface device in the memory section, an identifier for use by the switch controller in directing the data;
combining, by the memory interface device, the data read from the memory device and the identifier to produce combined data, and
forwarding, by the memory interface device, the combined data to the configurable switch.
receiving, by the memory section, the data to be stored in the storage locations in the memory device;
storing, by the temporary storage interface device, the data in a temporary storage device if the memory device to which the data is to be stored is busy;
retrieving, by the temporary storage interface device, the data from the temporary storage device when the memory device to which the data is to be stored is no longer busy; and
storing the data in the memory device.
means for storing data in storage locations; and
means for detecting a fault, in regard to the data stored by the means for storing, and transmitting a fault message, in response to the fault;
programmable means for forming connections between the means for storing and one or more interfaces according to a routing algorithm executed by the programmable means for forming connections;
means for receiving the fault message;
means for removing from service the means for storing associated with the fault message by changing the routing algorithm executed by the programmable means for forming connections.
a memory section controller capable of detecting a fault in the memory section and transmitting a fault message corresponding to the detected fault;
a switch controller that executes a routing algorithm; and
a selectively configurable switch fabric connected to the one or more memory sections and the one or more interfaces that provides a transmission path between the one or more memory sections and the one or more interfaces according to the routing algorithm executed by the switch controller; and
a management system that receives the fault message from the memory section controller and inactivating the memory section corresponding to the fault message by changing the routing algorithm executed by the switch controller.
9. The storage system of claim 8 wherein the switch is connected to a non-volatile storage device and wherein the management system instructs the non-volatile storage device to load data into the one or more of the memory sections via the switch.
10. The storage system of claim 8 further including an interface that connects the management system to an external management system such that configuration management may be performed through the external management system.
11. The storage system of claim 8 wherein the one or more memory sections includes a memory interface device that receives data from a memory device and receives an identifier from the memory section controller, the memory interface device comprising circuitry that combines the data from the memory device with the identifier.
12. The storage system of claim 8 wherein the management system includes:
one or more control processors for determining and providing the routing algorithm to the switch controller; and
one or more administration processors for collecting statistical data from the one or more switches and the one or more memory sections.
13. The storage system of claim 8 wherein at least one of the one or more memory sections further includes:
a temporary storage interface device that stores data into and retrieves data from the temporary storage device;
wherein at least one of the one or more switches forwards data to be stored in a memory section to the temporary storage interface device,
wherein the temporary storage interface device stores the data in the temporary storage device if a memory device to which the data is to be written is busy, and
wherein the temporary storage interface device retrieves the data from the temporary storage device when the memory device is no longer busy and forwards the data to the memory device.
14. A method for use in a storage system comprising:
determining, by a management system, a routing algorithm used by a switch controller that executes the routing algorithm to configure a configurable switch that connects the memory section to an interface;
connecting, by the configurable switch, the memory section to the interface according to the routing algorithm executed by the switch controller;
detecting, by a memory section controller, an error associated with the memory device and transmitting a fault message indicating the error and the memory device to the management system;
receiving, by the management system, the memory device indicated by the fault message; and
removing from service the memory device indicated by the fault message by changing the routing algorithm.
instructing, by the management system, the non-volatile storage device to load the copy of the data via the configurable switch into a replacement memory device corresponding to the memory device indicated by the fault message; and
storing the copy of the data from the non-volatile storage device in the replacement memory device.
US12476212 2002-10-31 2009-06-01 Methods and systems for a storage system Active - Reinstated 2022-12-24 US7958388B2 (en)
US10284199 US7197662B2 (en) 2002-10-31 2002-10-31 Methods and systems for a storage system
US11710407 US7543177B2 (en) 2002-10-31 2007-02-26 Methods and systems for a storage system
US12476212 US7958388B2 (en) 2002-10-31 2009-06-01 Methods and systems for a storage system
US20090240976A1 true US20090240976A1 (en) 2009-09-24
US7958388B2 true US7958388B2 (en) 2011-06-07
ID=32174818
US10284199 Expired - Fee Related US7197662B2 (en) 2002-10-31 2002-10-31 Methods and systems for a storage system
US11710407 Active - Reinstated US7543177B2 (en) 2002-10-31 2007-02-26 Methods and systems for a storage system
US12476212 Active - Reinstated 2022-12-24 US7958388B2 (en) 2002-10-31 2009-06-01 Methods and systems for a storage system
US (3) US7197662B2 (en)
US9336288B2 (en) 2013-06-03 2016-05-10 Bank Of America Corporation Workflow controller compatibility
US9460188B2 (en) 2013-06-03 2016-10-04 Bank Of America Corporation Data warehouse compatibility
US7802155B2 (en) * 2000-01-06 2010-09-21 Super Talent Electronics, Inc. Non-volatile memory device manufacturing process testing systems and methods thereof
US7873885B1 (en) * 2004-01-20 2011-01-18 Super Talent Electronics, Inc. SSD test systems and methods
WO2004107280A3 (en) 2003-05-28 2005-08-18 Ewi Holdings Inc System and method for electronic prepaid account replenishment
CN100352196C (en) * 2003-12-30 2007-11-28 国际商业机器公司 System and method for transmitting data pocket in infinite bandwidth network
US20050180352A1 (en) * 2004-02-18 2005-08-18 Trifus N.A., Llc Hub-based communications method and apparatus
WO2005101206A3 (en) * 2004-04-12 2006-03-02 Teeznar Corp Peer-to-peer distributed computational system and method
JP4500090B2 (en) * 2004-04-22 2010-07-14 株式会社日立製作所 Information management systems and information management method
US20060045244A1 (en) * 2004-08-24 2006-03-02 Darren New Method and apparatus for receipt printing and information display in a personal identification number delivery system
US7280644B2 (en) * 2004-12-07 2007-10-09 Ewi Holdings, Inc. Transaction processing platform for faciliating electronic distribution of plural prepaid services
JP4387968B2 (en) 2005-03-28 2009-12-24 富士通株式会社 Fault detection apparatus and fault detection method
US20090049167A1 (en) * 2007-08-16 2009-02-19 Fox David N Port monitoring
US8397011B2 (en) * 2007-10-05 2013-03-12 Joseph Ashwood Scalable mass data storage device
US8041991B2 (en) * 2008-11-18 2011-10-18 Lsi Corporation System and method for recovering solid state drive data
US8543863B2 (en) * 2009-11-18 2013-09-24 Microsoft Corporation Efficiency of hardware memory access using dynamically replicated memory
CN101753740A (en) * 2009-11-20 2010-06-23 华为终端有限公司 Communication method and gateway equipment based on SIP telephone
JP5640543B2 (en) * 2010-08-10 2014-12-17 富士通株式会社 Storage system and control method thereof
US8458514B2 (en) 2010-12-10 2013-06-04 Microsoft Corporation Memory management to accommodate non-maskable failures
CN102571478B (en) * 2010-12-31 2016-05-25 上海宽惠网络科技有限公司 server
US8966172B2 (en) * 2011-11-15 2015-02-24 Pavilion Data Systems, Inc. Processor agnostic data storage in a PCIE based shared storage enviroment
US8554963B1 (en) * 2012-03-23 2013-10-08 DSSD, Inc. Storage system with multicast DMA and unified address space
CN103995481A (en) * 2013-02-20 2014-08-20 阿斯科动力科技公司 Backup apparatus of transfer-switch controllers and transfer-switch controllers
US9667530B2 (en) * 2013-05-06 2017-05-30 International Business Machines Corporation Privacy preserving query method and system for use in federated coalition networks
GB201317944D0 (en) 2013-10-10 2013-11-27 Ibm Linear network coding in a dynamic distributed federated database
US6185644B2 (en)
US3713096A (en) 1971-03-31 1973-01-23 Ibm Shift register interconnection of data processing system
US3735362A (en) 1971-09-22 1973-05-22 Ibm Shift register interconnection system
US3748647A (en) 1971-06-30 1973-07-24 Ibm Toroidal interconnection system
US3812476A (en) 1971-11-16 1974-05-21 Texas Instruments Inc Memory device
US4064556A (en) 1975-06-23 1977-12-20 Sperry Rand Corporation Packed loop memory with data manipulation capabilities
US4065756A (en) 1976-03-15 1977-12-27 Burroughs Corporation Associative memory with neighboring recirculated paths offset by one bit
US4193121A (en) 1976-07-16 1980-03-11 Post Office Information handling apparatus having multiple ports distributed around shifting register rings
US4302632A (en) 1976-10-18 1981-11-24 Graphic Scanning, Inc. Digitally-controlled telephone answering switching system
US4334305A (en) 1979-02-15 1982-06-08 Cselt - Centro Studi E Laboratori Telecomunicazioni S.P.A. Data-handling system with operatively interconnected processors
US4510599A (en) 1983-04-04 1985-04-09 General Electric Company Prioritized unidirectional distributed bus accessing system
US4790418A (en) 1987-04-30 1988-12-13 Ford Motor Company Transmission clutch loop transfer control
US4984240A (en) 1988-12-22 1991-01-08 Codex Corporation Distributed switching architecture for communication module redundancy
US5060068A (en) 1988-10-31 1991-10-22 James E. Lindstrom Audio/video distribution system
US5062059A (en) 1986-12-08 1991-10-29 Sunriver Corporation Apparatus and method for communication between host CPU and remote terminal
US5084839A (en) 1990-02-05 1992-01-28 Harris Corporation Variable length shift register
US5119481A (en) 1987-12-22 1992-06-02 Kendall Square Research Corporation Register bus multiprocessor system with shift
US5200925A (en) 1988-07-29 1993-04-06 Mitsubishi Denki Kabushiki Kaisha Serial access semiconductor memory device and operating method therefor
US5581479A (en) 1993-10-15 1996-12-03 Image Telecommunications Corp. Information service control point, which uses different types of storage devices, which retrieves information as blocks of data, and which uses a trunk processor for transmitting information
WO1997007458A1 (en) 1995-08-15 1997-02-27 Emc Corporation Data storage system
US5737510A (en) * 1993-11-30 1998-04-07 Hitachi, Ltd. Disc array system having disc storage devices dispersed on plural boards and accessible at withdrawal of part of the boards
US5771367A (en) 1992-12-17 1998-06-23 International Business Machines Corporation Storage controller and method for improved failure recovery using cross-coupled cache memories and nonvolatile stores
US5883831A (en) 1996-09-30 1999-03-16 Sgs-Thomson Microelectronics S.A. Serial access memory with securing of write operations
US5909564A (en) 1997-03-27 1999-06-01 Pmc-Sierra Ltd. Multi-port ethernet frame switch
US5978295A (en) 1997-07-04 1999-11-02 Stmicroelectronics S.A. Sequential access memories
US6317377B1 (en) 1999-04-12 2001-11-13 Nec Corporation Semiconductor memory device
US6498741B2 (en) 1998-12-22 2002-12-24 Kabushiki Kaisha Toshiba Semiconductor memory device
US20030018930A1 (en) 2001-07-18 2003-01-23 Oscar Mora Peer-to-peer fault detection
US6587909B1 (en) 1996-06-05 2003-07-01 Hewlett-Packard Development Company, L.P. Installation and removal of components of a computer
US20040044744A1 (en) 2000-11-02 2004-03-04 George Grosner Switching system
WO2004025476A1 (en) 2002-09-16 2004-03-25 Tigi Corporation Storage system architectures and multiple caching arrangements
US20040068561A1 (en) 2002-10-07 2004-04-08 Hitachi, Ltd. Method for managing a network including a storage system
US6728799B1 (en) 2000-01-13 2004-04-27 Hewlett-Packard Development Company, L.P. Hybrid data I/O for memory applications
US20040088477A1 (en) 2002-10-31 2004-05-06 Bullen Melvin James Methods and systems for a memory section
US20040085818A1 (en) 2002-10-31 2004-05-06 Lynch William Thomas Methods and apparatus for improved memory access
US20040088514A1 (en) 2002-10-31 2004-05-06 Bullen Melvin James Methods and systems for a storage system including an improved switch
US20040168101A1 (en) 2002-04-09 2004-08-26 Atsushi Kubo Redundant memory system and memory controller used therefor
US6795937B2 (en) 2000-10-03 2004-09-21 Sun Microsystems, Inc. Multiple traps after faulty access to a resource
JPS5122794B1 (en) * 1970-06-24 1976-07-12
GB8514905D0 (en) * 1985-06-12 1985-07-17 Ici Plc Colouration process
US4762317A (en) * 1987-05-04 1988-08-09 Roadmaster Corporation Stationary exercise device
US5060058A (en) * 1989-06-07 1991-10-22 U.S. Philips Corporation Modulation system for projection display
US6870936B1 (en) * 1998-10-20 2005-03-22 Edward Y. Ajamian Control platform for multiple signal routing and interfacing in an audio/visual environment
US6334305B1 (en) * 1999-04-28 2002-01-01 MAGNETI MARELLI S.p.A. Self-adapting method for controlling titre in an injection unit for an internal combustion engine
US6032214A (en) 1990-04-18 2000-02-29 Rambus Inc. Method of operating a synchronous memory device having a variable data output length
US5636139A (en) 1993-10-15 1997-06-03 Image Telecommunications Corp. Information service control point which retreives information as blocks of data
US5604682A (en) 1993-10-15 1997-02-18 Image Telecommunications Corp. Information service control point which retrieves information as blocks of data and outputs the retrieved data via a communications link
"170 MHz FIFOs Using the Virtex Block SelectRAM+ Features" Xilinx Application Note XAPP131, Jun. 5, 2001 (6 pages).
"200-MHz SDRAM Controller Core Product Specification" Alliance Core, Jan. 10, 2000 (4 pages).
"74F1763 Intelligent DRAM Controller Product Information" Philips Semiconductors 2002 (2 pages).
"API Networks Accelerates Use of HyperTransport(TM) Technology With Launch of Industry's First HyperTransport-to-PCI Bridge Chip" API Networks Press Release, Apr. 2, 2001 (2 pages).
"API Networks Unveils Industry's First HyperTransport(TM) Switch to Bring Products to Market Quickly and Cost-Effectively" API Networks Press Release, Nov. 5, 2001 (4 pages).
"API Networks, enabling technology for next generation product . . . HyperTransport technology licensed by HP" (retrieved Mar. 18, 2002) , API Networks, Dec. 10, 2001 (2 pages).
"Asynchronous Operation" (retrieved Mar. 17, 2002), (2 pages).
"Basic DRAM Cell" (retrieved Mar. 17, 2002), MIT, Jun. 26, 1996 (1 page).
"Basic DRAM Operation" (retrieved Mar. 17, 2002) (2 pages).
"Block SelectRAM Overview" (retrieved Mar. 18, 2002) <http://www.xilinx.com/xil-prodcat-systemsolution.jsp?title=xam-memory-embedded-blockram-pag> (2 pages).
"Column Decode and Refresh Control Logic" (retrieved Mar. 17, 2002) , MIT, Jun. 26, 1996 (1 page).
"Comments" (retrieved Mar. 17, 2002), MIT, Jun. 26, 1995 (1 page).
"Data-In Tristate Buffer" (retrieved Mar. 17, 2002) , MIT, Jun. 26, 1996 (1 page).
"Data-out Precharging Circuits and Control Circuits" (retrieved Mar. 17, 2002) , MIT, Jun. 26, 1996 (1 page).
"DDR SDRAM" (retrieved Mar. 17, 2002), (2 pages).
"HHSPICE Verification of Latch" (retrieved Mar. 17, 2002), , MIT, Jun. 26, 1996 (1 page).
"HyperTransport(TM) I/O Link Specification," HyperTransport Technology Consortium, Rev. 1.03, Oct. 10, 2001 (217 pages).
"HyperTransport-to-PCI Bridge Chip from API Networks" Cahners Business Information 2002 (1 page).
"InfiniBridge MT21108 Infiniband Channel Adapter and Eight Port Switch" Product Specification, Mellanox Technologies, (date unknown) (4 pages).
"Introduction" (retrieved Mar. 17, 2002) (2 pages).
"Other Circuits" (retrieved Mar. 17, 2002) , MIT, Jun. 26, 1996 (1 page).
"Output Multiplexer" (retrieved Mar. 17, 2002), , MIT, Jun. 26, 1996 (1 page).
"Refresh Address Control" (retrieved Mar. 17, 2002) , MIT, Jun. 26, 1996 (1 page).
"Refresh Circuit" (retrieved Mar. 17, 2002) , MIT, Jun. 26, 1996 (1 page).
"Row Address Decoder and Row Driver" (retrieved Mar. 17, 2002) , MIT, Jun. 26, 1996 (1 page).
"Sections 5.4.1-5.4.5 of document describing DRAM and DRAM Controller" (author and data unknown).
"Simulations" (retrieved Mar. 17, 2002) , MIT, Jun. 26, 1996 (1 page).
"Synchronous Operation" (retrieved Mar. 17, 2002) (2 pages).
"Synthesizable High Performance SDRAM Controller" Xilinx Application Note XAPP134, Feb. 1, 2001 (16 pages).
"Using Block SelectRAM+ Memory in Spartan-II FPGAs," Xilinx Application Note XAPP173, Dec. 11, 2000 (15 pages).
"Using the Virtex Block SelectRAM+ Features" Xilinx Application Note XAPP130, Dec. 18, 2000 (11 pages).
"API Networks Accelerates Use of HyperTransport™ Technology With Launch of Industry's First HyperTransport-to-PCI Bridge Chip" API Networks Press Release, Apr. 2, 2001 (2 pages).
"API Networks Unveils Industry's First HyperTransport™ Switch to Bring Products to Market Quickly and Cost-Effectively" API Networks Press Release, Nov. 5, 2001 (4 pages).
"API Networks, enabling technology for next generation product . . . HyperTransport technology licensed by HP" (retrieved Mar. 18, 2002) <http://www.api-networks.com/pressreleases/pr121001.shtml>, API Networks, Dec. 10, 2001 (2 pages).
"Asynchronous Operation" (retrieved Mar. 17, 2002), <http://www4.tomshardware.com/mainboard/98q4/981024/ram-02.html> (2 pages).
"Basic DRAM Cell" (retrieved Mar. 17, 2002)<http://www.mit.edu:8001/people/tairan/6371/node59.html>, MIT, Jun. 26, 1996 (1 page).
"Basic DRAM Operation" (retrieved Mar. 17, 2002) <http://www4.tomshardware.com/mainboard/98q4/981024/ram-01.html> (2 pages).
"Block SelectRAM Overview" (retrieved Mar. 18, 2002) <http://www.xilinx.com/xil—prodcat—systemsolution.jsp?title=xam—memory—embedded—blockram—pag> (2 pages).
"Column Decode and Refresh Control Logic" (retrieved Mar. 17, 2002) <http://www.mit.edu:8001/people/tairan/6371/node61.html>, MIT, Jun. 26, 1996 (1 page).
"Comments" (retrieved Mar. 17, 2002)< http://www.mit/edu:8001/people/tairan/6371/node 70.html>, MIT, Jun. 26, 1995 (1 page).
"Data-In Tristate Buffer" (retrieved Mar. 17, 2002) <http://www.mit.edu:8001/people/tairan/6371/node64.html>, MIT, Jun. 26, 1996 (1 page).
"Data-out Precharging Circuits and Control Circuits" (retrieved Mar. 17, 2002) <http://www.mit.edu:8001/people/tairan/6371/node65.html>, MIT, Jun. 26, 1996 (1 page).
"DDR SDRAM" (retrieved Mar. 17, 2002), <http://www4.tomshardware.com/mainboard/98q4/981024/ram-10.html> (2 pages).
"HHSPICE Verification of Latch" (retrieved Mar. 17, 2002), <http://www.mit.edu:8001/people/tairan/6371/node73.html>, MIT, Jun. 26, 1996 (1 page).
"HyperTransport™ I/O Link Specification," HyperTransport Technology Consortium, Rev. 1.03, Oct. 10, 2001 (217 pages).
"Introduction" (retrieved Mar. 17, 2002) <http://www4.tomshardware.com/mainboard/98q4/981024/index.html> (2 pages).
"Other Circuits" (retrieved Mar. 17, 2002) <http:/www.mit.edu:8001/people/tairan/6371/node67.html>, MIT, Jun. 26, 1996 (1 page).
"Output Multiplexer" (retrieved Mar. 17, 2002), <http://www.mit.edu:8001/people/tairan/6371/node66.html>, MIT, Jun. 26, 1996 (1 page).
"Refresh Address Control" (retrieved Mar. 17, 2002) <http://www.mit.edu:8001/people/tairan/6371/node63.html>, MIT, Jun. 26, 1996 (1 page).
"Refresh Circuit" (retrieved Mar. 17, 2002) <http://www.mit/edu:8001/people/tairan/6371/node62.html>, MIT, Jun. 26, 1996 (1 page).
"Row Address Decoder and Row Driver" (retrieved Mar. 17, 2002) <http://www.mit.edu:8001/people/tairan/6371/node60.html>, MIT, Jun. 26, 1996 (1 page).
"Simulations" (retrieved Mar. 17, 2002) <http://www.mit.edu:8001/people/tairan/6371/node68.html>, MIT, Jun. 26, 1996 (1 page).
"Synchronous Operation" (retrieved Mar. 17, 2002) <http://www4.tomshardware.com/mainboard/98q4/981024/ram-07.html> (2 pages).
Bapat, Sheker, "Synthesizable 200 MHz ZBT SRAM Interface" Xilinix Application Note XAPP136, Jan. 10, 2000 (6 pages).
Clark, Tom, "Designing Storage Area Networks-A Practical Reference for Implementing Fibre Channel SANs," Addison-Wesley, 1999.
Clark, Tom, "Designing Storage Area Networks—A Practical Reference for Implementing Fibre Channel SANs," Addison-Wesley, 1999.
Communication Pursuant to Article 94(3) EPC for European Application No. 03 777 844.6, dated Feb. 11, 2008 (10 pages).
Communication Pursuant to Article 94(3) EPC for European Application No. 03 777 844.6, dated Oct. 28, 2008 (6 pages).
Communication Pursuant to Article 94(3) EPC for European Application No. 03 810 796.7-2210, dated Apr. 25, 2008 (4 pages).
Communication Pursuant to Article 94(3) EPC for European Application No. 03 810 796.7-2210, dated Jan. 29, 2009 (3 pages).
European Patent Office communication pursuant to Art. 96(2) EPC, dated Mar. 27, 2007, for European Application No. 38 810 796.7 (4 pages).
European Patent Office Communication Under Rule 71(3) EPC for EPO Application No. 03 777 844.6, dated Oct. 28, 2008 (6 pages).
Extended European Search Report, dated Apr. 8, 2009, for European Application No. 09153548.4 (12 pages).
Farley, Marc, "Building Storage Networks, Second Edition," Osbourne/McGraw-Hill, 2001, entire book.
International Search Report, dated Jan. 14, 2005, for International Application No. PCT/US03/33665.
International Search Report, dated May 27, 2004, for International Application No. PCT/US03/33679 (3 pages).
Ma, Alex, "Synchronous DRAM Controller" Powerpoint slides, EE527 Spring 1998 (21 pages).
Malavalli, Kumar, "Fibre Channel Classes of Service for Data Transport," Brocade Communications Services, Inc. 1997 (15 pages).
Mauro, Douglas R., et al., "Essential SNMP," O'Reilly & Associates, Inc., Jul. 2001.
McComas, Bert "PCI-X or InfiniBand Complementary New Technologies Go Head to Head" (retrieved Mar. 18, 2002) , Inquest Market Research, Jan. 19, 2001 (10 pages).
McComas, Bert "PCI-X or InfiniBand Complementary New Technologies Go Head to Head" (retrieved Mar. 18, 2002) <http://www.inqst.com/articles/pcixvib/pciarticle.htm>, Inquest Market Research, Jan. 19, 2001 (10 pages).
Notice of Allowance and Allowability, dated Jun. 24, 2008, for U.S. Appl. No. 10/284,268 (6 pages).
Notice of Allowance and Allowability, dated Jun. 28, 2007, for U.S. Appl. No. 11/030,881 (4 pages.) with Reply, filed May 8, 2007, listing allowed claims (13 pages).
Notice of Allowance and Allowability, dated Mar. 23, 2009, for U.S. Appl. No. 11/710,407 (6 pages.) with Reply, filed Feb. 23, 2009, listing allowed claims (14 pages).
Notice of Allowance and Allowability, dated Mar. 9, 2009, for U.S. Appl. No. 11/806,012 (6 pages.) with Reply, filed Dec. 4, 2008, listing allowed claims (14 pages).
Notice of Allowance and Allowability, dated May 1, 2007, for U.S. Appl. No. 10/284,278 (6 pages.) with Suppl. Amendment filed Jan. 25, 2007, listing allowed claims (14 pages).
Office Action, dated Apr. 11, 2007, for U.S. Appl. No. 10/284,268 (5 pages).
Office Action, dated Apr. 24, 2006, for U.S. Appl. No. 10/284,268 (16 pages).
Office Action, dated Apr. 24, 2006, for U.S. Appl. No. 10/284,278 (12 pages).
Office Action, dated Aug. 4, 2008, for U.S. Appl. No. 11/806,012 (7 pages).
Office Action, dated Dec. 31, 2007, for U.S. Appl. No. 10/284,268 (15 pages).
Office Action, dated Feb. 28, 2007, for U.S. Appl. No. 11/030,881 (7 pages).
Office Action, dated Jan. 11, 2006, for U.S. Appl. No. 11/030,881 (5 pages).
Office Action, dated Jul. 10, 2006, for U.S. Appl. No. 11/030,881 (5 pages).
Office Action, dated Jul. 27, 2005, for U.S. Appl. No. 11/030,881 (8 pages).
Office Action, dated Jun. 15, 2006, for U.S. Appl. No. 10/284,199 (12 pages).
Office Action, dated Mar. 18, 2005, for U.S. Appl. No. 10/284,268 (14 pages).
Office Action, dated Mar. 18, 2005, for U.S. Appl. No. 10/284,278 (12 pages).
Office Action, dated Mar. 27, 2008, for U.S. Appl. No. 11/710,407 (10 pages).
Office Action, dated Oct. 23, 2008, for U.S. Appl. No. 11/710,407 (11 pages).
Office Action, dated Sep. 21, 2005, for U.S. Appl. No. 10/284,268 (13 pages).
Office Action, dated Sep. 26, 2005, for U.S. Appl. No. 10/284,278 (13 pages).
Preston, W. Curtis, "Using SANs and NAS," O'Reilly & Associates, Inc., Feb. 2002.
Prince, Betty, "High Performance Memories New Architecture DRAMs and SRAMs-evolution and function Revised Edition," John Wiley & Sons, Ltd., 1996, entire book.
Prince, Betty, "High Performance Memories New Architecture DRAMs and SRAMs—evolution and function Revised Edition," John Wiley & Sons, Ltd., 1996, entire book.
RDRAM® Memory: Leading Performance and Value Over SDRAM and DDR, Document WP0001-R, Version 1.2, 2001.
RDRAM®: Maximizing the Value of PCs and Workstations, Document WP0003-R, Version 1.0, 2001.
Richmond, Robert, "AMD 64-Bit K8 Platform Preview" (retrieved Mar. 11, 2002) Sep. 14, 2000 (4 pages).
Supplemental European Search Report for European Application No. 03 777 844.6, dated Oct. 12, 2007 (26 pages).
The Economist, Jun. 22, 2002, 28th edition, A Match for Flash?, pp. 22-23.
Tran, Jennifer, "Synthesizable 1.6 Gbytes/s DDR SRAM Controller" Xilinx Application Note XAPP200, Mar. 21, 2000 (12 pages).
US20070174646A1 (en) 2007-07-26 application
US20040088393A1 (en) 2004-05-06 application
US7197662B2 (en) 2007-03-27 grant
US20090240976A1 (en) 2009-09-24 application
US7543177B2 (en) 2009-06-02 grant
US6378021B1 (en) 2002-04-23 Switch control method and apparatus in a system having a plurality of processors
US6725393B1 (en) 2004-04-20 System, machine, and method for maintenance of mirrored datasets through surrogate writes during storage-area network transients
US6766470B1 (en) 2004-07-20 Enhancing reliability and robustness of a cluster
US20040148376A1 (en) 2004-07-29 Storage area network processing device
US20060190552A1 (en) 2006-08-24 Data retention system with a plurality of access protocols
US20050021606A1 (en) 2005-01-27 Network storage appliance with integrated redundant servers and storage controllers
Owner name: RING TECHNOLOGY ENTERPRISES, INC., VIRGINIA
Free format text: CHANGE OF NAME;ASSIGNOR:RING TECHNOLOGY ENTERPRISES, LLC;REEL/FRAME:023639/0460
Owner name: RING TECHNOLOGY ENTERPRISES OF TEXAS, LLC, TEXAS
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:RING TECHNOLOGY ENTERPRISES, INC.;REEL/FRAME:023639/0572
Owner name: RING TECHNOLOGY ENTERPRISES, INC.,VIRGINIA
Owner name: RING TECHNOLOGY ENTERPRISES OF TEXAS, LLC,TEXAS
Owner name: PARRALLEL IRON LLC, TEXAS
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:RING TECHNOLOGY ENTERPRISES OF TEXAS, LLC;REEL/FRAME:025660/0089
Owner name: PARALLEL IRON LLC, TEXAS
Free format text: CORRECTIVE ASSIGNMENT TO CORRECT THE THE SPELLING OF THE ASSIGNEE NAME IS INCORRECT. PREVIOUSLY RECORDED ON REEL 025660 FRAME 0089. ASSIGNOR(S) HEREBY CONFIRMS THE IT IS CURRENTLY SPELLED PARRALLEL IRON LLC SHOULD BE PARALLEL IRON LLC.;ASSIGNOR:RING TECHNOLOGY ENTERPRISES OF TEXAS, LLC;REEL/FRAME:025746/0410
Owner name: PARALLEL IRON, LLC, DELAWARE
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:PARALLEL IRON, LLC;REEL/FRAME:028392/0931
2015-06-07 REIN Reinstatement after maintenance fee payment confirmed
Owner name: MOBILE NETWORKING SOLUTIONS, LLC, TEXAS
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:PARALLEL IRON, LLC;REEL/FRAME:041446/0006
2017-03-06 PRDP Patent reinstated due to the acceptance of a late maintenance fee
2017-03-09 SULP Surcharge for late payment