Source: http://www.google.com/patents/US7760626?dq=6437692
Timestamp: 2016-06-27 12:01:59
Document Index: 335541713

Matched Legal Cases: ['Application No. 200480041815', 'art 11', 'art 11', 'art 11', 'art 11', 'art 11', 'art 3', 'art 3']

Patent US7760626 - Load balancing and failover - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inPatentsProvided are techniques for static load balancing. For each data path in a network adapter team, a load balancing value is computed. A maximum value of the computed load balancing values is determined. A data path with the maximum value is selected for use in routing data. Also provided are techniques...http://www.google.com/patents/US7760626?utm_source=gb-gplus-sharePatent US7760626 - Load balancing and failoverAdvanced Patent SearchPublication numberUS7760626 B2Publication typeGrantApplication numberUS 10/815,349Publication dateJul 20, 2010Priority dateMar 31, 2004Fee statusPaidAlso published asUS20050259632Publication number10815349, 815349, US 7760626 B2, US 7760626B2, US-B2-7760626, US7760626 B2, US7760626B2InventorsNavneet Malpani, Xuebin Yao, Charles A. MustaOriginal AssigneeIntel CorporationExport CitationBiBTeX, EndNote, RefManPatent Citations (68), Non-Patent Citations (86), Referenced by (17), Classifications (13), Legal Events (4) External Links: USPTO, USPTO Assignment, EspacenetLoad balancing and failover
A Host Bus Adapter (HBA) is a hardware device that “connects” the operating system of a host computer and a communication path (e.g., a Small Computer System Interface (SCSI) (American National Standards Institute (ANSI) SCSI Controller Commands-2 (SCC-2) NCITS.318:1998) bus). The HBA manages the transfer of data between the host computer and the communication path.
Internet Small Computer Systems Interface (iSCSI) is a protocol (IETF RFC 3347, published February 2003; IETF Draft, published Jan. 19, 2003) that defines a technique for transporting SCSI commands/data to and from I/O devices across TCP (“Transmission Control Protocol”)-enabled networks (Internet Engineering Task Force (IETF) Request for Comments (RFC) 793, published September 1981). As such, iSCSI acts as a bridge between two independently designed protocols that have significantly different tolerances for, and facilities for, detecting and recovering from network congestion and from errors. iSCSI permits the existence of multiple parallel data paths to the same storage target.
HBA teaming refers to grouping together several HBAs to form a “team,” where each HBA in a team is connected to a particular target and may route data to that target. HBA teams may be built on an Internet Small Computer System Interface (iSCSI) portal group concept. A portal group concept may be described as a collection of network portals within an iSCSI Network Entity that collectively support the capability of coordinating a session with connections spanning these portals. HBA teaming may be used with Small Computer System Interface (SCSI) initiators running a Linux� operating system.
The data storage 140 is connected to network 170 and includes network adapters 142 a, 142 b. Network adapter 142 a includes a bus controller 144 a and a physical communications layer 146 a. Network adapter 142 b includes a bus controller 144 b and a physical communications layer 146 b. The data storage 140 includes one or more logical units (i.e., “n” logical units, where “n” may be any positive integer value, which in certain embodiments, is less than 128). Merely for ease of understanding, logical unit 0, logical unit 1, and logical unit “n” are illustrated. Each logical unit may be described as a separate storage device. Additionally, a logical unit number (LUN) is associated with each logical device. In certain embodiments, a network adapter team is organized based on the target and LUN (i.e., each network adapter that can route data to a particular LUN of a target is grouped into one network adapter team), and one network adapter may belong to different network adapter teams.
FIG. 4 illustrates operations for static load balancing in accordance with certain embodiments. Control begins at block 400 with receipt by the load balancing component 122 of input parameters for static load balancing. The input parameters may include, for example, a list of the data paths in the team, a total number of bytes transferred by the team (also referred to as “TotalTeamBytes”), a load balancing share of each data path (also referred to as “LoadBalancingShare”), and a number of bytes transferred on each data path (also referred to as “DataPathBytes”). The TotalTeamBytes may be described as a total number of bytes sent across all data paths for one or more commands, while DataPathBytes may be described as a number of bytes sent across a particular data path so far for one or more commands. For static load balancing, the load balancing component 122 assumes that each data path in a team has an associated value, which will be referred to as a “load balancing share” for ease of reference, that represents the percentage of a total I/O (read or write) workload that a given data path in the team can take. The overall sum of the load balancing share values for the team is 100%.
send SCSI command on this data path End For loop FIGS. 5A and 5B (whose relationship is shown in FIG. 5) illustrate operations for dynamic load balancing in accordance with certain embodiments. Dynamic load balancing ensures that a single data path does not become congested, as long as there is available bandwidth on other data paths, by dynamically adjusting I/O workload among data paths. Thus, dynamic load balancing ensures that a storage system's total available bandwidth is readily accessible by clients. In FIG. 5, it can be seen that processing may flow from FIG. 5A to FIG. 5B and then back to FIG. 5A. In FIG. 5A, control begins at block 500 with global parameters being set by, for example, a system administrator or other user. In particular, dynamic load balancing uses three global parameters: a timer interval value (also referred to as “TimerInterval”) that determines when a timer (e.g., a kernel timer) fires in order to dynamically set the load balancing parameters based on the current congestion; a change threshold value (also referred to as “ChangeThreshold”), which is a value to determine whether congestion on a data path exists; and a load balancing change percent value (also referred to as “LBChange %”), that is used to determine how much of the load balancing share to decrease for the congested data path (if congestion exists).
In block 504, the load balancing component 122 determines that the timer has fired. The load balancing component 122 implements dynamic load balancing when the timer fires every “TimerInterval” value. At the TimerInterval time, load balancing shares are recomputed. In certain embodiments, during the length of the interval, a SCSI command is directed on a data path based on the recomputed values using static load balancing.
In block 506, the load balancing component 122 computes an actual load balancing share (also referred to as “ActualLBShare”) and a difference load balancing value (also referred to as “DifferenceLB”) for each data path in the team for the selected data path. The actual load balancing share is computed by dividing the number of bytes transferred on the data path by the total number of bytes to generate a first value and multiplying the first value by 100. The difference load balancing value is computed by subtracting the load balancing share from the actual load balancing share.
(DifferenceLB>ChangeThreshold)), Reduce LoadBalancingShare of the DataPath by “LBChange %” value Select data path with lowest value of DifferenceLB, where lowest refers to inclusion of −ve numbers, and, if multiple data paths have the same lowest value of DifferenceLB, select data path from this group with highest ActualLBShare Increase LoadBalancingShare of the selected data path 3. Return with the updated LoadBalancingShare for all the data paths With dynamic load balancing, the static load balancing technique may be implemented once the LoadBalancingShares have been computed at the “TimerInterval value”. In certain embodiments, switching between static and dynamic load balancing is performed by a user. If a switch occurs, the LoadBalancingShares may be recomputed.
The iSCSI target1 806 returns a status indicator of “good” to the SCSI low level driver 804, which forwards the status indicator of “good” to the failover/load balancing components 802, which, in turn, forward the status indicator of “good” to the SCSI mid layer 800.
The described embodiments may be implemented as a method, apparatus or article of manufacture using programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof. The term “article of manufacture” and “circuitry” as used herein refers to a state machine, code or logic implemented in hardware logic (e.g., an integrated circuit chip, Programmable Gate Array (PGA), Application Specific Integrated Circuit (ASIC), etc.) or a computer readable medium, such as magnetic storage medium (e.g., hard disk drives, floppy disks, tape, etc.), optical storage (CD-ROMs, optical disks, etc.), volatile and non-volatile memory devices (e.g., EEPROMs, ROMs, PROMs, RAMs, DRAMs, SRAMs, firmware, programmable logic, etc.). Code in the computer readable medium is accessed and executed by a processor. When the code or logic is executed by a processor, the circuitry may include the medium including the code or logic as well as the processor that executes the code loaded from the medium. Thus, the “article of manufacture” may comprise the medium in which the code is embodied. Additionally, the “article of manufacture” may comprise a combination of hardware and software components in which the code is embodied, processed, and executed. Of course, those skilled in the art will recognize that many modifications may be made to this configuration. Additionally, the devices, adapters, etc., may be implemented in one or more integrated circuits on the adapter or on the motherboard.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS4403286 *Mar 6, 1981Sep 6, 1983International Business Machines CorporationBalancing data-processing work loadsUS5006781 *May 9, 1988Apr 9, 1991Onan CorporationMicroprocessor based integrated generator set controller apparatus and methodUS5086499May 23, 1989Feb 4, 1992Aeg Westinghouse Transportation Systems, Inc.Computer network for real time control with automatic fault identification and by-passUS5168208 *Jan 9, 1991Dec 1, 1992Onan CorporationMicroprocessor based integrated generator set controller apparatus and methodUS5390068 *Jul 27, 1992Feb 14, 1995Onan CorporationMicroprocessor based integrated generator set controller apparatus and methodUS5493689 *Mar 1, 1993Feb 20, 1996International Business Machines CorporationSystem for configuring an event driven interface including control blocks defining good loop locations in a memory which represent detection of a characteristic patternUS5495426 *Jan 26, 1994Feb 27, 1996Waclawsky; John G.Inband directed routing for load balancing and load distribution in a data communication networkUS5724569 *Jun 28, 1995Mar 3, 1998Bull S.A.Apparatus for evaluating database query performance having libraries containing information for modeling the various system components of multiple systemsUS5790775Oct 23, 1995Aug 4, 1998Digital Equipment CorporationHost transparent storage controller failover/failback of SCSI targets and associated unitsUS6052795Oct 29, 1996Apr 18, 2000Hitachi, Ltd.Recovery method and system for continued I/O processing upon a controller failureUS6081511 *Aug 14, 1996Jun 27, 2000Cabletron Systems, Inc.Load sharing for redundant networksUS6185601 *Apr 15, 1998Feb 6, 2001Hewlett-Packard CompanyDynamic load balancing of a network of client and server computersUS6381218Sep 11, 1998Apr 30, 2002Compaq Computer CorporationNetwork controller system that uses directed heartbeat packetsUS6430610Sep 2, 1998Aug 6, 2002Steeleye Technology, Inc.TCP/IP address protection mechanism in a clustered server environmentUS6438133 *Sep 9, 1998Aug 20, 2002Cisco Technology, Inc.Load balancing mechanism for a translational bridge environmentUS6453360 *Mar 1, 1999Sep 17, 2002Sun Microsystems, Inc.High performance network interfaceUS6470397Mar 30, 1999Oct 22, 2002Qlogic CorporationSystems and methods for network and I/O device driversUS6526521Jun 18, 1999Feb 25, 2003Emc CorporationMethods and apparatus for providing data storage accessUS6618798Jul 11, 2000Sep 9, 2003International Business Machines CorporationMethod, system, program, and data structures for mapping logical units to a storage space comprises of at least one array of storage unitsUS6654801Jan 4, 1999Nov 25, 2003Cisco Technology, Inc.Remote system administration and seamless service integration of a data communication network management systemUS6658018Jun 30, 1999Dec 2, 2003Intel CorporationMethod and system of providing advanced teaming functionality capable of utilizing heterogeneous adapters to improve utility and performanceUS6687735 *May 30, 2000Feb 3, 2004Tranceive Technologies, Inc.Method and apparatus for balancing distributed applicationsUS6711137 *Mar 12, 1999Mar 23, 2004International Business Machines CorporationSystem and method for analyzing and tuning a communications networkUS6769071Jan 23, 2001Jul 27, 2004Adaptec, Inc.Method and apparatus for intelligent failover in a multi-path systemUS6802021Jan 23, 2001Oct 5, 2004Adaptec, Inc.Intelligent load balancing for a multi-path storage systemUS6823477Jan 23, 2001Nov 23, 2004Adaptec, Inc.Method and apparatus for a segregated interface for parameter configuration in a multi-path failover systemUS6940853 *Jan 3, 2001Sep 6, 2005Juniper Networks, Inc.Datagram relaying apparatus with load distributing functionUS6941341 *Jan 26, 2004Sep 6, 2005Sandraic Logic, Llc.Method and apparatus for balancing distributed applicationsUS6990553Jan 13, 2004Jan 24, 2006Hitachi, Ltd.Method of controlling storage device controlling apparatus, and storage device controlling apparatusUS7003687May 15, 2002Feb 21, 2006Hitachi, Ltd.Fail-over storage systemUS7016367 *Jan 2, 2001Mar 21, 2006Juniper Networks, Inc.Systems and methods for allocating bandwidth for processing of packetsUS7111084Dec 28, 2001Sep 19, 2006Hewlett-Packard Development Company, L.P.Data storage network with host transparent failover controlled by host bus adapterUS7126910 *Dec 13, 2001Oct 24, 2006AlcatelLoad balancing technique for a resilient packet ringUS7134040Apr 17, 2002Nov 7, 2006International Business Machines CorporationMethod, system, and program for selecting a path to a device to use when sending data requests to the deviceUS7139242 *Dec 19, 2001Nov 21, 2006Proficient Networks, Inc.Methods, apparatuses and systems facilitating deployment, support and configuration of network routing policiesUS7281169Jan 30, 2004Oct 9, 2007Dell Products L.P.Method, software and system for multi-path fail-over recovery in sequential storage systemsUS7307948Oct 21, 2002Dec 11, 2007Emulex Design & Manufacturing CorporationSystem with multiple path fail over, fail back and load balancingUS7308604Jul 3, 2003Dec 11, 2007Thomson LicensingReal-time fail-over recovery for a media area networkUS7310341 *Nov 25, 2002Dec 18, 2007Alcatel Canada Inc.System and method for parallel connection selection in a communication networkUS7313681Nov 20, 2003Dec 25, 2007International Business Machines CorporationApparatus, system, and method for adapter fastloadUS7330972Sep 26, 2003Feb 12, 2008Intel CorporationRemote management device key administration using network adaptersUS7496104 *Aug 13, 2004Feb 24, 2009Nortel Networks LimitedEnhanced encapsulation mechanism using GRE protocolUS7603463 *Oct 13, 2009Nortel Networks LimitedMethod and apparatus for allocating processing capacity of system processing units in an extranet gatewayUS20020141343 *Dec 19, 2001Oct 3, 2002Bays Robert JamesMethods, apparatuses and systems facilitating deployment, support and configuration of network routing policiesUS20030074473 *Oct 12, 2001Apr 17, 2003Duc PhamScalable network gateway processor architectureUS20030126315Dec 28, 2001Jul 3, 2003Choon-Seng TanData storage network with host transparent failover controlled by host bus adapterUS20030140191Jan 24, 2002Jul 24, 2003Mcgowen Michael E.System, method, and computer program product for on-line replacement of a host bus adapterUS20040107304May 29, 2003Jun 3, 2004Paul GrunMethods and systems for efficient connection of I/O devices to a channel-based switched fabricUS20040153710Jul 7, 2003Aug 5, 2004Fair Robert LawrenceMultiple hierarichal/peer domain file server with domain based, cross domain cooperative fault handling mechanismsUS20040162901 *Feb 19, 2004Aug 19, 2004Krishna MangipudiMethod and apparatus for policy based class service and adaptive service level management within the context of an internet and intranetUS20040184483 *Jan 30, 2004Sep 23, 2004Akiko OkamuraTransmission bandwidth control deviceUS20040203827 *Nov 1, 2002Oct 14, 2004Adreas HeinerDynamic load distribution using local state informationUS20040236860 *Jan 26, 2004Nov 25, 2004Gary LogstonMethod and apparatus for balancing distributed applicationsUS20050185789Jan 30, 2004Aug 25, 2005Goodwin Kevin M.Forestalling actions that otherwise would defeat access-control mechanism for volumeUS20060173331 *Nov 23, 2005Aug 3, 2006Siemens Medical Solutions Usa, Inc.Ergonomic transducer housing and methods for ultrasound imagingUS20060193331 *Aug 7, 2003Aug 31, 2006Telecom Italia S.P.A.Method for the statistical estimation of the traffic dispersion in telecommunication networkUS20060242313 *Jun 7, 2006Oct 26, 2006Lewiz CommunicationsNetwork content processor including packet engineUS20060291392 *Jun 28, 2005Dec 28, 2006Alicherry Mansoor A KMulti-path routing using intra-flow splittingUS20070030804 *Jul 31, 2006Feb 8, 2007Kimball Karen EMethod and apparatus for automatic load-balancing on a multi-segment networkUS20070070893 *May 13, 2004Mar 29, 2007Siemens AktiengesellschaftMethod and network node for self-regulating, autonomous and decentralized traffic distribution in a multipath networkUS20070133406 *Dec 14, 2005Jun 14, 2007Jean-Philippe VasseurTechnique for enabling traffic engineering on CE-CE paths across a provider networkUS20070153763 *Dec 29, 2005Jul 5, 2007Rampolla Richard ARoute change monitor for communication networksUS20070271570 *May 17, 2006Nov 22, 2007Brown Douglas PManaging database utilities to improve throughput and concurrencyUS20080034249Aug 22, 2007Feb 7, 2008Husain Syed M AFail-Over In A Multi-Computer SystemUS20080291826 *May 24, 2007Nov 27, 2008Harris Stratex Networks Operating CorporationDynamic Load Balancing for Layer-2 Link AggregationUS20090193105 *Jul 30, 2009Cisco Technology, Inc.Local placement of large flows to assist load-balancingJP2002304331A Title not availableRU2202123C2 Title not available* Cited by examinerNon-Patent CitationsReference1"dpANS Fibre Channel Generic Services," www.t11.org; X3.288-199x, Rev. 3.1, Aug. 7, 1996; 100 pp.2"dpANS Fibre Channel Protocol for SCSI," X3.269-199X, Revision 12, Dec. 4, 1995; 75 pp.3"dpANSI Fibre Channel Arbitrated Loop," www.t11.org; X3.272-199x, Rev. 4.5, Jun. 1, 1995; 102 pp.4"dpANSI Fibre Channel Fabric Generic Requirements," X3.289-199x, Rev. 3.5, Aug. 7, 1996; 35 pp.5"dpANSI Fibre Channel Framing and Signaling," www.t11.org; INCITS working draft, Apr. 9, 2003; 106 pp.6"How to Get T10 Draft Standards (and other Publications)," www.t10.org, 3 pp.7"Information Technology-SCSI Architecture Model-2 (SAM-2)," T10 Project 1157-D, www.t10.org, 91 pp.8"Internet Protocol: DARPA Internet Program Protocol Specification," Information Sciences Institute for Defense Advanced Research Projects Agency, RFC 791, (Sep. 1981), 44 pp.9"Serial ATA 1.0a Specification," Serial ATA Specs and Design Guides, www.serialata.com Feb. 4, 2003; 2 pp.10"Serial ATA: High Speed Serialized AT Attachment," Serial ATA Workgroup rev. 1.0, Aug. 29, 2001; 36 pp.11"Transmission Control Protocol: DARPA Internet Program Protocol Specification," Information Sciences Institute for Defense Advanced Research Projects Agency, RFC 793, (Sep. 1981), 80 pp.12"Information Technology—SCSI Architecture Model—2 (SAM-2)," T10 Project 1157-D, www.t10.org, 91 pp.13 *A dynamic routing algorithm with load balancing heuristics for restorable connections in WDM networks; Lu Ruan; Luo, H.; Chang Liu; Selected Areas in Communications, IEEE Journal on vol. 22, Issue 9, Nov. 2004 pp. 1823-1829.14Abstract and Translation for Japanese Publication No. JP2002-304331, Oct. 18, 2002, 12 pp.15 *Adaptive routing and load balancing of ephemeral connections; Heusse, M.; Kermarrec, Y.; Universal Multiservice Networks, 2000. ECUMN 2000. 1st European Conference on Oct. 2-4, 2000 pp. 100-108.16Alacritech Inc., "Delivering High-Performance Storage Networking", 2001, 8 pp.17 *An efficient QoS routing algorithm based on nonlinear path distance for min-sharing interference and load balancing Xiaolong Yang; Shuheng Zhou; Min Zhang; Xuefei Tang; Jinde Liu; Computer and Information Technology, 2004. CIT '04. The Fourth International Conference on Sep. 14-16, 2004 pp. 380-385.18Aspen Systems, "Load Balancing Clustering", [online], [Retrieved 2003], retrieved from the Internet at .19Aspen Systems, "Load Balancing Clustering", [online], [Retrieved 2003], retrieved from the Internet at <URL: http://www.aspsys.com/clusters/load—balancing/>.20Brocade Communication Systems Inc, "Improving System Availability With Storage Area Networks", 2001, 22 pp.21Chinese First Office Action, Feb. 15, 2008, for CN Application No. 200480041815.9, 29 pp.22Cisco Systems, Inc., "Cisco Load Balancing Solutions", FO-4553-c1, 1999, 50 pp.23Cisco Systems, Inc., "High Availability for Cisco SN 5420: Storage Router-Based iSCSI Solutions", White Paper, 2002, pp. 1-32.24Cisco Systems, Inc., "High Availability Quick Look", White Paper, [Retrieved on Sep. 18, 2006], retrieved from the Internet at .25Cisco Systems, Inc., "Load Balancing: A Multifaceted Solution for Improving Server Availability", White Paper, 1998, pp. 1-4.26Cisco Systems, Inc., "MultiNode Load Balancing: Technical Documents", [online], 2002, [Retrieved 2003], retrieved from the Internet at .27Cisco Systems, Inc., "Cisco Load Balancing Solutions", FO—4553—c1, 1999, 50 pp.28Cisco Systems, Inc., "High Availability Quick Look", White Paper, [Retrieved on Sep. 18, 2006], retrieved from the Internet at <URL: http://whitepapers.zdnet.co.uk>.29Cisco Systems, Inc., "MultiNode Load Balancing: Technical Documents", [online], 2002, [Retrieved 2003], retrieved from the Internet at <URL: http://www.cisco.com/warp/public/cc/pd/ibsw/mulb/tech/>.30Compaque Corp, "Intelligent Traffic Management-Global Load Balancing in a DISA Environment", Technical Guide, Feb. 2002, pp. 1-23.31Compaque Corp, "Intelligent Traffic Management—Global Load Balancing in a DISA Environment", Technical Guide, Feb. 2002, pp. 1-23.32 *Distributed routing for load balancing; Boel, R.K.; van Schuppen, J.H.; Proceedings of the IEEE vol. 77, Issue 1, Jan. 1989 pp. 210-221.33Dot Hill Systems Corp., "SANpath v3.1 Software Product Description", Jul. 2000, Revision A, pp. 1-11.34 *Dynamic congestion-based load balanced routing in optical burst-switched networks; Thodime, G.R.V.; Vokkarane, V.M.; Jue, J.P.; Global Telecommunications Conference, 2003. GLOBECOM '03. IEEE vol. 5, 2003 pp. 2628-2632 vol. 5.35Falconstor, "Superior Resiliency, Availability, and Performance with IPStor DynaPath", White Paper, 2001, pp. 1-9.36Great Britain Second Office Action, Jul. 27, 2007, for Application No. GB0616648.2, 4 pp.37IEEE Std 802.11a-1999, Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications, 12 pp.38IEEE Std 802.11a—1999, Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications, 12 pp.39IEEE Std 802.11b-1999, Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications, Sep. 16, 1999; 17 pp.40IEEE Std 802.11g-2003, Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications, Jun. 27, 2003; 22 pp.41IEEE Std 802.11g—2003, Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications, Jun. 27, 2003; 22 pp.42IEEE Std 802.3-2002, Part 3: Carrier sense multiple access with collision detection (CSMA/CD) access method and physical layer specifications, Mar. 8, 2002; 33 pp.43IEEE Std 802.3—2002, Part 3: Carrier sense multiple access with collision detection (CSMA/CD) access method and physical layer specifications, Mar. 8, 2002; 33 pp.44IEEE, "High Availability", [online], [Retrieved 2003], retrieved from the Internet at , 3 pp.45IEEE, "High Availability", [online], [Retrieved 2003], retrieved from the Internet at <URL: http://www.geocities.com/Paris/Musee/2712/tfcc.html>, 3 pp.46Intel Corp., "Advanced Networking Services: Teaming", [online], 2003, [Retrieved 2003], retrieved from the Internet at , 4 pp.47Intel Corp., "Advanced Networking Services: Teaming", [online], 2003, [Retrieved 2003], retrieved from the Internet at <URL: http://support.intel.com/support/network/adapter/ans/teaming.htm>, 4 pp.48Krueger, et al. "Small Computer Systems Interface protocol over the Internet (iSCSI) Requirements and Design Considerations," Network Working Group of The Internet Society, RFC 3347, (2002), 23 pp.49Linuxcare, "T3 Multipath Failover System FAQ", [online], 2000, [Retrieved 2003], retrieved from the Internet at , 3 pp.50Linuxcare, "T3 Multipath Failover System FAQ", [online], 2000, [Retrieved 2003], retrieved from the Internet at <URL: http://open-projects.linuxcare.com/t3/faq.html>, 3 pp.51 *Load balancing routing with bandwidth-delay guarantees; Gopalan, K.; Tzi-cker Chiueh; Yow-Jian Lin; Communications Magazine, IEEE; vol. 42, Issue 6, Jun. 2004 pp. 108-113.52Meinen, D., "High Availability Secure Path for Windows NT", Nov. 1999, 37 pp.53 *Modeling and performance analysis of QoS-aware load balancing of Web-server clusters; Computer Networks; vol. 40, Issue 2, Oct. 7, 2002, pp. 235-256.54Mylex, "Clustering & Fibre Channel: Next Wave in PC Computing", printed on Oct. 22, 2003, pp. 1-32.55Neomar, "Neomar LBFO Module: When Wireless Means Business", printed on Oct. 22, 2003, 2 pp.56Notice of Allowance, Dec. 1, 2009, for Application No. JP2007500707, 4 pages.57Office Action 1, Apr. 7, 2009, for Application No. JP2007-500707, 3 pp.58Office Action 1, Dec. 18, 2006, for Application No. GB0616648.2, 5 pp.59Office Action 2, Aug. 1, 2008, for Application No. CN200480041815.9, 11 pp.60Office Action 3, Jan. 24, 2008, for Application No. GB0616648.2, 5 pp.61Ottem, E., "SANs for High Availability Systems", WP-HA0201, 2001, pp. 1-6.62PCI-SIG "PCI Specifications," www.pcisig.com, copyright 2004; 18 pp.63PCT International Search Report and Written Opinion, Dec. 16, 2004, for International Application No. PCT/RU2004/000105.64PCT IPRP, Sep. 28, 2006, for International Application No. PCT/RU2004/000105.65Qlogic Corp., "Qlogic Fibre Channel Software: Enhancing Disk Adapter Performance, Function, and Control", White Paper, FC0132001-01 Rev B, May 2, 2000, pp. 1-21.66 *QoS-Aware Routing Schemes Based on Hierarchical Load-Balancing for Integrated Services Packet Networks (1999).67Response to Office Action 1, Jul. 3, 2008, for Chines Application No. CN200480041815.9, 13 pp.68Response to Office Action 1, Jun. 18, 2007, for Application No. GB0616648.2, 10 pp.69Response to Office Action 2, Nov. 12, 2007, for Application No. GB0616648.2, 6 pp.70Response to Office Action 2, Oct. 20, 2008, for Chinese Application No. CN200480041815.9, 21 pp.71Response to Office Action 3, Feb. 18, 2008, for Application No. GB0616648.2, 6 pp.72Response to Office Action, Aug. 6, 2009, for Application No. JP2007500707, 17 pages.73Robertson, A., "High-Availability Linux Project", [online], Sep. 24, 2003, [Retrieved 2003], retrieved from the Internet at , 7pp.74Robertson, A., "High-Availability Linux Project", [online], Sep. 24, 2003, [Retrieved 2003], retrieved from the Internet at <URL: http://www.linux-ha.org/>, 7pp.75Satran, et al., "iSCSI," IP Storage Working Group for Internet Engineering Task Force, www.ietf.org; Jan. 19, 2003; 51 pp.76Shanley, T. and D. Anderson, "PCI System Architecture, 4th edition," Mindshare, Inc., 1999; Chapters 1 & 2, 61 pp.77 *Simple mathematical modeling of efficient path selection for QoS routing in load balancing; Man-Ching Yuen; Weijia Jia; Chi-Chung Cheung; Multimedia and Expo, 2004. ICME '04. 2004 IEEE International Conference on.78Sun Microsystems, Inc., "Sun Clusters", White Paper, Oct. 1997, pp. 1-33.79Sun Microsystems, Inc., "Sun Multipath Failover Driver 1.0 for Windows NT User's Guide", White Paper, Revision 01, Feb. 2001, pp. 1-26.80 *Supporting QoS routing in mobile ad hoc networks using probabilistic locality and load balancing queensu.ca [PDF]ES Elmallah, HS Hassanein, HM - GLOBECOM-NEW 2001.81U.S. Appl. No. 10/599,055, filed Sep. 18, 2006, entitled "Failover and Load Balancing", invented by A. Belyakov, M. Sennikovsky, and A. Drozdov, 35 pages.82U.S. Appl. No. 12/749,407, filed Mach 29, 2010, entitled "Failover and Load Balancing", invented by A. Belyakov, M. Sennikovsky, and A. Drozdov, 33 pages.83Weber, et al. "Fiber Channel (FC) Frame Encapsulation," Network Working Group of The Internet Society, RFC 3643, (2003), 17 pp.84Weber, R., "It's the I-T Nexus, Stupid", 01-093r0, [Retrieved 2003], 2005, pp. 1-11.85Weber, R., "It's the I—T Nexus, Stupid", 01-093r0, [Retrieved 2003], 2005, pp. 1-11.86Zeus Technology, "Building High Performance, High-Availability Clusters", Jun. 16, 2000, Version 1.0, pp. 1-12.* Cited by examinerReferenced byCiting PatentFiling datePublication dateApplicantTitleUS7992039 *Mar 29, 2010Aug 2, 2011Intel CorporationFailover and load balancingUS8135765 *Jun 10, 2008Mar 13, 2012Software AgSystem and method for a genetic integration of a database into a high availability clusterUS8429452Jun 23, 2011Apr 23, 2013Intel CorporationFailover and load balancingUS8589480 *May 24, 2011Nov 19, 2013Sony Computer Entertainment America LlcAutomatic performance and capacity measurement for networked serversUS9008106 *Apr 24, 2013Apr 14, 2015Apple Inc.Enhanced encapsulation mechanism using GRE protocolUS9026651Sep 26, 2013May 5, 2015Sony Computer Entertainment America LlcAutomatic performance and capacity measurement for networked serversUS9088905 *Apr 23, 2012Jul 21, 2015Hughes Network Systems, LlcMethod and apparatus for load balancing on a priority level basis over shared communications channels of a communications systemUS9128900 *Apr 6, 2013Sep 8, 2015Hon Hai Precision Industry Co., Ltd.Method and server for managing redundant arrays of independent disks cardsUS9213953 *Sep 15, 2008Dec 15, 2015Amazon Technologies, Inc.Multivariable load balancing in a fulfillment networkUS9262351Mar 27, 2014Feb 16, 2016International Business Machines CorporationInter-adapter cooperation for multipath input/output systemsUS20090307246 *Jun 10, 2008Dec 10, 2009Klaus SchulerSystem and Method for a Genetic Integration of a Database into a High Availability ClusterUS20100185794 *Mar 29, 2010Jul 22, 2010Alexander BelyakovFailover and load balancingUS20120303694 *May 24, 2011Nov 29, 2012Sony Computer Entertainment Inc.Automatic performance and capacity measurement for networked serversUS20130235802 *Apr 24, 2013Sep 12, 2013Apple Inc.Enhanced Encapsulation Mechanism Using GRE ProtocolUS20130279334 *Apr 23, 2012Oct 24, 2013Hughes Network Systems, LlcMethod and apparatus for load balancing on a priority level basis over shared communications channels of a communications systemUS20140019989 *Sep 13, 2013Jan 16, 2014Fujitsu LimitedMulti-core processor system and scheduling methodUS20140082245 *Apr 6, 2013Mar 20, 2014Hon Hai Precision Industry Co., Ltd.Method and server for managing redundant arrays of independent disks cards* Cited by examinerClassifications U.S. Classification370/230, 370/428, 370/395.4International ClassificationH04L29/08, H04L12/28Cooperative ClassificationH04L67/1002, H04L67/101, H04L67/1034, H04L67/1012European ClassificationH04L29/08N9A11, H04L29/08N9A1C, H04L29/08N9A1D, H04L29/08N9ALegal EventsDateCodeEventDescriptionMar 31, 2004ASAssignmentOwner name: INTEL CORPORATION, CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MALPANI, NAVNEET;YAO, XUEBIN;MUSTA, CHARLIE A.;REEL/FRAME:015180/0402;SIGNING DATES FROM 20040326 TO 20040330Owner name: INTEL CORPORATION, CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MALPANI, NAVNEET;YAO, XUEBIN;MUSTA, CHARLIE A.;SIGNING DATES FROM 20040326 TO 20040330;REEL/FRAME:015180/0402Jul 1, 2004ASAssignmentOwner name: INTEL CORPORATION, CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MALPANI, NAVNEET;YAO, XUEBIN;MUSTA, CHARLES A.;REEL/FRAME:014813/0869;SIGNING DATES FROM 20040413 TO 20040429Owner name: INTEL CORPORATION, CALIFORNIAFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MALPANI, NAVNEET;YAO, XUEBIN;MUSTA, CHARLES A.;SIGNING DATES FROM 20040413 TO 20040429;REEL/FRAME:014813/0869Nov 30, 2010CCCertificate of correctionDec 27, 2013FPAYFee paymentYear of fee payment: 4RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services