Source: http://www.google.com/patents/US7068667?ie=ISO-8859-1&dq=Richard+B.+Fuller
Timestamp: 2015-01-27 14:38:12
Document Index: 428040351

Matched Legal Cases: ['Application No. 60', 'Application No. 60', 'Application No. 60', 'Application No. 60', 'Application No. 60', 'Application No. 60', 'Application No. 60', 'Application No. 60', 'Application No. 60']

Patent US7068667 - Method and system for path building in a communications network - Google PatentsSearch Images Maps Play YouTube News Gmail Drive More »Sign inAdvanced Patent SearchPatentsA method and system for managing an interconnect fabric that connects nodes. A network manager manages an interconnect fabric or network of routing devices (e.g., interconnect fabric modules, switches, or routers) to allow source nodes to transmit data to destination nodes. The network manager receives...http://www.google.com/patents/US7068667?utm_source=gb-gplus-sharePatent US7068667 - Method and system for path building in a communications networkAdvanced Patent SearchPublication numberUS7068667 B2Publication typeGrantApplication numberUS 10/046,334Publication dateJun 27, 2006Filing dateOct 26, 2001Priority dateApr 27, 2001Fee statusPaidAlso published asUS6993023, US6996058, US7042877, US7068666, US7164656, US20020159389, US20020159437, US20020159446, US20020159451, US20020159452, US20020159453, US20020159456, US20020159458, US20020159468, US20020161887, US20020161923, US20020167902, US20020181395, US20020184529, US20020188754, US20030189927, US20030202535, US20030202536, US20030204618, US20040004966Publication number046334, 10046334, US 7068667 B2, US 7068667B2, US-B2-7068667, US7068667 B2, US7068667B2InventorsMichael S. Foster, Michael A. DorsettOriginal AssigneeThe Boeing CompanyExport CitationBiBTeX, EndNote, RefManPatent Citations (129), Non-Patent Citations (5), Referenced by (24), Classifications (37), Legal Events (3) External Links: USPTO, USPTO Assignment, EspacenetMethod and system for path building in a communications networkUS 7068667 B2Abstract A method and system for managing an interconnect fabric that connects nodes. A network manager manages an interconnect fabric or network of routing devices (e.g., interconnect fabric modules, switches, or routers) to allow source nodes to transmit data to destination nodes. The network manager receives registration requests from source nodes to send data to destination nodes, configures the routing devices of the network to establish a path from each source node to its destination node, and provides a virtual address to each source node. The virtual address identifies a path from the source node to the destination node. The source node sends the data to its destination node by providing the data along with the virtual address to a routing device of the network.
setting each of the source-side ports to switch data sent to the identified virtual address through the destination-side port of its switch, wherein the switches are Fibre Channel, wherein the data is changed from an InfiniBand frame to a Fibre Channel frame.
7. The method of claim 1 wherein the switches are interconnect fabric modules.
8. The method of claim 1 wherein when a port of a switch receives data with a virtual address that has not been set for the port, the port does not forward the data.
9. A method for establishing a path between a source node and a destination node through a network of routing devices, the method comprising:
setting each of the identified source-side ports to route data sent to the identified virtual address through the identified destination-side port of its routing device, wherein the data is an InfiniBand frame, wherein the InfiniBand frame is changed to a Fibre Channel frame.
12. The method of claim 9 wherein each routing device has a virtual address table for mapping virtual addresses to another port of the routing device.
13. The method of claim 9 wherein when data is received at a port of a routing device, the virtual address of the data is used to retrieve an indication of another port and the data is sent out of the routing device through the other port.
14. The method of claim 9 wherein a path is established between the source node and each of a plurality of destination nodes by identifying ports of routing devices for each path.
15. The method of claim 9 wherein the routing devices are interconnect fabric modules.
16. The method of claim 9 wherein when a routing device receives data with a virtual address that has not been set for the routing device, the routing device does not forward the data.
17. The method of claim 9 wherein the identified virtual address is not currently used by any of the identified source-side ports.
18. The method of claim 9 wherein the identified virtual address is currently used by an identified source-side port when part of the path is shared by two source nodes sending data to the same destination node.
19. The method of claim 9 including providing the identified virtual address to the source node for use in sending data to the destination node.
20. A network manager for establishing a path between a source node and a destination node through a network of switches, comprising:
a component that configures each of the identified switches to route data sent to the identified virtual address through the identified switches from the source node to the destination node, wherein the data is an InfiniBand frame wherein the InfiniBand frame is changed to a Fibre Channel frame.
21. The network manager of claim 20 including:
22. The network manager of claim 20 including:
23. The network manager of claim 22 wherein the path from the source node to the destination node includes one port that is not in the path from the destination node to the source node.
24. The network manager of claim 22 wherein the path from the source node to the destination node is different from the path from the destination node to the source node.
25. The network manager of claim 20 wherein each switch has ports with a mapping of virtual addresses to another port of the switch.
26. The network manager of claim 20 wherein when data is received at a port of a switch, the identified virtual address is used to retrieve an indication of another port of the switch through which the data is transmitted.
27. The network manager of claim 20 wherein a path is established between the source node and each of a plurality of destination nodes by identifying ports of switches for each path.
28. The network manager of claim 20 wherein the switches are interconnect fabric modules.
29. The network manager of claim 20 wherein when a port of a switch receives data with a virtual address that has not been set for the port, the port does not forward the data.
30. The network manager of claim 20 wherein each switch has a source-side port and the identified virtual address is not currently used by any of the source-side ports.
31. The network manager of claim 20 wherein each switch has a source-side port and the identified virtual address is currently used by a source-side port when part of the path is shared by two source nodes sending data to the same destination node.
32. A network manager for establishing a path between a source node and a destination node through a network of routing devices, comprising:
setting each of the identified source-side ports to route data sent to the identified virtual address through the identified destination-side port of the routing device wherein the data is an InfiniBand frame, wherein the InfiniBand frame is changed to a Fibre Channel frame.
33. The network manager of claim 32 wherein a routing device is a switch.
34. The network manager of claim 32 wherein each port of each routing device has a means for mapping virtual addresses to another port of the routing device.
35. The network manager of claim 32 including means for, when data is received at a port of a routing device, retrieving an indication of another port using the identified virtual address and sending the data out of the routing device through the other port.
36. The network manager of claim 32 including means for establishing a path between the source node and each of a plurality of destination nodes by identifying ports of routing devices for each path.
CROSS-REFERENCE TO RELATED APPLICATION(S) This application claims the benefit of U.S. Provisional Application No. 60/287,069 entitled �METHOD FOR IMPLEMENTING A CLUSTER NETWORK FOR HIGH PERFORMANCE AND HIGH AVAILABILITY USING A FIBRE CHANNEL SWITCH FABRIC,� filed Apr. 27, 2001; U.S. Provisional Application No. 60/287,120 entitled �MULTI-PROTOCOL NETWORK FOR ENTERPRISE DATA CENTERS,� filed Apr. 27, 2001; U.S. Provisional Application No. 60/286,918 entitled �UNIFIED ENTERPRISE NETWORK SWITCH (UNEX) PRODUCT SPECIFICATION,� filed Apr. 27, 2001; U.S. Provisional Application No. 60/286,922 entitled �QUALITY OF SERVICE EXAMPLE,� filed Apr. 27, 2001; U.S. Provisional Application No. 60/287,081 entitled �COMMUNICATIONS MODEL,� filed Apr. 27, 2001; U.S. Provisional Application No. 60/287,075 entitled �UNIFORM ENTERPRISE NETWORK SYSTEM,� filed Apr. 27, 2001; U.S. Provisional Application No. 60/314,088 entitled �INTERCONNECT FABRIC MODULE,� filed Aug. 21, 2001; U.S. Provisional Application No. 60/314,287 entitled �INTEGRATED ANALYSIS OF INCOMING DATA TRANSMISSIONS,� filed Aug. 22, 2001; U.S. Provisional Application No. 60/314,158 entitled �USING VIRTUAL IDENTIFIERS TO ROUTE TRANSMITTED DATA THROUGH A NETWORK,� filed Aug. 21, 2001, and is related to U.S. patent application Ser. No. 10/062,199 entitled �METHOD AND SYSTEM FOR VIRTUAL ADDRESSING IN A COMMUNICATIONS NETWORK,� U.S. patent application Ser. No. 10/066,014 entitled �METHOD AND SYSTEM FOR LABEL TABLE CACHING IN A ROUTING DEVICE,� U.S. patent application Ser. No. 10/039,505 entitled �METHOD AND SYSTEM FOR MULTIFRAME BUFFERING IN A ROUTING DEVICE,� ; U.S. patent application Ser. No. 10/046,333 entitled �METHOD AND SYSTEM FOR DOMAIN ADDRESSING IN A COMMUNICATIONS NETWORK,� U.S. patent application Ser. No. 10/039,404 entitled �METHOD AND SYSTEM FOR INTERSWITCH LOAD BALANCING IN A COMMUNICATIONS NETWORK,� U.S. patent application Ser. No. 10/046,572 entitled �METHOD AND SYSTEM FOR INTERSWITCH DEADLOCK AVOIDANCE IN A COMMUNICATIONS NETWORK,� U.S. patent application Ser. No. 10/039,877 entitled �METHOD AND SYSTEM FOR CONNECTION PREEMPTION IN A COMMUNICATIONS NETWORK,� U.S. patent application Ser. No. 10/061,564 entitled �METHOD AND SYSTEM FOR MULTICASTING IN A ROUTING DEVICE,� U.S. patent application Ser. No. 10/046,640 entitled �METHOD AND SYSTEM FOR NETWORK CONFIGURATION DISCOVERY IN A NETWORK MANAGER,� U.S. patent application Ser. No. 10/039,703 entitled �METHOD AND SYSTEM FOR RESERVED ADDRESSING IN A COMMUNICATIONS NETWORK,� U.S. patent application Ser. No. 10/039,814 entitled �METHOD AND SYSTEM FOR RECONFIGURING A PATH IN A COMMUNICATIONS NETWORK,� U.S. patent application Ser. No. 10/066,217 entitled �METHOD AND SYSTEM FOR ADMINISTRATIVE PORTS IN A ROUTING DEVICE,� U.S. patent application Ser. No. 10/039,784 entitled �PARALLEL ANALYSIS OF INCOMING DATA TRANSMISSIONS,� U.S. patent application Ser. No. 10/066,159 entitled �INTEGRATED ANALYSIS OF INCOMING DATA TRANSMISSIONS,� U.S. patent application Ser. No. 10/062,245 entitled �USING VIRTUAL IDENTIFIERS TO ROUTE TRANSMITTED DATA THROUGH A NETWORK,� U.S. patent application Ser. No. 10/044,182 entitled �USING VIRTUAL IDENTIFIERS TO PROCESS RECEIVED DATA ROUTED THROUGH A NETWORK,� U.S. patent application Ser. No. 10/044,164 entitled �METHOD AND SYSTEM FOR PERFORMING SECURITY VIA VIRTUAL ADDRESSING IN A COMMUNICATIONS NETWORK,� and U.S. patent application Ser. No. 10/068,329 entitled �METHOD AND SYSTEM FOR PERFORMING SECURITY VIA DE-REGISTRATION IN A COMMUNICATIONS NETWORK�, which are all hereby incorporated by reference in their entirety.
TECHNICAL FIELD The described technology relates to a network manager for routing devices of an interconnect fabric.
BACKGROUND The Internet has emerged as a critical commerce and communications platform for businesses and consumers worldwide. The dramatic growth in the number of Internet users, coupled with the increased availability of powerful new tools and equipment that enable the development, processing, and distribution of data across the Internet have led to a proliferation of Internet-based applications. These applications include e-commerce, e-mail, electronic file transfers, and online interactive applications. As the number of users of, and uses for, the Internet increases so does the complexity and volume of Internet traffic. According to UUNet, Internet traffic doubles every 100 days. Because of this traffic and its business potential, a growing number of companies are building businesses around the Internet and developing mission-critical business applications to be provided by the Internet.
Existing enterprise data networks (�EDNs�) that support e-commerce applications providing services to customers are straining under the demand to provide added performance and added services. The growing customer demands for services, along with a highly competitive market, has resulted in increasingly complex ad hoc EDNs. Affordable, high-performance EDN solutions require extensive scalability, very high availability, and ease of management. These attributes are significantly compromised or completely lost as existing solutions are grown to meet the demand.
BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a network diagram illustrating various nodes of an example Fibre Channel fabric-based interconnect network that are inter-communicating using virtual identifiers.
FIG. 2 is a flow diagram illustrating the discovery processing of a component of the interconnect fabric module in one embodiment.
FIG. 3 is a flow diagram illustrating the discovery processing of the network manager in one embodiment.
FIG. 4 is a flow diagram illustrating the process of establishing a path by the network manager in one embodiment.
FIG. 5 is a flow diagram illustrating the processing of an identify virtual address component of the network manager in one embodiment.
FIG. 6 is a flow diagram illustrating the processing of an initialize label table component of the network manager in one embodiment.
FIG. 7 is a block diagram illustrating a distributed network manager in one embodiment.
FIG. 8 is a flow diagram illustrating the processing of a component of an interconnect fabric module that processes reserved addresses in one embodiment.
DETAILED DESCRIPTION A method and system for managing an interconnect fabric that connects nodes is provided. In one embodiment, a network manager manages an interconnect fabric or network of routing devices (e.g., interconnect fabric modules, switches, or routers) to allow source nodes to transmit data to destination nodes. The network manager receives registration requests from source nodes to send data to destination nodes, configures the routing devices of the network to establish a path from each source node to its destination node, and provides a virtual address to each source node. The virtual address identifies a path from the source node to the destination node. The source node sends the data to its destination node by providing the data along with the virtual address to a routing device of the network. Upon receiving the data and the virtual address, a source-side port of each routing device in the path uses the virtual address to identify a destination-side port through which the data and the virtual address are to be transmitted. The network manager configures the routing devices by setting the mappings from a source-side port to a destination-side port for each routing device in the path. The routing devices receive data via source-side ports and transmits data via destination-side ports.
In one embodiment, a routing device is an interconnect fabric module (�IFM�) with high-speed switching capabilities. An interconnect fabric module can be dynamically configured to interconnect its communications ports so that data can be transmitted through the interconnected ports. Multiple interconnect fabric modules can be connected to form an interconnect fabric through which nodes (e.g., computer systems) can be interconnected. In one embodiment, data is transmitted through the interconnect fabric as frames such as those defined by the Fibre Channel standard. Fibre Channel is defined in ANSI T11 FC-PH, FC-PH-2, FC-PH-3, FC-PI, and FC-FS industry standard documents which are hereby incorporated by reference. One skilled in the art will appreciate, however, that the described techniques can be used with communications standards other than Fibre Channel. In particular, the described techniques can be used with the InfiniBand standard, which is described in the InfiniBand Architecture Specification, Vols. 1�2, Release 1.0, Oct. 24, 2000, which is hereby incorporated by reference. The interconnect fabric module may allow the creation of an interconnect fabric that is especially well suited for interconnecting devices utilizing multiple information types such as might be required by the devices of an enterprise data network (�EDN�).
In one embodiment, a virtual address may be part of a �virtual identifier� (e.g., source or destination identifier) that includes a domain address. A destination identifier of a frame may be set to a virtual identifier. The destination identifiers of the frames received by the interconnect fabric modules are used to forward the frame. Each interconnect fabric module is assigned a domain address. The interconnect fabric modules that are assigned the same domain address are in the same domain. The interconnect fabric modules use of the domain addresses to forward frames between domains. The network manager may configure the interconnect fabric modules with inter-domain paths. When an interconnect fabric module receives a frame with a destination domain address that matches its domain address, then the frame has arrived at its destination domain The interconnect fabric module then forwards the frame in accordance with the destination virtual address since it has arrived at its destination domain. If, however, the domain addresses do not match, then the frame has not arrived at its destination domain. The interconnect fabric module forwards the frame using an inter-domain path. Each port of an interconnect fabric module may have a domain address table (configured by the network manager) that maps the domain addresses to the destination port through which frames with that domain address are to be forwarded. Thus, an interconnect fabric module may selectively use virtual addresses and domain addresses when forwarding frames.
In some embodiments, one or more virtual identifier (�VI�) Network Interface Controller (�NIC�) facilities on each node (e.g., one VI NIC for each network interface) facilitate the use of virtual identifiers in communicating data When a VI NIC on a node receives an indication that a data communication to one or more remote nodes is to occur, such as from an application executing on the node, the VI NIC will identify an appropriate transmittal virtual identifier that can be used to route the data communication through the network to the appropriate remote destination nodes without being assigned to or directly associated with those destination nodes. Such data communications can include both transitory connectionless transmittals of data (e.g., unidirectional transmittals from a source to a destination) and non-transitory connections that allow multiple distinct transmittals of data (e.g., a persistent dedicated connection that allows a connection-initiating source and a connection destination to transmit data back and forth).
The VI NIC can identify an appropriate transmittal virtual identifier for routing a data communication in various ways. In some embodiments, the VI NIC will register some or all outgoing data communications with a network manager for the network, and will receive an appropriate transmittal virtual identifier to be used for that communication from the network manager. If an indicated data communication corresponds to a previously registered data communication (e.g., to an existing connection or to a previous communication to the same destination and in the same transmission manner), however, the VI NIC could instead in some embodiments use the previously received transmittal virtual identifier for that data communication rather than perform an additional registration for the indicated data communication. The manners in which a data communication can be transmitted vary with the transmission characteristics that are supported by a network, and can include factors such as a particular Class Of Service (�COS�) or transmission priority.
FIG. 1 is a network diagram illustrating various nodes of an example Fibre Channel fabric-based interconnect network that are inter-communicating using virtual identifiers. In this example embodiment, multiple interconnect fabric modules (�IFMs�) 110 with high-speed switching capabilities are used as intermediate routing devices to form an interconnect fabric, and multiple nodes 105, a network manager 115 and a Multi-Protocol Edge Switch (�MPEX�) 120 are connected to the fabric. Each of the nodes has at least one VI NIC that uses virtual identifiers when communicating and receiving data. The MPEX is used to connect the Fibre Channel or InfiniBand network to an external network, such as an Ethernet-based network, and similarly includes at least one VI NIC. Data is transmitted through the interconnect fabric using frames such as those defined by the Fibre Channel or InfiniBand standards.
As described above, the network manager may dynamically discover the topology of the network using various different techniques. In the embodiment described below, each interconnect fabric module identifies which of its ports are connected to other devices. The network manager uses this information to send a message through each port that is connected to another device to identify the connected-to device. FIG. 2 is a flow diagram illustrating the discovery processing of a component of the interconnect fabric module in one embodiment. Each port of an interconnect fabric module identifies whether it is connected to a port of another device, such as another switch or a node. The interconnect fabric module then provides to the network manager an indication of which of its ports are connected to other ports to assist in the discovery process. In blocks 201�204, the component determines whether each port is currently connected to another port. In block 201, the component selects the next port. In decision block 202, if all the ports have already been selected, then the component completes, else the component continues at block 203. In decision block 203, the component determines whether the selected port is connected to another port. This determination may be made based on various voltage levels of the communications links. If there is a connection, then the component continues at block 204, else the component loops to block 201 to select the next port of the interconnect fabric module. In block 204, the component notes the selected port as connected to another port and loops to block 201 to select the next port of the interconnect fabric module.
FIG. 3 is a flow diagram illustrating the discovery processing of the network manager in one embodiment. The network manager first retrieves an indication of which ports of the interconnect fabric modules are connected to other devices. The network manager then sends a query message through each of the indicated ports to the connected-to port. When the connected-to port receives the query message, it responds with an identification of its interconnect fabric module and its port number. In this way, the network manager can discover the topology of the interconnect fabric. In blocks 301�303, the network manager retrieves the indications of which ports of the interconnect fabric modules are connected to other ports. In block 301, the network manager selects the next interconnect fabric module that has not yet been selected. In decision block 302, if all the interconnect fabric modules have already been selected, then the network manager continues at block 304, else the network manager continues at block 303. In block 303, the network manager retrieves an indication of which ports of the selected interconnect fabric module are connected to other ports. The network manager may send the message using either in-band our out-of-band communications. The network manager then loops to block 301 to select the next interconnect fabric module. In blocks 304�310, the network manager determines the identity of each of the connected-to ports. In block 304, the network manager selects the next interconnect fabric module. In decision block 304, if all the interconnect fabric modules have already been selected, then the network manager completes its discovery process, else the network manager continues at block 306. In blocks 306�310, the network manager loops sending a query message through each port of the selected interconnect fabric module that is connected to another port. In block 306, the network manager selects the next port of the selected interconnect fabric module that is connected to another port. In decision block 307, if all such ports are already selected, then the network manager loops to block 304 to select the next interconnect fabric module, else the network manager continues at block 308. In block 308, the network manager sends a query message through the selected port of the selected interconnect fabric module. In block 309, the network manager receives the identification of the connected-to port of the selected port of the selected interconnect fabric module. The identification may include an indication of the interconnect fabric module and the port number of the connected-to port. In block 310, the network manager stores a mapping between the selected port of the selected interconnect fabric module and the connected-to port of the connected-to interconnect fabric module. These mappings define the topology of the network. The network manager then loops to block 306 to select the next port of the selected interconnect fabric module that is connected to another device.
FIG. 4 is a flow diagram illustrating the process of establishing a path by the network manager in one embodiment. A path is typically established when a node registers with the network manager. An establish path component of the network manager may receive an indication of a source node and a destination and then identify paths of ports of interconnect fabric modules from the source node to the destination node and from the destination node to the source node. The component then identifies virtual addresses for the paths and initializes the label tables of the ports of the interconnect fabric modules along the identified paths. A label table of a port contains mappings from virtual addresses to destination-side ports through which a frame sent to that virtual address is to be forwarded. In block 401, the component identifies the paths. In one embodiment, the path from the source node to the destination node and the path from the destination node to the source node use the same ports of the same interconnect fabric modules. That is, the paths use the same communications links. Alternatively, the path in one direction may be different from the path in the other direction. One skilled in the art will appreciate that various well-known techniques for identifying paths can be used. In block 402, the component invokes an identify virtual address component passing an indication of the path and an indication that the virtual address to be used by the source node when sending a communications to the destination node (e.g., the destination virtual address). The invoked component may select a virtual address that is not currently in use by any of the source-side ports of the path. A source-side port of the path is a port that receives data sent by a source node, and a destination-side port of the path is a port through which data is transmitted on its way to the destination node. In block 403, the component invokes in identify virtual address component passing an indication of the path and that the virtual address is to be used by the destination node (e.g., the source virtual address). In block 404, the component invokes a component to initialize the label tables of the source-side ports of the path with the destination virtual address. The invoked component transmits instructions to the each source-side port of the path indicating that the port is to update its label table to map the source virtual address to a destination-side port of the interconnect fabric module. In block 405, the component invokes a component to initialize the label tables of the destination-side ports of the path with the source virtual address. The component then completes.
FIG. 5 is a flow diagram illustrating the processing of an identify virtual address component of the network manager in one embodiment. In this embodiment, the identify virtual address component is provided an indication and a path along with an indication of whether a virtual address for the source node or the destination node is to be identified. The component may check every port along the path to identify a virtual address that is not currently used by a port along the path. Alternatively, the component may identify virtual addresses based on a sequential ordering. That is, the component may keep track of the last identified virtual address and increment that virtual address to identify the next virtual address. In this way, each virtual address is unique. In blocks 501�505, the component loops selecting the next virtual address and determining whether it is available. The virtual address may not be available to a port along the path when that port already uses that virtual address. In blocks 501, the component selects to the next virtual address. In decision block 502, if all the virtual addresses have already been selected, then the component indicates that a virtual address could not be identified, else the component continues at block 503. In blocks 503�505, the component loops selecting each port along the path and determining whether that port already uses the selected virtual address. In block 503, the component selects the next interconnect fabric module and port of the path. In decision block 504, if all the interconnect fabric modules and ports of the path have already been selected, then the component uses the selected virtual address as the identified virtual address and then completes, else the component continues at block 505. In decision block 505, if the selected virtual address is available at the selected interconnect fabric module and selected port, then the component loops to block 503 to select the next port along the path, else the component loops to block 501 to select the next virtual address.
FIG. 6 is a flow diagram illustrating the processing of an initialize label table component of the network manager in one embodiment. The initialize label table component sends a command to each port along the path indicating to add a mapping from the identified virtual address to the other port of that interconnect fabric module along the path. The component is passed in indication of the path, the virtual address, and an indication of whether the virtual address is a source virtual address or a destination virtual address. In block 601, the component selects the next interconnect fabric module and port in the path based on whether the source or destination virtual address has been passed. In decision block 602, if all the interconnect fabric modules along the path have already been selected, then the component completes, else the component continues at block 603. In block 603, the component sends a message to be selected port of the interconnect fabric module indicating to add to its label table a mapping from the virtual address to the other port of the path. The component then loops to block 601 to select the next interconnect fabric module and port in the path.
In one embodiment, the crosspoint switch of an IFM may have more outputs than the number of ports of the IFM. For example, a crosspoint switch may have 34 inputs and outputs, but the IFM may have only 32 ports. The IFM may use these additional ports of the crosspoint switch to route upper layer protocol frames, such as frames directed into a name server or other administrative services. In one embodiment, the additional output ports of the crosspoint switch may be connected to a manager device for the IFM. An interconnect fabric module may have a list of �reserved� addresses that designate an upper layer protocol port. When an IFM determines that an address of its frame matches one of the reserved addresses, it enables the routing of that frame to an upper layer protocol port. The routing to upper layer protocol ports may use the same arbitration mechanism as used for routing to non-upper layer protocol ports as described in the Patent Application entitled �Interconnect Fabric Module.� Alternatively, when the crosspoint switch does not have extra output for an upper layer protocol port, an output can be selectively switched between a communications port and an upper layer protocol port depending on whether the address of the destination identifier is reserved.
FIG. 7 is a block diagram illustrating a distributed network manager in one embodiment. In this embodiment, the network manager may be implemented on a series of manager devices connected directly to the interconnect fabric modules. The distributed network manager may communicate with each other using in-band communication of the interconnect fabric or using out-of-band communication that is independent of the interconnect fabric The crosspoint switch of an interconnect fabric module may have reserved ports for the distributed network manager. When an interconnect fabric module receives data that designates one of the reserved ports, then the interconnect fabric module forwards the data to the distributed network manager through the reserved port.
FIG. 8 is a flow diagram illustrating the processing of a component of an interconnect fabric module that processes reserved addresses in one embodiment. This component forwards the frame to the network manager via either in-band or out-of-band communications. With the use of in-band communications the frame can be routed to the appropriate interconnect fabric module, which can then send the frame to the network manager using the out-of-band communications. In block 801, if the virtual address of the received frame is a reserved address, then the component continues at block 802, else the component completes. In decision block 802, if the virtual address parameter within the frame is in the label table, then the frame is to be forwarded using in-band communications and the component continues at block 804, else the frame is to be forwarded directly to the network manager at the IFM's manager device using out-of-band communications and the component continues at block 803. In block 803, the component forwards frame to the administrative port and then completes. In block 804, the component forwards the frame based on the port map of the label table and then completes.
Patent CitationsCited PatentFiling datePublication dateApplicantTitleUS4777595 *Mar 24, 1986Oct 11, 1988Digital Equipment CorporationApparatus for transferring blocks of information from one node to a second node in a computer networkUS4872160Aug 30, 1988Oct 3, 1989American Telephone And Telegraph Company, At&T Bell LaboratoriesIntegrated packetized voice and data switching systemUS5247516Mar 28, 1991Sep 21, 1993Sprint International Communications Corp.Configurable composite data frameUS5303233May 20, 1991Apr 12, 1994Nec CorporationArrangement favorably carrying out a connection by the use of an intermediate conversion parameter between input and output virtual channel identifiers (VCI's)US5327426Sep 27, 1991Jul 5, 1994Echelon CorporationMethod and apparatus for preventing unnecessary retransmission of messages in a networked messaging systemUS5327552Jun 22, 1992Jul 5, 1994Bell Communications Research, Inc.Method and system for correcting routing errors due to packet deflectionsUS5339318Oct 22, 1992Aug 16, 1994Fujitsu LimitedVPI and VCI assignment system in ATM systemUS5343473Aug 7, 1992Aug 30, 1994International Business Machines CorporationMethod of determining whether to use preempt/resume or alternate protocol for data transmissionUS5412653Oct 15, 1993May 2, 1995International Business Machines CorporationDynamic switch cascading systemUS5432783Feb 7, 1994Jul 11, 1995Bell Communications Research, Inc.System and method for providing egress policing for broadband virtual private networksUS5440547Jan 5, 1994Aug 8, 1995Kabushiki Kaisha ToshibaData-transfer routing management for packet-oriented digital communication system including ATM networksUS5548639Oct 22, 1992Aug 20, 1996Fujitsu LimitedDistributed control of telecommunication network for setting up an alternative communication pathUS5550816Dec 29, 1994Aug 27, 1996Storage Technology CorporationMethod and apparatus for virtual switchingUS5590119Aug 28, 1995Dec 31, 1996Mci Communications CorporationDeterministic selection of an optimal restoration route in a telecommunications networkUS5675579Jun 7, 1995Oct 7, 1997Tandem Computers IncorporatedMethod for verifying responses to messages using a barrier messageUS5675807Jun 7, 1995Oct 7, 1997Tandem Computers IncorporatedInterrupt message delivery identified by storage location of received interrupt dataUS5680402Oct 5, 1994Oct 21, 1997International Business Machines CorporationPriority broadcast and multi-cast for unbuffered multi-stage networksUS5689505Jan 16, 1996Nov 18, 1997Lucent Technologies Inc.Buffering of multicast cells in switching networksUS5689506Jan 16, 1996Nov 18, 1997Lucent Technologies Inc.Multicast routing in multistage networksUS5689689Jun 7, 1995Nov 18, 1997Tandem Computers IncorporatedClock circuits for synchronized processor systems having clock generator circuit with a voltage control oscillator producing a clock signal synchronous with a master clock signalUS5734719Dec 10, 1996Mar 31, 1998International Business Systems, IncorporatedDigital information accessing, delivery and production systemUS5751932Jun 7, 1995May 12, 1998Tandem Computers IncorporatedFail-fast, fail-functional, fault-tolerant multiprocessor systemUS5751955Jun 7, 1995May 12, 1998Tandem Computers IncorporatedMethod of synchronizing a pair of central processor units for duplex, lock-step operation by copying data into a corresponding locations of another memoryUS5774067Oct 16, 1997Jun 30, 1998International Business Machines CorporationFlash-flooding multi-stage interconnection network with parallel path seeking switching elementsUS5790546Dec 4, 1995Aug 4, 1998Cabletron Systems, Inc.Method of transmitting data packets in a packet switched communications networkUS5790776Jun 7, 1995Aug 4, 1998Tandem Computers IncorporatedApparatus for detecting divergence between a pair of duplexed, synchronized processor elementsUS5805804Mar 12, 1997Sep 8, 1998Oracle CorporationMethod and apparatus for scalable, high bandwidth storage retrieval and transportation of multimedia data on a networkUS5809025Mar 15, 1996Sep 15, 1998Motorola, Inc.Method for routing a cell in a cell-based switching systemUS5818842Jan 20, 1995Oct 6, 1998Newbridge Networks CorporationTransparent interconnector of LANs by an ATM networkUS5838894Jun 7, 1995Nov 17, 1998Tandem Computers IncorporatedLogical, fail-functional, dual central processor units formed from three processor unitsUS5867501Jun 7, 1995Feb 2, 1999Tandem Computers IncorporatedEncoding for communicating data and commandsUS5872783 *Jul 24, 1996Feb 16, 1999Cisco Systems, Inc.Arrangement for rendering forwarding decisions for packets transferred among network switchesUS5881246Jun 12, 1996Mar 9, 1999Bay Networks, Inc.System for generating explicit routing advertisements to specify a selected path through a connectionless network to a destination by a specific routerUS5892766 *Feb 22, 1996Apr 6, 1999Fujitsu, Ltd.Method and apparatus for coordinating access to an output of a routing device in a packet switching networkUS5892923Dec 28, 1995Apr 6, 1999Hitachi, Ltd.Parallel computer system using properties of messages to route them through an interconnect network and to select virtual channel circuits therewithinUS5898830Oct 17, 1996Apr 27, 1999Network Engineering SoftwareFirewall providing enhanced network security and user transparencyUS5914953Jun 7, 1995Jun 22, 1999Tandem Computers, Inc.Network message routing using routing table information and supplemental enable information for deadlock preventionUS5917820Jun 10, 1996Jun 29, 1999Cisco Technology, Inc.Efficient packet forwarding arrangement for routing packets in an internetworkUS5940596Aug 4, 1997Aug 17, 1999I-Cube, Inc.Clustered address caching system for a network switchUS5943339Mar 21, 1996Aug 24, 1999Northern Telecom LimitedFor conveying narrow band traffic over a virtual trunk groupUS5953338Dec 13, 1996Sep 14, 1999Northern Telecom LimitedDynamic control processes and systems for asynchronous transfer mode networksUS5964835Jun 7, 1995Oct 12, 1999Tandem Computers IncorporatedStorage access validation to data messages using partial storage address data indexed entries containing permissible address range validation for message sourceUS5999531Apr 17, 1998Dec 7, 1999Cabletron Systems, Inc.Method and system for identifying ports and forwarding packets in a multiport switchUS6021263Feb 16, 1996Feb 1, 2000Lucent Technologies, Inc.Management of ATM virtual circuits with resources reservation protocolUS6021495May 30, 1997Feb 1, 20003Com CorporationMethod and apparatus for authentication process of a star or hub network connection ports by detecting interruption in link beatUS6028863Mar 4, 1997Feb 22, 2000Fujitsu LimitedMethod and apparatus for negotiating connection identifierUS6032205Mar 6, 1998Feb 29, 2000Hitachi, Ltd.Crossbar switch system for always transferring normal messages and selectively transferring broadcast messages from input buffer to output buffer when it has sufficient space respectivelyUS6034956 *Jun 29, 1998Mar 7, 2000International Business Machines CorporationMethod of simultaneously attempting parallel path connections in a multi-stage interconnection networkUS6041049May 6, 1997Mar 21, 2000International Business Machines CorporationMethod and apparatus for determining a routing table for each node in a distributed nodal systemUS6047323Jun 27, 1996Apr 4, 2000Hewlett-Packard CompanyCreation and migration of distributed streams in clusters of networked computersUS6078963Jan 16, 1998Jun 20, 2000At&T Corp.Router with de-centralized processing using intelligent portsUS6085238Apr 22, 1997Jul 4, 2000Matsushita Electric Works, Ltd.Virtual LAN systemUS6094712Dec 4, 1996Jul 25, 2000Giganet, Inc.Computer network interface for direct mapping of data transferred between applications on different host computers from virtual addresses to physical memory addresses application dataUS6104700Feb 3, 1998Aug 15, 2000Extreme NetworksPolicy based quality of serviceUS6108708May 21, 1997Aug 22, 2000Nec CorporationConnection-oriented network using distributed network resources and predetermined VPIs for fast VC establishmentUS6147976 *Jun 24, 1996Nov 14, 2000Cabletron Systems, Inc.Fast network layer packet filterUS6151316Dec 18, 1997Nov 21, 2000Advanced Micro Devices, Inc.Apparatus and method for synthesizing management packets for transmission between a network switch and a host controllerUS6151689Dec 9, 1996Nov 21, 2000Tandem Computers IncorporatedDetecting and isolating errors occurring in data communication in a multiple processor systemUS6157967Dec 30, 1997Dec 5, 2000Tandem Computer IncorporatedMethod of data communication flow control in a data processing system using busy/ready commandsUS6169742Dec 18, 1997Jan 2, 2001Advanced Micro Devices, Inc.Multiport data network switch having direct media access control link to external managementUS6172991Feb 17, 1998Jan 9, 2001Nec CorporationATM Network with a filtering table for securing communicationUS6195335Jul 6, 1998Feb 27, 2001International Business Machines CorporationData switchUS6215412Jun 2, 1995Apr 10, 2001International Business Machines CorporationAll-node switch-an unclocked, unbuffered, asynchronous switching apparatusUS6216173Feb 3, 1998Apr 10, 2001Redbox Technologies LimitedMethod and apparatus for content processing and routingUS6216200Mar 14, 1995Apr 10, 2001Mips Technologies, Inc.Address queueUS6219706Oct 16, 1998Apr 17, 2001Cisco Technology, Inc.Access control for networksUS6262976Sep 17, 1999Jul 17, 2001Ordered Networks, Inc.System and method for network flow optimization using traffic classesUS6278714Feb 6, 1998Aug 21, 2001Sun Microsystems, Inc.Efficient hardware implementation of virtual circuit bunchingUS6292488May 22, 1998Sep 18, 2001Compaq Computer CorporationMethod and apparatus for resolving deadlocks in a distributed computer systemUS6292839Dec 9, 1998Sep 18, 20013Com CorporationMethod and system for reflexive tunnelingUS6301252Apr 16, 1998Oct 9, 2001Mantra Communications IncorporatedControl and management of ATM networks from server computersUS6304549May 8, 1997Oct 16, 2001Lucent Technologies Inc.Virtual path management in hierarchical ATM networksUS6381242Aug 29, 2000Apr 30, 2002Netrake CorporationContent processorUS6381244Mar 17, 1998Apr 30, 2002Fujitsu LimitedConnectionless communication methodUS6385197Jul 9, 1999May 7, 2002Allied Telesyn International Corp.Virtual port trunking method and apparatusUS6396815Aug 28, 2000May 28, 2002Virata LimitedProxy-controlled ATM subnetworkUS6400730Feb 8, 2000Jun 4, 2002Nishan Systems, Inc.Method and apparatus for transferring data between IP network devices and SCSI and fibre channel devices over an IP networkUS6411806Jul 6, 2000Jun 25, 2002Mobile Satellite Ventures LpVirtual network configuration and management system for satellite communications systemUS6430626Aug 1, 2000Aug 6, 2002Compaq Computer CorporationNetwork switch with a multiple bus structure and a bridge interface for transferring network data between different busesUS6460088May 21, 1999Oct 1, 2002Advanced Micro Devices, Inc.Method and apparatus for port vector determination at egressUS6493347Jul 24, 1997Dec 10, 2002Juniper Networks, Inc.Memory organization in a switching deviceUS6510151Sep 19, 1996Jan 21, 2003Enterasys Networks, Inc.Packet filtering in connection-based switching networksUS6535518Aug 3, 2000Mar 18, 2003Simpletech Inc.System for bypassing a server to achieve higher throughput between data network and data storage systemUS6542739Oct 6, 2000Apr 1, 2003Mobile Satellite Ventures, LpPriority and preemption service system for satellite related communication using central controllerUS6556547Dec 15, 1998Apr 29, 2003Nortel Networks LimitedMethod and apparatus providing for router redundancy of non internet protocols using the virtual router redundancy protocolUS6563831Aug 19, 1998May 13, 2003Avici SystemsRouter with virtual channel allocationUS6578122 *Mar 1, 2001Jun 10, 2003International Business Machines CorporationUsing an access key to protect and point to regions in windows for infinibandUS6591310 *May 11, 2000Jul 8, 2003Lsi Logic CorporationMethod of responding to I/O request and associated reply descriptorUS6594329 *Nov 1, 1999Jul 15, 2003Intel CorporationElastic bufferUS6594712 *Oct 20, 2000Jul 15, 2003Banderacom, Inc.Inifiniband channel adapter for performing direct DMA between PCI bus and inifiniband linkUS6597691Aug 30, 1999Jul 22, 2003Ancor Communications, Inc.High performance switchingUS6601148 *Mar 1, 2001Jul 29, 2003International Business Machines CorporationInfiniband memory windows management directly in hardwareUS6608819Nov 30, 2000Aug 19, 2003Mcdata CorporationMethod for scoring queued frames for selective transmission through a switchUS6611883 *Nov 16, 2000Aug 26, 2003Sun Microsystems, Inc.Method and apparatus for implementing PCI DMA speculative prefetching in a message passing queue oriented bus systemUS6614758Jan 10, 2002Sep 2, 2003Broadcom CorpLoad balancing in link aggregation and trunkingUS6622193 *Nov 16, 2000Sep 16, 2003Sun Microsystems, Inc.Method and apparatus for synchronizing interrupts in a message passing queue oriented bus systemUS6636516Mar 7, 2000Oct 21, 2003Nec CorporationQOS-based virtual private network using ATM-based internet virtual connectionsUS6650646Aug 14, 1998Nov 18, 2003Alcatel Canada Inc.Digital communications systemUS6651099Jun 30, 2000Nov 18, 2003Hi/Fn, Inc.Method and apparatus for monitoring traffic in a networkUS6697379May 17, 1999Feb 24, 2004Inria Institut National De Recherche En Informatique Et En AutomatiqueSystem for transmitting messages to improved stations, and corresponding processingUS6704831 *Nov 16, 2000Mar 9, 2004Sun Microsystems, Inc.Method and apparatus for converting address information between PCI bus protocol and a message-passing queue-oriented bus protocolUS6707800Oct 1, 1998Mar 16, 2004Hughes Electronics CorporationATM network with central call processorUS6760775Mar 6, 2000Jul 6, 2004At&T Corp.System, method and apparatus for network service load and reliability managementUS6775719 *Sep 28, 2000Aug 10, 2004Intel CorporationHost-fabric adapter and method of connecting a host system to a channel-based switched fabric in a data networkUS6810428Jun 5, 1998Oct 26, 2004Iwics Inc.Method of operation of a multi-station networkUS6831916 *Sep 28, 2000Dec 14, 2004Balaji ParthasarathyHost-fabric adapter and method of connecting a host system to a channel-based switched fabric in a data networkUS6847613Mar 12, 2001Jan 25, 2005Hitachi, Ltd.Method of monitoring quality of communication for each flowUS6859867 *May 31, 2000Feb 22, 2005Intel CorporationTranslation and protection table and method of using the same to validate access requestsUS6895006Oct 2, 2000May 17, 2005Juniper Networks, Inc.Unicast/multicast systemUS6914911 *Jan 17, 2001Jul 5, 2005Telefonaktiebolaget Lm EricssonCombining narrowband applications with broadband transportUS6917614Sep 17, 1999Jul 12, 2005Arris International, Inc.Multi-channel support for virtual private networks in a packet to ATM cell cable systemUS6917987 *Mar 26, 2001Jul 12, 2005Intel CorporationMethodology and mechanism for remote key validation for NGIO/InfiniBand� applicationsUS6938169Dec 10, 1999Aug 30, 2005Sun Microsystems, Inc.Channel-specific file system views in a private network using a public-network infrastructureUS6944152Aug 22, 2000Sep 13, 2005Lsi Logic CorporationData storage access through switched fabricUS6948004 *Mar 28, 2001Sep 20, 2005Intel CorporationHost-fabric adapter having work queue entry (WQE) ring hardware assist (HWA) mechanismUS6990528 *Oct 19, 2000Jan 24, 2006International Business Machines CorporationSystem area network of end-to-end context via reliable datagram domainsUS20010010692Jul 24, 1997Aug 2, 2001Juniper NetworksMemory organization in a switching deviceUS20010030968Jan 17, 2001Oct 18, 2001Magnus HallenstalCombining narrowband applications with broadband transportUS20020028656Feb 1, 2001Mar 7, 2002Yechiam YeminiMethod and apparatus for providing forwarding and replication services on a dynamically addressed networkUS20020049875 *Jan 31, 2001Apr 25, 2002Biran GioraData communications interfacesUS20020049878 *Jan 31, 2001Apr 25, 2002Giora BiranData communications interfacesUS20020071450 *Dec 8, 2000Jun 13, 2002Gasbarro Dominic J.Host-fabric adapter having bandwidth-optimizing, area-minimal, vertical sliced memory architecture and method of connecting a host system to a channel-based switched fabric in a data networkUS20020073257 *Dec 7, 2000Jun 13, 2002Ibm CorporationTransferring foreign protocols across a system area networkUS20020124117 *Mar 1, 2001Sep 5, 2002Ibm CorporationInfiniband memory windows management directly in hardwareUS20020124148 *Mar 1, 2001Sep 5, 2002Ibm CorporationUsing an access key to protect and point to regions in windows for infinibandUS20020141424 *Mar 28, 2001Oct 3, 2002Gasbarro Dominic J.Host-fabric adapter having work queue entry (WQE) ring hardware assist (HWA) mechanismUS20020159385 *Apr 26, 2001Oct 31, 2002Susnow Dean S.Link level packet flow control mechanismUS20020184392 *Mar 26, 2001Dec 5, 2002Balaji ParthasarathyMethodology and mechanism for remote key validation for NGIO/infiniBandTM applicationsUS20020191599 *Mar 30, 2001Dec 19, 2002Balaji ParthasarathyHost- fabrec adapter having an efficient multi-tasking pipelined instruction execution micro-controller subsystem for NGIO/infinibandTM applications* Cited by examinerNon-Patent CitationsReference1"FloodGate-1 Data Sheet, Enterprise Traffic Management," Checkpoint Software Technologies Ltd., http://www.checkpoint.com, Jan. 1998 (8 pages).2"Getting Started with FireWall-1, A Simple Configuration," Checkpoint Software Technologies Ltd., <http://www.messagenet.co.uk/products/manuals/firewall/tutorial.htm>, Jan. 1997 (7 pages).3InfiniBand, <<http://searchsmb.techtarget.com/gDefinition/0,,sid44<SUB>-</SUB>gci>214596,00.html>, Jun. 5, 2001 (3 Pages).4Kohalmi, S., Anatomy of an IP Service Edge Switch: Accelerating Advance IP Services with a Pipelined Architecture, Quarry Technologies, Inc., Jan. 2001 (10 pages).5Meggyesi, Z., "Fiber Channel Overview," <http://hsi.web.cern.ch/HSI/fcs/spec/overview.htm> Research Institute for Particle and Nuclear Physics, Aug. 15, 1994 (10 Pages).Referenced byCiting PatentFiling datePublication dateApplicantTitleUS7200674 *Jul 19, 2002Apr 3, 2007Open Invention Network, LlcElectronic commerce community networks and intra/inter community secure routing implementationUS7865586Mar 31, 2008Jan 4, 2011Amazon Technologies, Inc.Configuring communications between computing nodesUS8046480Mar 31, 2008Oct 25, 2011Amazon Technologies, Inc.Embedding overlay virtual network addresses in underlying substrate network addressesUS8055789Mar 27, 2007Nov 8, 2011Amazon Technologies, Inc.Configuring intercommunications between computing nodesUS8060630 *Nov 27, 2002Nov 15, 2011Symantec Operating CorporationCreating and configuring virtual fabrics in storage area networksUS8151318 *Feb 25, 2003Apr 3, 2012Cisco Technology, Inc.Method and apparatus for reliably and asymmetrically distributing security information within a fibre channel fabricUS8166201Oct 18, 2011Apr 24, 2012Amazon Technologies, Inc.Configuring intercommunications between computing nodesUS8224931Mar 31, 2010Jul 17, 2012Amazon Technologies, Inc.Managing use of intermediate destination computing nodes for provided computer networksUS8228848Nov 17, 2009Jul 24, 2012Sierra Wireless, Inc.Method and apparatus for facilitating push communication across a network boundaryUS8396946Mar 31, 2010Mar 12, 2013Amazon Technologies, Inc.Managing integration of external nodes into provided computer networksUS8429739Mar 31, 2008Apr 23, 2013Amazon Technologies, Inc.Authorizing communications between computing nodesUS8488446Oct 27, 2010Jul 16, 2013Amazon Technologies, Inc.Managing failure behavior for computing nodes of provided computer networksUS8510420Jun 27, 2012Aug 13, 2013Amazon Technologies, Inc.Managing use of intermediate destination computing nodes for provided computer networksUS8543554Aug 10, 2010Sep 24, 2013ScalArc Inc.Method and system for transparent database query cachingUS8549347Dec 20, 2010Oct 1, 2013Amazon Technologies, Inc.Techniques for network replicationUS8560646Sep 28, 2010Oct 15, 2013Amazon Technologies, Inc.Managing communications using alternative packet addressingUS8612627Mar 3, 2010Dec 17, 2013Amazon Technologies, Inc.Managing encoded multi-part communications for provided computer networksUS8644188Jun 25, 2009Feb 4, 2014Amazon Technologies, Inc.Providing virtual networking functionality for managed computer networksUS8645508Mar 3, 2010Feb 4, 2014Amazon Technologies, Inc.Managing external communications for provided computer networksUS8683023Jun 30, 2010Mar 25, 2014Amazon Technologies, Inc.Managing communications involving external nodes of provided computer networksUS8738745Mar 31, 2010May 27, 2014Amazon Technologies, Inc.Managing use of intermediate destination hardware devices for provided computer networksUS8763091Aug 24, 2010Jun 24, 2014ScalArc Inc.Method and system for user authentication offload in a transparent database load balancerUS8798043 *Jun 30, 2003Aug 5, 2014Brocade Communications Systems, Inc.Apparatus and method for routing traffic in multi-link switchUS8812730May 17, 2011Aug 19, 2014Sierra Wireless, Inc.Method and apparatus for network port and network address translation* Cited by examinerClassifications U.S. Classification370/398, 709/223, 370/397, 709/238, 709/245, 370/399, 370/422International ClassificationH04J1/16, G06F15/16, H04J3/16, G06F11/30, G06F15/173, H04L12/50, H04L12/56, H04L12/28Cooperative ClassificationH04L67/1014, H04L69/22, H04L69/08, H04L67/1031, H04L67/1008, H04L67/1002, H04L67/322, H04L67/1029, H04L63/0245, H04L63/0236, H04L49/552, H04L49/357, H04L49/101European ClassificationH04L29/08N9A7, H04L29/08N31Q, H04L29/08N9A1E, H04L29/08N9A, H04L49/55A, H04L29/08N9A1B, H04L29/06E, H04L49/35H2, H04L29/08N9A9Legal EventsDateCodeEventDescriptionDec 27, 2013FPAYFee paymentYear of fee payment: 8Dec 28, 2009FPAYFee paymentYear of fee payment: 4Oct 26, 2001ASAssignmentOwner name: BOEING COMPANY, THE, WASHINGTONFree format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:FOSTER, MICHAEL S.;DORSETT, MICHAEL A.;REEL/FRAME:012504/0994Effective date: 20011025RotateOriginal ImageGoogle Home - Sitemap - USPTO Bulk Downloads - Privacy Policy - Terms of Service - About Google Patents - Send FeedbackData provided by IFI CLAIMS Patent Services