Patent Publication Number: US-6701448-B1

Title: Establishing and maintaining master and standby LAN emulation server connections to provide a fault tolerant ELAN

Description:
FIELD OF THE INVENTION 
     The present invention relates generally to LAN emulation. More specifically, a system for establishing and maintaining master and standby LAN emulation server connections to provide a fault tolerant LAN without creating excess traffic or load on an LECS is disclosed. 
     BACKGROUND OF THE INVENTION 
     Purpose of LAN Emulation 
     In general, the purpose of Asynchronous Transfer Mode (ATM) LAN Emulation (LANE) is to emulate a multiple-access, broadcast media (e.g., Etheniet) over a point-to-point, virtual circuit oriented media (ATM). The specification for LAN Emulation over an ATM may be found in the LANE ATM specification, “LAN emulation over ATM” version 1.0 Document No. AF-LANE-0021 which is herein incorporated by reference, “LAN emulation over ATM Version 1.0 Addendum” Document No. AF-LANE-0050.000 which is herein incorporated by reference (collectively, the “LANE specification version 1”), and in version 2 of the LANE ATM specification, “LAN emulation over ATM Version 2 LUNI Specification Documnent No. AF-LANE-0084.000 which is herein incorporated by reference (the “LANE specification version 2”). It is necessary to emulate the services available over the LAN so that the existing base of software can take advantage of the ATM network. LANE provides an easy way to connect existing LAN applications into the ATM environment by supporting “unmodified” interaction amongst legacy applications on the ATM network, and by supporting the interconnection of ATM networks with legacy LAN&#39;s. This provides interoperability between applications on the ATM network and those on legacy LANs. 
     LAN Emulation Components 
     FIG. 1A shows the components of an emulated LAN (ELAN) implemented over an ATM network  100 . The ELAN components include at least one LEC  102 . LEC  102  may include end stations (e.g., hosts), routers or bridges and each LEC  102  is responsible for setting up the association with the ELAN assets to provide LANE services to a local host or router. Two other components of the LANE service are a LAN Emulation Server (LES)  104  and a Broadcast and Unknown Server (BUS)  106 . LES  104  provides a registration facility to allow clients to register unicast and multicast media access control (MAC) addresses and/or ring route descriptors, and a forwarding service for control frames. BUS  106  receives frames from LEC  102  and retransmits them to other LEC&#39;s on the ELAN. The LANE Service may use more than one BUS, and the BUS  106  may be distributed over multiple physical entities or collocated on a single box. In certain implementations, a single LES/BUS is collocated on one router. Another ELAN component, an LECS  108 , enables LEC  102  to find and to connect to LES  104  for the proper ELAN over the ATM network  100 . 
     LES  104  and LECS  108  are referred to as “edge devices” on the ELAN because they are not required for the operation of the ATM network itself, but are required in order for the ATM network to emulate a LAN. The functions and protocols implemented for these devices are further described in the LANE specification. 
     The LECS, LES and BUS may be implemented on various network devices including switches, hosts, routers or bridges. Any device which includes an ATM interface and can support virtual circuits may function as the LECS or LES, including a PC with an ATM interface. Such devices include a processor capable of executing logic operations and memory, along with an ATM interface. Based on the operations executed by the processor, the device is able to process outside events, record those events in memory, and also to determine states and store those states in memory. 
     FIG. 1B shows how LEC  102  connects to the LANE service so that LEC  102  may perform data forwarding, address resolution and registration of MAC addresses with LES  104 . When LEC  102  wishes to connect to an ELAN, it locates LECS  108  on ATM network  100  at a well-known address, and then sets up a point to point configure direct connection  112  with LECS  108 . LECS  108  services configuration requests from LEC  102  and tells LEC  102  which ELAN it should join and the address of the LES that it should use. LEC  102  then sets up a control direct connection  114  with LES  104 . LES  104  establishes a point to multipoint connection  116  with LEC  102 , as well as with any other LEC  110  on the ELAN. Finally, LEC  102  establishes a multicast send point to point connection  118  with BUS  106  and a point to multipoint multicast forwvard connection  120  is set up between BUS  106  and LEC  102  and any other LEC  110 . 
     Once LEC  102  registers with LES  104 , it receives address resolution requests for unregistered destinations from LES  104 . LES  104  provides a registration facility to allow clients to register unicast and multicast MAC addresses and/or token ring route descriptors, and a forwarding service for control frames. Optionally, it may generate control frames. Bridges may register with LES  104  any MAC addresses that they are capable of foraarding. BUS  106  receives frames from clients or other multicast servers, and retransmits them to other clients. There is at least one BUS in an emulated LAN which handles broadcast and multicast addresses. 
     In order for the ELAN to function, the servers that function as LES  104  and LECS  108  must remain operational. Wien LEC  102  wants to participate in the ELAN, it must first ask LECS  108  for the address of LES  104  and then execute a joining procedure with the LES. When this transaction is completed, LEC  102  connects to LES  104 . This completes the joining of LEC  102  to the ELAN. LEC  102  must maintain a control direct connection with LES  104  as long as it continues to be connected to the ELAN. LES  104  and LECS  108  therefore represent potential single points of failure for the ELAN. Although the LANE specification details the operation of the LECS&#39;s and the LES&#39;s as described above, there is no provision for a system of redundant or backup servers. 
     What is needed is a redundancy scheme that provides a way for backup or slave servers to automatically and transparently take over the tasks of a failed master server so that the LANE service may continue to operate without significant disruption. Because LES  104  and LECS  108  may be interconnected to a numerous and changing group of LEC&#39;s, a way is needed to both detect failure of these edge devices and redesignate a slave or backup device as the master device and also to reconnect the LEC&#39;s to the new master devices with minimum disruption to the LEC&#39;s. 
     In addition, in a system where a large number of LEC&#39;s may be connected to an LES, a method is needed to direct LEC&#39;s to a backup LES upon failure of a master LES that does not require all of the LEC&#39;s to contact the LECS. Also, it would be desirable if a method of determining a master LES from among a group of available LES&#39;s could be devised that would minimize switching among LES&#39;s while a system is being brought on line. Finally, it is important that such a system be able to accommodate older LECS&#39;s, LES&#39;s and LEC&#39;s that have not been modified to operate in the system as well as LECS&#39;s, LES&#39;s and LEC&#39;s that have been designed to operate in the system. Thus, a robust redundancy scheme compatible with the LANE specification that reduces backend traffic between LES&#39;s and the master LECS and that also reduces the need for clients to contact an LECS when an LES fails is needed. 
     SUMMARY OF THE INVENTION 
     A redundancy scheme that reduces backend traffic between LES&#39;s and the master LECS and that also reduces the need for clients to contact an LECS when an LES fails is disclosed. The master LECS broadcasts configuration information to all LES&#39;s that includes a list of the master LES and any available backup LES&#39;s. This information is relayed by the LES&#39;s to modified LEC&#39;s that keep track of the available LES&#39;s. The LEC&#39;s use such information instead of contacting the LECS to determine the master LES before joining the LAN. When the master LES fails, LEC&#39;s may switch over to the highest priority backup LES without contacting the LECS. In addition, a method is disclosed for designating the first LES made available as the master LES to avoid moving LEC&#39;s when a higher priority LES comes on line. Finally a system wherein LES&#39;s verify LEC&#39;s without contacting the LECS is disclosed. 
     It should be appreciated that the present invention can be implemented in numerous ways, including as a process, an apparatus, a system, a device, a method, or a computer readable medium. Several inventive embodiments of the present invention are described below. 
     In one embodiment, a method of transferring an enhanced LAN Emulation Client from a first LAN Emulation Server to a second LAN Emulation Server includes determining at an LECS the addresses of the first LES and the second LES. The first LES and the second LES are connected to the LECS. The addresses of the first LES and the second LES are sent to the enhanced LEC so that the addresses of the first LES and the second LES may be stored at the enhanced LEC. When the enhanced LEC determines that the first LES has failed, the address of the second LES is looked up at the enhanced LEC and the enhanced LEC connects to the second LES. 
    
    
     These and other features and advantages of the present invention will be presented in more detail in the following specification of the invention and the accompanying figures which illustrate by way of example the principles of the invention. 
     BRIEF DESCRIPTION OF THE DRAWINGS 
     The present invention will be readily understood by the following detailed description in conjunction with the accompanying drawings, wherein like reference numerals designate like structural elements, and in which: 
     FIG. 1A shows the components of an emulated LAN (ELAN) implemented over an ATM network. 
     FIG. 1B shows how a LEC connects to the LANE service so that the LEC may perform data forwardings address resolution and registration of MAC addresses with an LES. 
     FIG. 2 is a block diagram illustrating a LAN emulation system  200  that services LEC&#39;s  201 A,  201 B, and  201 C. 
     FIG. 3 is a block diagram illustrating a LAN emulation system  300  that reduces the traffic to the master LECS. 
     FIG. 4 is a block diagram illustrating several of the packet formats described in connection with FIG.  3 . 
     FIG. 5 is a state diagram illustrating the state of an LES upon the occurrence of certain events. 
     FIG. 6 is a flow chart illustrating a process  600  running on an enhanced LES for processing join requests from LEC&#39;s. 
     FIG. 7 is a flow chart illustrating how an LECS handles connection events. 
     FIG. 8 is a flow chart illustrating a process  800  executed by an enhanced LECS for the purpose of handling CONFIG requests. 
     FIG. 9 is a flowchart illustrating a process executed by an LEC upon receiving a targetless LEARP on a control distribute connection from an LES. 
     FIG. 10 is a flow chart illustrating a process  1000  executed on an enhanced LEC for the purpose of handling various connection events. 
    
    
     DETAILED DESCRIPTION 
     A detailed description of a preferred embodiment of the invention is provided below. While the invention is described in conjunction with that preferred embodiment, it should be understood that the invention is not limited to any one embodiment. On the contrary, the scope of the invention is limited only by the appended claims and the invention encompasses numerous alternatives, modifications and equivalents. For the purpose of example, numerous specific details are set forth in the following description in order to provide a thorough understanding of the present invention. The present invention may be practiced according to the claims without some or all of these specific details. For the purpose of clarity, details relating to technical material that is known in the technical fields related to the invention has not been described in detail in order not to unnecessarily obscure the present invention in such detail. 
     Definitions 
     The following terms and acronyms are used in the instant specification. Their definitions are provided here to assist in understanding the preferred embodiments described herein. 
     ATM address The ATM address is the address given to the User Network Interface (UNI) to identify the endpoint to which the user wishes to connect. 
     BUS Broadcast and Unknown Server. The BUS receives frames from LEC&#39;s and retransmits them to other LEC&#39;s on the ELAN. 
     ELAN Emulated LAN. The emulation of a single Ethernet LAN or Token Ring LAN segment over ATM. 
     LAN Local Area Network 
     LANE LAN Emulation. The mechanism which emulates a multicast LAN media over a circuit switched network. 
     LEARP LAN Emulation Address Resolution Protocol 
     LEC LAN Emulation Client. The LEC is responsible for setting up the association with the LES and BUS to provide LAN emulation services to the local host or router. 
     LECS LAN Emulation Configuration Server. The LECS exists somnewhere in the ATM network at a well-known address and tells the LEC which ELAN it should join. 
     LES LAN Emulation Server. The LES provides a registration facility to allow clients to register unicast and multicast MAC addresses and/or ring route descriptors, and a forwarding service for control frames. 
     MAC Address A MAC address is an address of a device at tile sublayer of the data link layer, defined by the IEEE 802 committee that deals with issues specific to a particular type of LAN. The types of LAN for which MAC addresses are available include token ring, FDDI, and Ethenet. A MAC address is generally intended to apply to a specific physical device no matter where it is plugged into the network. Thus, a MAC address is generally hardcoded into the device—on a router&#39;s ROM, for example. This should be distinguished from the case of a network layer address, described above, which changes depending upon where it is plugged into the network. MAC is an acronym for Media Access Control. 
     Packet A packet is a collection of data and control information including source and destination node addresses, formatted for transmission from one node to another. In the context of this invention, it is important to note that hosts on one LAN send packets to hosts on another LAN through a router or bridge connecting the LANs. 
     Systems and Environment 
     As described below, the present invention provides methods for providing backup LECS&#39;s and LES&#39;s in a LAN emulation on an ATM network. In addition, the present invention provides apparatus for performing these backup methods. The apparatus of this invention may be specially constructed for the required purposes, or it may be a general-purpose programmable machine selectively activated or configured by firmware or a computer program. Such devices preferably include an ATM interface along with a memory and a processor capable of executing logic operations. Based on the operations executed by the processor, the device is able to process outside events, record those events in memory, and also to determine states and store those states in memory. Examples of suitable machines include routers, bridges, switches, workstations. personal computers, and virtually any intelligent machine on a network. In each case, Such machine must be configured to act as an LEC, LES, or LECS in accordance with this invention. 
     In one example, the LEC, LES, or LECS of this invention is implemented on a router. Generally, such router may include a master central processing unit (CPU), an ATM interface, and low, medium, and high-speed interfaces. In preferred embodiments, the CPU is responsible for such router tasks as routing table computations and network management. It may include one or more microprocessor chips such as the MIPS R4600 reduced instruction set computer (RISC) chip, or other available chips. In a preferred embodiment, non-volatile RAM and/or ROM also form part of the CPU. Of course, there are many acceptable ways in which memory could be coupled to the system. 
     The interfaces may be provided as interface cards. Generally, they control the sending and receiving of data packets over the network and sometimes support other peripherals used with the router. The low and medium speed interfaces may include a multiport communications interface, a serial communications interface, and a token ring interface. The high-speed interfaces may include an FDDI interface and a multiport Ethernet interface. Preferably, each of these interfaces (low/medium and high speed) includes (1) a plurality of points appropriate for communication with the appropriate media, and (2) an independent processor, and in some instances (3) volatile RAM. The independent processors control such communications intensive tasks as packet switching and filtering, and media control and management. By providing, separate processors for the communications intensive tasks, this architecture permits the master microprocessor to efficiently perform routing computations, network diagnostics, security functions, etc. 
     The low and medium speed interfaces are coupled to the master CPU through a data, control, and address bus. High-speed interfaces are connected to the bus through a fast data, control, and address bus which is in turn connected to a bus controller. 
     As noted, a properly configured device will include an ATM interface and means for supporting virtual circuits. The appropriate hardware and software for providing ATM interfaces is generally known in the field. In 1991, an entity known as the ATM Forum was founded to standardize ATM technology. A substantial body of information regarding deployment of ATM technology is available from the ATM Forum. Cites to many references pertaining to ATM technology are available through the ATM Forum&#39;s World Wide Web site at www.ATMForum.com. Specific references include McDysan et al., “ATM Theory and Application,” McGraw Hill, 1995; Minoi et al., “ATM &amp; Cell Relay Service for Corporate Environments,” McGraw Hill, 1994; and Prycker, “Asynchronous Transfer Mode—Solution for Broadband ISDN, 2nd Edition, Ellis Horvood, 1993. Each of these references is incorporated herein by reference for all purposes. 
     Specific ATM-capable hardware includes the 7000 series of routers and the LightStream® line of ATM switches available from Cisco Systems, Inc. of San Jose Calif. In addition, add-on ATM adapters are available from Cisco Systems and other companies providing networking equipment. 
     LAN emulation and the associated redundancy capability of this invention may be provided on the above-mentioned ATM hardware via software or hardware modules. In a preferred embodiment, software for implementing the redundancy capability is provided as a modified version of the suite of programs provided by Cisco Systems, Inc. as the Cisco Internet work Operating System (Cisco IOS®) software for ATMs. Of course, other systems such as workstations running a UNIX operating system (e.g., a SPARCStation available from Sun Microsystems of Mountain View California) may be configured to implement the present invention. 
     FIG. 2 is a block diagram illustrating a LAN emulation system  200  that services LEC&#39;s  201 A,  201 B, and  201 C. The three LEC&#39;s are connected to an LES  210 . Each LEC has a data connection to LES  210  that allows point to point communication in both directions as well as broadcast communication of data to all of the LEC&#39;s. These data connections are not shown in FIG.  2 . Instead, only the control direct connection from the LES to each of the LEC&#39;s is shown. 
     In addition to LES  210 , an LES  21   1  and a LES  212  are included in the system as well. The extra LES&#39;s serve as backups should LES  210 , which is designated as the master LES, fail. Each of the LES&#39;s makes a back channel control direct connection with the master LECS  220 . Master LECS  220  keeps track of all of the LES&#39;s that are connected to the master LECS in a memory  221 . In addition, master LECS  220  keeps track of which LES has been designated as the master LES at any given time. In the example shown, LES  210 , which is labeled LES A, has been designated as the master LES and this is shown for the purpose of illustration in memory  221  using a check mark. In practice, the master state of LES  210  may be noted by setting a flag in memory  221  or by some other appropriate technique. In addition to the control direct connection maintained by each LEC to a master LES, each LEC also may temporarily establish a CONFIG direct connection to master LECS  220  for the purpose of obtaining the address of the master LES when an LEC wishes either to join the ELAN for the first time or to rejoin the ELAN after the connection between the LEC and the master LES has been broken for some reason. 
     A similar LAN emulation system to that shown in FIG. 2 is also described in U.S. Pat. No. 5,835,481 by Akyol et. al. (“the &#39;481 patent”) which is herein incorporated by reference for all purposes. This system allows the ELAN to continue to function after a master LES fails. If the master LES fails, then each of the LEC&#39;s contact the master LECS. The master LECS then sends the address of the new master LES to the LEC&#39;s so that they may rejoin the ELAN. In certain systems having a large number of LEC&#39;s, it has been discovered that this scheme may create a very large amount of traffic from the LEC&#39;s to the master LECS  220 . It should be noted that, as described in the &#39;481 patent, backup LECS&#39;s which are not shown in FIG. 2 may also be included for the purpose of serving in the event that LECS  220  fails. Hence, LECS  220  may either be the only LECS or a master LECS. 
     To understand the amount of traffic potentially generated at LECS  220 , it is useful to consider the chain of events that occurs each time an LEC joins or rejoins the ELAN. First, the LEC sends a CONFIG request to the master LECS. The LECS responds to the CONFIG request by sending a CONFIG response to the LEC. The CONFIG response from the LECS contains the address of the master LES to which the LEC should establish a connection for the purpose of joining the ELAN. After the LEC receives the CONFIG response from the LECS, the LEC sends adjoin request to the master LES. The master LES then contacts the LECS for the purpose of verifying that the LEC is allowed to join the ELAN. To that end, the LES sends a CONFIG request to the LECS. This CONFIG request may also be referred to as a verification request. The LECS then sends a CONFIG response to the LES, instructing the LES either to allow the client to join the ELAN or to drop the client. If the client is allowed to join, then the LES sends adjoin response to the LEC and the LEC successfully joins the ELAN. 
     As can be seen from the above narrative, the LECS is contacted twice (once by the LEC and once by the LES) each time that an LEC joins the ELAN. When an LES fails and a large group of LEC&#39;s are dropped, the LECS may be flooded with CONFIG requests from each of the dropped LEC&#39;s as well as requests from the new master LES to verify each of the LEC&#39;s attempting to establish connections with it. Thus, although the system shown in FIG. 2 effectively provides a set of backup LES&#39;s, it would be useful if the system could be modified both to reduce the amount of LEC traffic to the LECS as well as the amount of backend traffic between the LES&#39;s and the LECS. 
     FIG. 3 is a block diagram illustrating a LAN emulation system  300  that reduces the traffic to the master LECS. Again, as noted above, the master LECS may be a master LECS determined as described in the  481  patent or the master LECS may be the only LECS provided for the system. Master LECS  302  stores an LES list in a memory  304 . The LES list includes each of the LES&#39;s connected to the master LECS. In addition, the LECS determines which of the LES&#39;s is the master LES. This determination is further described in FIG.  7 . In addition to keeping track of which LES is the master LES itself, master LECS  302  also sends packets to each of the LES&#39;s that indicate which LES is the master LES. Master LECS  302  also provides a list of all the LES&#39;s available in the system. 
     In the example shown, three LES&#39;s,  312 ,  314 , and  316 , are shown connected to LECS  302 . Each of the LES&#39;s also includes a memory that contains a flag or other data structure used for the purpose of storing the state of the LES, that is, whether the LES is the master LES or a backup LES. The packets sent from master LECS  302  to the LES&#39;s on the CONFIG direct connections to the LES&#39;s are illustrated in FIG.  4 . In addition to checking the control packets sent by master LECS  302 , each LES forwards a list of LES&#39;s provided by the master LECS to any LEC&#39;s that may be connected to an LES. 
     In the example shown, three LEC&#39;s,  322 ,  324 , and  326 , are shown connected to LES  312 , which has been designated as the master LES. In addition to a point to point control direct connection made from each of the LEC&#39;s to each of the LES&#39;s, a control distribute connection is also established between each LEC and each of the LES&#39;s for the purpose of broadcasting control information to the LEC&#39;s. In the diagram shown, the control direct connection from LEC  322  to each of the LES&#39;s is shown as well as the control distribute connection between LEC  322  and LES  312 . LEC  324  and LEC  326  also participate in the same control distribute connection. The control direct connections between LEC  324  and the LES&#39;s and LEC  326  and the LES&#39;s are not shown for the purpose of simplifying the diagram, but it should be recognized that the connections exist. 
     LES  312  takes the LES list including the designation of the master LES received from the master LECS and relays it to the LEC&#39;s via the control distribute connection with the LEC&#39;s. In one embodiment, the LES list is relayed from the LES to the LEC&#39;s using a targetless LEARP packet. (The LES list is first derived by the LES from the CONFIG response frame sent from the LECS to the LES.) Targetless LEARP packets are specified in version 2 of the LAN emulation specification. As will be described further in FIG. 4, a targetless LEARP packet includes one or more type link value (TLV) fields that may be used to carry information not specified in the LAN emulation specification. According to the specification, a targetless LEARP packet is supposed to follow a normal LEARP packet. By utilizing a targetless LEARP packet to carry information from the LECS to the LEC&#39;s via the LES&#39;s, the system provides a way to send the LES list to the LEC&#39;s. 
     If an LEC is not configured to accept such information, then the LEC may either ignore the targetless LEARP that it receives or treat the targetless LEARP as an error since it was not preceded by a normal LEARP packet. LEC&#39;s that are configured to receive such targetless LEARPS and extract the LES list from the TLV field in the LEARP packet that includes that list are referred to as enhanced LEC&#39;s. 
     As is described further in FIG. 4, a control packet sent from the master LECS to an LES may contain one or more TLV fields. An LES that is capable of recognizing such a TLV field that includes a list of the LES&#39;s is referred to as an enhanced LES. It should be noted that an enhanced LES may either determine whether it is the master LES by looking at the master LES address provided by the master LECS in the target ATM address field or by checking whether the address of the enhanced LES is the first address listed in the TLV field used to provide the LES list. 
     Thus, enhanced LES&#39;s are able to determine their own state based on control packets received from the LECS. In addition, enhanced LES&#39;s pass along the TLV field in packets sent to the LEC&#39;s. As described above, a targetless LEARP is used in one embodiment to transfer this information to the LEC&#39;s. In some embodiments a control direct connection from the LES to the LEC&#39;s may be used in a system that does not support the targetless LEARP on the control distribute. In addition, it should be noted when an LEC first joins the ELAN, the CONFIG response received from the master LECS by the LEC may also contain the TLV that contains a listing of all available LES&#39;s. Thus, each LEC discovers not only the address of the master LES but also the addresses of all backup LES&#39;s that are available. 
     Each LEC opens a control direct connection with both the master LES and one or more of the backup LES&#39;s. In one embodiment, the LES list is ordered. The first LES in the list is the master LES; the second LES in the list is the first backup LES; the third LES in the list is the second backup LES, and so on. Thus, when the master LES fails, the LEC already has a connection set up to the first backup LES and may transfer to that backup LES without contacting the master LECS. This avoids all of the LEC&#39;s contacting the master LECS upon the failure of the master LES. 
     Traffic on the back end connection between the LES&#39;s and the master LECS is also reduced because the LES&#39;s may accept all connections from LEC&#39;s without verifying such connections with the master LECS. Since the enhanced LEC&#39;s in the system are aware of all the LES&#39;s and which LES is the master LES, the backup LES&#39;s may accept a connection from an LEC without being concerned that the LEC is mistakenly connecting to the wrong master LES. Because each LEC is continually receiving an updated list of LES&#39;s from the LECS, the LES need not break the connection or refuse the connection from the LEC for the purpose of sending the LEC back to the LECS to get the correct master LES address. 
     As noted above, it is important that the system also be able to accommodate unenhanced LEC&#39;s as well as enhanced LEC&#39;s. Unenhanced LEC&#39;s receiving the targetless LEARP packet that includes the LES list, as described above, ignore the targetless LEARP packet and therefore do not discover the LES list information. In addition, unenhanced LEC&#39;s receiving a CONFIG response from the LECS including the TLV field that includes the LES list are not able to extract the LES list from the TLV field. 
     Such unenhanced LEC&#39;s are detected by the backup LES&#39;s and their connections are refused or dropped whenever such LES&#39;s are in the backup state. In one embodiment, unenhanced LEC&#39;s are recognized because such LEC&#39;s do not include a blank or dummy TLV field in the position used by the LECS and LES&#39;s to transfer the LES list information. Enhanced LEC&#39;s, on the other hand, include such a TLV field in packets directed from the enhanced LEC&#39;s to the LES&#39;s. Thus, the enhanced LES&#39;s may accept all connections from enhanced LEC&#39;s and accept connections from unenhanced LEC&#39;s only when the LES is in the master state. 
     It should also be noted that in some embodiments, enhanced LES&#39;s may be included in systems that include unenhanced LECS&#39;s. Unenhanced LECS&#39;s are LECS&#39;s that are not configured to send LES list information to the LES&#39;s and LEC&#39;s serviced by the unenhanced LECS. When placed in such a system, enhanced LES&#39;s treat each LEC as if it were an unenhanced LEC and the system functions in a similar manner as the system described in FIG.  2 . Since each of the LEC&#39;s only receives the address of the master LES from the unenhanced LECS, a backup LES must break or refuse a connection made to it by an LEC so that the LEC will return to the unenhanced LECS and obtain the correct master LES address. 
     In one embodiment, the master LECS periodically sends a packet to the LES&#39;s that includes a TLV field containing the list of LES&#39;s. For example, the master LES may every minute send a CONFIG response packet to each of the LES&#39;s containing such a TLV field. The LES&#39;s then forward the list contained in the TLV field in another TLV field included as part of a targetless LEARP packet. In addition, if the master LES ever fails, and that failure is detected by the LECS, then the LECS sends an unsolicited CONFIG response packet that includes the TLV of all the correct updated LES list for the purpose of immediately updating the LES list and informing both the LES&#39;s and the LEC&#39;s via the LES&#39;s of the new master LES address. 
     It should be noted that, in the discussion above, certain types of packets are described as carrying the LES list in a TLV field. It should be noted that other types of packets including data packets may be used to transfer the LES list in different embodiments. Also, other fields or parts of a packet may be used to carry the LES list downstream from the LECS. Also, other types of packets including data packets may be used to transfer the LES list from the LES&#39;s to the LEC&#39;s. 
     The system shown in FIG. 3 illustrates how traffic to the LECS is reduced in two ways. First, when an LES fails, each LEC may move to the first backup LES without sending a CONFIG request to the LECS. Second, the LES&#39;s no longer need to verify each CONFIG request received from an LEC since the LEC&#39;s continuously receive updates of the LES list, including the address of the current master LES. As is described in connection with FIG. 7, traffic to the LECS may be reduced in yet another way when the LECS operates in non-preemptive mode by avoiding changing the master LES when a higher priority LES becomes available and connections have already been established to a lower priority LES. 
     FIG. 4 is a block diagram illustrating several of the packet formats described in connection with FIG. 3. A join request packet  410  includes a standard join request portion  412  as well as a TLV  414  that correspond to the TLV that contains the list of LES&#39;s. When an enhanced client sends such a join request containing this TLV field, even though the TLV field does not contain meaningful data for the LES, the existence of the TLV field indicates that the enhanced client is, in fact, an enhanced client. Likewise, a CONFIG request  420  from an enhanced client also includes a TLV field  424  in addition to the fields included in a standard CONFIG request. Again, the existence of the TLV field in a packet sent from an enhanced client indicates that the client is an enhanced client. In one embodiment, enhanced LEC&#39;s can send TLVs in a CONFIG request to an LECS or a join request to an LES only when instructed to do so through a command in a router or switch. 
     A CONFIG response  430  sent by an enhanced LECS includes standard fields found in CONFIG response packets. Inside the target ATM address field  432  the enhanced LECS sends the master LES address. In addition, the LECS sends a TLV field  434  that contains a list of all available LES&#39;s, including backup LES&#39;s. A targetless LEARP packet  440  sent from an LES to an enhanced LEC includes various standard fields as well as a TLV field  444  that is used to send the list of LES&#39;s to the client. Thus, all of the information that needs to be transferred within the LAN emulation system for the client to maintain a list of available LES&#39;s is sent using TLV fields in standard packet formats. 
     It should be noted that other TLV fields may be used for other purposes as well. Also, the existence of a certain TLV field in packets originating from an enhanced LEC serves as an indicator that the enhanced LEC is an enhanced LEC. Next, the various processes that run on each enhanced LECS, LES and LEC are described. These processes reduce the amount of traffic generated at the LECS while accommodating other devices that may not be enhanced. 
     FIG. 5 is a state diagram illustrating the state of an LES upon the occurrence of certain events. Immediately after starting up, the LES state is undetermined. In various embodiments, the LES state may be initialized to master or backup. As soon as the LES receives a CONFIG direct packet from the master LECS, the state of the LES is determined by looking at the address of the master LES and comparing that address to the LES&#39;s own address. If the master LES matches the LES&#39;s own address, the LES transitions to a master state  504 . If the master LES is some other address, the LES transitions to a backup state  506 . If, while the LES is in backup state  506 , it receives an unsolicited CONFIG response packet with its own address as the master LES address, then the LES transitions to the master state. If, while in the master state, the LES receives a packet with another address as the address of the master LES, then the LES transitions to the backup state. 
     When in the backup state, the LES rejects join requests and tears down connections with any unenhanced LEC&#39;s that may be connected to or attempt to connect to the LES. Likewise, the LES also rejects connections from all LEC&#39;s when it is in the backup state and it has determined that the LECS for the ELAN is an unenhanced LECS. If an unenhanced LECS is serving the ELAN, then even enhanced clients only receive a single LES address from the LECS that corresponds to only the master LES. Thus, when an unenhanced LECS is present, enhanced LEC&#39;s behave similarly to unenhanced LEC&#39;s because the enhanced LEC&#39;s only are notified of the master LES address. 
     FIG. 6 is a flow chart illustrating a process  600  running on an enhanced LES for processing join requests from LEC&#39;s. The process starts at  602 . In a step  604 , the LES receives a join request. Next, in a step  606 , the state of the LES is checked. If the LES is in the master state, then control is transferred to a step  608  and the join request is accepted. The process then ends. 
     If the LES is in the slave state, then control is transferred to a step  620 . In step  620  it is checked whether the join request came from an enhanced client. If the join request came from an enhanced client, then control is transferred to a step  630 . In step  630 , it is checked whether the LECS to which the LES is connected is an enhanced LECS. If the LECS is an enhanced LECS, then the join request is accepted in a step  632  and the process ends. If the LECS to which the LES is connected is not an enhanced LECS, then the LES validates the join request in a step  634  by sending a CONFIG request to the LECS on the backend connection. 
     If it was determined at step  620  that the request did not come from an enhanced client, then control is transferred directly from  620  to  634  and the join request is validated by sending a CONFIG request to the LECS. If the LECS indicates that the request should be accepted, then the LECS accepts the request and if the LECS indicates that the request should be rejected, then the LES rejects the request. 
     Thus, the LES is able to handle join request from both enhanced and unenhanced clients in an appropriate manner according to the type of LECS, enhanced or unenhanced, to which the LES is connected. When an LES transitions from the master state to the slave state, a similar process is executed. If the LES is connected to an enhanced LECS, then connections to enhanced clients are maintained. Connections from unenhanced clients are dropped and those clients therefore send a CONFIG request to the LECS and are directed to the new master LES. If the LES is connected to an unenhanced LECS, then all of the LEC&#39;S connected to the LES are dropped when the LES transitions to the slave state. That is because in a system with an unenhanced LECS, all clients must contact the LECS to find out the new master LES address. 
     FIG. 7 is a flow chart illustrating how an LECS handles connection events. After coming online, the LECS enters a state  704  wherein the LECS waits for connection events and timer interrupts. When a new connection is made to the LECS, control is transferred to a step  706 . In step  706 , it is checked whether the LECS is in preemption mode. If the LECS is not in preemption mode, then control is transferred to a step  708 , the list of LES&#39;s is updated, and the LECS returns to state  704 . 
     If the LECS is in preemption mode, then control is transferred to a step  710  where it is checked whether the new connection is from an LES that is a higher priority LES than the master LES. If the LES is a higher priority LES, then control is transferred to a step  712  and the LECS designates a new LES as the master LES. The LECS list of LES&#39;s is updated in a step  714  and in a step  716  the LECS sends a CONFIG response packet to every LES containing the new LES list and the new LES master. The LECS then returns back to state  704 . If it is determined in step  710  that the new connection is not from a higher priority LES than the master LES, then control is transferred to a step  722  and the LECS list is updated and the LECS returns to state  704 . 
     If a connection to an LES is lost, then control is transferred to a step  730  and the list of LES&#39;s is updated at the LECS. Next, in a step  732 , it is determined whether or not the connection lost was the connection to the master LES. If it was not the master LES connection, then the LECS returns to state  704 . If the connection lost is the master LES connection, then control is transferred to a step  734  and the LECS designates a new master LES. Next, in a step  736 , the LECS sends a CONFIG response to all of the LES&#39;s indicating the new master LES and the new LES list. 
     If the LECS timer that controls the periodic sending of CONFIG response packets from the LECS to all of the LES&#39;s expires, then control is transferred to a step  738  and the LECS sends a CONFIG response that includes the address of the master LES as well as the list of LES addresses. The LECS then returns back to state  704 . Next, the LECS periodically sends packets to all of the LES&#39;s that include an updated LES list. When an event occurs that causes the LECS to change the master LES, then the LECS immediately sends a packet to each LES updating them with the new master LES address. When a new LES establishes a connection to the LECS, then the LECS may or may not designate a new master LES depending on whether the LECS is operating in preemption mode or not. 
     FIG. 8 is a flow chart illustrating a process  800  executed by an enhanced LECS for the purpose of handling CONFIG requests. The process starts at  802 . In a step  804 , a CONFIG request is received. Next, in a step  806 , the LECS determines whether the request contains the TLV field that is used by the LECS to provide the LES list. If the CONFIG request does not contain the TLV field, then the CONFIG request came from an unenhanced LES. Therefore, control is transferred to a step  808  and the LECS determines whether or not the unenhanced LES is a master LES. If the unenhanced LES is not a master LES, then control is transferred to a step  810  and the request is rejected. If the LES is a master LES, then control is transferred to a step  812  and the request is accepted. In either event, the process ends and the LECS is ready to handle another CONFIG request. 
     Thus, the enhanced LECS continues to behave as an unenhanced LECS would when it receives CONFIG requests from unenhanced LES&#39;s. If the CONFIG request contains a TLV used to send the list of LES addresses, then control is transferred directly from step  806  to step  812  and the request is automatically accepted as is appropriate for requests from enhanced LES&#39;s. 
     FIG. 9 is a flowchart illustrating a process executed by an LEC upon receiving a targetless LEARP on a control distribute connection from an LES. Process  900  starts at  902 . In a step  904 , the LEC receives an untargeted LEARP. Next, in a step  406 , the LEC checks whether it had previously received a normal LEARP. If a normal LEARP was previously received, the control is transferred to a step  408  and the untargeted LEARP is handled according to the LANE specification and the process ends. If the LEC had not previously received normal LEARP, then control is transferred to a step  410  and the enhanced LEC checks for the TLV that contains the list of LES addresses from the LECS. 
     If such a TLV is detected, then control is transferred to a step  412  and the LEC stores the list of LES&#39;s and notes whether or not the address of the master LES has changed. The process then ends. If no TLV is detected in step  410 , then the untargeted LEARP is either an error or is an untargeted LEARP generated for some unrelated purpose and the process ends. Thus, enhanced LEC&#39;s check untargeted LEARP&#39;s and use the untargeted LEARP&#39;s that contain a TLV listing LES&#39;s to update the list of LES&#39;s stored at the enhanced LEC. 
     FIG. 10 is a flow chart illustrating a process  1000  executed on an enhanced LEC for the purpose of handling various connection events. Process  1000  starts  1002 . The enhanced LEC enters a state  1004  where it waits for connection events. If the enhanced LEC receives notice that a new backup LES is on line, then control is transferred to a step  1010  and a connection is established with the new backup LES. If the enhanced LEC receives notice that a backup LES has been removed from the list of LES&#39;s, then control is transferred to a step  1012  and the connection to the LES is torn down. If the enhanced LEC is notified that a new master LES has been designated by the LECS, then the LEC changes its data connection to the new LES. If the LEC detects a new master LES, then the LEC changes its data connection to the new LES in a step  1016 . If the LEC detects that the master LES has failed, then the LEC changes its data connection to the next LES on the list of LES&#39;s in a step  1014 . In addition, a process is initiated in a step  1018  wherein the LEC periodically attempts to reconnect to the failed master LES. After each connection event is processed in the manner described, the LEC returns back to state  1004  and waits for another connection event. 
     A LAN emulation system has been described that includes a master LES and one or more backup LES&#39;s. Enhanced LEC&#39;s receive packets that include a list of all of the LES&#39;s and that designate a master LES. Enhanced LES&#39;s accept all connections from enhanced clients without verifying the clients with the LECS. In this manner, a large number of clients may participate in the ELAN without generating an excessive amount of traffic and overwhelming the LECS. 
     If a system is known to contain only enhanced LECS&#39;s, LES&#39;s, and LEC&#39;s, then the system may be used in a mode where the LES&#39;s are no longer designated as master LES&#39;s and backup LES&#39;s. Instead, each available LES may service LEC&#39;s in parallel with the other LES&#39;s. In one embodiment of such a system, LEC&#39;s are programmed to connect alternatively to different LES&#39;s for the purpose of load balancing among the LES&#39;s. 
     Although the foregoing invention has been described in some detail for purposes of clarity of understanding, it will be apparent that certain changes and modifications may be practiced within the scope of the appended claims. It should be noted that there are many alternative ways of implementing both the process and apparatus of the present invention. Accordingly, the present embodiments are to be considered as illustrative and not restrictive, and the invention is not to be limited to the details given herein, but may be modified within the scope and equivalents of the appended claims.