Abstract:
A method and apparatus for the non-disruptive addition of one or more nodes to an active inter-nodal network is provided. A telecommunications switching system includes a plurality of nodes interconnected by an inter-nodal network which carries packetized information among the nodes. A host which issues various control instructions is connected to at least one node in the system. When one or more new nodes are to be added to the system, the two nodes adjacent to the location selected for the new node are instructed to begin a synchronized routine for simultaneously beginning to temporarily operate in a special loop back mode of operation which allows telecommunications traffic to continue to travel on the inter-nodal network while the new node or nodes are being physically connected into the system. Thereafter, a sequence of instructions provides for a synchronized routine for both neighbor nodes to return to open operating mode from loopback mode simultaneously. The instruction sequences are issued by a host and a master node of the system for implementing the non-disruptive addition of the new node or nodes to the system.

Description:
FIELD OF THE INVENTION 
     The present invention relates generally to the field of telecommunications and, more specifically, to a system for expansion of a telecommunications network with minimal disruption of traffic service on the network. 
     BACKGROUND OF THE INVENTION 
     A fundamental consideration in any telecommunications system design is switching capacity. Switching capacity must be analyzed in terms of current demand and projected demand in order to find a solution that is cost effective for both present and future service. For example, assume that a developing country is in the process of building a basic telecommunications system and intends to provide service to most of its current population. Such a population is most likely geographically distributed among small areas of high density (cities) and larger areas of low density (suburban and rural). In addition, the population is probably growing, but at different rates in different areas. Thus, the challenge for a telecommunications system designer is to provide sufficient switching capacity to support satisfactory service to most or all of the population while also anticipating likely increases in future demand and providing for economical expansion. 
     A second fundamental consideration in telecommunications system design is providing for the addition of new features or services in the future. Telecommunications equipment and service continues to evolve rapidly, due in large part to the advent of digital technology. Even more dramatic advances are likely in the future, particularly as previously separate industries such as the Internet, cable television and local telephone operating companies integrate services. Again, the challenge is to create a system which economically serves a present need, while also providing flexible and inexpensive ways to integrate new features and services as they become available. Ideally, such new features and services can be added to an existing system without disruption of service, but this has not always been possible. 
     Several of these fundamental considerations are addressed in a system described in commonly-assigned U.S. Pat. No. 5,544,163, Aug. 6, 1996, entitled EXPANDABLE TELECOMMUNICATIONS SYSTEM, which is incorporated herein in its entirety. Briefly, the patent describes an open, high speed, high bandwidth digital communications network for connecting multiple programmable telecommunications switches to form a large capacity, non-blocking switching system. In a preferred embodiment described therein, the network is implemented using one or more inter-nodal networks which provide a medium for transferring information over the network, and a plurality of programmable switches, each of which appears as a node on the network and serves a group of ports. Additional switches (nodes) may be added to the network as desired to increase the system&#39;s switching capacity. 
     Each node includes circuitry for transmitting and receiving variable-length, packetized information over the network, thus enabling each node to receive information from or transmit information to all other nodes. The network may carry any type of information present in the system including voice, data, video, multimedia, control, configuration and maintenance, and the bandwidth of the network may be divided or shared across various information types. 
     In addition, devices or resources other than programmable switches may also act as nodes on the network, thereby gaining direct access to all information passing through the network. More specifically, voice processing resources such as voice mail/message systems or other enhanced service platforms may, by becoming nodes, gain direct access to all ports served by the system without the need for a large central switch. The system&#39;s ability to transfer information of any type, in a readily usable form, at high speed across the network enables any service, feature or voice processing resource which is available at a given node to be provided to any port of the same or any other node. 
     The programmable switching nodes and the other nodes on an expandable telecommunications system are connected by a physical medium. When it is desired to modify an active inter-nodal network in order to enhance the services or increase the capacity provided by the system, by the inclusion of one or more additional nodes, the physical medium must be modified to accommodate an additional node. More specifically, the physical connections between the nodes in the inter-nodal network adjacent to the area to be expanded must be temporarily physically disconnected in order to connect the new node or nodes into the network. When the physical medium is disconnected, that portion of the inter-nodal network cannot, of course, carry telecommunications traffic. Typically, in such a case, the system, in whole or in part, must be taken out of service, thus leading to possibly unacceptable levels of service interruption. 
     Also, a new node which is added to the system must be properly configured before beginning operation to avoid undesired interference with other nodes. 
     There remains a need, therefore, for a telecommunications system which can be expanded by the addition of one or more new nodes with minimal, if any, disruption of telecommunications service provided by the system. 
     There remains a further need for a method of adding a new node to an existing, active system with essentially no interruption of service in the system. 
     SUMMARY OF THE INVENTION 
     These and other needs are satisfied by the present invention which provides, in brief summary, a method and system for non-disruptive addition of one or more nodes to an active inter-nodal network in a telecommunications system. In a preferred embodiment the invention is implemented in a method and an associated system which includes a plurality of interconnected nodes, which may be nodes used for telecommunications switching, or other nodes used for voice processing resources such as voice mail/messaging and the like. The nodes are interconnected by an inter-nodal network which carries packetized information among all the nodes served by the inter-nodal network. 
     Each node has an open mode of operation and a special mode of operation referred to as loopback mode. More specifically, each node has an “A” I/O port, which has its own transmit and receive coupling, and a “B” I/O port, which has its own transmit and receive coupling. In the open mode of operation, packets traverse between the node and the inter-nodal network by coming into the node through the receive coupling of the “A” port and exiting the node by the transmit coupling of the “B” I/O port. In loopback mode, one of the I/O ports is effectively disconnected from the system while the other “loops back” and functions to both transmit and receive packets using its own transmit and receive coupling and an alternate communication path to continue to pass packets onto and receive packets from the inter-nodal network. 
     A host is connected in communicating relationship with at least one of the nodes in the system. The host controls certain aspects of the system&#39;s operations by sending messages addressed to all nodes or specific messages to individual nodes. The system is also configured so that one of the nodes is a master node. The master node is capable of sending control messages addressed to the other, non-master, nodes in the network. In accordance with the invention, the master does this messaging, in part, by using a unique inter-nodal network control word. The inter-nodal network control word is a set of bits contained within a frame which in turn contains one or more packets originating from a particular node. In accordance with this aspect of the invention, a unique messaging sequence involves an interplay between messages issued by the host, which may be controlled by a user, and messages issued in turn by the master node, using the inter-nodal network control word. The inter-nodal network control word allows nodes to receive and execute instructions in a synchronized manner while avoiding the necessity of forcing the host to attempt to simultaneously communicate with multiple nodes. In still other circumstances, a non-master node will be enabled to write into the inter-nodal network control word to communicate certain types of information to the master node. This is also a time-saving technique which can by utilized to avoid interruption of system operation. 
     In accordance with the invention, when it is desired to expand an existing, active inter-nodal network, a location is selected along the inter-nodal network to connect one or more new nodes. As used herein, the term “new node” shall include one or more new nodes which can be programmable switching nodes, voice processing resource nodes, or other nodes, or a combination thereof as desired in a particular application. The two nodes in the active system which are adjacent to the selected location are identified. The two nodes between which the new node is to be added shall be referred to herein as the “neighbor” nodes, as they will neighbor the new node when it is added to the system. 
     The new node is prepared by the host for addition to the network by being programmed with instructions about the installation procedure it needs to follow. The inter-nodal network is then checked to ascertain whether any existing nodes are coincidentally operating in loopback mode. If no existing nodes are operating in loopback mode, the expansion of the system by the addition of the new node may continue. If, on the other hand, there is a node operating in loopback mode, then the fault or other condition causing that node to operate in loopback mode will have to be resolved first. This is because performing the installation procedure while any node is concurrently operating in loopback mode could cause other nodes to become isolated, and thus, cause interruption of service. 
     Once it is determined that all existing nodes are operating in open mode, the host issues a message addressed to the master node, instructing it to notify the two neighbor nodes, using the inter-nodal network control word, to begin a first synchronized routine to change their respective I/O ports to operate in loopback mode after a predetermined time delay. After this predetermined time period elapses, both neighbor nodes change their respective ports to operate in loopback mode essentially simultaneously. Accordingly, packetized information traversing the inter-nodal network continues to travel through the network uninterrupted except for that portion of the inter-nodal network between the two neighbor nodes. That portion is now temporarily isolated and inactive. Thus, the physical disconnection of that portion of the inter-nodal network can now take place without interruption of the remainder of the network. Thereafter, the new node or nodes are physically connected into the network. 
     In accordance with another aspect of the invention, the new node follows a sequence of instructions, triggered by certain host-issued messages, which causes the node to wait while the inter-nodal network, which continues to carry normal traffic, is configured to include the new node. This sequence of instructions is a short cut around the normal sequence of messages which would be followed by a node when an inter-nodal network is being brought into service and initialized. In accordance with the present invention, the new node follows the short cut instructions so that it enters the network in a running state as if it had always been in the network. This avoids the new node following its pre-programmed instructions for network initialization. 
     After the new node is connected into the inter-nodal network and the configurations tasks are accomplished, the two neighbor nodes must return to the open mode of operation out of loopback mode. In a manner similar to the sequence followed to place the two neighbor nodes into loopback mode, both nodes must be brought out of loopback mode approximately simultaneously. Otherwise, if one node returns its respective port to normal mode first, this leaves only one node operating in loopback mode, and the network will not function properly. 
     In order to avoid this, in accordance with the invention, a message is issued by the host, addressed to the master node, instructing it to notify the two neighbor nodes to commence a second synchronized routine, this time to discontinue loopback mode and return to open mode after a predetermined time delay. The predetermined time delay allows both nodes to receive the instruction and begin the timing sequence prior to either one of them returning to the open mode of operation. In this way, both neighbor nodes, between which the new node is now inserted, return to open mode essentially simultaneously. 
     After the two neighbor nodes return to open mode, the host sends a message to the new node in response to which the new node attempts to verify that it is recognizable by the master node. The new node does this by sending a message to the master node, using the inter-nodal network control word. The new node waits for an acknowledgment from the master node. When the acknowledgment is received, then a final verification is performed to ascertain that the new node or nodes as well as the neighbor nodes have open ports. Thereafter, the new node can be configured to transmit and receive packets to and from the inter-nodal network, or to perform a redundancy function by being configured in a receive only mode, as desired in the application. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     The above and further advantages of the invention may be better understood by referring to the following description in conjunction with the accompanying drawings in which: 
     FIG. 1A is a schematic diagram of an expandable telecommunications system which employs a single inter-nodal network to transfer information among nodes; 
     FIG. 1B is a schematic diagram of the system of FIG. 1A also depicting a new node to be added to the system and the two nodes adjacent to the new nodes operating in loopback mode in accordance with a preferred embodiment of the present invention; 
     FIG. 1C is a schematic diagram of the system of FIG. 1A expanded in accordance with the present invention by the addition of a new node; 
     FIG. 2 is a block diagram which illustrates the telecommunications system and the I/O ports of each node in the system, also illustrating the operation of the neighbor nodes during the installation procedure of a new node; 
     FIGS. 3A and 3B are block diagrams which illustrate the open mode of operation and a loopback mode of operation of a node; 
     FIG. 4 is a block diagram illustrating a preferred embodiment of a frame structure which includes the inter-nodal network control word; 
     FIG. 5 is a message flow diagram which illustrates certain aspects of the messaging sequence used to place the two neighbor nodes in a loopback mode of operation when a non-master node initially received the operative host-issued message; 
     FIG. 6 is a timing diagram which illustrates the time relationship of certain messaging sequences; 
     FIG. 7 is a message flow diagram which illustrates certain aspects of the messaging sequence used to place the two neighbor nodes in a loopback mode of operation when the master node initially receives the operative host-issued message; 
     FIG. 8 is a flow diagram illustrating the logical states in which a new node operates while being added to the system; 
     FIG. 9 is a message flow diagram which illustrates the expand network messaging sequence when a non-master node receives the initial message from the host about expanding the inter-nodal network; and 
     FIG. 10 is a message flow diagram which illustrates the expand network messaging sequence when the master node receives the initial message from the host about expanding the network. 
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
     FIG. 1A shows a large capacity, expandable, fully programmable telecommunications switching system  2 . Details of the construction and operation of the system  2  may be found in the above-cited U.S. Pat. No. 5,544,163. To aid in understanding of the present invention, certain aspects of the system  2  are discussed here. 
     The telecommunications system  2  comprises a host  4 , and three nodes,  6   a  through  6   c,  connected in communicating relationship by an inter-nodal network  12 . It should be understood that a larger or small number of nodes being either programmable switching nodes or other types of non-switching (e.g., voice processing or communications services) nodes or bridge nodes may be used in essentially any combination in the system  2 . It is desired to expand the telecommunications system  2  by the addition of another node, node  6   d,  as illustrated in FIG.  1 B. The new node  6   d  may be of any type that is compatible with inter-nodal network  12 . FIG. 1C illustrates new node  6   d  as it is incorporated into the system while the nodes proximate to it, node  6   a  and node  6   c  are in loopback mode as schematically illustrated in FIG. 1C by looped back port  7  on node  6   a  and looped back port  9  on node  6   c.    
     FIG. 2 illustrates in greater detail the telecommunications system  2 . The inter-nodal network  12  is configured for providing high speed digital communications among nodes  6   a  through  6   c  as represented by the solid line  12   a.  The node  6   a  includes a host interface which is connected in communicating relationship with the host  4  by a local area network (LAN) such as Ethernet or some other communication link  8 . In such configuration, the node  6   a  may receive messages from the host  4  which are intended for one of the other nodes and pass such messages to the appropriate node over the inter-nodal network  12   a.  Other types of host/node interfaces may be used instead of or in addition to the LAN/link  8 . In addition, one or more of the remaining nodes  6   b  and  6   c  may also be directly connected in communicating relationship with the host  4  via LAN/link  8  (not shown). In the absence of a direct link from the host  4  to each of nodes  6   b  and  6   c,  such nodes are preferably connected in communicating relationship with node  6   a  in such fashion as to allow messages to be transmitted to and received from host  4 . 
     The inter-nodal network  12  is preferably implemented using one or more fiber optic rings. However, the inter-nodal network  12  may also be implemented with any suitable communication network, such as, for example, wide area networks, wireless communications networks, the PSTN, ATM, SONET and the Internet. 
     The overall operation of the system  2  is controlled by the host  4 , which is commonly implemented with a personal computer (PC), workstation, fault tolerant or other computer on which a user&#39;s application software runs. The host  4  communicates with the nodes  6   a  through  6   c  by exchanging messages over the LAN/link  8 . The messages are typically used to configure the nodes as well as to direct call processing functions such as making connections and providing communication services (i.e., tone detection, tone generation and conferencing). 
     Although only a single host  4  is shown, use of the LAN  8  to provide a host/node communications path permits multiple hosts to control the system  2  (or parts thereof) by configuring a host as a “client” and each node as a “server.” A host can be implemented as a printed circuit card that is physically connected within a node, or it may be an external host. 
     Each of the nodes  6   a  through  6   c  may include interfaces  10   a  through  10   c  with the public switched telephone network (PSTN) (not shown) or a private network (not shown). The term “private network” is intended in a broad sense to refer to any network or line or other interface other than the PSTN. Network/line interfaces  10   a  through  10   c  may terminate either digital networks or analog trunks/lines, or combinations of both types. The network/line interfaces of a given node may include suitable interfaces for performing telecommunications using ATM, Signaling System 7 (SS7), ISDN, T1-Robbed Bit, E1-CAS, TCP/IP or other communications protocols. 
     Additional details of the redundancy, fault isolation and expanded switching capacity features of system  2  are described in commonly-assigned U.S. Pat. No. 5,923,643, which is incorporated herein in its entirety. Briefly, in that patent, a system is described in which a fault within a particular node or a portion of an inter-nodal network may be isolated and prevented from degrading system performance. This is accomplished in part by providing each node with a loopback mode of operation. In the event of a failure of either a node or a portion of an inter-nodal network, the loopback mode of operation may be invoked to isolate the failure and allow the remainder of the system to continue to operate without interruption of service. The loopback mode of operation may also be advantageously used in the present invention as described below. 
     More specifically, with reference to FIG. 2, each of the nodes of  6   a  through  6   c  includes two I/O ports, denoted A and B, which are physically interfaced with the inter-nodal network  12   a.  In open operating mode, each of nodes  6   a  through  6   c  receives packetized information through port A and transmits packetized information to the other nodes through port B as indicated by the solid arrows. 
     An alternative mode of operation, the loopback mode, may be understood with reference to FIGS. 3A and 3B. As shown by the solid line in FIG. 3A, in the open mode of operation, packetized information is received from the inter-nodal network  12  (not shown in FIG. 3A) at port A of a network I/O card  40   a  and is passed through to a nodal switch  44   a  of node  6   a.  Information originating from the nodal switch  44   a  within node  6   a  is passed through the network I/O card  40   a  and then transmitted by way of port B as shown in FIG.  3 A. Stated another way, during the open mode of operation, information passing between the nodal switch  44   a  and the inter-nodal network  12  passes in one direction only, through each of ports A and B. 
     In contrast, as shown by the solid line in FIG. 3B, during a loopback mode of operation, port A is effectively disconnected from the inter-nodal network  12  while port B functions to both receive and transmit information. Thus, packets still pass through the node  6   a.  Conventionally, a particular node may operate in loopback mode as to a certain port in response to a message from the host or a detection of any number of conditions on the inter-nodal network  12 , such as either a fault, or during initialization of the system. 
     Communications over the inter-nodal network  12  are preferably carried out using a frame/packet structure as shown in FIG. 4. A frame  50  contains at least one packet  54  and may contain additional packets  58 ,  60  for carrying data, address and control information over the inter-nodal network  12 . The maximum number of packets frame  50  may contain is a largely function of the bandwidth of the inter-nodal network  12 . In a preferred embodiment of the present invention, frame  50  may contain up to thirty packets. 
     Each frame  50  begins with an inter-nodal network control word  64  which is preferably a 16-bit entity. Control word  64  is used to effect certain control functions with respect to a node which either receives the control word or transmits it to another node. Typically, the master node will use the control word  64  to direct a non-master node to perform a particular control function. However, as described herein, in certain instances, non-master nodes may use the control word  64  to communicate with the master node. Each packet  54 ,  58 ,  60  contains a start-of-packet (SOP) entity  66 , a source address (SRC)  68 , which is preferably a logical node identification (ID) of the node from which frame  50  originates, a destination address (DST)  70 , which is preferably a logical node ID of the node to which the frame  50  is destined. Following those entities is a payload  72  which, in the illustrative embodiment, has a capacity of 512 bytes of data. An end-of-packet (EOP) entity  74  follows the payload  72 . The end of frame  50  is represented by an end-of-transmission (EOT) entity  78 . 
     Details of a messaging sequence followed in accordance with the present invention when a node, such as the node  6   d  (FIG.  2 ), is to be added to an active inter-nodal network  12  will now be described. Assume that the inter-nodal network  12  has been appropriately configured and initialized and is now carrying data among nodes  6   a - 6   c.  Further assume that one of those nodes has become a master node through an arbitration process described in U.S. Pat. No. 5,923,643, referenced above. For this example, we will assume that node  6   b  is the master node. 
     Initially, prior to commencing the process of adding a new node, a determination must be made whether any node in the system  2  is currently operating in loopback mode. Preferably, the host  4  already has this information as a result of being automatically notified by any node when that node has gone into loopback mode for any reason (i.e., fault isolation, node addition in progress, etc.) If there is a node with a looped back port, it should first be restored to open mode before proceeding further with the process of adding a new node. Otherwise, there is a risk that a node may become isolated when the two neighbor nodes adjacent to the position selected for the new node loop back their respective ports. 
     With reference now to FIGS. 2,  5  and  6 , assuming that no node is operating in loopback mode, the host  4  begins the process of adding a new node by issuing a LOOPBK_PORT message  80   a  which is received by a system monitor (SYM) task  81  running on non-master node  6   a.  The SYM  81 , in turn, issues a LOOPBK_PORT message  80   b  to a ring configure (RCFG) task  83 , which in turn issues a LOOPBK_PORT message  80   c  to a ring control (RCTL) task  85  running on the non-master node  6   a.  This message and the others shown with solid arrows are messages which pass between processes internal to a node or they are API (Application Program Interface) messages issued or received by the host. The messages shown with a dashed line are messages which are sent, using control word  64 , over the inter-nodal network  12 . 
     In response to the LOOPBK_PORT message  80   c,  the RCTL task  85 , using control word  64 , passes a LOOPBK_LOOP_BACK_Indication message  84   a  to the master node  6   b  over the inter-nodal network  12 . As a result of receiving the LOOPBK_LOOP BACK_Indication message  84   a,  the master node  6   b  is informed that the neighbor nodes are to be instructed to loop back their respective ports. To ensure that the master node  6   b  receives the LOOPBK_LOOP_BACK_Indication  84   a,  this message is preferably sent continuously until the non-master node  6   a  (sending the message) receives a LOOPBK_LOOP_BACK_ACK  84   b  from the master node  6   b.    
     Once the LOOPBK_LOOP_BACK_ACK message  84   b  is received by the non-master node  6   a,  the non-master node  6   a  will stop sending any messages using the network control word  64 . This allows the master node  6   b  to be the only node in the network which controls the contents of the control word  64 . If the non-master node  6   a  does not receive the acknowledgment from the master node  6   b,  it will notify the host  4  that there has been an interruption in the procedure. 
     If, on the other hand, the master node  6   b  does send the acknowledgment and takes control of the control word  64 , it is programmed to thereafter start sending the LOOPBK_PREPARE_LOOP BACK message  85   a,  again using control word  64 . This message is sent to the neighbor nodes between which the new node is to be added. 
     The LOOPBK_PREPARE_LOOP BACK message  85   a  is sent by the master node, as indicated by reference numeral  89 , preferably continuously for approximately 200 ms to be sure that both neighbor nodes receive it. (It should be noted that acknowledgment messages are omitted from FIG. 6 for purposes of improved clarity.) Concurrently, the neighbor nodes initialize a timing procedure during which the nodes read the control word  64  approximately every 250 microseconds, as indicated by reference numeral  90 . This is to ensure that the neighbor nodes will react quickly to the anticipated instructions from the master node  6   b.    
     Assuming that the master node  6   b  does receive the LOOPBK_PREPARE_LOOPBACK_ACK message  85   b  from both nodes, it starts to send a LOOPBK_LOOP BACK_PORT message  86 . Both neighbor nodes  6   a  and  6   c  will receive this message with a maximum of Ims delay, as they read the control word  64  as indicated by reference numeral  92 . After each neighbor node  6   a  and  6   c  receives message  86 , each node will preferably schedule the transition of the operating mode of its respective I/O port from open to loopback mode in approximately 2 ms. This 2 ms delay allows both nodes to receive the instruction before either one of them actually transitions to loopback mode. Otherwise, one of the nodes might receive the message and immediately loop back its port thereby possibly causing the other node never to receive the loop back instruction. 
     After a 2 ms delay, the master node sends a LOOPBK_LOOP BACK_COMPLETED message  87  using control word  64 . The two neighbor nodes are programmed to wait for the message  87 . If the neighbor nodes do not receive it within a certain time period, then they will notify the host of a problem. Otherwise, after that time period, the non-master node  6   a  informs the host  4  of the successful completion of this phase of the process via messages  88   a,    88   b  and  88   c  of FIG.  5 . At this point, port “A” of node  6   a  and port “B” of node  6   c  are operating in loopback mode. 
     FIG. 7 illustrates the case in which the master node  6   b  is the node that receives the initial loop back port message  80   a  from the host  4  because it is one of the neighbor nodes in that instance. In such a case, the initial messaging sequence in which the message  80   a  passes to the SYM  81  task which in turn passes message  80   b  to the RCFG  83  task, and ultimately to the RCTL  85  task is the same as that described with reference to FIG.  5 . However, the LOOPBK_LOOP_BACK_Indication and LOOPBK_ACK sequence does not need to be followed because the master node  6   b  has been notified directly by the host  4  and the master node  6   b  simply sends the LOOPBK_PREPARE_LOOP BACK message  85  in order to begin the synchronized routine for the two neighbor nodes in that instance to loop back their respective ports simultaneously. The master node waits for the acknowledgement ACK message  85   b  from the non-master, and it also waits for its own ACK message  85   b ′. Assuming that such acknowledgments are received, the process continues in the manner described herein. 
     After both neighbor nodes, such as the neighbor nodes  6   a  and  6   c  of FIG. 2, have simultaneously looped back their ports, the physical connection of the new node  6   d  into the network  12  can take place. The new node  6   d  must be prepared, however, for entry into an active inter-nodal network. It is desired that the new node  6   d  operate as if it had always been part of the network  12 . 
     Accordingly, in order to accomplish this, the new node  6   d  is programmed in accordance with the present invention to short cut the normal sequence of operating states through which it would otherwise transition and, instead, to follow a special sequence of states until it is in a running state and the network  12  is ready to include it. More specifically, FIG. 8 is a state transition diagram illustrating the states in which the new node  6   d  will remain while both it and the network  12  are prepared for its addition thereto. The new node  6   d  is powered on and begins operations in the RESET state designated in step  93 . It is typically configured in hardware to immediately begin to look for an external connection by checking for light (i.e., the presence of an optical signal) on its “A” and “B” I/O ports. This is illustrated by the ESTABLISH LIGHT state  94  in FIG.  8 . At this point, the new node  6   d  is connected, typically by an Ethernet interface, to the host  4 . (FIG.  2 ). The system software needed to operate the node on the network is then down-loaded into an appropriate storage device within the node. The new node  6   d  is then preferably assigned, by the host  4 , a unique logical node ID which will identify it on the network  12 , as described in U.S. Pat. No. 5,923,643, referenced herein. 
     A message is then issued by the host  4 , which may be controlled by a user, to cause the new node to perform a diagnostic check, as desired in the particular circumstances. This is designated in FIG. 8 by the message box  95  entitled MSG: RIC_DIAGNOSTICS. Diagnostic sequences may then be run to check the operation of the node  6   d.  Any diagnostics should be run before the new node  6   d  is brought into service on the network to prevent undesirable service interruption. 
     After performing all diagnostic tasks triggered by the RIC-DIAGNOSTICS message  95 , the node waits for configuration messages in the GET CONFIG state  96 . Normally, upon receipt of such information, a node would automatically move to the ESTABLISH CLOCKWISE state  98  and through the remaining states  98  through  104 , in the normal sequence of events. However, in accordance with the invention, a message is then issued by the host  4  entitled MSG: PREPARE_FOR_ADDITION  106 , which causes the node  6   d  to operate on an alternative path in which it transitions to the WAIT — FOR_CONFIG_IN_ADDITION state  108 . This is a short cut from the normal sequence of states several of which are illustrated by example in steps  98  through  104  of FIG.  8 . 
     The new node  6   d  will wait in the state  108  until it receives configuration information from the host  4 . Before the host provides such information, however, it is preferred that the new node  6   d  is checked to ascertain that both of its I/O ports are open, that its transmitters are disabled in hardware, (this is so that the node cannot begin transmitting onto the network until it is desired that it do so), and that the diagnostic checks have been successful. Now, the configuration messages may be sent by the host  4  as indicated by block  110  in FIG.  8 . These configuration messages will preferably include the following elements: a logical network identification, as there may be more than one network in the system  2 , an indication that the new node is configured for the transmit mode (which becomes effective only after subsequent authorization as described hereinafter), and the number of packets, such as the packets described with reference to FIG. 4, it will use. Once this configuration information is obtained, the new node will automatically transition to the WAIT FOR ADDITION state  112  in which it will wait and do nothing until a special message is received from the host  4 , which message is MSG: ADD_NODE  114 . 
     At this point, the new node  6   d  is ready to be brought into service on the inter-nodal network  12 . It has been prepared and is waiting in its WAIT FOR ADDITION  112  state. Consequently, the two neighbor nodes  6   a  and  6   c  (FIG. 2) should now be returned in synchronized fashion to open operating mode. 
     The re-opening of the looped back ports in the neighbor nodes is managed in a manner similar to the sequence followed to place the nodes in loop back mode. With reference now to FIG. 9, assume that a non-master node, such as the node  6   a  in FIG. 2, is connected by a communication link to the host  4 . The host  4  issues an EXPAND_NETWORK message  120   a  which is received by the SYM task  81  on node  6   a.  The SYM task  81 , in turn, sends an EXPAND_NETWORK message  120   b  to the RCFG task  83 , which in turn sends message  120   c  to the RCTL task  85  of the non-master node. In response, the RCTL task  85 , using the control word  64 , passes an EXPNTK_EXPAND_IND  126   a  to the master node  6   b.  The master node  6   b  then sends an EXPNTK_EXPAND_ACK message  126   b  acknowledging receipt of message  126   a  from node  6   a.  Upon receipt of acknowledgment  126   b,  the non-master node  6   a  stops sending any messages using the inter-nodal network control word  64 . This allows the master node  6   b  to have sole use of control word  64 . 
     The master node  6   b  then sends a EXPNTK_PREPARE_OPEN message  128   a.  In a manner similar to that described with reference to the LOOP BACK series of messages, the master node sends the EXPNTK_PREPARE_OPEN message  128   a  continuously for approximately 200 ms to allow both neighbor nodes to receive it. Once the neighbor nodes receive that message, they acknowledge it with messages  128   b.    
     Assuming that both neighbor nodes receive message  128   a,  they initialize a timing procedure similar to that previously described to read control word  64  every 250 microseconds, to allow for a quick response time. After the 200 ms time period, the master node  6   b  then immediately sends the EXPNTK_OPEN_PORT message  130 . Upon receipt of this message, the two neighbor nodes schedule the opening of their looped back ports in 2 ms. After the 2 ms delay, the master node starts to send the EXPNTK_COMPLETED message  132 . The neighbor nodes, now with their respective I/O ports open, read control word  64  until each node receives the EXPNTK_COMPLETED message  132 . This verifies that the inter-nodal network is intact. 
     FIG. 10 illustrates a messaging sequence which may be used when it is the master node that receives the message from the host  4  because it is one of the neighbor nodes in that instance. In such a case, the master node can immediately begin to send the EXPNTK_PREPARE_OPEN message  128   a  to begin the synchronized routine which then proceeds in the same manner as discussed with reference to FIG.  9 . 
     At this point, under normal conditions, there are no looped back I/O ports in any node on the inter-nodal network  12 . In order to enable the new node  6   d  to come into service (e.g., make connections, etc.), that node must transition to the RUNNING state  116  (FIG.  8 ). The new node  6   d  will attempt to verify that it is visible on the inter-nodal network  12  by writing its node identification (ID) into the control word  64 . When the master node  6   b  receives the control word  64  containing the node ID for the new node, it returns that node ID in the control word  64  as an acknowledgment. When the new node  6   d  receives its own node ID back from the master node  6   b,  it then configures its transmitter to transmit, if it has previously been instructed to do so by the host during the configuration step. The new node  6   d  then transitions to the RUNNING state  116  and is now in service on the inter-nodal network  12 . 
     It should be understood the foregoing arrangement may be used to add more than one node to an inter-nodal network at one time without an interruption of service, so long as only one break is made in the inter-nodal network. 
     While the invention has been particularly shown and described with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention. Furthermore, the terms and expressions which have been employed are used as terms of description and not of limitation, and there is no intention in the use of such terms and expressions, of excluding any equivalents of the features shown and described or portions thereof, but it is recognized that various modifications are possible within the scope of the invention claimed.