Abstract:
A method for constructing and utilizing a distributed data communication network that will meet its performance specifications in the most economical and efficient manner. The steps of the method include clustering terminals based on their proximity and traffic; selecting backbone node sites based on cost; designing the access network composed of terminals, concentrators, backbone nodes and the links between them; generating a backbone link topology; and determining routes in the backbone given the backbone topology and a traffic matrix.

Description:
This application is a continuation of U.S. Pat. application Ser. No. 07/475,871, filed on Feb. 6, 1990, now abandoned. 
    
    
     FIELD OF THE INVENTION 
     This invention relates to distributed data communication networks, and more particularly, to a method for constructing and utilizing a distributed data communication network that will meet its performance specifications in the most economical and efficient manner. 
     BACKGROUND OF THE INVENTION 
     A distributed data communication network is a hierarchical system of hardware components arranged so that each hardware device is connected, directly or indirectly, to every other device. At the lowest level are user terminals or hosts, which form part of the access network. These terminals are linked to one or more concentrators, a statistical multiplexer often with protocol converters and several low data rate output lines and a fewer number of high data rate output data lines. The concentrators form the second level of the hierarchy and, together with the terminals, form the access network. 
     The concentrators, which may be connected to other concentrators in a hierarchical fashion, are ultimately connected to the backbone, which forms the highest level in the hierarchy. The backbone consists of high speed, high capacity links that terminate at backbone nodes. A backbone node consists of one or more devices, and includes a switching device for routing traffic within the backbone. The data traffic from the concentrators enters the backbone at a backbone node. 
     Existing methods for utilizing a distributed data communication network have deficiencies that are overcome by the present invention. Some of these methods were only capable of evaluating each level of the hierarchy independently of all the others. For example, the topology of the backbone links connecting the backbone nodes would be arranged for optimal performance independently of the access network. This method, whereby each level of the hierarchy is optimized independently of all the others, does not necessarily produce a distributed data communication network that operates most economically or efficiently. 
     Other so called vendor-specific methods can only be used to design a distributed data communication network with equipment provided by a limited number of vendors, and in many cases with equipment provided by only a single vendor. Still other generic methods, while not limited to specific vendors&#39; hardware, do not address the hardware capabilities with sufficient detail to permit the selection of particular hardware components. Finally, other existing methods can only be used to design backbone links composed of T1 links or 56 Kbps links, but not a hybrid of both. 
     SUMMARY OF THE INVENTION 
     The present invention is directed to a method and apparatus used in constructing and utilizing a distributed data network that overcomes the limitations of the prior art. More specifically, the invention utilizes an interactive personal computer-based tool used to select and arrange distributed data communication networks that make use of existing communications equipment. 
     The invention employs a sequence of steps for the optimization of equipment that automatically achieves effective network configurations. It also provides for graphical network display and editing. These features allow users to generate customized configurations as well as to modify the configurations generated by the system. Furthermore, users can invoke network analysis that will evaluate any design. All the data utilized by the invention is maintained by a relational data base management system. 
     The system for analyzing the network consists of five modules, each containing its own subsystem for analyzing, evaluating and constructing the network. Four of the modules are used to construct the actual network, with each module designing a different level of the hierarchy. The remaining module finds the optimal routes for the traffic in the backbone network once the basic network has been fixed. 
     In general, the modules are arranged sequentially such that the outputs from one module are used as inputs in the next module. However, the output from a later module can be fed-back as input into an earlier module in order to improve the network&#39;s configuration. Ideally, each aspect of the process requires the results of all the other parts of the process as input. For example, in determining the geographic location of concentrators, it would be desirable to have as input the location of the backbone nodes in order to compute the cost of connecting a concentrator to the backbone. It would also be desirable to have as input the complete backbone topology so that an estimate of the cost of transporting traffic from the concentrators through the backbone could be made. Obviously, it is not in general possible to have the rest of the configuration available when any particular part of the process is begun. By using an iterative procedure, however, the results of a configuration step occurring later in the process can be fed-back into a step earlier in the cycle. This method results in a network that is better optimized to meet its specification requirements than the networks that use the existing methods. 
     Each module in the process takes as input relevant data from other parts of the configuration or, when such data is not available, parameters are used to estimate such data. During the initial iterations, these parameters are specified based on experience with similar types of problems, or if no such experience is available, they are specified on the basis of general knowledge of network principles. Each parameter has a reasonable default value which can be used if no other basis is available for its specification. 
     The user has as much or as little control over the decision making process as desired. It is possible for the system to operate in an almost completely automatic mode, producing a complete design with little user interaction. Alternatively, the user can continuously interact with the system by inputting a partially selected arrangement, requiring that the system complete the construction, and finally inputting the corrections to the configuration that have been made by the system. 
     The above is a brief discussion of some of the prior art and features of the invention. Other advantages of the invention can be gleaned from the detailed discussion of the preferred embodiment that follows. 
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS 
     FIG. 1 is a schematic diagram depicting a distributed data communication network, which includes backbone nodes, concentrators, and traffic sources, as well as the data link lines between them. 
     FIG. 2 is a flow diagram illustrating the steps of the present invention. 
     FIG. 3 is a map showing the distribution of terminals to be clustered. 
     FIG. 4 is an input data table for some of the terminals shown in FIG. 3. 
     FIG. 5 shows a map of the output for the candidate cluster sites and the candidate concentrator sites. 
     FIG. 6 shows a map of the output for the candidate cluster sites and the candidate backbone sites. 
     FIG. 7 shows the output of the terminal clustering module in tabular form. 
     FIG. 8 shows the input data concerning the data terminals for the backbone node location module. 
     FIG. 9 shows a map of the data terminals and candidate backbone node sites selected by the backbone node location module. 
     FIG. 10 shows the output of FIG. 9 in tabular form. 
     FIG. 11 shows the concentrators chosen by the access configuration and analysis module in tabular form. 
     FIG. 12 shows the output data concerning the local access links determined by the access configuration and analysis module. 
     FIG. 13 shows the output data concerning the traffic through the backbone and the availability of traffic routes between backbone nodes. 
     FIG. 14 shows the final network configuration produced by the backbone link optimization module. 
     FIG. 15 shows the network configuration of FIG. 14 in tabular form. 
     FIG. 16 is an output table from the backbone routing, delay and reliability analysis module showing the routing information 
     FIG. 17 is an output table from the backbone routing, delay and reliability analysis module showing the link delay data. 
    
    
     DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 
     An example of a distributed data communications network is shown in FIG. 1. The network is depicted as a set of interconnected data terminals S 1  -S 16 . The terminals are connected to concentrators C 1  -C 6  such as Telenet Communications Corporation model number TP 3000 or TP4 series multiplexers. The concentrators are in turn connected to backbone nodes N 1  -N 3 , which contain data switches such as a Telenet Communications Corporation TP4/III or TP4/II series switch. The traffic sources S 1  -S 16  are connected to the concentrators C 1  -C 6  by data link lines TL 1-15  that are leased from the transmission providers. The concentrators C 1  -C 6  may be connected to other concentrators C 1  -C 6  by data link lines CL 1  -CL 2 , or, the concentrators can be connected directly to the backbone nodes by data links CL 3  -CL 6 . Both of these types of lines are leased from the transmission providers. The backbone nodes N 1  -N 3  are connected together by backbone data link lines BL 1  -BL 3 , which are high-volume leased data lines. The invention is a method for determining the set of location/device combinations for the backbone nodes and concentrators, as well as the links between them, given a set of terminal locations and the traffic, that will have the least cost while meeting the specification requirements. The traffic is the quantity of data per unit time that is transmitted through the lines. 
     The hardware components shown schematically in FIG. 1 can represent a variety of hardware devices. Each terminal/host S i  represents either a physical user terminal, or a virtual terminal composed of an aggregate of more than one physical terminal. In general, a single physical location may have more than one hardware device. For example, a backbone node location N i  can have, in addition to a switch, other devices such as concentrators. The links TL i , Cl i , and Bl i  can also be analog or digital communication lines. 
     The five modules which make up the invention are terminal clustering, backbone node location, access network design and analysis, backbone link optimization and routing, and finally, delay and reliability analysis. FIG. 2 is a flow diagram illustrating these steps of the present invention. It includes Block 100, which is the initial set up of the data tables by the relational data base management system. Each of these five modules will be discussed in detail. 
     The first module in the network design system is the terminal clustering module, indicated by block 200 in FIG. 2, which forms clusters of terminals based on their proximity and traffic. Its purpose is to reduce the number of terminals that must be considered during the later stages of the design process. It can produce as output either virtual terminals, candidate backbone sites, or candidate concentrator sites. Each virtual terminal represents several real terminals, thereby reducing the number of traffic sources that must be handled by the backbone node location module, thus simplifying the construction problem. If desired, this module can also narrow down the set of candidate backbone sites and the set of candidate concentrator sites that were chosen as candidates initially. The smaller set of candidates chosen by this module are used as potential backbone sites and concentrator sites in the later modules. 
     This module takes as input a set of terminal locations that are to be clustered and a weight for each one that represents the data traffic carried by that location. The parameters of this module to be specified by the user include: the total number of clusters to be formed; the minimum weight of the smallest cluster; the maximum weight assigned to a single cluster; and the maximum distance from the center of the cluster that a terminal assigned to it can be located. If desired, the user can constrain each cluster to a local access and transport area (LATA). Also, the user can manually select the location of cluster centers. If the location of the cluster center is left unspecified, the location will be automatically determined by the module based on the weighted traffic volume of each terminal location coordinates (V,H). The output of this module are the clusters formed and the terminals associated with them, as well as the weight of each cluster. Thus, the output is essentially in the same form as the input and it can be thought of as defining a reduced problem of the same type. 
     The next module is the backbone node location module, indicated by block 300 in FIG. 2, which selects a set of backbone nodes based on costs. It uses as its input candidate backbone node locations, data traffic source locations (terminals or virtual terminals), and a traffic table, which specifies the traffic between these sources. The module works by balancing the costs of the backbone (backbone node switch hardware and backbone link data line costs) against the cost of the local access network (terminal-to-concentrator and concentrator-to-backbone node line costs). 
     As additional nodes are added to the network, the length of the lines, and hence the cost, between the local access network and the backbone nodes decreases. However, a point is reached at which the cost of adding an additional node to the network will be greater than the savings resulting from the decreased line costs. This module balances these considerations. In this evaluation mode, the invention can determine the cost savings or increases incurred in incrementally adding or deleting nodes from the backbone. Both the backbone and the local access efficiencies are approximated. The module can also take as input a partial list of backbone locations which have been chosen by the user and select the remaining sites. In another mode of operation, the module can evaluate the efficiency and performance of a backbone composed entirely of node locations specified by the user. This module&#39;s performance is determined by values of several key parameters. Among them are &#34;access&#34; parameters influencing local access cost, a &#34;detour&#34; parameter effecting backbone costs and &#34;link utilization&#34;, defining the allowable line utilization. 
     The access configuration and analysis module, indicated by block 400 in FIG. 2, consists of two parts. The first part configures the access network, which is composed of terminals, concentrators, the links between terminals and concentrators, the links between concentrators and concentrators, and the links between concentrators and backbone nodes. It also selects concentrators from the list of candidates. A concentrator candidate consists of a potential concentrator location as well as the number and type of devices to be placed at that location. The user therefore has the flexibility of specifying which types of devices are permissible at each location. 
     The inputs are the set of concentrator candidates, the set of terminals, and the terminal level requirements for both the number of data communication lines and the volume of traffic. The module uses a model of concentrator costs and capacities as well as terminal to concentrator connection costs. The user may also specify mandatory and existing concentrator sites and associate specific terminals to specific concentrators. Thus, it is possible to use this module for both initial and incremental configuration. 
     The local access module&#39;s key parameters define a number of terminal-concentrator links, cost parameters for that level of concentration (1-level, 2-level designs), and trade-off costs between terminal-concentrator and terminal-backbone connectivity. 
     The access analysis portion of this module calculates several performance attributes of the access network. Using as input the data traffic requirements between the terminals, the module routes the traffic through the access network and derives two reduced traffic requirement matrices. One list is for traffic accumulated at the concentrator level and the other is for traffic accumulated at the backbone level. For each backbone node, the average delay is computed to and from the backbone. Two reliability measures are calculated; the probability of a packet reaching the backbone assuming it has reached the node, and the expected flow to and from the backbone, given the probability of network link and node failures. This module allows for the design of very general hierarchical access networks. In particular, a lower level concentrator or terminal can be homed to several devices at a higher level. However, it is assumed that all the terminals are homed to concentrators. They cannot be homed directly to backbone nodes. The traffic requirements are determined between terminals, concentrators and backbone nodes. 
     The fourth module is the backbone link optimization module, indicated by block 500 in FIG. 2, which generates a backbone link topology. It uses as its input a set of backbone node locations, a backbone traffic matrix, which contains the traffic accumulated at the backbone level, and a list of links which are candidates for inclusion in the backbone topology. This module can generate a single speed topology considering transmission tariff, including the costs of leasing lines to carry data over the required distance. Alternatively, the module can design a two-level network. The lower of the two levels will be a high speed &#34;slow&#34; circuit switched network (a T1 network), which is used to provide individual channels for the higher level, packet-switched network. 
     Two different evaluation techniques may be accessed through this module. One generates a 56 Kbps level network and possibly also a T1 level network as well. The other generates a T1 level network based on a set of 56 Kbps channel demands. 
     Whenever this system is run, the following summary statistics are produced: the total cost of all physical backbone links, including direct backbone node to backbone node 56 Kbps links and T1 links; and the total costs of all logical channels derived from T1 links. The module normally derives the costs of these logical channels using the standard tariffs. The traffic weighted average backbone node to backbone node delay through the backbone is also calculated. In addition, an estimate is made of the traffic weighted average backbone node to backbone node availability. The detour factor, which is the cost of the shortest path between a pair of backbone nodes divided by the cost of a direct link between these same nodes, is also determined. The resulting value is the traffic weighted average for each node. 
     The backbone routing, delay, and reliability analysis module, indicated by block 600 in FIG. 2, finds routes in the backbone network given a backbone topology (connectivity) and a backbone traffic matrix. The module functions on the level of backbone nodes clusters; both the topology and traffic matrix are specified on the cluster level. The module also includes an algorithm for routing in the TP-7 network &#34;slow&#34; circuit switching. In addition to producing routes for all requirements, the module yields an analysis of delay and throughput. If the results of this analysis are not satisfactory, then the topology may be changed either by using the manual editing capabilities provided with the analysis module or by returning to the topological optimization module and rerunning the algorithms with different parameter settings. 
     After the user has obtained the results from any of the above modules, one or more of the modules can be rerun with different parameter settings. This capability provides a mechanism for feeding back the users&#39; own assessment of the results of each algorithm. An additional and more important type of feedback can be accomplished by rerunning a module based on the results obtained from a later module. Whenever an earlier module is rerun, however, all the results from the later modules are invalidated. For example, if the backbone node locations are recalculated (using the backbone node location module) after the local access and the backbone link designs have been completed, then both the local access design and the backbone link design will be invalid. 
     The present invention provides explicit support for feedback between two of the modules. As illustrated in FIG. 2, the results from the access design module can be fed-back into the backbone node location module. This type of feedback is particularly important because the concentrators are actually the true traffic sources as far as the backbone nodes are concerned. Furthermore, it is almost always desirable that each backbone node location coincide with the concentrator location. Therefore, in order to accurately locate the backbone nodes, it is important to first determine the concentrator locations. 
     This feedback of the concentrator locations into the backbone node location module is accomplished by using the terminal cluster module to regenerate both virtual terminals and candidate backbone sites. The current set of concentrator locations, which have already been chosen, should be used as fixed cluster centers. The backbone node location module can now be reentered in the virtual terminal mode and rerun to define new backbone locations given selected concentrators, creating a feedback iteration. The result should be an improved set of backbone node locations. 
     For the purposes of illustrating the method of the present invention, numerical examples will be presented which correspond to the distribution of terminals shown in FIG. 3. The first Step 200 is terminal clustering, for which it is necessary to input each terminal&#39;s geographic coordinates, the traffic associated with it and an indication as to whether it should be considered as a candidate backbone or concentrator location. A portion of this information is shown in tabular form in FIG. 4. 
     The outputs of the terminal clustering module are shown in FIGS. 5-7. FIG. 5 shows candidate cluster sites as solid circles and candidate concentrator sites as solid triangles. FIG. 6 shows the candidate cluster sites and the candidate backbone sites, which are now indicated by solid triangles. FIG. 7 shows the output of the terminal clustering in tabular form, indicating both the candidate concentrator and the candidate backbone sites. 
     For the backbone node location module 300, it is necessary to provide as input, in addition to the information above concerning the candidate backbone node locations, a traffic table specifying the traffic between the terminals. An example of this input table is shown in FIG. 8. The traffic volume between the terminals can be specified in the form of bytes, packets, or calls. 
     Based on these inputs, the method of the invention configures the network shown in FIG. 9. Information about the selected backbone nodes is shown in tabular form in FIG. 10. 
     Using the above tables as input, the access configuration and analysis module 400 yields the outputs shown in FIGS. 11-13. Information about the concentrators chosen for this network is presented in tabular form in FIG. 11. Information about the local access links is presented in FIG. 12, while information about the traffic through the backbone and the availability of traffic routes between backbone nodes is presented in FIG. 13. 
     The backbone link optimization module 500 produces as output the final network configuration shown in FIG. 14. This configuration is also shown in tabular form in FIG. 15. 
     The backbone routing, delay and reliability analysis module 600 uses as input the data from FIG. 15. A sample output resulting from the method of the invention is shown in FIGS. 16 and 17. FIG. 16 includes the routing information and FIG. 17 includes the link delay data. 
     The preferred embodiment of the present invention described above represents one desirable, workable embodiment of the present invention. It is to be understood that the methodology described herein is not limited to specific forms disclosed by way of example and illustration, but may assume other embodiments and methods limited only by the scope of the appended claims and their equivalents.