# Patent

## Title
Load balancing in a network environment

## Abstract

This application is being filed with the following related applications on the same date, U.S. patent Ser. Nos. 09/139,948; 09/139,797 and 09/140,180.

## Background
The present invention is generally directed to an IP multicast interface used in network connections with gateway devices.

Personal computers are gaining widespread popularity as the state of technology is improving. Over the past few decades, their utilization has proliferated both for personal purposes and for use in business and scientific communities. Establishing good computing communications have become a necessity as individual users try to connect to one another for exchange of information, and to larger computers to take advantage of their higher processing capabilities. This need to communicate between different computing hosts or nodes have evolved into the creation of distributed networks. A distributed network is an aggregate of individual systems that are connected to one another electronically. Distributed networks can be organized in a number of ways, either remotely extending over great distances using wide area networks or WANs, or locally through the use of a Local Area Network, better known as a LAN.

A LAN usually consists of a number of nodes or hosts located within an office, a building or at other close proximations. Being a type of a distributed network, a LAN loosely couples processors and workstations. Generally, workstations on a LAN do not share a central memory but do share common servers. In this way a LAN increases the power and flexibility of workstations by enabling them to access shared data without jeopardizing the security of each individual resource.

A LAN system that has been in wide use in recent years is produced by Novell, Inc. of Provo, Utah. In a Novell system, a LAN device driver is implemented on top of the local operating systems to be coupled and device driver commands at the LAN workstations are directed to and from the workstations onto the LAN to the target servers.

As networks have grown and particularly as LANs have come into widespread use, many businesses and organizations have faced the necessity of interconnecting and managing a confederation of networks and LANS. Each network itself can in turn comprises of a plurality of logical networks which in turn run independent and different networking protocols. The challenge has not become only to interconnect local area networks to one another, but to do so and still provide remote server access through WANs or other devices.

Three basic methods are now available to interconnect both local and remote area networks to one another as to provide wide used access and remote information exchange capabilities. These three methods are 1) bridges or data-link devices that connect similar networks together; 2) routers that perform routing services by maintaining a routing table in each host; and 3) gateway devices that carry out protocol conversions and other connectivity functions. Typically, a device driver for the gateway is provided with modems or other physical ports that can be linked to switched communication WANs.

A gateway facility allows the interconnection of multiple independently controlled communication networks to one another in a way that logical units in the network can communicate with one another without any changes to the network. A logical network runs a single networking protocol, but a processing organization can be composed of a dozen of logical networks running six or seven networking protocols. A gateway provides transparent interconnection of these single networking protocols, so that a single multiport transport network is formed.

In the existing mechanisms, gateway devices are implemented on top of the LAN device drivers as a switched communications device interface. The user initialization of the communication link-up procedure redirects the user hardware commands to the gateway. The communications interface in the gateway driver then institutes and maintains the switched communications link, diverting hardware resources of the driver to do so. The connection and access procedures are then executed using the gateway ports and modems in order to link the user's system with the switched communications network. A remote connection is established through the LAN/WAN which sets up a point to point configuration through the port along the communication line between the user and the communications device in use.

The procedure described above have many complexities associated with it. The complexities connected with the configuration assessment of gateway devices is an on-going concern of the designers of such devices today. These configuration concerns contribute to many limitations that exist with today's gateway devices. These limitations often make the interconnection of networks running different protocols non-transparent. Because many of the present gateways are transport layer protocol-specific, it is possible that a gateway cannot interconnect a network running for example a TCP/IP protocol and a network running the SNA protocol. Furthermore, a variety of gateway devices have been developed which connect TCP/IP to different operating system and give connectivity to the LAN/WAN environments, but each time the configuration has to be redefined and reassessed before connectivity is accomplished successfully. Furthermore, each gateway device can usually implement only a subset of the TCP/IP functions. Most currently existing gateway devices do not support many of the TCP/IP functions and performance problems have been encountered due to increased bandwidth of the LAN/WAN arenas. One of the communication protocols used to communicate with the gateway is LCS or LAN Channel Station. A control command interface exists in the LCS protocol that requires all the configuration information to be set prior to the TCP/IP connect sequence. Nonetheless, the control interface does not have a command sequence to enable the use of the TCP/IP functions which have been implemented in the gateway devices. To reduce the complexity of configuring gateway devices, an expanded control command interface is needed.

This application incorporates by reference U.S. Pat. No. 5,999,974 issued Dec. 7, 1999.

A method for addressing IP multicast problems using a special set of control commands in a computing network environment having at least one host and a plurality of clients connected to one another via a local area network (LAN). The method comprises: sending a special control command by a first platform that desires to add a new IP multicast address; providing a LAN MAC multicast address as well as an IP multicast address with the special control command; a controlling platform receiving said connection requests and registering said information provided about each new IP multicast address in a multicast routing table; creating one entry in said table for each connection request received if said connection request contains a new IP multicast address entry; initiating a use count mechanism to keep track of numbers of communication stacks which have registered with a certain IP multicast address, thus allowing different virtual hosts to register new entries; and routing all inbound IP packets destined for an IP multicast address to all communication stacks which have registered with said controlling platform.

A method of dynamically configuring and monitoring every host connected to a gateway device in a computing network environment.

The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of practice, together with further objects and advantages thereof, may best be understood by reference to the following description taken in connection with the accompanying drawings in which:

FIG. 1 is a block diagram illustration of a computing environment;

FIG. 2 depicts the header format including a LAN MAC header for a block of data;

FIG. 3 is a block diagram illustration of the prior art showing a computing environment;

FIG. 4 depicts a diagram showing different functions available such as IP Multicast function in a computing network environment;

FIG. 5 is a block diagram illustration of a computing environment having a gateway device and a plurality of clients as practiced by one embodiment of the present invention;

FIGS. 6A and 6B, and 6C and 6D are tables showing values at the onset of SETIPM and DELIPM command respectively; and

FIG. 7 depicts a multicast routing table example as used by one embodiment of the present invention.

FIG. 1 is an illustration of an example showing the connection of a large computing network environment, such as one provided by a large mainframe computer, to a Local Area Network (LAN) like the Internet. The large rectangle in FIG. 1 shown at 100 is a computing network environment such as an IBM S/390 computer. The operating system(s) for the computing environment is shown as 110 and can be one of many available operating systems such as OS/390 or MVS/ESA. At the next layer, shown at 120, the software products and applications reside, some examples being DB2, TSO, CICS, and IMS. The Common Gateway Interface or CGI is shown at 130 and interfaces with the communication devices such as an IBM Internet Connection Server, to access VTAM (shown at 142), TCP/IP (shown at 144), IPX (shown at 146) or other similar communication protocols. The network connection to the LAN and/or Internet (shown at 160-174) can be accomplished by means of any indicated communications controller, such as an Open Systems Adapter connection, hereinafter OSA. The network connection from the computing environment servers can be a channel (150) or an integrated platform (155) connection. An Open System Adapter or an OSA shown at 158 is an integrated connectivity platform and a product of IBM Corp. of Armonk. OSA provides direct attachment to token ring (shown at 172), ethernet (shown at 174), FDDI's or fiber distributed data interfaces, and ATM (not shown). Multiple host systems can be accessed from a single workstation through a single OSA LAN/WAN port. Integrated on the OSA card is a channel adapter, a control unit and LAN adapters. OSA supports a variety of communication protocols, namely SNA/APPN, TCP/IP and IPX. OSA allows the sharing of applications and/or LAN ports among logical partitions.

There are many protocols used to connect the communication stacks, for example CS/390 TCP/IP Stack, to a channel attached platform or an integrated platform such as the OSA. In the following sections, the teachings of the present invention may be described in reference with a system using OSA and CS/390 in a setup using TCP/IP with MVS operating system. However, the workings of the present invention is not limited to the use of OSA, CS/390, MVS or TCP/IP and any such reference will be used for demonstrative purposes only.

A variety of gateway devices have been developed which connect to TCP/IP and give connectivity to LAN/WAN environment. In the existing systems, when a host is communicating with a LAN or other similar networks a Media Access Control Header, also known as a MAC Address, along with the LAN type, has to be first established so that the host and the LAN understand what kind of device and at what address the communication is being determined to address the particular needs of one another.

FIG. 2 shows how a block of data may be transferred in the existing systems. A Block Header 210 can indicate the start of a new data batch. Each block header 210 is usually followed by the appropriate MAC Header or MAC Address and LLC shown at 212 and 214. The Protocol header and the Protocol data follow as shown in 216 and 218. A new block of data is shown at 220 and 230. Usually the applications receive user data which is to be sent to the LAN in a data buffer. The applications send the user data to the protocol layer.

Building a MAC header and the deciphering of data each time is not an easy task. In a large communicating environment, each time the host and LAN devices communicate a MAC Header has to be created which can be both inefficient in terms of time and memory consumption. Each time communication is to be established information including the device(s) address on both the sending and receiving ends, MAC header, LAN type, device type, protocol versioning and the like has to be exchanged. This process requires substantial user configuration in existing systems.

With the addition of some new TCP/IP functions, such as International Business Machines' Offload and Assist functions (see related application 9-97-097), the configuration process can become an insurmountable task. Some platforms such as "Assist" and "Offload" have varying degrees of functionality which each requires its own specific type of configuration file. As new TCP/IP functions are being added, the configuration process in these platforms continues to grow in complexity.

One such function is the Video Conferencing function that can be performed with the TCP/IP protocol. FIG. 3 is an illustration of the prior art showing a multicast computing environment. A multicast environment is used for many purposes today, one example being video-conferencing. In way of example, if clients C2312 and C4314 are to participate in a video-conferencing session only, as depicted by FIG. 3, there must be a way to allow them to do so without clients C1311 and C3313 being affected. To do so all clients have to be listening at all times to all broadcasted messages to determine which one is intended for them. The broadcasted message usually contain the MAC header followed by an IP Packet header. Each client has to listen at all times and check all messages until it finds the broadcasted message with the correct MAC header and IP address. This process is very inefficient and can tie up both time and resources. This problem is even more noticeable when working with TCP/IP functions.

The most common implementation used today for TCP/IP platforms such as "Assist" and "Offload" is to run the LAN Device Driver on the platform itself. The remaining functions supported on these platforms vary from platform to platform. All implementations which have the device driver on the platform and the IP protocol which supports the IP Multicast function on a separate platform need to communicate the necessary IP Multicast information to each other. The current method used today is to have the user enter this information on both platform's configured files. This adds yet another level of complexity to the TCP/IP configuration. Many of these implementations have no means of ensuring the accuracy and verifying that the matching IP Multicast information is centered in both of the platforms. one of the new TCP/IP functions being exploited in the TCP/IP network is the use of IP Multicast Addressing. The IP Multicast Addressing is being used to try and reduce the Broadcast traffic which is currently being used for many of the currently supported routing protocols. Some of the new IP protocols which are using multicast are OSPF (Open Shortest Path First) and RIP2 (Routing Information Protocol 2).

For all IP Multicast Addresses supported by a TCP/IP stack, a corresponding LAN MAC Group address must be registered with the device driver connected to the LAN media. This is needed because the IP Multicast packets are encapsulated inside a LAN Multicast packet. This LAN Multicast packet will only be received by a LAN Device Driver which has registered the proper LAN MAC Multicast Address with the LAN adapter.

FIG. 4 is an illustration of the workings of the related invention of U.S. Pat. No. 5,999,974. FIG. 4 provides some of the functions supported and used by TCP/IP and on the OSA platform. These functions include IP Multicast. The host and its operating system and applications is shown at 400 and 401 respectively. The IP protocols are shown at 402 (IP) and TCP and UDP at 404 and 406, respectively. The TCP/IP or UDP/IP functions available are shown at 440. IP Multicast is one of these functions as shown at 458. (Multiple other applications can be supported as well such as SAP and RIP). The LAN connection is shown at 380 and the gateway connection at 430.

In such setups, usually a control command interface exists in the gateway device to allow TCP/IP or other protocols to connect to the gateway device in order to begin data transfers. However, this control command interface requires all the configuration information to be set prior to the TCP/IP connect sequence. Furthermore, the control interface does not have a command sequence to enable the use of the TCP/IP functions which have been implemented in the gateway device. The TCP/IP program product running on such operating systems as MVS and VM/VSE have been developed by a number of companies. But many of today's TCP/IP implementations have encountered performance problems due to the increased bandwidth of the LAN/WAN arenas. To try and improve the MVS and other similar operating system's performance using TCP/IP or other communication protocols, various TCP/IP functions have been identified which could better be implemented in a gateway device which has LAN/WAN connectivity.

To solve the IP Multicast configuration problem, the present invention uses a new type of control command. This control command is sent from the platform which desires to add a specific IP Multicast address to the platform containing the device driver which must register the LAN MAC Group address with the LAN adapter. The control command contains the specific IP Multicast Address along with the corresponding LAN MAC Multicast address. The platform upon receiving this command will register the LAN MAC Multicast address with the LAN adapter.

The OSA adapter is a type of TCP/IP "Assist" platform. It contains the LAN device driver which interfaces with the LAN media. The IP protocols which support the different IP Multicast functions all currently reside on the operating system connected to the host(s). The OSA adapter can support multiple instances of TCP/IP which could be running on different host images under the operating system. Each image can support different IP Multicast functions. To accommodate this functionality, when the IP Multicast control command is received, the specific host image from which the command was received is registered in a Multicast table. This table entry will contain the host image, IP Multicast Address and LAN MAC Multicast Address. As IP Multicast frames are received from the LAN, the Multicast table will be searched to find all the host images which have registered the specific IP Multicast address. The IP Multicast packet will then be routed only to these one or more specific host images. A host image also needs the ability to remove an IP Multicast Address from the network. To accommodate this requirement, a corresponding Delete IP Multicast command has been developed. This command will cause the IP MAC Multicast Address to be removed from the LAN adapters registration and the IP Multicast address to be removed from the Multicast table.

FIG. 5 depicts a plurality of clients C1501, C2502, C3503 and C4504 are in processing communications with at least one host 500 via a LAN 550. The host(s) comprises of a plurality of servers and host images. Server S1511 corresponds to host image 1521; server S2512 corresponds to host image 2522; and servers S3513 and S4514 correspond to host image 3533. An OSA adapter is also shown at 560 coupled to the LAN. By the use of a SET IPM (IP Multicast) command, at initialization time, hosts images present in the computing network are identified and registered. A corresponding table is provided for storing the characteristics of client, host images including the address, the protocol used and the LAN MAC header information and IP address. The table is then kept dynamically so that with the addition and registration of each new client, host image, the table is updated. When a multicast communication is made the table will be searched for a matching entry. When the corresponding entry is made, the message is routed to all host images which have registered with the corresponding IP multicast address (via SETIPM command). However, if no corresponding match is made in the table, the packet is dropped and resources are freed.

Correspondingly, a Del IPM (IP Multicast) command is also provided to un-register components from the table. Again this updating of the table is done dynamically as new components are registered or un-registered. When a DEL IPM command is used, all corresponding entries in the table will become unavailable for that component or erased.

Routing Table Overview

The OSA Multicast Routing table is built using SETIPM and DELIPM commands from the TCP/IP stack.

The SETIPM command is used to add an entry to the OSA Multicast routing table. The SETIPM commands contain the IP Multicast Address and its associated Group MAC Address. The IP Multicast Address is used to communicate with a group of TCP/IP stations in the network. This group of TCP/IP stations all want to share the data in the network. To perform this task, the TCP/IP data which is sent is packaged as a MAC frame which contains the Group MAC Address which is associated to the IP Multicast address.

The DELIPM command is used to remove an entry from the OSA Multicast Routing Table. The DELIPM command contains the IP Multicast Address to be removed from the OSA Multicast Routing Table. The OSA Multicast Routing table is indexed using a hashing algorithm on the IP Multicast Address. The Routing Table contains a Collision Chain for multiple entries which hash to the same OSA Multicast Routing Table index.

FIG. 7 depicts the concept of Routing Table. Use count 741-743 provides how many times the IP address was registered by for different and separate images or sub-environments. Each SETIPM will increment this count while each DELIPM decrements this count. The IP address and the MAC group address is provided for each entry in 721-723 and 731-733 respectively. The base entry is provided at 710 while the Collision chain is provided as 750.

SETIPM Command and Flow

In an embodiment of the present invention the OSA receives a SETIPM command from CS/390 as the TCP/IP stack receives connection requests from TCP/IP application requesting IP Multicast services. For each connection request received, the CS/390 TCP/IP stack builds a corresponding SETIPM command. This registers the IP Multicast Address and its corresponding MAC Group Address with the OSA adapter. The OSA adapter then updates its Multicast Routing table with the SETIPM information. The IP Multicast Address is hashed to compute a one byte index into the IP Multicast Address table. If the entry being added is a new entry, a new entry is allocated in the table. A collision chain is kept for entries which Hash to the same index. The uniqueness in this implementation is that OSA can have connections to multiple OS/390 Multicast Routers in the same or different virtual modes or Host Partitions (LPARs). OSA can therefore receive the same SETIPM information from multiple different CS/390 TCP/IP stacks. Since each CS/390 TCP/IP stack can register with the same IP Multicast Addresses, OSA must be able to Route inbound IP packets destined for an IP Multicast Address to all of the CS/390 TCP/IP stacks which have registered with the OSA adapter. OSA keeps a "use count" in its routing table entries. The "use count is used to track the number of CS/390 TCP/IP stacks which have registered with the IP Multicast Address. The entry also contains the list of CS/390 TCP/IP stacks which have registered with the IP Multicast Address. This is the list used to forward inbound IP Multicast packets to all the registered CS/390 TCP/IP Stacks.

FIGS. 6A and 6B are tables composed of the information obtained at onset of SETIPM command.

DELIPM Command

The DELIPM command is used by CS/390 TCP/IP stacks to unregister IP Multicast Addresses from the OSA adapter. Entries are removed when either the CS/390 TCP/IP applications issue a Disconnect for the TCP/IP session or the TCP/IP stack itself is shutdown. When an entry is removed from the OSA Multicast Routing Table, the "use count" in the entry is decremented. If the "use count" is still non-zero, the CS/390 TCP/IP stack address is removed from the OSA Multicast Routing Table entry, but the entry will remain active for the other TCP/IP stacks. Once the "use count" goes to 0x00, the entry is then removed from the OSA Multicast Routing table.

FIGS. 6C and 6D are tables composed of the information obtained at onset of DELIPM command.

While the invention has been described in detail herein in accordance with certain preferred embodiments thereof, many modifications and changes therein may be effected by those skilled in the art. Accordingly, it is intended by the appended claims to cover all such modifications and changes as fall within the true spirit and scope of the invention.

## Claims

1. In a computing network environment having host images in processing communication with one or more clients by way of a controlling platform and a local area network (LAN), a method for registering multicast addresses in said controlling platform, said method comprising the steps of: first host image to said controlling platform, said first connection request comprising first information including a first MAC multicast address, a first network protocol multicast address and a first host image identifier; registering said first information in a multicast routing store; second host image to said controlling platform, said second connection request comprising second information; registering said second information in said multicast routing store; and packets destined for said first network protocol multicast address to all host images which have registered for said first network protocol multicast address with said controlling platform.

2. The method of claim 1, wherein said controlling platform is an Open System Adapter (OSA) platform.

3. The method of claim 1, wherein said controlling platform contains a LAN device driver which interfaces with a LAN media.

4. The method of claim 1, wherein network protocol multicast packets are received from said LAN to be routed to communication stacks identified by information in said multicast routing store.

5. The method of claim 1, wherein said communication stack comprises an application selected from the group consisting of TCP/IP, SNA/APPN, UDP/IP,IPX/IP and VTAM.

6. The method of claim 1, wherein said network environment comprises a plurality of local area networks (LANs).

7. The method of claim 6, wherein said local area networks comprise different network protocol types with different and independent communication stacks.

8. The method of claim 1, wherein said network environment comprises at least one wide area network (WAN).

9. The method of claim 1, wherein said first connection request comprises a SETIPM command.

10. The method of claim 1, wherein said network environment comprises at least one gateway device.

11. The method of claim 1, wherein said network protocol multicast address is an Internet Protocol (IP) multicast address.

12. The method of claim 1, wherein said multicast routing store comprises a first entry, said first entry holding registered information from one or more host images registered to said first network protocol multicast address.

13. The method of claim 1, comprising the further step of registering in said multicast routing store, an indication of the number of host images that are registered to a network protocol multicast address.

14. The method of claim 13, wherein said indication of the number of host images registered comprises a counter field.

15. The method of claim 14, wherein said registering said first information step comprises incrementing said counter field.

16. The method of claim 15, wherein if said counter field is decremented to a zero value, said entry is completely removed from said multicast routing store.

17. The method of claim 15, wherein said entry remains active if said counter field does not contain a value of zero.

18. In a computing network environment having host images in processing communication with one or more clients by way of a controlling platform and a local area network (LAN), a method for registering multicast addresses in said controlling platform, said method comprising the steps of: host image to said controlling platform, said first disconnection request comprising first information including a first MAC multicast address, a first network protocol multicast address and a first host image identifier; un-registering said first information in a multicast routing store; host image to said controlling platform, said connection request comprising second information; said second information in said multicast routing store; and packets destined for said first network protocol multicast address to all host images which have registered for said first network protocol multicast address with said controlling platform.

19. The method of claim 18, wherein said registered second information is removed when said second communication stack has shutdown.

20. The method of claim 1, wherein said disconnection request comprises a DELIPM command.

21. The method of claim 18, wherein said multicast routing store comprises a first, said first entry holding registered information from one or more host images registered to said first network protocol multicast address.

22. The method of claim 18, comprising the further step of registering in said multicast routing store, an indication of the number of host images that are registered to a network protocol multicast address.

23. The method of claim 22, wherein said indication of the number of host images registered comprises a counter field.

24. The method of claim 23, wherein said un-registering said second information step comprises decrementing said counter field.

25. The method of claim 23, wherein said removing step comprises removing said first host image identifier.

26. The method of claim 18, wherein said multicast routing store comprises a first entry, said first entry holding registered information from one or more host images registered to said first network protocol multicast address.

27. The method of claim 26, wherein said removing step comprises removing said first entry when said first host image identifier is the only host image identifier in the entry.

28. The method of claim 18, comprising the further step of removing registered first information when said first host image issues a disconnect command.

29. The method of claims 1 or 18, wherein IAN Channel Station (LCS) protocol is used for said connection request.

30. The method of claim 18, wherein LAN Channel Station (LCS) protocol is used for said disconnection request.

31. The method of claims 1 or 18, wherein said local area network comprises a network protocol selected from the group consisting of ethernet, token ring, FDDI, and ATM.