From: ken@cs.cornell.edu (Ken Birman)
Subject: Notes on IP Multicast option for ISIS
Organization: Cornell Univ. CS Dept, Ithaca NY 14853
Date: Tue, 22 Dec 1992 21:22:30 GMT

I am working on IP multicast for Isis, as readers of this group
are probably aware.  I just want to outline the structure of what
I am doing and solicit feedback or ideas before the design becomes
locked into stone.

When will IP mcast be used?
   - I plan to require that all (or most) machines using Isis support
     the IP Multicast standard, which was developed by Steve Deering at
     Stanford and has become part of Solaris 2.1; Steve's code is also
     available for many other platforms, but perhaps not as a standard
     offering from the vendor.

     The code defines some setsockopt() options: SO_REUSEADDR, IP_MULTICAST_TTL,
     IP_ADD_MEMBERSHIP, IP_DROP_MEMBERSHIP.  It also includes a multicast
     router, called "mrouted".  If these are not up, IP multicast use triggers
     a point-to-point mechanism slightly but not drastically slower than
     the usual Isis UDP scheme.

   - You will need to have an authorization for "ipmcast" in your isis.rc
     file.

   - You will need to specify ISIS_IPMCAST as a flag to isis_remote_init()

   - The group(s) that will use IP multicast should specify PG_IPMCAST as 
     an option to pg_join() (clients find out automatically).

   - You will need to specify a range of port numbers dedicated to use by
     Isis for IP multicast.  You can decide how many and what base address
     Isis should use.  Isis will reserve them as soon as a program specifies
     ISIS_IPMCAST.  Ideally, one port per group using PG_IPMCAST, but we
     can manage if the number is smaller.  In the limit, you can specify that
     just one port be used, but in this case processes will get and discard
     some messages not intended for them.

   - Only messages sent in diffusion mode (to the full set of members and
     clients) will be multicast.  However, you can overlap groups and in this
     way can build up other patterns pretty easily.

How does it work?
   - We fragment your message into 4k chunks and stick a small (44 byte)
     header on each.  

   - For each group joined using PG_IPMCAST as an option, we allocate an IP
     multicast address and register the members and clients.

   - Members will multicast a packet initially, then retransmit point to point
     if some processes NACK or fail to ACK it.

   - Once received, packets travel the same code path as other Isis packets,
     so all the usual Isis ordering properties and so forth hold.

Summary:
   - isis_remote_init(..... ISIS_IPMCAST); to start it up
   - pg_join(.... , PG_IPMCAST, ....); to enable for a group
   - Everything else works as usual, transparently.

From experiments (done, however, with a different version of the code) we
expect to see a benefit for groups with more than about 4 members.  However,
we have yet to experiment seriously with the new code on Solaris, for
technical reasons, so I won't have solid numbers for a little longer.
My guess -- just a guess -- is that performance will be fairly flat as
a function of group size until groups+clients gets pretty large, probably
40 or 50 or more.  By then, acks coming back may begin to be a serious
load on the sender and cause performance to slowly taper off.

Users contemplating huge fanouts, say to hundreds of processes, will want
to build some form of tree structure.  Read the posting from 2 weeks ago
about the "O" option to cbcast if you do something like this, with multiple
overlayed groups.

We are willing to take on beta users starting in mid January.  

Please feel free to comment or flame about this.  I am interested in
feedback and hearing questions.
-- 
Kenneth P. Birman                              E-mail:  ken@cs.cornell.edu
4105 Upson Hall, Dept. of Computer Science     TEL:     607 255-9199 (office)
Cornell University Ithaca, NY 14853 (USA)      FAX:     607 255-4428
Newsgroups: comp.sys.isis