From: lowry@watson.ibm.com (Andy Lowry)
Newsgroups: comp.os.research
Subject: Re: A challenge
Date: 10 Apr 92 15:30:03 GMT
Originator: osr@ftp

In article <32507@darkstar.ucsc.edu> ast@cs.vu.nl (Andy Tanenbaum) writes:
   ... I hereby challenge all the people who have commented on my list
   to do better.  List what YOU consider the five most important
   things we have learned about distributed operating systems in the
   past 20 years.

Well, let's just say these are the most important things I think
*I've* learned.  I think it's clear by now that there's not going to
be any quick consensus on this! :-)

The top one on my list would have to be:

1. Distributed applications are just too complicated for programmers
   to deal with.
   We need to make programmers as unaware as possible of the execution
   platform on which their programs will eventually be running.  Don't
   make them worry about where processes will be run.  Don't make them
   worry about how their data will be represented.  Don't make them
   worry about what to do when the hardware fails.  Don't make them
   worry about which threads will share which address spaces.  Don't
   make them worry about what architecture they'll be running on, or
   what language the programs they're interacting with were written
   in.  Just give them a programming model that's powerful enough to
   support the complex interactions they will increasingly have to
   incorporate into their programs, and flexible enough to admit a
   vast variety of implementation choices, so compilers and run-time
   systems can collaborate to choose efficient implementations.  Where
   needed, allow the programmer (or a separate "tuning specialist")
   supply hints to the compiler and run-time to help them make the
   best choices.

The rest of my points support the above:

2. Shared memory is not a good programming model.
   It forces programmers to write code that has nothing to do with the
   functional logic of their programs, in order to acquire locks or
   otherwise synchronize their access to shared memory.  If they don't
   do this, they run great risks of seeing intermittent errors when
   the underdog inevitably wins a race or two (excuse the somewhat
   overextended analogy based on the term "race condition").  Physical
   shared memory is a very useful architectural feature that should be
   exploited when available as an *implementation vehicle* for
   interactions that the programmer has specified via some other
   paradigm like procedure call.

3. We should be very worried about latency.
   Growth in network bandwidth is certain to vastly outstrip decreases
   in latency, so in many cases latency will be a primary
   performance bottleneck.  In keeping with point #1, we should not
   force programmers to be cognizant of this.  Instead we need to
   develop transparent mechanisms to battle latency.  The optimistic
   call-streaming ideas that I mentioned in a prior post are an
   example.  In fact, I expect that optimism will emerge as one of the
   major weapons to combat latency.

4. Toolkits are a bad way to provide support for distributed
   applications.
   Have you looked at your favorite RPC reference guide lately?  How
   many API's are there?  Probably several dozen at least.  To do
   what?  Make remote procedure calls look like local procedure calls?
   Give me a break!  Since when do I need several dozen library
   routines to enable my local procedure calls?  This is the wrong way
   to enable the development of programs that can be easily executed
   on distributed platforms.  Language-based approaches are far
   preferable.  Many of today's languages can be extended very
   minimally to support a fully dynamic and distribution-friendly
   programming model.  See the IBM T.J. Watson Research Research
   Report Number RC17341, "Interprocess Communication in Concert/C" by
   Auerbach etc., for an example of how this can be done.

5. Transparent fault tolerance will be crucial.
   Most people over the history of programming have paid little or no
   attention to the possibility that computers can fail while their
   programs are executing.  There's no reason to think that will
   change.  But the cost of unhandled failure will rise dramatically
   as programs interact more frequently and in far more complex ways
   than they do now.  Applications need to be made fault-tolerant, but
   it has to be done transparently.  Transactions provide a
   semi-transparent mechanism in this regard, but transactions are
   applicable only to a small fraction of the applications that people
   use, even today.  And they can impose a large performance hit due
   to the hard global synchronization requirement at transaction
   completion.  Furthermore, the all-or-nothing guarantees provided by
   transaction systems aren't good enough.  I don't want to have to
   reconstruct my entire computational network every time a machine
   goes down, just because some transaction manager decided all the
   applications that were interacting with that machine should be
   aborted.  Failure recovery should include reinstating the
   computational environment that was in effect when the failure
   occurred.  The failure should be transparent not only to programs,
   but also to users.  More general mechanism will be needed, based on
   tracking dependencies among process states and restoring globally
   consistent states via checkpoints and log.  There is an exquisite
   opportunity to exploit optimism to improve response time in this,
   since in actual fact, computers and networks don't fail very often.
   See Strom and Yemini's paper in ACM Transactions on Computer
   Systems, titled "Optimistic Recovery in Distributed Systems,"
   volume 3 number 3, pp. 204-226, August 1985.  For an example of how
   this can be done.
---
Andy Lowry, lowry@watson.ibm.com, (914) 784-7925
IBM Research, P.O. Box 704, Yorktown Heights, NY  10598