From: chk@rice.edu (Charles Koelbel)
Newsgroups: fj.mail-lists.parallel
Subject: July HPFF minutes, part 2 of 2
Date: 14 Aug 92 12:12:02 GMT
Distribution: fj

starting this discussion." There were 2 proposals for this
functionality in the email list, one from Guy Steele and the
other from Marc. Guy's proposal designed a feature internal
to HPF, while Marc's defined an interface to external
routines, or as Marc put it "really an escape to code
outside HPF." Such a feature is needed because HPF needs to
interact with non-HPF code (SPMD libraries, message-passing
code, etc.). Guy remarked that the two proposals were not in
conflict, although they had different origins and goals.
  Marc's first slide presented conventions enforced by the
HPF compiler on the called external routine.
     INTERFACE blocks for declarations in HPF, marked
       by the keyword LOCAL
     Synchronous transfer of control from HPF to the
       external routine
       All processors call the routine, execute
          cooperatively, and later transfer back
       This proposal does not specify what happens
          in the routine
     Alignment and/or distribution are performed as
       required by the INTERFACE block
       Distributions that the routine expects to
          see, HPF enforces (as for regular
          routines)
       Allow libraries to assume a given
          distribution (otherwise lots of queries
          would be necessary)
Relatively little discussion on this interface was
necessary, immediately mostly just clarifying points on the
original slide. (I smooth out grammar and fix handwriting
for these notes.)
  The next slide placed some requirements on the LOCAL
routine to ensure smooth functioning with HPF, and
mechanisms to access HPF data structures from those
routines.
     *  Requirements from the external routine
       Keep multiple copies of data in a consistent
          state (on return from the routine)
     * . Mechanism for external routine access to HPF
       data structures
       Array descriptors are passed as arguments
          along with data
          These are explicit arguments in the
            external routine, not in the HPF call
            itself
       Intrinsics for use of array descriptors
          Query functions
          Functions to access data using global
            indices
The format of an array descriptor is not specified in the
proposal, just functions for encapsulation of its
operations. These can be supplied as a standard module in
Fortran 90, or by other mechanisms in other languages.
  Marc's next slide had only one point.
     HPF calling conventions and external restrictions
       make sense for both SIMD and MIMD machines;
       does the external data access mechanism also?
       SIMD vendors should check it (the proposal
          was written with MIMD in mind)
  This opened the floor for discussion. Andy Meltzer asked
if SIMD or MIMD style synchronization was assumed in the
LOCAL function. Marc replied that it is a black box
interface so any paradigm can be used internally. He
speculated that EXTERNAL is probably a better name for the
feature. Richard Swift suggested "FOREIGN" as a keyword,
since LOCAL and EXTERNAL already have meanings in Fortran
90.
  Piyush Mehrotra asked if the external could access
nonlocal data; Marc replied that FETCH and STORE were
defined in the proposal, but the list there was not
exhaustive. Piyush Mehrotra followed by asking if a local or
global index space was used for HPF arguments within the
function, and Marc replied that local was used, with
intrinsics for global indexing operations. Clemens-August
Thole questioned the need for FETCH and STORE at all, saying
that communication could be handled in other (possibly more
efficient) ways for a given machine. Marc said he wanted to
avoid syntax arguments in this talk. Guy Steele asked about
the type declaration for descriptors; Marc admitted that it
was an error in his proposal to omit them.
  Peter Highnam argued that this defined a foreign
interface, so HPF can't say what is on the other side. He
agreed that the functionality of descriptors was needed, but
claimed that we shouldn't do syntax in this forum. Finally,
requiring foreign code to maintain consistency is incorrect
since it cannot be checked. Marc replied that a correct
external routine must have consistency on return, otherwise
the HPF code is in deep trouble. Richard Swift asked if an
external routine can redistribute HPF arrays; Marc replied,
"Not in this proposal." Chuck Koelbel and several others
pointed out that staying consistent with normal routines in
this respect was a very good idea. Mary Zosel commented that
she still can't get to the right library directly with
Marc's proposal, but at least it gives a wrapper layer.
  David Presberg argued that HPF shouldn't try to define
syntax in external routines, just the minimal information
flow over interface. His example was using this feature to
call a C library, where syntax would be problematic at best.
Ken Kennedy took the other side, arguing that it was very
useful to specify the FORTRAN intrinsics to avoid divergence
in compiler extensions. Randy Scarborough used the example
of matrix multiplication to show the usefulness of defining
a module whose only responsibility was partitioning arrays.
Dave Loveman agreed, but claimed that to do this HPF should
give very exact Fortran 90 interface.
  Clemens-August Thole defined 3 layers of intrinsics for
external routines:
     1. How is this array distributed?
     2. FETCH and STORE operations
     3. Pure message passing
He claimed that 1 was clearly needed to program the external
routine, and 3 is being standardized by another group (who
first met at Williamsburg, VA, in April). The remaining
category is 2, which he claimed had too many subtleties to
standardize quickly in this group.
  Barbara Chapman asked if the proposal was assuming array
sections passed as arguments were stored contiguously. The
answer was no; the storage scheme would be hidden in the
descriptor access functions. Rich Shapiro and Marina Chen
argued for keeping the proposal fully general by not
specifying storage structures. Marc Snir said he had tried
to stay mute on this issue, but the proposal may have to
expand the list of intrinsics. Clemens-August Thole pointed
out the importance of overlap areas in scientific
computations. James Cownie claimed that both overlaps and
Marc's proposal do not require contiguous storage, just that
the array descriptors exist so that they can be used. Andy
Meltzer mentioned that more people want to escape to C than
to Fortran 90 due to the availability of C compilers. James
assumed that everyone will have Fortran 90 to C links, and
that these should generalize to the HPF external linkages.
David Presberg noted that external compilers may not adapt
to HPF.
  Guy Steele reiterated that his and Marc's proposals were
not mutually exclusive, and recommended combining them. The
group concurred, and scheduled a first reading of the
combined proposal for the next meeting.

Storage and Sequence Association

  After the LOCAL subroutine discussion, Ken Kennedy called
for a break so everybody could read the storage and sequence
association proposal (which had been put in final form the
night before, and had copies distributed that morning).
Before splitting up, Mary Zosel announced a couple of
changes to the printed document:
     On page 2, near the bottom, a line of italics
       should be changed to plain text. (Somebody
       quickly asked, "Is this a printing
       discussion?")
     On page 3 and following pages, EDD was undefined;
       it should be "HPF alignment or distribution
       directive." (It was originally "Explicit Data
       Distribution.")
The group agreed that if that was the level of corrections
needed, the proposal must be in good shape. A 20 minute
break began.
  After the break, Ken Kennedy presented the storage
association proposal. This was a first reading of that
proposal, since it had been changed so heavily from the last
meeting. His first pair of slides concerned definitions used
in the proposal.
     *  Aggregate storage sequence: storage in variables
       that are associated by EQUIVALENCE & COMMON
     *  Aggregate variable group: a collection of
       variables whose storage sequences are
       associated to an aggregate storage sequence
     *  Aggregate variable group: a collection of
       variables in an aggregate storage group
     *  A variable is sequential if and only if any one
       of the following hold
       a. It is in a sequential COMMON
       b. It is in an aggregate variable group
       c. It is an assumed size array
       d. It is a component of sequenced type
       d. It is declared sequential
     *  Components: variables and aggregate variable
       groups in COMMON
     *  A COMMON is nonsequential if and only if all of
       the following hold
       a. It is not explicitly declared sequential
       b. Every instance of the COMMON has the same
          number of same-size components
       c. The type, shape, and mapping of any
          explicitly-mapped variable or variable
          group are the same in every program unit
          in which the COMMON occurs
Robbie Babb pointed out that there are lots of possible
conflicts in the COMMON definition, and asked if the default
was sequential or nonsequential. Ken responded that the
issue was dealt with later, but for now assume that
nonsequential is the default. The next slide gave some
examples to illustrate the definitions.
          common /foo/ a(100), b(100), c(100), d(100),
          e(100)
          real x(100), y(150)
          EQUIVALENCE (a(1),y(1)
          aggregate variable groups: (a,b,y), c, d, e, x
          sizes: (a,b,y) = 200, others 100
Piyush Mehrotra asked about inconsistent declarations of
COMMON; Ken replied that they make the COMMON sequential.
The subgroup had decided it was more important to support
consistently-declared COMMONs than full reshaping. David
Presberg noted that implementing full storage association is
a heavy burden on vendors. Piyush claimed that the same
problems happen here, because of EQUIVALENCE. Ken noted that
the effect Piyush wanted could be achieved by just declaring
a totally overlapping variable, which would force the
aggregate groups in all routines to be equivalent. After a
long discussion, Piyush agreed to provide an example of what
he was proposing, since everyone appeared confused.
  Ken's next slide gave the storage association rules for
HPF. In this context, "mappable" means "can be used in an
explicit HPF mapping directive."
     1. A nonsequential array variable is mappable. A
       variable in an aggregate variable group is
       mappable if its storage sequence is totally
       associated (i.e. completely covers) the
       aggregate storage sequence. Only one variable
       in an aggregate variable group may be
       explicitly mapped.
     2. The result variable of an array-valued function
       that is not an intrinsic is a nonsequential
       array.
Ken noted that the only sequential variables that can be
mapped are covering variables; this replaced the capability
of distributing the whole COMMON, as had been done in the
previous proposal. An example was meant to clarify the
rules.
          common /two/ e(10,10), g(10,100,1000), h(100),
          p(100)
          real cover(200)
          EQUIVALENCE (cover(1), h(1))
          align e...    OK, since it is in its own aggregate
          variable group
          align cover...   OK, since it covers the whole of
          its group
          can declare /two/ e(10,10), g(10,100,1000), x(200)
          elsewhere
  The next topic was the sequence association rules. These
had been presented in the same form at the last meeting, and
so this was treated as a second reading.
     1. When an array element or the name of an assumed-
       size array is used as an actual argument, the
       associated dummy must be a scalar or a
       sequential array.
       An array-element designator of a nonsequential
       array may not be associated with a dummy array
       argument
     2. A variable of default type CHARACTER (scalar or
       array) is nonsequential if it conforms to the
       requirements of definition DE-4. The size of a
       nonsequential, explicit-length, CHARACTER dummy
       argument must be the same size as the actual
       argument.
In the discussion, rule 2 was translated as "If you want to
map the arrays, the sizes must match." A short discussion of
assumed-length character strings followed, and the subgroup
will look at that issue some more.
  Returning to the storage association proposal, Piyush
Mehrotra presented his examples for extending the current
proposal.
          common /one/ q(100), r(100), s(100)
          real u(200)
          EQUIVALENCE u(1), q(1)
          u & s are mappable: that's good
          common /two/ f(100) g(100) h(100)
          real p(100)
          EQUIVALENCE p(1) f(50)
          p not mappable, (p,f,g) is an aggregate variable
          group
          common /three/ a(100), b(100), c(100)
          common /three/ d(150), e(50), c(100)    !new
          routine
          c not mappable, but there is no aggregate variable
          group in either routine
          claim: c should be mappable
David Presberg started his response with the claim, "I'm
about to put my foot in my mouth." In COMMON /two/ f and g
are clearly aligned, so h can reasonably be aligned as well.
In COMMON /three/, there are different parameters early in
the sequence, so there is no guarantee that array c can be
treated consistently in a separate compilation system. Ken
Kennedy defended Piyush, claiming that in any reasonable
implementation, if /two/ works, /three/ works as well.
Clemens-August Thole and Pres disagreed, noting that without
SEQUENTIAL (in Fortran 90), there is no guarantee that no
padding is inserted in COMMON. Joel Williamson suggested a
goal for compiling HPF: each array in COMMON acts like its
own COMMON (in the absence of EQUIVALENCE).
  Randy Scarborough claimed that the restrictions placed on
COMMON would mean that HPF needs explicit NOSEQUENTIAL
declarations, otherwise separate compilation cannot check
whether COMMON blocks are declared sequentially. Richard
Swift asked what problem we were trying to solve, to which
Piyush Mehrotra replied COMMON /three/ should be mappable.
After some discussion, Guy Steele asked how far Piyush
wanted to go in applying sequential storage models. Piyush
claimed that the length of arrays should be the only
consideration in defining aggregates. Peter Highnam remarked
that this was getting baroque, at which point Ken asked
Piyush if he were trying to kill the storage association
proposal. Peter continued by asking how to explain this
machinery to users, particularly since modules are coming
and would avoid much of the need for it.
  Ken tried to summarize the discussion so far:
     Piyush Mehrotra doesn't like requiring identical
       declarations of COMMON blocks in order to map
       the arrays.
     Peter Highnam uses this as argument against
       relaxing original rule, based on the difficulty
       of explaining the resulting rules to users.
Marc Snir characterized it as "Piyush wants to rely on
storage association to decide when to break storage
association." Ken commented that his statement was true, but
not a balanced view of the argument. Barbara Chapman stated
that Piyush's ideas are easier to implement than understand,
noting that they are very similar to what is already in
Vienna Fortran. The compiler should be able to figure out
aggregates without explicit EQUIVALENCE statements
everywhere. Randy Scarborough asked what the alternative
was; the only possibility seemed to be that a compiler sees
nonsequential COMMONs, which must be consistent or the
program will break on some machines. Robbie Babb made a
radical proposal: COMMON blocks can be declared as mappable
or not mappable; he was informed that that was the original
proposal, and this was an attempt to get around those
limitations.
  Chuck Koelbel argued against Piyush's brand of storage
association by converting the example to an extreme case:
          common /three/ a(100), b(100), c(100)
          common /three/ d(199), e, c(100)   !new routine
          Can you distribute c now?
He argued that this would be likely to break many machines.
Ken noted that this proposal was just trying to help users
porting large codes with lots of distributions. Don Heller
believed that the coding rule that would come from this
would be to allow only one array per COMMON, to avoid
accidental aliasing. Andy Meltzer noted that the future will
discourage all of this, but Ken replied that this is all for
interim porting problems in the first place. Randy
Scarborough proposed requiring COMMON blocks to be declared
identically to solve many of these problems.
  Richard Swift recalled the history of the proposal: it
started as a very simple proposal, then Ken Kennedy extended
it to allow some useful cases of association, and now we're
talking about extending it again. Ken said he would take it
as a friendly amendment that COMMON blocks must be identical
everywhere they are declared, including EQUIVALENCE
declarations. The discussion shifted somewhat to finding the
right terminology for this. Clemens-August Thole asked why
distinguish the proposal distinguished components from
aggregate variable groups; this was an artifact of how the
sections were written. Randy Scarborough pointed out a
potential problem if no EQUIVALENCE spans a variable in
COMMON, leading to lots of potential technical rewriting.
Marc Snir suggested wording the conditions as an identical
sequence of aggregates (i.e.. as Ken already intended, but
with clarification about what "identical" means). Peter
Highnam claimed that identical is too restrictive, using as
his example
          COMMON /foo/ complx(1000)
          COMPLEX complx
          REAL realx(2000)
          EQUIVALENCE (realx(1),complx(1))
Some questioned why this was considered doing users a favor.
Piyush favored the "identical" restriction, saying that it
at least gave some freedom to programmers.
  After a short discussion of some minor points, the
proposal was brought up for a straw poll as a first reading.
The vote to accept storage association (subject to details
of defining identical COMMON matching) was 26 yes, 1 no, and
5 abstain.
  The discussion then moved quickly to sequence
association. The first substantial issue opened was whether
assumed-length CHARACTER arguments could be distributed. Ken
Kennedy moved to send this question to the distribution
group, who should consider it in the context of ALIGN WITH
*. Clemens-August Thole asked whether there was a problem
with array sections; since these cannot be sequence
associated, there was not. Rich Shapiro asked if assumed-
size arrays could be distributed, and the issue was again
kicked back to Guy Steele's distribution group.
  Pending clarification of those issues, an official vote
to accept the sequence association proposal was taken. The
proposal passed 20 to 0, with 2 abstentions.

Low Performance Fortran

  Before moving to lunch Andy Meltzer made his now-
traditional presentation of Low Performance Fortran (LPF).
As he described it, this is another language model "real
similar to HPF." For the attendees, he had the disclaimer
"If you recognize something you think you said in this
presentation, it probably means that you did say it, and I
don't mean any offense by quoting you." After seeing the
first few points, Ken Kennedy remarked, "All I ask is that
you don't circulate this slide"; unfortunately, it was
already too late.
     Low Performance Fortran II
       LPF varieties of FORALL (to be thought of as
          a looping construct)
          FORSOME (triplet-spec [,mask]) stmt
            The set of iterations executed are
               implementation defined
          FORONE (triplet-spec [,mask]) stmt
            Statement is executed once
          FREEFORANY
            Some statement in the program is
               executed
          ONCE_AND_FORALL
            This is the last time the statement is
               executed
          BASKIN_ROBBINS_FORALL
            31 varieties, but you can't see them all
               at once and their descriptions are so
               similar that you can't tell what you
               voted for in the straw poll
          THERE_EXISTS
            Asserts that there is one, but since
               it's an existential statement, it
               doesn't matter
       Pointers
          It is impolite to point
       !LPF$DEPENDS_ON [line #]
          Asserts that the following statement
            depends on line #
       LPF Superset
          Contains addition and multiplication
            (regular LPF makes do with subtraction
            and division)
       TEMPLATES
          Some templates are "much-alike". If you
            want the same thing to happen to much-
            alike TEMPLATES, run them over with the
            same train
       COMMON
          Another name for COMMON is GAUCHE. There
            is only GAUCHE data and dummy arguments
            (called parameters)
       Actual & dummy arguments
          Actuals are copied to a randomly chosen
            PE's memory
          Parameters are hunted for by other PE's &
            copied back
       !LPF$ SOMEONE_ELSES
          Will choose some other application on
            machine and execute it
     LPP Low Performance Pascal (sister group)
       Have decided to go with Ada
       While marginally harder to implement, data
          hiding deemed worth it
       Will retain name to mislead users
     Coming Soon - To a Forum Near You!
       OPF - Obfuscated Performance Fortran
	   
Intrinsics

  After lunch, Rex Page presented the proposal from the
intrinsics subgroup. The reference document was from Rob
Schreiber (who could not attend the meeting), dated July 22;
there was some expansion from the draft at the last meeting,
but the spirit was the same.
  Rex's first slide treated the difference between
intrinsics and library functions.
     *  Only some procedures should be intrinsics.
     *  A Fortran 90 intrinsic is a procedure that is
       provided as an inherent part of the language
       processor (compiler).
     *  There are two types of intrinsics: those allowed
       in specification expressions and those that are
       not.
     *  We recommend: NUMBER_OF_PROCESSORS and
       PROCESSORS_SHAPE are intrinsics usable in
       specification expressions, and the rest of the
       functions defined here are not intrinsics.
The first point brought up discussion was the significance
of specification expressions: they can be used in
declarations, for example as array bounds. Andy Meltzer
pointed out that for reconfigurable machines or operating
systems that allowed partitioning of processors (that is,
virtually all MIMD machines), the recommendation made
NUMBER_OF_PROCESSORS and PROCESSORS_SHAPE load-time
constants, which has a major implementation impact on
handling symbols. Piyush Mehrotra and Rex clarified that
these functions are inquiries about the physical machine,
not the HPF PROCESSOR grids. Piyush suggested adding options
for distribution inquiry; that was taken under advisement.
  Mary Zosel started a long discussion by asking, "Are we
assuming a static number of processors in HPF?" Chuck
Koelbel came down firmly on both sides of the issue,
pointing out that if the number of processors was not
static, then the distribution patterns as they stand don't
make sense (their definitions require the number of
processors). However, there had been some interest from
other groups in porting HPF to workstation networks or other
situations that allowed (or required) dynamic process
creation. By the end of the discussion, both were sorry they
had brought the issue up. Alok Choudhary claimed that
physical processors are not covered in the rest of the HPF
model. Guy Steele noted that this is at bound of machine-
independent and dependent features, and HPF may be able to
do something with this. Rich Fuhler suggested adding the
constraint that the function must evaluate to a constant if
used in a specification expression context. Marc Snir noted
that there is no way to associate abstract and physical
processors within HPF; Guy replied that HPF does have the
PROCESSOR size constraint (implementations must allow a
processor grid of size NUMBER_OF_PROCESSORS()). Ken Kennedy
recalled awful arguments about this issue in PCF and urged
steering clear of it. Guy Steele observed that when the
NUMBER_OF_PROCESSORS constant is chosen is an implementation
issue, not a language issue. Richard Swift said he liked the
intent, but questioned whether this was the interface we
want.
  Finally, a straw poll was taken: those in favor of
NUMBER_OF_PROCESSORS() as proposed 23, opposed 1, abstain 7.
  The next slide covered the first set of recommended
standard library functions.
     *  Extended MAXLOC/MINLOC
       Add an optional argument to treat the main
          argument as an array of vectors
       Extended function delivers the
          maximum/minimum value for each vector
  The discussion was much briefer than previously. The
first question was whether overloading the function name
forces the extended functions to be intrinsics; quick
reference to the Fortran 90 standard indicated not. Rich
Shapiro noted the extensions give the same semantics as MAX
and MIN, thus fixing a Fortran 90 shortcoming. Returning
briefly to the "intrinsics" question, Marc Snir asked what
the difference was between a standard library and
intrinsics. Richard Swift noted an interface difference,
namely that the programmer must include modules declarations
for libraries (particularly when the function names are
overloaded, as MAXLOC is). Ken Kennedy claimed the
distinction is small, but Marc thought it was important. He
argued that added intrinsics are extending Fortran 90
generally, not just in the areas of data distributions and
parallel computation as HPFF originally intended. Don Heller
suggested that maybe a better design would be to generate
APL functionals in a library rather than new intrinsics. Guy
Steele had two responses: "That's what I thought I was
doing" and "I'm not sure it makes sense to recreate all of
APL in HPF."
  Rex then took a vote on whether some form of this (i.e.
either library functions or intrinsics) should be in HPF. 13
voted yes, 1 voted no, and 10 abstained.
  The next slide covered a new set of functions
     *  Elemental bit inquiries
       POPCNT - Number of 1's in the binary
          representation of an integer
       PARITY - Parity of the binary representation
          of an integer
       LEADZ - Leading 0 bits in the binary
          representation of an integer
       ILEN - Number of bits needed to represent an
          integer
The discussion was very brief. Most of the questions were
answered when Rex pointed out that these are elemental
functions and work like SIN. Several implementors affirmed
that all were needed on many machines (although ILEN can be
constructed from LEADZ). A straw poll on whether these
functions should be included drew 5 yes votes, 0 no votes,
and 16 abstains.
  Yet another slide introduced more functions.
     *  New reductions
       AND - corresponding to the IAND operation
          (bit-wise AND)
       OR - corresponding to the IOR operation (bit-
          wise inclusive OR)
       EOR - corresponding to the NEQV operation
          (bit-wise exclusive OR)
       PARITY - corresponding to the parity
          operation
Again, this just fixed some features (associative and
commutative operations) that had been left out of Fortran
90. There was no discussion. The straw poll to include these
functions found 13 yes votes, 0 no votes, and 13 abstains.
  The next slide defined a very broad class of functions.
     *  Parallel prefix and suffix operations
       XXX_PREFIX - for any reduction function XXX
          (including the new ones defined above)
       XXX_SUFFIX - for any reduction function XXX
       Each delivers a series of partial reductions
          (sums, products, etc.) of the input array
       The result is defined for all output array
          elements, even when the (optional) mask is
          false
       Reductions are restarted for each segment;
          segments are portions of ARRAY
          corresponding to runs of like values in
          SEGMENT (a LOGICAL array optional
          argument)
       Partial reductions start at the identity
          element for the appropriate operation
The discussion was short, but spirited. Some off by one
details of the SEGMENT feature were referred back to the
proposal. Dave Loveman noted that this creates lots of
functions (particularly when overloading by number of array
dimensions is taken into account), and asked if HPF needed
all of them. Chuck Koelbel asked which ones he wanted to
leave out, and Dave noted that the full set would take
valuable implementation time. Ken Kennedy referred to a
proposal by Demmel (on the mailing list) regarding prefix
operations on 2-by-2 matrix multiplies; Guy Steele wanted
more input, particularly on the numerical aspects, before
recommending those features. Marc Snir noted that all the
functions could be moved to a library by overloading, but
Rex Page noted they were hard to write just due to volume.
Guy claimed that most of the implementation difficulty is in
testing rather than writing. Richard Swift was concerned
about the library size; Andy Meltzer concurred.
  A straw poll was taken on whether to include the prefix
operations in HPF, giving 11 yes votes, 0 no votes, and 19
abstains.
  The next slide extended reductions in another way.
     *  Combining send procedures
       XXX_SEND( SRC, DEST, IDX1, ... )
       XXX is any Fortran 90 or HPF reduction
       CALL SUM_SEND(a,x,v) operates like x(v) =
          x(v) + a but all elements assigned to a
          corresponding to duplicate element in v
          contribute to the final value
       The number of IDX arguments is the rank of
          the destination array
       Source and all IDX arguments are conformable
The intent is to allow parallel accumulations of many
irregularly related array elements in a single call; for
example, one call can compute a histogram. CALL
SUM_SEND(a,x,idx) is equivalent to the following loop:
          DO i = 1, n
            x(idx(i)) = x(idx(i)) + a(i)
          END DO
The discussion was lively. Clemens-August Thole suggested
avoiding the name SEND because of possible conflicts with
message-passing routines; the subgroup agreed to look for a
better term. Piyush Mehrotra asked why these were
subroutines, and not functions; given current syntax and
semantics, they can't be called from FORALL constructs. Guy
Steele saw no strong reason against the function form. Rich
Shapiro noted that the same operations can be done with
FORALLs, but Guy responded that those constructs give the
compiler problems in idiom recognition. Mary Zosel smelled
something extremely important for unstructured mesh problems
in these subroutines, and was greeted with a general
"Right!" from the crowd. Richard Swift noted that HPF now
required lots and lots of library functions (by his count,
over 400 after resolving overloading); David Loveman agreed.
Ken Kennedy asked if HPF needed this support for irregular
computations, given that it doesn't have irregular
distributions. Rich Shapiro replied that these functions are
very useful, even without distributions of any kind. Ken was
still concerned over the interaction of this feature with
others. Richard Swift said his concern was the interface,
not the functionality.
  Guy Steele made two "meta-remarks" to help clarify the
situation. No single one of these functions is vital for HPF
(although each has some real application associated with
it). But he had promised HPF (particularly the intrinsics
group) to give them all the things Thinking Machines had
found useful, so the names, order of arguments, and other
details could be standardized. Andy Meltzer commented that
these libraries take a lot of time to develop, and HPFF
should decide in November whether to include them in the HPF
subset, superset (if any), or whatever.
  With that, a straw poll was taken on whether to include
these functions in HPF. There were 8 yes votes, 0 no votes,
and 21 abstains. Another vote was taken on whether anybody
objected to making these operations functions, finding 0
opposed.
  There was only one slide remaining, and little time left
before the meeting was scheduled to end.
     Sorting functions
       GRADE_UP - Sorts in ascending order
       GRADE_DOWN - Sorts in descending order
       Each delivers a list of subscripts for
          arranging the array in sorted order (i.e.
          a permutation array)
       If optional argument DIM is absent - The
          result has shape
          [size(shape(ARRAY),product(shape(array))]
       If DIM is present - The result has the same
          shape as array, and is sorted along the
          appropriate dimension
Maureen Hoffert asked if sorting was limited to integer and
real arrays. Guy Steele responded that any standard type
with a "less than" operation defined was allowed. Ken
Kennedy suggested sorting could take a comparison function
(like UNIX qsort). Robbie Babb asked what sorting had to do
with High Performance Fortran. Rich Shapiro noted that it is
hard to do parallel sorts (and many other parallel
functions); a good philosophy was to let vendor do it
instead of the applications programmer. Chuck Koelbel noted
that sorting is needed for particle push phases in PIC
algorithms, certainly a high-performance application. The
straw poll on including the sorting functions found 12 yes
votes, 0 no votes, and 16 abstains.
  Richard Swift asked for a poll on the intrinsics vs.
library question. Rich Shapiro noted that most of the
functions proposed were now implemented as calls in the CM
Fortran compiler, but it "doesn't follow the rules" on type
checking in some cases. If there is strong type checking,
the library needs lots more code. Robbie Babb asked if
generic functions help the library situation; Rex Page
wondered if it was legal to overload intrinsics (the
language lawyers believed so) and what the implementation
effects were (no answer). Ken Kennedy wanted a description
of the pros and cons for intrinsics versus libraries before
putting it to a vote, and the group agreed this was wise.

Official HPF Subset

  After a short break, the group considered the status of
the official HPF subset. The discussion started (rather than
ended) with a straw poll: Given all the new features should
HPFF allow HPF features to be omitted from the official
subset? 22 voted yes, 1 no, and 5 abstained.
  Mary Zosel suggested that each subgroup should propose
what goes into the official subset from the features they
considered. Ken Kennedy recommended that Mary and her group
prepare a proposal to the effect that HPF features can be
out of subset; assuming that passed, future meetings would
allow motions to remove specific features from the subset.
The reasoning behind this was that the group needed to know
what was in the language before deciding what was in the
subset. As Marc Snir pointed out, the argument for removing
features from the language (or subset) is size, so we need
to know how big it is.

Editorial Matters

  With all the proposals, counter-proposals, and other
discussions on the email lists, Mary Zosel asked whether
somebody could post summaries to the net? Reading all the
HPFF mail is too much. Chuck Koelbel muttered "I know" under
his breath. After a short discussion, in which it was made
clear that such postings would have to be automatic, David
Presberg suggested posting just the mail headings regularly
and directed Chuck to some software that might do this. He
is looking into that possibility.
  Dave Loveman summarized the status of the (proposed) High
Performance Fortran language document. Chuck Koelbel had
proposed two outlines, both of which had the problem that
they required too much writing to transform the proposals
being made into the right form. He had produced a new
outline that would hopefully balance readability and
completeness. In essence, each group would work on its own
chapter, integrated into a single outline. The subgroup
chairs had seen the outline, and most had agreed with the
structure. by next meeting he hoped to get a first draft
from each group and have a full (albeit rough) document
available. Ken Kennedy noted that little effort now will be
worth it later. Mary Zosel asked that he send the whole
outline to the core group, which Dave agreed to do once some
packaging issues had been resolved. Dave and Chuck Koelbel
would collaborate on providing FTP access.

Meeting Schedules

  The final topic for discussion was the schedule for the
next HPFF meeting, to be held September 9-11 in Dallas. The
topics to be discussed included
     Second readings
       Storage association
       Subroutine linkage for distributed arrays
       Distribution for pointers and allocatable
          arrays
       Intrinsics considered at this meeting
       FORALL
       INDEPENDENT (for DO only)
     First readings
       Local subroutines
       New intrinsic functions
       Fortran 90 subset
       INDEPENDENT (for FORALL and other constructs)
       Parallel I/O
As Ken Kennedy put it, we have lots of work to do. Consensus
was that the usual one-and-a-half day meeting was not
enough. Piyush Mehrotra was the first to suggest extending
the meeting into the afternoon of the last day, and the rest
agreed. After a short discussion of flight availability, the
group decided to run to about 4:00pm on the last day. At
Mary Zosel's suggestion, the subgroup leaders will get
together for dinner the night before to set the meeting
agenda and other details. Marc Snir suggested distributing
the proposal documents for first and second readings at
least one week in advance, in order to speed up the
discussions. Mary Zosel generalized this to distributing the
documents to the hpff-core mailing list, not just the
subgroup lists as had been done in the past.
  Ken Kennedy ended the meeting with a short report on the
HPF Birds-of-a-Feather session held at ICS. About 12
visitors, including Bob Voigt from NSF, had attended that
meeting and provided comments on the language. Ken
particularly mentioned some helpful comments from John Ried.
Most of the questions there had been raised in the
discussions, either in the plenary sessions or in the
subgroups. The next major opportunity for this kind of
feedback appeared to be the workshop at Supercomputing '92,
where the near-final draft of the language will be
distributed. Ken hoped that this would allow a great deal of
feedback before the December HPFF meeting, where the group
would have a chance to reconsider language features that had
been voted as "frozen" so far. Piyush Mehrotra suggested
that HPFF should move the December meeting back, in order to
allow more time for feedback. After some discussion, the
group decided that December 9-11 had no conflicts from
potential attendees; pending checking dates with the Dallas
hotel, the meeting will be rescheduled then.