From: Jeff Squyres <jsquyres@XXXXXXXXXXXX>
Date: June 24, 2005 11:15:32 PM GMT+02:00
To: Jeff Squyres <jsquyres@XXXXXXXXXXXX>
Cc: Brian Barrett <brbarret@XXXXXXXXXXXX>, Ewing (Rusty) Lusk  
<lusk@XXXXXXXXXXX>, William Gropp <ddd@XXXXXXXX>, George Bosilca Subject: Re: MPI-2 attributes question

After much more discussion amongst ourselves, we came to the  
following conclusions:

1. Example 4.13 is wrong: in the C->Fortran example (i.e., the first  
example), the Fortran var variable should get the address of i, not 5.

2. My cases 2 and 3 are wrong.  They should be:

  * 2. Fortran MPI-1 reads the attribute value.  The C pointer is cast
  * to a fortran INTEGER (i.e., MPI_Fint) -- potentially being
  * truncated if sizeof(void*) > sizeof(INTEGER).
  *
  * Example A: INTEGER ret
  *            CALL MPI_ATTR_GET(..., ret, ierr)
  *            --> ret will equal &foo, possibly truncaed
  * Example B: INTEGER ret
  *            CALL MPI_ATTR_GET(..., ret, ierr)
  *            --> ret will equal &bar, possibly truncaed
  *
  * 3. Fortran MPI-2 reads the attribute value.  The C pointer is cast
  * to a fortran INTEGER(KIND=MPI_ADDRESS_KIND) (i.e., a (MPI_Aint)).
  *
  * Example A: INTEGER(KIND=MPI_ADDRESS_KIND) ret
  *            CALL MPI_COMM_GET_ATTR(..., ret, ierr)
  *            --> ret will equal &foo
  * Example B: INTEGER(KIND=MPI_ADDRESS_KIND) ret
  *            CALL MPI_COMM_GET_ATTR(..., ret, ierr)
  *            --> ret will equal &bar

This makes all the text in MPI-2 make sense -- it's just Example 4.13  
that is wrong.

Do you agree?

On Jun 24, 2005, at 3:22 PM, Jeff Squyres wrote:

> Bill/Rusty --
>
> I'm asking you guys this question because:
>
> 1. We can't figure this out
> 2. I don't remember (and can't find) the address of the MPI  
> implementer's list ;-)
>
> I'm the unlucky slob who was tagged to write the attributes code  
> for Open MPI.  I wrote it about a year ago, but some recent test  
> failures made me go back and look at it again about a week ago.   
> I've been puzzling over MPI-2:4.12.7 for quite some time and asked  
> a whole bunch of other OMPI guys and none of us can figure this out.
>
> In particular, I'm referring to the paragraph that starts on MPI-2  
> p58, line 12: "MPI supports two types of attributes..."  I have  
> several problems with this text and the examples that follow it,  
> but perhaps I'm just missing something fundamental.
>
> Starting on line 16, it says:
>
> "When an address valued attribute is accessed from Fortran, the  
> MPI_xxx_GET_ATTR will convert the address into an integer and  
> return the result of this conversion.  This conversion is lossless  
> if new style (MPI-2) attribute functions are used, and an integer  
> of kind MPI_ADDRESS_KIND is returned.  This conversion may cause  
> truncation if old style (MPI-1) attribute functions are used."
>
> From that text, it sounds like if, in C, I write an attribute (that  
> will be a pointer, such as "int i = 5;", I'll put &i), if I use the  
> Fortran MPI-2 functions, I should get the value of the pointer back  
> (!).  However, Example 4.13 directly shows something different --  
> in C, it COMM_PUT_ATTR's a pointer to an int.  Then, in Fortran, it  
> COMM_GET_ATTR's that attribute and gets the value *pointed to by  
> the C pointer*.
>
> This, to me, seems like a contradiction.
>
> But this raises other questions:
>
> 1. If Example 4.13 is showing the spirit of what is intended (i.e.,  
> put a pointer in C, get the dereferenced value back in Fortran),  
> then the language about lossless / truncation is meaningless -- and  
> there was no point in extending the size of the MPI-2 Fortran  
> functions to be INTEGER(KIND=MPI_ADDRESS_KIND).
>
> 2. If the text is showing the spirit of what is intended (i.e., put  
> a pointer in C, get the pointer value back in Fortran), what is the  
> point of that?  I.e., what would one do with an address in Fortran?  
> (other than for MPI_WIN_BASE)
>
> 3. I think that Example 4.13 is not always right.  Specifically,  
> when running on a 64 bit platform (e.g., an Opteron, where sizeof 
> (int) == 4 and sizeof(INTEGER(KIND=MPI_ADDRESS_KIND)) == 8), the  
> only way that the fortran val variable could get the value of 5 is  
> if the C pointer is dereferenced as an int, which a) is not  
> consistent with the text above it, and b) defeats the point of  
> having the INTEGER be of size MPI_ADDRESS_KIND (also making the  
> lossless / truncation language irrelevant).
>
> What do you think?  Am I just missing something?
>
> Sidenote: Was all of this put in place solely for MPI_WIN_BASE?   
> (this attribute seems to contradict the [confusing] established  
> rules established -- it seems to be the one attribute that is *not*  
> dereferenced in Fortran when read)
>
> More generally, I see 9 cases for attribute handling:
>
> 1. C writes, C reads
> 2. C writes, Fortran MPI-1 reads
> 3. C writes, Fortran MPI-2 reads
> 4. Fortran MPI-1 writes, C reads
> 5. Fortran MPI-1 writes, Fortran MPI-1 reads
> 6. Fortran MPI-1 writes, Fortran MPI-2 reads
> 7. Fortran MPI-2 writes, C reads
> 8. Fortran MPI-2 writes, Fortran MPI-1 reads
> 9. Fortran MPI-2 writes, Fortran MPI-2 reads
>
> Here's an extended comment from my code where I explain what I  
> think should happen with each of them.  Does this all sound right  
> to you?
>
>  * Cases where C writes an attribute value:
>  * ----------------------------------------
>  *
>  * In all of these cases, a pointer was written by C (e.g., a pointer
>  * to an int -- but it could have been a pointer to anything, such as
>  * a struct).  These scenarios each have 2 examples:
>  *
>  * Example A: int foo = 3;
>  *            MPI_Attr_put(..., &foo);
>  * Example B: struct foo bar;
>  *            MPI_Attr_put(..., &bar);
>  *
>  * 1. C reads the attribute value.  Clearly, this is a "unity" case,
>  * and no translation occurs.  A pointer is written, and that same
>  * pointer is returned.
>  *
>  * Example A: int *ret;
>  *            MPI_Attr_get(..., &ret);
>  *            --> *ret will equal 3
>  * Example B: struct foo *ret;
>  *            MPI_Attr_get(..., &ret);
>  *            --> *ret will point to the instance bar that was written
>  *
>  * 2. Fortran MPI-1 reads the attribute value.  The C pointer is
>  * dereferenced as an int and cast to a fortran INTEGER (i.e.,
>  * MPI_Fint).
>  *
>  * Example A: INTEGER ret
>  *            CALL MPI_ATTR_GET(..., ret, ierr)
>  *            --> ret will equal 3
>  * Example B: INTEGER ret
>  *            CALL MPI_ATTR_GET(..., ret, ierr)
>  *            --> ret will be the INTEGER representation of the first
>  *                sizeof(int) bytes from the struct foo instance
>  *                (bar).
>  *
>  * 3. Fortran MPI-2 reads the attribute value.  The C pointer is
>  * dereferenced as an int and cast to a fortran
>  * INTEGER(KIND=MPI_ADDRESS_KIND) (i.e., a (MPI_Aint)).
>  *
>  * Example A: INTEGER(KIND=MPI_ADDRESS_KIND) ret
>  *            CALL MPI_COMM_GET_ATTR(..., ret, ierr)
>  *            --> ret will equal 3
>  * Example B: INTEGER(KIND=MPI_ADDRESS_KIND) ret
>  *            CALL MPI_COMM_GET_ATTR(..., ret, ierr)
>  *            --> ret will be the INTEGER representation of the first
>  *                sizeof(int) bytes from the struct foo instance  
> (bar).
>  *
>  * Cases where Fortran MPI-1 writes an attribute value:
>  * ----------------------------------------------------
>  *
>  * In all of these cases, an INTEGER is written by Fortran.
>  *
>  * Example: INTEGER FOO = 7
>  *          CALL MPI_ATTR_PUT(..., foo, ierr)
>  *
>  * 4. C reads the attribute value.  The value returned is a pointer
>  *    that points to an INTEGER (i.e., an MPI_Fint) that has a value
>  *    of 7.
>  *    --> NOTE: The external MPI interface does not distinguish  
> between
>  *        this case and case 7.  It is the programer's responsibility
>  *        to code accordingly.
>  *
>  * Example: MPI_Fint *ret;
>  *          MPI_Attr_get(..., &ret);
>  *          -> *ret will equal 7.
>  *
>  * 5. Fortran MPI-1 reads the attribute value.  This is the unity
>  *    case; the same value is returned.
>  *
>  * Example: INTEGER ret
>  *          CALL MPI_ATTR_GET(..., ret, ierr)
>  *          --> ret will equal 7
>  *
>  * 6. Fortran MPI-2 reads the attribute value.  The same value is
>  *    returned, but potentially sign-extended if sizeof(INTEGER) <
>  *    sizeof(INTEGER(KIND=MPI_ADDRESS_KIND)).
>  *
>  * Example: INTEGER(KIND=MPI_ADDRESS_KIND) ret
>  *          CALL MPI_COMM_GET_ATTR(..., ret, ierr)
>  *          --> ret will equal 7
>  *
>  * Cases where Fortran MPI-2 writes an attribute value:
>  * ----------------------------------------------------
>  *
>  * In all of these cases, an INTEGER(KIND=MPI_ADDRESS_KIND) is written
>  * by Fortran.
>  *
>  * Example A: INTEGER(KIND=MPI_ADDRESS_KIND) FOO = 12
>  *            CALL MPI_COMM_PUT_ATTR(..., foo, ierr)
>  * Example B: // Assume a platform where sizeof(void*) = 8 and
>  *            // sizeof(INTEGER) = 4.
>  *            INTEGER(KIND=MPI_ADDRESS_KIND) FOO = pow(2, 40)
>  *            CALL MPI_COMM_PUT_ATTR(..., foo, ierr)
>  *
>  * 7. C reads the attribute value.  The value returned is a pointer
>  *    that points to an INTEGER(KIND=MPI_ADDRESS_KIND) (i.e., a void*)
>  *    that has a value of 12.
>  *    --> NOTE: The external MPI interface does not distinguish  
> between
>  *        this case and case 4.  It is the programer's responsibility
>  *        to code accordingly.
>  *
>  * Example A: MPI_Aint *ret;
>  *            MPI_Attr_get(..., &ret);
>  *            -> *ret will equal 12
>  * Example B: MPI_Aint *ret;
>  *            MPI_Attr_get(..., &ret);
>  *            -> *ret will equal 2^40
>  *
>  * 8. Fortran MPI-1 reads the attribute value.  The same value is
>  *    returned, but potentially truncated if sizeof(INTEGER) <
>  *    sizeof(INTEGER(KIND=MPI_ADDRESS_KIND)).
>  *
>  * Example A: INTEGER ret
>  *            CALL MPI_ATTR_GET(..., ret, ierr)
>  *            --> ret will equal 12
>  * Example B: INTEGER ret
>  *            CALL MPI_ATTR_GET(..., ret, ierr)
>  *            --> ret will equal 0
>  *
>  * 9. Fortran MPI-2 reads the attribute value.  This is the unity
>  *    case; the same value is returned.
>  *
>  * Example A: INTEGER(KIND=MPI_ADDRESS_KIND) ret
>  *            CALL MPI_COMM_GET_ATTR(..., ret, ierr)
>  *            --> ret will equal 7
>  * Example B: INTEGER(KIND=MPI_ADDRESS_KIND) ret
>  *            CALL MPI_COMM_GET_ATTR(..., ret, ierr)
>  *            --> ret will equal 2^40
>
> If you're still reading this, many thanks for your time!  :-)
>
> -- 
> {+} Jeff Squyres
> {+} The Open MPI Project
> {+} http://www.open-mpi.org/
>
>

-- 
{+} Jeff Squyres
{+} The Open MPI Project
{+} http://www.open-mpi.org/