Mpi

Using a unique tag per MPI process does not allow to identify different
threads inside of the same MPI process.

Using the the destination rank as a tag:

MPI Process rank=0 id_thread=2 MPI_Recv( ... tag=0 ... ) -- Waiting message
from rank=1 id_thread=2
MPI Process rank=0 id_thread=3 MPI_Recv( ... tag=0 ... ) -- Waiting message
from rank=1 id_thread=1
MPI Process rank=1 id_thread=1 MPI_Send( ... tag=0 ... ) -- Sending message
to rank=0 id_thread=3
MPI Process rank=1 id_thread=2 MPI_Send( ... tag=0 ... ) -- Sending message
to rank=0 id_thread=2

Using the source rank as a tag:

MPI Process rank=0 id_thread=2 MPI_Recv( ... tag=1 ... ) -- Waiting message
from rank=1 id_thread=2
MPI Process rank=0 id_thread=3 MPI_Recv( ... tag=1 ... ) -- Waiting message
from rank=1 id_thread=1
MPI Process rank=1 id_thread=1 MPI_Send( ... tag=1 ... ) -- Sending message
to rank=0 id_thread=3
MPI Process rank=1 id_thread=2 MPI_Send( ... tag=1 ... ) -- Sending message
to rank=0 id_thread=2

In both cases, it could happen that rank=0 id_thread=3 receive the message
from rank=1 id_thread=2 for example.

MPI Process rank=0 id_thread=2 MPI_Recv( ... tag=6464 ... ) -- Waiting
message from rank=2 id_thread=2
MPI Process rank=0 id_thread=3 MPI_Recv( ... tag=3232 ... ) -- Waiting
message from rank=1 id_thread=1
MPI Process rank=1 id_thread=1 MPI_Send( ... tag=3232 ... ) -- Sending
message to rank=0 id_thread=3
MPI Process rank=2 id_thread=2 MPI_Send( ... tag=6464 ... ) -- Sending
message to rank=0 id_thread=2

Here, it cannot happen because they are using a different tag.

I made this decision thinking to allow different active
Collage::MPIConnections in the same MPI process.


On 1 July 2014 15:00, Stefan Eilemann <notifications@github.com> wrote:

> In co/mpiConnection.cpp:
>
> > +#include <boost/thread/thread.hpp>
> > +
> > +#include <set>
> > +#include <mpi.h>
> > +
> > +namespace
> > +{
> > +
> > +/*
> > + * Every connection inside a process has a unique MPI tag.
> > + * This class allows to register a MPI tag and get a new unique tag.
> > + *
> > + * Due to the tag is defined by a 16 bits integer on
> > + * ConnectionDescription ( the real name is port but it
> > + * is reused for this purpuse ). The listeners always
> > + * use tags [ 0, 65535 ] and others [ 65536, 2147483648 ].
>
> I agree with the reasoning for a tag. My argument is that you already have
> a unique identifier, the MPI rank of the peer process and you don't need
> another tag. Is this correct, or am I missing something?
>
> —
> Reply to this email directly or view it on GitHub
> <https://github.com/Eyescale/Collage/pull/109/files#r14401934>.
>