Our fundamental data structure approach is to ensure that every element can reference its neighbor information locally. Also that the mesh is described in a hierarchial and distributed manner with simple interfaces to user routines. The mesh contains components stored in dynamic arrays. This approach provides flexibility for mesh migration during load balancing, while keeping data components contiguous.
All communication is asychronous, for efficiency purposes.
At times, as in the mesh migration procedure , the communication pattern is unbalanced and irregular. Our software needs to communicate with the ParMeTiS parallel mesh partitioner, which is written in the C programming languauge. We have a single routine that acts at the conduit between our Fortran 90 system and the C ParMeTiS library.