enthought/distarray — pull request #247 by kwmsmith (2014-03-21 21:03:59+00:00)
Title: Client-side maps

Massive refactoring. Lots going on here. The biggest change was the addition of the `ClientMDMap` class, which is meant to help `DistArray` objects figure out the global index to process mapping (hence the "Map" name). In general, this map class is meant to handle the distribution, indexing, and communication tasks. This PR gets this started, and we can build on it from here.

Some details:

* A new class, `ClientMDMap` (for "MultiDimensional") was added in `distarray/client_maps.py`
* Several 1-dimensional client map classes were added to the same module.
* All `DistArray`s now use a `ClientMDMap` instance internally to help with indexing.
* `DistArray`s ask their `MDMap` object for several attributes that they used to ask their `LocalArray`s, reducing communication.
* `DistArray`s use their `MDMap` to help figure out what targets to talk to in `__getitem__` and `__setitem__`.
* Tests were added for the `MDMap` class and component classes.
* A number of the stateless functions that used to be in `distarray/local/construct.py` have been generalized and pulled into `distarray/metadata_utils.py`. These functions determine the `grid_shape` and normalize `dist`, among other things. Now both the client and engines need to call these functions, so they were moved out of `distarray/local`.
* Every time a `DistArray` is created, a `MDMap` is created either explicitly or implicitly for it to use.

This is just the start, there will likely be more changes and refactorings related to this redesign, but it will put us on strong footing.